OAuth sources

Most platforms authorise rather than hand out a token. Those connectors ask you to approve access at the source and come back; nobody types a credential into louvain, and louvain never holds one. The broker does, and each connection is named by its own id, so one connection can never drive another's.

They are ordinary connector definitions — the same registry, the same runner supervision, the same per-connection ingest token, the same org fence. Adding twenty-one platforms required no change to the ingest path and no change to the web app, because the Connections page renders whatever is registered.

A source appears only when the broker can serve it

Connecting a brokered source takes two things at the broker, and the second one is the quiet one:

1. an integration, which teaches it how to authorise; and
2. a sync, which is what actually produces records.

With an integration but no sync, a connection authorises, goes live, reports no error and delivers nothing — forever. Everything about it looks healthy. louvain checks both and says which is missing, rather than letting you watch an empty feed and guess which end is broken.

Registering a connector and being able to use it are different things: a brokered source needs an integration configured at the broker first. louvain asks the broker which ones exist — one call for all of them, on a timer, never while rendering a page — and greys out the rest with what to do about it. They are still listed, because hiding them would leave you wondering whether louvain supports Jira at all, and they sort below the ones you can use.

The API agrees with the page: creating a connection for a source that is not ready is refused with connector_not_ready rather than producing a connection that fails at the last step of a form you have already filled in.

If the broker is briefly unreachable, the last known answer stands rather than the catalogue emptying — a blip should cost a confusing error at worst, not the whole page.

Sign in, or hand louvain a service credential

Every one of these offers both, as two tabs on its setup form. Slack (OAuth) shares the Slack card with core's Socket Mode app and export import (it declares the slack family), so its two tabs sit beside theirs; its type stays slack-oauth, because the type is in every container id it has written.

Sign in with the provider takes one click and no administrator, and the feed reads exactly what you read there: your mail, your files, the channels you are in. For a team of ten that is frequently the right answer and the fastest one.

A service credential — a bot token, an app installation, a service account — is issued by an administrator in the source's own console, and the feed reads the workspace.

The reason to care is that the first option's ceiling is invisible. A company that connects Drive and Gmail with one admin's personal sign-in sees twenty-one healthy feeds, no errors and no gaps, and holds one person's documents. Nothing in the data says which part of the company is missing, because nothing was ever there to be missing. So louvain names it on the form before you choose, and puts a Reads as you or Reads the workspace chip on the connection afterwards.

Slack is the exception worth knowing: its OAuth install issues a bot token, so signing in there already reads the workspace.

If several people connect the same source

Nothing is processed twice. Every event carries a key built from the source's own ids — a Slack channel and timestamp, a Drive file id and its modified time — and ingest drops a repeat before it reaches the queue. Ten colleagues each signing in to the same Slack sees one copy of a public-channel message, one extraction and one set of claims. The keys are scoped per workspace, so one customer's traffic can never suppress another's.

Read this part before connecting anything

louvain shows a thing to whoever may see the container it came from. So the question that decides what your colleagues can see is not "can louvain read this source" — it is "can this source tell louvain who may see a thing."

Some platforms publish per-object membership: who is in a channel, who a file is shared with. Those sync it as first-class events, and the container ends up visible to exactly the people the source says may see it. Through the broker that is Google Drive and SharePoint, both of which publish a Permission model.

Slack and Teams belong in that group by capability and are not in it here: the broker's templates carry channels, users and messages, but no per-channel member list. louvain's own hand-written Slack connector does sync membership — the brokered one cannot, and says so rather than claiming a fence it has no data for.

Many platforms do not. Their content is ingested at container visibility — readable by whoever is granted that container in louvain, and nobody else. louvain does not widen to workspace-wide because an API declined to answer: silence is not permission, and inventing permission is the one thing a connector may not do (ADR-001). Each connector says which of the two it is, in the picker, before you connect it.

Fewer sources publish an ACL than you would hope. That is a fact about the industry, not a gap here, and the honest handling of it is a narrower default rather than a convenient one.

For those sources the whole connection is one container — a mailbox, a workspace, a site — stored as <connector>/<workspace>/all. Nothing finer buys anything: the only grant such a source can ever carry is to the person who connected it, and a container per thread stated that one fact once per thread. Measured on one imported mailbox, that was 6,860 containers, 6,860 grants and 9,722 permission tuples to say that its owner may read their own mail. Each message is still cited to its own thread through source.ref. A source that publishes membership keeps one container per object, because that object is what its membership names — and each connector declares which it is, so the check that a mailbox folds to one container and a file share does not runs against the table, not the code.

The picker shows each source's logo, what its container is, and a chip saying which of the two it is — so the permission answer is on the card, before you click it, not in a paragraph you might not read.

Messaging

What the reader is shown says who, not just what. A mail's text opens with the sentence a person reads first — "Email from Dana Voss <dana@acme.example> to Scott Waddell <scott@…>, copied to ops@…" — built from the From:, To: and Cc: lines (or the equivalent objects a mail API keeps), so direction and audience are in the text the extractor and the passage tier both hold. The sender's name and address travel as the actor's identity hints, and every recipient as a participant — hints that link identities, never grants: who may read the mail is still the connection's container.

Authors are colleagues. None of these publishes a member list through the broker, so each connection is one container.

Files

A document whose source names an author or owner opens with "Document by <name>.", and that person's name and address are the actor's identity hints.

A document's author is often a service account, so these declare actorsArePeople: false and their actors are counted in an answer's sources rather than drawn as people on the map — except the issue trackers, where the author is a colleague.

Records

None of the four publishes a per-record ACL, so the whole CRM is one container. A record is a summary somebody else already wrote, so these are ingested as relayed (ADR-014): stored, embedded, cited and exactly as visible as anything else, and ranked below a first-hand account of the same fact when both are available.

Connecting one

1. Connections → Add connection → pick the source. The picker groups by messaging, files and records, and each entry says what a container is there and whether membership comes with it.
2. Leave the workspace label blank unless you connect the same platform twice. It names which of your accounts a connection is, and it is part of a container's identity — so with one account there is nothing to distinguish, and with two it stops both accounts' channels landing in one namespace. louvain asks for it on the second connection and refuses without it.
3. Continue to the source creates the connection and opens the provider in a new tab. Approve there; louvain never sees the password or the token. If the broker is unreachable or misconfigured the error says which, and retrying re-authorises the connection that already exists rather than making a second one.
4. Come back and press Enable. Authorising and running are deliberately two acts, so an abandoned authorisation leaves an inert row rather than a source that quietly began reading.

What it does once enabled

The connection manager spawns one runner, which syncs each of the source's models in turn, maps the records to events and posts them to /v1/ingest/batch under the connection's own ingest token.

Content events carry no tuples — content never grants itself visibility. Membership records become ACL events on the same feed, with the container and subject namespaced by connector (user:google-drive/…, container:google-drive/…), which is what the readable-set projection resolves. A personal sign-in to a source with no membership model grants the person who connected it their one container, by the same route — one membership event, one tuple.

The broker runs in your stack

docker compose up starts the broker beside louvain (service nango, dashboard http://localhost:3003, no login locally; the production stack has the same service behind your own hostname with its own accounts). It holds the OAuth tokens and signs requests; nothing else. Once, in its dashboard, create the integration for a source — for Gmail, google-mail with your Google client id and secret and the gmail.readonly scope — and add the broker's callback, <NANGO_SERVER_URL>/oauth/callback, to the Google client's redirect URIs. Paste the environment's secret key into LOUVAIN_NANGO_SECRET_KEY. A hosted broker works too (LOUVAIN_NANGO_URL=https://api.nango.dev), but its free plan caps compute per month and cancels every sync when the cap is hit, mid-backfill.

Gmail reads a month, with bodies

Gmail is paged by louvain itself through the broker's proxy (the self-hosted broker runs no functions). The runner reads the last N days — the connection's History (days), via after: in the Gmail query, the window fixed when the backfill starts — fetches each message in full and decodes the body (plain part first, HTML stripped, 20k cap), then follows Gmail's history feed from the id captured before the backfill, so mail that arrived meanwhile is not missed. A few fetches in flight with a floor between them stay under Google's per-user quota; a quota refusal is a doubling sleep, not a failure. The broker's stock sync, by contrast, walks the whole mailbox with no window, fetches snippets, and delays new mail behind the full backfill.

Not everything in a mailbox is mail the owner would keep. Spam and trash are never read. Everything else is the connection's Gmail search setting, in Gmail's own syntax — the string you would type in the search box, so the filters you already keep are the filter here: -category:promotions -from:noreply@example.com label:clients. Empty means -category:promotions -category:social; updates and forums are kept by default because receipts, payslips and notifications are where a mailbox's facts tend to live. Gmail evaluates the query on the backfill and again on the live feed (the feed lists every arrival, so the same search is run over the last week and an arrival is read only where Gmail listed it); a withheld message is counted in the runner's log rather than silently absent. When a mailbox is busy enough that the week's listing cannot be enumerated, the runner says so and re-reads the window through the query itself rather than treating what it could not list as excluded — a message it cannot judge is never dropped and passed over. The re-read costs fetches and no extractions: the same mail carries the same idempotency key.

Every brokered connection takes a History (days) setting. A mailbox is years deep and every message is an extraction; set it to 30 to read the last month and nothing older. Records outside the window are counted in the runner's log and never submitted, so a short window is visible as a choice rather than mistaken for a thin source. Leave it empty to read everything the source holds. Grants are never windowed.

Several people connecting mailboxes that hold the same email produce one event per mailbox, each in its owner's container, and one model reading shared between them (see Limits). Nobody is granted a message because they were addressed on it: a grant is evidence that a connected mailbox holds the copy, never an inference from a header.

A connection made before a source folded to one container still holds its per-thread rows. scripts/collapse-containers.ts --org <slug> --connector <key> rewrites them onto the single container — every table, one transaction, and it refuses if folding would let any subject read a container they were not granted — folds the per-thread grant events into the one grant they now express, then scripts/rebuild-spicedb.ts clears the orphaned objects.

The cursor lives in louvain, not in the runner, at /v1/ingest/state. It advances only after ingest has accepted the events it covers: advancing first would turn one failed post into a permanent gap, because the source moves on and louvain would never learn what it missed.

Self-hosting the broker

LOUVAIN_NANGO_URL points at the broker and LOUVAIN_NANGO_SECRET_KEY authenticates to it. The default is the hosted service; the broker is self-hostable, and pointing that variable at your own instance means no customer credential leaves your infrastructure. louvain's core never learns the broker's name either way — these connectors live in ee/ and register through the same public extension API any third party would use.