Gateway and overlook

One local connection holding your provider keys, and an optional link to the SaaS overlook that never sees your code.

What it is#

The gateway is an optional loopback proxy, default 127.0.0.1:8788, for seats that talk to models with API keys. It does not replace Cursor / Claude / Codex plan logins — those seats keep their vendor account; Orchemax still governs them (MCP, bus, locks) without routing their model traffic.

  API-key agent ---> orch gateway ---> provider A  (key 1, key 2, key 3)
                          |        \--> provider B  (key 1, key 2)
                          |
                          +-- crush (shrink the payload)
                          +-- rotate on rate limits, cooldown the key
                          +-- optional exact-match response cache

  Cursor / Claude (plan) / Codex ---> vendor model API   (no gateway)

Your keys go in a machine-home vault — keys.env, shared by every workspace — several per provider when you have them:

orch gateway keys add <provider> [key]     # probes connectivity first
orch gateway keys list [provider]          # hints and tier only, never the secret
orch gateway keys remove <provider> <hint>
orch gateway status                        # live / cooldown, ring position, rotation counts

orch gateway status reads straight from the router and is content-free: no keys, no prompts. The agent never holds a key and never sees a 429 — the gateway rotates through the provider's ring and crushes every call.

Two places a provider key can live#

The gateway does not ask where you are on the plan ladder; it asks a credential resolver for a key and gets one. Which backend answers is decided once, by the plan:

One person with their own key has no CTO to entrust it to, and we are not going to ask them to upload it. A company is the other way round: a key on every laptop is a key that walks out the door with whoever leaves.

On Team and Enterprise the CTO parks the ring once:

orch server keys add <provider> <key> [more keys…]   # park or rotate
orch server keys list                               # hints only, never the secret

Each machine then leases a copy for fifteen minutes — long enough to outlive any single model call, short enough that a credential in a process nobody sealed stops working on its own. The gateway renews a minute before expiry, so a refresh is an ordinary step and never a failed request. The lease is held in memory: it is not written to keys.env, not written to orch.db, and masked out of logs like any other credential. Every lease is recorded against the device that asked for it, which is what makes the account's trail able to say which machine spent what.

The ceiling, stated plainly: a leased key does reach the machine's memory, so this buys central rotation and immediate revocation — unlink the device and its next lease fails — rather than a key the machine never sees at all. Nothing is left behind on disk when the machine goes away.

Two policy keys carry this in your entitlements, and a tenant row may override either:

A workshop with no server keeps working: below Team nothing on this path touches the network, and a Team machine that cannot reach the server is told so rather than quietly falling back to whatever key happens to be on disk.

Point an API-key agent at it with orch gateway connect (OpenAI-compat paste) or connect anthropic; optionally connect opencode --allow-user-scope writes OpenCode's JSON. Do not connect plan seats (Cursor / Claude subscription / Codex).

The response cache is off by default. gateway.cache.ttl_sec is the owner's decision — 0 disables it — streaming is never cached, entries are keyed per credential, and a cached response carries X-Orchemax-Cache: hit without counting as an upstream attempt.

Plan-billed seats keep their own login#

Anthropic's terms forbid a third party from routing requests through Free, Pro or Max credentials or from storing those tokens, so the gateway answers 403 with a one-line refusal to any request carrying an Anthropic OAuth bearer or the OAuth beta header. A Claude seat signed in with its plan routes no model traffic through Orchemax; an Anthropic API key is the only credential the passthrough lane will carry.

This is expressed per preset as agents.auth.<preset>:

A CLI that ignores a custom base URL while signed in defaults to account: forcing credentials on it would replace the subscription login and disable its connectors without buying anything.

Every worker gets its own key#

A dispatched worker never receives the workspace's gateway key. At dispatch Orchemax mints a key scoped to that run — orch_wk_…, bound to the session and the role, expiring at the dispatch timeout plus ten minutes — and that is the only model credential in the worker's environment. Provider keys the Chair itself holds are stripped from what the worker inherits.

Three things end the key, and the first one to happen wins:

• Sealing. The agent stops, the key is revoked in the same step.
• Expiry. A key nobody revoked stops authenticating on its own.
• The reaper. A worker killed mid-run never reaches sealing; the reap pass revokes the keys of every abandoned session.

orch gateway key workers     # live worker keys: id, session, role, expiry — never the secret

Because the key names its session, the gateway knows whose call it is serving: usage lands on that dispatch even though no worker CLI sends a session header of its own. Only the hash is stored, and orch_wk_ is masked out of logs like any other credential.

Sending usage to your own observability stack#

orch usage export ships the usage ledger, opt-in and only where you point it:

orch usage export                                    # gen_ai.*-keyed JSONL on stdout
orch usage export --otlp http://127.0.0.1:4318       # /v1/traces is appended
orch usage export --otlp $URL --since 24h --header "authorization=Bearer $TOKEN"

With --otlp each model call travels as one OpenTelemetry CLIENT span in OTLP/HTTP JSON, carrying gen_ai.system, gen_ai.request.model, gen_ai.operation.name and gen_ai.usage.input_tokens/output_tokens, plus orch.session and orch.agent. Calls from one session share a trace. Spans travel in batches, each POST with a ten-second deadline, and a collector that refuses fails the command instead of being swallowed.

An attempt that never reached a provider carries no gen_ai.usage.* key at all — orch.terminal says why, rather than a zero pretending to be a measurement. Nothing else crosses the wire: the attempt ledger behind this never stored a prompt or a completion, so there is nothing there to redact.

The overlook#

Linking a workspace to the Orchemax overlook is optional and reversible:

orch server login       # sign in and link this device
orch server link --token <token>
orch server status
orch server entitlements
orch server watch       # hold the live channel open and drain mail as it arrives
orch server unlink

orch server watch subscribes to GET /v1/orch/stream, a server-sent-events channel behind the same device token as every other /v1/orch route. The event carries one field, the outbox's highest pending id: it is a doorbell, not a delivery. The mail itself is drained through the sync pull that already exists, so a workshop three events behind still catches up from a single ring. An idle channel writes a comment line every 25 seconds to survive proxies that drop a silent stream.

Polling stays the fallback. A server without the route, a dropped connection or a reconnect still climbing its backoff costs latency, never delivery: --poll (default 60s) drains on its own interval regardless.

What crosses the wire is counters and metadata — entitlements, content-free work pulse, seat events. Source code does not. The usage ledger the client keeps never stores prompts, which is what makes the read-out safe to quote at all. An unlinked workshop keeps working, offline.

Agents check the plan with entitlements_status first. Denials are plain language for humans (and agents should quote them as-is):

• Outside schedule → Outside the Team work schedule — agents cannot dispatch right now… (see Team work schedules)
• Missing plan feature → X needs the Team plan (you have free). Upgrade: …
• Concurrent workers full → free plan allows 1 concurrent worker; 1 already running…

Tell the human; do not invent workarounds outside Orchemax. Structured fields (capability, plan_have, plan_need) still exist on the error value for tools; they are no longer the user-facing sentence.

Assist HITL (Team+)#

From Team upward the overlook includes Telegram Assist: link a chat under /app/assist, then /approve / /deny / /reject on the Orchemax bot (or the console Assist buttons) write HITL notices for the tenant. Professional includes soft Telegram notify (telegram_notify) without org Assist. Details: Notifications and approvals.

Live control from the console (Team+)#

Four values reach a linked installation live, inside the signed lease, and apply on its next entitlements refresh — nothing restarts:

Two rules hold for all four:

• A push only tightens. The server refuses a quota above what the plan allows, and the installation clamps again on its own side.
• A push is not a local setting. It rides the lease signature, so the workshop's own plan.enforce: false does not switch it off, and editing the machine's mirror invalidates the lease instead of loosening a cap.

webhook.set never carries a credential: give the name of the environment variable holding the signing secret (ORCH_HOOK_SECRET), not its value. The server rejects a payload whose leaves look like a live key.

On the machine, orch server entitlements fetches the new lease immediately instead of waiting for the next sync.

How it relates to the rest#

• Crush savings and cache hits show up in orch usage, as bytes kept off the wire rather than as a claim.
• Providers are catalogued user-wide with orch gateway setup, writing ai.yaml in the machine home; a workspace orch.yaml can override ai.providers.
• Secrets never go in chat. The vault or a user environment variable, then orch gateway.

Nothing to run here#

The gateway is optional. An agent with its own key works without it; you turn it on when you want one connection, key rotation and crush instead of one configuration per provider per agent.