Gateway and overlook

One local connection holding your provider keys, and an optional link to the SaaS overlook that never sees your code.

What it is#

The gateway is a loopback proxy, default 127.0.0.1:8788, that sits between your agents and your model providers.

  agent ---> orch gateway ---> provider A  (key 1, key 2, key 3)
                 |        \--> provider B  (key 1, key 2)
                 |
                 +-- crush (shrink the payload)
                 +-- rotate on rate limits, cooldown the key
                 +-- optional exact-match response cache

Your keys go in a machine-home vault — keys.env, shared by every workspace — several per provider when you have them:

orch gateway keys add <provider> [key]     # probes connectivity first
orch gateway keys list [provider]          # hints and tier only, never the secret
orch gateway keys remove <provider> <hint>
orch gateway status                        # live / cooldown, ring position, rotation counts

orch gateway status reads straight from the router and is content-free: no keys, no prompts. The agent never holds a key and never sees a 429 — the gateway rotates through the provider's ring and crushes every call.

Point an agent at it with orch gateway connect [claude|openai|opencode|commandcode]; OpenCode's config is written automatically.

The response cache is off by default. gateway.cache.ttl_sec is the owner's decision — 0 disables it — streaming is never cached, entries are keyed per credential, and a cached response carries X-Orchemax-Cache: hit without counting as an upstream attempt.

Plan-billed seats keep their own login#

Anthropic's terms forbid a third party from routing requests through Free, Pro or Max credentials or from storing those tokens, so the gateway answers 403 with a one-line refusal to any request carrying an Anthropic OAuth bearer or the OAuth beta header. A Claude seat signed in with its plan routes no model traffic through orch; an Anthropic API key is the only credential the passthrough lane will carry.

This is expressed per preset as agents.auth.<preset>:

Value Meaning
account The agent keeps its own plan login; orch injects no provider credentials
gateway The agent authenticates with the orch virtual key and all model traffic goes through the gateway

A CLI that ignores a custom base URL while signed in defaults to account: forcing credentials on it would replace the subscription login and disable its connectors without buying anything.

The overlook#

Linking a workspace to the Orchemax overlook is optional and reversible:

orch server login       # sign in and link this device
orch server link --token <token>
orch server status
orch server entitlements
orch server unlink

What crosses the wire is counters and metadata — entitlements, content-free work pulse, seat events. Source code does not. The usage ledger the client keeps never stores prompts, which is what makes the read-out safe to quote at all. An unlinked workshop keeps working, offline.

Agents check the plan with entitlements_status first. A denial has a fixed shape — plan deny: capability=… plan_have=… plan_need=… — upgrade to … (URL) — and the correct response is to tell the human, not to route around orch.

How it relates to the rest#

  • Crush savings and cache hits show up in orch usage, as bytes kept off the wire rather than as a claim.
  • Providers are catalogued user-wide with orch gateway setup, writing ai.yaml in the machine home; a workspace orch.yaml can override ai.providers.
  • Secrets never go in chat. The vault or a user environment variable, then orch gateway.

Nothing to run here#

The gateway is optional. An agent with its own key works without it; you turn it on when you want one connection, key rotation and crush instead of one configuration per provider per agent.