The agency problem is attribution before optimization
Imagine three customers: a support assistant for a retailer, a document classifier for a legal operations team and a marketing research workflow. Each has different approved models, monthly ceilings, log-retention requirements and people authorized to see usage. Sending every request through a single shared provider key and recording only total tokens makes a later invoice dispute almost impossible to resolve.
Define a stable tenant identifier on the server, an application identifier within that tenant, and a request identifier for every call. Never trust a customer-supplied tenant label without binding it to authenticated authorization. Cost reporting should identify the billed upstream provider and the gateway fee separately, with a clear distinction between measured usage and estimated cost. Avoid storing prompt text merely to achieve financial attribution.
- Tenant: who owns the workload and has authority over its data.
- Application: which product or workflow generated the request.
- Provider account: whose invoice ultimately pays for inference.
- Request: the traceable unit for usage, errors, model choice and policy decisions.
Choose a tenancy model before choosing a gateway
One model is agency-owned provider accounts with separate application keys and internal cost allocation. It is operationally convenient but can create disputes about who owns credits, rate limits and data processing. Another model is client-owned provider accounts, where the agency manages an integration but the client receives the provider invoice. A third model is a separately deployed gateway per client. That can improve infrastructure separation, but increases hosting, updates and on-call burden.
Do not equate a workspace filter with tenant isolation. Ask which database rows, encrypted credentials, logs, routing policies, administrator roles and backup copies are shared. Test whether a key from client A can list or use client B's models, change B's limits, read B's usage, or trigger a fallback through B's provider account. Record the actual result. A vendor's claim of encryption at rest does not establish that its application runtime cannot access plaintext credentials.
- Document key ownership, rotation responsibility and offboarding procedures.
- Check administrative roles and audit evidence for cross-client changes.
- Identify shared failure domains: database, gateway runtime, rate limit pools and providers.
- Require explicit data-retention and export terms for each client.
What vendor documentation can establish—and what it cannot
LiteLLM documents virtual keys with model access and spend tracking at key, user and team levels. Its documentation states that virtual-key and budget features require PostgreSQL; this means the database is part of your reliability and security boundary. It also describes nuanced inheritance between team keys, owner roles and budget settings. Verify the precise version and permissions you deploy instead of assuming that every key inherits the same restrictions.
Cloudflare AI Gateway documents spend limits scoped by custom metadata such as a user or team. It distinguishes identity derived from authenticated Cloudflare Access from identifiers supplied in request metadata. If an agency passes an untrusted client-provided user_id, it must enforce the binding in its own application. Cloudflare also describes its cost tracking as best-effort estimation, not a substitute for upstream provider invoices. Portkey documents conditional routing based on metadata; that is useful for segmenting policies but is not, by itself, proof of customer data isolation.
- LiteLLM: validate team/key ownership and database-backed budget enforcement.
- Cloudflare: validate metadata trust, rule limits and estimated-versus-invoiced cost.
- Portkey: validate routing conditions, credential scope and trace visibility.
- AetherGate: request a current demonstration of per-key policy, workspace boundaries and provider-key custody; do not assume enterprise isolation from marketing copy.
Run a three-client isolation drill
Create three synthetic tenants with different approved models and deliberately small test budgets. Generate traffic from each tenant, including a burst of parallel requests from one tenant. Confirm that the other two continue to operate and that the budget-exceeded event is attributed to the correct account. Then attempt a cross-tenant query using a low-privilege key and verify it is denied, with an audit record that does not reveal sensitive prompt content.
Next, simulate an upstream provider rate limit. If tenants share an upstream account, a provider-wide quota can still affect all of them even when the gateway enforces separate application budgets. Test the behavior and document the residual risk. Finally, revoke one tenant's application key and provider credential while the other tenants are active. A good result is predictable isolation, not an impressive demo dashboard.
- Check model allowlists, budget enforcement and policy-change permissions.
- Compare gateway records with actual provider invoices over the same window.
- Verify export and deletion behavior using non-sensitive test data.
- Measure p95 latency and failed-request rate for unaffected tenants during the drill.
Pricing and handoff: the agency margin calculation
The gateway subscription is only one line item. Include provider inference, hosting, database, support, monitoring, staff time, audit exports and migration. If an agency charges clients for AI usage, its contract should specify whether the amount is a pass-through provider charge, an estimate, or a separate managed-service fee. Do not silently present gateway cost estimates as exact provider invoices.
Offboarding is where a cheap integration can become expensive. Ask whether a client can take over its provider account, rotate credentials, export usage and policies, and migrate the same request format without the agency's continued involvement. Rehearse one client departure before committing the entire portfolio. This protects both the client and the agency from accidental lock-in.
- Require a reproducible monthly reconciliation and correction process.
- Separate recurring gateway subscription from model inference and support.
- Estimate per-client gross margin only after allocating real operational costs.
- Write an exit plan with a named owner and a tested credential-revocation sequence.
Who should shortlist which architecture?
An agency with a small number of prototypes may reasonably use direct provider integrations or a managed gateway with simple application-level accounting. An agency that wants to operate its own infrastructure should evaluate LiteLLM or another self-hosted gateway with database-backed policy controls. Teams already using Cloudflare can evaluate its metadata-scoped limits and routing. Larger agencies should demand contract-backed identity, audit and retention requirements before consolidating client workloads.
AetherGate can be considered for teams seeking provider-direct BYOK, gateway keys and spend policies under a flat platform subscription, but a purchase decision should depend on a demonstrated isolation drill, verified deployment controls and the client's actual economics. Its public Team plan is $249.99 per month before provider charges; that is a subscription to justify, not an automatic saving.