Self-hosted versus managed AI gateways: the real operating-cost decision

Self-hosted generally means your team runs the gateway application and its supporting services. A managed gateway runs its control or data plane under the vendor's operations, although some vendors offer hybrid or customer-environment deployments. A model marketplace can additionally act as the inference billing intermediary. These categories overlap; ask which exact component is operated by whom and which legal entity invoices provider tokens.

LiteLLM's production deployment documentation describes a proxy, PostgreSQL for keys and spend records, Redis for shared rate-limit and routing state in multi-instance setups, secret management and load balancing. That is a real service architecture, not simply a free executable. A managed gateway may eliminate some of those duties for the buyer but does not eliminate integration testing, upstream quotas, provider invoices or the need for application-side access controls.

Compute monthly total as upstream inference plus gateway fees plus infrastructure plus engineering operations plus observability and retention plus amortized migration work. For self-hosting, include compute, database, Redis, backups, TLS, network egress, upgrades and on-call response. For managed services, include subscription, metered requests or logs, contractual support and any BYOK or credit-purchase fees. If a marketplace resells inference, do not count the same provider charge twice.

Consider an illustrative internal planning case, not a vendor benchmark: a self-hosted stack costing $120 per month in infrastructure and six staff hours at an assumed fully loaded $60/hour costs $480 monthly before upstream inference and incident risk. A managed service priced at $249.99 monthly would be cheaper on those assumptions, but the result reverses if the team already operates the infrastructure and needs only one hour of upkeep. Replace every illustrative input with your actual quotes and observed labor.

Cloudflare's AI Gateway pricing documentation states that core analytics, caching and rate limiting are free, while logs and other features follow their own terms. It also describes a fee for Unified Billing credit purchases. That does not make every workload free: upstream inference, paid log retention and other services still matter. Its documented rate limits differ for Unified Billing and BYOK, so the chosen billing path affects capacity as well as cost.

LiteLLM's production guidance recommends shared Redis once multiple proxy instances are running and describes PostgreSQL-backed key and usage features. Portkey documents managed routing and fallback behavior; alternative provider attempts can change total inference cost. AetherGate's publicly listed Team plan has been $249.99 per month with BYOK provider inference billed separately. Treat all prices as dated observations and verify current contracts and production features before making a decision.

With self-hosting, your team owns deployment upgrades, schema migrations, database recovery, shared counters, incident response and security patching. You can control the environment and investigate deeply, but the gateway becomes another production system with its own failure modes. With managed hosting, the vendor may take on much of the platform operation, yet your application still needs circuit breakers, direct-provider rollback and visibility into upstream incidents.

Ask who can roll back a bad configuration, how long historical traces remain available, what happens when a region fails, and whether the gateway can continue forwarding requests when its management database is unavailable. Run an outage rehearsal: disable a non-production dependency, inspect errors, restore from backup and measure recovery time. A product's marketing uptime percentage is not evidence that your particular workload will meet its SLO.

Self-hosting can keep gateway runtime and credential storage inside your chosen environment, but the application may still decrypt upstream provider keys to send authorized requests. A managed BYOK service likewise needs a documented method for using provider credentials. Encryption at rest does not establish that a compromised application runtime cannot access plaintext keys. Require a threat model that identifies key creation, decryption, rotation, revocation and administrator access.

Compare prompt and response logging defaults, data residency, audit exports, deletion controls, incident notification terms and independent evidence for claimed security certifications. A private deployment is not automatically compliant, and a vendor's enterprise label is not a certification. Test least privilege with a low-scope application key, and verify that it cannot change its own spending policy or read another tenant's records.

Choose two real but permissioned workloads: one latency-sensitive request stream and one batch or agent workflow. Compare direct-provider calls, a self-hosted gateway and a managed candidate using the same models, input distribution and quality rubric. Measure p50/p95 latency, failures, successful-task cost, time spent operating the gateway and time to investigate a deliberately injected error. Include all retries and cached-token classes in cost reconciliation.

Then practice leaving each option. Export policies and usage, revoke the gateway-facing credential, restore the direct-provider path and verify that production-like traffic still works. For a small team with limited operations capacity, a paid managed service may be rational even if its subscription is higher than a free license. For a mature platform team with existing infrastructure, self-hosting may be the better control and cost choice. Neither outcome should be asserted before the trial.