The total-cost formula buyers should actually use
Monthly total cost = provider inference charges + gateway subscription + gateway usage or BYOK fees + hosting and operations + paid logging/retention + migration amortization. If you run more than one environment, include staging and disaster recovery. Do not double-count inference when a marketplace charges credits instead of direct provider billing.
Cost per successful task = monthly total cost divided by tasks that meet your acceptance criteria. A cheaper model that forces expensive retries or produces unacceptable answers may raise total cost. A model router only creates value when its decisions preserve the quality and latency your product needs.
- Provider input, output and cached-token charges, including any minimums.
- Gateway subscription, credit top-up percentage, per-request and BYOK fees.
- Compute, database, egress, backups, monitoring and on-call time.
- Data retention, audit export, compliance and support contracts.
- Migration, SDK changes, regression tests and ongoing model evaluation.
What published 2026 prices do — and do not — establish
Cloudflare says AI Gateway core features including analytics, caching and rate limiting are free, while logging depends on Workers Logs or legacy log limits. Portkey lists a free evaluation tier and a $49/month Production tier, with enterprise pricing negotiated. Bifrost markets a free open-source core and a separate enterprise offering; operating that core is not costless.
OpenRouter's current pricing lists a 5.5% fee on Standard credit purchases and 8% on Business credit purchases. Its BYOK terms include $25,000/month of list-price inference without OpenRouter fees on Standard and Business, then a 5% fee above the allowance. These are distinct billing paths; do not add the credit top-up fee to BYOK provider invoices as though both apply to the same request.
AetherGate lists Team at $249.99/month, Enterprise Growth at $999/month and Enterprise Scale at $2,499/month. Customers pay their own upstream providers separately and AetherGate says it does not add a token markup. Kong's published pricing includes control-plane, request-volume and proxied-model dimensions; get a quote for the exact deployment. LiteLLM can be self-operated, but production operations and enterprise-only features can change the effective cost.
- Lowest published starting gateway fee is not lowest total cost for every team.
- Vendor prices are snapshots checked October 25, 2026; read official pricing before purchase.
- Open-source and free-tier products have different usage rights, service guarantees and maintenance burdens.
- Do not compare vendor-published savings percentages as if they came from a shared independent benchmark.
Worked example: when does a $249.99 subscription break even?
Suppose a team already spends $10,000/month on provider inference. A $249.99 gateway subscription would require approximately a 2.50% reduction in provider spending to offset the subscription alone: 249.99 / 10,000 = 0.024999. At $25,000/month in provider spending, the equivalent threshold is about 1.00%. Those are arithmetic break-even points, not predicted savings from AetherGate.
Now suppose the gateway actually saves 4% of a $10,000 provider bill while maintaining quality and latency. The modeled provider saving is $400. Subtract a $249.99 subscription and the result is $150.01/month before integration, operations, taxes and any other fees. If savings are only 1%, the same example loses $149.99/month. The difference is why a trial must measure actual workload behavior.
- Illustrative scenario A: $10,000 provider spend, 4% measured reduction, $249.99 platform fee → $150.01 before other costs.
- Illustrative scenario B: $10,000 provider spend, 1% reduction, $249.99 platform fee → negative $149.99 before other costs.
- These scenarios assume the workload stays constant and accepted-output quality is unchanged; neither outcome is a product claim.
Where cost reduction really comes from
Request caching helps only when eligible requests repeat. Cloudflare documents exact-match caching: differences in the full request body can create different cache entries. Repeated prompts with stable content can benefit, while personalized requests often cannot. Prompt-prefix caching offered by upstream model providers is a different mechanism and needs its own token-price accounting.
Routing to cheaper models can lower average inference cost but must be evaluated on actual outputs. Context trimming can reduce input tokens yet harm retrieval or instructions if done indiscriminately. Retries and failovers can improve completion rate but also increase spend. A trustworthy cost report records original model, served model, retries, token classes and whether the task passed quality checks.
A simple 14-day cost audit
Start by exporting two weeks of provider invoices and gateway logs, with sensitive prompts removed. Group by application, model, task type and provider. Compute cost per successful task, p50/p95 latency and retry rate. Then trial one optimization at a time on a representative sample, comparing to a holdout with the same quality checks.
Do not let the gateway optimize only for cheaper tokens. Put hard limits on which models may serve high-stakes tasks and define an acceptance test before changing routing. A savings claim should include sample size, time period, quality acceptance, cost definition and exclusions.
- Week 1: baseline invoices, traffic, output quality and incident rate.
- Week 2: test routing, cache eligibility or context reduction separately.
- Report confidence limits, changed traffic mix and excluded failures.
- Include the subscription and engineering time before declaring positive ROI.
Who should choose which economics?
For experimentation or low volume, a free core, small paid tier or direct provider API may be the rational choice. For teams with existing Cloudflare infrastructure, test its free gateway features first. For teams that can run and maintain their own proxy, LiteLLM or Bifrost may offer strong economics. For organizations with larger bills and a need for provider-direct BYOK, AetherGate's fixed subscription may be worth testing against the alternatives—but only after the measured savings or governance benefits justify the fee.
The fair question is not 'which vendor is cheapest?' It is 'which deployment produces the lowest reliable cost for our accepted workload under our operational constraints?' That answer requires your data, not a generic top-ten article.