Four copies of one request can exist
A single LLM request can appear in the application log, the gateway request log, an observability callback and the upstream provider's systems. If the request invokes retrieval or external tools, additional services may receive parts of the prompt. Disabling one dashboard's logging does not establish that the other copies are absent. A complete data-flow diagram should enumerate each processor, region, storage system and administrator role.
Distinguish payload content from operational metadata. Most teams need request identifiers, model IDs, status, duration, token classes and estimated cost to debug production systems. They may not need the full prompt or completion persisted by default. Redaction, hashing and sampling are useful only if they operate before data reaches a log sink; masking a dashboard after storage is weaker.
- Application logs, gateway logs, provider retention and tools are separate scopes.
- Document attachments and tool arguments, not only plain text.
- Verify whether failed requests and exceptions follow the same privacy policy.
What vendor documentation says today
Cloudflare AI Gateway's logging documentation, updated October 26, 2026, states that logs are enabled by default for a gateway. It documents a per-request control to skip the entire log and another to retain metadata without raw request/response payloads. It also warns that those request headers do not necessarily control Logpush exports. Buyers must inspect both gateway settings and export pipelines.
LiteLLM documents message redaction while retaining metadata such as spend, as well as callbacks to external observability tools. Its controls and defaults depend on configuration and the particular callback integration. OpenRouter documents provider-side zero-data-retention routing and separate gateway-level content logging choices. These are not interchangeable guarantees; review current provider-specific terms and the exact request path.
- Cloudflare: verify default logging, payload-only suppression and Logpush separately.
- LiteLLM: verify redaction and all configured callback sinks.
- OpenRouter: verify both router settings and eligible upstream provider policies.
Retention and deletion require evidence
Ask for a retention matrix with separate entries for prompt content, completions, metadata, error traces, security events, backups and support exports. Each row should name the storage region, retention window, deletion mechanism, exception policy and parties with access. A single statement that data is encrypted or not used for training does not answer how long it is stored.
Deletion should be tested with a unique synthetic marker. Send a request containing that marker, confirm where it appears, disable collection, then exercise deletion and search again through authorized audit views. Include exported telemetry and backup retention in the assessment. Avoid using real customer secrets to test redaction; a controlled dummy string is enough to prove whether payloads leak into a log.
- Request a retention matrix and an export/deletion demonstration.
- Check whether deletion is immediate or subject to backup cycles.
- Separate contractual commitments from configurable product behavior.
Zero retention is not the same as private processing
A zero-data-retention provider still processes request content during inference. It does not mean data never leaves the application, that tools cannot store inputs, or that a gateway runtime cannot access payloads. OpenRouter's October 27 documentation explicitly separates eligible provider ZDR from its own metadata and optional content logging. Data residency is a different question again: processing region, support access and billing records may follow different rules.
Enterprise buyers should specify which workloads require a particular processing region, provider, retention policy or contractual data-processing addendum. If no eligible provider satisfies those constraints, the gateway should fail closed rather than silently route to an unapproved endpoint. Verify this with an intentionally unavailable allowed provider; do not rely on a green privacy badge.
- Test region/provider allowlists with a forced unavailable route.
- Ask which entities may access plaintext during inference.
- Review the upstream provider's retention agreement independently.
A metadata-first observability design
Keep a stable opaque request ID and authenticated tenant/application identifiers. Record the selected provider and model, start and finish timestamps, attempt count, HTTP status, input/output/cache token categories, cost estimate and correlation trace ID. For troubleshooting, use time-limited, authorized payload capture on a small synthetic or consented sample rather than indefinite full-text logging.
Operational metadata can still be sensitive. Tenant IDs, model usage patterns and timing can reveal business behavior, so enforce access controls, encryption, retention limits and export audits. Ensure application operators cannot turn on payload logging for another tenant without authorization and a traceable change record. During an incident, verify whether startup scripts, crash reporters or third-party SDKs create additional uncontrolled logs.
- Collect minimum viable metadata; redact before export.
- Limit privileged debugging and record every policy change.
- Test logging behavior for errors, streaming and tool calls.
How to compare gateway candidates
Score vendors on verifiable defaults, per-tenant policies, log sinks, deletion, role-based access, provider controls and contractual clarity. Cloudflare, LiteLLM and OpenRouter provide different mechanisms that require different operational responsibility. AetherGate's security and logging statements should be validated against its actual deployed services, not treated as proof of independently audited controls.
The right answer may be a self-operated proxy with carefully controlled sinks, a managed gateway with enforceable data-processing terms, or direct provider calls for a narrow workload. Select the option that meets the data-flow requirements at an acceptable operating cost. Procurement should request a reproducible test and written evidence, not simply a product demo.