1. Threat model before features
Assume an attacker can read application environment variables, access databases or compromise the application runtime. Document which independent systems must remain trustworthy.
2. Keep provider credentials outside the app
Encrypting keys in the same process as an application does not isolate them from a runtime compromise. Independent custody requires distinct administrative authority and a broker that never returns raw provider credentials to the app.
3. Authorize intent, not just possession
A valid credential proves possession, not permission for every provider, model, destination, budget or request. Bind authorization to those parameters and reject changed requests.
4. Make revocation durable and tenant-wide
A revoked signing key or contract should remain invalid across every credential of that tenant. Test stale policy, restart, concurrent nodes and fallback paths.
5. Reject replay and overspend
Enforce single-use or time-bounded authorization, request identity, immutable limits and atomic admission accounting before provider dispatch.
6. Verify evidence independently
An audit log under the same administrator can be rewritten. Signed ledger checkpoints retained by customers can offer stronger tamper detection, subject to the verifier and key-management assumptions.
7. Test failure modes
Practice revoked authorization, key rollover, broker unavailability, delayed telemetry, corrupted records, missing checkpoints and partial deployments. Fail closed for requests that need independent custody.