AI credentials can live in local configuration, application memory, a secrets manager or an isolated execution environment. The security question is where they become usable, which processes can read them and what happens when a request runs. Keeping a key out of a prompt does not establish who else can access it.
| Location or pattern | What it tells you | What to check |
|---|---|---|
| Local config or environment | A client or tool process may be able to read the secret. | Can the AI tool read files, run commands or inherit that environment? |
| Secrets manager | Storage and retrieval have dedicated controls. | Which identity retrieves the secret, and where does the returned value go? |
| Gateway or application memory | A service can authenticate calls without putting a key in the model prompt. | Can its host, operators, diagnostics or another process read the live value? |
| Isolated execution environment | Decryption or signing can be restricted to a separate execution boundary. | If attestation is claimed, what does it verify? How is key access restricted, and what leaves the boundary? |
These patterns can coexist: a secret may be stored in one system, decrypted in another and used in a third. Trace the complete request path.
Which credential are you tracing?
Separate the credential used to connect an AI client to a gateway from the credential used to access a downstream service. Revoking the client’s session and revoking a GitHub or cloud grant are different actions. This checklist focuses on downstream API keys, access tokens and signing secrets.
For the broader deployment checklist, read MCP Gateway Security.
Where does a credential become usable?
Follow one request from the AI tool to the destination. Identify the component that retrieves or decrypts the secret, adds an authentication header or signs the request. That is where storage protection becomes a runtime access question.
For example, a GitHub token might be encrypted in storage but loaded into a gateway’s ordinary application memory for each request. The model never receives it, yet that process handles the usable token. Ask which processes and operators can access that memory.
Illustrative authentication paths. The execution boundary depends on the implementation. Bearer tokens travel to the destination; AWS SigV4 does not transmit the secret access key.
What leaves the execution boundary?
OAuth bearer tokens are presented to the destination service. A party holding a valid token can use the authority it grants, so protect the token in storage and transport. Keeping it away from the model narrows one exposure path; it does not mean the token never travels.
AWS Signature Version 4 works differently. A signing key derived from the secret access key is used to calculate a request signature. The secret access key is not transmitted. Requests still carry authentication information, including an access key identifier and, for temporary credentials, a session token.
These are alternative authentication methods. Ask which one each integration uses and identify the particular secret and boundary behind a claim such as “credentials never leave”.
Is encryption at rest enough?
Encryption at rest protects stored data. It does not, by itself, establish who can access a decrypted value during a request. Check the decryption permissions and the process receiving the result. Customer-managed encryption keys also need an access-policy review.
AWS Nitro Enclaves attestation can be used with KMS policies that restrict key operations to matching enclave measurements. Ask for the attestation verification process and the relevant key-policy conditions. An enclave claim alone does not show how a particular deployment is configured.
What should you ask a gateway vendor?
Use one service and one workflow for the evaluation. Ask for a diagram and observable evidence, then check these six points.
1. Where is the usable secret?
Identify every component that can retrieve, decrypt or use it. Include the AI client, gateway, destination and any component that terminates encrypted transport.
2. Who can read it during a request?
Ask about application identities, host access, support access, debugging and memory capture. Distinguish access to ciphertext from permission to decrypt it.
3. What is sent to the destination?
Confirm whether the integration sends a bearer token, an API key or a signed request. Identify any intermediary that could see that authentication material.
4. What authorises credential use?
Demonstrate an allowed request and a policy-denied request. Confirm that the denied request stops at the gateway, rather than reaching the service and being rejected there.
5. What does revocation stop?
Test the client session and downstream grant separately. Check whether other copies, direct connections or already-issued tokens remain usable.
6. What evidence can you inspect?
Request a redacted decision record, the applicable policy and evidence for the claimed execution boundary. Confirm that diagnostics do not capture raw credentials.
Keep downstream permissions narrow as well. Hiding a credential does not reduce its authority, and an allowed request can still misuse that authority. OWASP’s secrets management guidance covers access controls, secret handling in memory, auditing and credential lifecycle management.
How does Gatekeeper handle this?
Gatekeeper applies deterministic policy to tool calls and uses an attested AWS Nitro Enclave for credential decryption and use. Its credential protection documentation describes authenticating the caller and evaluating policy before injecting a credential into an allowed downstream call. The AI tool receives the result without receiving the downstream credential.
The outbound protocol still matters: a bearer token goes to its destination over encrypted transport; a SigV4 request does not transmit the secret access key. Enclave isolation and downstream permissions address different parts of the request path.
For an initial check, use a non-critical test connection in enforcement mode. Configure policy to allow a read operation and deny a write operation. Review both decisions and confirm the denied request did not reach the service. Test revocation separately, and remove direct connections that could bypass Gatekeeper after the cutover.
Common questions about AI credentials
Does keeping a key out of the prompt mean the AI cannot access it?
Not necessarily. A tool that can read a configuration file, inspect an environment variable or run commands may reach a secret without it appearing in the initial prompt. Check the tool’s actual permissions and execution environment.
Does OAuth remove the need to protect credentials?
OAuth can replace direct use of an account password with a delegated access token. That token is still a credential. Check who holds it, its permissions and lifetime, whether refresh tokens exist and how access is revoked.
Are short-lived credentials enough?
A shorter lifetime reduces the period in which a copied credential can be used. It does not decide whether a particular action should run while the credential is valid. Review both the credential’s authority and the controls on its use.
Can a gateway protect credentials while still sending them downstream?
Yes. A gateway can keep a bearer token away from the AI client while presenting it to the destination service over TLS. Describe that boundary precisely rather than saying the token never leaves the gateway.
What does attestation prove?
Attestation provides signed evidence about an enclave’s measured environment. When tied to a suitable key policy, it can constrain where decryption is permitted. It does not establish that every application rule is correct or that the downstream system has appropriate permissions.
Start with one credential path
Choose a workflow your team wants to enable. Trace the credential from storage to use, identify what crosses each boundary and test an allowed call, a denied call and revocation. That gives security and platform teams a concrete basis for evaluating the design.
Gatekeeper puts policy and credential use inside an attested execution boundary. Explore Gatekeeper.
Related reading: Tool-call policy: should a model decide what AI can do?