7 min read

What limits the damage when something goes wrong?

Sooner or later a token is copied, and sooner or later the agent is talked into something. A good design assumes both, and decides in advance how little each one costs.

This lesson builds on the ones before it. If you arrived here first and a term is unfamiliar, lesson 1 is where they start, and each is in the glossary.

Two failures to plan for

Every lesson so far has ended on a caution. Here they are, side by side.

  • A credential leaks. Tokens end up in logs, in error reports, in debugging sessions and in the memory of the agent that used them. A plain token works for whoever holds a copy.
  • The agent is talked into something. An agent reads web pages, emails, documents and the output of tools, and any of them can contain text written to be taken as instructions. This is called prompt injection, and nobody has a way to rule it out.

You cannot prevent either for certain. What you can decide is what each one costs. Five controls do most of that work. Before you look, guess: does any one of them help with everything that can go wrong?

What goes wrong

What goes wrong: A token is copied from a log

Does it help?
A short-lived tokenPartlyThe copy works until the token expires. Minutes, not months.
A narrow scopePartlyIt limits the copy to reading one calendar. It does not stop it.
A token bound to a keyYesAn API that checks for the key refuses the token alone, and the key was never in the log. A proof caught beside it can be reused at that one address, briefly.
Credentials kept out of the model’s reachPartlyIt removes one place to leak from. Logs and proxies are others.
Asking a person firstNoReading a calendar is not a step anyone would be asked about.

A stolen token is a real token. Two things limit what it is worth: time, because it expires, and proof, because an API that checks the binding refuses a token that arrives without proof of the key.

What goes wrong: The agent is talked into doing something

Does it help?
A short-lived tokenNoThe agent is using its own, valid token. Expiry is beside the point.
A narrow scopePartlyA read-only token cannot delete, so this request is refused. Whatever the token does allow, the agent can still be talked into.
A token bound to a keyNoThe agent holds the key. It is the agent that was fooled.
Credentials kept out of the model’s reachNoNothing is being leaked. The agent is misusing access it really has.
Asking a person firstYesIf the API will not delete without Alice’s approval, the request stops in front of her.

A tricked agent uses real credentials, so neither time nor proof helps. What holds is what the token cannot do, and a person standing in front of anything that cannot be undone.

What goes wrong: The agent is talked into leaking its credentials

Does it help?
A short-lived tokenPartlyAn access token works only for the minutes it has left. A refresh token or an API key keeps working.
A narrow scopePartlyIt limits what the thief gets.
A token bound to a keyYesA bound token is refused without its key, as long as the key is somewhere the model cannot read.
Credentials kept out of the model’s reachYesThe model cannot repeat a secret it was never shown. It must not be able to choose where your code sends that secret, either.
Asking a person firstNoNobody is asked before text is sent.

Anything the model can read, text on a web page can ask it to repeat. So the strongest answer is about where things are kept: the credential never enters the model’s context, and ideally never enters its sandbox.

All three at a glance
What goes wrongA short-lived tokenA narrow scopeA token bound to a keyCredentials kept out of the model’s reachAsking a person first
A token is copied from a logPartlyPartlyYesPartlyNo
The agent is talked into doing somethingNoPartlyNoNoYes
The agent is talked into leaking its credentialsPartlyPartlyYesYesNo

Read the table down a column and no control says yes three times. That is why designs use several: each one covers a failure the others miss.

The five controls

Keep tokens short-lived. Minutes for an access token. A refresh token makes that painless for the agent, and the short life limits what a copy is worth. It is the cheapest control here, and the one already built in.

Ask for little, and ask late. One audience and the narrowest scope that does the job, requested when a task needs it. An agent that holds only calendar.read cannot be talked into deleting anything. It can still be talked into reading, so a narrow scope shrinks the damage without removing it.

And what it reads, it can be talked into sending somewhere else. No token setting stops that. Limit where the agent can send things, and take most care with an agent that can do all three: read private data, read text written by strangers, and send messages out.

Bind the token to a key. The agent makes a key pair and keeps the private half. The token names the public half, and every request carries a fresh signature made with the private one. An API that checks for that signature then refuses a copy of the token sent alone. This is a sender-constrained token. The signature is called proof of possession, and one common way to do it is DPoP. The private key is held by your code or by the platform’s key store, never where the model can read it: a key the model could repeat would be no better than a token it could repeat.

Keep credentials out of the model’s reach. The model decides which tool to call. Your code makes the call and attaches the token, and only to the API that token is for. The address comes from your code, never from the model: a model that can choose where a request goes can be talked into sending the token there without ever seeing it. Take secrets out of anything handed back to the model, error messages included. A stronger version puts the secret outside the agent’s sandbox altogether, behind a proxy that adds it to outgoing requests and only for the hosts it was meant for.

Ask a person when it matters. For anything that cannot be undone, the API refuses the everyday token and says what it needs. The agent sends the person back to the authorization server, to sign in again or to approve more. This is called step-up. When the request also describes the one action, the token that comes back is good for exactly that. If the person is not at the keyboard, the authorization server can reach them on their phone instead.

When a password is the only way in

Lesson 3 said an agent never asks a user for their password. Some services leave no alternative: an old internal system that knows only usernames and passwords, or a website with a sign-in page and no API. Signing in there as a person is impersonation however carefully it is done. The service cannot tell the agent from the person and will not limit it. So the work is to limit what that costs.

  • Give the agent an account of its own, where the service allows one, with only the access the job needs. Its actions are then recorded as the agent’s, and the account can be closed without locking a person out.
  • Keep that account’s password away from the model. It lives in a secrets store. Your code, or a proxy outside the agent’s sandbox, fills it in. The model asks to be signed in and never sees what was typed. This is for the agent’s own account. A person’s password is never stored at all, as the next point says.
  • If it has to be a person’s account, the person signs in. Show them the service’s own sign-in page, and let the password and any code from their phone go from them to the service. Nothing they type should pass through the model or be saved. What the agent holds afterward is a signed-in session, and while it lasts that session can do what the password could. Keep it short, keep it out of the model’s reach, and end it when the task is done.
  • Put your own limits in front. The service will refuse the agent nothing the account can do. So the limit has to be in the code that drives the session, not in the prompt: which pages it may open, and which actions wait for a person to approve.

None of this makes the arrangement good. It makes it smaller. Check that the service allows automated use at all, and where you have any say, ask its owner for the real thing: an API that accepts tokens with a scope.

One rule under all of it

The agent is the part of your system that reads untrusted text and acts on it. So nothing that matters should depend on the agent behaving. Put the limit in the token, enforce it at the API, keep the secret out of reach, and leave the agent free to be as helpful as it can within that.

That is the last of the lessons. You have the model the rest of this guide is built on: what an API can see, what a token says, how an agent gets one for a person or for itself, what happens down a chain, and what limits the damage. The pages from here on are the same ideas at full size.

Check yourself

Answer before you read on. Nothing is recorded.

1An attacker copies a token that is bound to a key. They do not have the key. What can they do with the token?
  • Right. Every request carries a signature made with the private key. The token names the matching public key, so a copy sent alone fails the check. The check is the API’s job: one that ignores the binding would accept the copy.

  • Not quite. That is true of a plain bearer token. A bound token has to arrive with a fresh signature made by the private key.

  • Not quite. They cannot make a proof at all. That takes the private key, which was never sent anywhere.

2An agent holding a read-only calendar token is tricked by a web page into trying to delete every event. What stops it?
  • Not quite. The page’s text is competing with the prompt, and may win. A prompt is an instruction to the model, not a limit on it.

  • Not quite. The token is valid when the agent uses it. A short life limits a thief, not the token’s rightful holder.

  • Right. The token’s scope does not cover deleting, so the API refuses. It checks on every request, and it makes no difference why the agent tried.

3Where should an agent’s long-lived credentials be kept?
  • Not quite. Code the agent runs there can read its environment, and so can anything that code was talked into running.

  • Right. The model asks for a tool to be called. Your code, or a proxy outside the sandbox, attaches the credential, and only for the address it was meant for. The model sees the result and never the secret.

  • Not quite. A tool’s description is part of what the model reads, so it can be repeated to anyone whose text the model reads.

What to keep

  • Plan for two failures: a credential leaks, and the agent is talked into something.
  • Time and proof limit a leak: short lives, and tokens bound to a key.
  • Scope and approval limit a tricked agent: narrow tokens, and a person asked at the API’s insistence.
  • Anything the model can read can be drawn out of it. Keep credentials where it cannot, and let your code decide where they are sent.
  • No token setting stops an agent passing on what it was allowed to read. Limit where it can send.
  • A limit in a prompt is a suggestion. A limit at the API is a control.

The names for these

The rest of this guide, and every specification, uses these terms. Each links to its entry in the glossary.

In this lessonIts nameDefined as
Text written to be taken as instructionsPrompt injectionText an agent reads, in a web page, an email or a tool result, that is written to be taken as instructions. It is why limits belong at the API and credentials belong outside the model’s context.
A token only the holder of a key can useSender-constrained tokenA token that only the holder of a particular key can use. The opposite of a bearer token.
Showing, on each request, that you hold the keyProof of possessionEvidence, supplied with a request, that the sender holds a particular private key. DPoP, mutual TLS and HTTP message signatures all provide it.
One common way of doing thatDPoPA mechanism that binds a token to a key held by the client. Each request carries a small signed proof.
The API sending the agent back for a fresh sign-in or a wider approvalStep-upAn API’s demand for stronger or fresher authentication, or a broader scope, before it will perform a sensitive action.

Go deeper

The same ground at full depth, when you want it.