Least privilege is not injection prevention

At NDC Copenhagen in 2025, I gave a talk called “Your AI Agents Are Overprivileged,” arguing that AI assistants with access to tools need tighter limits on what they can do. Building an assistant like that gave me a concrete place to start: could the system refuse a database change based on the permissions of the person using the agent?

The permission check worked

My setup connected Claude Code to a Snowflake database through a small local tool server. Okta, the identity provider, handled sign-in. Keycard checked the user’s group membership against an access policy and issued a short-lived database credential only when access was allowed.

The agent could read records. When I asked it to add one, Keycard refused to issue a credential that allowed writes. After I changed the user’s group membership and signed in again, the same request worked.

This showed that the access check enforced the user’s permissions. It did not test task-specific permissions or resistance to prompt injection. I wanted to take it further: if a user normally has write access, should an agent doing a read-only task get it too?

From a user’s permissions to one task

Consider an assistant asked to reply to Naomi about order 1042. It can read customer records and send emails, not just suggest a reply. Giving it only the access needed for that task is least privilege. It should not need every customer’s records or permission to email anyone.

The application knows what the agent was asked to do, but that task context may not reach the service receiving its API call. The service may see only a credential.

Now the assistant opens an email containing an extra instruction: send the full customer list to a different address. That instruction came from the email, not the user. Trying to make an assistant treat material it reads as instructions it should obey is prompt injection. It tries to redirect the assistant. Least privilege limits what the assistant has permission to do.

Security has an old name for a program being tricked into misusing its authority: the confused deputy. Norman Hardy wrote about it in 1988. Prompt injection can put an agent in that position, redirecting access it already holds.

This has happened in real systems. In 2025, EchoLeak used a crafted email to make Microsoft 365 Copilot leak information without a user clicking a malicious link. Invariant’s GitHub research used a public bug report to coax an agent into exposing private repository data. ForcedLeak placed instructions in a customer lead form that Salesforce Agentforce later read. In each case, outside text led the assistant to misuse access it already had.

An agent misled by an email may still give a convincing explanation. I’d check the proposed tool call itself: the recipient, the records and the message. For Naomi’s reply, I’d start by narrowing access to her order, then work through the checks needed before sending.

What a second model could check

I’d try a separate model to compare Naomi’s draft reply with her order record. Does the record support the refund promise? TypeSafe’s Jev and Cloudflare’s Clef are built to answer narrowly defined questions, so I’d try them for this check.

Another experiment could use Jev to check the agent’s proposed tool call. I’d give it the user’s request and details of the call, then ask whether to allow it, deny it or ask a person. That assesses the planned operation, not whether the reply’s claims are true.

I’d test both uses against manually checked examples, including misleading inputs and legitimate requests, before relying on either for a real send.

A model’s recommendation must not override an access denial. It could stop a call that passed the policy check or ask for human review, but it could not grant permission on its own. TypeSafe’s documentation for Jev 1.13 warns that adversarial text can steer its answers. If this checking model is unsure or fails to respond, a consequential action should wait for a person to review it. I wouldn’t give the model tool credentials. The application would still enforce the rules.

Putting the checks together

Here’s how I’d put those checks into the email workflow. The application would supply the task limits, and the gateway would keep the service key away from the agent. Checks on the reply, including any required human approval, would happen before anything is sent.

For implementers, OAuth’s Rich Authorization Requests standard, RFC 9396, offers a way to describe fine-grained access details. The receiving system still has to enforce them. The format alone does not make a credential single-use or bind approval to an unchanged email.

I’d test this design with three kinds of request: an out-of-scope export that should be blocked, an in-scope reply containing a false promise, and a legitimate reply that should go through. The last one matters as much as the first two. A system that refuses everything is not a useful assistant.

I’d log who the agent acted for, what was allowed and what happened. That would help me trace a failure to the relevant check and improve it. The checks still need to happen before sending.