An AI assistant helping with customer service may need to read a message, look up an account and prepare a reply. Each capability sounds reasonable on its own. Together, they create a question about who controls the information flowing between them.
In June 2025, Simon Willison described a risky combination he calls the “lethal trifecta”: private data, untrusted content and external communication.1 An attacker-controlled message can try to persuade a model to retrieve information and send it somewhere it should not go.
The business needs more than an instruction telling the assistant to be careful. We need boundaries the application can enforce.
Think of a mailroom where incoming envelopes can contain instructions to change the building’s access permissions. The envelope is part of the work. It shouldn’t also be the manager.
Map one support task before connecting the tools
For a fictional service company, the first task is drafting a response to a customer’s question about an existing request.
Draw the steps in plain language: receive the message, establish the customer context, retrieve permitted information, draft the answer and let an authorized person send it.
Treat the customer’s message as content
The message may contain a genuine question, pasted material or malicious instructions. The assistant must be able to work with its content without letting that content grant new authority.
A sentence claiming that the CEO has approved access to another customer’s records remains a sentence in an incoming message. The application’s identity and permission checks should determine access.
This also applies to retrieved documents and tool results. A CRM note can contain text originally supplied by someone outside the organization. Storing it internally does not automatically make every instruction inside it trustworthy.
See how we connected Mansa Medical’s website to its CRM to support an organised sales process.
Limit the record lookup to the actual task
The service retrieving account information should enforce the relevant user, tenant and record boundaries. It should return only the fields needed for this support task.
A tool that can export the full CRM is a much broader capability than one that can retrieve the current request’s permitted status. The broader tool may be convenient during development and unnecessary in the finished workflow.
The model’s proposed record identifier is an input to validate, not proof of entitlement. Authorization belongs in the service carrying out the lookup.
Separate reading, drafting and sending
These operations have different consequences. An assistant can help prepare language without being allowed to choose arbitrary recipients or send arbitrary attachments.
That distinction gives the team practical ways to reduce the risky combination.
Keep the destination outside untrusted control
For the sample workflow, the application could bind the reply to a verified conversation and prevent the model from replacing its recipient. It could also restrict which information may be included and require an authorized review before sending.
That review must be concrete. Show the intended recipient, actual message and attachments together. A vague approval button offers less help if the person cannot see what they are approving.
Also consider other communication routes. External requests, rendered links and loaded resources can move data too. Removing a send-email tool while leaving unrestricted outbound requests does not establish that information cannot leave.
A June 2025 research paper by Luca Beurer-Kellner and colleagues examines constrained agent designs and the trade-offs between flexibility and security.4 Its broader lesson is that system-level limits matter; a general-purpose model prompt is not a dependable permission boundary.
Pairs well with MCP can connect the AI assistant. It cannot organize the cupboard.
Test what happens when the content disagrees with the task
Use synthetic records and deliberately hostile test content. Try a message that asks for another account, claims new authority or attempts to change the destination of the reply.
The expected outcome should be established before the test. Which information must remain inaccessible? Which actions must be impossible? Where should the workflow stop or ask a person to take over?
Inspect the actual tool requests and the final output. A model refusing in its prose is insufficient if an earlier tool call already retrieved material it should never have received.
Keep useful audit information without turning logs into another unrestricted store of sensitive content. The team needs to understand the operation, the permissions applied and the result.
I would begin with the narrow draft-and-review task, then consider additional authority only where the business benefit justifies the controls and testing.
The point is to make a helpful assistant possible within a known job. Once the permission boxes are clear, the assistant can spend its energy explaining the customer’s status rather than deciding what the company meant by trust.