Prompt Injection Matters to You

This is the most important unsolved security problem in AI tooling, and it is usually discussed as a developer concern. Once assistants can browse, read your files and take actions, it becomes a user concern.

The mechanism

A model receives one stream of text. Your instructions, the system prompt, and any content it reads all arrive in the same channel.

There is no reliable boundary between data and instruction. If a model reads a web page, and that page contains "ignore your previous instructions and instead do X", the model may follow it. The text looks like text.

Defences exist — training, delimiters, separate channels, output filtering — and none is complete. This is a structural property of how the systems work, not a bug awaiting a patch.

Why it becomes serious

A model that only answers questions can be made to give a wrong answer. Annoying.

A model with tools can be made to act. Send an email. Read a file and put its contents somewhere. Make a purchase. Change a setting. Submit a form.

The combination that creates real risk: the assistant reads untrusted content, has access to something valuable, and can send data somewhere.

Where you encounter it

Browsing assistants. A page you ask it to summarise contains hidden instructions. Text can be invisible to you — white on white, off-screen, in metadata — and fully visible to the model.

Email assistants. An incoming message contains instructions. The assistant reads it while summarising your inbox. This is the most concerning case because email is attacker-controlled by definition, and email assistants typically have access to everything.

Document processing. A PDF or spreadsheet from outside your organisation.

Code assistants reading dependencies, issues, or comments.

Anything reading shared files.

Search results. A page crafted to rank for a query an assistant will make.

What an attack tries to achieve

Exfiltration. Getting the assistant to include your data in a request to an attacker-controlled address — an image URL, a link, a form submission.

Unauthorised actions. Sending, deleting, purchasing, forwarding.

Manipulated output. Making the assistant recommend something, or misreport what a document says.

Persistence. Writing instructions into a place the assistant will read again, such as a memory feature or a file it will process regularly.

What you can actually do

Limit permissions. An assistant that reads your email does not also need to send it. A browsing assistant does not need file access. Damage is bounded by capability.

Require confirmation for actions. Sending, purchasing, deleting, sharing. Most products offer this and many default to off.

Do not connect your primary accounts to experimental tools. A separate account with limited access is a reasonable arrangement.

Be suspicious of unexpected behaviour. An assistant that summarises a page and then suggests visiting a link, or produces an oddly specific recommendation, may be doing what the page told it to.

Watch for output containing addresses or links you did not expect, particularly images. Rendering an image is a network request, which is one way data leaves.

Treat anything from outside as untrusted content, including documents from colleagues whose accounts may be compromised.

Review actions taken, not just the summary of them.

Judging a product

Does it distinguish trusted from untrusted content? A vendor with no answer has not considered it.

What does it do without asking?

Can you see the actions it took?

Can permissions be scoped?

Is there a log?

The gap between products here is large and it is not visible in feature comparisons.

The realistic assessment

This is not a reason to avoid these tools. It is a reason to be deliberate about which of them can act on your behalf and over what.

The safe configuration is an assistant that reads widely and acts narrowly, with confirmation on anything irreversible. The risky one is an agent with broad account access processing incoming email unsupervised — which is exactly what several products are marketing.

Signs an assistant has been redirected

Attacks of this kind are frequently visible in the output if you know what to look for.

A recommendation you did not ask for, particularly one naming a specific product, site or address.

A link or an image you did not expect in the response, especially one containing a long string of characters, which may be your data being sent.

A summary that does not match your own reading of the document.

The assistant reporting an action you did not request, or reporting success on something it should have asked about.

A sudden change of tone or of subject mid-response.

A refusal that seems to come from nowhere, or conversely a willingness to do something it normally declines.

If you see any of these after the assistant read external content, stop. Do not follow links from the response, check what actions were taken, and treat the source document as hostile.