Agents: What They Cannot Do

"Agent" is used for everything from a chatbot with a search button to software that operates a computer unsupervised. The useful definition is narrower.

The definition worth using

An agent is a model that can take actions, observe results, and decide what to do next, repeatedly, without asking you between steps.

Three components: tools it can invoke, a loop that continues until a goal is met or a limit is hit, and permission to act without per-step approval.

A model that searches once and answers is not an agent. A model that searches, reads, decides the answer is incomplete, searches differently, and repeats, is.

What they are genuinely good at

Multi-step research. Following leads, checking several sources, noticing gaps and filling them. The loop is a real advantage over one-shot retrieval.

Software tasks with a checkable result. Writing code, running it, reading the error, fixing it. The test suite provides ground truth, which is what makes the loop converge.

Data transformation with verification. Processing files where the output can be checked.

Repetitive structured work across many similar items.

The pattern: agents work where there is a cheap, reliable signal of whether a step succeeded. Compilation errors, test results, HTTP status codes. Where that signal is absent, the loop wanders.

Where they fail

Compounding errors. A step that is 95 percent reliable is 60 percent reliable after ten steps. Long chains without checkpoints fail more often than the per-step accuracy suggests.

No sense of being stuck. An agent will retry a failing approach with cosmetic variations for a long time. Iteration limits exist because of this.

Confident wrong completion. The most expensive failure: reporting success having done something different from what was asked. A chat interface shows you the answer; an agent may have already acted.

Irreversible actions. Sending messages, making purchases, deleting files, committing code, changing settings. A wrong answer in a chat costs nothing; a wrong action does.

Prompt injection. This is the serious one and it is structural. If an agent reads a web page, a document or an email, text in that content can contain instructions. The model has no reliable way to distinguish content it is reading from instructions it should follow. An attacker who controls something the agent will read can attempt to direct it.

Where the agent also has the ability to act — send email, access files, spend money — that becomes a genuine security problem rather than a curiosity. There is no complete defence at present.

What this implies for use

Scope permissions narrowly. An agent that can read your files does not also need to send email. The damage is bounded by what it is allowed to do.

Require approval for irreversible actions. Sending, purchasing, deleting, publishing.

Prefer sandboxes. A separate account, a container, a working copy. Not your primary environment.

Set iteration limits and cost caps. Runaway loops are expensive.

Treat content it reads as untrusted. Any agent processing external material is processing potential instructions.

Check the work, not the report. An agent's summary of what it did is generated text like everything else.

The honest state of it

Agentic coding is the clearest success, because tests provide the feedback signal.

Research agents are useful and require verification of every claim.

Agents operating browsers and computers work impressively in demonstrations and remain unreliable over long sequences on unfamiliar interfaces.

Agents with autonomous authority over money, communications or production systems are not something to grant lightly, and the prompt injection problem is the reason.

Judging a product

What tools does it have, and can you restrict them?

What does it do without asking?

Can you see the actions it took, not just its summary?

What happens when it fails — does it stop, or keep going?

How is untrusted content handled? A vendor with no answer has not thought about it.

What are the cost and iteration limits?

The gap between demonstration and reliability is wider here than anywhere else in this field. Judge on failure handling, not on the demo.

A safe first configuration

For anyone trying agentic tools for the first time, a setup that limits the damage while you learn how they fail.

A separate account with access to nothing important. Not your main email, not your production systems.

Read-only where possible. Many useful agent tasks are research and analysis, which need no write access at all.

A sandbox for file operations. A dedicated directory, a container, a working copy of a repository — never your live files.

Confirmation on anything irreversible. Sending, publishing, purchasing, deleting.

A spending cap and an iteration limit, set before the first run.

Full action logging, reviewed after each run until you trust the behaviour.

Start with a task you could verify by hand. Run it, then check the work properly. Two or three of these teach you more about the tool's reliability than any amount of documentation.

Expand permissions only after you have seen how it behaves when it fails, which is the thing that matters and the thing demonstrations never show.