Prompt injection

Prompt injection is the vulnerability in which input alters a language model's behaviour or output in ways the system's designer did not intend. It sits at the top of the OWASP Top 10 for Large Language Model Applications as LLM01, which is a fair reflection of both how common it is and how hard it is to close.

The definition contains a detail that catches people out: the input does not have to be visible. OWASP notes that these inputs can affect the model even if they are imperceptible to humans, so injections do not need to be human-visible or readable, as long as the content is parsed by the model. White text, a character in an unusual encoding and an instruction inside an image are all in scope.

Direct and indirect

Direct injection is when a user's own prompt changes the model's behaviour unexpectedly. It may be deliberate — someone crafting a prompt to exploit the system — or accidental, where an ordinary input trips something.

Indirect injection is when the model consumes content from an external source such as a website or a file, and instructions inside that content are interpreted as instructions. This is the variant that matters for anyone publishing on the web, because it means the boundary between your page and someone's prompt is not enforced by anything.

OWASP's own scenarios make the shape concrete. In one, a user asks a model to summarise a web page containing hidden instructions, which cause it to insert an image linking to a URL, exfiltrating the private conversation. In another, an attacker modifies a document in a repository used by a retrieval-augmented generation application, so that when a user's query returns the modified content the malicious instructions alter the output. In a third — labelled unintentional — a company puts an instruction in a job description to identify machine-written applications, and an applicant who uses a model to polish their resume triggers it without knowing the instruction existed.

Why it cannot simply be patched

The reason there is no clean fix is architectural. A language model receives one stream of tokens. A system prompt, a retrieved document and a user message are separated by convention, not by a hardware boundary, and the model's job is to follow instructions in text. OWASP puts it plainly: given the stochastic influence at the heart of the way models work, it is unclear whether fool-proof methods of prevention exist.

It also closes off the two fixes people reach for first. Retrieval and fine-tuning aim to make outputs more relevant and accurate, but OWASP reports that they do not fully mitigate the vulnerability. Multimodal systems widen it rather than narrow it: OWASP notes that malicious actors could exploit interactions between modalities, such as hiding instructions in images that accompany benign text, and that the complexity of these systems expands the attack surface.

What actually reduces the damage

The OWASP mitigations are worth reading as a hierarchy, because they mostly limit blast radius rather than prevent the injection:

1. Constrain model behaviour — give specific instructions about role, capabilities and limits in the system prompt, enforce context adherence, and instruct the model to ignore attempts to modify core instructions.
2. Define and validate output formats — specify formats, request reasoning and source citations, and use deterministic code to check adherence.
3. Filter input and output — define sensitive categories, apply semantic filters and string checks.
4. Enforce least privilege — give the application its own API tokens, handle privileged functions in code rather than exposing them to the model, and restrict access to the minimum needed.
5. Require human approval for high-risk actions — human-in-the-loop controls on privileged operations.
6. Segregate and identify external content — separate and clearly denote untrusted content so it has less influence.
7. Test adversarially — regular penetration testing that treats the model as an untrusted user.

Points 4 and 5 carry most of the weight. An agent that cannot spend money, send mail or write to a repository without a person approving the action is one where a successful injection is an embarrassment rather than an incident.

The publisher's side of it

There is a version of this that is a content problem rather than a security problem. If instructions embedded in a page can steer a model, then some people will embed instructions in pages, and the pages doing it are competing for the same answers as yours.

Two things follow. Treat any suggestion to hide instructions in your own markup as an attack you are committing rather than a tactic — it is the behaviour every mitigation above is designed to detect, and being caught by an output filter is the good outcome. And when you check what assistants say about your brand, remember that a wrong claim may have arrived through a retrieved page rather than through anything you published; that is a different failure from a hallucination, and it is fixed at a different place.