AI search performance
The practice of measuring how content fares inside AI assistants' answers — what can actually be observed, what has to be sampled, and what cannot be attributed at all.
Karl-Gustav Kallasmaa, Founder & CEOLast updated AI search performance is the practice of measuring how a site's content fares inside the answers AI assistants generate — whether the brand is named, whether its URLs are cited, and where those citations sit. It differs from search reporting in one structural way: there is no impression log to read, so almost all of it has to be sampled.
Three things are observable, and they are not the same thing
Answer content. What a model says when asked a question in your category. Observable only by asking and reading the reply, which makes it a survey.
Citations. Which URLs a model attaches to an answer, and in what order. Observable in the same way, when the surface returns them at all. See source citation.
Crawler access. Which of your pages the named AI agents fetched, and when. This is the only census: every fetch lands in your server logs. See crawling and indexing.
Everything downstream of these — a visit, a signup, revenue — is not observable in this class of measurement at all.
Why there is no impression log
Search reporting works because a search engine records an impression against a query and hands you the aggregate. No assistant publishes an equivalent. Google's AI features guidance is explicit that traffic from AI features is folded into overall search data in Search Console under the Web search type rather than broken out, and that there are no additional requirements to appear in AI Overviews or AI Mode and no special optimisations necessary. So the platform tells you neither which of your clicks came from an AI feature nor how often you appeared without one.
That leaves sampling: put a fixed set of prompts to a fixed set of models on a schedule, store the verbatim answers, and compute rates from them. The verbatim answer matters more than the rate, because it is the evidence — you can read what was actually said, and re-run detection over the same history when the method improves.
The three failure modes of the measurement itself
Non-determinism. The same prompt to the same model can give different answers minutes apart. A rate computed from one draw per prompt per day is a thin sample, and most week-to-week movement in it sits under the noise floor. The defence is depth plus a significance test on deltas, so a movement that cannot be proven is reported as no proven change rather than as a change.
Naive detection. A brand mention is usually found by matching the brand name against the answer text. Attensira's limitations document that its match is case-insensitive substring with no entity resolution and no word-boundary check — so a brand whose name is a common noun carries systematic error that averaging cannot remove. Any tool doing this has the same problem; the difference is whether it says so.
Null read as zero. A model that was never queried, and a model that was queried and did not name you, are different facts. Collapsing them into one chart line invents a rejection that never happened. The same applies to a surface with no reader at all: unmeasured is not zero.
Access is upstream of all of it
Before content quality is worth discussing, the agents have to be able to fetch the page — and they are named separately per job. OpenAI's bots documentation names OAI-SearchBot for surfacing sites in ChatGPT search, ChatGPT-User for user-initiated fetches, GPTBot for training data, and OAI-AdsBot for ad safety checks, and asks that OAI-SearchBot be allowed for sites that want to appear in ChatGPT search answers. Anthropic's crawler documentation names ClaudeBot, Claude-User and Claude-SearchBot with the same split between training and answering.
The practical consequence: a rule written against a wildcard, or a WAF rule matching a vendor's name, opts you out of answers while you believe you only opted out of training. This is the single most common cause of a brand appearing nowhere, and it produces no error anywhere. See robots.txt.
What a defensible report looks like
- Every rate carries its denominator. A rate without an n is a claim, not a measurement.
- Deltas are gated. Movement that does not clear a significance test is reported as unproven, not drawn as a line.
- Unmeasured is drawn as unmeasured. Never as zero.
- Verbatim answers are retained. They are the audit trail, and they let detection be re-run over history.
- Attribution is not claimed. Presence in answers is reported as presence. Connecting it to revenue is the reader's inference, made with their own analytics, and it should be labelled as such.
For the assistants and agents this is measured across, see AI search.
Frequently asked questions
Is there a Search Console report for AI features?
No. Google includes that traffic in overall search data under the Web search type.
Why must it be sampled?
Because no platform publishes an impression log for answers; the only observation is asking and reading the reply.
Can an answer be attributed to a signup?
No. There is no identity join between a model's answer and a later visit.
What breaks first when a brand appears nowhere?
Access. Search and user-directed agents are named separately from training crawlers, and a blanket block removes retrieval silently.
Terms related to AI search performance
Search where a model composes the answer and fetches pages through named crawlers, rather than returning a ranked list of links for you to read.
Google's AI-generated summary at the top of a results page, and the snippet controls that decide whether your page can appear inside one.
How an AI answer attributes what it says to the pages it read, and why a citation is a distinct outcome from a click or a mention.
The two separate stages that decide whether a page can be retrieved at all, and the reason a serving rule on a blocked page is never read.
The standardised file that tells crawlers what they may fetch, now the main place where AI training and AI retrieval access are decided separately.