# AI Search Optimisation: How AI Search Retrieves and Cites > How AI-assisted search retrieves documents and attaches citations, what each platform documents, what is undisclosed, and what it means for Perth businesses. Source: https://aiseoperth.net.au/ai-search-optimisation/ --- Reference # AI search optimisation: how AI-assisted search retrieves and cites sources Author Dorian Menard, founder of Search Scope Published 9 September 2026 Reading time 8 min AI search optimisation is the work of making a source usable by AI-assisted search: systems that retrieve documents at answer time, compose a response with a language model, and attach citations to some of what they used. It differs from optimising for a ranked list because the unit is the retrieved passage and the outcome is being composed from and cited, not a position. What the platforms document is access and eligibility; how they select among eligible sources is largely undisclosed. ## Key facts Key facts about AI search optimisation Fact Detail Evidence Two knowledge paths A model can answer from what it learned in training or from documents retrieved at answer time. Only retrieval can be influenced by a site owner today; platforms document separate crawlers for each. DOCUMENTED Query fan-out Google documents that AI Overviews and AI Mode may issue multiple related searches across subtopics and data sources to develop a response. DOCUMENTED Citations Perplexity documents that its user-triggered fetcher includes a link to the page in its response; Google documents supporting links in AI Overviews and AI Mode. Citation display and completeness differ by platform. DOCUMENTED Retrieval providers Which web index each assistant searches is not fully documented by every platform. This site does not assert an undisclosed provider. DOCUMENTED Variance The same prompt returns different sources across runs, sessions and accounts. Measured, not assumed, in this publication's methodology. HYPOTHESIS ## What AI-assisted search is AI-assisted search is any search experience where a generative model composes the answer. It includes AI Overviews and AI Mode inside Google Search, chat assistants such as ChatGPT, Gemini, Claude and Copilot when they search the web, and citation-first engines such as Perplexity. What they share is a pipeline with three visible steps: retrieve documents, compose an answer, attribute some of the sources used. What differs is how much of each step the platform documents. ## Training knowledge versus retrieval DOCUMENTED Every major platform separates the crawler that gathers training data from the agent that fetches pages to answer a question, and lets site owners control them separately. OpenAI documents GPTBot for training and OAI-SearchBot for its search features, and states that each robots.txt setting “is independent of the others”. Anthropic documents ClaudeBot for training, Claude-SearchBot for search indexing and Claude-User for user-initiated fetches. Perplexity documents PerplexityBot for its search index and Perplexity-User for user actions, and says neither is used to collect content for training foundation models. Google’s Google-Extended token controls training and grounding for Gemini apps and “does not impact a site’s inclusion in Google Search”. The consequence for optimisation is that what a model learned in training cannot be edited, and what it retrieves can be. AI search optimisation is retrieval-side work. ## How retrieval happens DOCUMENTED Google is the platform that documents the most. Its site-owner guidance says AI Overviews and AI Mode “may use a query fan-out technique, issuing multiple related searches across subtopics and data sources, to develop a response”, and that while responses are generated its models “identify more supporting web pages”, showing “a wider and more diverse set of helpful links” than a classic search. Google’s AI Mode announcement describes the same technique as “breaking down your question into subtopics and issuing a multitude of queries simultaneously”. Eligibility is the normal Search index: a page must be indexed and eligible for a snippet. DOCUMENTED Gemini’s privacy documentation states that “some Gemini responses are grounded on Search results”, which places the Gemini app on the same retrieval substrate for grounded answers. DOCUMENTED OpenAI documents that OAI-SearchBot “is used to surface websites in search results in ChatGPT’s search features” and that ChatGPT-User “may visit a web page” when a user asks a question, but is “not used for crawling the web in an automatic fashion”. Perplexity documents the equivalent pair. Neither publishes how candidate pages are ranked once retrieved. What is not documented, by any platform, is the selection step: given several eligible, retrieved pages, which ones the answer draws on and why. Every ranked list of “AI ranking factors” in circulation is inference or observation dressed as documentation. ## How citations work DOCUMENTED Perplexity documents that its user-triggered fetcher “might visit a web page to help provide an accurate answer and include a link to the page in its response”. Google documents that AI Overviews and AI Mode show links to supporting websites and describes AI Mode as returning “prominent links for people to click on”. OpenAI’s documentation frames OAI-SearchBot opt-out as removal from “ChatGPT search answers” while sites “can still appear as navigational links”. INFERRED Two inferences follow and are labelled as such. First, a source can inform an answer without being cited, because no platform claims every retrieved document is attributed; counting visible citations therefore undercounts influence. Second, citation display varies by surface, so absence of a citation in one run is not evidence that the source was unused. This publication’s [methodology](https://aiseoperth.net.au/methodology/) records citations as shown and treats absence as a data point, not a conclusion. ## Why the same prompt gets different answers HYPOTHESIS Repeated runs of the same prompt return different sources and different brands. The likely contributors are the model’s sampling, the searches it constructs from the prompt, personalisation, inferred location, logged-in state and ongoing model changes. Platforms document some of these individually (Google’s fan-out, Gemini’s use of account data) but none publishes the combined effect. This site therefore treats variance as something to measure: at least three runs per prompt per platform, from controlled session profiles, with divergence reported. ## What AI search optimisation is, given all that Take the documented parts and the work follows: 1. **Be retrievable.** Allow the search and user agents named above through robots.txt and any firewall or CDN rule, serve content as HTML, and stay indexed on Google. 2. **Be the answer to the sub-questions.** Fan-out means a page can be retrieved for a sub-query the person never typed. Coverage of the comparison, cost, suitability and local questions around a subject is what gives an engine something to retrieve. 3. **Be identifiable.** An answer that names a business has to resolve it to one entity across the sources it retrieved. [Entity SEO](https://aiseoperth.net.au/entity-seo/) is that work. 4. **Be corroborated.** Retrieval returns several sources; the ones that agree get composed from. Independent sources stating the same facts matter because the engine reads them too. 5. **Measure with repetition.** One run is an anecdote. ## How it applies in Perth HYPOTHESIS A Perth prompt is a local prompt, and fan-out on a local prompt plausibly retrieves directories, review platforms, Google Business Profiles and national publishers alongside business sites. Which of those the engines compose Perth answers from is exactly what the [most cited domains study](https://aiseoperth.net.au/research/) is designed to count. Until it has data, the Perth-specific claim this page makes is the documented one: both Google surfaces are live in Australia, and the retrieval agents above are the ones a Perth site has to let in. ## What is documented and what is inferred - **Documented:** the training and retrieval split per platform; Google’s fan-out and eligibility rule; Gemini grounding on Search; Perplexity’s citation behaviour; OpenAI’s opt-out consequence. - **Inferred:** uncited influence; citation display varying by surface. - **Hypothesis:** the contributors to run-to-run variance; the Perth source mix. - **Undisclosed:** each platform’s selection logic among retrieved pages, and any web index provider a platform has not named. ## Related concepts - [ Reference What is AI SEO? The practice built on this mechanism. ](https://aiseoperth.net.au/what-is-ai-seo/) - [ Reference Google AI Overviews Google's generative answer surface. ](https://aiseoperth.net.au/google-ai-overviews/) - [ Reference ChatGPT SEO How ChatGPT reaches and shows sources. ](https://aiseoperth.net.au/chatgpt-seo/) - [ Reference Perplexity SEO A citation-first answer engine. ](https://aiseoperth.net.au/perplexity-seo/) ## Related research - [ Planned study Most Cited Domains in Perth AI Search Which domains the systems cite for Perth prompts. ](https://aiseoperth.net.au/research/) ## Sources 1. [AI features and your website](https://developers.google.com/search/docs/appearance/ai-features), Google Search Central (read 9 September 2026) DOCUMENTED 2. [AI Mode in Search gets new agentic features and expands globally](https://blog.google/products/search/google-search-ai-mode-update/), Google Blog, 20 May 2025 (read 9 September 2026) DOCUMENTED 3. [Overview of OpenAI crawlers](https://platform.openai.com/docs/bots), OpenAI (read 9 September 2026) DOCUMENTED 4. [Perplexity crawlers](https://docs.perplexity.ai/guides/bots), Perplexity (read 9 September 2026) DOCUMENTED 5. [Does Anthropic crawl data from the web, and how can site owners block the crawler?](https://support.anthropic.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler), Anthropic (read 9 September 2026) DOCUMENTED 6. [Google crawlers and fetchers (Google-Extended)](https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers), Google Search Central (read 9 September 2026) DOCUMENTED 7. [Gemini Apps Privacy Hub](https://support.google.com/gemini/answer/13594961), Google (read 9 September 2026) DOCUMENTED States that some Gemini responses are grounded on Search results. On this page - [Definition](#definition) - [What AI-assisted search is](#what-ai-assisted-search-is) - [Training knowledge versus retrieval](#training-knowledge-versus-retrieval) - [How retrieval happens](#how-retrieval-happens) - [How citations work](#how-citations-work) - [Why the same prompt gets different answers](#why-the-same-prompt-gets-different-answers) - [What AI search optimisation is, given all that](#what-ai-search-optimisation-is-given-all-that) - [How it applies in Perth](#how-it-applies-in-perth) - [What is documented and what is inferred](#what-is-documented-and-what-is-inferred) - [Sources](#sources) Evidence labels Claims on this page are labelled DOCUMENTED, OBSERVED, TESTED, INFERRED or HYPOTHESIS. [What the labels mean](https://aiseoperth.net.au/methodology/#evidence-labels). Published by Search Scope AI SEO Perth is published by Search Scope, a Perth SEO consultancy founded by Dorian Menard. Search Scope provides commercial SEO and AI search optimisation services; this website is maintained as its specialist Perth AI search research and reference publication. [Search Scope](https://searchscope.com.au/)