What OpenAI documents
DOCUMENTED OpenAI’s crawler documentation is the first-party source for how ChatGPT reaches the web, and it is precise about three agents.
| Agent | OpenAI’s stated purpose | Effect of blocking it |
|---|---|---|
| OAI-SearchBot | “Used to surface websites in search results in ChatGPT’s search features.” | “Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links.” |
| GPTBot | “Used to crawl content that may be used in training our generative AI foundation models.” | “Indicates a site’s content should not be used in training generative AI foundation models.” |
| ChatGPT-User | “When users ask ChatGPT or a CustomGPT a question, it may visit a web page with a ChatGPT-User agent.” | “Because these actions are initiated by a user, robots.txt rules may not apply.” Not used for automatic crawling. |
DOCUMENTED The settings are independent: a site “can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot”. If both are allowed, OpenAI “may use the results from just one crawl for both use cases”. It recommends allowing OAI-SearchBot and its published IP ranges, and says search systems take “~24 hours” to reflect a robots.txt change.
That is the whole of what ChatGPT SEO can state as fact from the platform. The rest is inference from the mechanism, or observation.
Two paths, one of them influenceable
INFERRED ChatGPT can answer from what its model learned in training or from pages it retrieves when it searches. The documentation implies the split by giving each path its own agent. Trained knowledge is fixed until a model is retrained; nothing a site publishes today changes it. Retrieval happens at answer time and reads what is on the web now. ChatGPT SEO is therefore retrieval-side work: being in the search index OAI-SearchBot builds, and being readable when ChatGPT-User fetches a page.
A common Perth-business experience follows from that split. A business updates its prices or address, and ChatGPT keeps giving the old ones for weeks. The likely causes, in order of how often they apply, are that the answer came from trained knowledge rather than retrieval, that retrieval found third-party pages still carrying the old facts, or that the business’s own page was not the one retrieved. Only the second and third can be fixed by the business, and both are entity consistency problems as much as ChatGPT problems.
What is not documented
The crawler documentation says nothing about how ChatGPT ranks the pages it retrieves, how many it reads per question, how it decides which to cite, or which underlying web index its search features draw on. This publication does not fill those gaps with a named provider or a factor list. Where it discusses ChatGPT’s source choices, it does so from its own repeated-run observations and labels them OBSERVED, or as hypotheses to test.
HYPOTHESIS Two hypotheses this publication holds and intends to test rather than assert: that ChatGPT answers Perth recommendation prompts with businesses that are consistently described across several sources more often than with businesses described inconsistently; and that directories and review platforms are cited for Perth recommendation prompts at a higher rate than business sites. Both are counts the ChatGPT Perth study is designed to produce.
What does not work, as far as the documentation goes
Nothing in OpenAI’s documentation describes a mechanism by which text written to the model on a page, hidden instructions, keyword density or self-declared markup changes which sources ChatGPT uses. The system reads pages to answer a question; a page that answers the question plainly is usable, and one that addresses the model is not. This publication treats prompt-injection and “LLM manipulation” tactics as unsupported and does not test them.
What a Perth business can check today
- Access. Confirm that OAI-SearchBot and ChatGPT-User are not blocked in robots.txt, by a firewall or by bot-management rules. The crawler access guide walks through checking each agent and reading the raw HTML the agent receives.
- Identity. Ask ChatGPT, in a fresh session, what the business does, where it is and who runs it. A hedged or confused answer is an entity problem. Run it more than once; single answers vary.
- Retrieval. Ask a Perth question the business should be an answer to, with search on, and note which pages are cited. If they are third-party pages carrying stale facts, fix those sources, not only the site.
- Referrals. Set up the GA4 view in the AI referral guide so that any traffic from chatgpt.com is visible before the work starts.
What is documented and what is inferred
- Documented: the three agents and their purposes; the effect of opting out of OAI-SearchBot; the independence of settings; the ~24-hour propagation.
- Inferred: the training-versus-retrieval split as it affects a site owner; the causes of stale answers.
- Hypothesis: consistency and source-class patterns for Perth prompts, pending observation.
- Undisclosed: ranking among retrieved pages, citation selection, and the underlying web index.