What Is OAI-SearchBot?
OpenAI runs several agents, and most robots.txt confusion comes from treating them as one. OAI-SearchBot is the one that decides whether ChatGPT search can show your site.
Quick Answer
OAI-SearchBot is OpenAI's crawler for ChatGPT search. OpenAI documents that it “is used to surface websites in search results in ChatGPT's search features,” and that “sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers,” though they may still appear as navigational links. It is controlled through robots.txt, changes take about 24 hours to apply, and its IP ranges are published at openai.com/searchbot.json.
OAI-SearchBot, GPTBot and ChatGPT-User are three different decisions
OpenAI's crawler documentation describes three agents that matter to a business website. OAI-SearchBot surfaces sites in ChatGPT search. GPTBot “is used to crawl content that may be used in training our generative AI foundation models.” ChatGPT-User is what visits a page “when users ask ChatGPT or a CustomGPT a question”; it “is not used for crawling the web in an automatic fashion,” robots.txt “may not apply” to it because a person initiated the request, and it “is not used to determine whether content may appear in Search.”
OpenAI states that each setting is “independent of the others.” Blocking GPTBot is a training decision and, on the published documentation, does not remove you from ChatGPT search. Blocking OAI-SearchBot does. The common mistake is a 2023-era rule that disallowed everything OpenAI, which removed the site from ChatGPT answers while the owner believed they had only opted out of training.
Allowing GPTBot does nothing for visibility
How to allow it, and how to check that it actually gets through
In robots.txt, give OAI-SearchBot its own group with Allow: / (or simply no Disallow), and make the GPTBot decision separately. OpenAI says robots.txt changes take “~24 hours” to propagate, so test after that, not before. Our robots.txt generator writes the groups separately.
A robots rule is not the whole story. A CDN, firewall or bot-management setting can return a 403 or a challenge page to OAI-SearchBot even when robots.txt allows it. Request your homepage with the OAI-SearchBot user-agent string and compare the status to a browser request, or use the AI crawler checker, which does both and reads robots.txt per agent. Verify against the published IP list rather than trusting the user-agent string alone.
What OpenAI does not document is how ChatGPT search ranks or chooses among the pages it can reach. Access is the documented part; everything after it is inference. How ChatGPT search picks sources keeps the two apart.
Frequently Asked Questions
Check whether OAI-SearchBot can reach your site
The free audit fetches your pages as each documented OpenAI, Anthropic and Perplexity agent and tells you exactly what blocked them.
Get a Free AI Visibility Audit