Skip to main content
USA-Based Digital Agency
AI Search Glossary

What Is OAI-SearchBot?

OpenAI runs several agents, and most robots.txt confusion comes from treating them as one. OAI-SearchBot is the one that decides whether ChatGPT search can show your site.

Quick Answer

OAI-SearchBot is OpenAI's crawler for ChatGPT search. OpenAI documents that it “is used to surface websites in search results in ChatGPT's search features,” and that “sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers,” though they may still appear as navigational links. It is controlled through robots.txt, changes take about 24 hours to apply, and its IP ranges are published at openai.com/searchbot.json.

OAI-SearchBot, GPTBot and ChatGPT-User are three different decisions

OpenAI's crawler documentation describes three agents that matter to a business website. OAI-SearchBot surfaces sites in ChatGPT search. GPTBot “is used to crawl content that may be used in training our generative AI foundation models.” ChatGPT-User is what visits a page “when users ask ChatGPT or a CustomGPT a question”; it “is not used for crawling the web in an automatic fashion,” robots.txt “may not apply” to it because a person initiated the request, and it “is not used to determine whether content may appear in Search.”

OpenAI states that each setting is “independent of the others.” Blocking GPTBot is a training decision and, on the published documentation, does not remove you from ChatGPT search. Blocking OAI-SearchBot does. The common mistake is a 2023-era rule that disallowed everything OpenAI, which removed the site from ChatGPT answers while the owner believed they had only opted out of training.

Allowing GPTBot does nothing for visibility

If OAI-SearchBot is blocked, allowing GPTBot will not put you in ChatGPT search answers. The reverse also holds: you can block GPTBot and remain fully visible. Decide each on its own.

How to allow it, and how to check that it actually gets through

In robots.txt, give OAI-SearchBot its own group with Allow: / (or simply no Disallow), and make the GPTBot decision separately. OpenAI says robots.txt changes take “~24 hours” to propagate, so test after that, not before. Our robots.txt generator writes the groups separately.

A robots rule is not the whole story. A CDN, firewall or bot-management setting can return a 403 or a challenge page to OAI-SearchBot even when robots.txt allows it. Request your homepage with the OAI-SearchBot user-agent string and compare the status to a browser request, or use the AI crawler checker, which does both and reads robots.txt per agent. Verify against the published IP list rather than trusting the user-agent string alone.

What OpenAI does not document is how ChatGPT search ranks or chooses among the pages it can reach. Access is the documented part; everything after it is inference. How ChatGPT search picks sources keeps the two apart.

Frequently Asked Questions

OAI-SearchBot is the OpenAI crawler used to surface websites in ChatGPT’s search features. OpenAI documents that sites opted out of it will not be shown in ChatGPT search answers.
No. GPTBot crawls content that may be used for model training. OAI-SearchBot governs ChatGPT search inclusion. OpenAI describes the settings as independent of each other.
Yes. OpenAI documents that it uses OAI-SearchBot and GPTBot robots.txt tags and that changes take about 24 hours to take effect. ChatGPT-User, the user-initiated fetch agent, may not be subject to robots.txt because a person requested the page.
Compare the source address against the IP ranges OpenAI publishes at openai.com/searchbot.json (and the equivalent lists for GPTBot and ChatGPT-User). A user-agent string alone can be spoofed.

Check whether OAI-SearchBot can reach your site

The free audit fetches your pages as each documented OpenAI, Anthropic and Perplexity agent and tells you exactly what blocked them.

Get a Free AI Visibility Audit