What is OAI-SearchBot and should you block it?
OAI-SearchBot is the crawler that builds the index behind ChatGPT search. Block it and ChatGPT cannot cite your pages.
What OAI-SearchBot does
OpenAI says OAI-SearchBot is "used to surface websites in search results in ChatGPT's search features" and that content it collects is not used to train models. It is the crawler that decides whether ChatGPT can cite you.
On robots.txt: OAI-SearchBot respects robots.txt. OpenAI's own wording: "Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers."
ChatGPT search also draws on third-party search providers, which is generally understood to mean Bing, so a page that is in Bing's index has a second way in. The two are additive, and the one you control directly is this one.
What blocking it costs
Block OAI-SearchBot and ChatGPT search cannot index your pages, so it will not quote them, will not link to them, and will not send you the visit. The cost never shows up in analytics, because you never see the visits that did not happen. If your site sells anything, or exists to be found, this is the crawler to leave alone.
If what you actually want is to keep your text out of model training, block GPTBot instead. That is a separate crawler with a separate robots.txt group, and blocking it has no effect on ChatGPT search.
How many of the top 5,000 sites block it
In my September 2026 census of the Tranco top 5,000, 2,778 sites returned a robots.txt. 262 of them block OAI-SearchBot (9.4%), which makes it the 18th most-blocked of the 20 crawlers I checked. 339 files mention it by name; the rest of the mentions either allow it explicitly or restrict only part of the site.
The detail that matters: 258 of the 262 sites that block OAI-SearchBot also block GPTBot. Only 4 block OAI-SearchBot on its own. Almost nobody decides to stay out of ChatGPT search; they decide to stay out of training, or they write a catch-all that disallows everything, and this crawler gets caught in it. Blocking GPTBot correctly, with an explicit group for OAI-SearchBot, is what the other 302 GPTBot blockers did.
The robots.txt lines
To make sure it is allowed even when your catch-all disallows everything, give it its own group. A crawler follows the most specific group that names it, so this wins over the catch-all:
User-agent: OAI-SearchBot Allow: / User-agent: * Disallow: /
To block it (you probably do not want this), the two lines are:
User-agent: OAI-SearchBot Disallow: /
How to verify a request really came from OAI-SearchBot
Anyone can put the string OAI-SearchBot in a request. OpenAI publishes the addresses it uses at openai.com/searchbot.json. A request from outside those ranges is an impostor, and impostors using well-known crawler names are common enough that blocking by name alone catches the honest crawler and misses the rest.
Source: OpenAI's crawler documentation. Census: the September 2026 robots.txt census, raw data at /data/census.json.
Deciding whether to block GPTBot as well? That guide covers what blocking it does and does not cost.
Other OpenAI agents: ChatGPT-User, GPTBot. All 20 crawlers. Check your own robots.txt against all of them with the checker.