What is OAI-SearchBot and should you block it?

OAI-SearchBot is the crawler that builds the index behind ChatGPT search. Block it and ChatGPT cannot cite your pages.

What OAI-SearchBot does

OpenAI says OAI-SearchBot is "used to surface websites in search results in ChatGPT's search features" and that content it collects is not used to train models. It is the crawler that decides whether ChatGPT can cite you.

On robots.txt: OAI-SearchBot respects robots.txt. OpenAI's own wording: "Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers."

ChatGPT search also draws on third-party search providers, which is generally understood to mean Bing, so a page that is in Bing's index has a second way in. The two are additive, and the one you control directly is this one.

What blocking it costs

Block OAI-SearchBot and ChatGPT search cannot index your pages, so it will not quote them, will not link to them, and will not send you the visit. The cost never shows up in analytics, because you never see the visits that did not happen. If your site sells anything, or exists to be found, this is the crawler to leave alone.

If what you actually want is to keep your text out of model training, block GPTBot instead. That is a separate crawler with a separate robots.txt group, and blocking it has no effect on ChatGPT search.

How many of the top 5,000 sites block it

In my September 2026 census of the Tranco top 5,000, 2,771 sites returned a robots.txt. 241 of them block OAI-SearchBot (8.7%), which makes it the 18th most-blocked of the 20 crawlers I checked. 334 files mention it by name; the rest of the mentions either allow it explicitly or restrict only part of the site.

The detail that matters: 238 of the 241 sites that block OAI-SearchBot also block GPTBot. Only 3 block OAI-SearchBot on its own. Almost nobody decides to stay out of ChatGPT search; they decide to stay out of training, or they write a catch-all that disallows everything, and this crawler gets caught in it. Blocking GPTBot correctly, with an explicit group for OAI-SearchBot, is what the other 297 GPTBot blockers did.

The robots.txt lines

To make sure it is allowed even when your catch-all disallows everything, give it its own group. A crawler follows the most specific group that names it, so this wins over the catch-all:

User-agent: OAI-SearchBot
Allow: /

User-agent: *
Disallow: /

To block it (you probably do not want this), the two lines are:

User-agent: OAI-SearchBot
Disallow: /

The OAI-SearchBot user-agent string

OpenAI's documentation gives this string (read 2026-09-26). OpenAI gives it as an example, says the version number may change, and says that when it fetches robots.txt it may add a robots.txt marker to the string. robots.txt matches on the token OAI-SearchBot, not on the whole string, and anyone can send any string, so the address check below is what shows a request is real.

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot

How to verify a request really came from OAI-SearchBot

Anyone can put the string OAI-SearchBot in a request. OpenAI publishes the addresses it uses at openai.com/searchbot.json. A request from outside those ranges is an impostor, and impostors using well-known crawler names are common enough that blocking by name alone catches the honest crawler and misses the rest.

Source: OpenAI's crawler documentation. Census: the September 2026 robots.txt census, raw data at /data/census.json.

Other OpenAI agents: ChatGPT-User, GPTBot. All 20 crawlers. Check your own robots.txt against all of them with the checker.