What is PerplexityBot and should you block it?
PerplexityBot is the crawler that builds the index behind Perplexity. Block it and Perplexity cannot cite your pages.
What PerplexityBot does
Perplexity says PerplexityBot exists to surface and link websites in Perplexity search results, and that it is not used to collect content for training AI foundation models.
On robots.txt: PerplexityBot respects robots.txt, according to Perplexity's crawler documentation.
Perplexity is the most-blocked AI search crawler in the census, ahead of OAI-SearchBot and Claude-SearchBot, which is consistent with how often it appears in published block lists.
What blocking it costs
Block PerplexityBot and Perplexity cannot index your pages, so it will not quote them, will not link to them, and will not send you the visit. The cost never shows up in analytics, because you never see the visits that did not happen. If your site sells anything, or exists to be found, this is the crawler to leave alone.
How many of the top 5,000 sites block it
In my September 2026 census of the Tranco top 5,000, 2,778 sites returned a robots.txt. 382 of them block PerplexityBot (13.8%), which makes it the 12th most-blocked of the 20 crawlers I checked. 431 files mention it by name; the rest of the mentions either allow it explicitly or restrict only part of the site.
For comparison, 393 sites block at least one of the three AI search crawlers, and 305 of those still allow Googlebot: they are visible in Google and invisible in AI answers, usually by accident. The full census, with every crawler, is here.
The robots.txt lines
To make sure it is allowed even when your catch-all disallows everything, give it its own group. A crawler follows the most specific group that names it, so this wins over the catch-all:
User-agent: PerplexityBot Allow: / User-agent: * Disallow: /
To block it (you probably do not want this), the two lines are:
User-agent: PerplexityBot Disallow: /
How to verify a request really came from PerplexityBot
Anyone can put the string PerplexityBot in a request. Perplexity publishes the addresses it uses at www.perplexity.com/perplexitybot.json. A request from outside those ranges is an impostor, and impostors using well-known crawler names are common enough that blocking by name alone catches the honest crawler and misses the rest.
Source: Perplexity's crawler documentation. Census: the September 2026 robots.txt census, raw data at /data/census.json.
Other Perplexity agents: Perplexity-User. All 20 crawlers. Check your own robots.txt against all of them with the checker.