Which sites in the top 5,000 block GPTBot
534 of the 2,771 sites with a robots.txt block GPTBot, OpenAI's training crawler - about 19%. 297 of them still allow ChatGPT citations. The full named list, from a 2026-09-07 census.
GPTBot is OpenAI's training crawler. Blocking it keeps a site's content out of future model training runs - it does not remove the site from ChatGPT's answers. Citations are governed by a different crawler, OAI-SearchBot, and live user-requested fetches by a third, ChatGPT-User. The three are separate robots.txt decisions, and this page shows all three for every GPTBot blocker.
Of the 2,771 sites in the census that served a robots.txt, 534 block GPTBot. 297 of those still allow OAI-SearchBot - they opted out of training without giving up ChatGPT citations, which is the configuration most publishers actually want. 232 block GPTBot, OAI-SearchBot and ChatGPT-User alike, a full OpenAI opt-out. In the table, a red mark means that crawler is blocked too: the first mark column is OAI-SearchBot (can ChatGPT cite it?), the second is ChatGPT-User (can ChatGPT fetch it for a user?).
The reverse mistake is rare but real: 3 sites block the citation crawler while still allowing GPTBot - training on their content is fine, but ChatGPT may not cite them.
The Tranco list ranks raw hostnames, so it includes CDN and API endpoints nobody visits. 2 of the 534 rows look like infrastructure rather than sites you would browse. They are real robots.txt results so they stay in the table, but they are not evidence about a publisher.
Measured on 2026-09-07 against each site's live robots.txt, using the same matcher Google publishes, so wildcards and dollar anchors are handled the way a real crawler handles them. The AI search crawler blockers are their own list, the 32 recognizable names are on the board, GPTBot itself is profiled here, and the raw per-domain data is on the census page and in the open dataset.
Want to know if YOUR site's AI visibility changes?
Enter your domain and email. One email if a crawler you rely on gets newly blocked, or an existing block goes away. No spam.
| Rank | Site | Can ChatGPT cite it? | Can ChatGPT fetch it live? |
|---|---|---|---|
| 3 | gstatic.com | ✗ | ✗ |
| 11 | instagram.com | ✗ | ✗ |
| 13 | fbcdn.net | ✗ | ✗ |
| 16 | twitter.com | ✗ | ✗ |
| 26 | amazon.com | ✗ | ✗ |
| 32 | whatsapp.net | ✓ | ✓ |
| 50 | x.com | ✗ | ✗ |
| 54 | tiktok.com | ✗ | ✗ |
| 56 | whatsapp.com | ✓ | ✓ |
| 57 | pinterest.com | ✗ | ✗ |
| 60 | yahoo.com | ✓ | ✗ |
| 66 | msn.com | ✗ | ✗ |
| 84 | baidu.com | ✗ | ✗ |
| 105 | reddit.com | ✗ | ✗ |
| 120 | registrar-servers.com | ✗ | ✗ |
| 128 | snapchat.com | ✓ | ✓ |
| 153 | nytimes.com | ✗ | ✗ |
| 154 | gravatar.com | ✓ | ✓ |
| 181 | flickr.com | ✓ | ✓ |
| 190 | medium.com | ✓ | ✓ |
| 193 | cnn.com | ✗ | ✗ |
| 206 | forbes.com | ✓ | ✓ |
| 210 | t-online.de | ✓ | ✓ |
| 211 | bbc.com | ✗ | ✗ |
| 224 | bbc.co.uk | ✗ | ✗ |
| 231 | t.co | ✗ | ✗ |
| 235 | ebay.com | ✓ | ✓ |
| 240 | canva.com | ✓ | ✓ |
| 254 | meraki.com | ✗ | ✗ |
| 256 | imdb.com | ✗ | ✗ |
| 262 | launchpad.net | ✗ | ✓ |
| 274 | reuters.com | ✓ | ✓ |
| 287 | weibo.com | ✗ | ✗ |
| 303 | alibaba.com | ✓ | ✓ |
| 305 | weather.com | ✓ | ✓ |
| 323 | amazon.co.uk | ✗ | ✗ |
| 365 | bloomberg.com | ✗ | ✗ |
| 380 | amazon.de | ✗ | ✗ |
| 410 | globo.com | ✓ | ✓ |
| 416 | espn.com | ✓ | ✗ |
| 434 | claude.ai | ✗ | ✗ |
| 438 | goodreads.com | ✓ | ✓ |
| 452 | yahoo.co.jp | ✓ | ✓ |
| 454 | pixabay.com | ✓ | ✓ |
| 456 | sohu.com | ✗ | ✗ |
| 459 | go.com | ✓ | ✓ |
| 465 | edgecdn.ru | ✗ | ✗ |
| 470 | cnbc.com | ✗ | ✗ |
| 487 | amazon.co.za | ✗ | ✗ |
| 490 | quora.com | ✗ | ✗ |
| 491 | gcdn.co | ✗ | ✗ |
| 494 | aol.com | ✓ | ✗ |
| 497 | amazon.co.jp | ✗ | ✗ |
| 509 | behance.net | ✓ | ✓ |
| 541 | patreon.com | ✓ | ✓ |
| 555 | tandfonline.com | ✓ | ✓ |
| 564 | tripadvisor.com | ✓ | ✓ |
| 567 | usatoday.com | ✗ | ✗ |
| 575 | cookielaw.org | ✓ | ✓ |
| 597 | telegraph.co.uk | ✗ | ✗ |
| 611 | amazon.fr | ✗ | ✗ |
| 613 | threads.com | ✗ | ✗ |
| 619 | sharethrough.com | ✗ | ✗ |
| 629 | nintendo.com | ✓ | ✓ |
| 636 | amazon.ca | ✗ | ✗ |
| 643 | techcrunch.com | ✓ | ✗ |
| 651 | yelp.com | ✗ | ✗ |
| 660 | dailymail.co.uk | ✗ | ✗ |
| 668 | amazon.in | ✗ | ✗ |
| 673 | hstgr.net | ✗ | ✗ |
| 685 | hihonorcloud.com | ✗ | ✗ |
| 687 | cbsnews.com | ✓ | ✓ |
| 694 | dailymail.com | ✗ | ✗ |
| 695 | fb.com | ✗ | ✗ |
| 697 | blackhub.team | ✗ | ✗ |
| 699 | cnet.com | ✗ | ✗ |
| 700 | pinimg.com | ✗ | ✗ |
| 703 | amazon.es | ✗ | ✗ |
| 704 | kick.com | ✓ | ✓ |
| 707 | amazon.it | ✗ | ✗ |
| 709 | wp.pl | ✓ | ✓ |
| 713 | quantserve.com | ✗ | ✗ |
| 718 | globalsign.com | ✓ | ✓ |
| 724 | spiegel.de | ✗ | ✗ |
| 732 | creativecdn.com | ✗ | ✗ |
| 739 | wattpad.com | ✓ | ✓ |
| 743 | bandcamp.com | ✓ | ✓ |
| 764 | ted.com | ✓ | ✓ |
| 780 | mercadolivre.com.br | ✓ | ✗ |
| 789 | apnews.com | ✓ | ✓ |
| 790 | adform.net | ✗ | ✗ |
| 814 | academia.edu | ✓ | ✓ |
| 831 | lefigaro.fr | ✗ | ✗ |
| 845 | amazon.com.br | ✗ | ✗ |
| 846 | hbr.org | ✓ | ✓ |
| 849 | amazon.com.au | ✗ | ✗ |
| 858 | ebay.co.uk | ✓ | ✓ |
| 859 | nbcnews.com | ✗ | ✗ |
| 860 | figma.com | ✓ | ✓ |
| 862 | pexels.com | ✓ | ✓ |
| 875 | ebay.de | ✓ | ✓ |
| 900 | latimes.com | ✓ | ✓ |
| 925 | genius.com | ✓ | ✗ |
| 940 | mailchi.mp | ✗ | ✗ |
| 943 | webmd.com | ✓ | ✓ |
| 947 | mercadolibre.com.ar | ✓ | ✗ |
| 958 | repubblica.it | ✓ | ✓ |
| 969 | corriere.it | ✓ | ✓ |
| 970 | healthline.com | ✓ | ✓ |
| 979 | btloader.com | ✗ | ✗ |
| 983 | stbid.ru | ✗ | ✗ |
| 989 | pixiv.net | ✓ | ✓ |
| 992 | elmundo.es | ✗ | ✗ |
| 1002 | archiveofourown.org | ✓ | ✗ |
| 1023 | letterboxd.com | ✓ | ✓ |
| 1038 | amazon.com.mx | ✗ | ✗ |
| 1043 | pbs.org | ✓ | ✓ |
| 1045 | theconversation.com | ✗ | ✗ |
| 1057 | huffpost.com | ✗ | ✗ |
| 1058 | n-tv.de | ✓ | ✓ |
| 1061 | ouest-france.fr | ✗ | ✗ |
| 1065 | marca.com | ✗ | ✗ |
| 1075 | bfmtv.com | ✗ | ✗ |
| 1076 | flightradar24.com | ✓ | ✓ |
| 1089 | idnes.cz | ✓ | ✓ |
| 1092 | tabelog.com | ✓ | ✓ |
| 1095 | franceinfo.fr | ✗ | ✗ |
| 1101 | namecheaphosting.com | ✗ | ✗ |
| 1107 | skysports.com | ✓ | ✓ |
| 1113 | cookpad.com | ✓ | ✗ |
| 1127 | netcraze.io | ✗ | ✗ |
| 1133 | mercadolibre.com.mx | ✓ | ✗ |
| 1136 | leparisien.fr | ✓ | ✓ |
| 1151 | xiaohongshu.com | ✗ | ✗ |
| 1155 | economist.com | ✗ | ✗ |
| 1161 | pimpbunny.com | ✗ | ✗ |
| 1180 | ad.nl | ✗ | ✗ |
| 1181 | spankbang.com | ✓ | ✓ |
| 1188 | nu.nl | ✗ | ✗ |
| 1202 | rightmove.co.uk | ✓ | ✓ |
| 1203 | rocketmoney.dev | ✗ | ✗ |
| 1204 | dcinside.com | ✓ | ✓ |
| 1210 | lequipe.fr | ✗ | ✗ |
| 1217 | mercadolibre.com | ✓ | ✗ |
| 1230 | gazzetta.it | ✓ | ✓ |
| 1237 | furaffinity.net | ✗ | ✗ |
| 1241 | pandora.com | ✓ | ✓ |
| 1247 | hln.be | ✗ | ✗ |
| 1269 | nrk.no | ✗ | ✗ |
| 1274 | ladepeche.fr | ✓ | ✓ |
| 1275 | abc.net.au | ✓ | ✗ |
| 1290 | actu.fr | ✗ | ✗ |
| 1291 | vietnam.vn | ✓ | ✓ |
| 1292 | jstor.org | ✓ | ✓ |
| 1296 | f95zone.to | ✗ | ✓ |
| 1297 | justdial.com | ✓ | ✓ |
| 1300 | hugedomains.com | ✓ | ✓ |
| 1305 | yr.no | ✗ | ✗ |
| 1327 | huffingtonpost.com | ✗ | ✗ |
| 1344 | e621.net | ✓ | ✓ |
| 1352 | sport.es | ✗ | ✗ |
| 1364 | comix.to | ✓ | ✓ |
| 1367 | dw.com | ✗ | ✗ |
| 1376 | nikkansports.com | ✓ | ✗ |
| 1381 | ilmeteo.it | ✓ | ✓ |
| 1382 | dagbladet.no | ✓ | ✓ |
| 1401 | zedge.net | ✓ | ✓ |
| 1410 | iltalehti.fi | ✗ | ✗ |
| 1411 | aljazeera.com | ✓ | ✗ |
| 1416 | spankbang.party | ✓ | ✓ |
| 1424 | omg10.com | ✓ | ✓ |
| 1428 | cardmarket.com | ✓ | ✓ |
| 1441 | pussyspace.com | ✓ | ✓ |
| 1444 | ekstrabladet.dk | ✗ | ✗ |
| 1449 | crn77.com | ✓ | ✓ |
| 1459 | hdtube.porn | ✓ | ✗ |
| 1469 | bblaa.com | ✓ | ✓ |
| 1471 | dogdrip.net | ✓ | ✓ |
| 1472 | eenadu.net | ✗ | ✗ |
| 1473 | toyhou.se | ✓ | ✓ |
| 1476 | xnxx.health | ✓ | ✓ |
| 1490 | usnews.com | ✗ | ✗ |
| 1491 | myinstants.com | ✓ | ✓ |
| 1495 | ay267.com | ✓ | ✓ |
| 1498 | joyclub.de | ✓ | ✓ |
| 1499 | exhentai.org | ✗ | ✗ |
| 1507 | 255md.com | ✓ | ✓ |
| 1510 | bt.dk | ✓ | ✓ |
| 1513 | shahvani.com | ✓ | ✓ |
| 1530 | wnacg.com | ✓ | ✓ |
| 1532 | aka.ms | ✗ | ✗ |
| 1548 | buzzfeed.com | ✓ | ✓ |
| 1556 | myreadingmanga.info | ✓ | ✓ |
| 1564 | ladies.de | ✗ | ✗ |
| 1568 | nya2.com | ✓ | ✓ |
| 1571 | syosetu.org | ✓ | ✓ |
| 1573 | imgsrc.ru | ✓ | ✓ |
| 1597 | tjk.org | ✗ | ✗ |
| 1605 | colliderporn.com | ✓ | ✓ |
| 1609 | arabx.cam | ✓ | ✓ |
| 1633 | artfight.net | ✓ | ✗ |
| 1647 | sportfilm800.com | ✓ | ✓ |
| 1649 | keirin.jp | ✗ | ✗ |
| 1660 | sedoparking.com | ✗ | ✗ |
| 1664 | twkan.com | ✓ | ✓ |
| 1671 | bef77.com | ✓ | ✓ |
| 1674 | momon-ga.com | ✓ | ✓ |
| 1675 | manhwaweb.com | ✓ | ✓ |
| 1681 | animanch.com | ✓ | ✓ |
| 1692 | espn.com.ve | ✓ | ✗ |
| 1701 | hai8g.com | ✓ | ✓ |
| 1713 | pornoborshch.com | ✓ | ✓ |
| 1720 | scan-manga.com | ✓ | ✓ |
| 1723 | xxxhindi.to | ✓ | ✓ |
| 1725 | lectormangass.net | ✓ | ✓ |
| 1726 | loteriadehoy.com | ✓ | ✓ |
| 1739 | bpexch.live | ✓ | ✓ |
| 1742 | xn--12c1ezaww.com | ✓ | ✓ |
| 1749 | xxxvideosind.com | ✓ | ✓ |
| 1762 | moviesda33.com | ✓ | ✓ |
| 1765 | filmyzilla45.com | ✓ | ✓ |
| 1779 | shutterstock.com | ✓ | ✓ |
| 1781 | threads.net | ✗ | ✗ |
| 1809 | newsweek.com | ✓ | ✗ |
| 1810 | qcloud.com | ✗ | ✗ |
| 1819 | attn.tv | ✗ | ✗ |
| 1823 | themoviedb.org | ✗ | ✗ |
| 1824 | avito.st | ✗ | ✗ |
| 1851 | prokerala.com | ✓ | ✓ |
| 1876 | kir2kos.net | ✓ | ✓ |
| 1884 | shaalaa.com | ✓ | ✓ |
| 1889 | zdnet.com | ✗ | ✗ |
| 1900 | bisp.gov.pk | ✓ | ✓ |
| 1906 | bgtee.com | ✓ | ✓ |
| 1913 | ydparty04.tv | ✓ | ✓ |
| 1924 | adrta.com | ✗ | ✗ |
| 1926 | vdy.to | ✓ | ✓ |
| 1927 | wikihow.com | ✗ | ✗ |
| 1934 | ovh.com | ✓ | ✗ |
| 1935 | blacktoon415.com | ✓ | ✓ |
| 1941 | iqiyi.com | ✓ | ✗ |
| 1943 | case-battle.id | ✓ | ✓ |
| 1945 | x-arxx.com | ✓ | ✓ |
| 1948 | filmyzilla52.com | ✓ | ✓ |
| 1950 | moviesda34.com | ✓ | ✓ |
| 1956 | vidylee.click | ✓ | ✓ |
| 1958 | moviezwap.land | ✓ | ✓ |
| 1959 | blacktoon417.com | ✓ | ✓ |
| 1963 | xnhau.you | ✓ | ✓ |
| 1965 | rexify.com.ng | ✓ | ✓ |
| 1968 | xnhau.xxx | ✓ | ✓ |
| 1969 | filmyzilla46.com | ✓ | ✓ |
| 1972 | xnhau.bio | ✓ | ✓ |
| 1973 | scientificamerican.com | ✓ | ✗ |
| 1977 | sciencemag.org | ✓ | ✗ |
| 1990 | secureserver.net | ✗ | ✗ |
| 2001 | thelancet.com | ✓ | ✗ |
| 2003 | oreilly.com | ✓ | ✓ |
| 2008 | fastcompany.com | ✓ | ✓ |
| 2010 | si.edu | ✓ | ✓ |
| 2012 | jamanetwork.com | ✓ | ✓ |
| 2028 | strava.com | ✓ | ✓ |
| 2029 | pnas.org | ✓ | ✓ |
| 2031 | euronews.com | ✓ | ✓ |
| 2034 | meetup.com | ✓ | ✓ |
| 2050 | science.org | ✓ | ✗ |
| 2052 | lnkd.in | ✓ | ✗ |
| 2058 | sureserver.com | ✗ | ✗ |
| 2064 | vtb.ru | ✗ | ✗ |
| 2066 | jhu.edu | ✓ | ✓ |
| 2072 | clevertap-prod.com | ✗ | ✗ |
| 2077 | ign.com | ✗ | ✗ |
| 2100 | sogou.com | ✗ | ✗ |
| 2104 | acronis.com | ✓ | ✓ |
| 2116 | mashable.com | ✗ | ✗ |
| 2152 | salesforceliveagent.com | ✗ | ✗ |
| 2172 | ztomy.com | ✗ | ✗ |
| 2184 | mos.ru | ✓ | ✗ |
| 2188 | medicalnewstoday.com | ✓ | ✓ |
| 2211 | variety.com | ✗ | ✗ |
| 2217 | mirror.co.uk | ✗ | ✓ |
| 2250 | ad.gt | ✗ | ✗ |
| 2273 | juicyscore.com | ✗ | ✗ |
| 2277 | nyu.edu | ✓ | ✗ |
| 2301 | nba.com | ✓ | ✗ |
| 2324 | weibo.cn | ✗ | ✗ |
| 2330 | mparticle.com | ✗ | ✗ |
| 2345 | inc.com | ✓ | ✓ |
| 2382 | congress.gov | ✗ | ✗ |
| 2402 | umd.edu | ✓ | ✓ |
| 2403 | co.com | ✗ | ✗ |
| 2408 | tappx.com | ✗ | ✗ |
| 2425 | rollingstone.com | ✗ | ✗ |
| 2428 | tidal.com | ✓ | ✓ |
| 2430 | abcnews.com | ✓ | ✗ |
| 2431 | slate.com | ✓ | ✓ |
| 2436 | sendbird.com | ✓ | ✓ |
| 2438 | nejm.org | ✓ | ✗ |
| 2452 | redditmedia.com | ✗ | ✗ |
| 2462 | express.co.uk | ✗ | ✓ |
| 2488 | pcmag.com | ✗ | ✗ |
| 2500 | a2hosting.com | ✓ | ✓ |
| 2529 | glassdoor.com | ✓ | ✓ |
| 2539 | news.com.au | ✗ | ✗ |
| 2546 | hollywoodreporter.com | ✗ | ✗ |
| 2550 | openssl.org | ✗ | ✗ |
| 2553 | nextdoor.com | ✗ | ✗ |
| 2557 | smh.com.au | ✗ | ✗ |
| 2568 | zeit.de | ✓ | ✓ |
| 2571 | urldefense.com | ✗ | ✗ |
| 2579 | rockstargames.com | ✓ | ✓ |
| 2582 | technologyreview.com | ✗ | ✗ |
| 2612 | salthasher.com | ✓ | ✓ |
| 2615 | expressapisv2.net | ✗ | ✗ |
| 2624 | pxf.io | ✗ | ✗ |
| 2630 | thehill.com | ✗ | ✗ |
| 2643 | tmdb.org | ✗ | ✗ |
| 2656 | gamespot.com | ✓ | ✓ |
| 2670 | theglobeandmail.com | ✗ | ✗ |
| 2680 | smi2.ru | ✗ | ✗ |
| 2689 | ksapisrv.com | ✗ | ✗ |
| 2707 | mastodon.social | ✓ | ✓ |
| 2713 | cell.com | ✓ | ✗ |
| 2727 | newscientist.com | ✗ | ✗ |
| 2729 | ko-fi.com | ✓ | ✓ |
| 2746 | huntress.io | ✗ | ✗ |
| 2768 | bizjournals.com | ✓ | ✓ |
| 2774 | instacart.com | ✓ | ✓ |
| 2778 | system-monitor.com | ✗ | ✗ |
| 2800 | chicagotribune.com | ✗ | ✗ |
| 2811 | france24.com | ✗ | ✗ |
| 2832 | wa.gov | ✗ | ✗ |
| 2836 | salesforce-scrt.com | ✗ | ✗ |
| 2850 | sueddeutsche.de | ✓ | ✗ |
| 2860 | yumpu.com | ✓ | ✓ |
| 2863 | faz.net | ✗ | ✗ |
| 2877 | jwpltx.com | ✗ | ✗ |
| 2894 | mapquest.com | ✗ | ✗ |
| 2900 | prezi.com | ✓ | ✓ |
| 2926 | imageshack.us | ✗ | ✗ |
| 2936 | asahi.com | ✗ | ✗ |
| 2955 | theregister.com | ✓ | ✓ |
| 2963 | thehindu.com | ✗ | ✗ |
| 2969 | globalcdn.co | ✗ | ✗ |
| 2985 | irishtimes.com | ✓ | ✗ |
| 2990 | 1drv.ms | ✗ | ✗ |
| 3000 | medscape.com | ✓ | ✓ |
| 3003 | intelbras.com.br | ✓ | ✓ |
| 3018 | ctvnews.ca | ✗ | ✗ |
| 3026 | today.com | ✗ | ✗ |
| 3029 | faqs.org | ✓ | ✓ |
| 3033 | dehasher.com | ✓ | ✓ |
| 3039 | newsbreak.com | ✗ | ✗ |
| 3050 | hellotalk8.com | ✗ | ✗ |
| 3053 | vinted.com | ✓ | ✓ |
| 3085 | llamarpc.com | ✓ | ✓ |
| 3098 | skybridge.click | ✗ | ✗ |
| 3099 | loom.com | ✓ | ✓ |
| 3101 | cbssports.com | ✓ | ✓ |
| 3107 | crunchbase.com | ✓ | ✓ |
| 3118 | tagesschau.de | ✓ | ✓ |
| 3177 | metro.co.uk | ✗ | ✗ |
| 3187 | myanimelist.net | ✓ | ✓ |
| 3202 | billboard.com | ✗ | ✗ |
| 3222 | flipboard.com | ✗ | ✓ |
| 3238 | quizlet.com | ✓ | ✓ |
| 3253 | nzherald.co.nz | ✗ | ✗ |
| 3262 | infojobs.net | ✗ | ✗ |
| 3263 | tripadvisor.co.uk | ✓ | ✓ |
| 3273 | yahoo.net | ✓ | ✗ |
| 3287 | consumerreports.org | ✓ | ✓ |
| 3307 | foursquare.com | ✓ | ✗ |
| 3320 | nicovideo.jp | ✓ | ✓ |
| 3321 | permutive.app | ✗ | ✗ |
| 3334 | lavanguardia.com | ✓ | ✓ |
| 3341 | amazon.ae | ✗ | ✗ |
| 3347 | tagesspiegel.de | ✓ | ✓ |
| 3353 | lifehacker.com | ✗ | ✗ |
| 3354 | kicker.de | ✓ | ✗ |
| 3359 | tds.net | ✓ | ✗ |
| 3376 | deadline.com | ✗ | ✗ |
| 3436 | rs6.net | ✗ | ✗ |
| 3441 | flightaware.com | ✓ | ✓ |
| 3447 | macomnet.ru | ✓ | ✗ |
| 3450 | orf.at | ✓ | ✓ |
| 3451 | ladsp.com | ✗ | ✗ |
| 3460 | dictionary.com | ✓ | ✓ |
| 3467 | thefreedictionary.com | ✗ | ✗ |
| 3473 | lenovomm.com | ✗ | ✗ |
| 3474 | skimresources.com | ✗ | ✗ |
| 3498 | abc.es | ✗ | ✗ |
| 3503 | cuny.edu | ✗ | ✗ |
| 3504 | worldcat.org | ✓ | ✗ |
| 3509 | br.de | ✓ | ✓ |
| 3527 | glance.net | ✗ | ✗ |
| 3537 | stuff.co.nz | ✗ | ✗ |
| 3552 | ndr.de | ✓ | ✓ |
| 3576 | amazon.nl | ✗ | ✗ |
| 3579 | politico.eu | ✗ | ✗ |
| 3582 | smugmug.com | ✗ | ✗ |
| 3587 | ebsco.com | ✓ | ✓ |
| 3588 | proquest.com | ✓ | ✗ |
| 3592 | espncricinfo.com | ✓ | ✓ |
| 3616 | boston.com | ✓ | ✓ |
| 3629 | nydailynews.com | ✗ | ✗ |
| 3634 | xkcd.com | ✓ | ✓ |
| 3637 | amazon.eg | ✗ | ✗ |
| 3641 | entrust.net | ✗ | ✗ |
| 3642 | vinted.fr | ✓ | ✓ |
| 3645 | ansa.it | ✓ | ✗ |
| 3646 | lonelyplanet.com | ✓ | ✓ |
| 3660 | gifshow.com | ✗ | ✗ |
| 3671 | chosun.com | ✓ | ✓ |
| 3675 | yle.fi | ✓ | ✗ |
| 3684 | gitea.com | ✗ | ✗ |
| 3687 | aljazeera.net | ✓ | ✗ |
| 3692 | ardmediathek.de | ✓ | ✓ |
| 3693 | amazon.se | ✗ | ✗ |
| 3706 | oregonstate.edu | ✓ | ✓ |
| 3724 | wiki.gg | ✓ | ✓ |
| 3729 | realestate.com.au | ✓ | ✓ |
| 3739 | oaistatsig.com | ✓ | ✓ |
| 3743 | liberation.fr | ✗ | ✗ |
| 3746 | codeberg.org | ✗ | ✗ |
| 3754 | global-e.com | ✓ | ✓ |
| 3763 | zimbra.com | ✗ | ✗ |
| 3767 | studocu.com | ✓ | ✓ |
| 3796 | yomiuri.co.jp | ✗ | ✗ |
| 3805 | pcworld.com | ✓ | ✓ |
| 3819 | toyota.com | ✓ | ✓ |
| 3821 | rte.ie | ✓ | ✗ |
| 3829 | nos.nl | ✓ | ✗ |
| 3830 | sportradarserving.com | ✗ | ✗ |
| 3844 | news24.com | ✗ | ✗ |
| 3847 | wordreference.com | ✗ | ✗ |
| 3855 | codepen.io | ✓ | ✓ |
| 3886 | findagrave.com | ✓ | ✓ |
| 3887 | geforce.com | ✗ | ✗ |
| 3908 | bostonglobe.com | ✓ | ✓ |
| 3920 | aol.de | ✓ | ✗ |
| 3928 | stern.de | ✓ | ✗ |
| 3952 | ilsole24ore.com | ✗ | ✗ |
| 3968 | udn.com | ✓ | ✓ |
| 3975 | vaticannews.va | ✓ | ✓ |
| 3977 | rfi.fr | ✗ | ✗ |
| 3991 | screenrant.com | ✓ | ✗ |
| 4000 | xhpingcdn.com | ✗ | ✗ |
| 4024 | siteforce.com | ✗ | ✗ |
| 4025 | namebright.com | ✓ | ✓ |
| 4033 | merkur.de | ✓ | ✓ |
| 4041 | 123rf.com | ✓ | ✓ |
| 4046 | thestar.com | ✗ | ✗ |
| 4057 | kompas.com | ✗ | ✗ |
| 4078 | handelsblatt.com | ✓ | ✓ |
| 4079 | hse.ru | ✓ | ✓ |
| 4081 | seattletimes.com | ✗ | ✗ |
| 4104 | ahajournals.org | ✓ | ✓ |
| 4114 | mainichi.jp | ✗ | ✗ |
| 4119 | amazon.sa | ✗ | ✗ |
| 4129 | emojipedia.org | ✓ | ✓ |
| 4147 | amazon.pl | ✗ | ✗ |
| 4156 | teacherspayteachers.com | ✓ | ✓ |
| 4184 | ixl.com | ✓ | ✓ |
| 4208 | braintreegateway.com | ✗ | ✗ |
| 4225 | derstandard.at | ✓ | ✗ |
| 4229 | tvtropes.org | ✓ | ✗ |
| 4231 | opensea.io | ✓ | ✓ |
| 4233 | ketchcdn.com | ✗ | ✗ |
| 4237 | blender.org | ✓ | ✓ |
| 4238 | marinetraffic.com | ✓ | ✓ |
| 4255 | infura-ipfs.io | ✗ | ✗ |
| 4263 | snopes.com | ✓ | ✗ |
| 4265 | swr.de | ✓ | ✓ |
| 4273 | home.blog | ✗ | ✗ |
| 4285 | 20minutos.es | ✓ | ✓ |
| 4299 | computerworld.com | ✓ | ✓ |
| 4309 | gazeta.pl | ✓ | ✓ |
| 4321 | ebay.ca | ✓ | ✓ |
| 4326 | liquipedia.net | ✓ | ✗ |
| 4348 | radiofrance.fr | ✗ | ✗ |
| 4352 | howtogeek.com | ✓ | ✗ |
| 4353 | mcusercontent.com | ✗ | ✗ |
| 4366 | mdr.de | ✓ | ✓ |
| 4375 | wdr.de | ✓ | ✓ |
| 4391 | srf.ch | ✓ | ✓ |
| 4415 | ugtelset.ru | ✗ | ✗ |
| 4424 | d-net.pro | ✗ | ✗ |
| 4443 | biorxiv.org | ✓ | ✗ |
| 4446 | paycomonline.net | ✗ | ✗ |
| 4447 | 20minutes.fr | ✗ | ✗ |
| 4477 | rg.ru | ✓ | ✓ |
| 4495 | vinted.co.uk | ✓ | ✓ |
| 4501 | ebay.com.au | ✓ | ✓ |
| 4543 | cnki.net | ✗ | ✗ |
| 4549 | huya.com | ✗ | ✗ |
| 4566 | feednews.com | ✗ | ✗ |
| 4582 | amazon.com.tr | ✗ | ✗ |
| 4611 | theregister.co.uk | ✓ | ✓ |
| 4640 | ucsc.edu | ✓ | ✓ |
| 4643 | lesoir.be | ✗ | ✗ |
| 4663 | rm358.com | ✓ | ✓ |
| 4665 | xda-developers.com | ✓ | ✗ |
| 4680 | msnbc.com | ✗ | ✗ |
| 4685 | mooo.com | ✗ | ✗ |
| 4689 | collider.com | ✓ | ✗ |
| 4690 | meb.gov.tr | ✓ | ✗ |
| 4695 | imhentai.xxx | ✓ | ✓ |
| 4700 | boxofficemojo.com | ✗ | ✗ |
| 4703 | polygon.com | ✓ | ✗ |
| 4713 | turbify.com | ✓ | ✓ |
| 4717 | wowhead.com | ✓ | ✓ |
| 4718 | join.chat | ✓ | ✓ |
| 4722 | adweek.com | ✓ | ✓ |
| 4746 | helsinki.fi | ✗ | ✗ |
| 4757 | hurriyet.com.tr | ✗ | ✗ |
| 4763 | yandexadexchange.net | ✗ | ✗ |
| 4765 | channel4.com | ✗ | ✗ |
| 4768 | onlamp.com | ✓ | ✓ |
| 4802 | atlasobscura.com | ✓ | ✓ |
| 4803 | nine.com.au | ✗ | ✗ |
| 4806 | telegraaf.nl | ✗ | ✗ |
| 4813 | iol.co.za | ✓ | ✗ |
| 4858 | drive2.ru | ✗ | ✗ |
| 4870 | dr.dk | ✗ | ✗ |
| 4899 | aps.org | ✓ | ✓ |
| 4918 | softpedia.com | ✓ | ✓ |
| 4923 | arte.tv | ✗ | ✗ |
| 4939 | dayforcehcm.com | ✗ | ✗ |
| 4941 | deccanherald.com | ✗ | ✗ |
| 4945 | lesechos.fr | ✗ | ✗ |
| 4948 | tripadvisor.fr | ✓ | ✓ |
| 4978 | sbs.com.au | ✓ | ✓ |
| 4980 | hindawi.com | ✓ | ✓ |
| 4992 | imageshack.com | ✗ | ✗ |
Not on this list and want to be sure? Check your own site. If you meant to block training but are blocking citations too, this guide covers how the mistake usually happens and how to undo it. Robots.txt is not the only way a site goes missing from AI answers - a fully-allowed page can still render blank to a crawler if its content only exists after JavaScript runs.