Can AI read newsbreak.com?
newsbreak.com blocks OAI-SearchBot and PerplexityBot in its robots.txt, so ChatGPT and Perplexity cannot cite it. Read live from the site's own robots.txt with Google's reference parser.
OpenAI and Perplexity cannot index you for AI answers (robots.txt blocks OAI-SearchBot, PerplexityBot).
If you want this changed, I can write the exact fix for newsbreak.com and confirm it works once it is live. It is paid work, quoted up front before anything starts. Email me at reese@lastminutedealshq.com.
From the census: This site was also in our 5,000-site AI-crawler census (Tranco rank 3039, collected 2026-09-07). Its robots.txt blocked 11 training crawlers and 3 AI-search crawlers we track at that time. Its llms.txt in that census was 1887 bytes with 3 links, with a blockquote summary. The same census found 2 malformed User-agent lines in its robots.txt, an invalid product token that silently becomes an unscoped rule. See the full pattern.
Want a daily check on this site?
Daily monitoring for newsbreak.com: a re-check every day of its AI-crawler access, readable text and llms.txt. Each result and each change shows on a private status page that also has an Atom feed. You also get an email when a watched status changes. $19/mo after a 7-day free trial, cancel anytime. See a live example of the status page.
Start free 7-day trialRelated guides
- Why your site is not showing up in ChatGPT
- Should you block GPTBot?
- The 296 sites that block AI search but not Google: deliberate or accidental?
- Google AI Overviews: what actually controls whether you appear
All of them are on the guides page.
Check your own site or see the crawler census.
Do you fix AI search visibility for clients? A list of sites whose server refuses the ChatGPT or Perplexity search crawler comes with a free 25-row sample.