Can AI read glassdoor.com?

glassdoor.com allows the AI search crawlers in its robots.txt, so ChatGPT and Perplexity are able to read and cite it. Read live from the site's own robots.txt with Google's reference parser.

✓AI search crawlers can reach it
OAI-SearchBot and PerplexityBot are allowed.
?Homepage is readable by crawlers
Could not fetch the homepage (403), so I cannot tell how much text a crawler would get.
✗Has an llms.txt
No /llms.txt. It is an emerging, optional convention, not required to be cited today. You can generate one at https://ai-visibility.lastminutedealshq.com/llms-txt
About the crawlers named here: OAI-SearchBot, PerplexityBot. Each page has what the vendor says it does, the robots.txt lines, how many of the top 5,000 sites block it and how to verify a request.

I could not fetch this site's homepage, so I cannot tell whether it sends readable text.

From the census: This site was also in our 5,000-site AI-crawler census (Tranco rank 2529, collected 2026-09-07). Its robots.txt blocked 10 training crawlers and 1 AI-search crawler we track at that time. The same census found 1 malformed User-agent line in its robots.txt, an invalid product token that silently becomes an unscoped rule. See the full pattern.

Want a daily check on this site?

Daily monitoring for glassdoor.com: a re-check every day of its AI-crawler access, readable text and llms.txt. Each result and each change shows on a private status page that also has an Atom feed. You also get an email when a watched status changes. $19/mo after a 7-day free trial, cancel anytime. See a live example of the status page.

Start free 7-day trial
Add this badge to your site:
Want to be counted for the October re-run?
Share this result: Post on X ·

Related guides

All of them are on the guides page.

Check your own site or see the crawler census.

Do you fix AI search visibility for clients? A list of sites whose server refuses the ChatGPT or Perplexity search crawler comes with a free 25-row sample.