Can AI read sfgate.com?
sfgate.com blocks PerplexityBot in its robots.txt, so Perplexity cannot cite it. Its homepage also sends an almost-empty page that fills in with JavaScript, so a crawler reads no text. Read live from the site's own robots.txt with Google's reference parser.
✗AI search crawlers can reach it
Blocked: PerplexityBot
✗Homepage is readable by crawlers
The page returned very little text. Worth checking that real content is in the HTML, not loaded later by JavaScript.
✗Has an llms.txt
No /llms.txt. It is an emerging, optional convention, not required to be cited today. You can generate one at https://ai-visibility.lastminutedealshq.com/llms-txt
Perplexity cannot index you for AI answers (robots.txt blocks PerplexityBot).
Your homepage may not be readable by AI crawlers (little or no text in the HTML).