TrustMRR tracked 1M+ AI bots in 30 days

AI bots data for TrustMRR: - OpenAI and Anthropic used llms.txt heavily to answer users, index pages, and train their models. - Markdown pages are crawled 50% of the time; AI bots still use HTML. - /mcp is the page that ChatGPT/Claude crawl the most to answer users. n=1M+ AI bots in the last 30 days

Zeno Rocha@zenorocha · Aug 12, 2026how to make your site readable by ai agents: 1. add an llms.txt so search tools stop guessing urls 2. check your server logs, not your analytics 3. serve a markdown version of every page 4. support content negotiation headers 5. rethink your JS-only widgets