AI labs can now fetch my *.md files
It works! AI labs started indexing, training, and serving my new *.md files ✨ Right after pushing to prod, I asked ChatGPT to check {{startup_markdown_URL}}, and it couldn't because of "cache missed" or "400 timeout fetching". So I added all 8,831 URLs to the sitemap.xml and added the following headers: "Content-Type": "text/plain; charset=utf-8" "Content-Disposition": "inline" I can't tell if this helped, but 24 hours later, ChatGPT was able to fetch them properly 😊 Maybe AI assistants need the page to be indexed first, before being able to crawl it?


@marclou · Jul 13, 2026
I added wildcard path matching to DataFast's bot traffic feature So it's easier to know if a group of URLs is crawled by AI assistants > datafa.st/docs/bot-traff…
- 3 replies
- 1 reposts
- 19 likes
- 11.1K views





