The llms.txt file has been generating real buzz in AI and SEO circles, but the adoption story has two very different sides. This episode of the SEO.co podcast digs into the data behind the llms.txt adoption gap in 2026, separating the genuine momentum on the publisher side from the near-total silence on the crawler side — and explaining what that disconnect actually means for your strategy.
Here's what the episode covers:
- What llms.txt is (and isn't): A markdown-based, human-curated index proposed in late 2024 to help LLMs navigate site content — not a blocking directive, not a ranking signal, and not yet a ratified web standard from any major standards body.
- The adoption paradox: Publisher uptake grew nearly 9× in a single year, reaching 10–16% of major websites — yet an Ahrefs analysis found 97% of those files received zero crawler requests in May 2026.
- What the data actually shows: An SE Ranking study found that including llms.txt as a variable in an AI citation prediction model made the model less accurate — suggesting the file isn't even neutral noise at this stage.
- Where the major platforms stand: Google's John Mueller and Gary Illyes have both been unusually explicit: Google does not support llms.txt and has no plans to. OpenAI, Anthropic, and Perplexity have said very little — and that silence is telling.
- The Shopify signal: In May 2026, Shopify auto-deployed llms.txt and agent-facing endpoints across its entire platform — hinting that the file's real future may be in Business-to-Agent transactions, not search citation.
- A practical publish-or-defer framework: Documentation-heavy sites, developer platforms, ecommerce catalogs, and regulated brands have a defensible case for shipping it now; standard content marketing properties chasing generative engine optimization citation lift are better served shoring up entity coverage, structured data, and technical SEO fundamentals first.
The episode makes clear that llms.txt is genuinely cheap to implement — roughly three days of combined engineering and content work for a mid-sized site — which is why teams ship it defensively even when ROI is murky. But cheap doesn't mean strategic, and the data right now points to a file that looks responsible on an audit while delivering close to nothing in measurable AI visibility. The smarter play, for most sites, is to treat it as optional plumbing rather than a priority line item.
For more from the show, check out the episode Search Intent Mismatch: Why Good Content Fails to Rank — a strong companion listen if you're thinking about how content structure and signals interact with how AI systems evaluate and surface pages.
SEO.co