The verdict up front
As of mid-2026, no major AI provider reads llms.txt in production, large-scale studies find no relationship between having the file and being cited, and AI crawlers do not even request it. If llms.txt is on your SEO roadmap, the evidence says delete the line item and spend the effort on the things that measurably correlate with AI visibility.
What llms.txt was supposed to be
The proposal was appealing: a robots.txt-style file that tells language models what your site is about and which pages matter, so AI assistants would understand and cite you better. It cost twenty minutes to add, so thousands of sites did — and an industry of tutorials, generators and audits grew around it through 2025.
What Google actually said
Google's Gary Illyes confirmed in July 2025 that Google does not support llms.txt and is not planning to. John Mueller went further, comparing it to the keywords meta tag — the canonical example of metadata that site owners fill in and no search system reads (Search Engine Journal's report). No other major provider — OpenAI, Anthropic, Meta, Mistral — has committed to reading it in production either.
What the data shows at scale
- Ahrefs, 137,000 sites (May 2026): 97 percent of llms.txt files received zero traffic. The bots the file is addressed to are not reading it.
- SE Ranking, ~300,000 domains: no statistically significant correlation between having an llms.txt file and how often a domain is cited in AI answers.
- Crawler logs: AI bots do not request llms.txt on domains where it does not exist — meaning they are not even probing for it. A crawler that wanted the file would ask for it and get 404s. They do not ask (a good independent evidence roundup).
Why it failed
Standards work when the consumer side adopts them. robots.txt works because crawlers agreed to read it. llms.txt shipped the publisher side of a handshake that no model provider ever agreed to. LLMs build answers from search indexes and crawled content — they already have your pages, and a self-description file adds nothing they trust more than the pages themselves. There was never a mechanism by which the file could work; it just felt proactive.
The real cost was never the twenty minutes
The file itself is harmless. The damage is what it displaced: teams reported llms.txt as their AEO strategy while skipping the things the evidence supports. If your AI-visibility plan fits in a text file, you do not have one.
What the evidence says works instead
Two findings with actual data behind them: brand mentions predict AI citations about three times better than backlinks, and cited content is measurably fresher than what merely ranks — see the freshness effect. Add extractable structure (question-shaped headings, self-contained answer passages) and you have the whole defensible playbook. We track which AI assistants actually send us visitors using this GA4 setup — measured, not assumed.
Frequently asked questions
Does llms.txt improve AI visibility or SEO?
No evidence supports it. Google has said it does not read the file, an Ahrefs study of 137,000 sites found 97% of llms.txt files received zero traffic, and an SE Ranking analysis of roughly 300,000 domains found no correlation between the file and AI citations.
Do AI crawlers read llms.txt?
Crawler log analyses show AI bots do not request llms.txt even on sites where it exists, and do not probe for it on sites where it does not. No major provider has committed to reading it in production.
Should I delete my llms.txt file?
It is harmless to leave in place — the cost is not the file but the false sense of strategy. Keep it or delete it; either way, move your AI-visibility effort to brand mentions, freshness and extractable content structure, which have measurable correlations with citations.
What should I do instead of llms.txt?
Build brand mentions across the platforms AI engines cite (reviews, communities, YouTube), keep your highest-value content fresh and updated in place, and structure pages with question-shaped headings and self-contained answers that models can quote.