How it works
The file sits at /llms.txt and lists the site's purpose and links to its most useful pages, sometimes with a fuller llms-full.txt version containing page content.
Adoption has grown, but use hasn't followed. Ahrefs found 28% of domains in its sample published one, yet 97% of files got zero requests in May 2026 [2]. OtterlyAI saw 84 requests for llms.txt among 62,100+ AI bot visits [3].
Google's AI search guide says you don't need llms.txt or other machine-readable AI files to appear in its generative AI features [1]. John Mueller has compared it to the old keywords meta tag [3].
Why it matters
llms.txt is a good example of a tactic that spread faster than the evidence. It's cheap to create and won't hurt, but it isn't a substitute for the things that do affect AI visibility: crawlable, clear pages, strong SEO and a credible brand. It may still help some developer tools and coding assistants that read documentation, which is where Mueller suggests it belongs [2].
By the numbers
of llms.txt files got zero requests in May 2026 (Ahrefs)
Ahrefs: We analyzed 137K sites: 97% of llms.txt files never get read, Jun 2026[2]of domains in Ahrefs' SEO-aware sample published one
Ahrefs: We analyzed 137K sites: 97% of llms.txt files never get read, Jun 2026[2]of AI bot visits requested llms.txt in OtterlyAI's study
Optimizely: Does llms.txt work? The honest answer, 2025 to 2026[3]Key components
Practical example
A Pune SaaS company's agency charged for an 'AI visibility package' built around an llms.txt file. Six months on, server logs showed no AI crawler had requested it, and AI mentions hadn't changed. The company kept the file (it costs nothing) but moved budget to clearer product pages, comparison content, documentation and PR, which did move its AI visibility.
Benefits and limitations
Benefits
- Quick and harmless to create.
- May help developer and coding tools that read docs.
- A useful exercise in summarising your key pages.
Tools and implementation
- A text editor: the file is plain Markdown.
- Server logs: whether anything requests it.
- CMS plugins that generate llms.txt automatically.
- Google's AI search guide: what actually matters for Google.
Common mistakes
- Paying for llms.txt as an 'AI SEO' solution.
- Using it instead of fixing crawlability and content.
- Putting content in it that contradicts your pages.
- Expecting it to control how AI uses your content (that's robots.txt).
- Not checking logs to see if anything reads it.
Related glossary terms
Our SEO services in Pune focus AI visibility work on what the evidence supports, and our website design and development services can add llms.txt in minutes if you want it anyway. Further reading: how to choose an SEO company in Pune.
Get a free consultationFrequently asked questions
Do I need an llms.txt file?
Not for Google: its guide says you don't need llms.txt to appear in AI Overviews or AI Mode [1]. It won't hurt, but don't expect it to improve AI visibility on its own.
Do AI crawlers read llms.txt?
Rarely. Ahrefs found 97% of files got no requests in May 2026 [2], and OtterlyAI saw about 0.1% of AI bot visits request it [3].
What is the difference between llms.txt and robots.txt?
Robots.txt tells crawlers what they may access and is widely respected. llms.txt is a voluntary summary for AI tools that few, if any, major platforms use.
Who created llms.txt?
It was proposed in September 2024 as a way to help language models find a site's key content [2]. It isn't an official web standard.
Sources
- Search Engine Journal: Google says llms.txt is purely speculative for now (May 2026). Cross-checked with: Semrush: Google publishes guide to optimizing for generative AI search.
- Ahrefs: We analyzed 137K sites: 97% of llms.txt files never get read (Jun 2026). Cross-checked with: Search Engine Journal: 97% of llms.txt files got no requests.
- Optimizely: Does llms.txt work? The honest answer (2025 to 2026). Cross-checked with: walshy.dev: Stop shipping AI files nobody reads.
Last updated: 18 November 2026. Every statistic on this page links to its original source and was cross-checked against a second source where one was available.
