Does llms.txt Actually Work? What the 2026 Data Shows

If you only read the headlines, you'd think this question was settled twice over. One camp declared llms.txt dead the moment Google said it wouldn't use the file. The other camp keeps publishing "add llms.txt for instant AI visibility" guides as if citations were waiting on the other side of a one-click plugin.

Both camps are wrong, and the reason is the same: almost nobody bothered to look at the data.

Now, nearly two years after Jeremy Howard first proposed the standard, we finally have it. Several independent teams have analysed server logs, scanned hundreds of thousands of domains, and run controlled citation tests. The picture they paint is clearer—and more useful—than either the hype or the obituaries suggest.

So let's answer the question properly. Does llms.txt actually work? The honest answer is: yes, for one specific job, and no, for the job most people are buying it to do. Here's the evidence.

First, What "Work" Even Means

Before we touch a single statistic, we have to separate two completely different claims that get blurred together constantly. This confusion is the single biggest reason people end up disappointed.

Claim 1: llms.txt helps AI coding assistants and agents use your content at inference time. This is what Howard originally proposed. You're working in Cursor or Claude Code, you point it at a library's docs, and the assistant pulls in clean Markdown instead of choking on messy HTML. This is an inference-time context-loading job.

Claim 2: llms.txt makes ChatGPT, Perplexity, and Google AI Overviews cite your site more. This is the Generative Engine Optimization (GEO) promise that took over the SEO conversation in 2025. It treats llms.txt as a ranking signal for AI search.

Almost every "does it work" argument falls apart because someone is defending Claim 1 while their opponent is attacking Claim 2. They're both right. They're just talking about different files doing different jobs. Keep these two claims in separate boxes for the rest of this article and everything snaps into focus.

The Crawler Evidence: Do AI Systems Even Fetch the File?

The most basic test of Claim 2 is embarrassingly simple: if AI search engines used llms.txt, their crawlers would request it. So do they?

Multiple independent log studies through 2025 and into 2026 say: barely.

• A 90-day experiment by OtterlyAI logged 62,100 AI-bot requests to a test site. Just 84 of them—0.1%—touched /llms.txt. A single average content page got around 265 hits. The dedicated AI file underperformed a random blog post by a wide margin.
• A scan of 1,000 domains by Flavio Longato (August 2025) found no requests from GPTBot, ClaudeBot, or PerplexityBot to llms.txt files at all. Bing made 7 requests across all 1,000 sites; OpenAI's search bot made 10. Googlebot accounted for 95% of the traffic—to regular pages, not the AI file.
• An analysis of over 515 million LLM-bot events by Limy (May 2026) found only 408 requests targeting llms.txt across GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, and Google-Extended combined. Their word for the share was "statistically negligible."

To put that last one in perspective: out of more than half a billion crawler events from the exact bots that power AI search, the file the entire GEO industry told you to create got requested four hundred times.

There is some contrary signal, and honesty demands we include it. Ray Martinez, founder of Archer Education, posted server logs showing OpenAI's GPTBot fetching his llms.txt roughly every 15 minutes. Mintlify, citing tracking firm Profound, reports Microsoft and OpenAI crawlers actively fetching both llms.txt and llms-full.txt on the sites they monitor.

How do we square that with the big log studies? The most defensible reading is exploratory crawling on a small subset of sites, not production retrieval. Some bots are clearly poking at the file. But "a bot fetched it" and "the model used it to decide what to cite" are very different things—and the firms reporting the positive signal (Profound sells GEO tracking, Mintlify sells docs hosting) have a commercial stake worth keeping in mind.

The Citation Evidence: Does Having One Change Anything?

Fetching is one thing. The question that actually matters for Claim 2 is whether sites with an llms.txt get cited by AI more than sites without one. Here the evidence is even cleaner, because two large studies controlled for it directly.

SE Ranking analysed roughly 300,000 domains in late 2025. About 10% of them had an llms.txt file. The correlation between having one and being cited in AI answers? Effectively zero. The detail that should end the debate: when their data scientists removed the llms.txt variable from their predictive model, the model got more accurate. The file wasn't a weak signal—it was noise.

A separate scan of 37,894 AI-cited domains by Trakkr found sites with llms.txt averaged 6.8 citations versus 6.7 for sites without. Run through a statistical significance test, that gap returned a p-value of 0.85—about as close to "literally no effect" as real-world data ever gets.

So for Claim 2, the verdict from the data is unambiguous: publishing an llms.txt file does not measurably increase your AI citations in 2026. Not because it's controversial, but because the largest available datasets show no effect.

If you want the tactics that do correlate with AI citations, that's a separate post worth your time—see our guide on llms.txt and SEO for where to actually spend your effort.

What Google Actually Says (All Three Contradictory Things)

Google's position deserves its own section because the company has, remarkably, said three different things.

Google Search says no. Back in April 2025, John Mueller compared llms.txt to the long-discredited keywords meta tag, noting that you can tell from your server logs that AI services don't even check for it. At Search Central Live in July 2025, Gary Illyes stated plainly that Google doesn't support llms.txt and isn't planning to. Google's 2026 AI-search guidance still lists it among the things you don't need.

Google's other product teams say yes. Chrome's Lighthouse 13.3 (updated May 2026) added an experimental "Agentic Browsing" audit that checks for llms.txt, describing it as "an emerging convention." Several Google docs properties—Chrome for Developers, the Gemini API site—publish their own llms.txt files.

And Google Search briefly published one by accident. In December 2025, someone spotted an llms.txt on Google's own Search documentation. Mueller responded "hmmn :-/" and the file 404'd within hours; he later explained it was a CMS rollout, not an endorsement.

The lesson isn't that Google is lying. It's that different teams are making different bets. Google Search—the part that decides AI Overviews—is firmly negative. Google's developer-tooling teams find the format genuinely useful for agents. Which is exactly the Claim 1 / Claim 2 split showing up inside a single company. Note also that the Lighthouse audit is gentler than the headlines suggested: per Chrome's own docs, it only flags sites where fetching the file returns a server error. If you have no llms.txt at all, you're marked "Not Applicable," not failed.

So Where Does llms.txt Actually Work?

Here's the part the obituaries miss. While Claim 2 was quietly collapsing under the weight of the data, Claim 1 was getting stronger.

Coding assistants and IDE agents genuinely use it

This was always the point. When you feed a library's llms.txt to an AI coding assistant, you're solving the exact problem Howard described: getting clean, current documentation into a model that may have a stale knowledge cutoff. Tools like Cursor and Windsurf support documentation commands that pull these files directly.

The most rigorous test comes from LangChain's Lance Martin, who benchmarked four ways of feeding documentation to a coding agent across five real tasks. The ranking he found: an optimized llms.txt beat a vector database, which beat a standard llms.txt, which beat dumping everything into context. His framing is the clearest one-liner in this whole debate—llms.txt is "just RAG with full documents as retrieval units." That's a use case with measured results, not vibes.

MCP servers turn it into infrastructure

LangChain shipped a tool called mcpdoc, an MCP server that exposes llms.txt files to Cursor, Windsurf, and Claude Desktop as callable tools. This matters because it fixes llms.txt's oldest weakness—discovery. The file no longer has to be magically found by a crawler; an agent is explicitly handed it.

Stripe is using it to program what AI says about Stripe

This is the most interesting development of all, and it points to where the real value lives. Stripe's llms.txt includes an "Instructions for Large Language Model Agents" section that explicitly steers assistants away from deprecated APIs—telling them, in effect, "never recommend the legacy Card Element." That's not SEO. That's a company shaping what every AI coding assistant says about its product. As autonomous agents start doing real work, this kind of machine-readable instruction layer becomes genuinely valuable.

The market backs this up. Mintlify—which auto-generates llms.txt for thousands of docs sites and co-developed the llms-full.txt format with Anthropic—raised a $45M Series B at a $500M valuation in April 2026, on the back of roughly 10x year-over-year revenue growth. The layer of the web where llms.txt actually gets used is scaling fast.

The Adoption Picture: Thriving in Tech, Absent Everywhere Else

If you've seen wildly different adoption numbers thrown around, that's because they measure different slices of the web—and all of them can be true at once:

• BuiltWith counted roughly 844,000 sites with an llms.txt by late 2025.
• SE Ranking's 300,000-domain sample put adoption at about 10%.
• ProGEO.ai found only 7.4% of the Fortune 500 had one as of early 2026—versus nearly 93% with a robots.txt.
• Trakkr's sector breakdown found SaaS and dev tools at 24%, but government and academic sites at 1.5%, and review/reference sites at essentially 0%.

The shape is consistent across every source: llms.txt has deep penetration in developer tooling, AI companies, and technical documentation, and near-zero presence on the mainstream web. That's not a standard fading—it's a standard that found its niche and stayed there. For a fuller tour of who's implemented it, see our roundup of companies using llms.txt.

The Honest Verdict for 2026

Let's put the two claims back together and answer the title question directly.

Does llms.txt work for getting cited by AI search engines? No. The 300,000-domain study, the 90-day crawler log, the 37,894-domain citation scan, and the 515-million-event analysis all point the same way: no measurable effect. If someone is selling you llms.txt as an AI-visibility tactic, the evidence isn't on their side.

Does llms.txt work for feeding documentation to AI coding assistants and agents? Yes—and the case is getting stronger. Benchmarks show it beating vector RAG for documentation tasks, MCP servers have solved its discovery problem, and companies like Stripe are using it to actively shape agent behaviour.

The standard isn't dead, and it isn't magic. It's a developer-documentation tool that got mis-sold as an SEO tool, and most of the disappointment around it traces back to that single category error.

What You Should Actually Do

Your move depends entirely on which job you're hiring llms.txt to do.

Create an llms.txt file if you:

• Publish developer documentation or maintain an API where coding-assistant accuracy matters
• Want to shape how AI agents describe and use your product (the Stripe play)
• Have agent-driven or programmatic workflows that fetch your content
• Can auto-generate the file so maintenance is effectively free

The cost is trivial with the right tooling, and the upside in the agentic-web era is real. You can build one in minutes with our free generator and check it with our validator.

Don't prioritize llms.txt if you:

• Run a general business, e-commerce, or content site expecting more AI citations
• Are being sold it purely as a GEO or "AI visibility" deliverable
• Need proven ROI before investing maintenance time

For those sites, spend the same hours on the things that do correlate with citations: original data and research, presence on review platforms, community mentions on Reddit and Quora, fast page loads, and solid structured data. Our llms.txt and SEO guide breaks down where that effort actually pays off.

If you're on WordPress and your SEO plugin offers one-click generation

Turn it on—see our WordPress guide. It costs nothing, won't hurt, and gets you into the directories. Just don't expect a traffic bump.

The Bottom Line

A year ago we asked whether llms.txt was dead. The 2026 data answers that question better than any opinion could: it was never dying—it was being measured against the wrong goal.

Judged as an SEO tactic, llms.txt fails the evidence test cleanly and repeatedly. Judged as what it was actually built to be—a clean, machine-readable feed for AI coding assistants and agents—it works, it's spreading where it matters, and the infrastructure around it is attracting serious money.

So before you create one, don't ask "will this help my AI rankings." Ask "do AI agents need to read my documentation." If the answer is yes, llms.txt is one of the cheapest, highest-leverage files you can ship. If the answer is no, your effort belongs somewhere else—and now you have the data to know the difference.