llms.txt is a plain text file, written in markdown, that lives at yourdomain.com/llms.txt and lists a site's most important pages for AI systems to read. It occupies the same spot as robots.txt, but instead of controlling access it recommends which pages a model should read. Setting one up takes under an hour. Current data, though, shows it is not the citation lever most guides claim it is.
What Is llms.txt?
llms.txt is a markdown index file at the root of a domain. It gives an AI model a condensed map of a site's key pages instead of full HTML to parse. Jeremy Howard, co-founder of Answer.AI, published the llms.txt proposal on September 3, 2024, according to Answer.AI. The llms.txt spec gives the reason: context windows are too small to hold most websites in full. Converting HTML navigation, ads, and scripts into clean text is difficult and imprecise.
The file has a companion format, llms-full.txt, which exports a site's entire content as one long markdown document instead of an index of links. Most implementations use the short index version and reserve the full export for smaller documentation sites.
How the llms.txt Format Works
An llms.txt file follows a fixed markdown structure, starting with an H1 for the site name and an optional one-line summary. From there, H2-headed sections list linked pages with a short note on each. Any section titled "Optional" signals pages a model can skip if it needs a shorter context. That is the file's main job: telling the model what matters most, not just what exists.
That structure only helps if a model actually requests the file, and that is where the format runs into a wall. Gary Illyes addressed llms.txt at a Search Central Deep Dive event on July 23, 2025. According to Search Engine Land, he said Google does not support the file and is not planning to. John Mueller made the same point earlier, comparing the file to the keywords meta tag, according to Search Engine Journal. That is a signal the site owner controls, which search engines learned decades ago to ignore.
Google's own tooling treats the file differently from Google Search. Chrome's Lighthouse now runs an llms.txt audit inside an Agentic Browsing category, according to Chrome for Developers. The audit asks whether the file helps AI agents understand a site without crawling all of it. Its pass conditions, though, make this a thinner endorsement than it looks. A missing file returns Not Applicable rather than a failure, because Chrome documents the file as optional. Lighthouse flags only a server error on a file that already exists.
The category carries no 0 to 100 score either, just a fraction of checks passed. Chrome labels it experimental and based on proposed standards, according to Chrome for Developers. Both Google positions hold once you separate the mandates. Chrome is drafting for a browser and AI-agent web that does not fully exist yet. Search ranks on signals it trusts now, across traditional and AI results alike.
Does llms.txt Actually Get You Cited by AI?
No published study has found llms.txt increasing AI citation rate. The largest study to date found no correlation at all, and its citation prediction model got more accurate once llms.txt was dropped as a variable. That is the finding to weigh before spending implementation time on this file over other GEO work.
According to SE Ranking, a study of nearly 300,000 domains found llms.txt on 10.13% of the domains it sampled. Having the file made no measurable difference to how often a domain was cited in AI answers.
Server logs back this up from the other direction. According to Ahrefs, a study of 137,210 domains in May 2026 found that 28% published an llms.txt file. Among those, 97% received zero requests that month. Among the 3% that saw any traffic, SEO audit tools were the largest requester category, and AI retrieval bots accounted for about 1% of requests.
No major model provider, including OpenAI, Anthropic, and Google, has publicly confirmed using llms.txt as a signal in production web retrieval. The one place it does see real use is developer tooling. Developers point coding assistants like Cursor and Claude Code at llms.txt files to pull API documentation into context, the use case Howard designed it for.
How to Create and Implement an llms.txt File
Building a basic llms.txt takes four steps: pick your priority pages, write the file, upload it, and confirm it resolves.
- List your 10 to 20 most important pages. Product category pages, comparison pages, and FAQ pages are the highest-value entries for an ecommerce brand.
- Write the file in markdown, starting with an H1 for your brand name and an optional one-line summary. Add H2 sections that group the linked pages by type, such as Products, Guides, and Company.
- Upload it to yourdomain.com/llms.txt. It must sit at the root, not in a subfolder, or crawlers that do check for it will not find it.
- Verify it resolves by requesting the URL directly in a browser and confirming a 200 response with clean markdown, not a redirect or 404.
Keep it under a few hundred lines. A file that tries to list every page on the site defeats the purpose of a condensed index.
When llms.txt Is Worth Doing (and What Actually Moves AI Citations)
llms.txt costs almost nothing to set up, which is why it is worth the hour. A stale file with dead links is worse than no file at all, so review it whenever your key pages change. It should not, though, replace the GEO work that has a measured effect on citations.
The techniques with actual evidence behind them work on the content and structure a model reads, regardless of whether it ever fetches a special file. That means clean schema markup that disambiguates your entity and consistent entity signals across every source a model draws from. It also means content built with answer-first structure that a crawler can extract straight from the HTML.
The distinction that matters is whether the crawler has to go looking. llms.txt only works if a bot chooses to request a file it has no reason to want. OmniGro's dual-layer architecture serves a cleaner version of the page at the URL the crawler already requested. Nothing depends on opt-in behavior that most bots do not exhibit.
Making sure AI crawlers can reach and index your pages does more for citation rate than any index file at the root. That work is covered in why Bing indexing matters for ChatGPT visibility.
FAQs
Does llms.txt improve SEO rankings?
No. Google has stated directly that it does not use llms.txt for Search or AI Overviews, and no ranking factor depends on the file's presence.
Does the Chrome Lighthouse audit make llms.txt a ranking signal?
No. Lighthouse's Agentic Browsing category is experimental and sits apart from Search, feeding no Google ranking system. The audit does not even require the file. A 404 returns Not Applicable, and only a broken file gets flagged.
Do I need both llms.txt and llms-full.txt?
Most sites only need the short index version. llms-full.txt suits documentation-heavy sites where a model benefits from ingesting the entire content in one file.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers what they are allowed to access. llms.txt is an index of recommended pages with no enforcement mechanism; a model can ignore it entirely, and most currently do.
Will llms.txt matter more once the standard is formalized?
Possibly. Google has ruled it out for Search. W3C has an open strategy issue tracking llms.txt, opened in April 2025 and at the investigation stage, according to W3C, not a formal standardization track. Until major AI providers commit to reading it in production, treat any future benefit as unconfirmed.
Should ecommerce brands skip llms.txt entirely?
Not entirely. Add it once as a low-effort, low-risk checklist item. Then put the remaining GEO budget into schema, entity consistency, and answer-first content, where the evidence for citation impact actually exists.
Conclusion
llms.txt costs an hour and rarely causes harm if you keep it current, but the real GEO budget belongs elsewhere. Put it into schema, entity consistency, and content built to be extracted from the page itself, not a file most crawlers never request. To see which of these is actually costing you citations, get a free AI visibility assessment. We'll show you where your brand stands across ChatGPT, Claude, Perplexity, Gemini, and Google AI Mode.