Frequently asked
questions

The llms.txt file is a new web standard that provides AI systems with a streamlined, easy-to-understand version of your website content. It helps large language models (LLMs) overcome context window limitations by delivering essential information in a clean, focused format.

While robots.txt tells search crawlers which pages they can access, llms.txt helps AI systems understand your content structure and meaning. robots.txt is about permissions and access control, while llms.txt is about content comprehension.

An llms.txt file must start with an H1 heading (the project name), followed by a blockquote summary of your site. Then it can include additional markdown sections with H2 headers organizing links to your important content. There is also an Optional section for secondary information.

While llms.txt provides navigation structure, the optional llms-full.txt contains complete documentation content. For simpler websites, llms.txt alone is usually sufficient. For documentation sites, having both can be beneficial for AI comprehension.

Current AI systems do not automatically discover these files. Users can provide the file content to AI tools (like Claude or ChatGPT) by pasting the URL, uploading the file, or copying the contents directly into their prompts. This gives the AI important context about your site.

The llms.txt specification was proposed by Jeremy Howard, co-founder of Answer.AI, to solve the challenge of AI systems struggling to process complex website documentation within their limited context windows.

The llms.txt file should be placed at the root of your website domain (e.g., example.com/llms.txt), similar to robots.txt and sitemap.xml. This makes it easy for AI systems and users to find it.

Adoption is growing rapidly. Companies like Anthropic, Mintlify, Cursor, and many documentation platforms have implemented llms.txt support. Any site that wants to be better understood by AI assistants can benefit from adding this file.

Our generator makes it simple - just enter your website URL and we will analyze your site structure to create an appropriate llms.txt file. You can then edit it to highlight the most important content before adding it to your website.

No, any website can benefit from an llms.txt file. While it is particularly valuable for technical documentation, it can also help AI systems understand e-commerce sites, blogs, personal websites, educational resources, and corporate sites. It is for any content you want AI to comprehend better.

Sitemap.xml lists all indexable pages but does not help with content processing. It contains too many pages to fit in the context window of an AI context window and does not distinguish between important and less important content. llms.txt is a curated, AI-friendly summary of your most relevant content.

llms.txt uses Markdown format with a specific structure: an H1 title, a blockquote description, optional additional details, and then sections with H2 headers containing link lists. Each link should have a description. The Optional section marks content that can be skipped when context is limited.