How to Create Your First llms.txt File: A Step-by-Step Guide

Creating an llms.txt file for your website is one of the most effective ways to make your content AI-friendly. It helps large language models (LLMs) understand and navigate your content more efficiently, leading to better representation in AI-generated responses.

Having worked with many websites to implement the llms.txt standard, I've created this guide to walk you through the process of creating your first llms.txt file, from planning to implementation.

Use llms-txt.io

First, if you don't want to create the file yourself, you can use our free tool to generate it for you.

You can use llms-txt.io to generate both the llms.txt and llms-full.txt files, in seconds.

Prerequisites

Before we begin, you'll need the following.

1. Access to your website's root directory - The llms.txt file should be placed at the root of your domain (e.g., example.com/llms.txt or docs.example.com/llms.txt).
2. An inventory of your important content - Take some time to identify the most important content, documentation, blog posts, or resources on your website.
3. Markdown knowledge - Basic familiarity with Markdown formatting is helpful but not required.

Step 1: Understand the llms.txt Structure

The llms.txt file follows a specific structure defined in the standard:

# Title

> Optional description goes here

Optional details go here

## Section name

- [Link title](https://link_url): Optional link details

## Optional

- [Link title](https://link_url)

Let's break down the components:

1. Title (H1) - The only required section, typically your project, company, or website name.
2. Description (Blockquote) - A brief summary of what your website or project is about.
3. Details - Additional context about your project.
4. Sections (H2) - Categories for organizing your content.
5. Links - Markdown links to your content with optional descriptions.
6. Optional Section - A special section for less critical information.

Step 2: Create the Title and Description

Start by creating a new text file named llms.txt. The first elements to add are the title and description:

# Your Company Name

> Brief, compelling description of what your company or project does.

For example, here's what Tiptap (a rich text editor framework) uses:

# Tiptap

> Tiptap is the headless and extensible rich-text editor framework tailored to modern web and app development needs.

Tips for writing an effective description:

• Keep it under 50 words
• Focus on the core value proposition
• Use clear, jargon-free language
• Include key information AI systems might need to understand your content

Step 3: Add Optional Details

After the description, you can add additional context that helps explain your project:

# Your Company Name

> Brief, compelling description of what your company or project does.

Your Company Name helps users [core benefit]. Founded in [year], we specialize in [specialization] for [audience].

This section is optional but can provide valuable context. For example, Tiptap includes their core benefits after the description:

Core Benefits
- Build rich text editors similar to Notion within days
- Supports React, Next, Vue, Svelte, JavaScript and many more
- Robust, battle-tested open source framework under the MIT license
- With over 100 extensions highly adaptable

Keep this section concise and focused on information that helps AI systems understand your content better.

Step 4: Plan Your Sections

Before adding links, plan the main sections of your llms.txt file. Your organization strategy should reflect your website's structure and content types.

Here are common approaches.

1. Product-based organization (like Stripe):

   ## Payments
   ## Billing
   ## Connect
   ## Terminal
   

2. Content-type organization:

   ## Documentation
   ## API Reference
   ## Guides
   ## Blog Posts
   

3. User journey organization:

   ## Getting Started
   ## Core Features
   ## Advanced Usage
   ## Support
   

Choose a structure that makes the most sense for your content. For a company blog, you might organize by topic or date. For an API, you'd likely organize by endpoints or features.

Step 5: Add Content Links

Now, add links to your important content under each section.

## Documentation

- [Getting Started Guide](https://example.com/docs/getting-started.md): A beginner-friendly introduction to our platform.
- [API Reference](https://example.com/docs/api-reference.md): Complete documentation of all available endpoints.
- [Authentication](https://example.com/docs/authentication.md): Learn how to authenticate requests to our API.

Each link should follow this format:

- [Link Title](URL): Brief description of what this content covers.

The description is optional but highly recommended as it provides context about the linked content.

Create .md Versions of Your Pages

The llms.txt standard recommends linking to markdown (.md) versions of your pages. If you don't already have markdown versions, you can do this:

1. Create markdown versions of your key pages
2. Use a tool to automatically convert HTML to markdown
3. Append .md to your regular URLs (e.g., example.com/docs/guide.html.md) and make sure your server serves these properly

If creating separate markdown files isn't feasible right now, you can still link to your regular HTML pages - the standard is flexible.

Step 6: Add the Optional Section

The Optional section is a special part of the llms.txt standard designed for content that's less critical but still relevant.

## Optional

- [Company History](https://example.com/about/history.md): Our journey from founding to today.
- [Changelog](https://example.com/changelog.md): Detailed history of product updates and changes.
- [Community Guidelines](https://example.com/community/guidelines.md): Rules and best practices for our community.

This section helps AI systems prioritize your content when context windows are limited. Items in the Optional section may be excluded when an AI system needs to conserve space.

Step 7: Review and Refine

Once you've created a draft of your llms.txt file, review it and make sure it meets the following.

1. Completeness: Have you included all your most important content?
2. Organization: Is the structure logical and easy to navigate?
3. Accuracy: Are all links correct and descriptions accurate?
4. Balance: Is there a good balance between completeness and conciseness?

Refine your file based on this review, focusing on making it as helpful as possible for AI systems.

Step 8: Validate Your Markdown

Before publishing, validate your Markdown to make sure it's correctly formatted.

1. Use a Markdown validator or linter to check for syntax errors
2. Preview the Markdown in an editor that supports Markdown rendering
3. Check that all links are correctly formatted and working

Even small formatting errors can affect how AI systems parse your file, so this validation step is important.

Step 9: Publish Your llms.txt File

Now it's time to publish your llms.txt file.

1. Upload the file to your website's root directory
2. Ensure it's accessible at yourdomain.com/llms.txt
3. Verify it loads correctly by visiting the URL in your browser

For most websites, this means placing the file in the same directory as your index.html or robots.txt file.

Step 10: Consider Creating an llms-full.txt File

While not required, many websites also provide an llms-full.txt file that contains the complete content of linked pages. This allows AI systems to access all your content in one request.

Creating an llms-full.txt file involves a process that's more time-intensive.

1. Combining the content of all pages referenced in your llms.txt file
2. Preserving the structure and formatting of each page
3. Clearly marking the source of each content section

Tools are available to help generate this file automatically from your llms.txt file.

Complete llms.txt Example

Here's a complete example of what a simple llms.txt file might look like.

# Acme Corp

> Acme Corp provides cloud-based project management tools for remote teams.

Founded in 2020, Acme Corp helps distributed teams collaborate efficiently with our suite of project tracking, document management, and team communication tools.

## Getting Started

- [Quick Start Guide](https://acmecorp.com/docs/quickstart.md): Set up your first project in under 5 minutes.
- [User Onboarding](https://acmecorp.com/docs/onboarding.md): Comprehensive guide for new users.
- [Team Setup](https://acmecorp.com/docs/team-setup.md): Learn how to invite and organize your team members.

## Core Features

- [Project Management](https://acmecorp.com/features/projects.md): Create, organize, and track projects from start to finish.
- [Document Collaboration](https://acmecorp.com/features/documents.md): Real-time collaborative document editing for teams.
- [Task Tracking](https://acmecorp.com/features/tasks.md): Assign, prioritize, and monitor tasks across your organization.
- [Team Chat](https://acmecorp.com/features/chat.md): Integrated messaging for teams and projects.

## API Documentation

- [API Overview](https://acmecorp.com/api/overview.md): Introduction to the Acme API capabilities.
- [Authentication](https://acmecorp.com/api/auth.md): Secure your API requests with OAuth 2.0.
- [Projects API](https://acmecorp.com/api/projects.md): Endpoints for managing projects programmatically.
- [Tasks API](https://acmecorp.com/api/tasks.md): Create and manage tasks via API.

## Optional

- [Pricing Plans](https://acmecorp.com/pricing.md): Details on our subscription options.
- [Blog](https://acmecorp.com/blog/index.md): Project management tips and product updates.
- [Roadmap](https://acmecorp.com/roadmap.md): Upcoming features and improvements.
- [Changelog](https://acmecorp.com/changelog.md): History of product updates and changes.

Different Approaches for Different Sites

The structure of your llms.txt file should reflect the nature of your website. Let's look at some examples here.

For Documentation Sites

Focus on organizing by topic categories with clear hierarchies.

## Getting Started
## API Reference
## Guides
## Troubleshooting

For Product Companies

Organize by product lines or features.

## Product A
## Product B
## Integrations
## Support

For Blogs and Content Sites

Organize by content categories or topics.

## Marketing Guides
## Case Studies
## Industry News
## Tutorials

For Small Business Websites

Focus on key business information.

## Services
## About Us
## Portfolio
## Contact

Tips for Success

Based on analyzing llms.txt implementations from companies like Stripe, Zapier, and Cloudflare, here are some best practices.

1. Be concise - Prioritize brevity while maintaining clarity.
2. Use consistent formatting - Maintain the same pattern for all links and descriptions.
3. Link to the most useful pages - Focus on content that provides the most value to users.
4. Keep descriptions informative - Briefly explain what each link contains.
5. Update regularly - Revisit your llms.txt file when you add important new content.
6. Consider AI users - Think about how AI systems might use your content when organizing your file.
7. Start small - Begin with your most critical content, then expand over time.

Testing Your llms.txt Implementation

After publishing your llms.txt file, you can test it by:

1. Directly asking AI systems - Ask an AI assistant about your website and see if it references information from your llms.txt file.
2. Checking for proper rendering - Make sure the file is properly rendered as Markdown when accessed directly.
3. Validating links - Verify that all linked content is accessible.

Conclusion

Creating an llms.txt file is a straightforward yet powerful way to make your content more accessible to AI systems. By following this step-by-step guide, you've taken an important step toward ensuring your website's content is properly represented in the AI-driven future of information retrieval.

Remember that your llms.txt file is a living document. As your website evolves, be sure to update your llms.txt file to reflect new and important content.