llms.txt: What It Is, the Spec, and How to Create One

What is llms.txt?

llms.txt is a Markdown file served at /llms.txt on your domain that provides a structured, LLM-friendly description of your website. It was proposed by Jeremy Howard, the founder of fast.ai, as a way to help AI models understand what a site does without parsing every page. You can read the full proposal at llmstxt.org.

Think of it as the equivalent of robots.txt, but for AI comprehension rather than crawl permission. Where robots.txt tells crawlers what they may and may not access, llms.txt tells language models what your site is about and which pages are most important. For a concise definition, see our glossary entry on llms.txt.

The file uses Markdown deliberately. LLMs handle Markdown far better than raw HTML. A well-structured Markdown file with headings, links, and brief annotations gives an AI model a clear, parseable overview of a site in a single read.

One important note: llms.txt is a community proposal, not an official standard. It has not been ratified by the W3C, IETF, or any standards body. That said, adoption is growing as more site owners recognize the value of making their sites legible to AI systems, and the simplicity of the format makes it easy to adopt with no risk.

What the spec says

The llms.txt proposal defines a straightforward format. The file lives at the root of your domain at /llms.txt and follows a specific Markdown structure:

• An H1 heading with your site or company name. This is the first line of the file and tells the model whose site it is reading.
• A blockquote with a one- or two-sentence description of the site. This gives the model immediate context about what the business does.
• Sections with links. Each section groups related pages under a heading (such as Services, Resources, or Blog) with Markdown links and brief annotations explaining what each page covers.

The proposal also describes an optional companion file, /llms-full.txt, which includes the full text content of key pages. While llms.txt is a concise overview designed to fit in a single context window, llms-full.txt gives AI models the complete content without requiring them to fetch each page individually.

The key principle is balance: concise enough for a single context window, comprehensive enough to be useful. An AI model that reads your llms.txt should come away knowing what you do, what your most important pages are, and how to accurately describe your business.

What to include in your llms.txt

The spec gives a framework, but the content you put in the file depends on your site. Here is what we recommend based on the proposal and our own implementation:

• Site identity. Your name, what you do, when you were founded, and where you are located. This is the foundation the model uses to represent you accurately.
• Core pages. Homepage, about, contact, pricing — each with a one-line description. These are the pages an AI model is most likely to reference when answering questions about your business.
• Services and products. Each major offering with a brief explanation of what it includes, who it is for, and what makes it distinct.
• Content hubs. Blog, resources, guides, tools — with links to the most important pieces. You do not need to list every post, just the ones that best represent your expertise.
• Contact information. How to reach you, where you are located, and any relevant business details that help the model answer practical questions.
• Last updated date. So AI models know how fresh the information is. This is especially important because models may cache or store retrieved content.

Webvello’s own llms.txt is 540 lines and covers all of the above. We include every service page, every resource guide, our tools, glossary entries, and blog posts with annotations. It is the same approach we use across our AI search optimization practice: give the model exactly the information it needs to represent you correctly.

How AI models use llms.txt

When an AI model with web access encounters your site, it can read llms.txt to quickly understand the site’s purpose and structure. This is more efficient than crawling and parsing every page. Instead of extracting meaning from HTML, navigation menus, and footer links, the model gets a clean, pre-structured overview in a format it handles natively.

This helps the model give accurate answers about your business. If someone asks an AI assistant what services you offer, what your pricing looks like, or how to contact you, a well-written llms.txt gives the model the information it needs to answer correctly rather than guessing from fragments of HTML.

Browsing-enabled AI assistants like ChatGPT, Perplexity, and others can benefit from this. Some have begun checking for llms.txt when they access a site. However, adoption is not universal — not every AI system looks for the file yet. It should complement your existing structured data and AI search strategy, not replace them.

For a broader view of how to structure your site for AI extraction, see our guide on self-contained units for AI extraction and structuring service pages for AI.

llms.txt vs robots.txt vs sitemap.xml

These three files live at your site root and serve different purposes for different consumers. They complement each other; none replaces the others.

• robots.txt tells crawlers what they can and cannot access. It is about permission. A crawler reads it to know which paths are off-limits. You can check how crawlers interact with your site using our AI crawler checker.
• sitemap.xml tells crawlers where your pages are. It is about discovery. A search engine reads it to find URLs it might not encounter through links alone.
• llms.txt tells AI models what your site does and what matters most. It is about comprehension. A language model reads it to understand your business, services, and content structure in one pass.

A site that has all three gives both traditional search engines and AI models everything they need: permission boundaries, a complete page inventory, and a human-readable overview of what the site is about.

Step-by-step: creating your llms.txt

Here is how to create an llms.txt file for your site, following the structure defined in the proposal:

1. Start with an H1. Use your site or company name as a Markdown H1 heading. This is the first line of the file.
2. Add a blockquote description. Write a one- to two-sentence summary of what your site does. Use a Markdown blockquote (>) so the model can distinguish it from the rest of the content.
3. List your most important pages. Use Markdown links with brief annotations. Each link should include a short description of what the page covers and why it matters.
4. Organize by section. Group pages under descriptive headings: Services, Resources, Blog, Tools, Contact. This mirrors how the model will parse the file.
5. Add contact info and a last-updated date. Include your location, how to reach you, and the date you last revised the file. Freshness matters for AI retrieval.
6. Save at your site root. The file must be accessible at /llms.txt. In a Next.js project, this means placing it at public/llms.txt. For other frameworks, put it wherever static files are served from the root.
7. Optionally create llms-full.txt. If you want to give AI models the full text of your key pages without requiring them to fetch each one, create a companion file at /llms-full.txt with expanded content.

Once the file is live, you can verify it is accessible by visiting yourdomain.com/llms.txt in a browser. It should render as plain text or Markdown.

Frequently asked questions

About the work

Webvello is an AI search optimization and generative engine optimization company founded in 2024, based in Roseville, California. We help businesses control how AI models represent them — through structured data, llms.txt, and content architecture designed for LLM extraction. Founder George Shvaya came to this work from cybersecurity and systems engineering. Our AI Search Playbook covers the broader program, and our work portfolio shows the results.

Sources: the llms.txt proposal at llmstxt.org, created by Jeremy Howard. Webvello’s own llms.txt implementation at webvello.com/llms.txt (540 lines, verified November 3, 2026). The proposal may evolve; check the linked page for the latest version.