Skip to content

Glossary llms.txt

llms.txt (a Markdown guide to a site for AI agents)

What the llms.txt proposal is, the exact file format, how it differs from robots.txt and sitemaps, and how Markdown copies of pages fit in.

Updated 19 November 20261 min readDeutsch

llms.txt is a proposed Markdown file, placed at /llms.txt or under a path of a website, that gives AI agents a short summary of the site and curated links to the pages worth reading.

The idea#

Web pages are built for people, with navigation, scripts and ads around the content. The llms.txt proposal, first written in 2024 by Jeremy Howard, adds one Markdown file that tells agents what a site is and where its most useful content lives. The file can sit at the root, /llms.txt, or under a path such as /docs/llms.txt, where it covers the pages under that path.

The format#

The sections come in this order:

  1. an H1 with the name of the project or site, the only required section;
  2. a blockquote with a short summary;
  3. optional paragraphs or lists with more detail, without headings;
  4. optional H2 sections, each with a list of links written as name, optionally followed by a colon and notes.

A section named "Optional" holds links an agent can skip when it needs a shorter context.

  # Example Shop

  > Outdoor shoes and clothing, shipped within the EU.

  ## Products

  - [Trail running shoes](https://example.com/trail.md): sizes, materials and care
  - [Returns](https://example.com/returns.md): return policy and deadlines

Markdown copies of pages#

The proposal also asks sites to offer a clean Markdown version of pages agents may need, at the same URL with .md appended or replacing the file extension, and to point to it with rel="alternate" type="text/markdown".

llms.txt, robots.txt and sitemaps#

FilePurpose
robots.txtWhich automated access a site accepts
sitemap.xmlEvery indexable page, for search engines
llms.txtA short, curated overview that agents read on demand

Questions

Does llms.txt control which AI crawlers may visit?

No. Access rules belong in robots.txt. llms.txt gives agents context and links when they need information about a topic.

Is llms.txt an official standard?

No. It is a proposal, first written in 2024 by Jeremy Howard and since updated as version 2.

Sources

  1. The /llms.txt file, v2 · llmstxt.org checked 19 November 2026

You choose what goes in and where it goes out.

Central AI does everything in between. Your first 3 products are free.

  1. Your inputYour shop, your files or your product pages.
  2. Central AIResearches, verifies and writes everything in between. Every value is backed by sources you can check; what the sources do not support is held back.
  3. Your channelsProduct data and listings for the channels you choose.