Tutorials

llms.txt Guide: Help AI Understand Your Website

Sep 24, 2026

In one line

llms.txt is a plain-text Markdown file in your site root that helps AI models understand your website. Format, setup steps, validation, and llms-full.txt.

llms.txt is a plain-text file placed in your website's root directory. It uses Markdown to present a short site summary and links to your most important pages, so large language models can understand your site quickly. This guide walks through the format, a worked example, and how to validate the result.

What Is llms.txt

llms.txt is a community-proposed standard. It calls for a plain-text file in the site root that organizes your most important content in Markdown, written for large language models to read. Its intended audience is not humans or traditional search engines, but the models behind AI tools such as ChatGPT, Claude, and Perplexity.

A typical web page is cluttered with navigation menus, ads, and scripts. A model has to strip all of that away before it can extract the substance. llms.txt hands over a clean summary with links, effectively a reading guide prepared for AI.

One caveat: llms.txt is still a proposal, and the major AI companies have made no formal commitment to reading it. But the cost of creating one is minimal, and a file written once keeps working. Deploying it early carries no downside.

How robots.txt, sitemap.xml, and llms.txt Divide the Work

The three files complement each other rather than compete. Each can be summed up in one line:

robots.txt manages permissions: it tells crawlers which paths they may or may not fetch
sitemap.xml manages the inventory: it lists your page URLs for search engines
llms.txt manages understanding: it tells language models what your site does and which content matters most

AI crawlers such as GPTBot (OpenAI), ClaudeBot (Anthropic), and PerplexityBot still check robots.txt before fetching anything. So even with a well-written llms.txt, confirm that robots.txt is not blocking these user agents. If it is, the models never reach your site at all.

The File Format, Part by Part

llms.txt is ordinary Markdown, built from four parts:

An H1 heading: your site or company name, and the only element the spec requires
An intro: a blockquote (lines starting with >) immediately after, explaining in two or three sentences what the site is and who it serves
Sectioned link lists: H2 headings for each section, with links formatted as - [Page title](URL): one-line description
An Optional section: an H2 literally named "Optional" is a reserved keyword for secondary content that models may skip when context is tight

Links can simply point to your existing web pages. If you maintain separate Markdown versions of those pages, link to those instead, since models parse them more cleanly.

Worked Example: Writing One Step by Step

Suppose you run a Hong Kong commercial kitchen supplier, Example Catering Equipment Co., selling commercial refrigerators and cooking ranges with installation and repair services. Here is how to write its llms.txt in five steps.

Step one, write the H1 and the intro. Use your actual company name, and state your business, products, and service area plainly.

Step two, pick the key pages. Do not cram in the whole site. Choose only the pages that answer customer questions: product categories, service descriptions, and FAQs.

Step three, list the links under sections, each with a one-line description. Step four, move secondary content such as company history into the Optional section. Step five, save the file as plain text in UTF-8 encoding, named llms.txt.

The finished file looks like this (example.com stands in for the real domain):

# Example Catering Equipment Co.

> Example Catering Equipment Co. is a Hong Kong commercial kitchen
> equipment supplier, offering sales, installation, and repair of
> commercial refrigerators, cooking ranges, and dishwashing equipment
> across Hong Kong, Kowloon, and the New Territories.

## Products

- [Commercial refrigeration](https://www.example.com/products/refrigeration): Specs and buying guide for upright and counter refrigerators
- [Cooking equipment](https://www.example.com/products/cooking): Chinese wok ranges, Western ovens, and griddles

## Services

- [Installation and repair](https://www.example.com/services): On-site installation process, maintenance plans, and pricing

## FAQ

- [FAQ](https://www.example.com/faq): Delivery times, warranty terms, payment methods

## Optional

- [About us](https://www.example.com/about): Company background and team

Placement, Validation, and the llms-full.txt Variant

The file must sit in the site root, meaning it opens directly at yourdomain.com/llms.txt. Each subdomain is independent, so a blog subdomain needs its own copy.

Validation takes three steps:

Open the URL in a browser and confirm it loads correctly, displays as plain text, and shows no garbled characters
Click through every link in the file and confirm none of them are dead
Paste the full contents into any AI chat tool, ask it to summarize your website, and check whether its understanding is accurate

llms-full.txt is the advanced variant. Instead of listing links, it merges the full text of your key pages into one large file, so a model can read everything without fetching page by page. It suits documentation sites and content-heavy sites, at the cost of a larger file that must be regenerated whenever content changes. The two files can coexist: llms.txt as the reading guide, llms-full.txt as the full text.

FAQ

What is llms.txt?

llms.txt is a plain-text file in a website's root directory that introduces the site's key content to large language models through a curated list of links.

Does llms.txt replace robots.txt or sitemap.xml?

No. robots.txt manages permissions, sitemap.xml manages the page inventory, and llms.txt manages understanding. The three complement each other.

What is the difference between llms.txt and llms-full.txt?

llms.txt lists only links and short descriptions. llms-full.txt merges the full text of key pages into a single file, so models do not need to fetch each page separately.

Will AI definitely read llms.txt?

There is no guarantee. It is currently a community proposal rather than an official standard, but it costs almost nothing to create, and deploying it early carries no downside.

How often should llms.txt be updated?

Update it whenever you add important pages or your business changes, and make sure every link in the file stays valid.

Check Your Site's AI Visibility

Writing a good llms.txt is one step toward better AI visibility, but only one. To find out what else your site is missing in the eyes of AI search, such as incomplete structured data or blocked AI crawlers, you can run it through UDomain's free AEO Auditor (ai.ud.hk/aeo-auditor). AEO here means Answer Engine Optimization, the practice of making your content citable by AI answer engines; it has nothing to do with the Authorized Economic Operator customs program that shares the acronym. Enter your URL, get a report, and work through the checklist item by item.