A Markdown file at your site root that gives AI models a curated overview of your product, docs and key pages.

Is llms.txt an official standard?

No. It's a community proposal from September 2024, unconfirmed by OpenAI or Google, but increasingly common on AI-facing sites.

Does llms.txt improve AI search rankings?

No engine confirms it as a ranking factor. It's a context aid; the citation levers remain crawl access, content quality and structure.

How do I create an llms.txt file?

Write a Markdown file with your product name, a one-line summary and curated links with descriptions, upload it to your site root, and validate it with curl or AEO Pulse Labs' Site Audit.

Back to blog llms.txt Markdown file as a map for AI models

llms.txt: What It Is, How to Generate One, and Why AI Crawlers Look for It

llms.txt is a Markdown file that gives AI models a map of your site. The format, a working example, and what it actually affects.

5 min read

There's a small file that more and more AI-facing sites are adding at their root: llms.txt. Here's what it is, what it does, and an honest take on whether you need one.

What llms.txt is

llms.txt is a proposed standard (from Jeremy Howard, September 2024): a plain Markdown file at yoursite.com/llms.txt that gives language models a curated, human-readable map of your site — what you do, what's important, and where to find it. Think of it as robots.txt's cousin: robots.txt tells crawlers what they may fetch; llms.txt tells models what's worth reading.

Why LLMs like Markdown

Markdown is close to plain text, dense with meaning per token, and structurally simple — headings, lists, links. LLMs parse it more reliably than HTML full of navigation and scripts. That's the whole idea: your value proposition and key resources in a format models ingest cleanly.

The format

H1: your site or product name.
A blockquote: one or two sentences describing what you offer.
H2 sections (e.g., Docs, Product, Pricing) with Markdown links — each with a short description of what the target contains.

Keep it under a page. Curated, not exhaustive: an llms.txt with 500 links is a sitemap wearing a costume, and models will skim it the way humans do.

A working example

# AEO Pulse Labs
> AI search visibility platform: monitor brand mentions and citations across ChatGPT, Perplexity, Gemini and Claude; track AI crawlers; create AI-ready content.

## Product
- [AI Visibility Tracking](https://aeopulselabs.io/visibility): See how often AI engines cite your brand.
- [Free AI Readiness Grader](https://aeopulselabs.io/grader): Score any URL for AI search in 60 seconds.

## Guides
- [What is AEO?](https://aeopulselabs.io/blog/what-is-aeo): Answer Engine Optimization explained.

Does it affect AI citations? The honest answer

No engine has confirmed llms.txt as a ranking or citation factor — and Google has publicly downplayed it. Treat it as context, not a lever: it's cheap, takes an hour, and helps models that choose to read it understand your site faster. It is not a substitute for the things that actually drive citations: crawl access for AI bots, clean content, schema markup, and answer-first structure.

Common mistakes

Dumping every URL instead of the 10-20 that matter.
No descriptions on links.
Letting it go stale when you ship new pages.
Blocking AI crawlers in robots.txt while advertising an llms.txt — the map is useless if the territory is fenced.
Forgetting to link it from your site.

How to validate

Fetch your file like a model does: curl https://yoursite.com/llms.txt and confirm it returns 200 with correct Markdown. AEO Pulse Labs' Site Audit checks llms.txt automatically — presence, freshness, and whether your robots.txt lets AI crawlers actually reach the pages it points to. You can also see how the Site Audit fits into the full AI-readiness workflow.

Written by the AEO Pulse Labs team. Our Site Audit validates your llms.txt, your robots.txt AI-crawler access, and the schema AI engines need to cite you.

How to Optimize for AI Crawlers SEO vs GEO vs AEO: What's the Difference?