What is llms.txt and does your site need one?

AI tools now read websites for people: a chatbot checks your pricing page, a coding assistant looks up your docs. But web pages are built for human eyes, full of menus, scripts and layout that a language model (LLM, the tech behind tools like ChatGPT and Claude) has to dig through. llms.txt is a proposal for giving those tools a shortcut.

What it is

llms.txt is a plain Markdown file (a simple text format with headings and links) that lives at the root of your site, like https://example.com/llms.txt. Think of it as a short guide to your site written for AI: who you are, and which pages matter most.

It was proposed by Jeremy Howard in 2024, and the format is described in the llms.txt proposal. It's a community proposal, not an official web standard. It can also sit in a subfolder, so /docs/llms.txt would cover just your documentation.

What goes in it

The format is intentionally small. In order, the file has:

• A title (an H1 heading) with your site or business name. This is the only required part.
• A short summary in a blockquote, covering the key facts someone needs to know.
• Optional notes in normal paragraphs or lists.
• Sections of links under H2 headings, each link with an optional one-line description.

A section named Optional is the convention for nice-to-have links that an AI tool can skip when it's short on space. Here's what a small business might publish:

# Bayou Plumbing Co.

> Licensed plumbing repair and water heater installation serving the west Houston area.

We offer same-day service on weekdays. Pricing is quoted after an on-site visit.

## Services

- [Water heaters](https://example.com/water-heaters): Repair, replacement, tankless installs
- [Drain cleaning](https://example.com/drains): Clogs, camera inspections

## Company

- [Service area](https://example.com/service-area): Cities and ZIP codes we cover
- [Contact](https://example.com/contact): Phone, hours, booking form

## Optional

- [Blog](https://example.com/blog): Maintenance tips

How it's different from robots.txt and sitemap.xml

These three files sit side by side at your site root, but they do different jobs:

File	Job
robots.txt	Tells bots what they may and may not crawl
sitemap.xml	Lists every page you want search engines to find
llms.txt	Gives AI tools a short, curated summary of your site

One common mix-up: llms.txt does not block or allow anything. If you want to control which AI crawlers can access your site, that's still robots.txt.

Why it matters (and where it doesn't)

Here's the honest part. Google's teams give different answers depending on what you're asking about, as Search Engine Journal points out.

For Google Search, it does nothing. Google's AI search optimization guide lists llms.txt among the things you don't need to appear in AI Overviews or AI Mode. Google's John Mueller has even compared it to the keywords meta tag, the old tag search engines stopped trusting years ago. So adding one won't boost your rankings.

For AI agents, it can help. Chrome's Lighthouse tool now includes an llms.txt audit in its agentic browsing checks, which look at how easily AI agents can use a site. Google's Chrome docs note that without the file, agents may have to crawl more of your site to work out what it's about. The audit treats the file as optional: a missing file isn't counted against you, but a server error when fetching it is.

Adoption is real, if uneven. The proposal notes that the AI companies publish their own, for example Anthropic and OpenAI for their developer docs, and it's most useful today for documentation that coding assistants read.

How Jrooky checks it

Jrooky checks whether your site serves a file at /llms.txt and whether it loads successfully (a 200 response). This check is informational only: it shows up in your results, but it doesn't change your score. That matches where things stand today, where the file is a reasonable extra rather than a requirement.

Does your site need one?

It depends on who reads your site.

• Worth adding if you have documentation, a software product, or info people often ask AI assistants about (services, pricing, policies). It takes a few minutes.
• Low priority if your only goal is Google rankings. It won't move them.

If you add one:

1. Write it by hand, or use a tool that generates it. Platforms like Wix and WordPress plugins like Yoast SEO can create one for you.
2. Link only to your most important pages, with a short description for each.
3. Keep it current. An outdated summary can point AI tools at wrong prices or old pages.
4. Make sure it loads without errors at https://yoursite.com/llms.txt.

Run a free scan to see whether your site already serves an llms.txt file.