Guides · AI search

What is llms.txt? What the file does and how to create one.

In short: llms.txt is a Markdown text file at yourdomain.com/llms.txt that gives AI agents a short, curated overview of your site with its most important links. Jeremy Howard proposed it in September 2024; version 2 is dated August 10, 2026. It does nothing for your Google rankings, because Google Search ignores it. It’s useful where AI agents read your content on someone’s behalf.

By the Claire teamLast checked 12 min read

At a glance

As of October 8, 2026

Google: creating an llms.txt will neither harm nor help your visibility in Google Search, which ignores it.

Source: Google Search Central: Optimizing for generative AI features (updated Jul 10, 2026)

Chrome’s Lighthouse has an llms.txt audit and calls the file an emerging convention for AI agents.

Source: Chrome for Developers: Lighthouse llms.txt audit (updated May 5, 2026)

Every Shopify store serves an llms.txt automatically, mirroring its agent description at /agents.md.

Source: Shopify docs: llms.txt.liquid

Yoast SEO generates the file in WordPress with one click and updates it weekly; Wix creates one for every site.

Source: Yoast: llms.txt functional specification

Contents

Basics

What is an llms.txt file? A signpost for AI agents, at a fixed address.

Websites are built for people: navigation, cookie banners, scripts. An AI agent answering a question about your business has to dig through all of that. llms.txt tells it in a few lines what the site is about and where the key information lives.

01

An open proposal, not a standard

The proposal comes from Jeremy Howard, published September 3, 2024 on llmstxt.org; the current version 2 was modified on August 10, 2026. No standards body has adopted it; the community develops it on GitHub.

Source:llmstxt.org: The /llms.txt file, v2 (modified Aug 10, 2026)

02

What an llms.txt should contain

Per the spec, in this order: an H1 with the site or project name (the only required part), a blockquote with a short summary, any number of paragraphs or lists without headings, and sections with H2 headings, each containing a list of links. A section called “Optional” marks links an agent can skip when context is short.

The file normally sits at the root. Since version 2 it may also live in a subpath such as /docs/llms.txt and then covers everything below it; when several apply, the most specific wins.

# Site name

> One or two sentences: who you are, what you offer, for whom.

Other key facts in plain text, e.g. locations, starting prices, service area.

## Key pages

- [Services](https://example.com/services): what we actually offer
- [Pricing](https://example.com/pricing): current plans and what they include

## Optional

- [Careers](https://example.com/jobs)

Source:llmstxt.org: The /llms.txt file, v2 (modified Aug 10, 2026)

03

What’s new in version 2

The August 2026 revision brings three practical changes.

  • Markdown versions of pages at the same URL, either with .md appended (page.html.md) or with the extension replaced (page.md).
  • Discovery via link relations: rel="alternate" type="text/markdown" points to a page’s Markdown version, rel="describedby" to the llms.txt that covers it, as an HTML link or HTTP header.
  • A clear purpose: agents read the file and follow the relevant links. The file stays small; details live behind the links.

Source:llmstxt.org: Changes in v2

04

llms.txt vs. robots.txt vs. sitemap.xml

llms.txt replaces neither robots.txt nor your sitemap; it complements both. The spec notes that a sitemap is no substitute: it’s usually too big for a language model and full of pages it doesn’t need.

Purpose

robots.txt
controls which crawlers may fetch what
sitemap.xml
lists all pages for search engines
llms.txt
curated overview for AI agents

Format

robots.txt
own syntax
sitemap.xml
XML
llms.txt
Markdown

Scope

robots.txt
rules
sitemap.xml
complete
llms.txt
a selection

Effect on Google

robots.txt
controls crawling
sitemap.xml
helps discovery
llms.txt
none

Sources:llmstxt.org: The /llms.txt file, v2 (modified Aug 10, 2026)Google Search Central: Optimizing for generative AI features (updated Jul 10, 2026)

05

What about llms-full.txt?

llms-full.txt isn’t part of the spec, but documentation platforms use it widely. It holds the full text of many pages in one file. On October 8, 2026, Anthropic’s documentation llms-full.txt was about 42 MB, OpenAI’s about 8.7 MB. For a business website that rarely makes sense; the short llms.txt is enough.

Sources:docs.claude.com/llms-full.txtdevelopers.openai.com/llms-full.txt

Reality check

Does llms.txt actually work? Not for Google, increasingly for agents.

Opinions range from “must-have for AI visibility” to “useless hype”. Neither is right. What matters is who reads the file: not Google Search, but more and more AI agents and developer tools.

01

Reasons not to overrate it

Three official statements temper expectations.

  • Google: you don’t need new machine-readable files to appear in Google Search. Creating an llms.txt for other services is completely fine and will neither harm nor help your visibility in Google Search, including AI Overviews and AI Mode.
  • OpenAI: its crawler documentation handles ChatGPT search through robots.txt rules for OAI-SearchBot. It doesn’t say ChatGPT search reads llms.txt.
  • Webflow calls the format experimental, not yet widely adopted, with unclear effects on AI search.

Sources:Google Search Central: Optimizing for generative AI features (updated Jul 10, 2026)OpenAI: Overview of OpenAI crawlersWebflow Help: Upload an llms.txt file

02

Reasons to create one anyway

The real audience is agents acting for people, and that use is growing.

  • Per the Chrome team, without the file an agent may spend more time figuring out a site’s structure and main content. Lighthouse checks that it loads without errors; if it’s missing, the audit is “not applicable”, because the file is optional for now.
  • Shopify generates an agent description at /agents.md for every store, and /llms.txt mirrors it by default.
  • AI companies use it themselves: OpenAI, Anthropic and Google publish llms.txt files for their developer docs.
  • The effort is small: about half an hour for a typical business site, and several platforms generate it automatically.

Sources:Chrome for Developers: Lighthouse llms.txt audit (updated May 5, 2026)Shopify docs: llms.txt.liquiddevelopers.openai.com/llms.txt

03

Our recommendation by site type

Our take, based on the sources above.

Software, SaaS, developer docs

Recommendation
create it, with Markdown versions
Why
coding agents already use these files

Shopify store

Recommendation
review the default, customize only if needed
Why
Shopify generates it; the goal is agent shopping

Services, local business, B2B

Recommendation
create it when you have 30 minutes
Why
little effort, no risk

Pure blog

Recommendation
low priority
Why
content and crawler access matter more

Examples

llms.txt examples: our file, a Shopify store, OpenAI’s docs.

Three files we retrieved on October 8, 2026 show how different an llms.txt can look: a company website, an online store and a developer documentation site.

01

Our own llms.txt on byclaire.co

This is how the file on byclaire.co starts and ends (shortened). It’s served as text/plain and runs to just over 2,000 words.

What works: the blockquote answers who we are; then come hard facts like location, supported systems and prices, exactly what people ask an AI about. Every link has a description, and languages are marked.

What we improved on October 8, 2026: the thank-you page is gone, it doesn’t help any agent. Instead of about 100 links in a single “Pages” section, there are now separate sections for product, use cases, industries, integrations, comparisons, free tools and guides, so an agent can skip what it doesn’t need. Still open: prices in the file have to be updated with every change.

# Claire by byclaire.co

> Your AI colleague for visibility in ChatGPT and Google. Claire
> finds where a brand is missing on Google and in AI answers
> (ChatGPT, Gemini, Perplexity, Claude), closes the gaps and
> shows the results.

Claire is an AI colleague for visibility in ChatGPT and Google,
built by wordsmattr GmbH (Vienna, Austria), not a blog writer
or text generator. […]

Plans start at €199 per month (Starter, about 30 articles
per month) and €299 per month (Growth, […]).

## Product

- [Home (English)](https://goclaire.org/): what Claire does,
  how it works, case studies, pricing, FAQ
[…]

## Integrations

- [Shopify SEO & blogging guide](https://goclaire.org/integrations/shopify):
  Shopify SEO basics, setting up a Shopify blog, […]
[…]

## Comparisons · ## Free tools and templates · ## Guides
[…]

## Legal

- [Imprint](https://goclaire.org/imprint): Imprint and contact details

Source:byclaire.co/llms.txt

02

A Shopify store: instructions for shopping agents

At allbirds.com/llms.txt the auto-generated file is titled “Agent Instructions”. It tells agents how to search products, build a cart and prepare checkout, points to the Universal Commerce Protocol (UCP) with its own endpoint, and states that payment requires the buyer’s explicit approval.

For stores, llms.txt is less a table of contents than a manual for AI shopping. More in our agentic commerce guide.

Sources:allbirds.com/llms.txtShopify docs: llms.txt.liquid

Guide: agentic commerce
03

OpenAI’s developer docs: one file per area

developers.openai.com/llms.txt is just an index pointing to separate llms.txt files per area, such as the API, Ads and Agentic Commerce. Every doc page is also available as Markdown by appending .md to the URL, exactly what version 2 of the spec describes.

Sources:developers.openai.com/llms.txtOpenAI: Overview of OpenAI crawlers

How to

How to create an llms.txt file: six steps.

For most sites, a file with 10 to 30 well-described links is enough. Our free generator drafts one from your sitemap; you do the final edit.

01

Step 1: Decide whom the file should help

List three to five questions customers would ask an AI assistant about you. For a heating contractor: Who installs heat pumps in my area? Roughly what does it cost? How do rebates work? The file should show the shortest path to those answers.

02

Step 2: Pick the key pages

Only pages that answer those questions: services, pricing, locations, about, your best guides, contact. Leave out what doesn’t help an agent: thank-you pages, logins, filter and search pages, internal URLs. The spec sets no number, but the file should stay small enough to fit into a language model’s context.

Source:llmstxt.org: The /llms.txt file, v2 (modified Aug 10, 2026)

03

Step 3: Write it

Plain, specific, no marketing speak. Webflow’s yardstick works well: write it as if you were briefing a company intern who has never seen your site. A template for a local business:

# [Company name]

> [Company name] is a [type of business] in [city/region].
> We [core service] for [audience].

Founded [year]. Service area: [cities/radius].
First consultation: [free/paid]. Contact: [phone], [email].

## Services

- [Service 1](https://[domain]/service-1): [one line: what exactly, for whom]
- [Service 2](https://[domain]/service-2): [one line]

## Pricing and process

- [Pricing](https://[domain]/pricing): [what it includes, as-of date]
- [Process](https://[domain]/process): [from inquiry to delivery]

## Guides

- [Most important guide](https://[domain]/guides/…): [which question it answers]

## Optional

- [About us](https://[domain]/about): [team, history]

Source:Webflow Help: Upload an llms.txt file

Free llms.txt generator
04

Step 4: Upload it

Depending on your platform, it’s one click or one upload to the root.

PlatformHow
WordPressTurn on the llms.txt feature in Yoast SEO. Yoast creates the file and updates it weekly, by default with your most recently updated content. Alternatively AIOSEO or an FTP upload.
ShopifyNothing to do; the file exists. To customize, add an agents.md.liquid template (applies to all three paths) or llms.txt.liquid (only /llms.txt) to your theme.
WebflowSite settings → SEO → LLMs.txt → Upload file (UTF-8, under 100 KB). It goes live on your custom domain after publishing.
WixWix generates the file for every site automatically.
Custom serversPlace the file in your site’s public root directory.
Shopify serves exactly one file at the primary domain, without locale or market subfolders.

Sources:Yoast: llms.txt functional specificationShopify changelog: Customize /llms.txt, /llms-full.txt and /agents.mdShopify docs: llms.txt.liquidWebflow Help: Upload an llms.txt filellmstxt.org: The /llms.txt file, v2 (modified Aug 10, 2026)

05

Step 5: Test it

Four checks, five minutes in total.

  1. Open yourdomain.com/llms.txt in a browser: you should see the text, not your homepage or an error page.
  2. Check the content type in your browser’s developer tools (Network): text/plain or text/markdown, not text/html.
  3. Run Lighthouse: its llms.txt audit flags server errors when fetching the file.
  4. Test with an agent: give an AI assistant only your llms.txt and ask about your business. If it answers correctly, the file does its job.

Sources:Chrome for Developers: Lighthouse llms.txt audit (updated May 5, 2026)llmstxt.org: The /llms.txt file, v2 (modified Aug 10, 2026)

06

Step 6: Keep it current

Set a fixed date, quarterly or whenever prices or services change. Outdated facts are worse than none, because an agent repeats them unchecked. Yoast and Shopify update automatically, but take a look now and then.

Mistakes

llms.txt best practices: six mistakes we see often.

Most mistakes are quick to fix. The most common one is invisible: the URL doesn’t return a file at all, but your homepage.

01

Six mistakes and how to avoid them

Check your file against this list once.

  1. HTML instead of text: single-page apps and some hosts return the homepage with status 200 for unknown URLs. Check the content type.
  2. Marketing speak instead of facts: “innovative solutions for every need” helps no agent. Say what you do, for whom, where and at what price.
  3. Pasting everything: llms.txt is a signpost, not a full-text archive.
  4. Internal URLs: staging domains, admin areas and confidential documents don’t belong in it.
  5. Wrong file name: it must be exactly llms.txt.
  6. Wrong expectations: if you wait for better Google rankings you’ll be disappointed, because Google doesn’t use the file.

Sources:Webflow Help: Upload an llms.txt fileGoogle Search Central: Optimizing for generative AI features (updated Jul 10, 2026)

Priorities

What matters more for AI visibility: access, text, content.

llms.txt is a small extra. Whether ChatGPT, Perplexity or Google’s AI Overviews mention you depends on other things.

01

Crawler access

For ChatGPT to show your pages in search, OAI-SearchBot must be allowed to fetch them; for Google, Googlebot. An llms.txt is useless if your robots.txt or CDN blocks these crawlers.

Source:OpenAI: Overview of OpenAI crawlers

Free AI crawler checker
02

Text without JavaScript, and content no one else has

Content that only appears after scripts run is invisible to many crawlers. And what gets cited is content with your own numbers, experience and examples, which Google calls the most important lever for AI answers.

Source:Google Search Central: Optimizing for generative AI features (updated Jul 10, 2026)

Blog: AI Overviews SEO

Questions

Good to know.

Is llms.txt an official standard?

No. It’s an open proposal by Jeremy Howard, published on llmstxt.org in 2024 and revised as version 2 in August 2026. No standards body has adopted it. Shopify, Yoast and Wix still support it, and OpenAI, Anthropic and Google use it for their developer docs.

Does ChatGPT read llms.txt?

That isn’t officially confirmed. OpenAI’s crawler documentation handles ChatGPT search via robots.txt rules for OAI-SearchBot and doesn’t mention llms.txt for it. To be found in ChatGPT’s web search, what matters most is that OAI-SearchBot can crawl your site.

Is llms.txt useful for SEO?

Not for Google. Google states that an llms.txt neither harms nor helps visibility in Google Search, because Search ignores it. Its value lies with AI agents and developer tools that read it.

What should llms.txt contain?

An H1 with your site’s name, a short blockquote summary, key facts in plain text, and sections with links to the pages that answer your customers’ most common questions, each with a one-line description. Secondary pages go into an “Optional” section.

What’s the difference between llms.txt and robots.txt?

robots.txt controls which crawlers may fetch which parts of your site. llms.txt explains to AI agents what your site is about and where the key information is. One is an access rule, the other a signpost. You need robots.txt either way; llms.txt is optional.

How do I generate llms-full.txt, and do I need it?

llms-full.txt bundles the full text of many pages and is common on documentation platforms; Shopify serves one for stores, and docs tools generate it from your pages. It isn’t part of the spec and can get huge (Anthropic’s was about 42 MB on October 8, 2026). Business sites rarely need it.

Do I need Markdown versions of my pages?

No, they’re optional. Version 2 recommends them for pages agents use often, especially technical documentation. Webflow considers well-structured HTML pages linked from llms.txt sufficient for now.

Is there a free llms.txt generator?

Yes. Yoast SEO and AIOSEO create the file in WordPress, Wix and Shopify generate it automatically, and Claire has a free generator that drafts one from your sitemap and page titles. Review any generated file yourself: a generator doesn’t know your key pages as well as you do.

About this guide

Author, method and sources

Written by the Claire team at wordsmattr GmbH in Vienna. Claire is our own product, your AI colleague for visibility in ChatGPT and Google: she finds where you’re missing, closes the gaps and shows you the results. First published on , last checked against the sources on . We review this guide every quarter and whenever a provider changes its documentation.

How we researched

  • Search demand: Google Ads via DataForSEO, United States, monthly average, retrieved Oct 5, 2026 (llms.txt 5,400, llms txt file 880, what is llms txt 480, llms txt generator 480, llms txt example 170).
  • Search results: Google top 10 with “People also ask” for “llms.txt” in the U.S. and Germany, retrieved Oct 5 and Oct 8, 2026.
  • Facts: only the spec on llmstxt.org and documentation from Google, Chrome, OpenAI, Shopify, Webflow and Yoast, all checked again on Oct 8, 2026.
  • Examples: the files on byclaire.co, allbirds.com and developers.openai.com and the size of Anthropic’s and OpenAI’s llms-full.txt, retrieved Oct 8, 2026.

Sources, all retrieved on October 8, 2026

  1. llmstxt.org: The /llms.txt file, v2 (modified Aug 10, 2026)
  2. llmstxt.org: Changes in v2
  3. Google Search Central: Optimizing for generative AI features (updated Jul 10, 2026)
  4. Chrome for Developers: Lighthouse llms.txt audit (updated May 5, 2026)
  5. Shopify docs: llms.txt.liquid
  6. Shopify changelog: Customize /llms.txt, /llms-full.txt and /agents.md
  7. Webflow Help: Upload an llms.txt file
  8. Yoast: llms.txt functional specification
  9. OpenAI: Overview of OpenAI crawlers
  10. byclaire.co/llms.txt
  11. allbirds.com/llms.txt
  12. developers.openai.com/llms.txt
  13. docs.claude.com/llms-full.txt (about 42 MB)
  14. developers.openai.com/llms-full.txt (about 8.7 MB)

More guides

Keep reading. Same rules: answer first, sources dated.

All guides

Your llms.txt in two minutes. Free, drafted from your sitemap.

Claire’s free llms.txt generator reads your sitemap and page titles and drafts a file that follows llmstxt.org. Edit titles, descriptions and order right in your browser, then download it.

  • Draft that follows llmstxt.org
  • Edit in the browser
  • Free, no email
Create your llms.txt