A familiar idea with a new name
If you have ever poked around the back of a website, you have met files like this before. There is robots.txt, which tells search crawlers what they are allowed to visit. There is sitemap.xml, which hands them a map of every page worth indexing. Both sit quietly at the root of a site and speak to machines, not to people.
llms.txt is a proposed file in that same family, except its intended reader is a language model rather than a search crawler. It lives at the root too, at yourdomain.com/llms.txt, where any system that knows to look can find it. The idea is simple: give the model a clean, deliberate starting point instead of leaving it to piece your site together on its own.
What llms.txt actually is
At heart it is a plain markdown file (a simple text format that uses plain symbols for headings and links) that curates your site for a model. It opens with a title and a short summary of what you do, then lays out a tidy list of your most important pages, each with a one line description and a link. Think of it as a guided tour you write yourself, so the model does not have to guess your priorities from the sprawl of your whole site.
You may also see a second file mentioned, llms-full.txt. That one goes further: instead of just linking to your key pages, it includes the actual text of those pages inline, so a model can read the content directly without following links. It is the fuller, heavier version of the same idea, useful mainly for documentation heavy sites.
What it is not
It is worth being honest about the limits, because the hype tends to skip them. A few things llms.txt is not:
- It is not an official web standard yet. It is a proposal that has gained traction, not a rule that browsers or search engines are bound to follow.
- It is not read by everything. Adoption across AI systems is uneven, and some models may ignore the file entirely. You cannot assume it is being consumed just because it exists.
- It is not a ranking trick. There is no evidence that adding one boosts where you appear, and it does not replace the slow work of being genuinely useful and findable.
- It is not a shortcut past the basics. Anyone promising that a single file will make AI suddenly recommend you is overselling it.
Why it can still be worth it
None of that makes it pointless. The case for llms.txt is modest but real. It is cheap to add: a single text file you write once and rarely touch again. It costs almost nothing and breaks nothing.
It also has a quiet side benefit. Writing one forces you to state plainly what your most important pages are and to describe each in one clear line. That clarity is good for you regardless of whether a model ever reads it. And if adoption does grow, you already have a clean, machine readable summary of your site sitting where systems expect to find it. Low risk, possible upside, a little future proofing. For most sites that is a fair trade.
What a good one looks like
A solid llms.txt is short and disciplined. It starts with a clear H1 title (the main top-level heading, holding your product or company name), followed by a one paragraph summary of what you do, often written as a blockquote (an indented, set-off line) so it stands out. After that come grouped sections, each a small heading with a few links underneath.
The grouping is where it earns its keep. Rather than dumping every URL, you sort your pages into sensible buckets, for example:
- Docs: your guides, references and how-to pages, each with a short description.
- Product: your core feature and pricing pages, the ones that explain what you sell.
- Guides: your best long form articles, the content you would actually want quoted.
Plain language, no fluff
Two habits make the difference between a useful file and a wasted one. First, write the descriptions in plain language. This is not the place for marketing copy or clever taglines. A model wants to know what a page is, not how excited you are about it.
Second, point only to pages that genuinely help someone understand your site. A short list of your best, clearest pages beats an exhaustive index of everything you have ever published. If a page would not help a human get oriented, it does not belong in the file either.
So, should you bother?
Yes, if it is cheap and your fundamentals are already in order. A clean llms.txt is a sensible thing to have once the important parts of your site are working: crawlers can reach you, pages load quickly, and every page that matters says clearly what it is. In that situation, adding one is a small, tidy finishing touch.
It is not a priority if those fundamentals are shaky. A polished llms.txt on a site that blocks crawlers, loads slowly, or never says plainly what you do is effort spent in the wrong order. Foundation first, then the thin extras on top. That sequence matters more than the file itself.
If you are not sure which camp you are in, HowFindable scans your site for free and shows you where you stand on the basics as well as the AI specific signals. If you want the fixes, a sensible llms.txt is one of the files we generate for you, ready to upload. But the file is the easy part. Getting the order right is the point.