llms.txt Checker
Validate whether your site exposes an llms.txt file and gives AI agents a clear guide to your most useful resources.
An llms.txt checker verifies whether a site provides an AI-readable guide file and whether that file points agents toward important pages, docs, and Markdown resources.
The llms.txt checker tests the conventional root path, response status and content type, required Markdown structure, linked public resources, and related discovery signals. llms.txt is a voluntary proposal: passing the check does not guarantee that a crawler reads the file or that an AI system cites the site.
llms.txt checker results explained
| Check | Pass condition | Common failure |
|---|---|---|
| Root retrieval | Canonical `/llms.txt` returns a public successful response | 404, login, preview host or redirect loop |
| Markdown structure | One clear H1 with concise context and organized links | HTML error page or unstructured URL dump |
| Linked resources | Canonical, public, descriptive and useful destinations | Broken, private, duplicated or stale links |
| Scope | Curated high-value resources | Copying the entire sitemap |
| Maintenance | Owner and retest process exist | File becomes stale after launches or migrations |
Check your website's AI discoverability signals.
Run a free scan for robots.txt, sitemap discovery, Link headers, Markdown readiness, and AI bot access.
Detailed guide
The first check is retrieval. Request `/llms.txt` on the canonical HTTPS host and record the final URL, status code, redirects, content type, and body. A file that exists only on a preview domain, requires cookies, returns an HTML error template, or redirects through authentication is not a reliable public guide.
The current proposal uses Markdown. A useful file begins with one H1 naming the project or site, may include a short blockquote summary and optional details, then organizes curated links under H2 sections. The point is a concise map of valuable resources, not a second sitemap containing every URL.
Every linked resource should be public, canonical, current, and genuinely useful to an agent or reader. Check for 404s, redirect chains, duplicate URLs, private dashboards, stale documentation, and links whose labels do not describe the destination. Never expose account, billing, API, webhook, staging, or user-data routes.
Validate llms.txt independently from robots.txt and sitemap.xml. Robots rules express crawler access policy; sitemaps support URL discovery; llms.txt curates context. A site can pass one and fail another. Link headers and Markdown alternates are also separate optional mechanisms and should be reported separately.
Treat the file as maintained documentation. Assign an owner, update it when canonical docs or product facts change, and retest the links after migrations. The best outcome is not a green badge; it is a small, accurate guide that continues to point at the strongest public sources.
What this checker analyzes
- llms.txt availability
- links to key pages or documentation
- relationship with llms-index.md
- Link header exposure
- machine-readable descriptions
Why it matters
llms.txt can help AI agents understand which resources matter. It is not a ranking guarantee, but it can improve machine guidance.
Common issues
- No llms.txt file
- File exists but has no useful links
- Outdated links
- No Markdown index
- No Link header pointing to the file
How to use this checker
Start with a live scan of your website URL. Review the status of each signal, then fix the highest-impact blockers first. Technical blockers should usually be handled before content optimization because AI systems need access before they can evaluate page quality.
- Request the canonical `/llms.txt` URL.
- Verify status, final URL, content type, and visible body.
- Check the H1, summary, optional details, and H2 link groups.
- Resolve every linked URL and remove private or duplicate destinations.
- Compare the file with the sitemap, robots policy, and canonical documentation.
- Assign an owner and retest after URL or product changes.
What a strong result looks like
A strong result means important pages are crawlable, key resources are discoverable, and the content gives AI systems enough structure to understand the topic quickly.
- robots.txt allows the crawlers you want to support.
- sitemap.xml exposes important URLs.
- headers or HTML links point to useful AI-readable resources.
- content includes concise answers and supporting context.
Who should use it
This checker is useful for founders, marketers, SEO teams, developers, agencies, and technical content teams that want to improve AI search readiness without guessing.
It is especially useful before launching new landing pages, documentation, product pages, comparison pages, or AI visibility campaigns.
Implementation checklist
- Confirm the page returns a successful HTTP status.
- Confirm the page is not blocked by robots.txt.
- Make sure the page appears in your sitemap or is internally linked.
- Add direct answer content for the primary user question.
- Add related links to nearby AEO, GEO, llms.txt, or AI crawler topics.
- Document technical changes so they can be repeated across the site.
How Pro helps fix it
Pro provides llms.txt guidance, generator-style prompts, and implementation steps for connecting llms.txt with your content structure.
FAQ
What is llms.txt?
llms.txt is a proposed text file that gives AI systems a concise guide to important website resources.
Is llms.txt required?
No. It is optional, but it can help provide clearer machine-readable guidance.
Where should llms.txt be placed?
Usually at the site root, such as example.com/llms.txt.
Sources and methodology
The validation order follows the public llms.txt proposal: retrieve the conventional root file, inspect Markdown structure, resolve linked resources, check safe scope, and record limitations. The specification was reviewed on 2026-08-16 and remains a voluntary proposal rather than a ranking requirement.
These references support the changeable facts and study findings discussed above. Results depend on each source's sample, date, market, query set, and measurement method.