llms.txt Validator

llms.txt is a community convention (documented at llmstxt.org) for a Markdown file that gives AI systems a curated overview of your site—who you are, what matters, and which pages or docs to read next. It sits at your domain root (https://yoursite.com/llms.txt), similar to how robots.txt and sitemap.xml live at well-known paths. Unlike a full HTML crawl, it is meant to be concise and LLM-friendly so assistants and agents can load useful context without parsing navigation, ads, and scripts.

As more people discover brands through ChatGPT, Claude, Perplexity, and similar tools, sites need a clear, machine-readable summary AI can use on demand—not only pages built for classic Google crawlers. llms.txt complements sitemaps by highlighting the handful of URLs and notes that actually fit an LLM context window. It is especially useful for documentation, APIs, and product sites where coding agents and assistants need a fast map of authoritative content. It is not a guaranteed ranking or citation boost, but it is a low-effort signal of AI readiness as the agentic web grows.

Per the llmstxt.org convention, start with a single H1 for the project or site name (the only required section), then a blockquote summary with the key context an LLM needs. Optional paragraphs can follow, then H2 sections that hold Markdown link lists: each item is a [name](url) link, optionally followed by a colon and a short note. A special H2 named "Optional" marks secondary links that may be skipped when a shorter context is enough. Keep language concise and prefer absolute or clearly resolvable URLs to markdown-friendly pages when you have them.

Adoption is uneven and still evolving. Major AI search surfaces do not publicly treat llms.txt as a proven ranking or citation factor the way Google treats sitemaps, and many crawler-log studies show limited fetching of /llms.txt by named bots. Where it shows up more clearly is in documentation and agent workflows—tools and assistants that fetch a URL on demand can load llms.txt (or linked .md pages) as context. Treat it as helpful infrastructure for AI discovery, not as a substitute for strong content, crawlability, and traditional SEO.

robots.txt tells crawlers what they may or may not fetch; llms.txt does not grant or deny access—it curates guidance and links for LLMs and agents when they need to understand your site. Publish llms.txt at the site root as plain text/Markdown, not buried in a subfolder or CMS-only path crawlers never see. Common mistakes include missing the H1, skipping the blockquote summary, using broken or relative links without a clear base, stuffing the entire sitemap instead of a curated set, and expecting the file alone to improve AI citations without clear, high-quality destination pages.