How to Create an llms.txt File for Your Website

How to Create an llms.txt File for Your Website

To create an llms.txt file for your website, you must write a clean markdown document named llms.txt, structure it with standard headings outlining your site’s primary categories and API endpoints, and host it at the root directory of your domain (e.g., yourdomain.com/llms.txt). This file acts as a high-density, machine-readable summary that helps AI search agents and LLM web crawlers understand your site architecture and content without consuming high server bandwidth parsing bloated HTML pages.

—

[Visual: A file explorer window showing the root directory of a site. A developer right-clicks and creates a new file called 'llms.txt'.]

Speaker: "Forget robots.txt. In 2026, the most important file on your server is llms.txt."


[Visual: Side-by-side comparison. Left: A giant wall of messy HTML. Right: A beautifully clean markdown file summarizing the site.]


Speaker: "It's a simple markdown file placed at your domain root that tells AI crawlers exactly what your site does and where your main endpoints are."


[Visual: A PageSpeed Insights screen shows a green checkmark next to "llms.txt found".]


Speaker: "Google's new Agentic Browsing metric requires this file to give you a passing grade. If you want to appear in AI search summaries, you need one today. Link in the description for a template!"

—

Why AI Agents Require an llms.txt File

Traditional web scraping is extremely expensive for AI companies and highly inefficient for website servers. When an AI search engine like ChatGPT or Gemini wants to understand your site, it has to download and parse entire pages of HTML, CSS, and Javascript.

An llms.txt file acts as a “cheat sheet” for the web agent. It uses standard markdown to present a concise summary of the website, key links, search endpoints, and structural directories.

Standard Structure of an llms.txt File

Your llms.txt should be written in clean, concise markdown. It is typically divided into three primary sections:

  • Brand Summary: A single-sentence declaration of what the website or company is.
  • Main Navigation / Resource Links: A list of primary page paths and descriptions.
  • Optional Full Index Reference: A link pointing to a secondary, larger index file named llms-full.txt which contains more detailed documentation if needed.

Here is a standard example template you can copy:

# Brand Name / Project Title

> Brief description of the website, target audience, and primary functionality.


Core Pages


    • [Pricing](https://yourdomain.com/pricing): Detailed subscription pricing and plan features.


    • [Docs](https://yourdomain.com/docs): Full developer guides and setup documentation.


API Endpoints


    • [Product Search API](https://yourdomain.com/api/search): Endpoint query format ?q=search_term.

How to Deploy Your llms.txt

  • Save the File: Save the text file with the name llms.txt. Do not name it .md or .html.
  • Upload to Root: Upload the file directly to your website’s public root directory (the same folder that hosts your robots.txt and .htaccess files).
  • Verify Access: Open a browser tab and navigate to https://yourdomain.com/llms.txt. Ensure the content loads as plain text.
  • Set Up Redirection (Optional): If you are running a complex React or Next.js app, configure a rewrite rule so that requests to /llms.txt serve the public asset correctly.