geo llms Command: Generate /llms.txt From Sitemap
Generating llms.txt
Section titled “Generating llms.txt”geo llms auto-generates an /llms.txt file from your sitemap so AI crawlers can understand your site’s structure.
What is llms.txt?
Section titled “What is llms.txt?”llms.txt is a plain-text Markdown file placed at the root of your website (https://yoursite.com/llms.txt). It works like robots.txt but for AI language models: instead of telling crawlers what to block, it tells them what to read and how to understand your site.
Spec: llmstxt.org
A well-structured llms.txt helps AI crawlers:
- Understand what your site is about before crawling individual pages
- Discover your most important pages quickly
- Categorize your content into logical sections
- Extract a concise site summary for use in AI-generated answers
Why It Matters
Section titled “Why It Matters”AI crawlers (Perplexity, Claude, ChatGPT Search) use llms.txt as a trust and structure signal. Sites without it are treated as generic; sites with it are indexed more accurately. When a user asks an AI a question your site answers, having a well-structured llms.txt increases the probability of being cited.
# Minimal — auto-detects sitemap from robots.txtgeo llms --base-url https://yoursite.com
# Full optionsgeo llms \ --base-url https://yoursite.com \ --site-name "MySite" \ --description "Free online calculators for finance and math" \ --output ./public/llms.txt \ --sitemap https://yoursite.com/sitemap.xml \ --max-per-section 10 \ --fetch-titlesAll Flags
Section titled “All Flags”| Flag | Default | Description |
|---|---|---|
--base-url |
(required) | Root URL of the site (e.g. https://yoursite.com) |
--output |
./llms.txt |
Output file path |
--sitemap |
auto-detected | Direct URL to the sitemap XML; skips auto-detection |
--site-name |
derived from URL | Human-readable name for the site |
--description |
none | Short description of the site (1–2 sentences) |
--max-per-section |
20 |
Max URLs per content section |
--fetch-titles |
false | Fetch each URL to extract its <title> tag (slower but richer output) |
--check-drift |
false | Check an existing llms.txt against the current sitemap instead of generating one; exits 1 on stale URLs |
--llms-file |
fetched live | Local llms.txt to check with --check-drift, instead of fetching {base-url}/llms.txt |
Checking for drift
Section titled “Checking for drift”llms.txt is generated once and then trusted by AI agents as a map of the site, and nothing checks
whether that map is still accurate as pages get renamed or removed. --check-drift compares the
URLs listed in llms.txt against the site’s current sitemap (reusing the sitemap fetch geo llms
already does for generation, so it costs no extra per-link HTTP request):
geo llms --base-url https://yoursite.com --check-drift- Stale URLs: listed in llms.txt, gone from the sitemap. These actively hurt citability: an
agent following the link hits a 404 or redirect instead of the content it was told to expect.
Presence of any stale URL exits with code 1, so this is safe to run in CI. The command also exits with code 1 when llms.txt,
the
--llms-filepath or the sitemap cannot be read. - Missing URLs: in the sitemap, not yet in llms.txt. This means llms.txt is behind, not wrong.
Regenerate it with
geo llms --base-url ... --output ....
Sitemap Auto-Detection
Section titled “Sitemap Auto-Detection”When --sitemap is not specified, the script:
- Fetches
https://yoursite.com/robots.txt - Looks for a
Sitemap:directive - If found, uses that URL as the sitemap
- If not found, tries
https://yoursite.com/sitemap.xmlas a fallback - If the sitemap is a sitemap index (contains multiple sitemaps), it fetches and merges all of them
If your sitemap is in a non-standard location, always pass --sitemap explicitly.
Structure of the Generated File
Section titled “Structure of the Generated File”The script generates a structured Markdown file. Example output:
# MySite
> Free online calculators for finance, math, and health. Accurate, fast, no signup required.
## Finance Tools
- [Mortgage Calculator](https://yoursite.com/finance/mortgage): Calculate monthly payments for fixed and adjustable-rate mortgages- [Compound Interest Calculator](https://yoursite.com/finance/compound-interest): Compute growth over time with compound interest- [Loan Amortization](https://yoursite.com/finance/amortization): Full amortization schedule for any loan
## Math
- [Percentage Calculator](https://yoursite.com/math/percentage): Calculate percentages, increases, and decreases- [Fraction Simplifier](https://yoursite.com/math/fractions): Reduce fractions to their simplest form
## Blog & Articles
- [What is GEO?](https://yoursite.com/blog/what-is-geo): Introduction to Generative Engine Optimization- [robots.txt for AI Bots](https://yoursite.com/blog/robots-txt-ai): Complete guide to configuring AI crawlers
## Optional
- [About](https://yoursite.com/about)- [Contact](https://yoursite.com/contact)- [Privacy Policy](https://yoursite.com/privacy)Structure breakdown:
# H1— your site name> blockquote— your site description (parsed by AI as the summary)## H2 sections— content categories (auto-detected from URL patterns)- [Title](URL): description— links with optional descriptions- Links without descriptions go under
## Optional
Where to Put the File
Section titled “Where to Put the File”Place llms.txt at the root of your web server so it’s accessible at https://yoursite.com/llms.txt.
Static Sites (Astro, Next.js, plain HTML)
Section titled “Static Sites (Astro, Next.js, plain HTML)”# Astrogeo llms --base-url https://yoursite.com --output ./public/llms.txt
# Next.js (app router)geo llms --base-url https://yoursite.com --output ./public/llms.txt
# Plain HTMLgeo llms --base-url https://yoursite.com --output ./llms.txtAfter running: deploy as you normally would. The file will be served at /llms.txt.
WordPress
Section titled “WordPress”# Generate the file locallygeo llms \ --base-url https://yoursite.com \ --sitemap https://yoursite.com/sitemap.xml \ --output llms.txt
# Upload to WordPress root (where wp-config.php lives)scp llms.txt user@yourserver.com:/var/www/html/llms.txtOr use a plugin like “WP Robots Txt” to serve a custom llms.txt via code.
Generic Site (any server)
Section titled “Generic Site (any server)”# Generategeo llms --base-url https://yoursite.com --output llms.txt
# Upload via FTP/SCP to the web rootscp llms.txt user@yourserver.com:/var/www/yoursite.com/llms.txtVerify It’s Live
Section titled “Verify It’s Live”curl https://yoursite.com/llms.txtExpected: Markdown content starting with # YourSiteName.
If you get a 404, the file is not in the right location. Check your web root directory.
Example for Specific Frameworks
Section titled “Example for Specific Frameworks”Astro — with custom sitemap
Section titled “Astro — with custom sitemap”geo llms \ --base-url https://yoursite.com \ --site-name "MySite" \ --description "Free tools and calculators" \ --sitemap https://yoursite.com/sitemap-index.xml \ --output ./public/llms.txt \ --max-per-section 15 \ --fetch-titlesNext.js — standard setup
Section titled “Next.js — standard setup”geo llms \ --base-url https://yoursite.com \ --site-name "MySite" \ --output ./public/llms.txtWordPress — with Yoast SEO sitemap
Section titled “WordPress — with Yoast SEO sitemap”geo llms \ --base-url https://yoursite.com \ --sitemap https://yoursite.com/sitemap_index.xml \ --site-name "MySite" \ --description "Your WordPress site description" \ --output llms.txtGeneric site — no sitemap
Section titled “Generic site — no sitemap”If you have no sitemap, create a simple llms.txt manually following the structure above, or generate a sitemap first (many free tools exist online), then pass it with --sitemap.