llms.txt Generator

Give AI a map of your website

Paste a URL and get a spec-compliant llms.txt— a curated, LLM-friendly guide to your site's pages — in under a minute.

Works on most public websites. Generation typically takes 10–60 seconds.

Options

Crawl depth

Pages to crawlStop after visiting this many pages.50

No limitCrawl the entire site, however large — ignores the slider above.

Crawl linked pagesFollow links from your homepage to discover more pages (BFS).AI-enhanced descriptionsUse an LLM via OpenRouter to write richer page summaries.Unblock protected sitesEscalate to a real browser when a page blocks simple requests.Honor robots.txtRespect the site's crawler rules (recommended).Auto-updateRe-check this site on a schedule and refresh the hosted file when its content changes. Checks run nightly at midnight UTC. Leave off to generate once.

How it works

1Paste your URL

Drop in your website's homepage — no setup or account needed.

2We crawl & summarize

We visit your pages and write a short, clear description of each one.

3Download your llms.txt

Get a spec-compliant file ready to drop at the root of your site.