Some AI bots request llms.txt, but those visits do not prove they use its contents in an answer. For your business, the useful question is whether maintaining the file deserves attention ahead of other website improvements. An llms.txt file provides a brief website summary and links to selected content, as we explain in our guide to llms.txt. Understanding what bot visits actually show can help you set more practical AI search priorities.
How Often Do AI Bots Request llms.txt?
Published research suggests these requests are limited. In Ahrefs’ June 2026 study of 137,210 domains with traffic that May, 97 percent of roughly 38,000 existing llms.txt files received no requests. In a separate two month study published by Evil Martians, just 37 of approximately 770 combined requests to its llms.txt and llms-full.txt files came from named AI clients.
Those findings cover different populations and cannot predict your website’s traffic. Ahrefs examined its analytics customers, while Evil Martians measured only its own website. Both give you a reason to ask for evidence before making the file a major marketing investment.
Which Bots Are Visiting and Why Does It Matter?
Visitors can include AI training crawlers, coding agents, search tools, and ordinary website scanners. Ahrefs identified GPTBot and Claude Code among prominent named visitors. However, named AI categories accounted for only 19.5 percent of requests to files that received traffic in its May sample.
If your developer reports llms.txt activity, ask which clients requested it and whether the requests succeeded. A website audit tool checking that the file exists represents a different activity from an AI assistant retrieving information. Separating those visits makes your reporting more useful than a single total labeled “AI traffic.”
What Can Your Server Logs Actually Tell You?
Server logs show requests and responses, not an AI system’s reasoning. A successful fetch confirms that your server delivered the file. It does not establish that an assistant followed its links, relied on its descriptions, or cited your business.
For example, ten successful downloads are evidence of access, not ten recommendations to potential customers. Even a later visit to a listed page would not, by itself, prove inclusion in an answer. Treat request counts as technical observations and assess customer inquiries separately.
Where Should You Focus Your AI Search Work?
Start with the pages you want prospective customers to find and trust. Google’s published guidance says Google Search, including its generative AI features, does not use llms.txt and continues to rely on foundational SEO. That statement applies specifically to Google Search.
Review whether your service pages explain what you offer and answer common questions. Our content gap analysis guide can help you identify questions your existing pages leave unanswered. Add internal links that help readers move from an explanation to a relevant service or next step. These improvements belong within a broader SEO and digital marketing strategy, with priorities tied to your business goals.
Bot requests provide useful context, but they do not establish a marketing return. We recommend keeping llms.txt maintenance secondary to useful content, accessible pages, and clear navigation. At Boston Web Marketing, our team brings SEO experience to evaluating those priorities. Contact us to review your website and identify your next improvements.