Google’s Search and Chrome documentation now level in several instructions on llms.txt, relying on whether or not the aim is Search visibility or agentic browser readiness.
Google Search not too long ago revealed a brand new optimization information that lists llms.txt among the many techniques you don’t want for generative AI options. The information teams it with content material chunking, AI-specific rewriting, and particular schema.
Days earlier, Google’s Lighthouse instrument shipped model 13.3, which added a brand new Agentic Searching class. The replace contains an llms.txt audit that checks whether or not a web site gives the file and flags server errors when retrieving it.
The Lighthouse documentation describes llms.txt as a method to offer “a machine-readable abstract of an internet site’s content material, particularly designed for LLMs and AI brokers.” It provides that with out the file, “brokers might spend extra time crawling the positioning to grasp its high-level construction and first content material.”
What Google Search Has Mentioned
Google’s Search workforce has maintained for over a yr that llms.txt isn’t a Google initiative or one thing Google plans to undertake.
John Mueller in contrast llms.txt to the key phrases meta tag, noting no AI providers used it and bots didn’t request the file. He referred to as constructing separate Markdown pages for bots “a silly concept.”
At Search Central Reside Deep Dive Asia Pacific, Gary Illyes and Amir Taboul confirmed Google was not pursuing llms.txt.
Google’s optimization information explicitly states llms.txt ought to be skipped, offering the latest direct assertion from the Search workforce.
What Chrome’s Lighthouse Now Does
Lighthouse 13.3 ships with the Agentic Searching class by default, checking WebMCP integration, agent accessibility, format stability, and llms.txt.
The llms.txt audit solely marks websites as “Not Relevant” in the event that they return a 404; errors flag the audit. The Lighthouse docs describe llms.txt as an “rising conference” at llmstxt.org, advising web site homeowners to create and place it of their root listing.
This class is separate from search engine marketing audits and signifies that llms.txt helps browser-based brokers perceive web site construction, not enhance search rankings or AI citations.
Google Has Been Right here Earlier than
Google’s inner groups have despatched combined indicators on llms.txt earlier than.
In December, Lidia Infante noticed an llms.txt file on Google’s Search Central developer documentation. Mueller responded on Bluesky with “hmmn :-/” and didn’t make clear additional.
Dave Sensible famous that the file appeared on a number of Google developer properties, together with developer.chrome.com and internet.dev. The sample prompt an inner CMS platform replace that mechanically deploys llms.txt information, not a Search workforce resolution.
The Search Central file was eliminated inside hours, however information on different Google properties remained.
Why This Issues
Google’s reply on llms.txt varies by use case.
For Google Search, llms.txt isn’t wanted for AI Overviews, AI Mode, or different generative AI Search options.
For browser-based brokers, Lighthouse considers llms.txt non-compulsory in an experimental machine interplay class.
Steering is cut up between completely different Google developer websites, which might result in conflicting directions when evaluating Lighthouse or its llms.txt documentation with Google’s Search docs.
Wanting Forward
Google hasn’t commented on the documentation hole between the 2 product groups.
For a lot of websites, making a primary llms.txt file is easy, however sustaining it’s questionable, provided that Google Search states it’s pointless for AI Search visibility.
Featured Picture: Inventory-Asso/Shutterstock
