Google’s John Mueller says markdown pages serve a selected goal for developer documentation websites however received’t assist most web sites, at the same time as search turns into extra agentic.
Mueller laid out his reasoning in a Bluesky thread. He was responding to a query from Lily Ray about why Google publishes LLMs.txt recordsdata and markdown pages, though they aren’t wanted for search efficiency.
His response targeted primarily on markdown variations of developer documentation, not llms.txt as a standalone file.
Mueller wrote:
“The quick reply is that it’s not accomplished for search. There’s extra to web sites than simply search engine optimisation :-).”
Mueller’s Discovery Vs. Performance Framework
His reasoning focuses on two totally different web site targets.
He known as the primary “discovery,” or being discovered through a search engine, and the second “performance,” which helps customers full duties on the web page.
Mueller acknowledged the time period wasn’t exact. “There’s most likely a extra correct time period for this,” he wrote within the thread.
He in contrast the excellence to calls to motion on conventional pages, stating:
“You don’t ‘do them’ for search engine optimisation (to be discovered), however in case you’re answerable for the web site general, making certain a excessive ‘discovery fee’ (search engine optimisation) along with a excessive conversion fee is helpful to justify your work.”
Why Developer Docs Are Totally different
On builders.google.com, he famous, markdown variations make sense.
Mueller stated;
“AI coding has gotten extremely popular, and these coding methods might be (I feel) environment friendly and correct with the code they produce if they’ll simply learn / parse reference materials, equivalent to developer documentation.”
He added that markdown will help AI methods “perceive the context of the documentation they’re , in addition to a simplified model of the reference web page.”
Mueller known as this a workaround slightly than a long-term want, including:
“OF COURSE they’ll learn HTML simply positive, so that is imo extra of a short lived crutch, maybe to avoid wasting tokens.”
Non-Developer Websites Ought to Skip It
For everybody else, Mueller was direct, stating:
“For non-developer websites, I don’t assume this makes a lot sense, even with extra agentic visitors sooner or later. Making a markdown model of a shoe’s specs isn’t going to get you extra gross sales (rivals recognize it tho).”
He went additional in a follow-up publish, pushing again on the concept websites ought to put together for a future the place brokers drive extra visitors.
Mueller added:
“And (I do know, no one reads this far), in case you assume that is necessary to organize for when brokers are all over the place: your web site (all websites) have way more necessary issues to do for search engine optimisation than to organize for a possible future state of affairs which will or could not come. Prioritize wants earlier than desires.”
Why This Issues
Mueller’s feedback present a extra detailed place than his earlier statements on the subject.
In February, Mueller known as the thought of serving markdown pages to bots “a silly thought.” His Bluesky feedback carve out an exception for developer documentation whereas holding the road for each different sort of web site.
The thread additionally arrived on the identical day we reported that Google’s steering on llms.txt now will depend on which product you ask. Google’s generative AI optimization information says to skip llms.txt, whereas Lighthouse 13.3 added an experimental audit that checks for the file as a part of agentic shopping readiness.
Trying Forward
Mueller’s distinction between discovery and on-page performance will help you consider whether or not agentic optimization is value their time. The take a look at is whether or not constructing for brokers proper now produces measurable outcomes for a selected web site.
The “prioritize wants earlier than desires” line captures a broader rigidity within the trade proper now. Distributors have been selling llms.txt and markdown optimization as rising practices, however neither Google’s search documentation nor unbiased information help investing in these for non-developer websites.
Featured Picture: kirill_makarov/Shutterstock
