The previous couple of weeks have been noisy. Google shipped one thing known as the Open Data Format. Then Google Builders introduced the Agentic Useful resource Discovery (ARD) specification.
In the meantime, each search engine optimization LinkedIn feed is lit up with somebody both declaring markdown the way forward for the net or explaining why it is best to ignore all of it.
The reality, as per traditional, sits someplace extra attention-grabbing than both camp.
The net is growing a parallel machine-readable infrastructure (MCP/WebMCP, OKF, ARD, LLMs.txt…) and SEOs who perceive what every layer really does, slightly than treating all of it as “AI search engine optimization” or a silver bullet, will make higher selections about the place to spend their time.
First: The Layer Cake
There are at the least six distinct issues being mentioned below the umbrella of “making your website AI-ready.” They sit at completely different layers and serve completely different functions:
- Crawlable HTML Pages: Nonetheless the muse. Nothing has modified right here. All the pieces else sits on high.
- Schema.org/Structured Knowledge: Semantic hints baked into HTML that inform machines explicitly what a web page is about. It’s, in essence, a vocabulary.
- LLMs.txt: Primarily a navigation file. Its function is to primarily inform an AI agent that’s already in your website which pages matter. However as John Mueller places it on the Search Off the File podcast:
“If somebody is already in your web site, possibly some sort of automated system is useful. The place if it goes, I need to go to Martin’s Splitt and purchase {a photograph}, then the LLM system can go to your web site and might go searching, like, how do you purchase {a photograph}? Possibly he has some pointers for me as an agent for getting pictures. That sort of is smart.”
- MCP/WebMCP: Earlier than ARD got here into play, we had been offered with one other resolution for the problem of interoperability. An MCP, in its easiest clarification, is a standardized approach for an AI to connect with your providers to extract information or take motion. WebMCP, because the title itself suggests, provides web sites a solution to interact with brokers straight. WebMCP is for reside browser interactions on a webpage; MCP is for instruments and providers past the web page.
- Open Data Format (OKF): A bundle of markdown recordsdata with YAML frontmatter.
- Agentic Useful resource Discovery (ARD): A brand new open spec for a way brokers discover and confirm instruments, expertise, and different brokers throughout the net. Right here, the main focus shouldn’t be your content material; it’s your capabilities.
For ecommerce, there’s one other layer price naming individually – the product feed – fairly probably the way forward for retail discovery.
Every layer does one thing completely different.
I may hold including to this checklist; there’s a brand new layer popping up each 5 minutes. I’m stopping right here. It’s ballooning.
What OKF Truly Is (And Isn’t)
Google printed the OKF spec quietly, bolted to a rebrand of Dataplex into Data Catalog.
The format itself is nearly disarmingly easy: a listing of markdown recordsdata, every with a small YAML header declaring a kind, title, description, useful resource, and a few tags. The recordsdata hyperlink to one another like every markdown doc would. That’s it.
As Google’s personal weblog places it, OKF is “simply markdown, simply recordsdata, simply YAML frontmatter.”
search engine optimization Suganthan Mohanadasan has a transparent breakdown of this. He describes OKF as one ground in a stack that now contains sitemap.xml (which URLs exist), LLMs.txt (which pages you most need learn), and OKF (the library itself). They stack slightly than compete.
The confusion units in not once you have a look at what OKF is, however what it does and through which layer of the agentic and search mayhem it sits.
In my thoughts, OKF shouldn’t be a retrieval system. It doesn’t substitute crawling. And, personally, I don’t see a future the place AI methods now not ingest large quantities of HTML or the place search and RAG are usually not a multistep complicated pipeline that consists of self-reported and “unbased” alerts.
Any self-reported system can and might be gamed. So considering you may simply slam a bunch of markdown recordsdata in your website and be THE most popular alternative in retrieval and discovery is far-fetched.
OKF is a higher-signal supply amongst many. It might scale back parsing price and enhance sign high quality, but it surely doesn’t substitute present pipelines.
It’s additionally price being trustworthy right here: OKF was constructed for knowledge groups, not advertising websites.
It arrived as a solution to share inner information, i.e., desk schemas, runbooks, metric definitions, between AI brokers inside organizations. Pointing it at a public web site to me appears a bit like we’re but once more repurposing.
Francois Vanderseypen makes essentially the most exact level about what OKF really is and isn’t: a directed graph of markdown recordsdata is an online of paperwork, not a information graph (at the least not in its purest sense). An actual KG has specific, queryable, typed relations. OKF leaves what a hyperlink implies completely as much as the producer, and an LLM nonetheless has to deduce the semantics each single time it reads it.
For me, this factors to the crux of how I perceive the net and what we do as SEOs. OKF doesn’t change the stack. It provides another enter into it.
It’s not a shortcut. There aren’t any shortcuts.
The Schema.org Parallel, And Why It Issues
One of many patterns to grasp right here is the one Schema.org already went by.
Structured knowledge adopted a predictable arc:
Adoption – rating enhance – widespread use (and gaming) – platform studying – diminished dependency as a rating sign.
FAQ schema had a second in SERPs, then Google discontinued the FAQ wealthy end result. The platforms study from the alerts, fold the teachings into the algorithm, and the express markup turns into much less mandatory.
OKF and LLMs.txt could comply with the identical path. They’re most dear early, as clear alerts in a world the place AI methods are nonetheless studying to parse the net.
Over time, if the codecs work, the methods study. Express markup turns into redundant or stays a verification layer. For instance, in ecommerce, particularly, schema and feed alignment has change into increasingly more essential. One other notch within the name for co-ownership of the product feed between search engine optimization and paid groups!
There’s additionally a subtler level price making right here concerning the relationship between schema.org and discovery. Jarno van Driel’s deep dive on product variants in Search Engine Journal illustrates this nicely: For years, Google Search and Google Service provider Heart had conflicting structured knowledge necessities, forcing publishers to duplicate markup. Schema.org evolves to shut gaps, but it surely’s sluggish, it’s complicated, and implementation continues to be typically a large number.
Structured knowledge has by no means been a plug-and-play rating lever. OKF gained’t be both.
Ought to You Convert Your Web site To Markdown?
It’s an enormous fats no from me. That doesn’t imply I gained’t take a look at it and apply fastidiously!
And John Mueller stated it on the Search Off the File podcast:
“Relating to issues like a search engine or most likely additionally in generic LLM system, having a web site that makes use of regular HTML for the pages is vital. As a result of a search engine or crawler can simply go to that web page. It might recognise all the different hyperlinks which might be inside the web site.”
The structural data in HTML – nav hyperlinks, footers, header hierarchies, inner hyperlinks – is how crawlers perceive your website’s form. Markdown recordsdata strip all of that out. You’d be breaking discovery with a purpose to marginally enhance machine readability of particular person pages.
Lately, on LinkedIn, I even noticed a bit of analysis displaying how “Your navigation may be consuming your LLM (it’s ChatGPT Deep Analysis in reality) studying funds.” Fascinating findings, however please don’t take away your navigation to “avoid wasting tokens”!
Jono Alderson makes this level brilliantly: “A web page is not only a container for phrases. It’s an editorial artifact.” Hierarchy, emphasis, placement, what comes first, what’s outstanding, what’s tucked in a footnote … these aren’t fairly decorations for people. “They’re alerts about which means.”
“While you flatten a web page into markdown, you don’t simply take away litter. You take away judgment, and also you take away context.” And the second you publish a machine-only illustration, you’ve created a second candidate model of actuality.
The boring repair nonetheless works: Semantic HTML, clear construction, wise hierarchy, content material that exists when the web page hundreds.
John Mueller covers the markdown debate extensively within the podcast: The parallel variations drawback, the dynamic rendering classes we already realized the onerous approach, and why sustaining a shadow model of your website for AI doubles your upkeep burden and creates a debugging nightmare no one will inform you about.
The one exception Mueller carves out is developer documentation:
“In case you have one thing like developer documentation, the place, once more, if the agent or the LLM system already is aware of about your web site and the consumer says, how do I usethis API? Then for those who give the LLM system a Markdown file, it’s lots simpler for it to grasp.”
Now, I can positively see an easy use case there.
What ARD Is Truly Doing
The Agentic Useful resource Discovery specification, introduced by Google on June 17, 2026, is a special beast completely. It arrived solely a few days behind OKF, not a coincidence, and is already making large waves.
The issue ARD solves is a coordination one. Proper now, an agent needs to be wired to every device, MCP server, or API it makes use of earlier than it could possibly do something with it.
That works once you’re connecting a handful of recognized providers. It stops scaling the second the variety of accessible capabilities grows past what any staff can pre-configure by hand.
ARD strikes that discovery out of setup and into runtime. The agent finds what it wants when it wants it, slightly than solely realizing what it was informed about upfront.
It’s constructed on two primitives:
- Catalogs: An ai-catalog.json file hosted in your area, describing your accessible capabilities (MCP servers, A2A brokers, OpenAPI instruments). Possession of the area acts because the cryptographic basis for identification and belief.
- Registries: Engines like google for the agentic net. They crawl catalogs, index them, and return matching capabilities with the metadata wanted to confirm the writer earlier than connecting.
If OKF is about packaging information for consumption, ARD is about promoting capabilities for connection.
These are parallel efforts at completely different layers of the rising agentic stack. Each shipped inside inches of one another and now adopted with the velocity of sunshine by some very large gamers within the sport, i.e., Hugging Face and their Uncover Device.
It’s probably a extra pragmatic wager than the formal logic layer that got here earlier than it and by no means reached net scale. Time will inform.
A Hole Price Watching
Inside days of each specs delivery, a contributor opened companion points on the ARD and OKF repos stating one thing fundamental was lacking: There’s no agreed media sort for an OKF bundle, so a catalog can checklist one however can’t really acknowledge it as OKF with out sniffing the contents.
Within the meantime, publishers are already promoting bundles in manufacturing utilizing their very own interim sorts, which, as the difficulty itself notes, gained’t agree with one another.
On the face of it, this seems like a small ask, only a request for a shared label.
After a little bit of a dive into this explicit rabbit gap, it seems that’s fairly regular follow. Ready for full settlement earlier than anybody ships something is strictly how a spec dies in committee, and delivery quick and patching as actual adoption surfaces is an age-old technique.
Software/json itself wasn’t formally registered till 2006, roughly 5 years after JSON was already in vast, casual use. No person fearful about that, as a result of the price of the label being unsettled was low: A parser may reject one thing or fall again ungracefully.
However OKF is completely different, as a result of what occurs after the fetch is completely different. The artifact behind the label is a bundle an autonomous agent is supposed to ingest, confirm, and doubtlessly act on, inside a discovery system constructed particularly for agent-to-agent and agent-to-tool connection. Get the kind flawed right here, or depart an agent to deduce it, and the chance isn’t a parse error; it’s a system appearing on one thing it shouldn’t have trusted, with nobody checking the end result first.
I ponder concerning the danger concerned in settling this later slightly than sooner on this case. I assume it is dependent upon how briskly it will get resolved relative to how briskly adoption runs forward of it.
What This Means If You’re An search engine optimization
Just a few trustworthy conclusions and my present considering:
For many advertising and content material websites, not a lot has modified. HTML, well-structured for people, continues to be the fitting basis. A contact-us type and a clear website structure will serve you higher than any OKF bundle ever will. Discovery nonetheless is dependent upon hyperlinks, authority, consumer alerts … and indexing.
LLMs.txt is a signpost, not an search engine optimization device. It’s helpful for serving to an agent navigate inside your website as soon as it’s already there. It very probably doesn’t make an enormous distinction in how brokers discover you within the first place. And, most likely by no means will.
MCP/WebMCP. Neither is pressing for many advertising websites right this moment, however for those who’re constructing something with programmatic interfaces or ecommerce flows you need brokers to navigate, that is the route the infrastructure is heading.
OKF makes numerous sense for those who’re sitting on structured inner information, i.e., documentation, API references, product specs … and also you need to make it simpler for brokers to devour. The free OKF generator Suganthan constructed will produce a bundle and provide you with a graph view of your inner hyperlink construction as a facet profit. The structural audit alone appears price it. However I might be operating it on my web site first, not on my shopper’s web site.
ARD is price watching for those who’re constructing providers with programmatic interfaces. In case you have instruments, brokers, or APIs you need discoverable by different brokers, ARD is the rising customary for a way that will get completed. Simply know the identification layer beneath it, what an agent is definitely taking a look at when it finds your catalog entry, continues to be being settled in actual time, so I’d deal with this as infrastructure to look at intently slightly than construct vital paths on simply but.
The schema adoption cycle may repeat. These codecs are most dear now, as early alerts. Implement them if you are able to do it cheaply. Don’t construct your technique round them holding worth perpetually and don’t financial institution on them as a silver bullet.
Finally, pay attention to the shiny issues – if your organization has greater fish to fry, i.e., a horrible web site, a model nobody is aware of or cares for, an viewers you don’t perceive … then cope with this primary earlier than you get caught up in any of those new shiny issues.
The Underlying Shift
What all of this factors to is an online that’s genuinely rising a second layer or a 3rd head, one written for machines alongside the one written for browsers and people.
Sitemap.xml informed crawlers which URLs existed. Robots.txt informed them the place to not go. LLMs.txt, OKF, and ARD are comparable infrastructure for agentic methods: navigation hints, content material packaging, and functionality discovery.
None of it’s obligatory right this moment. None of it replaces strong HTML, authoritative content material, wise construction, or the factor that truly sits beneath all of it: a model price discovering.
However the SEOs who perceive what every layer really does, slightly than treating it as a single undifferentiated “AI search engine optimization” class, will make higher bets on the place to spend their time.
My cash is on the second layer, a parallel infrastructure written for machines, not a alternative for what already exists.
The third head state of affairs, the place agentic methods absolutely diverge from the human net, would require a special set of bets than any of us are at present making.
Large because of Jarno van Driel, Jono Alderson, Chris Inexperienced, Suganthan Mohanadasan, Kristine Schachinger, Gianluca Fiorelli, Victor Pan, Renee Bigelow (and anybody else I’ve missed) for some good discussions on this matter over the previous couple of weeks.
Extra Assets:
Featured Picture: Collagery/Shutterstock
