For the final couple of years, the subject of AI visibility has dominated search engine optimisation discussions. search engine optimisation groups have cleaned code and chunked content material, all the better for AI bots to entry and ingest. In case your model has began incomes common citations in AI-generated solutions or exhibiting up in AI Overviews, you could be forgiven for considering the battle for AI visibility is almost received.
None of that’s improper, and none of it’s wasted effort. It’s simply not the entire story.
Nonetheless, our audit of fifty main web sites discovered that, whereas most have made it simpler for AI to discover them, virtually none have made it attainable for AI to really perceive them. And practically two-thirds go away the query of which AI bots can entry which content material completely to luck.
3 Layers Of AI Visibility
Ask a advertising workforce to outline “AI visibility,” and the reply will probably give attention to mentions and citations. Ought to somebody ask any of the main AI platforms a query related to your class, you need your model and/or product to have a greater than common probability of showing within the response.
Defining AI visibility purely when it comes to citations and mentions is previous search engine optimisation considering – get ranked, get discovered, get clicked – carried over to a brand new and really totally different type of search. However visibility in AI isn’t merely about getting your model, product and related hyperlinks in entrance of the precise eyeballs. It’s additionally about how AI understands your data and interacts along with your web site.
In case you’ve parented a baby by major faculty, you already know that instructing them to learn isn’t the identical as instructing them to grasp. A six-year-old studying phonics may learn the phrases of their early studying e-book moderately fluently. Tick, get your self a cookie from the jar. However ask them about what they’ve simply learn, and chances are you’ll discover they absorbed or understood lower than you thought.
As soon as a baby has mastered each studying and comprehension, they will put what they be taught into motion – comply with a recipe to prepare dinner a meal, write their very own tales or essays, perform additional analysis to search out solutions to questions arising from the preliminary textual content, and so forth.
Studying is just one step in direction of comprehension. And comprehension is just one step in direction of company; what you resolve to do with that data upon getting it.
This is the reason we developed an audit framework for AI readiness with three distinct layers:
1. Retrievability
Can AI fetch and parse your content material with out stumbling? That is the studying bit, the foundational layer most search engine optimisation groups already optimize for, instantly associated to AI visibility within the sense of citations and model mentions.
However except you additionally optimize for the second layer, you’re principally crossing your fingers and hoping AI interprets your content material appropriately.
2. Attribution And That means
Can AI decide what your pages are about and who owns them? That is how AI is aware of, with out merely guessing, which quantity on the web page is the product value, and which is the low cost value accessible solely to loyalty card holders. It’s how AI determines who the model and/or creator actually is, and whether or not to contemplate them a trusted authority on the topic. It’s how AI tells whether or not an article is about Jaguar the automotive model, or one of many many Jaguar soccer golf equipment.
Giving AI better confidence that it has appropriately interpreted your content material does two issues: It will increase the probability of your content material being cited in related AI responses, whereas additionally decreasing the chance of these responses misrepresenting your model, your merchandise or your claims.
3. Agent Transaction And Discovery
Can AI brokers entry your web site’s capabilities to hold out duties, corresponding to finishing transactions? This remaining layer is about company, and it’s the distinction between AI merely parroting data again to the consumer and with the ability to work together with your enterprise on their behalf.
Beneath every of those layers, we grouped the probably protocols search engine optimisation groups may implement to obtain these outcomes. For instance, implementing ARIA labeling or together with AI user-agent directives within the robots.txt would largely relate to Retrievability, whereas the presence of an entity map or JSON-LD schema would relate to Attribution and That means.
Altogether, our framework consists of 27 audit parts – 11 in Layer One, three in Layer Two, and 13 in Layer Three – rated and scored based on their present maturity as business requirements:
- Established: Manufacturing-ready requirements in lively use right this moment.
- Rising: Actual protocols gaining traction with early movers, and rising quick.
- Frontier: Requirements nonetheless being debated with no settled implementation thus far.
Armed with this framework, we audited 50 main web sites throughout retail, SaaS, journey, publishing and finance to see the place the commonest gaps in AI readiness could be.
I’ll admit it’s a tad unfair of me to use the identical framework and scoring system throughout all the cohort, no matter business and enterprise mannequin. When utilizing our AI readiness framework with purchasers, we weight the scoring based on their business. However utilizing the framework as a benchmarking software throughout a diversified cohort wouldn’t work if we measured various things or utilized totally different scoring to every web site.
The aim was to focus on the place the most important gaps are total, somewhat than guessing on the nuances of each particular person enterprise case or business choice. A low rating doesn’t essentially imply a web site is underprepared for AI if different clues counsel they’ve intentionally adopted this method.
See additionally: Machine-First Structure: How To Construct Web sites Machines Can Determine, Learn, Cite & Use
What The Knowledge Reveals
Utilizing an instrumented browser, we had been capable of seize stay HTTP responses, the rendered DOM, uncooked server HTML, and machine-discovery endpoints. All information was captured on the identical day: June 12, 2026.
We then scored 12 Established indicators in opposition to our agentic readiness framework (0/1/2), with the ultimate rating rendered as a share of the attainable most. Whereas we additionally tracked Rising and Frontier protocols, we excluded these from the scoring.
- The best rating went to Airbnb.com with 79.2%.
- On common, websites have carried out simply over half of the established protocols (imply 56.6%, median 58.3%).
- Solely 10 websites scored under 50%.
Whereas that may sound like a move mark, it isn’t. Once you pull the three layers aside, it turns into clear the place there’s nonetheless work to be finished.
1. Retrievability (Common 74.4%)
- Established Audit Parts (8/11)
- txt & AI Person-Agent Directives.
- Accessibility Tree Integrity.
- ARIA Labeling & Descriptive Names.
- Semantic HTML & Doc Hierarchy.
- Token-Environment friendly DOM Density.
- Server-Rendered / Clear HTML Supply.
- Kind & Enter Machine Usability.
- Sitemap Declaration.
That is the layer that overlaps most closely with standard technical search engine optimisation, together with parts which will have already been in place, making any new tweaks to optimize for AI retrievability simpler to implement. Unsurprisingly, most websites carry out moderately properly, with solely three scoring under 50%. And there could possibly be legit causes for these low scores, as you’ll see.
2. Attribution And That means (Common 38.5%)
- Established Audit Parts (2/3)
- JSON-LD Schema & Semantic Richness.
- Content material Indicators Coverage.
Scores drop away sharply between the primary and second layers.
The excellent news is we detected JSON-LD structured information on the homepage of 35 out of fifty web sites (70%), with all however three scoring the 2-point most. The dangerous information is that also leaves practically a 3rd with none in any respect.
Schema is a crucial a part of how AI understands what your content material truly means: It is a product, that is the value, and that is the model that sells it. Poor or non-existent schema in all probability received’t undermine your web site’s AI retrievability, and the LLM will nonetheless make its greatest guess at what all the pieces means. However that greatest guess will also be improper, resulting in inaccurate AI responses and even outright hallucinations.
Maybe extra telling is that solely 5 of the 50 audited websites have carried out Cloudflare’s Content material Indicators Coverage. It is a set of directives inside the robots.txt file that spell out precisely what crawlers are permitted to do along with your content material in relation to go looking indexing, stay AI question responses, and mannequin coaching. A Content material Indicators Coverage means your AI technique isn’t restricted to a simplistic binary alternative – “block all AI” or “enable all AI” – permitting you to take a extra strategic method.
3. Agent Transaction And Discovery (Common 2.1%)
- Established Audit Parts (2/13)
- OAuth Discovery (Authorization Server).
- OAuth Protected Useful resource Metadata.
Agentic browsers and agentic commerce have solely been round since late 2025. Consequently, of the 13 protocols we recognized as instantly associated to Layer 3, we may solely class two as Established.
Each ebay.com and wikipedia.org block same-origin fetch by way of CSP. However of the 48 websites the place endpoint testing was attainable, 46 scored zero. And whereas airbnb.com and vercel.com have each carried out OAuth authorization server metadata, neither has carried out OAuth protected useful resource metadata, that means they solely scored 50% every for Layer 3.
OAuth discovery and OAuth protected metadata make it attainable for AI to work together along with your web site. For instance, the OAuth discovery metadata tells a shopper utility what it’s allowed to entry, the best way to establish itself, and the place to get the knowledge it must securely entry an OAuth authorization server or protected API.
Nonetheless, our workforce is carefully monitoring 9 different Rising parts in Layer 3 that would see the house evolve shortly.
Some relate to the Mannequin Context Protocol (MCP), which permits AI to question your servers instantly. For instance, as a substitute of piecing collectively data out of your web site product pages, AI will get what it wants straight out of your real-time product database, drastically decreasing processing time and the chance of AI hallucination.
After which there are the comparatively new ecommerce protocols that enable AI to finish transactions inside AI conversations, like Google’s Common Commerce Protocol (UCP) and OpenAI’s Agentic Commerce Protocol (ACP), that are prone to turn into established within the close to future.
Layer 3 may be very a lot the world to look at, with the potential to confer first-mover benefit on any manufacturers keen to experiment.
A Low Rating Isn’t At all times A Unhealthy Rating
I’m not suggesting each enterprise ought to embrace AI in the identical approach, following our framework of established protocols like a guidelines to be accomplished. Whereas it’s technically attainable for a web site to attain 100%, that doesn’t imply it ought to.
Some publishers inside the cohort – together with the BBC, CNN and The Guardian – particularly block most, if not all, AI bots. That is comprehensible. Their enterprise fashions rely on individuals visiting their websites to get the most recent information and skim their specific model of reporting.
In the meantime, industries corresponding to retail, SaaS, journey and monetary providers are more and more depending on reaching prospects and influencing choices in these AI areas. And as extra prospects begin interacting with these companies by way of AI brokers as a substitute of by their web sites, all three layers will turn into more and more essential.
Then once more, one of many lowest total scores (29.2%) belongs to amazon.com, which was additionally one of many three websites to attain under 50% in Layer 1. Now, I don’t suppose for one minute that Amazon has merely missed AI search. Just like the BBC, Amazon has particular AI crawler directives in place to dam virtually all AI bots from accessing the positioning, so that is clearly by design. And relying on their causes for blocking the bots, there won’t be a lot level in implementing a number of the different measures.
Conversely, some websites, like airbnb.com and cloudflare.com, don’t have any formal blocks in place however have carried out particular entry guidelines for a lot of the main AI bots. Others, together with eBay.com and tripadvisor.com, have been much more selective, implementing guidelines for the AI bots they do need and blocking these they don’t.
Every of the above websites has made deliberate, bot-specific choices and up to date their robots.txt with the suitable AI directives. They’ve begun to implement a plan for AI even when, in some circumstances, that plan is to dam it completely – wherein case an AI readiness rating is sort of moot.
However practically two-thirds of the audited websites (29 of fifty) seem to have made no deliberate choice about AI agent entry in some way. Nothing is blocked, and nothing is explicitly allowed both, with no guidelines for crawlers to comply with.
What AI bots can entry, how they interpret what they discover, and the way they use that data in AI responses is basically left to probability.
Indicators Are Not Ensures
At this level, we also needs to point out llms.txt, which fits past the AI directives and content material indicators inside robots.txt to supply AI programs with a curated, human-readable information to a web site’s content material and construction.
Our framework classifies llms.txt as a Frontier aspect, excluded from scoring, as a result of it’s at the moment an unratified commonplace with no agreed specification physique behind it. Nonetheless, we discovered 11 of the 50 websites have revealed one, suggesting there’s some enthusiasm for the thought of offering AI with a helpful “CliffsNotes” information to a model and its content material.
However earlier than you instruct your workforce to replace your web site’s robots.txt or add an llms.txt, it’s value clarifying what these information can – and might’t – do.
Any AI directives documented in your robots.txt are desire statements, somewhat than hard-coded guidelines. Compliance is voluntary. The most important AI crawlers – GPTBot, ClaudeBot, Google-Prolonged and Applebot-Prolonged – broadly respect them, however past that group, issues get patchier. Some bots might crawl your content material regardless.
Content material indicators carry even much less enforcement weight. Nicely-behaved bots can select to comply with them, however there’s no technical mechanism to pressure compliance. And, for now at the very least, llms.txt remains to be little greater than a sign of intent.
For instance, Expedia has revealed an llms.txt file that reads like a love letter to AI, setting out the model’s canonical identification and explaining in neatly chunked copy what the model is about, its capabilities, and so forth.
But when it got here to the remainder of the positioning’s AI structure, Expedia.com scored simply 33.3% total – one of many lowest in all the cohort. No JSON-LD structured information, no sitemap declaration, and solely 1 / 4 of the content material is delivered server-side. Publishing such a well-written llms.txt file whereas leaving a lot else unaddressed is somewhat like placing an indication in your window saying “open for enterprise” however forgetting to unlock the door.
This doesn’t imply such measures are wasted, simply that outcomes are removed from assured. However when has something been assured to work in search engine optimisation? It’s why optimization has all the time been about strengthening as many indicators as attainable.
Updating your robots.txt with detailed directions or making a complete llms.txt aren’t replacements for doing the remainder of the work to enhance your web site’s AI structure.
Block, Enable, Or Do Nothing: The Selection Is Yours
Mentions and citations had been by no means the entire story; they’re solely the primary layer of true AI visibility. Your model could possibly be cited frequently and nonetheless lose prospects due to inaccuracies or hallucinations, or as a result of the AI didn’t have sufficient confidence in your data to advocate your merchandise in essentially the most related conversations.
Airbnb, eBay, Amazon and others aren’t chasing mentions (within the latter’s case, positively not). Each has made deliberate, particular decisions about which AI programs can prepare on their content material, and which might act on a buyer’s behalf within the second.
The choices that may outline your model’s AI presence received’t all be made by your advertising workforce. A few of them might have already been made for you, hidden inside default settings, legacy robots.txt information, and safety insurance policies written for a distinct period.
Whether or not you’ve deliberate and optimized for it or not, AI is nearly actually studying your content material. However that is nonetheless your content material. You’ll be able to nonetheless exert some management over how AI accesses, interprets, and interacts along with your model.
These are all solvable issues – however provided that you resolve to unravel them.
Extra Sources:
Featured Picture: SvetaZi/Shutterstock
