Increase your abilities with Development Memo’s weekly knowledgeable insights. Subscribe totally free!
For years, SEOs have operated on a easy assumption: The extra floor your content material covers, the extra doubtless it’s to floor in AI-generated solutions. Actually, each “finest observe” in traditional web optimization content material pushes you towards extra: extra subtopics, extra sections, extra phrases. Construct the “final information.”
An evaluation of 815,000 query-page pairs throughout 16,851 queries and 353,799 pages says in any other case:
- Fan-out protection is almost irrelevant to quotation charges.
- Two indicators truly predict whether or not ChatGPT cites your web page.
- Six concrete modifications to your current content material library assist.
1. The Research
AirOps ran 16,851 queries by means of ChatGPT 3 times every by means of the UI, capturing each fan-out sub-query, each URL searched, each quotation made, and each web page scraped. Oshen Davidson constructed the pipeline. I analyzed the information.
Every question generates a mean of two fan-out queries. ChatGPT retrieves roughly 10 URLs per sub-search, reads by means of them, then selects which of them to quote. We scored how properly every web page’s H2-H4 subheadings matched these fan-out queries utilizing cosine similarity on bge-base-en-v1.5 embeddings. That rating is what we name fan-out protection: the share of subtopics a web page addresses at a 0.80 similarity threshold. (The 0.80 similarity threshold cutoff was used to resolve whether or not a subheading counts as a match to a fan-out question. Consider it as a relevance bar.)
The query: Do pages with larger fan-out protection get cited extra?
You’ll discover much more data within the co-written AirOps report.
2. Density Barely Strikes The Needle
Throughout 815,484 rows, the connection between fan-out protection and quotation is weak.
Masking 100% of subtopics provides 4.6 share factors over overlaying none. That hole shrinks additional once you management for question match (how properly the web page’s finest heading matches the unique question). Amongst pages with robust question match (>= 0.80 cosine similarity):
Average protection (26-50%) outperforms exhaustive protection. Pages that cowl all the things rating decrease than pages that cowl 1 / 4 of the subtopics. The “final information” technique produces worse outcomes than a centered article that covers two to a few associated angles properly.
3. What Really Predicts Quotation
These two indicators dominate: retrieval rank and question match.
1. Retrieval rank is the strongest predictor by a large margin. A web page at place 0 in ChatGPT’s net search outcomes (the primary URL returned by its search device) has a 58% quotation price. By place 10, that drops to 14%. We ran every immediate 3 times consecutively for this evaluation, and pages cited in all three runs have a median retrieval rank of two.5. Pages by no means cited: median rank 13.
2. Question match (cosine similarity between the question and the web page’s finest heading) is the strongest content material sign. Pages with a 0.90+ heading match have a 41% quotation price in comparison with the 30% price for pages under 0.50. Even amongst top-ranked pages (place 0-2), larger question match provides 19 share factors.
Fan-out protection, phrase rely, heading rely, area authority: all secondary. Some are flat. Some are inversely correlated.
4. The Wikipedia Exception
One web site kind breaks the sample. Wikipedia has the worst retrieval rank within the dataset (median 24) and the bottom question match rating (0.576). It nonetheless achieves the best quotation price: 59%.
Wikipedia pages common 4,383 phrases, 31 lists, and 6.6 tables. They’re encyclopedic within the literal sense. ChatGPT cites Wikipedia from deep within the search outcomes the place each different web site kind will get ignored.
That is density working as a sign, however at a scale no writer can replicate. Wikipedia’s content material is exhaustive, richly structured, and cross-linked throughout hundreds of thousands of matters. A 3,000-word company weblog submit with 15 subheadings just isn’t the identical factor.
5. The Bimodal Actuality
58% of pages retrieved by ChatGPT on this dataset are by no means cited. 25% are at all times cited after they seem. Solely 17% fall in between.
The always-cited and never-cited teams look practically similar on most content material metrics: related phrase counts (~2,200), related heading counts (~20), related readability scores (~12 FK grade), related area authority (~54). The on-page indicators we will measure don’t separate winners from losers.
What separates them is retrieval rank. All the time-cited pages rank close to the highest after they floor. By no means-cited pages rank within the backside half. The retrieval system, no matter indicators it makes use of internally, is the gatekeeper. All the things else is a tiebreaker.
6. What This Means For Your Content material
Typical web optimization content material writing knowledge says cowl extra subtopics, add extra sections, construct density. The information says the traditional strategy produces “blended” pages, the 17% within the center that get cited generally and ignored different occasions.
Combined pages have the best phrase counts, essentially the most headings, and the best area authority within the dataset. They’re the “final guides.” They’re additionally the least dependable performers in ChatGPT.
The pages that win persistently are centered. They:
- Match the question instantly of their headings,
- Are typically shorter (the quotation candy spot is 500-2,000 phrases), and
- Have sufficient construction (7-20 subheadings) to arrange the content material with out diluting it.
Construct the web page that’s the finest reply to 1 query. Not the web page that adequately solutions 20.
Featured Picture: Tero Vesalainen/Shutterstock; Paulo Bobita/Search Engine Journal
