A randomized subject experiment finds Google’s AI Overviews cut back natural clicks to exterior web sites by 38% on queries the place they seem, whereas self-reported search satisfaction stays almost unchanged when the summaries are eliminated.
The working paper by researchers on the Indian Faculty of Enterprise and Carnegie Mellon College was posted to SSRN this month. Authors Saharsh Agarwal and Ananya Sen describe it as the primary randomized subject experiment to check how AI Overviews have an effect on consumer habits in an actual searching surroundings.
How The Experiment Labored
Agarwal and Sen constructed a Chrome extension that randomly assigned 1,065 U.S. members to certainly one of three teams. Folks have been recruited from Prolific and used Chrome on desktop. In addition they needed to meet minimal browsing-history thresholds, so the pattern displays energetic desktop Chrome customers reasonably than all Google customers.
The management group noticed Google Search usually. A “Conceal AIO” group had the extension take away AI Overviews in actual time. A 3rd group was redirected to Google’s AI Mode for all searches. The examine ran for 2 weeks per participant between January and February 2026.
Researchers pre-registered the experiment with the AEA RCT Registry earlier than information assortment. Over 95% of customers within the Conceal AIO group didn’t detect any adjustments throughout the examine.
What The Researchers Discovered
AI Overviews appeared on 42% of queries, and eradicating them elevated outbound clicks from 0.38 to 0.61 per search. They decreased outbound natural clicks by 38% on triggered queries, with zero-click search rising from 54% to 72%.
Results have been strongest when AI Overviews appeared on the prime of the web page, which occurred 85% of the time. Eradicating top-position AI Overviews almost doubled outbound clicks, however decrease ones had no impact.
Sponsored clicks and search frequency remained regular, indicating substitution between AI Overviews and natural visits.
The Person Expertise Discovering
The endline survey used a 1-to-5 Likert scale to evaluate members’ search expertise. Responses from the management and Conceal AIO teams have been almost similar throughout all measures, together with satisfaction, info high quality, and ease of discovering info.
The researchers wrote that AI Overviews “divert site visitors away from publishers with out delivering measurable enhancements in consumer expertise.”
How AI Mode In contrast
Individuals directed to AI Mode had decrease outbound click on charges, greater zero-click charges, and decrease satisfaction at endline in comparison with different teams.
The authors notice that these outcomes are exploratory, as greater attrition, some uninstalling of the extension, or discovering workarounds might have influenced the outcomes.
Why This Issues
Impartial measurements of the impression of AI Overviews on site visitors have principally been correlational. Pew Analysis discovered customers click on 8% of the time with AI Overviews, in comparison with 15% with out. Ahrefs analyzed GSC information and reported a 58% drop in click-through price for top-ranking pages when AI Overviews appeared.
This experiment provides a special method by randomly assigning customers to see AI Overviews or not, isolating the causal impact.
Google VP Liz Reid claims AI Overviews lower “bounce clicks,’ however gives no information backing the user-benefit aspect. The Agarwal and Sen paper examined a associated query with a randomized design, discovering no measurable change in satisfaction or ease of discovering data.
Trying Forward
The paper is a draft on SSRN and isn’t peer-reviewed. Authors will add extra outcomes, and we’ll present an replace if findings change.
