OpenAI’s automated crawl exercise is estimated to have roughly tripled after the launch of GPT-5, in line with a brand new evaluation from Botify and visitor writer Chris Lengthy.
In Botify’s dataset, OpenAI’s search crawler is now producing extra log occasions than its coaching crawler. That’s a reversal from the interval earlier than GPT-5.
Lengthy, co-founder of the web optimization consultancy Nectiv, analyzed roughly 7 billion OpenAI-bot log occasions from Botify’s enterprise consumer dataset spanning November 2024 by means of March 2026.
What The Knowledge Reveals
Two of the three OpenAI person brokers Botify measured noticed exercise spike across the GPT-5 launch.
OAI-SearchBot, which retrieves content material when ChatGPT performs internet searches, recorded about 3.5x extra occasions after August 2025. That works out to roughly 2.2 billion extra occasions in Botify’s dataset.
GPTBot, which collects coaching information, recorded about 2.9x extra occasions over the identical interval. That’s one other 1.8 billion occasions.
The third person agent, ChatGPT-Consumer, moved in the wrong way. Lengthy stories a 28% drop in ChatGPT-Consumer log occasions between December 2025 and March 2026. ChatGPT-Consumer fires when a ChatGPT session fetches a web page on behalf of a person, so the drop measures logged user-initiated fetches reasonably than ChatGPT utilization total.
Lengthy presents two attainable readings. One is that fewer classes could also be triggering real-time web page fetches. The opposite, recommended by Botify’s group, is that OpenAI could also be relying extra on saved or listed sources, decreasing the necessity to fetch pages in actual time. Lengthy doesn’t decide between them.
Search Bot Now Outpaces Coaching Bot
Earlier than GPT-5, OAI-SearchBot and GPTBot ran at roughly even volumes in Botify’s dataset, with a ratio of about 0.95 search occasions per coaching occasion. After GPT-5, that ratio rose to about 1.14.
The sample strains up with what Dan Petrovic wrote in August 2025 about GPT-5, arguing that OpenAI was sourcing extra solutions from dwell search than from skilled reminiscence. Botify’s information is in keeping with that learn.
Trade Breakdown
The post-GPT-5 search bot will increase assorted by trade. Healthcare websites noticed about 740% extra OAI-SearchBot exercise after launch; Media and Publishing, 702%; and Marketplaces, Software program, and Retail, 190-216%.
Journey websites had the smallest rise at 30%. The search and coaching steadiness additionally varies. Lengthy stories a +256% OAI-SearchBot to GPTBot crawl distinction for Media/Publishing, the most important hole. Software program and Web lean towards search, Healthcare and Retail favor coaching, with -50% and -33%. GPTBot is extra lively total.
Botify and Lengthy recommend OpenAI routes immediate varieties in a different way: information inquiries set off dwell search, well being and product queries depend on skilled information.
How OpenAI’s Crawl Compares To Google’s
Even after tripling, OpenAI’s crawl exercise is way smaller than Google’s.
In Botify’s most up-to-date 30-day window, Googlebot registered 18.2 billion occasions, in contrast with 887 million occasions from OpenAI’s crawlers mixed. That places OpenAI at about 4% of Google’s crawl quantity.
A yr earlier, the identical comparability was 15 billion Google occasions to 207 million OpenAI occasions, or about 1.38%. The hole is closing, although Google’s crawl remains to be roughly 20 occasions bigger in absolute phrases.
Bingbot registered about 5.49 billion occasions in the latest window, placing OpenAI at roughly 14% of Bing.
Methodology & Business Context
The dataset is Botify’s, overlaying enterprise purchasers in retail, ecommerce, know-how, publishing, journey, and marketplaces. The evaluation was performed by Lengthy as a visitor writer on Botify’s weblog.
For transparency, Botify sells log file evaluation and AI bot administration software program, and the publish promotes a follow-up webinar and a product demo.
The dataset skews towards giant enterprise web sites reasonably than a consultant cross-section of the online.
Why This Issues
In Botify’s dataset, OAI-SearchBot now generates extra log occasions than GPTBot. Websites that block solely GPTBot should not blocking the bot OpenAI says is used to floor web sites in ChatGPT search solutions.
Websites that block OAI-SearchBot could also be excluding themselves from ChatGPT search solutions.
How This Suits With Different Studies
Botify’s findings line up with patterns different distributors have reported. An Alli AI evaluation coated earlier this month discovered OpenAI’s ChatGPT-Consumer made 3.6x extra requests than Googlebot in a smaller WordPress-heavy pattern. A Hostinger evaluation discovered OAI-SearchBot’s web site protection reaching 55% whereas GPTBot protection fell. Akamai’s latest bot site visitors report confirmed OpenAI main AI bot site visitors to publishing websites.
The stories recommend that AI coaching crawls and AI search crawls should be measured individually, particularly as OAI-SearchBot exercise grows.
