Google up to date its Googlebot documentation to make clear details about file measurement limits.
The change entails shifting details about default file measurement limits from the Googlebot web page to Google’s broader crawler documentation. Google additionally up to date the Googlebot web page to be extra particular about Googlebot’s personal limits.
What’s New
Google’s documentation changelog describes the replace as a two-part clarification.
The default file measurement limits that beforehand lived on the Googlebot web page now seem within the crawler documentation. Google stated the unique location wasn’t probably the most logical place as a result of the boundaries apply to all of Google’s crawlers and fetchers, not simply Googlebot.
With the defaults now housed within the crawler documentation, Google up to date the Googlebot web page to explain Googlebot’s particular file measurement limits extra exactly.
The crawling infrastructure docs record a 15 MB default for Google’s crawlers and fetchers, whereas the Googlebot web page now lists 2 MB for supported file varieties and 64 MB for PDFs when crawling for Google Search.
The crawler overview describes a default restrict throughout Google’s crawling infrastructure, whereas the Googlebot web page describes Google Search–particular limits for Googlebot. Every useful resource referenced within the HTML, resembling CSS and JavaScript, is fetched individually.
Why This Issues
This matches a sample Google has been operating since late 2025. In November, Google migrated its core crawling documentation to a standalone website, separating it from Search Central. The reasoning was that Google’s crawling infrastructure serves merchandise past Search, together with Buying, Information, Gemini, and AdSense.
In December, extra documentation adopted, together with faceted navigation steering and crawl price range optimization.
The most recent replace continues that reorganization. The 15 MB file measurement restrict was first documented in 2022, when Google added it to the Googlebot assist web page. Mueller confirmed on the time that the restrict wasn’t new. It had been in impact for years. Google was simply placing it on the report.
When managing crawl budgets or troubleshooting indexing on content-heavy pages, Google’s docs now describe the boundaries otherwise relying on the place you look.
The crawling infrastructure overview lists 15 MB because the default for all crawlers and fetchers. The Googlebot web page lists 2 MB for HTML and supported text-based recordsdata, and 64 MB for PDFs. Google’s changelog doesn’t clarify how these figures relate to 1 one other.
Default limits now stay within the crawler overview documentation, whereas Googlebot-specific limits are on the Googlebot web page.
Wanting Forward
Google’s documentation reorganization suggests there’ll doubtless be extra updates to the crawling infrastructure website within the coming months. By separating crawler-wide defaults from product-specific documentation, Google can extra simply doc new crawlers and fetchers as they’re launched.
