Search Console crawl stats, the honest crawl health picture

What crawl stats actually measure

Crawl stats (in the Settings area of Search Console, under Crawling) answers a narrower question than the Pages report: how much Google fetched from your servers over the selected period, broken down by request count, response codes, times and host content types. You can think of it as the Googlebot view of your access log.

The three headline graphs are Crawl requests, Crawled bytes, and Page load time. Below them the defaults matter: crawl requests and crawler bytes summarize Googlebot behavior, and the host breakdown tells you which of your servers (www, api, images subdomain) the bot is spending time on.

The crawl rate is bound by a cap Google calculates from your server health. A healthy site (fast, few errors, no recrawl rot) gets a higher cap. A site that experiences returns like 4xx and 5xx in volume can see the rate pulled back. That is the mechanism behind "Google disappeared from the log".

The morning bread-and-butter: distinguishing normal patterns

A pattern you will see in any real log and any Search Console:

• Crawl volume is bursty: a spike after a sitemap submission or new content publication, then the queue drains.
• Peaks align with Google discovering something new, or with a re-crawl of content that changed.
• A gentle drop in the last two days is noise, not an event.

The crawl-stats report is sampled for larger sites and shows daily totals. To compare honestly, keep the same host, time range and (if you can) response-type filter on every reading.

Sign 1: crawl requests drop sharply

A real drop (more than half, sustained for a week+, with no content change) asks three questions:

1. Is robots.txt suddenly blocking a chunk of the site? Read robots.txt and test with the URL inspection live test.
2. Did the server start failing on 5xx, or returning slow responses? Check your access log and the load-time line in this same report before treating it as a Google decision.
3. Did the sitemap disappear or start failing? Check sitemap submission.

If none fit, the drop can still be the natural crawl queue; a large site that changes little does not earn a roaring crawl. Compare to the same month last year before fearing public.

Sign 2: requests stay flat or climb while Pages show a growing not-indexed tail

Crawling harder without more indexing means the separation of "crawled" and "indexed" is behaving: Googlebot fetches (requests), but indexing is not following. This is the pattern the indexing analysis loop solves: the crawl is healthy, the index-layer reasons are the override.

Sign 3: load time climbs while requests climb

A growing download-time line over a growing count usually means heavy duplicated content at crawl time (huge sitemaps, parameter bloat). The crawl budget and the web performance audit both have the closer look.

The crawl-side of the console and what it shares

Crawl stats is the "what" of Googlebot; the Pages report is the index-level "so what". Together they pin whether the behavior lives in the crawl leg (robots, server, sitemap) or the index leg (content, canonicals, quality) of the pipeline.