Keeping Googlebot Happy Revisited
I stumbled upon an older blog post I wrote a couple of years back discussing the advantages of keeping your site fast, specifically so Google can crawl it efficiently to…
I stumbled upon an older blog post I wrote a couple of years back discussing the advantages of keeping your site fast, specifically so Google can crawl it efficiently to keep their index as fresh as possible. I loaded up Google’s Webmaster Tools to see what kind of numbers we’re doing these days and was pretty happy with the results.
March 3, 2009
October 8, 2011
The graphs speak for themselves, but I’m pretty proud of the fact that we’ve been able to simultaneously increase the number of pages crawled per day while cutting the average page download time by a significant amount.
Here’s to keeping things fast and fresh!
Written by
Related posts
10 technical talks I’m excited about at GitHub Universe 2026
From verifying AI-written code to securing npm dependencies, these are the sessions I’m building my Universe agenda around.
Developer policy update: Transparency, state policy, and what’s ahead
Explore GitHub’s latest transparency data and learn more about policy updates affecting developers and open source.
Developers want more efficient software. Here’s what over 1,000 GitHub users told us they need.
New research from GitHub and Yale Program on Climate Change Communication finds strong demand for tools, measurement, and practical guidance that can help developers reduce wasted compute.