Keeping Googlebot Happy Revisited
I stumbled upon an older blog post I wrote a couple of years back discussing the advantages of keeping your site fast, specifically so Google can crawl it efficiently to…
I stumbled upon an older blog post I wrote a couple of years back discussing the advantages of keeping your site fast, specifically so Google can crawl it efficiently to keep their index as fresh as possible. I loaded up Google’s Webmaster Tools to see what kind of numbers we’re doing these days and was pretty happy with the results.
March 3, 2009
October 8, 2011
The graphs speak for themselves, but I’m pretty proud of the fact that we’ve been able to simultaneously increase the number of pages crawled per day while cutting the average page download time by a significant amount.
Here’s to keeping things fast and fresh!
Written by
Related posts
Why age assurance laws matter for developers
Youth safety requirements are moving down the tech stack to operating systems and app stores—raising new questions for open source developers.
How researchers are using GitHub Innovation Graph data to reveal the “digital complexity” of nations
Researchers share in an interview how they used GitHub data to predict GDP, inequality, and emissions in ways that traditional economic data misses, along with our Q4 2025 data release.
An update on GitHub availability
Here’s what we’ve done—and what we’re still doing—to improve our availability and reliability.