Keeping Googlebot Happy Revisited
I stumbled upon an older blog post I wrote a couple of years back discussing the advantages of keeping your site fast, specifically so Google can crawl it efficiently to…
I stumbled upon an older blog post I wrote a couple of years back discussing the advantages of keeping your site fast, specifically so Google can crawl it efficiently to keep their index as fresh as possible. I loaded up Google’s Webmaster Tools to see what kind of numbers we’re doing these days and was pretty happy with the results.
March 3, 2009
October 8, 2011
The graphs speak for themselves, but I’m pretty proud of the fact that we’ve been able to simultaneously increase the number of pages crawled per day while cutting the average page download time by a significant amount.
Here’s to keeping things fast and fresh!
Written by
Related posts
Introducing Agent HQ: Any agent, any way you work
At Universe 2025, GitHub’s next evolution introduces a single, unified workflow for developers to be able to orchestrate any agent, any time, anywhere.
Octoverse: A new developer joins GitHub every second as AI leads TypeScript to #1
In this year’s Octoverse, we uncover how AI, agents, and typed languages are driving the biggest shifts in software development in more than a decade.
Announcing the 2025 GitHub Partner Award winners 🎉
GitHub celebrates its 2025 Partner Award winners, honoring global, regional, and technology partners for driving innovation, collaboration, and impact across the developer ecosystem.