Keeping GoogleBot Happy
One of the interesting side effects I hadn’t considered when we rolled out some fairly significant caching updates on GitHub in the beginning of January was how much Google’s crawler…
One of the interesting side effects I hadn’t considered when we rolled out some fairly significant caching updates on GitHub in the beginning of January was how much Google’s crawler would take full advantage of the speed increase.
The graph lays it all out pretty clearly what happens when your site is more responsive. The number of pages the bot is able to index goes up dramatically when the time spent downloading the page is reduced dramatically. More pages indexed on Google inevitably means more revenue for us as our traffic grows, so this is a solid win beyond making sure existing GitHubbers are happy.
Written by
Related posts
Still a developer. Just outside. Our latest GitHub Shop collection is here.
The ESC collection lets you escape the confines of your desk and get out into the sun where good ideas are bound to happen.
Take your local GitHub sessions anywhere
Kick off work in VS Code or the CLI, finish it from your phone. Remote control for GitHub Copilot sessions is now generally available on github.com and GitHub Mobile.
GitHub availability report: April 2026
In April, we experienced 10 incidents that resulted in degraded performance across GitHub services.