Keeping GoogleBot Happy
One of the interesting side effects I hadn’t considered when we rolled out some fairly significant caching updates on GitHub in the beginning of January was how much Google’s crawler…
One of the interesting side effects I hadn’t considered when we rolled out some fairly significant caching updates on GitHub in the beginning of January was how much Google’s crawler would take full advantage of the speed increase.
The graph lays it all out pretty clearly what happens when your site is more responsive. The number of pages the bot is able to index goes up dramatically when the time spent downloading the page is reduced dramatically. More pages indexed on Google inevitably means more revenue for us as our traffic grows, so this is a solid win beyond making sure existing GitHubbers are happy.
Written by
Related posts
Explore the best of GitHub Universe: 9 spaces built to spark creativity, connection, and joy
See what’s happening at Universe 2025, from experimental dev tools and career coaching to community-powered spaces. Save $400 on your pass with Early Bird pricing.
Agents panel: Launch Copilot coding agent tasks anywhere on GitHub
Delegate coding tasks to Copilot and track progress wherever you are on GitHub. Copilot works in the background, creates a pull request, and tags you for review when finished.
Q1 2025 Innovation Graph update: Bar chart races, data visualization on the rise, and key research
Discover the latest trends and insights on public software development activity on GitHub with the quarterly release of data for the Innovation Graph, updated through March 2025.