Database Troubles
This morning we had some database problems. With help from Engine Yard we were able to identify the slower queries and eliminate them. Here you can see our database exploding:…
This morning we had some database problems. With help from Engine Yard we were able to identify the slower queries and eliminate them.
Here you can see our database exploding:
img http://img.skitch.com/20090130-x5dfywbure73aqfjt3bqspckf3.png
And the sweet release when we deployed the fix:
img http://img.skitch.com/20090130-rkcn1b4d566f2x226f46deuq3.png
Basically, our events
table has swelled to over 30m rows taking up 18gb of disk space. We have some solid indexes in place, but saving new events and querying are both becoming problems. As we plan to do more fancy things with the dashboard in the future this needs to be addressed.
Currently the plan is twofold:
-
Split
events
table intoevents
andevents_archive
. Older events will go intoevents_archive
, archiving will happen nightly on a cron. This will dramatically reduce the size of theevents
table. -
Re-evaluate every query that touches
events
, make sure it’s using the best possible index.
We’ll update you as we make progress. Thanks!
Written by
Related posts

Your guide to GitHub Universe 2025: The schedule just launched!
Create your own agenda of favorites, sign up for one-on-on mentoring sessions, and register if you haven’t already. We’ll see you there!

Explore the best of GitHub Universe: 9 spaces built to spark creativity, connection, and joy
See what’s happening at Universe 2025, from experimental dev tools and career coaching to community-powered spaces. Save $400 on your pass with Early Bird pricing.

Agents panel: Launch Copilot coding agent tasks anywhere on GitHub
Delegate coding tasks to Copilot and track progress wherever you are on GitHub. Copilot works in the background, creates a pull request, and tags you for review when finished.