GitHub Availability Report: January 2021
Introduction In January, we experienced one incident resulting in significant impact and degraded state of availability for the GitHub Actions service. January 28 04:21 UTC (lasting 3 hours 53 minutes)…
![](https://github.blog/wp-content/uploads/2021/04/Blog_ENGINEERING_for-social.png?resize=1200%2C630)
Introduction
In January, we experienced one incident resulting in significant impact and degraded state of availability for the GitHub Actions service.
January 28 04:21 UTC (lasting 3 hours 53 minutes)
Our service monitors detected abnormal levels of errors affecting the Actions service. This incident resulted in the failure or delay of some queued jobs for a period of time. Jobs that were queued during the incident were run successfully after the issue was resolved.
We identified the issue as caused by an infrastructure error in our SQL database layer. The database failure impacted one of the core microservices that facilitates authentication and communication between the Actions microservices, which affected queued jobs across the service. In normal circumstances, automated processes would detect that the database was unhealthy and failover with minimal or no customer impact. In this case, the failure pattern was not recognized by the automated processes, and telemetry did not show issues with the database, resulting in a longer time to determine the root cause and complete mitigation efforts.
To help avoid this class of failure in the future, we are updating the automation processes in our SQL database layer to improve error detection and failovers. Furthermore, we are continuing to invest in localizing failures to minimize the scope of impact resulting from infrastructure errors.
In summary
We’ll continue to keep you updated on the progress we’re making on ensuring reliability of our services. To learn more about how teams across GitHub identify and address opportunities to improve our engineering systems, check out the GitHub Engineering blog.
Tags:
Written by
Related posts
![](https://github.blog/wp-content/uploads/2023/10/Collaboration-DarkMode-2.png?resize=400%2C212)
2024 is the biggest global election year in history. What’s at stake for developers?
GitHub is considering what is at stake for our users and platform, how we can take responsible action to support free and fair elections, and how developers contribute to resilient democratic processes.
![](https://github.blog/wp-content/uploads/2023/10/AI-DarkMode-2.png?resize=400%2C212)
GitHub named a Leader in the Gartner first-ever Magic Quadrant for AI Code Assistants
This year, as part of its annual Magic Quadrant series, Gartner published a first-of-its-kind report analyzing the state of play in the AI Code Assistants market–and named GitHub a Leader.
![AI in software development 2024 survey results with colorful graphics and percentages.](https://github.blog/wp-content/uploads/2024/08/ai-survey-header.png?resize=400%2C212)
Survey: The AI wave continues to grow on software development teams
We surveyed 2,000 people on software development teams at enterprises in the U.S., Brazil, India, and Germany about the use, experience, and expectations around generative AI tools in software development.