This Morning’s Git Protocol Outage
Anonymous Git clones were offline this morning for several hours (SSH and HTTP clones remained functional). Several days ago we removed the client inactivity timeout from our backend HAProxy configuration…
Anonymous Git clones were offline this morning for several hours (SSH and HTTP clones remained functional). Several days ago we removed the client inactivity timeout from our backend HAProxy configuration to prevent long-running RPC calls from incorrectly being severed. It turns out that some Git connections will behave badly and sit idle keeping a connection open without any data transfer. Without the timeout in place these connections eventually consume every spot within HAProxy’s session limit, meaning that any subsequent connections will be dropped. We have reinstated the timeouts and will carefully review any changes we need to make here to ensure this does not happen again.
At the same time, our monitoring of anonymous Git clones is lacking, so we were not notified of the problem as soon as we should have been. I’m now in the process of putting in place a more robust monitoring solution for the Git protocol so that in case there are problems in the future we will be notified in a timely fashion.
I’m very sorry about the problems this morning. We take all matters of availability very seriously and will do everything we can to make sure these problems do not arise again.
Written by
Related posts
![](https://github.blog/wp-content/uploads/2023/10/Collaboration-DarkMode-2.png?resize=400%2C212)
2024 is the biggest global election year in history. What’s at stake for developers?
GitHub is considering what is at stake for our users and platform, how we can take responsible action to support free and fair elections, and how developers contribute to resilient democratic processes.
![](https://github.blog/wp-content/uploads/2023/10/AI-DarkMode-2.png?resize=400%2C212)
GitHub named a Leader in the Gartner first-ever Magic Quadrant for AI Code Assistants
This year, as part of its annual Magic Quadrant series, Gartner published a first-of-its-kind report analyzing the state of play in the AI Code Assistants market–and named GitHub a Leader.
![AI in software development 2024 survey results with colorful graphics and percentages.](https://github.blog/wp-content/uploads/2024/08/ai-survey-header.png?resize=400%2C212)
Survey: The AI wave continues to grow on software development teams
We surveyed 2,000 people on software development teams at enterprises in the U.S., Brazil, India, and Germany about the use, experience, and expectations around generative AI tools in software development.