GitHub data, ready for you to explore with BigQuery
GitHub data is available for public analysis using Google BigQuery, and we’d like to help you take it for a spin. If you’d like to find out more about what…
![](https://github.blog/wp-content/uploads/2017/01/a09618e8-de3d-11e6-9e4e-8f964270dbbe.png?resize=1200%2C630)
GitHub data is available for public analysis using Google BigQuery, and we’d like to help you take it for a spin.
If you’d like to find out more about what data is available and how it’s been used so far, watch this conversation between GitHub Data Analyst Alyson La and Google Developer Advocate Felipe Hoffa. You’ll learn the story behind the datasets and what types of analysis they make possible. You’ll also see how we’ve visualized data with Tableau and Looker.
There’s a lot of data out there, but it’s all available through BigQuery in two large data sets. The original, community-led GitHub Archive project launched in 2012 and captures almost 30 million events monthly, including issues, commits, and pushes. Last year, we worked with Google to release The GitHub Public Data Set, separate tables with information on all projects that have open source licenses, including commits, file contents, and file paths.
You can also use the GH torrent project to complement the existing datasets with additional metadata.
We ran a list of queries on the datasets above to create the open source section of our Octoverse report, but anyone can run an analysis. Here are the results of some of the queries run so far.
- “This should never have happened” has appeared in code comments more than a million times (hear this data point for yourself in this Changelog episode)
- Where does open source happen? GitHub top countries shares which countries have the most open source developers per capita
- How reliable is GitHub? Felipe runs a query to find out in GitHub reliability with BigQuery
- There are a lot of feels in open source. Geeksta examines how emotions are expressed in GitHub commit messages
- Are bigger pull requests better? Jessie Frazelle analyzed the top 15 projects on GitHub in terms of pull requests opened vs. pull requests closed
Happy exploring!
Tags:
Written by
Related posts
![](https://github.blog/wp-content/uploads/2023/10/Collaboration-DarkMode-2.png?resize=400%2C212)
2024 is the biggest global election year in history. What’s at stake for developers?
GitHub is considering what is at stake for our users and platform, how we can take responsible action to support free and fair elections, and how developers contribute to resilient democratic processes.
![](https://github.blog/wp-content/uploads/2023/10/AI-DarkMode-2.png?resize=400%2C212)
GitHub named a Leader in the Gartner first-ever Magic Quadrant for AI Code Assistants
This year, as part of its annual Magic Quadrant series, Gartner published a first-of-its-kind report analyzing the state of play in the AI Code Assistants market–and named GitHub a Leader.
![AI in software development 2024 survey results with colorful graphics and percentages.](https://github.blog/wp-content/uploads/2024/08/ai-survey-header.png?resize=400%2C212)
Survey: The AI wave continues to grow on software development teams
We surveyed 2,000 people on software development teams at enterprises in the U.S., Brazil, India, and Germany about the use, experience, and expectations around generative AI tools in software development.