Rotating credentials for GitHub.com and new GHES patches

On December 26, 2023, GitHub received a report through our Bug Bounty Program demonstrating a vulnerability which, if exploited, allowed access to credentials within a production container. We fixed this vulnerability on GitHub.com the same day and began rotating all potentially exposed credentials.

After running a full investigation, we assess with high confidence, based on the uniqueness of this issue and analysis of our telemetry and logging, that this vulnerability has not been previously found and exploited. While we are confident the impact was isolated to the bug bounty researcher, our procedures call for rotation of credentials in any event where they are exposed to a third-party out of an abundance of caution.

This vulnerability is also present on GitHub Enterprise Server (GHES). However, exploitation requires an authenticated user with an organization owner role to be logged into an account on the GHES instance, which is a significant set of mitigating circumstances to potential exploitation. A patch is available today—January 16, 2024—for GHES versions 3.8.13, 3.9.8, 3.10.5, and 3.11.3. We recommend that GHES customers apply the patch as soon as you are able.

Rotating credentials across our production systems caused a number of service disruptions between December 27 and 29. We recognize the impact these had on our customers that rely on GitHub and have improved our credential rotation procedures to reduce the risk of unplanned downtime going forward.

What you can do

The majority of these credential rotations are part of our normal operations and cause no customer impact. However, some keys that we are rotating now, as of the time this blog was published, may require some additional action.

GitHub commit signing key

What this key does

The private GitHub GPG commit signing key is used for signing commits you create on GitHub. These include commits created in the web editor, through a codespace, via the command line in a codespace, or via pull request operations.

You may be affected by this rotation if you are verifying GitHub-signed commits outside of GitHub or if you have a GitHub Codespace with commit signing enabled and have not pushed your commits from your codespace to your GitHub repository.

What we did

Starting today, January 16, 2024, all commits created on GitHub will be signed with a new GPG key. The previous GPG key expired on January 16, 2024 20:00:00 UTC.

Starting January 23, 2024, any newly uploaded commits signed with the previous key will no longer show as verified.

What you need to do

If you verify GitHub.com commits outside of GitHub, including for verification in GHES, you will need to import our new public key hosted here. We strongly recommend regularly pulling the public key to ensure you’re using the most current data from GitHub. This will also allow for seamless adoption of new keys in the future.

If you have a GitHub Codespace with commit signing enabled and have not pushed your commits from your codespace to your GitHub repository, you will need to push all commits created before January 16, 2024, from your codespace to your repository. This must be completed before January 23, 2024. After January 23, 2024, those older commits will no longer be marked verified unless you resign them, which can be done with the amend flag or equivalent. If you sign your commits using your own GPG key, then no actions are required.

GitHub Actions, GitHub Codespaces, and Dependabot customer encryption keys

What these keys do

These public keys are used by customers to encrypt GitHub Actions, GitHub Codespaces, and Dependabot secrets before sending them to GitHub via the API to store for subsequent usage in the product.

You may be affected by this rotation if you use the following endpoints and have cached or hardcoded the related public key(s):

GitHub Actions:

• Organization public key
• Repository public key
• GitHub Actions environment public key

GitHub Codespaces:

• Organization public key
• Repository public key
• GitHub Codespaces user public key

Dependabot:

• Organization public key
• Repository public key

What we did

We rotated these keys on January 16, 2024.

What you need to do

If you have hardcoded or cached the old public keys, you may notice an error message when you attempt to send new secrets to GitHub, where key_identifier is a unique identifier for the specific public key for your endpoint:

Provided key #{key_identifier} is not the latest version available. Call GET /secrets/public-key and resign the data using the latest key.

We strongly recommend regularly pulling the public keys from the API to ensure you’re using the most current data from GitHub. This will also allow for seamless adoption of new keys in the future.

Acknowledgement

We thank Ngo Wei Lin (@Creastery) of STAR Labs (@starlabs_sg) for participating in the GitHub Bug Bounty Program and practicing safe security research under the guidance of our rules of engagement. For others looking to participate in the GitHub Bug Bounty Program, you can submit any future bug reports here.

Written by

Jacob DePriest

@jacobdepriest

VP, Deputy CSO