Three new Campus Experts are joining the fall 2022 batch of the MLH Fellowship to work with open source maintainers and get real-world experience.
We periodically see incidents on the registry where npm accounts are compromised by malicious actors and then used to insert malicious code into popular packages to which these accounts have access. Examples include the recent takeovers of the
rc packages. At times, these account takeover (ATO) events have involved npm accounts where two-factor authentication was not enabled.
For the last several months, the npm team has been investing in infrastructure and security improvements to automate monitoring and analysis of newly published versions of packages in order to identify malware and other malicious code in real time.
There are two main categories of malware publish events that occur in the npm ecosystem: malware that is published due to account takeovers and malware that is published by attackers through their own accounts. Even though high-impact account takeovers are relatively infrequent, when compared to direct malware published from attackers using their own accounts, account takeovers can be wide reaching when targeted at maintainers of popular packages. While our detection and response time to popular package takeovers has been as low as 10 minutes in recent incidents, we continue to evolve our malware detection capabilities and notification strategies toward a more proactive response model.
That said, these detection capabilities do not solve for the aforementioned account security challenges, which are core to improving the overall safety and security of the npm ecosystem. This is why we will begin to require two-factor authentication (2FA) during authentication for maintainers and admins of popular packages on npm, starting with a cohort of top packages in the first quarter of 2022. We are currently evaluating next steps to ensure that the strongest and most user-friendly authentication options, such as WebAuthn, are available and accessible to developers using npm. Expect to see updates from us in the coming weeks on the steps we’re taking to help developers adopt account security controls and adjustments to any impacted workflows.
Strong authentication and the use of 2FA have been recognized as best practice for many years, and the IT security space is undergoing a massive push toward zero trust architecture, which relies heavily on strong identity and user authentication. We increasingly feel that software development ecosystems must be in step with this push toward strong authentication as part of protecting the software supply chain. The consequences of not doing so, in an ecosystem as critical as npm, are far-reaching. We must push the bar for account security hygiene higher, and GitHub is committed to making strong account security easier to achieve and adopting the latest standards for the npm ecosystem.
As stewards of the registry, the security and trustworthiness of npm is crucial to all of us at GitHub, and we believe transparency is critical to maintaining that trust. Today, we are disclosing two recent security issues impacting the npm registry itself and the steps we’ve taken toward remediation.
First, on October 26 we identified an issue caused by routine maintenance of one of our publicly available npm services. During maintenance on the database that powers the public npm replica at
replicate.npmjs.com, records were created that could expose the names of private packages. This briefly allowed consumers of
replicate.npmjs.com to potentially identify the names of private packages due to records published in the public changes feed. No other information, including the content of these private packages, was accessible at any time. Package names in the format of
@owner/package for private packages created prior to October 20 were exposed between October 21 13:12:10Z UTC and October 29 15:51:00Z UTC. Upon discovery of the issue, we immediately began work on implementing a fix and determining the scope of the exposure. On October 29, all records containing private package names were removed from the replication database. While these records were removed from the
replicate.npmjs.com service on this date, the data on this service is consumed by third-parties who may have replicated the data elsewhere. To prevent this issue from occuring again, we have made changes to how we provision this public replication database to ensure records containing private package names are not generated during this process.
Second, on November 2 we received a report to our security bug bounty program of a vulnerability that would allow an attacker to publish new versions of any npm package using an account without proper authorization. We quickly validated the report, began our incident response processes, and patched the vulnerability within six hours of receiving the report.
We determined that this vulnerability was due to inconsistent authorization checks and validation of data across several microservices that handle requests to the npm registry. In this architecture, the authorization service was properly validating user authorization to packages based on data passed in request URL paths. However, the service that performs underlying updates to the registry data determined which package to publish based on the contents of the uploaded package file. This discrepancy provided an avenue by which requests to publish new versions of a package would be authorized for one package but would actually be performed for a different, and potentially unauthorized, package. We mitigated this issue by ensuring consistency across both the publishing service and authorization service to ensure that the same package is being used for both authorization and publishing.
This vulnerability existed in the npm registry beyond the timeframe for which we have telemetry to determine whether it has ever been exploited maliciously. However, we can say with high confidence that this vulnerability has not been exploited maliciously during the timeframe for which we have available telemetry, which goes back to September 2020. We’d like to thank Kajetan Grzybowski (DrBrix) and Maciej Piechota (haqpl) for reporting this vulnerability.
Whether it’s protecting users on the registry from account takeovers with 2FA, including npm in the GitHub security bug bounty program, or our partnership with the security community through the Open Source Security Foundation, we’re determined to continue to invest in the security of npm and the broader software security supply chain. We’ll be in touch in the coming weeks via this blog to share more on the rollout of 2FA requirements on npm in addition to sharing more about bringing WebAuthn support to the npm registry.