advanced-security

Subscribe to all “advanced-security” posts via RSS or follow GitHub Changelog on Twitter to stay updated on everything we ship.

~ cd github-changelog
~/github-changelog|main git log main
showing all changes successfully

We have started creating and storing CodeQL databases for the most popular open-source projects on GitHub.com. If you use CodeQL for security research, you can now obtain these databases easily and directly through the CodeQL extension for Visual Studio Code, which makes it much easier to write and run your own custom CodeQL queries.

Using CodeQL for security research

The CodeQL engine powers GitHub code scanning: it analyses source code and flags up potential security problems (for example, in pull requests). By default, code scanning runs a large set of open source queries that are able to identify the most important and common security problems.

CodeQL is also a powerful tool for variant analysis and other types of security research. CodeQL treats source code as data, and anyone can write custom CodeQL queries to explore a codebase and identify vulnerabilities. Like code search on steroids!

The first step of any CodeQL analysis is extracting the source code into a CodeQL database. This database contains a relational representation of the source code — including elements like the abstract syntax tree, the data flow graph, and the control flow graph. You can create CodeQL databases yourself using the CodeQL CLI, but with the feature we shipped today, it's much quicker to get started: you can download a ready-built CodeQL database from GitHub.com.

Downloading CodeQL databases from GitHub.com in VS Code

To download a CodeQL database for use in the CodeQL extension in VS Code:

  1. Make sure you have set up the CodeQL extension for VS Code. For more information, see Setting up CodeQL in Visual Studio Code.
  2. Open the CodeQL databases view in the extension.
  3. Hover over the sidebar, click the GitHub icon, and specify the owner/repo identifier of the public repository you'd like to analyze.

    image

Once you've downloaded a CodeQL database, you're ready to start your research. Find more information in the CodeQL documentation.

FAQs

How many CodeQL databases are available?

We currently store databases for over 200,000 repositories on GitHub.com. That list is constantly growing and evolving to make sure that it includes the most interesting codebases for security research.

What languages are can you download CodeQL databases for?

We create and store databases for all of the languages that we support in CodeQL code scanning. For more information, see About code scanning with CodeQL.

Can I download CodeQL databases outside VS Code?

Yes, you can also download CodeQL databases using the GitHub REST API. For more information, see Downloading databases from GitHub.com in the CodeQL CLI documentation.

Why is there no CodeQL codebase available for my favourite open source repository?

If there is a repository that you'd like to analyze, but a CodeQL database is not available yet, then you can trigger the creation (and storing) of a database by enabling GitHub code scanning with the CodeQL engine. Alternatively, you could fork the repository and enable code scanning on the fork. For more information, see the code scanning documentation.

See more

At the organization level, you can now view (GET) and update (PATCH) enablement status as well as configure the setting to automatically enable new repositories for the following GitHub security products:

  • Dependency graph
  • Dependabot alerts
  • Dependabot security updates

If you are a GitHub Advanced Security customer, you can also view and update the same settings for:

  • Secret scanning
  • Push protection

In addition, GitHub Advanced Security customers can view and update the enablement status for GitHub Advanced Security at the organization level.

Learn more about the organization REST API and send us your feedback

Learn more about GitHub Advanced Security

See more

The default code scanning query suites include checks for the most important security vulnerabilities for each supported language, so that any potential problems can be surfaced to developers before they are committed to their repository. However, in some situations a particular check is not relevant for a codebase and you might prefer to not run that CodeQL query. You can now easily exclude queries using code scanning query filters.

Query filters use the same syntax as CodeQL query suites and you can filter on any CodeQL query metadata property. Query filters must be specified in a custom code scanning configuration file, which you refer to from your code scanning analysis workflow file.

In your code scanning workflow file, use the config-file parameter of the init action to specify the path to the configuration file you want to use:

- uses: github/codeql-action/init@v2
  with:
    config-file: path/to/config/file.yml

In your configuration file, specify the query filters you want to use. For example, to exclude the Unsafe HTML constructed from library input query from the default code scanning query suite for JavaScript you can specify its id in an exclude block:

name: "My code scanning CodeQL config"

query-filters:
- exclude:
     id: js/html-constructed-from-input

For more information about how to use query filters, see Configuring code scanning in the code scanning documentation.

See more

GitHub Advanced Security customers using secret scanning can now specify a custom link that will show in the error message when push protection detects and blocks a potential secret. Admins can use the custom link to provide their developers with a point of reference on best practices with secrets.

Learn more about protecting pushes with secret scanning.

Custom link displayed in a push protection error message

See more

GitHub secret scanning protects users by searching repositories for known types of secrets. By identifying and flagging these secrets, our scans help prevent data leaks and fraud.

We have partnered with Prefect to scan for their access tokens and help secure our mutual users on public and private repositories. The Prefect service account API keys are not associated with a user and are restricted to a specific tenant, but they are recommended for application and automation use. GitHub will forward access tokens found in public repositories to Prefect, who will immediately email the owner of the leaked key. More information about Prefect API Tokens can be found here.

GitHub Advanced Security customers can also scan for Prefect tokens and block them from entering their private and public repositories with push protection.

See more

GitHub secret scanning protects users by searching repositories for known types of secrets. By identifying and flagging these secrets, our scans help prevent data leaks and fraud.

We have partnered with ReadMe to scan for their API keys and help secure our mutual users on public and private repositories. ReadMe’s API keys allow users to sync OpenAPI and Markdown files to their developer hubs using the rdme GitHub Action, as well as perform other programmatic updates using the ReadMe API. We’ll forward exposed API keys found in public repositories to ReadMe, who will immediately revoke the token and notify the project administrators via email. More information about ReadMe’s API keys can be found here.

GitHub Advanced Security customers can also scan for ReadMe tokens and block them from entering their private and public repositories with push protection.

See more

GitHub secret scanning protects users by searching repositories for known types of secrets. By identifying and flagging these secrets, our scans help prevent data leaks and fraud.

We have partnered with UNIwise to scan for their access tokens and help secure our mutual users on public and private repositories. The WISEflow API Key allows for institutions to manage key aspects of their license, such as exams and their life cycle. GitHub forwards access tokens found in public repositories to UNIwise, who will immediately disable the API Key and contact the customer. More information about WISEflow API Keys can be found here

GitHub Advanced Security customers can also scan for UNIwise tokens and block them from entering their private and public repositories with push protection.

See more

We’ve expanded access to GitHub’s security overview pages in two ways:

  1. All GitHub Enterprise accounts now have access to the security overview, not just those with GitHub Advanced Security
  2. All users within an enterprise can now access the security overview, not just admins and security managers

Security overview provides a centralized view of risk for application security teams, engineering leaders, and developers who work across many repositories. It displays code scanning, Dependabot, and secret scanning alerts across every repository you have access to in an organization or enterprise. The security overview also shows you where you have unknown risks because security features haven’t been enabled.

Learn more about security overview and send us your feedback

See more

It's now easier to debug CodeQL analysis problems in code scanning: click Re-run jobs from the GitHub Actions workflow run page, check the Enable debug logging box, and hit the Re-run jobs button.

Re-run all jobs

The data will be uploaded as an Actions artifact named debug-artifacts, attached to the workflow run. Such artifacts contain CodeQL logs, CodeQL databases, and the SARIF files that were produced.

Actions artifacts

These artifacts will help you when you're debugging problems with CodeQL code scanning. When contacting GitHub support, you might be asked for this data.

As part of the analysis, CodeQL extracts your source code into a relational database format. The debug artifacts include more detailed information about CodeQL extraction errors and warnings that occurred during database creation. If you want to permanently enable debug logging for the CodeQL analysis, or would like more information about troubleshooting CodeQL, please follow these instructions.

This feature is now available to all users on GitHub.com and will also be available in GitHub Enterprise Server 3.7.

See more

GitHub Advanced Security customers who are watching a repository's secret scanning alerts will now receive an email notification when a contributor bypasses a secret blocked by push protection. Previously, notifications were not sent if the secret was marked as a false positive or as used in tests.

Learn more about protecting pushes with secret scanning

See more

GitHub secret scanning protects users by searching repositories for known types of secrets. By identifying and flagging these secrets, our scans help prevent data leaks and fraud.

We have partnered with Zuplo to scan for their API keys connected to a Zuplo API Gateway, which allows users to add API key authentication to their APIs. We'll forward access tokens found in public repositories to Zuplo, who follow customer preference to either notify their customers via email or automatically revoke the token. More information about Zuplo API tokens can be found here.

We continue to welcome new partners for public repository secret scanning. GitHub Advanced Security customers can also scan their private repositories for leaked secrets and prevent Zuplo keys from accidental leaks with push protection.

See more

GitHub Advanced Security customers can now use cursors to paginate over alert results they retrieve via the repository and organization level REST APIs.

Paginating with cursors, using the new before and after query parameters, can help assure data consistency and improve response times. To receive an initial cursor on your first request, include an empty "before" or "after" query string in your API call.

Learn more about the secret scanning REST API
Learn more about private repository scanning with Advanced Security

See more

GitHub Advanced Security customers can now see an overview of code scanning alerts at the enterprise level. This page provides a repo-centric view of application security risks, as well as an alert-centric view of all secret scanning, Dependabot and now code scanning alerts. This view is beta and will be followed in the coming weeks with an enterprise level REST API to retrieve code scanning alerts.

Code scanning alerts at the enterprise level

Learn more about security overview
Learn more about GitHub Advanced Security

See more

GitHub secret scanning protects users by searching repositories for known types of secrets. By identifying and flagging these secrets, we help protect users from data leaks and fraud associated with exposed data.

We have partnered with SendGrid to scan for their access tokens, which allow users to retrieve account information and statistics. We'll forward access tokens found in public repositories to SendGrid. SendGrid will then either suspend the detected token or send it to their fraud team for manual review, depending on the token scope. More information about SendGrid API tokens can be found here.

GitHub Advanced Security customers can also scan for SendGrid's API keys and block them from entering their private and public repositories via secret scanning’s push protection feature.

Learn more about secret scanning
Partner with GitHub on secret scanning

See more

GitHub Advanced Security customers can now perform dry runs of their custom patterns when editing a pattern. Dry runs allow admins to understand a pattern's impact across an organization and to hone the pattern before publishing and generating alerts.

Admins can compose a new pattern or edit a published pattern then 'Save and dry run' to retrieve results from their selected repositories. Scan results will appear on screen as they're detected, but admins can leave the page and later come back to their saved pattern's dry run results.

For more information:

See more