codeql

GitHub code scanning helps open source maintainers and organizations find potential vulnerabilities in their code, before these can make their way into deployments. CodeQL, our very own analysis engine, powers the majority of those checks. Over the past few months, we have been working hard to improve the depth and breadth of our analysis to cover more CWEs, add support for a host of new language versions, and improve our platform compatibility.

Before we dive into the details: If you haven’t tried GitHub code scanning with CodeQL yet, you can enable it now on your repositories by following this guide! It’s free for open-source projects and available as part of GitHub Advanced Security for our enterprise customers.

All improvements below are available to users of GitHub code scanning on GitHub.com today, and will be part of the next GitHub Enterprise Server release (GHES version 3.5). Users of other GHES versions can also update their CodeQL version to benefit from these analysis improvements straight away.

Language Support

Today, CodeQL already supports JavaScript/TypeScript, Python, Ruby, Java, C#, Go, and C/C++. These languages are themselves under constant development, and we now support the following language versions:

• C# 10 / .NET 6,
• Python 3.10,
• Java 17, and
• TypeScript 4.5

The standard language features in those language releases are now fully supported by CodeQL.

Performance and Compatibility

For our Linux users, we have fixed an issue that caused the CodeQL CLI to be incompatible with systems running glibc version 2.34 and older.

For users of the CodeQL Apple Silicon support (beta), we are now bundling a native Java runtime for improved performance. Rosetta 2 and macOS Developer Tools are still required for other CodeQL components.

Security Coverage

The Common Weakness Enumeration (CWE) system is an industry-standard way of cataloging insecure software development patterns. CodeQL runs hundreds of queries out of the box that are able to detect an even greater number of CWEs. We went back through our existing queries, and aligned dozens of them with updated CWE IDs to give users better insight into the potential impact of a security issue when an alert is flagged up by code scanning.

We’ve added and improved detection for a large number of CWEs. These are the most significant changes:

• CWE-190 – Integer Overflow: The cpp/uncontrolled-arithmetic query for C/C++ detects potential user-controlled inputs to calculations that could produce an overflow condition
• CWE-319 – Cleartext Transmission of Sensitive Data: The cpp/cleartext-transmission query for C/C++ detects network transmissions of sensitive data without encryption
• CWE-120 – Buffer Overflow: The cpp/very-likely-overrunning-write query for C/C++ now detects cases of out-of-bounds writes based on advanced range analysis
• CWE-732 – Incorrect Permission Assignment for Critical Resource: The cpp/open-call-with-mode-argument (and optional cpp/world-writable-file-creation) query for C/C++ detect issues that could lead to stack memory disclosure or attacker-writable files
• CWE-295 – Improper Certificate Validation: The java/insecure-trustmanager query for Java now detects missing or lax certificate handling that could lead to man-in-the-middle attacks
• CWE-829 – Inclusion of Functionality from Untrusted Control Sphere: The js/insecure-dependency query for JavaScript/TypeScript detects dependency downloads over unencrypted communication channels
• CWE-347 – Improper Verification of Cryptographic Signature: The js/jwt-missing-verification query for JavaScript/TypeScript detects scenarios in which a JWT payload is not verified with a cryptographic secret or public key
• CWE-918 – Server-Side Request Forgery: SSRF detection queries for Python have been improved, and now differentiate between partially and fully (py/full-ssrf) user-controlled URLs

Behind the scenes, we’re also working on support for mobile application security, with additional support for Kotlin and Swift on our roadmap. In the meantime, we’ve also added more coverage for mobile security issues for our existing Java support:

• CWE-266 – Incorrect Privilege Assignment: The java/android/intent-uri-permission-manipulation query detects Intent URI permission manipulation in Android applications
• CWE-470 – Use of Externally-Controlled Input to Select Classes or Code: The java/android/fragment-injection and java/android/fragment-injection-preference-activity queries detect issues that could enable malicious applications to bypass access controls
• CWE-927 – Use of Implicit Intent for Sensitive Communication: The java/android/implicit-pendingintents query detects potential information disclosure through the use of PendingIntents

We’ve improved the depth of CodeQL's Python analysis by adding support for more libraries and frameworks, including:

• FastAPI
• aiomysql
• aiopg
• asyncpg
• Django REST framework
• The os.path module
• Flask-Admin
• toml
• ruamel.yaml
• SQLAlchemy

As a result, CodeQL can now detect even more potential sources of untrusted user data, steps through which that data flows, and potentially dangerous sinks in which this data could end up. This results in an overall improvement of the quality of the code scanning alerts.

We carefully choose and prioritize the libraries and frameworks supported by CodeQL based on their popularity and through user feedback. These improvements are now available to users of CodeQL code scanning on GitHub.com, and will also be available in the next release of GitHub Enterprise Server (3.4).

The latest release of the CodeQL CLI supports including markdown-rendered query help in SARIF files so that the help text can be viewed in the code scanning UI. This functionality is now available for code scanning on GitHub.com and will be available in GitHub Enterprise Server 3.4.

The CodeQL query help text is displayed in the code scanning UI whenever the query generates an alert. The query help explains the problem in more detail, and shows examples of vulnerable and fixed code. Until now, code scanning only displayed the query help for alerts generated by the default CodeQL queries. With the release of CodeQL CLI 2.7.1, the query help for your own custom queries will be uploaded to GitHub and displayed in code scanning.

Writing query help for custom CodeQL queries

When you write your own queries, we recommend that you write a query help file so that other users can properly understand the impact an alert has on the security of their code. For custom query help in your repository there are no restrictions on the content, but we recommend that you follow the Query help style guide to make the help text as useful as possible.

You should write query help for custom queries in your repository in a markdown file alongside the corresponding query. CodeQL code scanning looks for query help files written in markdown that share the same name as the corresponding query file. For example, if your query file is MyCustomQuery.ql, the query help file should be named MyCustomQuery.md.

For users of 3rd party CI/CD systems

When using CodeQL with GitHub Actions, the query help will automatically be imported from markdown files that are stored alongside the corresponding custom queries. The query help is inserted into SARIF files generated during the analysis step and made available in the code scanning UI.

If you use a different CI/CD system, you have to add the --sarif-add-query-help flag to the codeql database analyze command to include the query help in your SARIF results files. For more information, see Analyzing databases with the CodeQL CLI.

It's now easier to debug problems with CodeQL code scanning: an optional flag in the Actions workflow file will trigger diagnostic data to be uploaded as an artifact to your Actions run. To do this, you can modify the init step of your Actions workflow:

- name: Initialize CodeQL
  uses: github/codeql-action/init@v1
  with:
    debug: true

The data will be uploaded as an Actions artifact named debug-artifacts, attached to the workflow run. Such artifacts contain CodeQL logs, CodeQL databases, and the SARIF files that were produced.

These artifacts will help you when you're debugging problems with CodeQL code scanning. When contacting GitHub support, they might ask for this data too.

Learn more about Troubleshooting the CodeQL workflow.

We’ve improved the depth of CodeQL's analysis by adding support for more libraries and frameworks and increasing the coverage of our existing library and framework models. JavaScript analysis now supports most common templating languages, and Java now covers more than three times the endpoints of previous CodeQL versions. As a result, CodeQL can now detect even more potential sources of untrusted user data, steps through which that data flows, and potentially dangerous sinks in which this data could end up. This results in an overall improvement of the quality of the code scanning alerts.

We carefully choose and prioritize the libraries and frameworks supported by CodeQL based on their popularity and through user feedback. These improvements are now available to users of CodeQL code scanning on GitHub.com, and will also be available in the next release of GitHub Enterprise Server (3.3).

Java

We've improved coverage for the following libraries:

• Guava
• JAX-RS
• Java Standard Library
• Apache Commons Lang library
• Apache Commons Collections
• Spring
• JSON-java

JavaScript

We've added support for the following templating languages:

• Handlebars
• Mustache
• Nunjucks
• Hogan
• Swig
• EJS

Learn more about CodeQL and code scanning.

Developers and security researchers using the CodeQL CLI and VS Code extension can now build databases and analyze code on machines powered by Apple Silicon (e.g. Apple M1).

In order to use the CodeQL CLI and/or the VS Code extension on Apple Silicon, please make sure to install the Xcode command-line developer tools and Rosetta 2.

For detailed instructions on how to set up the CLI on supported platforms, please refer to the CodeQL CLI guide.

Learn more about CodeQL and code scanning.

Ruby is the 10th most popular language within the open source community. To help more open source maintainers and organizations find potential vulnerabilities in their code, we’ve added Ruby support (beta) to the CodeQL engine that powers GitHub code scanning.

Our CodeQL analysis identifies security issues in your code, along with the flow of data to the vulnerable location. To help secure services and tools created with Ruby, the CodeQL beta release spots many of the most common security issues:

• SQL injection (CWE-089)
• ReDoS (regular expression denial-of-service, CWE-1333)
• OS command and argument injection (A1:2017-Injection, CWE-078, CWE-088)
• XML entity expansion (CWE-611, CWE-827)
• Reflected cross-site scripting (XSS) (A7:2017-Cross-Site Scripting, CWE-079)
• Stored XSS (CWE-079, CWE-116)
• Unsafe deserialization (CWE-502)
• Hard-coded credentials (CWE-798)

We’ve been putting this beta through its paces by analyzing some of the world's largest Ruby codebases at GitHub and select customers. The feedback has been overwhelmingly positive, and in many cases CodeQL identified real vulnerabilities, all while keeping the number of false-positives at a minimum.

CodeQL for Ruby is available by default in GitHub.com code scanning, the CodeQL CLI, and the CodeQL extension for VS Code starting today. It will also be included in GitHub Enterprise Server 3.4. Ruby joins the list of supported CodeQL languages, which also includes C/C++, C#, Java, JavaScript/TypeScript, Python, and Go.

We currently support all common Ruby versions, up to and including 3.02. Check out the documentation for more details on compatibility.

To start using the new Ruby analysis, simply update your existing workflow file by adding Ruby to the language matrix:

jobs:
  analyze:
    name: Analyze
    ...
    strategy:
      fail-fast: false
      matrix:
        # add here
        language: ['ruby']

If you’re new to code scanning, set up a CodeQL analysis workflow from the Security tab in your repository.

Want to contribute or write your own CodeQL queries for Ruby? This guide will help you get started.

To give us feedback, join the Ruby beta discussion in the public CodeQL repository, which is also a good place to ask questions about anything CodeQL.

GitHub code scanning with CodeQL works seamlessly with GitHub Actions. For users of other CI/CD systems, we provided a way to run the code analysis using the CodeQL runner. The CodeQL runner acts as the glue between your CI/CD system and the CodeQL analysis engine (CLI).

In response to customer feedback, we have now transferred all CodeQL runner functionality natively into the CodeQL CLI. This means that there are fewer components to install, use, and update if you're running CodeQL code scanning on a 3rd party CI/CD system.

As of CodeQL CLI 2.6.2 (released today), there is now full feature parity between the CodeQL CLI and the runner. We are therefore announcing the deprecation of the CodeQL runner. This affects users of 3rd party CI/CD systems; users of CodeQL code scanning on GitHub Actions are not affected.

Deprecation timeline: GitHub Enterprise Server (GHES)

The CodeQL runner will continue to be part of the GitHub Enterprise Server releases up to and including the 3.3.x series. It will no longer be part of GitHub Enterprise Server starting the 3.4.0 (scheduled for 2022). The CodeQL CLI is a feature-complete replacement for the CodeQL runner, and is fully compatible with GitHub Enterprise Server.

Deprecation timeline: GitHub.com (including GitHub Enterprise Cloud)

The CodeQL bundle will continue to contain the CodeQL runner until March 2022. In the run-up to the final release we will start displaying clear deprecation warnings. After March 2022, code scanning API change on GitHub.com will likely not be compatible with older CodeQL runner releases. The CodeQL CLI is a feature-complete replacement for the CodeQL runner, and is fully compatible with GitHub.com.

How does this affect me?

If you’re using CodeQL code scanning on GitHub Actions, you are not affected by this change.

If you’ve configured code scanning to run the CodeQL runner inside another CI/CD system, we recommend migrating to the CodeQL CLI as soon as possible. Version 2.6.2 of the CodeQL CLI has full feature parity with the CodeQL runner and allows you to analyze codebases on 3rd party CI/CD systems in the same way as the CodeQL runner.

New users of CodeQL code scanning should use CodeQL CLI version >= 2.6.2.

What actions should I take?

You should configure your CI/CD system to use the CodeQL CLI before March 2022 and/or before upgrading to GHES 3.4.0. When setting up the CodeQL CLI, we recommend that you test the CodeQL CLI set up to verify that the CLI is correctly configured to create and analyze databases.

Learn more about how to migrate from the CodeQL runner to CodeQL CLI 2.6.2 here.

Code scanning with CodeQL now generates diagnostic information for all supported languages.

Before analyzing your code, CodeQL first creates a CodeQL database containing all of the important information about your codebase. This database is then analyzed by executing CodeQL queries against it.

The new diagnostic information includes important information to help learn more about the CodeQL analysis.

Diagnostic information includes:

• Lines of code in your codebase (as a baseline)
• Lines of code in the CodeQL database extracted from your code
• Lines of code in the CodeQL database excluding auto-generated files and external libraries
• Number of files successfully analyzed
• Number of files that generate extractor errors and warnings during database creation

You can see the detailed diagnostic information in GitHub CodeQL Actions Logs.

You also can view Lines of code in your codebase and Lines of code in the CodeQL database by going to the Security tab and selecting Code scanning alerts.

Diagnostic information queries are available in CodeQL CLI 2.5.6 and later. The CodeQL bundle includes both the CodeQL CLI and a compatible set of queries.