Building organization-wide governance and re-use for CI/CD and automation with GitHub Actions

Setting the scene

A strong focus on automation can reap benefits, particularly in your development workflow and DevOps lifecycle. We know that continuous integration (CI) can help accelerate development and enhance overall quality, while continuous delivery (CD) and continuous deployment (CD) can help reduce time-to-production and test for quality in live environments (for example, API fuzzing, infrastructure-as-code [IaC] validation, performance and load testing, and much more).

While many companies are continuing along their DevOps journey, there isn’t one clear path to adopting these principles. In fact, there are usually multiple application teams creating similar build and deployment pipelines in parallel. These teams operate at different maturity levels (that is, how rigorous their quality gates are; from simple builds and linting, through to high levels of test coverage and automated tests in a live environment). In some cases, a team might be managing a separate tool, such as Jira, TeamCity or similar. In others, teams may be standardized on an underlying platform but choose alternate approaches, such as using a Command Line Interface (CLI) in Bash or PowerShell, or some in-platform ‘helper’ approach (such as GitHub Actions, Azure DevOps Tasks or similar).

This potential disparity brings several challenges in a business setting:

• The multitude of platforms adopted typically causes significant operational overhead and challenges in consolidating your toolkit.
• Teams are likely reinventing the wheel. By analyzing the languages, frameworks, target platforms and deployment approaches used, you’ll see patterns emerge.
• Quality is likely not well-governed across an organization. This can introduce risk when pushing to production. How can you be certain whether a component has been rigorously performance tested, or pushed to production after only checking for a successful build?
• In high-regulatory environments, you may need to demonstrate compliance against certain checks. With a decentralized CI/CD model, attesting compliance is challenging to justify. As a result, it requires a significant amount of work to ensure compliance across application teams.

So, with the groundwork laid out, what are potential solutions? Fortunately, GitHub Actions has a few features that may be able to help.

How can GitHub help?

GitHub Actions is GitHub’s answer to automation and CI/CD, with the ability to trigger based on several GitHub events. Alongside a rich ecosystem of community and third-party actions, the platform provides a number of primitives to assist you in governing your workflows.

Let’s explore the platform features available to help govern CI/CD at scale across your company.

Branch protection rules and required status checks

With GitHub Actions, your workflows are stored as source in your repository alongside your code. That means they can benefit from the same governance practices as your other code.

When writing code, you typically want to follow a consistent process to bring changes to your codebase. Ideally, there would be a set of quality gates that must be passed before the changes can be brought into production. This is where branch protection rules come in. They allow you to enforce standards, so that quality can be maintained.

Protection rules can be combined with status checks to ensure that your code is meeting a set of conditions.

These checks could include tasks like test execution, build verification or validating that no new security vulnerabilities have been brought into the project. Required status checks can be mandated on a repository by using branch protection rules, encouraging practices that lead to higher quality code being pushed to production.

Reusable workflows

Instead of repeating the same set of steps across multiple workflows, you can define them once in a reusable workflow. And, well, reuse them!

Think of them like a function in software, which is generic and reusable. Or, if you’re familiar with IaC, think of reusable workflows like a template, acting as a “cookie cutter” for different patterns of your workflow.

With that in mind, it’s typical to see a reusable workflow take several parameters and use those to determine the action (pun intended!) within the workflow. As an example, below is a reusable workflow, which:

• Takes a config-path as an input string (think of this as a parameter to a function).
• Passes envPat as a named secret value. GitHub Actions recognizes that this value should be obfuscated.
• Executes the workflow on a GitHub-hosted Ubuntu runner. This workflow runs the actions/labeler GitHub Action using the provided inputs and secrets.

on:
  workflow_call:
    inputs:
      config-path:
        required: true
        type: string
    secrets:
      envPAT:
        required: true
jobs:
  reusable_workflow_job:
    runs-on: ubuntu-latest
    environment: production
    steps:
    - uses: actions/labeler@v4
      with:
        repo-token: ${{ secrets.envPAT }}
        configuration-path: ${{ inputs.config-path }}

The above reusable workflow shows a workflow that could be scaled across a company for repeated use. An application team would pass in the relevant secrets and define their own configuration file (in a path of their choosing, as opposed to a predefined path), while executing the needed checks and balances.

Application teams can consume the reusable workflow in their own workflows, using a snippet similar to the example below:

jobs:
  call-workflow-passing-data:
    uses: octo-org/example-repo/.github/workflows/reusable-workflow.yml@main
    with:
      config-path: .github/labeler.yml
    secrets:
      envPAT: ${{ secrets.envPAT }}

This raises a question. When might you want to share workflows? First, you’ll want to consider how broadly to share them. Are some business-specific (for example, relating to the processes of a business unit), or are some company policies and should be reused throughout?

Division-wide

These workflows could be shaped by a business unit’s processes, a given business unit choosing a specific cloud provider, division-wide governance policies, or numerous other scenarios.

Consider creating a repository for the purpose of sharing workflows across the division. That way, application teams in the division can consume from the centrally maintained repository. If any workflows are sensitive, then you could consider creating a private repository, and only sharing as needed.

Company-wide

Some workflows may warrant sharing across the entire company, for example, companywide policies or practices. This makes sense for scenarios where a business wants to provide paved paths that include built-in guard rails.

Consider a business that has adopted common languages or similar target deployment platforms across teams. They may want to provide templatized workflows that include unit testing and linting as standard. Or, performance testing to some standard endpoints for a given cloud provider’s hosting platform.

In essence, the scope of commonality for these workflows exists at the company level. You could once again consider a repository for the purposes of sharing these reusable assets across the company.

Required workflows

Required workflows were recently released in public beta. They ensure that a specified workflow is executed in a pull request (appearing as a status check), and are configured at the GitHub organization scope. This is useful when you want to take a step further than empowerment, and mandate that specific steps are completed.

As an example, consider the rollout of a security scanning tool. To ensure that all teams are scanning for security issues in their projects before merging code to production, you could consider making this a required workflow.

However, it’s worth considering the tradeoff when adopting this approach. How opinionated and imposing do you want to be on application teams that are using GitHub Actions? Alternatively, how much do you want to empower those teams to choose the appropriate reusable workflows for their scenario?

This decision will depend on your team’s risk appetite and whether cultural norms would allow for standardization of practices across the company.

GitHub Actions Importer

We know that adopting CI/CD is not as simple as creating a new workflow. In many cases, you already have incumbent tooling (perhaps multiple) to complete your DevOps automation needs. Fortunately, GitHub has released the GitHub Actions Importer.

This tooling helps you to migrate pipelines from Azure DevOps, CircleCI, GitLab, Jenkins, and Travis CI. While not completely foolproof, the team have found that on average over 90% of tasks and constructs used in a workflow have been successfully converted. In case you missed it, the tooling became generally available last month (March)!

In real terms, this can help accelerate your adoption of GitHub Actions. In turn, you could then convert those workflows into reusable workflows, and share your commonly-used recommended patterns across the organization, benefitting your wider engineering community.

After all, that’s exactly what innersource is about: contributing to the success of others, and building on the work of others!

What guardrails can you put in place?

For some organizations and engineering leaders, the prospect of reusing CI/CD and automation practices across the company may seem daunting. However, there are several considerations to help mitigate risk while ensuring your teams can continue innovating and being successful.

Permissions and secrets

To deploy your application to an environment, such as Azure, AWS, GCP, or on-premises, you must grant some level of access to your CI/CD platform. This typically means providing passwords, or certificates and having robust operational procedures to manage those.

But what if you didn’t have to worry about secrets at all? What if you could deploy to a target deployment environment without using a secret? Fortunately, that is possible using GitHub Actions and OpenID Connect (OIDC). Check out my extensive blog post on the topic.

Take a moment and think about how you can combine this with the concepts we’ve explored so far. Your application teams can depend on a number of reusable workflows shared internally. Employees can submit a pull request to enhance those workflows, which would be reviewed by (and benefit) the wider community. Those reusable workflows may contain actions that leverage GitHub Action’s OIDC capabilities and remove the need for passwords.

In other words, you are starting to pull together a series of templates that bring together recommended practices from across the organization and reduce the toil by removing credentials and passwords as a requirement (and therefore the need for password rotations and similar). This is a win for application and operations teams.

Manage the permissions of your workflows

Each GitHub Action workflow run is executed with a set of permissions so that it can interact with GitHub Services. For example, pushing a new package to GitHub Packages. We recently updated the default permissions so that it is read-only by default. However, you can explicitly set these permissions in your workflow’s YAML definition.

Governance: allow/deny specific actions and reusable workflows

Organizations typically have rigorous component governance processes in place, helping them understand the dependencies they have adopted in the software they build. But how do you govern the use of GitHub Actions and reusable workflows?

At the enterprise level, you can use policies to restrict the use of GitHub Actions, specifying whether all actions and reusable workflows are allowed, only those within the enterprise, or a more controlled list.

When selecting the option “Allow enterprise, and select non-enterprise, actions and reusable workflows,” you can access further granularity. This includes only allowing actions created by GitHub, marketplace actions by verified creators, and allowing specified actions and reusable workflows. You can find more information about these governance options in GitHub Docs.

These same policies can be set at the organization level, or you can set GitHub Actions Permissions at the repository level. This allows you to empower decision-making at the level which makes most sense in your company.

Dependencies: keeping GitHub Actions up to date

Just like open source dependencies, it’s important to keep your GitHub Actions up to date. When using a GitHub Action, you can specify a version number, which ties to a Git commit ID, branch name, or version number associated with a Git tag.

If there is a newer version, Dependabot can generate a pull request to update your GitHub Action workflow and point to the most recent version.

Find out more about keeping your actions up to date with Dependabot.

Wrap-up

Automation in the developer lifecycle is critical to accelerating delivery, maintaining quality, and delivering value to your users. However, silos across businesses can prevent teams from collaborating effectively.

GitHub Enterprise and GitHub Actions can help bring teams together to share internal CI/CD best practices. We have also explored several opportunities to establish policies and governance at scale by building paved paths or guardrails using reusable workflows. This allows you to set teams up for success and empower them to do their best work, while adopting recommended procedures across the organization.