Using GitHub Actions for MLOps & Data Science

Background

Machine Learning Operations (or MLOps) enables Data Scientists to work in a more collaborative fashion, by providing testing, lineage, versioning, and historical information in an automated way.  Because the landscape of MLOps is nascent, data scientists are often forced to implement these tools from scratch. The closely related discipline of DevOps offers some help, however many DevOps tools are generic and require the implementation of “ML awareness” through custom code. Furthermore, these platforms often require disparate tools that are decoupled from your code leading to poor debugging and reproducibility.

To mitigate these concerns, we have created a series of GitHub Actions that integrate parts of the data science and machine learning workflow with a software development workflow. Furthermore, we provide components and examples that automate common tasks.

An Example Of MLOps Using GitHub Actions

Consider the below example of how an experiment tracking system can be integrated with GitHub Actions to enable MLOps. In the below example, we demonstrate how you can orchestrate a machine learning pipeline to run on the infrastructure of your choice, collect metrics using an experiment tracking system, and report the results back to a pull request.

A screenshot of this pull request.

For a live demonstration of the above example, please see this talk.

MLOps is not limited to the above example. Due to the composability of GitHub Actions, you can stack workflows in many ways that can help data scientists. Below is a concrete example of a very simple workflow that adds links to mybinder.org on pull requests:

name: Binder
on: 
  pull_request:
    types: [opened, reopened]

jobs:
  Create-Binder-Badge:
    runs-on: ubuntu-latest
    steps:

    - name: checkout pull request branch
      uses: actions/checkout@v2
      with:
        ref: ${{ github.event.pull_request.head.sha }}

    - name: comment on PR with Binder link
      uses: actions/github-script@v1
      with:
        github-token: ${{secrets.GITHUB_TOKEN}}
        script: |
          var BRANCH_NAME = process.env.BRANCH_NAME;
          github.issues.createComment({
            issue_number: context.issue.number,
            owner: context.repo.owner,
            repo: context.repo.repo,
            body: `[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/${context.repo.owner}/${context.repo.repo}/${BRANCH_NAME}) 👈 Launch a binder notebook on this branch`
          }) 
      env:
        BRANCH_NAME: ${{ github.event.pull_request.head.ref }}

When the above YAML file is added to a repository’s .github/workflow directory, pull requests can be annotated with a useful link as illustrated below [1]:

A Growing Ecosystem of MLOps & Data Science Actions

There is a growing number of Actions available for machine learning ops and data science. Below are some concrete examples that are in use today, categorized by topic.

Orchestrating Machine Learning Pipelines:

• Submit Argo Workflows – allows you to orchestrate machine learning pipelines that run on Kubernetes.
• Publish Kubeflow Pipelines to GKE – Kubeflow Pipelines is a platform for building and deploying portable, scalable machine learning (ML) workflows based on Docker containers.

Jupyter Notebooks:

• Run parameterized Notebooks – run notebooks programmatically using papermill.
• Repo2Docker Action – Automatically turn data-science repositories into Jupyter-enabled Docker containers using repo2docker.
• fastai/fastpages – share information from Jupyter notebooks as blog posts using GitHub Actions & GitHub Pages.

End-To-End Workflow Orchestration:

• Examples and templates for utilizing Azure Machine Learning from GitHub Actions.

Experiment Tracking

• Fetch runs from Weights & Biases – W&B is an experiment tracking and logging system for machine learning and is free for open-source projects.

This is by no means an exhaustive list of the things you might want to automate with GitHub Actions with respect to data science and machine learning.   You can follow our progress towards this goal on our page, which contains links to blog posts, GitHub Actions, talks, and examples that are relevant to this topic.

We invite the community to create other Actions that might be useful for the community. Some ideas for getting started include data and model versioning, model deployment, data validation, as well as expanding upon some of the areas mentioned above. A great place to start is the documentation for GitHub Actions, particularly on how to build Actions for the community!

Related Materials

• Our page with relevant materials.
• GitHub Actions official documentation.
• Hello world Docker Action: A template to demonstrate how to build a Docker Action for other people to use
• Using self-hosted runners.
• This talk introducing Actions for data science, including some live-coding!
• Awesome Actions: A curated list of interesting GitHub Actions by topic
• Useful GitHub Actions to know about when getting started:
  • actions/checkout: Allows you to quickly clone the contents of your repository into your environment, which you often want to do. This does a number of other things such as automatically mount your repository’s files into downstream Docker containers.
  • mxschmitt/action-tmate: Provides a way to debug Actions interactively. This uses port forwarding to give you a terminal in the browser that is connected to your Actions runner. Be careful not to expose sensitive information if you use this.
  • actions/github-script: Gives you a pre-authenticated ocotokit.js client that allows you to interact with the GitHub API to accomplish almost any task on GitHub automatically. Only these endpoints are supported.

Footnotes:

[1] This example workflow will not work on pull requests from forks.  To enable this, you have to trigger a PR comment to occur via a different event.

Written by

Hamel Husain

@hamelsmu