Measuring enterprise developer productivity

In a recent paper written by Nicole Forsgren and her colleagues, “The SPACE of developer productivity: There’s more to it than you think,” there is an irony that is hard to escape. If you ask any individual whether they had a “productive day” they can almost certainly tell you, objectively, whether their work day was productive. It was a simple task for that individual to report whether they had reached their flow during that day, and, possibly depending on their mood, reported a high level of productivity. Now, try applying that same question to the diverse teams and organizations across your business. You will find (or have found) that answering this question, which is simple at a scale of 1, does not scale linearly’ in fact the complexity may scale exponentially.

The authors assert that by measuring developer productivity using a multidimensional framework, SPACE, we can more accurately measure productivity and make better decisions, and capture insight into the many layers of organizational, team and developer productivity. The SPACE framework presents five categories important to consider when measuring productivity.

S – Satisfaction & Well Being
P – Performance
A – Activity
C – Collaboration & Communication
E – Efficiency & Flow

No single measure can be taken alone to draw conclusions about productivity. Teams and organizations should leverage at least three of the dimensions to gain deeper insight into how your teams are working. Here are just a few examples of how the SPACE productivity framework can be used with GitHub tools today.

Satisfaction

At GitHub, we strive to keep the developer at the heart of everything we do. Satisfied developers deliver additional value to end users by writing higher quality, more secure software faster. It’s no surprise that “Research has shown that times of high productivity are also highly correlated with feeling more satisfied and happy with work” (Forsgren et al. 2021). We encourage you to survey your developers and teams to understand their level of satisfaction individually and as an aggregate. Try using a GitHub Action to open issues across your repositories with a link to an anonymous form (sneak peek: Issue Forms may be available later this year) and tagging your development teams for insight. If that doesn’t generate enough responses, consider leveraging Microsoft Teams or Slack to prompt a poll.

Performance

“Performance is the outcome of a system or process.” (Forsgren et al. 2021). In 2021, with the advent of GitOps, DevOps and DevSecOps we may be primed to measure developer output across systems. There can be many factors that impact the health of your systems, so it may be problematic to directly correlate the health of your systems to the team that develops and maintains them. However, consider recording the quality of your software through pipeline automation which tests against service disruption, bug reports, application performance and security incidents. Developers are not always in control of the features they are tasked with implementing—so be wary of relying too heavily on any single metric such as feature usage. We may broadly generalize Customer Satisfaction scores and tie that back to team or organizational performance. If quality can be deduced to application health, integrating an automated health check in your deployment or monitoring pipelines could be a useful activity. I’ve attached a sample workflow step below.

steps:
  - name: Check the deployed service URL
    uses: jtalk/url-health-check-action@v1.4
    with:
      # Check the following URLs one by one sequentially
      url: https://example.com|http://example.com
      # Follow redirects, or just report success on 3xx status codes
      follow-redirect: no # Optional, defaults to "no"
      # Fail this action after this many failed attempts
      max-attempts: 3 # Optional, defaults to 1
      # Delay between retries
      retry-delay: 5s # Optional, only applicable to max-attempts > 1

In CY 21, we are adding additional audit log events that will provide greater insight into your pipeline health and performance. Consider exporting and analyzing that data for trends such as process performance, uptime metrics and pipeline execution frequency.

Activity

Activity is the most ubiquitous productivity measure and often the most misused. “Activity is a count of actions or outputs completed in the course of performing work.” (Forsgren et al. 2021). GitHub Enterprise Cloud produces a plethora of events as your development teams collaborate on software delivery. If you aren’t already consuming platform Audit Log events, consider integrating this simple Enterprise Audit Log CLI with a cron workflow to export Git and other events into your preferred reporting tool. We plan to provide a more native streaming mechanism in the near future. If you are already consuming platform level events across your organization, consider leveraging a tool like this pull request stats action, which is designed to reduce the time taken to review pull requests, encourage quality on reviews and help decide which people to assign as reviewers.

Communication & Collaboration

The Open Source community that works on GitHub and the many enterprises that consume or contribute back to open source are a great example of the potential for increased productivity through communication and collaboration across organizations. It’s no surprise that most enterprises have caught on to this accelerator, our data tells us that > 95% of organizations consume open source. At GitHub, we often talk about modeling the open source philosophy inside our customers organizations or Innersourcing. Active code sharing with internal repositories, discussions and strategic technical alignment are all practices deeply tied to the innersourcing model. Consider launching an innersource initiative within your business to maximize developer productivity and software delivery. If you are already on the innersource journey it might be useful to explore metrics such as project onboarding time for new and existing members, project discoverability, and review quality. Codespaces make it easier for developers to onboard to a new company or start contributing to an open source project. Project maintainers can configure a repository so that when you create a codespace for the repository the project’s dependencies are included automatically. You can start coding faster by reducing time spent configuring your environment.

Efficiency & Flow

Developers often describe flow as an uninterrupted period of time when they are able to focus on delivering tasks related to project work items. Achieving a prolonged flow means more task completion in a day and greater overall satisfaction with work, thus a virtuous cycle of productivity. “Certain aspects of efficiency and flow may be hard to measure but often it is possible to spot and remove inefficiencies in the value stream.”(Forsgren et al. 2021). Our task is to create an environment where developers can achieve and maintain flow for the greatest amount of time every day while helping them feel satisfied with their daily activities. It may be useful to consider including questions around perceived efficiency and flow in your satisfaction survey. Absent individual perception, it could also be prudent to understand the amount of time a developer spends in their IDE actually pseudocoding and developing features. Keep in mind that time spent in an IDE alone is not a tell-all for developer productivity. Brainstorming, code reviews and other non-IDE based activities can also be productive throughout the typical developer day.

With great power comes great responsibility. Go forth and deliver great software while enabling your teams to do their best work!

Check out the full report in ACM Queue.

Written by

Kevin Alwell

@alwell-kevin