Real-time feedback for students using continuous integration tools

For teachers it can be a challenge to get students the help they need exactly when they get stuck, especially in large courses. But when teachers use an automated testing suite like Travis CI with their assignments, students can hone in on their mistakes and iterate to improve.

Students see the value of a test-driven workflow as they are learning, and teachers save time grading assignments: a win for everyone.

Omar Shaikh, Lecturer at San Francisco State University, saw an opportunity to improve the assignment workflow for the C++ course he took over:

“The problem with the manual submission process was that, if a student is missing a small semicolon, or they were almost correct, the TA has no way of knowing because it doesn’t compile. Yes, the TA can go into the code and try to grade it, but it’s very inefficient.”

Build confidence through passing tests

Exercism, a tool that helps code newbies and experienced programmers learn new languages, uses a testing framework to guide students in increments.

“It helps people who are new to programming break down problems into smaller pieces” said Exercism creator Katrina Owen, “a lot of students spent a lot of time being stuck before I introduced test suites. Seeing the green check marks when their tests pass can reassure students that they’ve mastered the skill and the problem is solved for now.”

Experienced Ruby teacher (and TravisCI engineer) Renée Hendricksen agrees: “When you have assignments that are ‘write code to get these tests to pass’ the student gets instant feedback if they are on the right track.”

“When you use a ‘test-first’ teaching method,” she explained, “students get used to reading and understanding errors early. When they start with a failing test, they can look into whether it’s an error because of setup or their application. Let the red-green-refactor guide you.”

Efficient grading for problem sets

Instead of grinding through a lot of time looking for a missing period or semicolon, teachers have insight into exactly which piece of the code didn’t pass and go from there.

For Hendrickson, a testing suite makes grading more efficient because she can see where a student needs help:

“If I see a failing test I can look there immediately and address where they struggled. If all the tests are green, I can focus on working code and give them guidance for style and composition improvement, without getting bogged down explaining syntax issues.”

A step-by-step guide to using tests for immediate feedback

Omar connects his course organization to GitHub Classroom, and then to Travis CI. While there are many ways to implement automated testing, here are the steps to implement his particular workflow:

Step 1: Create the tests

• Make an organization for each class, and apply for the organization discount at the GitHub Education website.
• Prepare homework assignments, their solutions, and test cases. For C++ assignments, Omar used Google’s Gtest library for test cases. Create a private repository in the organization, and move homework files there.
• Connect the assignment repository to Travis CI.

Setting up a new individual assignment in GitHub Classroom

Step 2: Make the assignment and connect it to GitHub Classroom

• Omar removes code segments so that students complete them as their homework. He includes a few lines of contextual information as comments embedded in code (such as “Complete this Section”), and creates a public repository with the name of the homework assignment: hwx-template (where x is the homework number).
• Omar navigates over to GitHub Classroom, creates a new individual assignment, and points the assignment to the template repository.
• Omar posts the assignment link from GitHub Classroom to his class forums. As students click on the link, Classroom creates a new private repository for each of them.
• In Travis CI, Omar enables the tests on each of the student repositories. Travis will then build the repository after every push, based on the directions Omar put in the starter code template repository.

Example of a few student repositories on Travis CI for “hw3”

Step 3: Students clone the assignments and complete their homework using the recommended IDE

• Omar uses the CLion C++ IDE from JetBrains, which offers free licenses for educators, and manages Google Tests efficiently.
• Students clone their particular assignment repository with detailed instructions in the README.md.
• Students open the project in the IDE to start coding and running unit tests. As students complete their assignments, they push their changes to GitHub.
• Instructors can see if the build is passing via Travis CI and in the “commit” view on the GitHub web interface.

Graders can look at which tests failed for insight into where students struggle with the material

Use this workflow in your own course

If you’re ready to implement Travis CI and GitHub Classroom, here are some tools to get you started:

• a sample repository you can use as an example with all the moving parts
• a Travis build of that sample
• Omar’s problem sets, which have been tested by his team of teaching assistants and also contain detailed instructions.

To help you put all of the pieces together, Omar made a video walkthrough:

—

This is a post in our “Teacher Spotlight” series, where we share the different ways teachers use GitHub in their classrooms. Check out the other posts:

• GitHub Issues and user testing as authentic assessment at the University of Victoria featuring Alexey Zagalsky
• Invest in tools students can grow with: GitHub and RStudio for data science at Duke University, featuring Mine Çetinkaya-Rundel
• GitHub Classroom for AP Computer Science at Naperville North High School, featuring Geoff Schmit
• How CS50 at Harvard uses GitHub to teach computer science, featuring David Malan

Join this week’s discussion in the community forum: How to automatically gather or collect assignments?