Software I enhancements, Part 3 – Testing continued

I found a couple of problems with our CI workflow that we built in the last post.

  1. We’re not running any tests
  2. The docker container we’re using to build the solution doesn’t pass out the exit code of our command; if a build fails inside the container we won’t know about it as long as our docker-compose command exits with code 0.

I picked xUnit to test our C# code because it’s got a more familiar assert structure that’s similar to jUnit. Similar products like nUnit are great, but I didn’t want to spend a lot of time re-learning a new test framework. To get started we’ll use Nuget to install the packages necessary to run xUnit tests.

To install these packages, search for xunit in the browse section and install each package with the menu on the right

I installed xUnit base, xUnit runners for visual studio and the console (one for dev on the destkop, the other for our CI container), and an analyzer to help me write better tests. Now we need a test project to run when we need to test our application. To create one you can either create a blank class library and add the necessary imports and references to use xUnit, or you can use one of the templates included with Visual Studio 2017. I opted for the latter.

Microsoft also provides MSTest project templates in Visual Studio 2017

After creating the test project you’ll be given a single empty Fact (a basic test method in xUnit) method. Let’s go ahead and add a few simple Assertions to test that our tests are passing and failing properly.

Great! Our tests pass and fail like we want them to.

So now that we have test capability in our solution we need to add it to our build container. Open up your docker compose project and add dotnet test to the cmd that the container will run.

version: '2'

    image: microsoft/aspnetcore-build:1.0-1.1
      - .:/src
    working_dir: /src
    command: /bin/bash -c "dotnet restore ./C482.sln && dotnet test && dotnet publish ./C482.sln -c Release -o ./obj/Docker/publish"


Once I added the line I ran a docker-compose up command on my CI file like so:

docker-compose -f .\ up

When I run this command I get an error that leads me to believe that my dotnet test command isn’t pointing to my test project.

Adding a path to the test project fixed this issue, though. And we can run tests in our container after we add the path to our test project.

version: '2'

    image: microsoft/aspnetcore-build:1.0-1.1
      - .:/src
    working_dir: /src
    command: /bin/bash -c "dotnet restore ./C482.sln && dotnet test ./C482.Tests/C482.Tests.csproj && dotnet publish ./C482.sln -c Release -o ./obj/Docker/publish"

Nice, let’s clean up our failing tests and commit our changes. There’s one more thing we need to do before we can begin TDD-ing it up. Right now our build container will always pass even if our container fails its command because docker-compose will always exit 0 as long as the container runs. We need to somehow extract the exit code from the container and pass it back to the shell that Travis runs for us. After a little bit of googling I settled on a solution found here on StackOverflow. The idea is to run the container, wait for it to finish, and then see what the state of the exited container is. I moved the building process to a shell script (big thanks to the Ministry of Programming) that we can execute on Travis.

docker-compose -f up
docker-compose ps -q | xargs docker inspect -f '{{ .State.ExitCode }}' | while read code; do  
    if [ "$code" == "1" ]; then    
       exit -1

Then we just add the correct lines to the travis.yml file.


sudo: required

install: true

 - docker

 - chmod +x ./ && ./

I removed the csharp language designation to make the build go faster. Now we’re using a ruby environment but skipping the “install” step that would normally set up a ruby environment. Now our build passes when there are no tests. Let’s push some failing tests and see if we can pass and fail like we expect to.

Sure enough, we CAN break the build now!

Being good stewards of our program, it’s our duty to fix the broken build. Let’s remove the broken tests.

Alright, next up we’ll outline the school project and start laying down some code for our models.