Build and Test .NetCore2.0 app with Jenkins

Posted in automation on September 20, 2017 by Adrian Wyssmann ‐ 7 min read

.net jenkins docker

Recently, for a job interview I had to solve a development task: Create a .NET C## application which converts digital numbers into roman numbers

jenkins build success

The details of the task can be found in my Gitlab project. As I recently attended a presentation regarding dotnetcore 2.0 I’ve decided to do the tasks with exactly that framework to have a cross-platform solution. Well the code itself is not a masterpiece, but was a good experience for me to actually do some real coding again 🙂

More interestingly was the challenge to setup a build pipeline. Even so the project is hosted on GitLab, I have decide to try out using a dedicated Jenkins server running from a container instead of Gitlab’s built-in CI/CD.

Start Jenkins container

Well, first step is to find the correct image, which is essential, as some might be deprecated. Best choice is obviously the one from the jenkins repo. For a start let’s keep it simple and don’t use dedicated agents, so the sole build server is enough. Starting the container is as simple as explained in the documentation

docker run -p 8080:8080 -p 50000:50000 -v jenkins_home:/var/jenkins_home jenkins/jenkins:lts

I have chosen to map the data instead using a data container, but as already mentioned in Docker and Data Storage data volumes will give you more flexibility. Once the server is running, I to the initial configuration and then I am ready to setup my build pipeline. It’s a simple one, for now manually triggered

  1. Checkout code from GitLab
  2. Run shell script

The shell script looks as follows:

#!/usr/bin/env bash
cd Parser
dotnet restore
dotnet build Parser.csproj
cd ../Parser.Test
dotnet restore
dotnet test Parser.Test.csproj

You might notice that this approach does only build on Linux - Jenkins running on Linux host, no Windows agents, Bash script - so a proper setup would look different. However my goal here is to build the .NetCore application on Linux. Once the jenkins job is created I simply run it which - you might expected it - ends in

10:36:01 + dotnet build ParserParser.csproj
10:36:01 /tmp/jenkins7090288280163475905.sh: 2: /tmp/jenkins7090288280163475905.sh: dotnet: not found
10:36:01 Build step 'Execute shell' marked build as failure
10:36:01 Finished: FAILURE

Jenkins does not come with dotnet so this has to be installed in the container.

Create my own Jenkins Image

Docker is great for that, you simply take the Jenkins image and aggregate the stuff you need. To add dotnet best starting point is the official documentation at MS https://docs.microsoft.com/en-us/dotnet/core/linux-prerequisites?tabs=netcore2x.

So what we need is the to register the Microsoft Product key as trusted and then add the ms repo for dotnet http://packages.microsoft.com/repos/microsoft-debian-jessie-prod jessie main

FROM jenkins/jenkins
USER root
RUN curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg
RUN mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg
RUN apt-get update

## Downgrading of libcurl is a required workaround to avoid segmentation fault when using dotnet build/run/...
## https://github.com/dotnet/core/issues/963
RUN apt-get --assume-yes install dotnet-runtime-2.0.0 dotnet-sdk-2.0.0 nuget
RUN sudo sh -c 'echo "deb [arch=amd64] http://packages.microsoft.com/repos/microsoft-debian-jessie-prod jessie main" > /etc/apt/sources.list.d/dotnetdev.list'

## drop back to the regular jenkins user - good practice
USER jenkins
#COPY plugins.txt /usr/share/jenkins/ref/
#RUN /usr/local/bin/plugins.sh /usr/share/jenkins/ref/plugins.txt

So let’s try to build the image

ubuntu@docker-host: ~$ docker build -t test .
Sending build context to Docker daemon  7.168kB
Step 1/8 : FROM jenkins/jenkins
 ---> 280369e5a27e
Step 2/8 : USER root
 ---> Using cache
 ---> cee30906a2f8
Step 3/8 : RUN curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg
 ---> Using cache
 ---> 6d76cc7e4cbd
Step 4/8 : RUN mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg
 ---> Using cache
 ---> 9c8a74f178b7
Step 5/8 : RUN sh -c 'echo "deb [arch=amd64] http://packages.microsoft.com/repos/microsoft-debian-jessie-prod jessie main" > /etc/apt/sources.list.d/dotnetdev.list'
 ---> Running in 9ccb85a07084
 ---> 1577e9389882
Removing intermediate container 9ccb85a07084
Step 6/8 : RUN apt-get update
 ---> Running in 29620f79ab26
Ign:1 http://deb.debian.org/debian stretch InRelease
Get:2 http://security.debian.org stretch/updates InRelease [62.9 kB]
Get:3 http://deb.debian.org/debian stretch-updates InRelease [91.0 kB]
Get:4 http://packages.microsoft.com/repos/microsoft-debian-jessie-prod jessie InRelease [2846 B]
Get:5 http://deb.debian.org/debian stretch Release [118 kB]
Get:6 http://deb.debian.org/debian stretch Release.gpg [2373 B]
Get:7 http://packages.microsoft.com/repos/microsoft-debian-jessie-prod jessie/main amd64 Packages [2897 B]
Get:8 http://deb.debian.org/debian stretch-updates/main amd64 Packages [5553 B]
Get:9 http://security.debian.org stretch/updates/main amd64 Packages [213 kB]
Get:10 http://deb.debian.org/debian stretch/main amd64 Packages [9497 kB]
Fetched 9996 kB in 0s (10.4 MB/s)
Reading package lists...
 ---> 20a30f6297df
Removing intermediate container 29620f79ab26
Step 7/8 : RUN apt-get --assume-yes install dotnet-runtime-2.0.0 dotnet-sdk-2.0.0 nuget
 ---> Running in 0d4de4ba1125
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 dotnet-runtime-2.0.0 : Depends: libssl1.0.0 but it is not installable
                        Depends: libicu52 but it is not installable
E: Unable to correct problems, you have held broken packages.
The command '/bin/sh -c apt-get --assume-yes install dotnet-runtime-2.0.0 dotnet-sdk-2.0.0 nuget' returned a non-zero code: 100

Sadly there are some “old” packages required so it required some investigation to figure out where to find these packages. So for debian jessie it requires the following 2 repos in addition:

Once this is done, the image is build successfully. The Dockerfile looks like this:

FROM jenkins/jenkins
USER root
RUN curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg
RUN mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg
RUN sh -c 'echo "deb [arch=amd64] http://httpredir.debian.org/debian jessie-backports main" > /etc/apt/sources.list.d/backports.list'
RUN sh -c 'echo "deb [arch=amd64] http://httpredir.debian.org/debian oldstable main" > /etc/apt/sources.list.d/jessiestable.list'
RUN sh -c 'echo "deb [arch=amd64] http://packages.microsoft.com/debian/8/prod jessie main" > /etc/apt/sources.list.d/dotnetdev.list'
RUN apt-get update

RUN apt-get --assume-yes  install dotnet-runtime-2.0.0 dotnet-sdk-2.0.0 nuget

## drop back to the regular jenkins user - good practice
USER jenkins
#COPY plugins.txt /usr/share/jenkins/ref/
#RUN /usr/local/bin/plugins.sh /usr/share/jenkins/ref/plugins.txt

Running the image

The image is uploaded to DockerHub (papanito/jenkins) so it can be easily used.

docker run -d -p 8080:8080 -p 50000:50000 --restart always -v /var/jenkins_home:/var/jenkins_home --name jenkins papanito/jenkins

So let’s trigger the jenkins job again, expecting it runs without problems… nope

13:01:06 /tmp/jenkins858828605718566111.sh: line 3:  3252 Segmentation fault      (core dumped) dotnet restore Parser\
13:01:06 MSBUILD : error MSB1009: Project file does not exist.
13:01:06 Switch: Parser.Test
13:01:06 /tmp/jenkins858828605718566111.sh: line 4:  3276 Segmentation fault      (core dumped) dotnet restore Parser.Test\
13:01:06 MSBUILD : error MSB1009: Project file does not exist.
13:01:06 Switch: ParserParser.csproj
13:01:06 /tmp/jenkins858828605718566111.sh: line 5:  3300 Segmentation fault      (core dumped) dotnet build Parser\Parser.csproj
13:01:06 MSBUILD : error MSB1009: Project file does not exist.
13:01:06 Switch: Parser.TestParser.Test.csproj
13:01:07 /tmp/jenkins858828605718566111.sh: line 6:  3324 Segmentation fault      (core dumped) dotnet test Parser.Test\Parser.Test.csproj
13:01:07 Build step 'Execute shell' marked build as failure
13:01:07 Finished: FAILURE

It took me quite some time to figure out how to overcome to this problem, but finally found it hidden in one of the comments of an existing issue dotnet restore segfaults on Debian 9.0/stretch Testing". Downgrading the libcurl package shall solve the problem, so the Dockerfile needs to be adjusted accordingly:

FROM jenkins/jenkins
USER root
RUN curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg
RUN mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg
RUN sh -c 'echo "deb [arch=amd64] http://httpredir.debian.org/debian jessie-backports main" > /etc/apt/sources.list.d/backports.list'
RUN sh -c 'echo "deb [arch=amd64] http://httpredir.debian.org/debian oldstable main" > /etc/apt/sources.list.d/jessiestable.list'
RUN sh -c 'echo "deb [arch=amd64] http://packages.microsoft.com/debian/8/prod jessie main" > /etc/apt/sources.list.d/dotnetdev.list'
RUN apt-get update

## Downgrading of libcurl is a required workaround to avoid segmentation fault when using dotnet build/run/...
## https://github.com/dotnet/core/issues/963
RUN apt-get --assume-yes --allow-downgrades install dotnet-runtime-2.0.0 dotnet-sdk-2.0.0 nuget libcurl3=7.38.0-4+deb8u5

## drop back to the regular jenkins user - good practice
USER jenkins
#COPY plugins.txt /usr/share/jenkins/ref/
#RUN /usr/local/bin/plugins.sh /usr/share/jenkins/ref/plugins.txt

After successfully building, uploading the image and restarting the container with the latest image the jenkins job finally succeeded:

...
13:40:01 Microsoft (R) Build Engine version 15.3.409.57025 for .NET Core
13:40:01 Copyright (C) Microsoft Corporation. All rights reserved.
13:40:01
13:40:04   Parser -> /var/jenkins_home/workspace/romannumeral-ci/Parser/bin/Debug/netcoreapp2.0/Parser.dll
13:40:04
13:40:04 Build succeeded.
13:40:04     0 Warning(s)
13:40:04     0 Error(s)
13:40:04
13:40:04 Time Elapsed 00:00:02.45
13:40:05   Restoring packages for /var/jenkins_home/workspace/romannumeral-ci/Parser.Test/Parser.Test.csproj...
13:40:05   Restore completed in 21.25 ms for /var/jenkins_home/workspace/romannumeral-ci/Parser/Parser.csproj.
13:40:06   Generating MSBuild file /var/jenkins_home/workspace/romannumeral-ci/Parser.Test/obj/Parser.Test.csproj.nuget.g.props.
13:40:06   Generating MSBuild file /var/jenkins_home/workspace/romannumeral-ci/Parser.Test/obj/Parser.Test.csproj.nuget.g.targets.
13:40:06   Restore completed in 457.61 ms for /var/jenkins_home/workspace/romannumeral-ci/Parser.Test/Parser.Test.csproj.
13:40:08 Build started, please wait...
13:40:12 Build completed.
13:40:12
13:40:12 Test run for /var/jenkins_home/workspace/romannumeral-ci/Parser.Test/bin/Debug/netcoreapp2.0/Parser.Test.dll(.NETCoreApp,Version=v2.0)
13:40:12 Microsoft (R) Test Execution Command Line Tool Version 15.3.0-preview-20170628-02
13:40:12 Copyright (c) Microsoft Corporation.  All rights reserved.
13:40:12
13:40:12 Starting test execution, please wait...
13:40:13 [xUnit.net 00:00:00.8303810]   Discovering: Parser.Test
13:40:13 [xUnit.net 00:00:00.9879387]   Discovered:  Parser.Test
13:40:14 [xUnit.net 00:00:01.0535661]   Starting:    Parser.Test
13:40:14 [xUnit.net 00:00:01.2785747]   Finished:    Parser.Test
13:40:14
13:40:14 Total tests: 49. Passed: 49. Failed: 0. Skipped: 0.
13:40:14 Test Run Successful.
13:40:14 Test execution time: 2.0174 Seconds
13:40:14 Finished: SUCCESS

Next steps

Currently the job is just triggered manually, so it would be best if job is triggered once changes are pushed to the git repository. In addition the building and testing shall also happen on Windows. For this to happen, I need to setup a Jenkins windows agent.

← Why I use Visual Studio Code on Linux
Introduction to Maven →