Unit Testing and Code coverage in Java

I previously discussed code coverage. In this post I would like to focus on code coverage for Java, as a concrete example. As part of that I also introduce the basics of unit testing for Java.

Unit testing #

unit testing is a method of software testing, where individual units of the source code is tested. A unit can be a complete module or function (procedural programming), or a class or method (object-oriented programming). Unit test allows to test and find issues early in the development lifecycle, as input, output and error behavior of a unit, are usually clear for a developer, when she writes the code. In addition, it also helps developers to think and improve the design of their units. The idea is, to test the unit in isolation, to ensure it’s proper function. This may not always be that simple, as some units may depend on other units, external data or states. This will require the use of mocks/stubs, to satisfy these external dependencies and simulate their behavior.

Unit tests are available for the most known programming languages, which each offers it’s own unit test framework or test harness, which may offer more than just executing the tests, but also thinks like

• setup and teardown of test data or test environments - also called a [test fixture]
• parametrization of tests, where a test is executes with different data
• test reporting

The methodology test driven development is a practice which has been established and used by developers. It improves the thinking process about the requirements and before writing the actual code. It basically goes like this

1. Write the skeleton of your unit
2. Write a test case for the unit, based on the requirements/specifications
3. Run the test against the unit - all test should fail with expected results, as no functionality has been implemented yet
4. Add code to your unit, which make your test pass
5. Run the test against the unit - this time the tests should pass
6. Refactor your code and improve it - if you re-run your tests, they still should pass as the expected input and output are still the same
7. Repeat: If requirements/specification changes, you first will adjust your test cases and then update the unit.

This approach should also help the developers to keep the units small, so the debugging is easier.

To be clear, even so you have a very good unit testing, this is not sufficient for testing an entire application/system. Some errors may not be detected as you cannot evaluate every execution path of the whole software. Using mocks just simulates the behavior of an external element but may not fully represent it’s real behavior. That’s wgy you also need integration tests, as well as manual tests, in addition to your unit tests.

Unit Tests with Junit #

Let’s do a simple example. Let’s create a simple java class which returns either do this or do that. Ideally you would follow the TDD approach where you first write your test, implement the productive code, but for simplicity I just write some code and then write the tests for it. Let’s take the same example as in this post:

public class UnitTestExample {
    public String doThis(int x, int y) {
        return "do this";
    }

    public String doThat(int x, int y) {
        return "do that";
    }

    public String simpleFunction(int x, int y) {
        if  (5 <= x || y < 10) {
            return doThis(x,y);
        } else {
            return doThat(x,y);
        }
    }
}

Let’s focus on the method simpleFunction(int x, int y) which we can test. In order to do so you need a unit test framework, which provides you the necessary harness to implement and run units tests for your code. In case of Java it is JUnit. Unit tests are usually written in a separate class and annotated with @Test. Remember, the outcome of the test is different depending on the input parameters:

For that we use Parametrized tests and end up with the following test class, which imports all required packages:

import java.util.Arrays;
import java.util.Collection;

import org.junit.Test;
import org.junit.Assert;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;

@RunWith(Parameterized.class)//import UnitTestExample;

public class UnitTestExampleTest {
    @Parameters
    public static Collection<Object[]> data() {
        return Arrays.asList(new Object[][] { 
                 { 4, 10, "do that"},
                 { 5, 10, "do this"},
                 { 4, 9, "do this"},
                 { 5, 9, "do this"},
           });
    }

    private int fx;
    private int fy;

    private String fExpected;

    public UnitTestExampleTest(int x, int y, String expected) {
        this.fx = x;
        this.fy = y;
        this.fExpected = expected;
    }

    @Test
    public void simpleFunctionTest() {
        String  retval = new UnitTestExample().simpleFunction(fx, fy);
        Assert.assertEquals(fExpected, retval);
    }
}

To run the tests, a test runner is required. usually your IDE has a native graphical runner built in. But you can also run tests from your command line You can also run it on the command line

1. Install java and Junit

2. Register CLASSPATH which includes to junit.jar and hamcrest-core.jar

   CLASSPATH=.:/usr/share/java/junit.jar:/usr/share/java/hamcrest-core.jar
   

3. Compile class and test class

   javac UnitTestExample.java UnitTestExampleTest.java
   

4. Now you can run the tests

   java org.junit.runner.JUnitCore UnitTestExampleTest
   JUnit version 4.13.2
   ....
   Time: 0.008
   
   OK (4 tests)
   

Measure Code Coverage #

As mentioned in my pervious post, measurement of code coverage is usually done on unit level. So what do you need to measure the code coverage?

• Source Code
• Unit tests
• A tool which measures the coverage

For Java the most common tool is Jacoco, a free code coverage library. Usually you would use a build tool like Maven to perform the steps below, but for better understanding I will do all on the command line.

So first of all we need Jacoco, which we can download from the website. It comes with several binaries and for measuring the code coverage, we are interested in the Java Agent

JaCoCo uses class file instrumentation to record execution coverage data. Class files are instrumented on-the-fly using a so called Java agent. This mechanism allows in-memory pre-processing of all class files during class loading independent of the application framework.

So we can create a code coverage report, by using this agent and execute the tests. This works as follows

java -javaagent:./jacoco/lib/jacocoagent.jar org.junit.runner.JUnitCore UnitTestExampleTest
JUnit version 4.13.2
....
Time: 0.018

OK (4 tests)

This will create a file called [jacoco.exec], which contains the code coverage information. From this file you can create human readable reports, for example a html-report:

java -jar ./jacoco/lib/jacococli.jar report jacoco.exec --classfiles . --html .

This is how the report looks like:

Wrap-up #

So now you should now the relation between unit testing and code coverage and how to get the code coverage information using Jacoco. Sure, the steps shown above are not very convenient, especially if you have a lot of classes. Anyway, unit testing and code coverage reporting are usually handled by your build tooling - for example Maven or Gradle.