Code coverage

Code quality is important and as part of this we are usually also interested in the code coverage. But what is code coverage and what we can do with it?

What is code coverage? #

[code-coverage] is the measurement of how much of your source code is covered i.e. executed trough test cases. Usually code coverage is often only related with [unit-tests] but that’s not really correct. Every test on all levels of the [Test-Pyramid], cover certain lines your code. However as the [unit-tests] are so near to the code, it’s usually easier to measure. The higher up in the [Test-Pyramid] your tests are, the more difficult it is to tell exactly which code is covered with your tests.

When we speak about code coverage but there are coverage criteria which can be measured:

• Function coverage – has each function (or subroutine in the program been called?
• Statement coverage – has each statement in the program been executed?
• Edge coverage – has every edge in the Control flow graph been executed?
• Branch coverage – has each branch (also called DD-path of each control structure (such as in if and case statements) been executed? For example, given an if statement, have both the true and false branches been executed? This is a subset of edge coverage.
• Condition coverage (or predicate coverage) – has each Boolean sub-expression evaluated both to true and false?

Let’s have look ath a simple program and the it’s program flow. If a program is executed it happens in a well defined manner from top to bottom and from left to right. For example

#!/usr/bin/python
print("1. statement")
print("2. statement")

So the above python script will print first 1. statement, then 2. statement. However complex programs also have control structures which has an effect on the flow. Such structures are

• branches
• loops
• function calls
• others like break

Let’s have an example of a simple, but non-linear example

#!/usr/bin/python
def doThis(x,y):
    return "do this"

def doThat(x,y):
    return "do that"


def simpleFunction(x,y):
    if  (5 <= x or y < 10):
        return doThis(x,y)
    else:
        return doThat(x,y)

Let’s draw the possible flows of the program:

Covering the full code, would required to go trough all paths, which can be achieved by choosing the right values for x and y. So for the left path you could choose x=5 and y whatever. For the right path x=1 and y=10. So with 2 test cases you simply can cover all paths. But as you might suspect, this is not enough. Code coverage may tell you, how good your code is covered, but does not say anything about the quality of your code.

Why not? For instance, if you approach your unit test more methodically, you can see that there are different cases, which make the statement true or false. So both, the left and right condition can either be true or false, which then makes the complete statement either true or false (or-condition):

So, to properly test the conditions, we need 4 test cases. Now, assuming you can easily mistype <= and < so for instance you write, 5 < x || y <= 10 you might also understand choosing any value for x and y for cases at the border is important. Let’s say we pick any number as in the table below, test cases will cover 100% of your code and pass.

However, the test cases will not detect the mistype I mentioned above, unless we carefully choose the values for x and y which are in the border of the

Where is code coverage useful? #

As shown above, high code coverage does not necessarily mean your code is properly tested but it certainly help to identify uncovered areas of your code. An other important indicator is also the measure of the trend, means code coverage should be rather increased than decreased. A decreasing (% of coverage) usually means you add code which is not tested. I recommend to have a look at Test-driven development so your developers also think about testing while writing the code.