If this is the first time you come across the Go coverage profile then do read my last post: Test coverage in Go, working with packages and sub-packages. It is an introduction to Go’s cover tool, how to generate the coverage profile and how to use it to aggregate the coverage from the sub-packages to the root package.
If you are already familiar with the cover tool and the coverage profile then let’s dive in and see a few cases that you need to be aware of if you are trying to aggregate the coverage from the sub-packages all the way to the root of your package.
🖥️ The full code is on my Github repo: https://github.com/efrag/blog-posts
1️⃣ Sub-packages with test files
This is a recap of what I covered in my previous blog post. We examined the case where we had a package with 2 sub-packages each of which defined its own test file. I am using the following command that runs the tests, generates the coverage profile and then aggregates the results to the package level:
go test --count=1 -coverprofile=coverage.out ./... ; \
cat coverage.out | \
awk 'BEGIN {cov=0; stat=0;} \
$3!="" { cov+=($3==1?$2:0); stat+=$2; } \
END {printf("Total coverage: %.2f%% of statements\n", (cov/stat)*100);}'Generating the coverage profile in this case and aggregating the result gave us the correct % coverage of our entire package.
2️⃣ Sub-packages without test files
In this case we are going to examine what happens if we introduce a new sub-package in our package, but this time we have not yet written any tests for it. The new package is called age and it has a single go file in it, called years.go that allows us to calculate how many years have passed since a user provided year. We have not written any tests for our new package yet but we still want to see how this affects our test coverage so we run our command again. The output that we get on the terminal looks like this:
$ go test --count=1 -coverprofile=coverage.out ./... ; cat coverage.out | awk 'BEGIN {cov=0; stat=0;} \
$3!="" { cov+=($3==1?$2:0); stat+=$2; } \
END {printf("Total coverage: %.2f%% of statements\n", (cov/stat)*100);}'
? github.com/efrag/blog-posts/go_test_coverage/age [no test files]
ok github.com/efrag/blog-posts/go_test_coverage/calculator 0.002s coverage: 16.7% of statements
ok github.com/efrag/blog-posts/go_test_coverage/size 0.004s coverage: 42.9% of statements
Total coverage: 30.77% of statementsThis somewhat matches what we expected but not quite. We can see the standard Go output that there are [no test files] in the new sub-package and we can see that we still calculate the coverage correctly for the calculator and size sub-packages as we made no change there. BUT we also see that the overall coverage has not change even though we added code that is not tested.
The way that the test and cover commands work, is the cause of this since there are no test files in the age directory the output of test simply mentions that there are no test files and since there are no test files there is nothing generated in the coverage profile! So let’s go ahead and add a years_test.go file in the age package with no tests whatsoever in it. It’s an empty test file as this point and rerun our command. The output we get now is different than before 🎉.
$ go test --count=1 -coverprofile=coverage.out ./... ; cat coverage.out | awk 'BEGIN {cov=0; stat=0;} \
$3!="" { cov+=($3==1?$2:0); stat+=$2; } \
END {printf("Total coverage: %.2f%% of statements\n", (cov/stat)*100);}'
? github.com/efrag/blog-posts/go_test_coverage/age 0.013s coverage: 0.0% of statements [no tests to run]
ok github.com/efrag/blog-posts/go_test_coverage/calculator 0.002s coverage: 16.7% of statements
ok github.com/efrag/blog-posts/go_test_coverage/size 0.004s coverage: 42.9% of statements
Total coverage: 20.00% of statementsLooking specifically at the output for the age package we can see that us adding the empty test file has triggered the coverage report to include the age package in its output (reporting a 0% coverage) and we can see that go test reports that it found [no tests to run]. Most importantly though, the parsing of the coverage profile now correctly calculates the aggregate coverage to be 20.00% instead of 30.77% that we were seeing just before.
3️⃣Sub-packages with multiple files
So then the next question is do we need to have a test file for each go file in a directory or does one test file per directory is enough to trigger the cover tool to run for all files in that directory?
And the answer to that is that a single test file per directory is enough! Adding a months.go file in the age sub-package without doing anything else with the test files updates the coverage profile correctly and considers all the statements in that directory, making the aggregate % coverage from the script go down to 13.33%.
github.com/efrag/blog-posts/go_test_coverage/age/months.go:8.48,9.14 1 0 github.com/efrag/blog-posts/go_test_coverage/age/months.go:13.2,13.29 1 0 github.com/efrag/blog-posts/go_test_coverage/age/months.go:17.2,18.24 2 0 github.com/efrag/blog-posts/go_test_coverage/age/months.go:22.2,23.49 2 0 github.com/efrag/blog-posts/go_test_coverage/age/months.go:27.2,27.58 1 0 github.com/efrag/blog-posts/go_test_coverage/age/months.go:9.14,11.3 1 0 github.com/efrag/blog-posts/go_test_coverage/age/months.go:13.29,15.3 1 0 github.com/efrag/blog-posts/go_test_coverage/age/months.go:18.24,20.3 1 0 github.com/efrag/blog-posts/go_test_coverage/age/months.go:23.49,25.3 1 0 github.com/efrag/blog-posts/go_test_coverage/age/years.go:8.40,9.14 1 0 github.com/efrag/blog-posts/go_test_coverage/age/years.go:13.2,15.24 2 0 github.com/efrag/blog-posts/go_test_coverage/age/years.go:19.2,19.32 1 0 github.com/efrag/blog-posts/go_test_coverage/age/years.go:9.14,11.3 1 0 github.com/efrag/blog-posts/go_test_coverage/age/years.go:15.24,17.3 1 0 (...)
As we can see from the snippet above both the years and months statements have been added to the coverage profile which means we correctly aggregate the result.
4️⃣ More nesting of sub-packages
And finally, let’s have a look at how the output changes when we create one more package under our size sub-package. The new package is called surface it only has one file in it and we have already added an empty test file in that directory (since we now know we need it 😉). Our output now looks like this
$ go test --count=1 -coverprofile=coverage.out ./... ; cat coverage.out | awk 'BEGIN {cov=0; stat=0;} \
$3!="" { cov+=($3==1?$2:0); stat+=$2; } \
END {printf("Total coverage: %.2f%% of statements\n", (cov/stat)*100);}'
? github.com/efrag/blog-posts/go_test_coverage/age 0.013s coverage: 0.0% of statements [no tests to run]
ok github.com/efrag/blog-posts/go_test_coverage/calculator 0.002s coverage: 16.7% of statements
ok github.com/efrag/blog-posts/go_test_coverage/size 0.004s coverage: 42.9% of statements
ok github.com/efrag/blog-posts/go_test_coverage/size/surface 0.006s coverage: 0.0% of statements [no tests to run]
Total coverage: 12.90% of statementsFrom the output we can see that we did calculate the aggregate % coverage for our package correctly. There is only one new statement in our new package that’s why the difference is so small, as we went from 13.33% to 12.90%.
So, what would happen if we did actually include tests in our surface package that do test our very simple function? Would Go change the % covered for the size package overall? The answer to that is actually no. Adding a test for the surface package does change what we calculate as part of the aggregate coverage (based on the coverage profile) but does not change the output of the Go test command.
$ go test --count=1 -coverprofile=coverage.out ./... ; cat coverage.out | awk 'BEGIN {cov=0; stat=0;} \
$3!="" { cov+=($3==1?$2:0); stat+=$2; } \
END {printf("Total coverage: %.2f%% of statements\n", (cov/stat)*100);}'
? github.com/efrag/blog-posts/go_test_coverage/age 0.013s coverage: 0.0% of statements [no tests to run]
ok github.com/efrag/blog-posts/go_test_coverage/calculator 0.002s coverage: 16.7% of statements
ok github.com/efrag/blog-posts/go_test_coverage/size 0.004s coverage: 42.9% of statements
ok github.com/efrag/blog-posts/go_test_coverage/size/surface 0.002s coverage: 100.0% of statements
Total coverage: 16.13% of statementsAs we can see the coverage of size is still reported as 42.9% which is the coverage for the statements directly under the size folder and does not take into account our newly added surface package.
Conclusion
Test coverage and the coverage profile in Go are really useful tools and knowing how and when to aggregate the results that they produce can be extremely handy.
If you do want to use the profile and some sort of automated way of aggregating your results upwards:
- make sure you either have a
_test.gofile in every directory you have go files - create it on the fly before running the tests and coverage so that you don’t assume your coverage is actually higher than what it really is
- aggregating at different levels (for example at the go_test_coverage but also at the size level) would be better done in code (shell script or otherwise) as it will require more manipulation of the output