Because many floating point numbers can’t be represented exactly in a computer, most testing frameworks provide a way of checking that a floating point result is “close enough” to the expected value. In a recent Python project, I learned that pytest has a solution to this problem that I’d never seen before.

When first working with floating point numbers, many programmers are surprised when two numbers that should be equal are not. For example, in binary, 0.1 + 0.2 does not exactly equal 0.3 because 0.1 does not have a finite represenation in the binary floating point representation that our computers use. This is similar to how 1/3 does not have a finite representation in decimal.

The solution most people come up with is to check whether the absolute difference between the expected and actual values is small enough. If you know that the numbers you’ll be working with are in a certain range, this can be fine. But if you’re working with either very large or very small numbers, this may not work the way you’d expect.

A solution to this problem is to compare the relative difference between the two numbers. This is a bit trickier, though, as you have to worry about what happens when the numbers are close to 0.

Because of these complexities, most unit testing frameworks provide a way of checking that a floating point result is close enough to the expected value.

Here are some examples:

Ruby Minitest assertions
assert_in_delta(0.3, 0.1 + 0.2, 0.0005)

assert_in_epsilon(0.3, 0.1 + 0.2, 0.0001)  # relative difference
Ruby RSpec
expect(0.1 + 0.2).to be_within(0.0005).of(0.3)
Java JUnit
assertEquals(0.3, 0.1 + 0.2, 0.0005)
JavaScript Mocha + Chai
expect(0.1 + 0.2).to.be.closeTo(0.3, 0.0005)
JavaScript Jest
expect(0.1 + 0.2).toBeCloseTo(0.3, 3)  // number of digits, not absolute value

Hopefully you get the idea. This is pretty much what I expect to find when I encounter a new testing framework.

However, I recently worked on a Python project where I used pytest as the test framework.

pytest uses Python’s built-in assert statement in its tests and is able to produce very nice failure messages from that. However, assert by itself doesn’t provide any niceties for comparing floating point numbers.

To solve this problem, pytest includes an approx function that creates an object that knows how to compare itself to a floating point value appropriately.

Here are some examples of how to use it:

Python pytest with approx
assert 0.1 + 0.2 == approx(0.3)

 
assert 0.1 + 0.2 == approx(0.3, rel=1e-3)

 
assert 0.1 + 0.2 == approx(0.3, abs=1e-3)

As you can hopefully see, approx provides a lot of flexibility with a very simple API. If you provide a rel value, it does relative comparison (the default). If you instead provide an abs value, it does absolute comparison. It can even handle comparing arrays or dictionaries containing numbers.

After so many years of using different testing frameworks in different languages, I was genuinely surprised to run into such a different approach to the problem. I think this is a really nice, effective solution and it’s got me thinking about how it might be applied to other languages and testing frameworks.