Python 3 Function Annotations

In this tutorial I’ll show you how to take advantage of general-purpose function annotations and combine them with decorators. You will also learn about the pros and cons of function annotations, when it is appropriate to use them, and when it is best to use other mechanisms like docstrings and plain decorators. To annotate means to add notes, explanations, or comments.

Annotations provide additional information about the expected types or other metadata; they are not evaluated or treated as regular code statements.

Function Annotations

Function annotations are specified in PEP-3107. The main motivation was to provide a standard way to associate metadata to function arguments and return value. A lot of community members found novel use cases but used different methods such as custom decorators, custom docstring formats and adding custom attributes to the function object.

It’s important to understand that Python doesn’t bless the annotations with any semantics. It purely provides a nice syntactic support for associating metadata as well as an easy way to access it. Also, annotations are totally optional.

The syntax for annotations looks like this for functions and classes:

1	def func(arg: type) -> return_annotation:
2	pass

Let’s take a look at an example. Here is a function foo() that takes three arguments called a, b, and c and prints their sum. Note that foo() returns nothing. The first argument a is not annotated. The second argument b is annotated with the string ‘annotating b’, and the third argument c is annotated with type int. The return value is annotated with the type float. Note the -> syntax for annotating the return value.

from typing import NewType

annotating_b = NewType('annotating_b', str)

def foo(a, b: annotating_b, c: int) -> float:
print(a + b + c)

The annotations have no impact whatsoever on the execution of the function. Let’s call foo() twice: once with int arguments and once with string arguments. In both cases, foo() does the right thing, and the annotations are simply ignored.

1	foo('Hello', ', ', 'World!')
2	Hello, World!
3	foo(1, 2, 3)
4	6

Default Arguments

Default arguments are specified after the annotation:

from typing import NewType

custom_argument =NewType ('an argument that defaults to 5',str)

def foo(x: custom_argument = 5):
    print(x)


foo(7)
7
foo()
5

Accessing Function Annotations

The function object has an attribute called annotations. It is a mapping that maps each argument name to its annotation. The return value annotation is mapped to the key return, which can’t conflict with any argument name because return is a reserved word that can’t serve as an argument name. Note that it is possible to pass a keyword argument named return to a function:

def bar(*args, **kwargs: 'the keyword arguments dict'):
    print(kwargs['return'])
d = {'return': 4}
bar(**d)
4

Let’s go back to our first example and check its annotations:

def foo(a, b: str, c: int) -> float:
    return(a + b + c)
    
print(foo.__annotations__)
{'b': 'annotating b', 'c': <class 'int'>, 'return': <class 'float'>}

This is pretty straightforward. If you annotate a function with an arguments array and/or keyword arguments array, then obviously you can’t annotate individual arguments.

1	def foo(args: 'list of unnamed arguments', *kwargs: 'dict of named arguments'):
2	return(args, kwargs)
3	print(foo.__annotations__)
4	{'args': 'list of unnamed arguments', 'kwargs': 'dict of named arguments'}

If you read the section about accessing function annotations in PEP-3107, it says that you access them through the func_annotations attribute of the function object. This is out of date as of Python 3.2. Don’t be confused. It’s simply the annotations attribute.

What Can You Do With Annotations?

This is the big question. Annotations have no standard meaning or semantics. There are several categories of generic uses. You can use them as better documentation and move argument and return value documentation out of the docstring. For example, this function:

def div(a, b):
    """Divide a by b 
args:
a - the dividend
b - the divisor (must be different than 0)
return:
the result of dividing a by b
"""
return a / b

Can be converted to:

1	def div(a: 'the dividend',
2	b: 'the divisor (must be different than 0)') -> 'the result of dividing a by b':
3	"""Divide a by b"""
4	return a / b

While the same information is captured, there are several benefits to the annotations version:

If you rename an argument, the documentation docstring version may be out of date.
It is easier to see if an argument is not documented.
There is no need to come up with a special format of argument documentation inside the docstring to be parsed by tools. The annotations attribute provides a direct, standard mechanism of access.

Another usage that we will talk about later is optional typing. Python is dynamically typed, which means you can pass any object as an argument of a function. But often functions will require arguments to be of a specific type. With annotations you can specify the type right next to the argument in a very natural way.

Remember that just specifying the type will not enforce it, and additional work (a lot of work) will be needed. Still, even just specifying the type can make the intent more readable than specifying the type in the docstring, and it can help users understand how to call the function.

Yet another benefit of annotations over docstring is that you can attach different types of metadata as tuples or dicts. Again, you can do that with docstring too, but it will be text-based and will require special parsing.

Finally, you can attach a lot of metadata that will be used by special external tools or at runtime via decorators. I’ll explore this option in the next section.

Multiple Annotations

Suppose you want to annotate an argument with both its type and a help string. This is very easy with annotations. You can simply annotate the argument with a dict that has two keys: type and help.

def div(a: dict(type=float, help='the dividend'),
        b: dict(type=float, help='the divisor (must be different than 0)')
    ) -> dict(type=float, help='the result of dividing a by b'):
    """Divide a by b"""
    return a / b
    
print(div.__annotations__)
{'a': {'type': <class 'float'>, 'help': 'the dividend'}, 
'b': {'type': <class 'float'>, 'help': 'the divisor (must be different than 0)'}, 'return': {'type': <class 'float'>,
'help': 'the result of dividing a by b'}}

Combining Python Annotations and Decorators

Annotations and decorators go hand in hand. For a good introduction to Python decorators, check out my two tutorials: Deep Dive Into Python Decorators and Write Your Own Python Decorators.

First, annotations can be fully implemented as decorators. You can just define an @annotate decorator and have it take an argument name and a Python expression as arguments and then store them in the target function’s annotations attribute. This can be done for Python 2 as well.

However, the real power of decorators is that they can act on the annotations. This requires coordination, of course, about the semantics of annotations.

Let’s look at an example. Suppose we want to verify that arguments are in a certain range. The annotation will be a tuple with the minimum and maximum value for each argument. Then we need a decorator that will check the annotation of each keyword argument, verify that the value is within the range, and raise an exception otherwise. Let’s start with the decorator:

def check_range(f):
    def decorated(*args, **kwargs):
        for name, range in f.__annotations__.items():
            min_value, max_value = range
            if not (min_value <= kwargs[name] <= max_value):
                msg = 'argument {} is out of range [{} - {}]'
                raise ValueError(msg.format(name, min_value, max_value))
        return f(*args, **kwargs)
        return decorated

Next, define a custom class Range that represents the range of values.

1	def Range(min_value: int, max_value: int) -> tuple:
2	return min_value, max_value

Now, let’s define our function and decorate it with the @check_range decorators. The Range in this case acts as a type annotation. Even though it’s not a built-in type annotation, it serves a similar purpose by providing additional information to the range of expected values. We can think of it as a custom annotation.

1	@check_range
2	def foo(a: Range(0, 8), b: Range(5, 9), c: Range(10, 20)):
3	return a * b - c

Let’s call foo() with different arguments and see what happens. When all arguments are within their range, there is no problem.

1	foo(a=4, b=6, c=15)
2	9

But if we set c to 100 (outside of the (10, 20) range), then an exception is raised:

1	foo(a=4, b=6, c=100)
2	ValueError: argument c is out of range [10 - 20]

When Should You Use Decorators Instead of Annotations?

There are several situations where decorators are better than annotations for attaching metadata.

One obvious case is if your code needs to be compatible with Python 2.

Another case is if you have a lot of a metadata. As you saw earlier, while it’s possible to attach any amount of metadata by using dicts as annotations, it is pretty cumbersome and actually hurts readability.

Finally, if the metadata is supposed to be operated on by a specific decorator, it may be better to associate the metadata as arguments for the decorator itself.

Dynamic Annotations

Annotations are just a dict attribute of a function.

1	type(foo.__annotations__)
2	<class 'dict'>

This means you can modify them on the fly while the program is running. What are some use cases? Suppose you want to find out if a default value of an argument is ever used. Whenever the function is called with the default value, you can increment the value of an annotation. Or maybe you want to sum up all the return values. The dynamic aspect can be done inside the function itself or by a decorator.

def add(a, b) -> 0:
    result = a + b
    add.__annotations__['return'] += result
    return result
    
print(add.__annotations__['return'])
0
add(3, 4)
7
print(add.__annotations__['return'])
7
add(5, 5)
10
print(add.__annotations__['return'])
17

Conclusion

Function annotations are versatile and exciting. They have the potential to usher in a new era of introspective tools that help developers master more and more complex systems. They also offer the more advanced developer a standard and readable way to associate metadata directly with arguments and return value in order to create custom tools and interact with decorators. But it takes some work to benefit from them and utilize their potential.

Learn Python

Learn Python with our complete Python tutorial guide, whether you’re just getting started or you’re a seasoned coder looking to learn new skills.