Introducing flake8-logging

LOG001 use logging.getLogger() to instantiate loggers

It’s possible to construct a logger directly:

import logging

logger = logging.Logger(__name__)

When doing so, you may be unaware that the documentation says:

Note that Loggers should NEVER be instantiated directly, but always through the module-level function logging.getLogger(name).

This edict is because a directly-instantiated logger is not hooked into the “logging tree”. It will bypass all configured levels, filters, formatters, and handlers. Instead, its messages will only be sent to the last resort handler, which normally means appearing on stderr with minimal formatting. Potentially, such messages will not end up in your logging destination, and so you won’t be alerted to issues.

So yeah, use getLogger() to correctly instantiate loggers:

 import logging

-logger = logging.Logger(__name__)
+logger = logging.getLogger(__name__)

flake8-logging’s LOG001 rule detects direct instantiation of Logger at the module level:

$ flake8 example.py
example.py:3:10: LOG001 use logging.getLogger() to instantiate loggers

LOG005 use exception() within an exception handler

I’ve often seen a plain logging.error() call inside exception handlers:

import logging
from dancing import salsa

try:
    salsa()
except Exception:
    logging.error("Doing the Salsa failed")

Failures then show only the message:

$ python example.py
ERROR:root:Doing the Salsa failed

There’s no exception information, making future debugging attempts harder.

You can add exception information by either adding exc_info=True, or by switching to the exception() method which automatically sets that:

 try:
     salsa()
 except Exception:
-    logging.error("Doing the Salsa failed")
+    logging.exception("Doing the Salsa failed")

Failures then capture the exception information:

$ python example.py
ERROR:root:Doing the Salsa failed
Traceback (most recent call last):
  File "/..../example.py", line 5, in <module>
    salsa()
  File "/..../dancing.py", line 22, in salsa
    data = dance_data["salsa"]
           ~~~~~~~~~~^^^^^^^^^
KeyError: 'salsa'

Error aggregation tools like Sentry often hook into the logging system to capture this information.

flake8-logging’s LOG005 rule detects use of error() within exception handlers, recommending exception() instead:

$ flake8 example.py
example.py:7:5: LOG005 use exception() within an exception handler

LOG011 avoid pre-formatting log messages

f-strings are very convenient and it’s natural to use them for formatting log messages:

logging.error(f"Couldn’t chop {vegetable}")

This bypasses one of logging’s features though, its delayed message formatting. This can use several styles, but it defaults to Python’s older %-style string formatting:

logging.error("Couldn’t chop %s", vegetable)

The logging module’s delayed message formatting is designed to avoid the cost of formatting when the message won’t be logged. Using an f-string or similar to pre-format the message avoids that optimization.

The runtime cost of formatting is rarely a large concern, but it can add up in a tight loop. It can also be sizeable if you have a lot of debug-level messages that are rarely enabled.

Additionally, pre-formatting log messages presents an extra challenge to aggregation tools. They cannot use the log message template to group messages, so may have to rely on other heuristics.

flake8-logging’s LOG011 rule detects pre-formatted log messages using f-strings, str.format(), and vanilla % formatting:

$ flake8 example.py
example.py:5:17: LOG011 avoid pre-formatting log messages

Development

I worked on flake8-logging whilst working on my client Silvr’s project. I noticed some antipatterns recurring from projects I’ve seen in the past, so decided that a new Flake8 plugin would be the best thing to help stop them for good. Thanks to Anna Bierońska, Pascal Fouque, Rémy Hubscher for reviewing flake8-logging and my changes on Silvr.

I looked at some prior plugins before creating flake8-logging:

• flake8-logging-format which was initially created for what LOG011 covers. It has since been expanded to a couple of other logging-related rules.
• flake8-logging-arg-count which has one rule, now covered by LOG012.

These didn’t cover everything that I wanted to check for, but they served as good inspiration.

Fin

Charlie Marsh, if you’re reading this, you’re welcome to copy flake8-logging into Ruff.

I hope to continue working on new rules and improvements, all documented in the issue tracker.

May your logging be ever smoother,

—Adam