I’m available for freelance work. Let’s talk »

Functional strategies in Python

Friday 13 March 2020

I got into a debate about Python’s support for functional programming (FP) with a friend. One of the challenging parts was listening to him say, “Python is broken” a number of times.

Python is not broken. It’s just not a great language for writing pure functional programs. Python seemed broken to my friend in exactly the same way that a hammer seems broken to someone trying to turn a screw with it.

I understand his frustration. Once you have fully embraced the FP mindset, it is difficult to understand why people would write programs any other way.

I have not fully embraced the FP mindset. But that doesn’t mean that I can’t apply some FP lessons to my Python programs.

In discussions about how FP and Python relate, I think too much attention is paid to the tactics. For example, some people say, “no need for map/­filter/­lambda, use list comprehensions.” Not only does this put off FP people because they’re being told to abandon the tools they are used to, but it gives the impression that list com­pre­hensions are somehow at odds with FP constructs, or are exact replacements.

Rather than focus on the tactics, the important ideas to take from FP are strategies, including:

  • Write small functions with no side-effects
  • Don’t change existing data, make new data
  • Combine functions to make larger functions

These strategies all lead to modularized code, free from mysterious action at a distance. The code is easier to reason about, debug, and extend.

Of course, languages that are built from the ground up with these ideas in mind will have great tools to support them. They have tactics like:

  • Immutable data structures
  • Rich libraries of higher-order functions
  • Good support for recursion

Functional languages like Scheme, Clojure, Haskell, and Scala have these tools built-in. They are of course going to be way better for writing Functional programs than Python is.

FP people look at Python, see none of these tools, and conclude that Python can’t be used for functional programming. As I said before, Python is not a great language for writing purely function programs. But it’s not a lost cause.

Even without those FP tools in Python, we can keep the FP strategies in mind. Although list comprehensions are presented as the alternative to FP tools, they help with the FP strategies, because they force you to make new data instead of mutating existing data.

If other FP professionals are like my friend, they are probably saying to themselves, “Ned, you just don’t get it.” Perhaps that is true, how would I know? That doesn’t mean I can’t improve my Python programs by thinking Functionally. I’m only just dipping my toes in the water so far, but I want to do more.

For more thoughts about this:

  • Gary Bernhardt: Boundaries, a PyCon talk that discusses Functional Core and Imperative Shell.
  • If you want more Functional tools, there are third-party Python packages like:
    • pyrsistent, providing immutable data structures
    • pydash, providing functional tools
    • fnc, providing functional tools

Comments

[gravatar]
I've always thought of list comprehensions as the functional alternative to for loops, to use when there are no side effects. I was even under the impression they were inspired by functional programming. Aren't similar constructs common in Haskell?
[gravatar]
I like toolz (https://github.com/pytoolz/toolz/), it is a great library. And cytoolz (https://github.com/pytoolz/cytoolz) if you need to go faster.
[gravatar]
Somebody on the Internet 9:00 PM on 14 Mar 2020
I agree with the gist of the article, but I don't think it's quite true to say that list comprehensions *force* you to create new data. Sure, you always get a new list back, but the new list can contain mutated data from the source list (or other sequence).

stuff = [...]
stuff = [thing.mutate() for thing in stuff]

I think it'd be more accurate to say "encourages," and I don't think this is just being pedantic.
[gravatar]
You probably already know, but there is an O'Reilly mini-book called "Functional Programming in Python" by David Mertz. Free pdf version available from them here:
https://www.oreilly.com/programming/free/files/functional-programming-python.pdf
[gravatar]
Pshaw! (Not you, the doubters.)

Python's a fine functional programming language, for a general purpose language.

In particular, there's one feature that I use over and over again, and that you don't have many other general purpose languages - you can just take a method from an object and use it as if it were a standalone function:
with open(fname, 'w') as fp:
    do_something_with(fp.write)
So slick. If you do that in Java or C++, the compiler will scream at you.

In Javascript, there's no error, and but you get this weird thing you can't actually call without knowing the original object _and_ using either .call() or .apply(). It is my belief that anyone who has tried to do functional programming in JS has made this mistake at least once.

----

Also, each new list comprehension creates to a brand-new list, so it's absolutely true that a list comprehension file forces you to create new data.

A generator expression does not:
stuff = # some iteratable
stuff = (i.mutate() for i in stuff)
stuff = (i for i in stuff if i and some_condition(i))
for i in stuff:
   #...
has one element in memory at a time and never creates a list.
[gravatar]
Awesome article! Also, if you love functional programming, check out https://github.com/dry-python/returns

It has the following features:
- Brings functional programming to Python land
- Provides a bunch of primitives to write declarative business logic
- Enforces better architecture
- Fully typed with annotations and checked with `mypy`, [PEP561 compatible](https://www.python.org/dev/peps/pep-0561/)
- Adds emulated Higher Kinded Types support
- Provides type-safe interfaces to create your own data-types with enforced laws
- Has a bunch of helpers for better composition
- Pythonic and pleasant to write and to read
- Support functions and coroutines, framework agnostic
- Easy to start: has lots of docs, tests, and tutorials

Add a comment:

Ignore this:
Leave this empty:
Name is required. Either email or web are required. Email won't be displayed and I won't spam you. Your web site won't be indexed by search engines.
Don't put anything here:
Leave this empty:
Comment text is Markdown.