Written by Nikita Sobolev✏️
Dark forces cast their wicked spells to leak into our realm of precious Python programs.
They spam their twisted magic uncontrollably and pollute our readable code.
Today I am going to reveal several chthonic creatures that might already live inside your codebase and accustom themselves enough to start making their own rules. We need a hero to protect our peaceful world from these evil entities. And you will be this hero to fight them!
All heroes need weapons enchanted with light magic to serve them well in their epic battles.
wemake-python-styleguide will be your sharp weapon and your best companion.
Let’s start our journey!
Space invaders
Not so long ago, space invaders were spotted in Python. They take bizarre forms.
5:5 E225 missing whitespace around operator
x -=- x
^
5:5 WPS346 Found wrong operation sign
x -=- x
^
10:2 E225 missing whitespace around operator
o+=+o
^
14:10 E225 missing whitespace around operator
print(3 --0-- 5 == 8)
^
14:10 WPS346 Found wrong operation sign
print(3 --0-- 5 == 8)
^
14:11 WPS345 Found meaningless number operation
print(3 --0-- 5 == 8)
^
14:12 E226 missing whitespace around arithmetic operator
print(3 --0-- 5 == 8)
^
14:13 WPS346 Found wrong operation sign
print(3 --0-- 5 == 8)
^
This is how our code base should look afterward:
x = 1
x += x
o = 2
o += o
print(3 + 5 == 8)
Readable and clean!
Mystical dots
Some citizens report that some strange codeglyphes are starting to appear. Look, here they are!
print(0..__eq__(0))
# => True
print(....__eq__(((...))))
# => True
What is going on here? Looks like a partial float and Ellipsis to me, but better be sure.
21:7 WPS609 Found direct magic attribute usage: __eq__
print(0..__eq__(0))
^
21:7 WPS304 Found partial float: 0.
print(0..__eq__(0))
^
24:7 WPS609 Found direct magic attribute usage: __eq__
print(....__eq__(((...))))
^
Ouch! Now we are sure. It is indeed the partial float with dot property access and Elipsis with the same dot access. Let’s reveal all the hidden things now:
print(0.0 == 0)
print(... == ...)
And still, it is better not to provoke wrath and not to compare constants in other places.
Misleading path
We have a new incident. Some values have never seen returned from a function. Let’s find out what is going on.
def some_func():
try:
return 'from_try'
finally:
return 'from_finally'
some_func()
# => 'from_finally'
We are missing 'from_try' due to a broken entity in our code, how this can be addressed?
31:5 WPS419 Found `try`/`else`/`finally` with multiple return paths
try:
^
Turns out wemake-python-styleguide knew it all along the way! It teaches us to never return from finally. Let’s obey it.
def some_func():
try:
return 'from_try'
finally:
print('now in finally')
The C-ursed legacy
Some ancient creature is awakening. It hasn’t been seen for decades. And now it has returned.
a = [(0, 'Hello'), (1, 'world')]
for ['>']['>'>'>'], x in a:
print(x)
What is going on here? One can implicitly unpack values inside loops. And the target for unpacking might be almost any valid Python expression.
But, we should not do a lot of things from this example:
44:1 WPS414 Found incorrect unpacking target
for ['>']['>'>'>'], x in a:
^
44:5 WPS405 Found wrong `for` loop variable definition
for ['>']['>'>'>'], x in a:
^
44:11 WPS308 Found constant compare
for ['>']['>'>'>'], x in a:
^
44:14 E225 missing whitespace around operator
for ['>']['>'>'>'], x in a:
^
44:21 WPS111 Found too short name: x
for ['>']['>'>'>'], x in a:
^
Looks like the ['>'\]['>'>'>'] is just ['>'\][0] because '>' > '>' is False.
This case is solved.
Signature of the black sorcerer
How complex can an expression be in Python? The Black Sorcerer leaves his complex mark on all classes he touches:
class _:
# There are four of them, do you see it?
_: [(),...,()] = {((),...,()): {(),...,()}}[((),...,())]
print(_._) # this operator also looks familiar 🤔
# => {(), Ellipsis}
How can this signature be read and evaluated? Looks like it consists of several parts:
– Declaration and type annotation: _: [(),...,()] =
– Dictionary definition with a set as a value: = { ((),...,()): {(),...,()} }
– Key access: [((),...,())]
While it does not make any sense to human beings from this world, it is still a valid Python code that can be used for something evil. Let’s remove it:
55:5 WPS122 Found all unused variables definition: _
_: [(),...,()] = {((),...,()): {(),...,()}}[((),...,())]
^
55:5 WPS221 Found line with high Jones Complexity: 19
_: [(),...,()] = {((),...,()): {(),...,()}}[((),...,())]
^
55:36 WPS417 Found non-unique item in hash: ()
_: [(),...,()] = {((),...,()): {(),...,()}}[((),...,())]
^
57:7 WPS121 Found usage of a variable marked as unused: _
print(_._) # this operator also looks familiar
^
And now this complex expression (with Jones Complexity rate of 19) is removed or refactored. Any the Signature of the Black Sourcerer is removed from this poor class. Let’s leave it in peace.
Metamagic
Our regular classes start to hang out with some shady types. We need to protect them from this bad influence.
Currently, their output really strange:
class Example(type((lambda: 0.)())):
...
print(Example(1) + Example(3))
# => 4.0
Why 1 + 3 is 4.0 and not 4? To find out, let’s unwrap the type((lambda: 0.)()) piece:
– (lambda: 0.)() is just 0. which is just 0.0.
– type(0.0) is float
– When we write Example(1) it is converted to Example(1.0) inside the class.
– Example(1.0) + Example(3.0) is Example(4.0)
Let’s be sure that our weapon is sharp as always:
63:15 WPS606 Found incorrect base class
class Example(type((lambda: 0.)())):
^
63:21 WPS522 Found implicit primitive in a form of lambda
class Example(type((lambda: 0.)())):
^
63:29 WPS304 Found partial float: 0.
class Example(type((lambda: 0.)())):
^
64:5 WPS428 Found statement that has no effect
...
^
64:5 WPS604 Found incorrect node inside `class` body
...
^
We have found all the possible issues here. Our classes are safe. Time to move on.
Regenerators
So similar and yet so different. Regenerator is found in our source code. It looks like an average generator expression, but it’s something totally different.
a = ['a', 'b']
print(set(x + '!' for x in a))
# => {'b!', 'a!'}
print(set((yield x + '!') for x in a))
# => {'b!', None, 'a!'}
This is a bug in Python — yes, they do exist. And since python3.8 is a SyntaxError, one should not use yield and yield from outside of generator functions.
Here’s our usual report about the incident:
73:7 C401 Unnecessary generator - rewrite as a set comprehension.
print(set(x + '!' for x in a))
^
76:7 C401 Unnecessary generator - rewrite as a set comprehension.
print(set((yield x + '!') for x in a))
^
76:11 WPS416 Found `yield` inside comprehension
print(set((yield x + '!') for x in a))
Also, let’s write comprehensions correctly as suggested.
print({x + '!' for x in a})
This was a hard one to solve. But in the end, Regenerator is gone and so are wrong comprehensions. What’s next?
Email evil clone
If one needs to write an email address, the string is used. Right? Wrong!
There are unusual ways to do regular things. And there are evil clones of regular datatypes.
We are going to discover them.
class G:
def __init__(self, s):
self.s = s
def __getattr__(self, t):
return G(self.s + '.' + str(t))
def __rmatmul__(self, other):
return other + '@' + self.s
username, example = 'username', G('example')
print(username@example.com)
# => username@example.com
How does it work?
– @ is an operator in Python, it’s behavior can be modified via __matmul__ and __rmatmul__ magic methods
– .com is an attribute com dot access, it can be modified via __getattr__
One big difference between this code and other examples is that this one is actually valid. Just unusual. We should probably not use it. But, let’s write this into our knowledge quest book.
Fallacy of the walrus
The darkness has fallen onto Python. The one that has split the friendly developer community, the one that brought the controversy.
You have gained the power to program in strings:
from math import radians
for angle in range(360):
print(f'{angle=} {(th:=radians(angle))=:.3f}')
print(th)
# => angle=0 (th:=radians(angle))=0.000
# => 0.0
# => angle=1 (th:=radians(angle))=0.017
# => 0.017453292519943295
# => angle=2 (th:=radians(angle))=0.035
# => 0.03490658503988659
What is going on here?
– f'{angle=} is a new (python3.8+) way to write f'angle={angle}
– (th:=radians(angle)) is an assignment expression, yes you can do assignments in strings now
– =:.3f is the formatting part, it returns the expression and its rounded result value
– print(th) works because (th:=radians(angle)) has the local scope effect
Should you use assignment expressions? Well, that’s up to you.
Should you assign values inside strings? Absolutely not.
And here’s a friendly reminder of things that you can (but also probably should not) do with f strings themselves:
print(f"{getattr(__import__('os'), 'eman'[None:None:-1])}")
# => posix
Just a regular module import inside a string — move on, nothing to see here.
Luckily, we are not allowed to write this line in our real code:
105:1 WPS221 Found line with high Jones Complexity: 16
print(f"{getattr(__import__('os'), 'eman'[None:None:-1])}")
^
105:7 WPS305 Found `f` string
print(f"{getattr(__import__('os'), 'eman'[None:None:-1])}")
^
105:18 WPS421 Found wrong function call: __import__
print(f"{getattr(__import__('os'), 'eman'[None:None:-1])}")
^
105:36 WPS349 Found redundant subscript slice
print(f"{getattr(__import__('os'), 'eman'[None:None:-1])}")
^
And one more thing: f strings cannot be used as docstrings:
def main():
f"""My name is {__file__}/{__name__}!"""
print(main().__doc__)
# => None
Conclusion
We fought many ugly monsters that spawned inside our code and made Python land a better place to live. You should be proud of yourself, hero!
That was an epic journey. And I hope you learned something new: to be stronger for the next battles to come. The world needs you!
That’s it for today. Stay safe, traveler.
Useful links
Editor's note: Seeing something wrong with this post? You can find the correct version here.
Plug: LogRocket, a DVR for web apps
LogRocket is a frontend logging tool that lets you replay problems as if they happened in your own browser. Instead of guessing why errors happen, or asking users for screenshots and log dumps, LogRocket lets you replay the session to quickly understand what went wrong. It works perfectly with any app, regardless of framework, and has plugins to log additional context from Redux, Vuex, and @ngrx/store.
In addition to logging Redux actions and state, LogRocket records console logs, JavaScript errors, stacktraces, network requests/responses with headers + bodies, browser metadata, and custom logs. It also instruments the DOM to record the HTML and CSS on the page, recreating pixel-perfect videos of even the most complex single-page apps.
Try it for free.
The post Frustrations in Python appeared first on LogRocket Blog.
Top comments (0)