Converting your Python 2 code to Python 3
Python 2 reached the end of life on January 1, 2020. Python 3 has been available since 2008, but converting from 2 to 3 has been slow because of dependencies on libraries that were not available in Python 3 initially, earlier versions of python 3 were slower than python 2 and also because Python 2 was working quite well for many people. The Python 2 End of Life means no more supported releases and no bug fixes from Python. Some Linux distributions such as Ubuntu and Red Hat will be supporting Python 2 for some time but migrating to Python 3 is a better solution.
I thought Python 3 was backward compatible with Python 2?
Not 100%. The change from Python 2 to Python 3 was taken as an opportunity to "fix" some issues with Python 2. Among them promoting Unicode more uniformly throughout the language and to clear up some issues in the syntax such as print being a statement rather than a function.
A few of the differences between Python 2 and Python 3
Several statements such as print, exec and others have been changed to built-in functions. Examples:
print 'hello world' # valid python 2 not in python 3
print ('hello world') # valid python 3 and later versions of python 2
Integer Division has been changed as well. Example:
print (5 / 2) # in python 2 result is 2 in python 3 result is 2.5
// has been added to python 3 to support old behavior
Many functions now return unicode/bytes in python 3 instead of ascii/str strings in python 2. Example:
type (subprocess.checkout_output('cmd')) # returns <class 'bytes'> <class 'bytes'=""> in python 3 vs <type 'str'><type 'str'=""> in python 2
To convert the python 3 <class 'bytes'><class 'bytes'=""> to a string you'll want to use the .decode('ascii') option.
type (subprocess.check_output('ls').decode('ascii')) # will result in a <class 'str'><class 'str'=""> in python 3
Some libraries have been reorganized.
E.g. The python2 library urllib2 has been replaced with the urllib in python 3. Urllib on Python 3 is also available in modules such as urllib.request and urllib.error so you can import them to get similar functionality to python 2.
from urllib import urlopen # python 2
from urllib.request import urlopen # python 3
Converting from Python 2 to Python 3 mostly Automatically
While you can make all the changes to convert your python script from python 2 to python 3 by hand there is a tool that will do a lot of the heavy lifting for you. 2to3 is a package that is installable from the pip library and is also available as a separate package in many distributions repositories. To install via pip3
% pip3 install 2to3
To install on Debian/Ubuntu
% sudo apt install 2to3
To install on Fedora/CentOS
% sudo yum install 2to3
Note: installing from pip will get you the latest version, while the distribution versions may be a bit older, so pip installation might work better for you.
Running 2to3
$ 2to3 --help # will provide you the help message from the system
A couple of interesting options
-l # will provide a list of the fixes available in 2to3. E.g. print, exec, urllib, and others -x # explicitly not run a transformation, use if one of the "fixes" doesn't work for your code base -o # output dir, put transformed files into another location --add-suffix=SUFFIX # put a suffix on converted files --add-suffix='3', will convert .py files to .py3 files -w # overwrite current files with modified files
Assuming you have a directory of python 2 code (scripts) and a destination folder for the python 3 code (scripts3) you can do the following: The command to do the conversion is as follows
% 2to3 scripts -n -w -o scripts3
Options are as follows
scripts - source dir -n - no backups -w - write-unchanged files, write file even if no changes are required by 2to3 -o - output directory scripts3 - output directory, where the converted scripts are written
This way the original code is not modified and the user can review converted code in the scripts3 folder and still refer to the original code as well.
Modernize is a wrapper around 2to3
It has similar behavior to 2to3 except it supports the --six-unicode flag which will use the six helper functions that help with support Python 3.1 and Python 3.2. I would probably use 2to3 unless there is a compelling feature added to modernize that will encourage you to use it. Modernize is also available with some distributions such as Debian testing and Arch. To install
# pip3 install modernize
To run
# python-modernize --help
Tips
Dos
Do conversion in small chunks Do use tools like pylint to help you figure out problematic code, before you convert it Do compare the code before and after the conversion Do use the debugger to validate critical code
Don'ts
Do not overwrite your code with 2to3, recommend a separate directory Do not trust in the tools to do everything correctly
Test, Test, Test
Regardless of how you convert your code from Python 2 to Python 3 you should test it thoroughly. For mission critical python code I would even suggest using the python debugger to run the code line by line.
# python3 -m pdb# will run the script in the python debugger
Then type n
Conclusion
Python 2 is dead after a long and fruitful life. If you are still using Python 2 you should convert to Python 3. Python provides a tool in 2to3 that helps with the conversion from Python 2 to Python 3 and takes a lot of the legwork out of it. 2to3 is not a perfect tool, but it can do a lot of the work for you and reduce the complexity of converting from Python 2 to 3 by several orders of magnitude.
