DEV Community

Cover image for Pipfile and Pipenv : the future of Python dependencies management
Julien Tanay
Julien Tanay

Posted on • Originally published at Medium

Pipfile and Pipenv : the future of Python dependencies management

Yes, I heared you. pip is a great tool and has been around for quite a long time. But for 3 years or so, people (contributors) have been looking for a way to enhance our packages management experience. Think about the superpowers of composer, npm (or better, yarn) in your favorite tool.

What they offer is (more or less) a replacement for the age-old requirements.txt file : the Pipfile.

How I learn to stop worrying and love pip (again)

The new Pipfile differs from the old-style requirements file in several ways :

  • It uses TOML syntax to allow more configuration;
  • It declares dependencies groups (i.e. a default one plus a development one for... development packages) so you don't have to split your requirements into several files.
  • It comes with a Pipfile.lock file which pins the versions of your packages, in addition to some security bonus.

Let's have a look at a minimal, simple Pipfile:

[[source]]
url = 'https://pypi.python.org/simple'
verify_ssl = true
name = 'pypi'

[requires]
python_version = '3.6'

[packages]
flask = '*'

[dev-packages]
pytest = '*'
Enter fullscreen mode Exit fullscreen mode

What do we have here :

  • First we declare the source that pip is going to use. Typically, it will be pypi.
  • Then we declare our Python version requirement.
  • Finally, we declare our packages and development packages.

You got it ? Now here is the example Pipfile from the official repository on GitHub :

[[source]]
url = 'https://pypi.python.org/simple'
verify_ssl = true
name = 'pypi'

[requires]
python_version = '2.7'

[packages]
requests = { extras = ['socks'] }
records = '>0.5.0'
django = { git = 'https://github.com/django/django.git', ref = '1.11.4', editable = true }
"e682b37" = {file = "https://github.com/divio/django-cms/archive/release/3.4.x.zip"}
"e1839a8" = {path = ".", editable = true}
pywinusb = { version = "*", os_name = "=='nt'", index="pypi"}

[dev-packages]
nose = '*'
unittest2 = {version = ">=1.0,<3.0", markers="python_version < '2.7.9' or (python_version >= '3.0' and python_version < '3.4')"}
Enter fullscreen mode Exit fullscreen mode

See ? With the TOML syntax, you can be very specific about the version, source or os-variant you want to use. Note that all packages versions are not pinned, even if it was the preferred way of declaring dependencies.

Pip is going to support this specification in the furure with a syntax like pip install -p Pipfile. Let's see how we can use all this magical stuff today.

Back to the Future

Meet Pipenv (yep, that's another project from Kenneth Reitz, the guy behind httpbin, pep8.org, and, yes, request).

Pipenv is the current preferred implementation of Pipfile and allows you to manage your virtualenvs like a boss. It aims to ease and enhance how we manage our python environments (a la virtualenvwrapper, but with pip skills).

Here is a simple workflow :

1- In your project directory, create your virtualenv. Pipenv will store it in a centralized place instead of your project root directory :

pipenv --three # Yeah, 3 is the way to go
Enter fullscreen mode Exit fullscreen mode

2- Install some dependencies :

pipenv install flask
pipenv install --dev pytest # Notice the --dev flag
Enter fullscreen mode Exit fullscreen mode

3- Use your virtualenv. It will open a new shell :

pipenv shell
Enter fullscreen mode Exit fullscreen mode

Note that we were not in our virtualenv before this command. Everything is handled for us by Pipenv.

From this point, you are good to go. Pipenv will help you manage your environment throughout your development process.

Finally, you shouldn't be worried about compatibility issues with your deployment server (which is surely still using pip as you are reading this) : you can generate a good old requirements.txt by running :

pipenv lock -r > requirements.txt
Enter fullscreen mode Exit fullscreen mode

Pipenv handles your dependencies and it even adds some security reinforcements by specifying each package's hash.

That's it. Go test Pipenv and tell me what you think !

Cheers

NOTE: This post was originally posted on Medium.

Top comments (11)

Collapse
 
radumas profile image
Raphael Dumas • Edited

I wanted to have pipenv shell be the default integrated terminal in VSCode on Windows. Here are the settings I modified:

    "terminal.integrated.shell.windows": "C:/Anaconda3/Scripts/pipenv.exe",
    "terminal.integrated.shellArgs.windows": ["shell"]

This works when an integrated terminal is opened. However, there doesn't seem to be saved history within a pipenv shell session (there would normally be history saved for a cmd session).

Is there a way to set up a command history so you can up-arrow for previous commands?

Collapse
 
djiit profile image
Julien Tanay

Hi there! On Unix you could try something like SHELL=/usr/bin/zsh pipenv shell. Sadly I don't know the Windows equivalent, but the idea is : force-set your shell when running the pipenv cmd.

Cheers

Collapse
 
radumas profile image
Raphael Dumas

Oops, my comment was incomplete. I meant it works in VSCode in Windows, but my outstanding issue is a lack of command history in the pipenv shell

Collapse
 
rhymes profile image
rhymes

Loving pipenv (+ pyenv for multiple Python versions) so far.

The only thing that seems to be lacking is I can't find an easy way to know which packages have new available versions (like "bundle outdated" or "yarn outdated").

Did you find a solution?

Collapse
 
djiit profile image
Julien Tanay

There is pipenv check which uses the safety package from the guys at pyup.io and checks if there is any security update on major libraries -- but it won't cover all the packages updates.

Collapse
 
rhymes profile image
rhymes

Just leaving this here:

pipenv does have an "outdated" command according to this issue github.com/kennethreitz/pipenv/iss...

$ pipenv update --dry-run
Collapse
 
djiit profile image
Julien Tanay

Nice! Thanks ;)

Collapse
 
yaguirre profile image
Yorman Aguirre

Hello from the future, I just start using pipenv. It was very useful your post.

Thanks :D

Collapse
 
djiit profile image
Julien Tanay

Thanks for tour message! I'm happy this little post can help πŸ™

Collapse
 
k4ml profile image
Kamal Mustafa

Last I check pipenv, it still have issue with packages in private vcs repo.

Collapse
 
k4ml profile image
Kamal Mustafa

Look like the PR has been merged, I'll give it another try.

github.com/kennethreitz/pipenv/pul...