Isolating your Python with VirtualEnv

By tom, April 16, 2017

You may have seen references to ‘virtualenv’ in documentation, in fact, I refer to it frequently in the docs for my input library. A few people have asked me what it is and why they should use it.

The simple explanation is that a virtual environment is a complete Python installation (interpreter, libraries, tools) which is separate from the system installation. Why might this be a good thing?

  1. Because you create it as a regular user, you can add libraries (by ‘pip install’ etc) without needing root access.
  2. Because you can set up a new virtual environment for each Python project you can be sure of exactly what libraries or Python versions you’re using for that project without worrying about you or someone else changing something later.

As an example, suppose you write some code that uses a particular library. It’s all fine, you can install the library into the system Python with ‘sudo pip install some-library’, you write your code, it’s all happy. A year later you write another piece of code, it uses the same library but now you want to take advantage of the latest version because of some new feature, you upgrade the library using pip, your system Python now has the latest version and your latest project works. A few days later you realise you need to run the code you wrote the previous year, and suddenly it doesn’t work! What’s happened? Turns out that the latest version of that library has changed the API you were using a year ago, so now you have a choice – you can abandon or upgrade your old code, or you can keep un-installing and re-installing the library with different versions depending on what you want to run.

How would this be different with virtual environments?

Well, first you’d have created a virtual environment for your first project, with the old library version installed and everything working. Then you’d have created a new virtual environment for your second project, with the newer version of the library installed. As the two environments are completely isolated from one another you won’t have over-written the old version of the library and both your projects will still run just fine.

While it’s not generally an issue for you on a Raspberry Pi, consider how much more annoying the above scenario would be if it was someone else who upgraded the library to make their code work!

Hopefully what you’ve read so far has convinced you that virtual environments are the way to go, so how do you go about creating and using them? Fortunately this is easy:

To create a new virtual environment you just do this:

> virtualenv ~/my-virtual-env

This will create a new installation of Python 2. Want Python 3? Just specify when you create the virtual environment:

> virtualenv --python=python3 ~/my-python3-virtual-env

You can be as specific as you like with the Python version, so if you really need to use Python2.3 for some reason you can (assuming it’s installed!)

Once you’ve created a new virtual environment you have to activate it. This sets up various paths so that when you type ‘python’ into your command line you get version from the virtual environment and not the system one. Inside your new virtual environment is a ‘bin’ directory, this contains a script called ‘activate’ which you need to source from the command line. The complete process, running on my Mac, looks like this:

Tengu:~ tom$ virtualenv  ~/my-virtual-env
New python executable in /Users/tom/my-virtual-env/bin/python
Installing setuptools, pip, wheel...done.
Tengu:~ tom$ source ~/my-virtual-env/bin/activate
(my-virtual-env)Tengu:~ tom$ which python
/Users/tom/my-virtual-env/bin/python

The ‘(my-virtual-env)’ before your command prompt is a nice reminder that you’re working inside your virtual environment.

On caveat with all this is that you won’t have access to the Python libraries you might have installed previously. In general this isn’t an issue, you can install almost everything by just doing ‘pip install’ once you’ve activated your environment. On the Pi in particular where quite a few libraries (e.g. for the SenseHat) are installed by default you’ll want to get the ones you’re using each time you create a new virtual environment. The plus side to this is that you’ll know exactly what libraries you’re using – very useful if you’re trying to run your code somewhere else or want to distribute it so others can use it.

Once you’ve activated your virtual environment, everything you do will use it instead of the system one. You can install whatever libraries you like without worrying about root privileges or potential version clashes. To really power up your virtual environments though you’ll also want to learn about setuptools, of which more later!

What do you think?

You must be logged in to post a comment.