virtualenv 1.4.4

Virtual Python Environment builder

Status and License

virtualenv is a successor to workingenv, and an extension of virtual-python.

It is written by Ian Bicking, and sponsored by the Open Planning Project. It is licensed under an MIT-style permissive license.

You can install it with easy_install virtualenv, or from the hg repository or from a tarball easy_install virtualenv==tip.

What It Does

virtualenv is a tool to create isolated Python environments.

The basic problem being addressed is one of dependencies and versions, and indirectly permissions. Imagine you have an application that needs version 1 of LibFoo, but another application requires version 2. How can you use both these applications? If you install everything into /usr/lib/python2.4/site-packages (or whatever your platform’s standard location is), it’s easy to end up in a situation where you unintentionally upgrade an application that shouldn’t be upgraded.

Or more generally, what if you want to install an application and leave it be? If an application works, any change in its libraries or the versions of those libraries can break the application.

Also, what if you can’t install packages into the global site-packages directory? For instance, on a shared host.

In all these cases, virtualenv can help you. It creates an environment that has its own installation directories, that doesn’t share libraries with other virtualenv environments (and optionally doesn’t use the globally installed libraries either).

The basic usage is:

$ python virtualenv.py ENV

If you install it you can also just do virtualenv ENV.

This creates ENV/lib/python2.4/site-packages (or ENV/lib/python2.5/site-packages on Python 2.5, etc), where any libraries you install will go. It also creates ENV/bin/python, which is a Python interpreter that uses this environment. Anytime you use that interpreter (including when a script has #!/path/to/ENV/bin/python in it) the libraries in that environment will be used. (Note for Windows: scripts and executables on Windows go in ENV\Scripts\; everywhere you see bin/ replace it with Scripts\)

It also installs Setuptools for you, and if you use ENV/bin/easy_install the packages will be installed into the environment.

If you use the --distribute option, it will install distribute for you, instead of setuptools, and if you use ENV/bin/easy_install` the packages will be installed into the environment.

To use Distribute just call virtualenv like this:

$ python virtualenv.py --distribute ENV

You can also set the environment variable VIRTUALENV_USE_DISTRIBUTE (since 1.4.4) and be a good Comrade

Creating Your Own Bootstrap Scripts

While this creates an environment, it doesn’t put anything into the environment. Developers may find it useful to distribute a script that sets up a particular environment, for example a script that installs a particular web application.

To create a script like this, call virtualenv.create_bootstrap_script(extra_text), and write the result to your new bootstrapping script. Here’s the documentation from the docstring:

Creates a bootstrap script, which is like this script but with extend_parser, adjust_options, and after_install hooks.

This returns a string that (written to disk of course) can be used as a bootstrap script with your own customizations. The script will be the standard virtualenv.py script, with your extra text added (your extra text should be Python code).

If you include these functions, they will be called:

extend_parser(optparse_parser):

You can add or remove options from the parser here.

adjust_options(options, args):

You can change options here, or change the args (if you accept different kinds of arguments, be sure you modify args so it is only [DEST_DIR]).

after_install(options, home_dir):

After everything is installed, this function is called. This is probably the function you are most likely to use. An example would be:

def after_install(options, home_dir):
    if sys.platform == 'win32':
        bin = 'Scripts'
    else:
        bin = 'bin'
    subprocess.call([join(home_dir, bin, 'easy_install'),
                     'MyPackage'])
    subprocess.call([join(home_dir, bin, 'my-package-script'),
                     'setup', home_dir])

This example immediately installs a package, and runs a setup script from that package.

Bootstrap Example

Here’s a more concrete example of how you could use this:

import virtualenv, textwrap
output = virtualenv.create_bootstrap_script(textwrap.dedent("""
import os, subprocess
def after_install(options, home_dir):
    etc = join(home_dir, 'etc')
    if not os.path.exists(etc):
        os.makedirs(etc)
    subprocess.call([join(home_dir, 'bin', 'easy_install'),
                     'BlogApplication'])
    subprocess.call([join(home_dir, 'bin', 'paster'),
                     'make-config', 'BlogApplication',
                     join(etc, 'blog.ini')])
    subprocess.call([join(home_dir, 'bin', 'paster'),
                     'setup-app', join(etc, 'blog.ini')])
"""))
f = open('blog-bootstrap.py', 'w').write(output)

Another example is available here.

activate script

In a newly created virtualenv there will be a bin/activate shell script, or a Scripts/activate.bat batch file on Windows.

On Posix systems you can do:

$ source bin/activate

This will change your $PATH to point to the virtualenv bin/ directory, and update your prompt. Unlike workingenv, this is all it does; it’s a convenience. But if you use the complete path like /path/to/env/bin/python script.py you do not need to activate the environment first. You have to use source because it changes the environment in-place. After activating an environment you can use the function deactivate to undo the changes.

On Windows you just do:

> \path\to\env\bin\activate.bat

And use deactivate.bat to undo the changes.

The --no-site-packages Option

If you build with virtualenv --no-site-packages ENV it will not inherit any packages from /usr/lib/python2.5/site-packages (or wherever your global site-packages directory is). This can be used if you don’t have control over site-packages and don’t want to depend on the packages there, or you just want more isolation from the global system.

Using Virtualenv without bin/python

Sometimes you can’t or don’t want to use the Python interpreter created by the virtualenv. For instance, in a mod_python or mod_wsgi environment, there is only one interpreter.

Luckily, it’s easy. You must use the custom Python interpreter to install libraries. But to use libraries, you just have to be sure the path is correct. A script is available to correct the path. You can setup the environment like:

activate_this = '/path/to/env/bin/activate_this.py'
execfile(activate_this, dict(__file__=activate_this))

This will change sys.path and even change sys.prefix, but also allow you to use an existing interpreter. Items in your environment will show up first on sys.path, before global items. However, this cannot undo the activation of other environments, or modules that have been imported. You shouldn’t try to, for instance, activate an environment before a web request; you should activate one environment as early as possible, and not do it again in that process.

Making Environments Relocatable

Note: this option is somewhat experimental, and there are probably caveats that have not yet been identified. Also this does not currently work on Windows.

Normally environments are tied to a specific path. That means that you cannot move an environment around or copy it to another computer. You can fix up an environment to make it relocatable with the command:

$ virtualenv --relocatable ENV

This will make some of the files created by setuptools or distribute use relative paths, and will change all the scripts to use activate_this.py instead of using the location of the Python interpreter to select the environment.

Note: you must run this after you’ve installed any packages into the environment. If you make an environment relocatable, then install a new package, you must run virtualenv --relocatable again.

Also, this does not make your packages cross-platform. You can move the directory around, but it can only be used on other similar computers. Some known environmental differences that can cause incompatibilities: a different version of Python, when one platform uses UCS2 for its internal unicode representation and another uses UCS4 (a compile-time option), obvious platform changes like Windows vs. Linux, or Intel vs. ARM, and if you have libraries that bind to C libraries on the system, if those C libraries are located somewhere different (either different versions, or a different filesystem layout).

Currently the --no-site-packages option will not be honored if you use this on an environment.

Compare & Contrast with Alternatives

There are several alternatives that create isolated environments:

• workingenv (which I do not suggest you use anymore) is the predecessor to this library. It used the main Python interpreter, but relied on setting $PYTHONPATH to activate the environment. This causes problems when running Python scripts that aren’t part of the environment (e.g., a globally installed hg or bzr). It also conflicted a lot with Setuptools.

• virtual-python is also a predecessor to this library. It uses only symlinks, so it couldn’t work on Windows. It also symlinks over the entire standard library and global site-packages. As a result, it won’t see new additions to the global site-packages.

  This script only symlinks a small portion of the standard library into the environment, and so on Windows it is feasible to simply copy these files over. Also, it creates a new/empty site-packages and also adds the global site-packages to the path, so updates are tracked separately. This script also installs Setuptools automatically, saving a step and avoiding the need for network access.

• zc.buildout doesn’t create an isolated Python environment in the same style, but achieves similar results through a declarative config file that sets up scripts with very particular packages. As a declarative system, it is somewhat easier to repeat and manage, but more difficult to experiment with. zc.buildout includes the ability to setup non-Python systems (e.g., a database server or an Apache instance).

I strongly recommend anyone doing application development or deployment use one of these tools.

Other Documentation and Links

• James Gardner has written a tutorial on using virtualenv with Pylons.

• Blog announcement.

• Doug Hellmann wrote a description of his command-line work flow using virtualenv (virtualenvwrapper) including some handy scripts to make working with multiple environments easier. He also wrote an example of using virtualenv to try IPython.

• Chris Perkins created a showmedo video including virtualenv.

• Using virtualenv with mod_wsgi.

• virtualenv commands for some more workflow-related tools around virtualenv.