Tutorial 4 - Virtual Environments

A virtual environment is a tool that helps to keep dependencies required by different projects separate by creating isolated Python virtual environments for them. Virtual environments create containers for each project, so that the Python interpreter, libraries, and scripts installed in one virtual environment are isolated from those installed in other virtual environments, as well as they are isolated from the libraries and packages that are installed in the main Python installation.

Therefore, where we create and activate a virtual environment for a specific project, the project becomes an independent application, which is isolated from the system-installed Python and from all other Python libraries. This project-specific virtual environment provides its own Python interpreter, and its own pip to install libraries, separated from other Python libraries.

By creating and managing separate environments for different projects, there is no chance of breaking currently installed packages in other environments. It also helps with reproducibility among developers and researchers working on large projects. For instance, if you have several different projects with their own virtual environments, they can have different versions of a library: for example, one project can use TensorFlow 1.12, and another can use TensorFlow 2.5. This way, we won’t worry whether an update to the TensorFlow library in the main system-installed Python would impact the code in all our projects.

There are several tools for managing virtual environments. Python 3.3 provides a standard module venv, which is most commonly used for managing virtual environments. It allows to manage separate package versions for different projects. When creating a new project, we can simply create a new virtual environment.

• The full official documentation for venv can be found here

• The full official user guide for venv can be found here

• The PEP proposal for venv can be found here

If you are looking for practical examples, it is recommended to consult the user guide. However, if you are looking for more information about specific details of venv, consulting the full documentation is recommended.

Installing venv

The module venv should be included in the standard Python library. If for some reason you must install it, this can be done with the following command:

python3 -m pip install --user virutalenv

Note: on Debian/Ubuntu systems, you will need to install the python3-venv package using the following command:

apt install python3.10-venv.

Creating a Virtual Environment

To create a virtual environment, run the following command:

To create a virutal envonment, run the following command:

python3 -m venv path/to/new/virutal/environment (in Unix/MacOS)

py -m pip install --user <virtual_environment_name> (in Windows)

Alternatively, there is a script installed with the venv library to make this more slightly more convenient:

pyvenv /path/to/new/virtual/environment (in Unix/MacOS)

python -m <virtual_environment_name> ./venv (in Windows)

Activating a Virtual Environment

Before we can start installing packages in the virtual environment, we must activate it. Doing so will put the virtual environment-specific Python and pip executables in your shell’s PATH.

To activate a virtual environment, run the following command:

source <path-to-venv>/bin/activate (in Unix/MacOS)

.\<virtual_environment_name>\Scripts\activate (in Windows)

To confirm that the virtual environment has been activated you can check the location of your Pytton interpreter:

which python (in Unix/MacOS)

where python (in Windows)

As long as the environment is active, you’ll be able to import packages installed in the environment.

To leave the environment run:

deactivate

Installing Packages

Make sure that the environment you wish to install packages into is active. Once this is done, it is as simple as installing packages through pip as you would normally. An example command to install requests (a popular library for making HTTP requests) is:

python3 -m pip install requests (in Unix/MacOS)

pip install requests (in Windows)

To check the list of all packages installed in the newly created virtual environment, use:

pip list

Similarly, we can generate a text file listing all installed libraries in a virtual environment with:

pip freeze > requirements.txt

This can be convenient, because if other users would like to replicate your virtual environment, instead of installing all libraries one by one, they can just run:

pip install -r requirements.txt

A Word of Caution and Some Best Practices

When a script is installed using venv, its shebang line (starting with #!) points to the environment’s Python interpreter. This means that these environments are inherently non-portable.

Moreover, if one is careful to include the absolute path to the desired environment interpreter in the shebang line, it is not necessary to activate the virtual environment before running the script.

For example, if you have an environment called env that has some non-standard packages installed, and you start writing a script that relies upon these packages but you did not want to (or can not) ensure the environment was active before running it, simply put something like the following as the shebang line:

#!/<virutal_environment_name>/bin/python

Conda Environment

Conda Env is a self-contained and isolated workspace within the Conda package management system, similar to venv. It allows to create and manage specific environments for different projects or applications, each with its own set of packages and dependencies.

The full official documentation for Conda Env can be found here.

Conda Env is pre-installed with Anaconda.

Creating a Conda Env with Commands

1. Open the terminal or an Anaconda Prompt.

2. Type: conda create --name myenv (or you can specify the python version: conda create -n myenv python=3.9).

3. Type y to proceed.

To activate a Conda Env, type:

conda activate myenv

To list all installed conda environments, type:

conda env list

To activate the base conda environment from the command prompt in Windows, just type:

conda activate

To deactivate the Conda Env you are in, type:

conda deactivate

Install Packages in a Conda Env

To install scikit-learn, for example, using Conda commands, type:

conda install -c anaconda scikit-learn

Or, you can use pip install as in:

pip install -U scikit-learn

BACK TO TOP