Conda create environment and everything you need to know to manage conda virtual environment

Typical python projects uses multiple packages for various tasks. And some of the packages are shared between projects as well.

Sharing same packages between projects can cause problems.

How?

When you update one of the packages used in a project, it might cause compatibility issues in the other packages that use it. On upgrading, it can also cause dependency issues. That is, dependent packages that are use the code of the upgraded package can break.

This issue is effectively handled by using virtual environments.

Some of the popular virtual environment implementations for Python are:
1. Virtualenv
2. Conda
3. pipenv
4. venv

Several others exist.

However the most popular ones are Conda, Pipenv and venv as well. Specifically, Conda is popular amongst Data Scientists whereas pipenv is popular amongst software engineers.

Conda is a package manager and a virtual environment and it provides the convenience of allowing you to manage what version of Python the virtual environment (and as a result your project) uses as well. So naturally, conda is very convenient and I use it my projects as well.

Install Conda

You can install conda via anaconda or miniconda

Source: LinuxnetMag

How to use conda environment?

Well, you need to know a few commands to create and activate the environment and effortlessly install and uninstall package versions you want to use.

Let’s look at them.

But before you start make sure you’ve installed Anaconda. If you use windows, in ‘Start’ you need to type and start the ‘Anaconda prompt’. If you are on Mac or Linux, you can do all of these in Terminal.

1. Create conda environment

Conda centrally manages the environments you create, so, you don’t have to bother about creating a folder for specific environments yourself. You can either start by creating an empty environment or mention the python version and packages you need at the time of creation itself.

(i) Create an empty environment

conda create --name {env_name}
conda create --name mlenv

(ii) Create an environment + specific python version

conda create --name {env_name} {python==3.7.5}
conda create --name mlenv python==3.7.5

This will also install packages like pip, wheel, setuptools. You can then activate the environment (see below) and

(iii) Create an environment + specific Python version + packages

conda create --name env_name python==3.7.5 package_name1 package_name2

Example:

conda create --name mlenv python==3.7.5 pandas numpy

2. Activate the environment

conda activate {env_name}

To deactivate whichever you are currently in, use:

conda deactivate

3. Install more packages

Once activated you can install more packages using either conda or with pip.

With Conda

conda install pkg_name1==1.x.y pkg_name2==1.x.y

With pip

pip install pkg_name2==1.x.y pkg_name2==1.x.y

or install multiple packages from requirements.txt.

pip install -r requirements.txt

However, I don’t recommend using pip inside conda environment, especially when you want to another person to be able to replicate your environment and run the programs. See the “Sharing environments across platforms” section below if you want to know the exact reason.

4. See the list of packages and environments

(i) Show list of packages in current environment

conda list

(ii) See list of packages in specific environment

conda list -n myenv

(iii) See list of environments

conda env list
# or
conda info --envs

Sample Ouptut:

# conda environments:
C:\Users\selva\.julia\conda\3
base * C:\Users\selva\AppData\Local\Continuum\anaconda3
envmnt C:\Users\selva\AppData\Local\Continuum\anaconda3\envs\envmnt
juliaenv C:\Users\selva\AppData\Local\Continuum\anaconda3\envs\juliaenv
mlcourse C:\Users\selva\AppData\Local\Continuum\anaconda3\envs\mlcourse

The current active environment will be marked with star (*).

5. Remove an environment

After making sure you are not in the environment:

conda env remove -n env_name

6. Build an identical environment.

To create an environment that is identical to an existing one, explicitly create a spec file of the environment you want to duplicate and use it at the time of creating the new env.

Step 1: Create spec file

conda list --explicit > spec-file.txt

Step 2:

conda create --name myenv --file spec-file.txt

You can do this in the same machine or a different machine as well, if you have the spec list.

If you want to install the packages in spec file, in an existing environment, run this.

conda install --name env_name --file spec-file.txt

7. Sharing environments across platforms (better way)

There is a common problem when you try to replicate your environment in another system / platform.

When you create an environment and packages, lets say you run something like this (insert your package names).

conda install python=3.7 pkg_name1 pkg_name2 pkg_name3

This downloads and installs numerous other dependent packages in order to make the packages you wanted to install work.
This can easily introduce packages that may not be compatible across platforms.

So instead, use this:

conda env export --from-history > environment.yml

Adding --from-history flag will install only the packages you asked for using conda. It will NOT include the dependency packages or packages you installed using any other method.

Now, How to re-create the environment?

Pass the environment.yml and the other person can re-create your environment by running:

conda env create -f environment.yml

8. Restore / Rollback to an earlier version of an environment

Conda maintains a history of changes you make to an environment, by changes, I mean the changes you made using the conda commands. It allows you to roll back changes by using revision numbers.

Upon activating an environment, first see the revisions and the revision numbers using this.

conda list --revisions

Then, roll back

conda install --rev 3