Python Virtual Environment: Difference between revisions
(23 intermediate revisions by the same user not shown) | |||
Line 4: | Line 4: | ||
* [[Python_Language#Virtual_Environment|Python Language]] | * [[Python_Language#Virtual_Environment|Python Language]] | ||
* [[Calling_Python_from_bash#Overview|Calling Python from bash]] | * [[Calling_Python_from_bash#Overview|Calling Python from bash]] | ||
* [[Poetry_Concepts#Virtual_Environments|Poetry]] | |||
=Overview= | =Overview= | ||
A virtual environment is a mechanism to isolate a set of installed dependencies. Virtual environments can be managed with tools like <code>[[virtualenv#Overview|virtualenv]]</code>, <code>[[Python Module venv|venv]]</code>, etc. and reside in directories called <code>venv</code> or <code>.venv</code> within the project's root. It is a good practice to avoid storing the content of <code>venv</code> or equivalent in source control. The content is populated locally on the developers' machines. | A virtual environment is a mechanism to isolate a set of installed dependencies. Virtual environments can be managed with tools like <code>[[virtualenv#Overview|virtualenv]]</code>, <code>[[Python Module venv|venv]]</code>, [[Poetry_Concepts#Virtual_Environments|Poetry]], etc. and reside in directories called <code>venv</code> or <code>.venv</code> within the project's root, or, as in the case for Poetry, in other locations. It is a good practice to avoid storing the content of <code>venv</code> or equivalent in source control. The content is populated locally on the developers' machines. | ||
=Virtual Environment Contents= | =Virtual Environment Contents= | ||
A virtual environment contains a relatively large number of files (thousands). | A virtual environment contains a relatively large number of files (thousands). | ||
Line 12: | Line 14: | ||
<font color=darkkhaki>What exactly does it contain? Does it contain a fully independent Python interpreter copy?</font> | <font color=darkkhaki>What exactly does it contain? Does it contain a fully independent Python interpreter copy?</font> | ||
= | =<span id='Manual_Creation'></span>Virtual Environment Creation= | ||
A virtual environment can be created manually as follows: | A virtual environment can be created manually as follows: | ||
<syntaxhighlight lang='bash'> | <syntaxhighlight lang='bash'> | ||
python -m venv <virtual-env-dir-name> | |||
</syntaxhighlight> | </syntaxhighlight> | ||
Example: | Example: | ||
<syntaxhighlight lang='bash'> | <syntaxhighlight lang='bash'> | ||
python -m venv .venv | |||
</syntaxhighlight> | </syntaxhighlight> | ||
The command creates a virtual Python environment in the specified target directory and also installs <code>pip</code>. | The command creates a virtual Python environment in the specified target directory, and installs the same Python version it is run with. By default, the Python binaries are '''linked''' from within <code>.venv/bin</code> directory. This is fine if the virtual environment is accessed from the system it was created, but it leads to broken links if the virtual environment is accessed from a different system. For more details on symbolic links versus copies, see "[[#Symbolic_Links_and_Copies|Symbolic Link and Copies]]" below. The command also installs <code>pip</code>. More more details on virtual environments and <code>pip</code> see "[[#Virtual_Environments_and_pip|Virtual Environments and pip]]" below. | ||
==Symbolic Links and Copies== | |||
When the virtual environment is created, the default behavior is to link the Python binaries from <code>.venv/bin</code> to the system installation. This is fine if the virtual environment is always accessed from the same system, but it will lead to broken symbolic links if the virtual environment is serialized and accessed from a different system. | |||
To ensure that the binaries are physically copied, use the <code>--copies</code> argument: | |||
<syntaxhighlight lang='bash'> | <syntaxhighlight lang='bash'> | ||
venv/bin/ | python -m venv --copies <virtual-env-dir-name> | ||
</syntaxhighlight> | |||
This will affect the following files: | |||
<font size=-1> | |||
.venv | |||
├─ bin | |||
├─ | |||
├─ python | |||
├─ python3 | |||
├─ python3.10 | |||
... | |||
</font> | |||
Another important aspect when using copies is to make sure the executable format of the binary file matches the image the virtual environment is accessed from: a virtual environment built on Mac will not work on a Linux image. | |||
==Virtual Environments and <tt>pip</tt>== | |||
The <code>venv</code> module installs <code>pip</code> as part of the virtual environment creation. After the virtual environment is created, it is usually a good idea to upgrade <code>pip</code>: | |||
<syntaxhighlight lang='bash'> | |||
.venv/bin/python -m pip install --upgrade pip | |||
</syntaxhighlight> | </syntaxhighlight> | ||
The dependencies can then be installed with: | The dependencies can then be installed with: | ||
<syntaxhighlight lang='bash'> | <syntaxhighlight lang='bash'> | ||
venv/bin/pip install -r requirements.txt | .venv/bin/pip install -r requirements.txt | ||
</syntaxhighlight> | </syntaxhighlight> | ||
For more details on virtual environments and dependencies see [[#Virtual_Environments_and_Dependencies|Virtual Environments and Dependencies]] below. | For more details on virtual environments and dependencies see [[#Virtual_Environments_and_Dependencies|Virtual Environments and Dependencies]] below. | ||
Line 34: | Line 61: | ||
To upgrade <code>pip</code> within an already initialized virtual environment: | To upgrade <code>pip</code> within an already initialized virtual environment: | ||
<syntaxhighlight lang='bash'> | <syntaxhighlight lang='bash'> | ||
venv/bin/ | venv/bin/python -m pip install --upgrade pip | ||
</syntaxhighlight> | </syntaxhighlight> | ||
=Virtual Environments and Dependencies= | =Virtual Environments and Dependencies= | ||
==Individual Dependencies== | |||
<syntaxhighlight lang='bash'> | |||
./venv/bin/pip install numpy | |||
</syntaxhighlight> | |||
==<tt>requirements.txt</tt> Dependencies== | |||
The dependencies can then be installed with: | The dependencies can then be installed with: | ||
<syntaxhighlight lang='bash'> | <syntaxhighlight lang='bash'> |
Latest revision as of 19:22, 19 September 2024
External
- https://peps.python.org/pep-0405/ PEP 405 – Python Virtual Environments
Internal
Overview
A virtual environment is a mechanism to isolate a set of installed dependencies. Virtual environments can be managed with tools like virtualenv
, venv
, Poetry, etc. and reside in directories called venv
or .venv
within the project's root, or, as in the case for Poetry, in other locations. It is a good practice to avoid storing the content of venv
or equivalent in source control. The content is populated locally on the developers' machines.
Virtual Environment Contents
A virtual environment contains a relatively large number of files (thousands).
What exactly does it contain? Does it contain a fully independent Python interpreter copy?
Virtual Environment Creation
A virtual environment can be created manually as follows:
python -m venv <virtual-env-dir-name>
Example:
python -m venv .venv
The command creates a virtual Python environment in the specified target directory, and installs the same Python version it is run with. By default, the Python binaries are linked from within .venv/bin
directory. This is fine if the virtual environment is accessed from the system it was created, but it leads to broken links if the virtual environment is accessed from a different system. For more details on symbolic links versus copies, see "Symbolic Link and Copies" below. The command also installs pip
. More more details on virtual environments and pip
see "Virtual Environments and pip" below.
Symbolic Links and Copies
When the virtual environment is created, the default behavior is to link the Python binaries from .venv/bin
to the system installation. This is fine if the virtual environment is always accessed from the same system, but it will lead to broken symbolic links if the virtual environment is serialized and accessed from a different system.
To ensure that the binaries are physically copied, use the --copies
argument:
python -m venv --copies <virtual-env-dir-name>
This will affect the following files:
.venv ├─ bin ├─ ├─ python ├─ python3 ├─ python3.10 ...
Another important aspect when using copies is to make sure the executable format of the binary file matches the image the virtual environment is accessed from: a virtual environment built on Mac will not work on a Linux image.
Virtual Environments and pip
The venv
module installs pip
as part of the virtual environment creation. After the virtual environment is created, it is usually a good idea to upgrade pip
:
.venv/bin/python -m pip install --upgrade pip
The dependencies can then be installed with:
.venv/bin/pip install -r requirements.txt
For more details on virtual environments and dependencies see Virtual Environments and Dependencies below.
Upgrade pip for an Already Initialized Virtual Environment
To upgrade pip
within an already initialized virtual environment:
venv/bin/python -m pip install --upgrade pip
Virtual Environments and Dependencies
Individual Dependencies
./venv/bin/pip install numpy
requirements.txt Dependencies
The dependencies can then be installed with:
venv/bin/pip install -r requirements.txt
TO TEST THAT: The dependencies installed in a virtual environment are used automatically if the interpreter is ./venv/bin/python
.
Also see:
Bash Wrapper that Bootstraps a Virtual Environment
Activated Virtual Environment Shell
An "activated" virtual environment means making the virtual environment Python interpreter the default interpreter for the shell session.
To activate the virtual environment:
source .venv/bin/activate
To "deactivate", run:
deactivate
or simply exit the shell.