Python Virtual Environment: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(12 intermediate revisions by the same user not shown)
Line 4: Line 4:
* [[Python_Language#Virtual_Environment|Python Language]]
* [[Python_Language#Virtual_Environment|Python Language]]
* [[Calling_Python_from_bash#Overview|Calling Python from bash]]
* [[Calling_Python_from_bash#Overview|Calling Python from bash]]
* [[Poetry_Concepts#Virtual_Environments|Poetry]]


=Overview=
=Overview=
A virtual environment is a mechanism to isolate a set of installed dependencies. Virtual environments can be managed with tools like <code>[[virtualenv#Overview|virtualenv]]</code>, <code>[[Python Module venv|venv]]</code>, etc. and reside in directories called <code>venv</code> or <code>.venv</code> within the project's root. It is a good practice to avoid storing the content of <code>venv</code> or equivalent in source control. The content is populated locally on the developers' machines.
A virtual environment is a mechanism to isolate a set of installed dependencies. Virtual environments can be managed with tools like <code>[[virtualenv#Overview|virtualenv]]</code>, <code>[[Python Module venv|venv]]</code>, [[Poetry_Concepts#Virtual_Environments|Poetry]], etc. and reside in directories called <code>venv</code> or <code>.venv</code> within the project's root, or, as in the case for Poetry, in other locations. It is a good practice to avoid storing the content of <code>venv</code> or equivalent in source control. The content is populated locally on the developers' machines.
 
=Virtual Environment Contents=
=Virtual Environment Contents=
A virtual environment contains a relatively large number of files (thousands).
A virtual environment contains a relatively large number of files (thousands).
Line 19: Line 21:
Example:
Example:
<syntaxhighlight lang='bash'>
<syntaxhighlight lang='bash'>
python -m venv venv
python -m venv .venv
</syntaxhighlight>
</syntaxhighlight>
The command creates a virtual Python environment in the specified target directory, and installs the same Python version it is run with. By default, the Python binaries are '''linked''' from within <code>venv/bin</code> directory. This is fine if the virtual environment is accessed from the system it was created, but it lead to broken links if the virtual environment is accessed from a different system. For more details on symbolic links versus copies, see "[[#Symbolic_Links_and_Copies|Symbolic Link and Copies]]" below. The command also installs <code>pip</code>. More more details on virtual environments and <code>pip</code> see "[[#Virtual_Environments_and_pip|Virtual Environments and pip]]" below.
The command creates a virtual Python environment in the specified target directory, and installs the same Python version it is run with. By default, the Python binaries are '''linked''' from within <code>.venv/bin</code> directory. This is fine if the virtual environment is accessed from the system it was created, but it leads to broken links if the virtual environment is accessed from a different system. For more details on symbolic links versus copies, see "[[#Symbolic_Links_and_Copies|Symbolic Link and Copies]]" below. The command also installs <code>pip</code>. More more details on virtual environments and <code>pip</code> see "[[#Virtual_Environments_and_pip|Virtual Environments and pip]]" below.
==Symbolic Links and Copies==
==Symbolic Links and Copies==
When the virtual environment is created, the default behavior is to link the Python binaries from <code>venv/bin</code> to the system installation. This is fine if the virtual environment is always accessed from the same system, but it will lead to broken symbolic links if the virtual environment is serialized and accessed from a different system.
When the virtual environment is created, the default behavior is to link the Python binaries from <code>.venv/bin</code> to the system installation. This is fine if the virtual environment is always accessed from the same system, but it will lead to broken symbolic links if the virtual environment is serialized and accessed from a different system.


To ensure that the binaries are physically copied, use the <code>--copies</code> argument:
To ensure that the binaries are physically copied, use the <code>--copies</code> argument:
Line 34: Line 36:


<font size=-1>
<font size=-1>
  venv
  .venv
  ├─ bin
  ├─ bin
        ├─
      ├─
        ├─ python  
      ├─ python  
        ├─ python3
      ├─ python3
        ├─ python3.10
      ├─ python3.10
        ...
      ...
</font>
</font>


Line 48: Line 50:
The <code>venv</code> module installs <code>pip</code> as part of the virtual environment creation. After the virtual environment is created, it is usually a good idea to upgrade <code>pip</code>:
The <code>venv</code> module installs <code>pip</code> as part of the virtual environment creation. After the virtual environment is created, it is usually a good idea to upgrade <code>pip</code>:
<syntaxhighlight lang='bash'>
<syntaxhighlight lang='bash'>
venv/bin/python3 -m pip install --upgrade pip
.venv/bin/python -m pip install --upgrade pip
</syntaxhighlight>
</syntaxhighlight>
The dependencies can then be installed with:
The dependencies can then be installed with:
<syntaxhighlight lang='bash'>
<syntaxhighlight lang='bash'>
venv/bin/pip install -r requirements.txt
.venv/bin/pip install -r requirements.txt
</syntaxhighlight>
</syntaxhighlight>
For more details on virtual environments and dependencies see [[#Virtual_Environments_and_Dependencies|Virtual Environments and Dependencies]] below.
For more details on virtual environments and dependencies see [[#Virtual_Environments_and_Dependencies|Virtual Environments and Dependencies]] below.
Line 59: Line 61:
To upgrade <code>pip</code> within an already initialized virtual environment:
To upgrade <code>pip</code> within an already initialized virtual environment:
<syntaxhighlight lang='bash'>
<syntaxhighlight lang='bash'>
venv/bin/python3 -m pip install --upgrade pip
venv/bin/python -m pip install --upgrade pip
</syntaxhighlight>
</syntaxhighlight>
=Virtual Environments and Dependencies=
=Virtual Environments and Dependencies=
==Individual Dependencies==
<syntaxhighlight lang='bash'>
./venv/bin/pip install numpy
</syntaxhighlight>
==<tt>requirements.txt</tt> Dependencies==
The dependencies can then be installed with:
The dependencies can then be installed with:
<syntaxhighlight lang='bash'>
<syntaxhighlight lang='bash'>

Latest revision as of 19:22, 19 September 2024

External

Internal

Overview

A virtual environment is a mechanism to isolate a set of installed dependencies. Virtual environments can be managed with tools like virtualenv, venv, Poetry, etc. and reside in directories called venv or .venv within the project's root, or, as in the case for Poetry, in other locations. It is a good practice to avoid storing the content of venv or equivalent in source control. The content is populated locally on the developers' machines.

Virtual Environment Contents

A virtual environment contains a relatively large number of files (thousands).

What exactly does it contain? Does it contain a fully independent Python interpreter copy?

Virtual Environment Creation

A virtual environment can be created manually as follows:

python -m venv <virtual-env-dir-name>

Example:

python -m venv .venv

The command creates a virtual Python environment in the specified target directory, and installs the same Python version it is run with. By default, the Python binaries are linked from within .venv/bin directory. This is fine if the virtual environment is accessed from the system it was created, but it leads to broken links if the virtual environment is accessed from a different system. For more details on symbolic links versus copies, see "Symbolic Link and Copies" below. The command also installs pip. More more details on virtual environments and pip see "Virtual Environments and pip" below.

Symbolic Links and Copies

When the virtual environment is created, the default behavior is to link the Python binaries from .venv/bin to the system installation. This is fine if the virtual environment is always accessed from the same system, but it will lead to broken symbolic links if the virtual environment is serialized and accessed from a different system.

To ensure that the binaries are physically copied, use the --copies argument:

python -m venv --copies <virtual-env-dir-name>

This will affect the following files:

.venv
  ├─ bin
      ├─
      ├─ python 
      ├─ python3
      ├─ python3.10
      ...

Another important aspect when using copies is to make sure the executable format of the binary file matches the image the virtual environment is accessed from: a virtual environment built on Mac will not work on a Linux image.

Virtual Environments and pip

The venv module installs pip as part of the virtual environment creation. After the virtual environment is created, it is usually a good idea to upgrade pip:

.venv/bin/python -m pip install --upgrade pip

The dependencies can then be installed with:

.venv/bin/pip install -r requirements.txt

For more details on virtual environments and dependencies see Virtual Environments and Dependencies below.

Upgrade pip for an Already Initialized Virtual Environment

To upgrade pip within an already initialized virtual environment:

venv/bin/python -m pip install --upgrade pip

Virtual Environments and Dependencies

Individual Dependencies

./venv/bin/pip install numpy

requirements.txt Dependencies

The dependencies can then be installed with:

venv/bin/pip install -r requirements.txt

TO TEST THAT: The dependencies installed in a virtual environment are used automatically if the interpreter is ./venv/bin/python.

Also see:

pip
requirements.txt

Bash Wrapper that Bootstraps a Virtual Environment

Calling Python from bash | bash Wrapper that Bootstraps a Virtual Environment

Activated Virtual Environment Shell

An "activated" virtual environment means making the virtual environment Python interpreter the default interpreter for the shell session.

To activate the virtual environment:

source .venv/bin/activate

To "deactivate", run:

deactivate

or simply exit the shell.