Python Language Modularization
External
Internal
Overview
Python Script
A script is a module whose aim is to be executed. It has the same meaning as "program" or "application", but it is usually used to describe simple and small program. It contains a stored set of instructions that can be handed over to the Python interpreter:
python3 ./my-script.py
Python scripts have .py
extensions.
The python code can be specified in-line with a here-doc:
python3 <<EOF
print('hello')
print('hello2')
EOF
Top-Level Script
Packages can be run as if they were scripts if the package provides a top-level script __main__.py
.
- TO PROCESS: https://docs.python.org/3/library/__main__.html
- Understand setup.py - it defines the entry point.
Standalone Program
Module
A module is an organizational unit of Python code. It is a file with the .py
extension containing Python code that can be imported inside another Python program. It can define functions, classes and variables. A module can also include runnable code. The file name is the module name with the suffix .py
appended. Modules have a namespace containing arbitrary Python objects. Modules are loaded into Python by the process of importing, where the code in one module is made available to Python code in another module. Modules and packages are managed by a Python package manager like pip, Conda, Pipenv and Poetry.
Does this apply to module?
import pulumi_aws as aws
aws.ec2.SecurityGroup(....)
Module Name
The module name may not contain "_"
Module Search Path
When a module is imported from a code file, the runtime looks at a list of directory names and ZIP files stored in the standard sys
module as the variable path
. This list can be accessed and modified:
import sys
for i in sys.path:
print(i)
/opt/brew/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python39.zip /opt/brew/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9 /opt/brew/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload /Users/ovidiu/my-project/venv/lib/python3.9/site-packages
The initial blank output line is the empty string '', which stands for current directory. The first match will be used. If a module with the same name as a module from standard library is encountered in the search path before the standard library, it will be used instead of the module coming from the standard library.
Extending Module Search Path
Externally
Set the environment variable PYTHONPATH
to a colon-separated list of directories to search for imported modules. A directory declared in PYTHONPATH
will be inserted at the beginning of the sys.path
list when Python starts up.
Internally from the Program
Append to sys.path
import sys
sys.path.append('/path/to/search')
Append to site.addsitedir
Another way is to use site.addsitedir
to add a directory to sys.path
. The difference between this and just plain appending is that when you use addsitedir
, it also looks for .pth
files within that directory and uses them to possibly add additional directories to sys.path
based on the contents of the files.
Package
A package is a Python module which can contain submodules or recursively, subpackages. Technically, a package is a Python module with a __path__
attribute. Packages allow for a hierarchical structuring of the module namespace using dot notation. In the same way that module avoid collisions between global variable names, packages avoid collision between module names. For example, the urllib package contains several modules: urllib.request
, urllib.error
, etc. Modules and packages are managed by a Python package manager like pip, Conda, Pipenv and Poetry.
Not clear yet how to define working packages. I've defined a package with an __init__.py
and a __main__.py
with a function defined inside __main__.py
, and after import I get:
ImportError: cannot import name 'my_test_function' from 'my_package' (/Users/ovidiu/.../main/python/my_package/__init__.py)
Regular Package
A traditional package, such as a directory containing an __init__.py
file.
__init__.py
__main__.py
The "main" module, which will be imported automatically when the package is imported.
Namespace Package
A PEP 420 package which serves only as a container for subpackages. Namespace packages may have no physical representation, and have no __init__.py
file.
Package Metadata
Name: pulumi
Version: 2.11.2
Summary: Pulumi's Python SDK
Home-page: https://github.com/pulumi/pulumi
Author:
Author-email:
License: Apache 2.0
Location: /Users/ovidiu/Library/Python/3.8/lib/python/site-packages
Requires: dill, grpcio, protobuf
Required-by: pulumi-aws, pulumi-kubernetes, pulumi-random, pulumi-tls
Requires
Required-by
Importing
Importing All The Code from a Module
import mymodule
Clarify the difference between importing a module and a package.
TO PROCESS: The import system: https://docs.python.org/3/reference/import.html
Importing only a Function
from urllib.request import urlopen
Importing from a Relative Path
Clarify this idiom:
sys.path.append("../sdk") import some_module as blah_some_module import some_other_module as blah_some_other_module
Library
The term library is simply a generic term for a bunch of code that was designed with the aim of being reused by many applications. It provides some generic functionality that can be used by specific applications. When a module or a package is published, people refer to it as a library.
Standard Library
If the published module or package is is available in the Python Module Index, it is referred to as a "standard library".
Python Standard Library
Notable Modules and Packages
These modules are available in the Python Standard Library, hence are referred to as "standard libraries".
Python Module Index
Python Package Index PyPI
It can be searched with pip search
.
Organizatorium
- Each installation of Python may have different modules installed. Python determines the path to its modules by examining the location of the
python3
executable.