Python Module Internal Representation and Introspection
Internal
Overview
The module Class
All module and package instances are represented internally as instances of the module
class.
Checking whether an Object Instance is a Module
import types
import mymodule
assert isinstance(mymodule, types.ModuleType)
assert not isinstance('something', types.ModuleType)
Attributes
All object instances declared in the module (variables, functions, classes, etc.) become attributes of the module instance and they are accessible with inspect.getmembers()
.
Additionally, the following special attributes are present:
__name__
The __name__
special variable contains the name of the module, when it was imported, as string.
import mymodule
assert mymodule.__name__ == 'mymodule'
When the module is executed directly with python mymodule.py
, it is never imported, so __name__
is set to the "__main__" string.
__file__
Once imported, the file associated with the module can be determined using the module object's __file__
attribute, as string:
import mymodule
[...]
print(mymodule.__file__)
The directory portion of __file__
should be one of the directories in sys.path
.
__doc__
The content of the module docstring, if declared, otherwise None
.
__cached__
__loader__
__spec__
__package__
An empty string for a top-level module, the name of the package for a package or for a module that was loaded as part of a package.
__path__
The __path__
attribute exists only for module
instances that represent packages, not for those instances that represent ordinary modules.
__path__
contains a list of the package root directories, where the component modules, subpackages, __init__.py
and __main__.py
live.
To check whether __path__
exists, use hasattr()
.
Dynamic Module Tree Traversal and Class Loading
def find_class(module_or_package: types.ModuleType, predicate) -> type:
"""
If given a module, look for a class whose name, as string, satisfies the predicate and return the first match.
If given a package, recursively load the modules while descending in the package structure, look for a class whose name, as string, satisfies
the predicate and return the first match.
:param module_or_package: the module or the package instance. Must be imported by the calling layer.
:param predicate: a function that examines the class name, as string, and returns True if the class is acceptable, False otherwise.
:return: the class instance, or None if no such class exists
"""
if not isinstance(module_or_package, types.ModuleType):
raise TypeError(f'invalid module: {module_or_package}')
if not isinstance(predicate, types.FunctionType):
raise TypeError(f'invalid predicate: {predicate}')
if not hasattr(module_or_package, '__path__'):
# module
module = module_or_package
for name, value in inspect.getmembers(module):
if isinstance(value, type):
# a class, apply the predicate
if predicate(name):
return value
else:
# package
package = module_or_package
paths = package.__path__
for p in paths:
path = Path(p)
if not path.is_dir():
raise IllegalStateError(f'package {package.__name__} path not a directory: {p}')
file_names = []
dir_names = []
for f in path.iterdir():
# module or sub-package. Import in the local namespace and proceed recursively.
if f.name.startswith('__'):
continue
name = f.name.replace('.py', '')
if f.is_file():
file_names.append(name)
else:
dir_names.append(name)
# Process modules first, to favor classes declared closest from the top
all_names = file_names
all_names.extend(dir_names)
for name in all_names:
module_or_package = importlib.import_module(f'.{name}', package.__name__)
cls = find_class(module_or_package, predicate)
if cls:
return cls
Usage:
def predicate(class_name: str) -> bool:
return class_name == 'SomeClass'
import test_package
cls = find_class(test_package, predicate)