Python Module logging Concepts
Internal
TODO
- Basic Tutorial: https://docs.python.org/3/howto/logging.html#logging-basic-tutorial
- Advanced Tutorial: https://docs.python.org/3/howto/logging.html#logging-advanced-tutorial
- Logging Cookbook: https://docs.python.org/3/howto/logging-cookbook.html#logging-cookbook
Overview
The standard Python library module to provide logging support is logging.
import logging
logging.basicConfig(level=logging.DEBUG)
some_logger = logging.getLogger('some-logger')
logging.debug('debugging')
some_logger.debug('debugging via some_logger')
The output is:
DEBUG:root:debugging
DEBUG:some-logger:debugging via some_logger
Logging Levels
The logging system can be configured to filter out log messages whose logging level is under a certain threshold. The logging levels, and the corresponding logging functions are, listed from the most important to the least important, are provided below. By default, the logging system is configured to allow log messages of level WARNING and above, INFO and below are discarded. A different logging threshold can be configured.
Standard Logging Levels
CRITICAL
Accessible as logging.CRITICAL. Has numeric value 50. Generated with logging.critical().
ERROR
Accessible as logging.ERROR. Has numeric value 40. Generated with logging.error().
WARNING
Accessible as logging.WARNING. Has numeric value 30. Generated with logging.warning().
WARN seems to be deprecated. warn() is deprecated.
INFO
Accessible as logging.INFO. Has numeric value 20. Generated with logging.info().
DEBUG
Accessible as logging.DEBUG. Has numeric value 10. Generated with logging.debug().
NOTSET
Accessible as logging.NOTSET. Has numeric value 0.
Setting Logging Level
import logging
logging.basicConfig(level=logging.DEBUG)
Logger
The logger is the programmatic way to generate logging messages. By default, logging.debug()|info()|... syntax implies the usage of the root logger, but custom loggers can be created.
Logger Hierarchy
If the logger name contains dot characters, they separate levels of a hierarchy. Closer to the left, the higher the logger in the logger hierarchy: a has a higher level than a.b . Each level in the hierarchy can be given different properties. At the top of the hierarchy there's a root logger.
The root Logger
At the top of the logger hierarchy, there is a special root logger, called ' '.
Custom Loggers
Custom loggers can be created with logging.getLogger() function. Each logger has a name, and the presence of dots in the name place the logger in the logger hierarchy.
import logging
some_logger = logging.getLogger('some.logger')
some_other_logger = logging.getLogger('some.other.logger')
Handler
The handlers direct logging messages to their destinations (terminal, file, database, etc.).
To configure the logging system to use a FileHandler:
import logging
logging.basicConfig(filename = 'some_file.log')
Handlers
FileHandler
Formatter
A formatter transforms the log message according to a certain pattern.
Format
The default format, without any customization is <LOGGING_LEVEL>:<logger_name>:<message>.
A format string can be provided to the configuration function, by specifying format variables.
logging.basicConfig(format='%(asctime)s %(levelname)s %(lineno)s %(message)s')
More examples:
format='[%(asctime)s] %(levelname)s:%(module)s:%(funcName)s: %(message)s'
logging.basicConfig(format=format, datefmt='%Y-%m-%d %H:%M:%S', level=logging.INFO)
Format Variables
Process this: LogRecord attributes: https://docs.python.org/3/library/logging.html#logrecord-attributes
There are several standard format variables. Custom format variables can also be used.
asctime
Date and time in ISO 8601 format.
levelname
lineno
message
module
pathname
The full pathname of the source file where the logging call was issued.
%(pathname)s
Filter
Message
Lazy Message Interpolation in Logging Functions
Avoid eager string interpolation in logging function for performance reasons: the interpolation should be done only if the corresponding logging level is enabled.
F-strings perform eager interpolations so they should be avoided.
Recommended style:
logger.error('caused by %s', msg)
%s works for integers too.