Python Module logging Concepts

Internal

• The logging module

TODO

• Basic Tutorial: https://docs.python.org/3/howto/logging.html#logging-basic-tutorial
• Advanced Tutorial: https://docs.python.org/3/howto/logging.html#logging-advanced-tutorial
• Logging Cookbook: https://docs.python.org/3/howto/logging-cookbook.html#logging-cookbook

Overview

The standard Python library module to provide logging support is logging.

import logging

logging.basicConfig(level=logging.DEBUG)
some_logger = logging.getLogger('some-logger')

logging.debug('debugging')
some_logger.debug('debugging via some_logger')

The output is:

DEBUG:root:debugging
DEBUG:some-logger:debugging via some_logger

Logging Levels

https://docs.python.org/3/library/logging.html#logging-levels

The logging system can be configured to filter out log messages whose logging level is under a certain threshold. The logging levels, and the corresponding logging functions are, listed from the most important to the least important, are provided below. By default, the logging system is configured to allow log messages of level WARNING and above, INFO and below are discarded. A different logging threshold can be configured.

Standard Logging Levels

CRITICAL

Accessible as logging.CRITICAL. Has numeric value 50. Generated with logging.critical().

ERROR

Accessible as logging.ERROR. Has numeric value 40. Generated with logging.error().

WARNING

Accessible as logging.WARNING. Has numeric value 30. Generated with logging.warning().

WARN seems to be deprecated. warn() is deprecated.

INFO

Accessible as logging.INFO. Has numeric value 20. Generated with logging.info().

DEBUG

Accessible as logging.DEBUG. Has numeric value 10. Generated with logging.debug().

NOTSET

Accessible as logging.NOTSET. Has numeric value 0.

Setting Logging Level

import logging

logging.basicConfig(level=logging.DEBUG)

Logger

The logger is the programmatic way to generate logging messages. By default, logging.debug()|info()|... syntax implies the usage of the root logger, but custom loggers can be created.

Logger Hierarchy

If the logger name contains dot characters, they separate levels of a hierarchy. Closer to the left, the higher the logger in the logger hierarchy: a has a higher level than a.b . Each level in the hierarchy can be given different properties. At the top of the hierarchy there's a root logger.

The root Logger

At the top of the logger hierarchy, there is a special root logger, called ' '.

Custom Loggers

Custom loggers can be created with logging.getLogger() function. Each logger has a name, and the presence of dots in the name place the logger in the logger hierarchy.

import logging

some_logger = logging.getLogger('some.logger')
some_other_logger = logging.getLogger('some.other.logger')

Handler

The handlers direct logging messages to their destinations (terminal, file, database, etc.).

To configure the logging system to use a FileHandler:

import logging
logging.basicConfig(filename = 'some_file.log')

Formatter

A formatter transforms the log message according to a certain pattern.

Format

The default format, without any customization is <LOGGING_LEVEL>:<logger_name>:<message>.

A format string can be provided to the configuration function, by specifying format variables.

logging.basicConfig(format='%(asctime)s %(levelname)s %(lineno)s %(message)s')

More examples:

format='[%(asctime)s] %(levelname)s:%(module)s:%(funcName)s: %(message)s'
logging.basicConfig(format=format, datefmt='%Y-%m-%d %H:%M:%S', level=logging.INFO)

Format Variables

Process this: LogRecord attributes: https://docs.python.org/3/library/logging.html#logrecord-attributes

There are several standard format variables. Custom format variables can also be used.

asctime

Date and time in ISO 8601 format.

pathname

The full pathname of the source file where the logging call was issued.

%(pathname)s

Message

Lazy Message Interpolation in Logging Functions

Avoid eager string interpolation in logging function for performance reasons: the interpolation should be done only if the corresponding logging level is enabled.

F-strings perform eager interpolations so they should be avoided.

Recommended style:

logger.error('caused by %s', msg)

%s works for integers too.

Patterns and Idioms

Custom Handling of Levels

https://docs.python.org/3/howto/logging-cookbook.html#custom-handling-of-levels