Python Module logging Concepts: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(36 intermediate revisions by the same user not shown)
Line 1: Line 1:
=Internal=
=Internal=
* [[Python Logging#Subjects|Python Logging]]
* [[Python_Module_logging#Overview|The logging module]]
 
=TODO=
<font color=darkkhaki>
* Basic Tutorial: https://docs.python.org/3/howto/logging.html#logging-basic-tutorial
* Advanced Tutorial: https://docs.python.org/3/howto/logging.html#logging-advanced-tutorial
* Logging Cookbook: https://docs.python.org/3/howto/logging-cookbook.html#logging-cookbook
</font>
 
=Overview=
The standard Python library module to provide logging support is <code>[[Python_Module_logging#Overview|logging]]</code>.
 
<syntaxhighlight lang='py'>
import logging
 
logging.basicConfig(level=logging.DEBUG)
some_logger = logging.getLogger('some-logger')
 
logging.debug('debugging')
some_logger.debug('debugging via some_logger')
</syntaxhighlight>
 
The output is:
<syntaxhighlight lang='text'>
DEBUG:root:debugging
DEBUG:some-logger:debugging via some_logger
</syntaxhighlight>
 
=Logging Levels=
{{External|https://docs.python.org/3/library/logging.html#logging-levels}}
 
The logging system can be configured to filter out log messages whose logging level is under a certain threshold. The logging levels, and the corresponding logging functions are, listed from the most important to the least important, are provided below. By default, the logging system is configured to allow log messages of level [[#WARNING|WARNING]] and above, [[#INFO|INFO]] and below are discarded. A different logging threshold [[#Setting_Logging_Level|can be configured]].
==Standard Logging Levels==
====<tt>CRITICAL</tt>====
Accessible as <code>logging.CRITICAL</code>. Has numeric value 50. Generated with <code>logging.critical()</code>.
====<tt>ERROR</tt>====
Accessible as <code>logging.ERROR</code>. Has numeric value 40. Generated with <code>logging.error()</code>.
====<tt>WARNING</tt>====
Accessible as <code>logging.WARNING</code>. Has numeric value 30. Generated with <code>logging.warning()</code>.
 
<font color=darkkhaki>WARN seems to be deprecated.</font> <code>warn()</code> is deprecated.
====<tt>INFO</tt>====
Accessible as <code>logging.INFO</code>. Has numeric value 20. Generated with <code>logging.info()</code>.
====<tt>DEBUG</tt>====
Accessible as <code>logging.DEBUG</code>. Has numeric value 10. Generated with <code>logging.debug()</code>.
====<tt>NOTSET</tt>====
Accessible as <code>logging.NOTSET</code>. Has numeric value 0.
 
==Setting Logging Level==
<syntaxhighlight lang='py'>
import logging
 
logging.basicConfig(level=logging.DEBUG)
</syntaxhighlight>


=Logger=
=Logger=
The logger is the programmatic way to generate logging messages. By default, <code>logging.debug()|info()|...</code> syntax implies the usage of the [[#The_root_Logger|root logger]], but [[#Custom_Logger|custom loggers]] can be created.


==Logger Hierarchy==
==Logger Hierarchy==
Line 8: Line 63:
If the logger name contains dot characters, they separate levels of a hierarchy. Closer to the left, the higher the logger in the logger hierarchy: <code>a</code> has a higher level than <code>a.b</code> . Each level in the hierarchy can be given different properties. At the top of the hierarchy there's a [[#The_root_logger|root logger]].
If the logger name contains dot characters, they separate levels of a hierarchy. Closer to the left, the higher the logger in the logger hierarchy: <code>a</code> has a higher level than <code>a.b</code> . Each level in the hierarchy can be given different properties. At the top of the hierarchy there's a [[#The_root_logger|root logger]].


===The root Logger===
==The root Logger==
   
   
At the top of the logger hierarchy, there  is a special root logger, called ''.
At the top of the logger hierarchy, there  is a special root logger, called ' '.
 
==Custom Loggers==
 
Custom loggers can be created with <code>logging.getLogger()</code> function. Each logger has a name, and the presence of dots in the name place the logger in the [[#Logger_Hierarchy|logger hierarchy]].
<syntaxhighlight lang='py'>
import logging
 
some_logger = logging.getLogger('some.logger')
some_other_logger = logging.getLogger('some.other.logger')
</syntaxhighlight>
 
=Handler=
 
The handlers direct logging messages to their destinations (terminal, file, database, etc.).
 
To configure the logging system to use a <code>[[#FileHandler|FileHandler]]</code>:
<syntaxhighlight lang='py'>
import logging
logging.basicConfig(filename = 'some_file.log')
</syntaxhighlight>
 
==Handlers==
====<tt>FileHandler</tt>====
 
=Formatter=
 
A formatter transforms the log message according to a certain pattern.
 
==Format==
 
The default format, without any customization is <code><LOGGING_LEVEL>:<logger_name>:<message></code>.
 
A format string can be provided to the configuration function, by specifying [[#Format_Variables|format variables]].
 
<syntaxhighlight lang='py'>
logging.basicConfig(format='%(asctime)s %(levelname)s %(lineno)s %(message)s')
</syntaxhighlight>
 
 
More examples:
<syntaxhighlight lang='py'>
format='[%(asctime)s] %(levelname)s:%(module)s:%(funcName)s: %(message)s'
logging.basicConfig(format=format, datefmt='%Y-%m-%d %H:%M:%S', level=logging.INFO)
</syntaxhighlight>
 
 
===Format Variables===
<font color=darkkhaki>Process this: LogRecord attributes: https://docs.python.org/3/library/logging.html#logrecord-attributes</font>
 
There are several standard format variables. Custom format variables can also be used.
====<tt>asctime</tt>====
Date and time in [[ISO_8601#Overview|ISO 8601]] format.
====<tt>levelname</tt>====
====<tt>lineno</tt>====
====<tt>message</tt>====
====<tt>module</tt>====
====<tt>pathname</tt>====
The full pathname of the source file where the logging call was issued.
<font size=-1>
%(pathname)s
</font>
 
=Filter=
 
=Message=
==Lazy Message Interpolation in Logging Functions==
Avoid eager string interpolation in logging function for performance reasons: the interpolation should be done only if the corresponding logging level is enabled.
 
[[Python_Language_String#F-String|F-strings]] perform eager interpolations so they should be avoided.
 
Recommended style:
<syntaxhighlight lang='py'>
logger.error('caused by %s', msg)
</syntaxhighlight>


=Format=
<code>%s</code> works for integers too.


The default format, without any customization is <code><LOGGING_LEVEL>:<logger_name>:<message></code>.
=Patterns and Idioms=
==Custom Handling of Levels==
{{External|https://docs.python.org/3/howto/logging-cookbook.html#custom-handling-of-levels}}

Latest revision as of 16:35, 26 May 2023

Internal

TODO

Overview

The standard Python library module to provide logging support is logging.

import logging

logging.basicConfig(level=logging.DEBUG)
some_logger = logging.getLogger('some-logger')

logging.debug('debugging')
some_logger.debug('debugging via some_logger')

The output is:

DEBUG:root:debugging
DEBUG:some-logger:debugging via some_logger

Logging Levels

https://docs.python.org/3/library/logging.html#logging-levels

The logging system can be configured to filter out log messages whose logging level is under a certain threshold. The logging levels, and the corresponding logging functions are, listed from the most important to the least important, are provided below. By default, the logging system is configured to allow log messages of level WARNING and above, INFO and below are discarded. A different logging threshold can be configured.

Standard Logging Levels

CRITICAL

Accessible as logging.CRITICAL. Has numeric value 50. Generated with logging.critical().

ERROR

Accessible as logging.ERROR. Has numeric value 40. Generated with logging.error().

WARNING

Accessible as logging.WARNING. Has numeric value 30. Generated with logging.warning().

WARN seems to be deprecated. warn() is deprecated.

INFO

Accessible as logging.INFO. Has numeric value 20. Generated with logging.info().

DEBUG

Accessible as logging.DEBUG. Has numeric value 10. Generated with logging.debug().

NOTSET

Accessible as logging.NOTSET. Has numeric value 0.

Setting Logging Level

import logging

logging.basicConfig(level=logging.DEBUG)

Logger

The logger is the programmatic way to generate logging messages. By default, logging.debug()|info()|... syntax implies the usage of the root logger, but custom loggers can be created.

Logger Hierarchy

If the logger name contains dot characters, they separate levels of a hierarchy. Closer to the left, the higher the logger in the logger hierarchy: a has a higher level than a.b . Each level in the hierarchy can be given different properties. At the top of the hierarchy there's a root logger.

The root Logger

At the top of the logger hierarchy, there is a special root logger, called ' '.

Custom Loggers

Custom loggers can be created with logging.getLogger() function. Each logger has a name, and the presence of dots in the name place the logger in the logger hierarchy.

import logging

some_logger = logging.getLogger('some.logger')
some_other_logger = logging.getLogger('some.other.logger')

Handler

The handlers direct logging messages to their destinations (terminal, file, database, etc.).

To configure the logging system to use a FileHandler:

import logging
logging.basicConfig(filename = 'some_file.log')

Handlers

FileHandler

Formatter

A formatter transforms the log message according to a certain pattern.

Format

The default format, without any customization is <LOGGING_LEVEL>:<logger_name>:<message>.

A format string can be provided to the configuration function, by specifying format variables.

logging.basicConfig(format='%(asctime)s %(levelname)s %(lineno)s %(message)s')


More examples:

format='[%(asctime)s] %(levelname)s:%(module)s:%(funcName)s: %(message)s'
logging.basicConfig(format=format, datefmt='%Y-%m-%d %H:%M:%S', level=logging.INFO)


Format Variables

Process this: LogRecord attributes: https://docs.python.org/3/library/logging.html#logrecord-attributes

There are several standard format variables. Custom format variables can also be used.

asctime

Date and time in ISO 8601 format.

levelname

lineno

message

module

pathname

The full pathname of the source file where the logging call was issued.

%(pathname)s

Filter

Message

Lazy Message Interpolation in Logging Functions

Avoid eager string interpolation in logging function for performance reasons: the interpolation should be done only if the corresponding logging level is enabled.

F-strings perform eager interpolations so they should be avoided.

Recommended style:

logger.error('caused by %s', msg)

%s works for integers too.

Patterns and Idioms

Custom Handling of Levels

https://docs.python.org/3/howto/logging-cookbook.html#custom-handling-of-levels