Events-log4j-parser: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(One intermediate revision by the same user not shown)
Line 21: Line 21:
=Usage=
=Usage=


  lg [''[[Events-api_Concepts#Query|query]]''] [''[[#Log_Format_Specification|log-format-specification]]''] [''command''] [''command options''] <log-file1> [log-file2 ...]
  lg [''[[Events-api_Concepts#Query|query]]''] [''[[#Expected_Log_Format_Specification|expected-log-format]]''] [''command''] [''command options''] <log-file1> [log-file2 ...]


The command is optional.  
The command is optional.  
Line 41: Line 41:
For more details about commands (procedures), see: {{Internal|Events-processing|Events Processing (events-processing)}}
For more details about commands (procedures), see: {{Internal|Events-processing|Events Processing (events-processing)}}


==Log Format Specification==
==Expected Log Format Specification==


The parser may be configured to expect a specific log4j format specification, which then it will try to use when parsing the file.  
The parser may be configured to expect a specific log4j format - or what log4j names [http://logging.apache.org/log4j/2.x/manual/layouts.html#PatternLayout layout]. The specification of the format, expressed in the same way a log4j configuration would, will be used to parse the file.


The log format specification is optional. If not specified, heuristic rules are used to parse the file, but the results may be not what you expect, if the heuristics fails.
The expected log format specification is optional. If not provided, heuristic rules are used to parse the file, but the results may be not what you expect, if the heuristics fails.


If the log format specification is provided, it should be the exact string that otherwise may be used to provide specification to the log4j API, enclosed in single quotes, to avoid various bash command line expansions. If the parser realizes that the log file content does not match the provided format specification, the parser will fail.
If the expected log format specification is provided, it should be the exact [http://logging.apache.org/log4j/2.x/manual/layouts.html#PatternLayout log4j pattern layout string] that otherwise would be used in the log4j configuration file. The format specification string must be enclosed in single quotes, to avoid various bash command line expansions. If the parser realizes that the log file content does not match the provided format specification, the parser will fail.


  -f &#39;''log4j-format-string''&#39; | --format=&#39;''log4j-format-string''&#39;
  -f &#39;''log4j-format-string''&#39; | --format=&#39;''log4j-format-string''&#39;
Line 54: Line 54:


  -f '%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%E%n'
  -f '%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%E%n'
{{Warn|If the log4j format string that configured the Java runtime that produced the log is available, it should be as simple as copying an pasting the format in the command line, enclose it within single quotes and run the parser.}}


==Query Syntax==
==Query Syntax==


The query is optional, If specified, it must follow the syntax described here: {{Internal|Events-api_Concepts#Query|Queries}}
The query is optional, If specified, it must follow the syntax described here: {{Internal|Events-api_Concepts#Query|Queries}}

Latest revision as of 15:31, 28 October 2017

Internal

Overview

A library that produces timed events from log4j logs.

GitHub

https://github.com/NovaOrdis/events-log4j-parser

Installation

Installs as a command line utility lg-<version>.zip.

TODO

./doc/Events log4j Parser TODO.docx.

Usage

lg [query] [expected-log-format] [command] [command options] <log-file1> [log-file2 ...]

The command is optional.

When the command is missing, the implied command is "output", which works as described here:

output

Unless output filtering options are used, the events are displayed in their raw format, which is how they appear in the original log.

Commands:

  • help
  • version
  • output
  • describe - displays a description of the events produced as the result of the parsing.
  • time-gaps - identify time gaps in the log
  • count (-c) - display the number of events that match the query.
  • exclude (-x) - filter out the events that match the query, and only display those that do not match. This command is equivalent with the grep --invert-match (-v) option.

For more details about commands (procedures), see:

Events Processing (events-processing)

Expected Log Format Specification

The parser may be configured to expect a specific log4j format - or what log4j names layout. The specification of the format, expressed in the same way a log4j configuration would, will be used to parse the file.

The expected log format specification is optional. If not provided, heuristic rules are used to parse the file, but the results may be not what you expect, if the heuristics fails.

If the expected log format specification is provided, it should be the exact log4j pattern layout string that otherwise would be used in the log4j configuration file. The format specification string must be enclosed in single quotes, to avoid various bash command line expansions. If the parser realizes that the log file content does not match the provided format specification, the parser will fail.

-f 'log4j-format-string' | --format='log4j-format-string'

Example:

-f '%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%E%n'

If the log4j format string that configured the Java runtime that produced the log is available, it should be as simple as copying an pasting the format in the command line, enclose it within single quotes and run the parser.

Query Syntax

The query is optional, If specified, it must follow the syntax described here:

Queries