Events User Manual - Parse Command: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(5 intermediate revisions by the same user not shown)
Line 43: Line 43:
</pre>
</pre>


More details about the httpd log format support can be found in the "[[events User Manual - Apache httpd Logs#Input_Format|Apache httpd log format]]
More details about the httpd log format support can be found in the "[[events User Manual - Apache httpd Logs#Input_Format|Apache httpd log format]]" section. A CSV file format can be specified in-line as follows:
" section. A CSV file format can be specified in-line as follows:


<pre>
<pre>
-i "timestamp, count, status-code"
-i "timestamp, count, status-code"
</pre>
</pre>
More details about the CSV format support can be found in the "[[Events_User_Manual_-_CSV_Support#Input_Format#Input_Format|CSV format]]" section.


When an in-line format is used, the runtime will apply heuristics and try to figure out what type of format was specified.
When an in-line format is used, the runtime will apply heuristics and try to figure out what type of format was specified.
Line 70: Line 71:
<blockquote style="background-color: #f9f9f9; border: solid thin lightgrey;">
<blockquote style="background-color: #f9f9f9; border: solid thin lightgrey;">
:[[events User Manual - Apache httpd Logs#Input_Format|Apache httpd log format]]
:[[events User Manual - Apache httpd Logs#Input_Format|Apache httpd log format]]
:[[events User Manual - CSV Support#Input_Format|CSV format]]
:[[events User Manual - Java Garbage Collection Logs#G1|G1 Garbage Collection Log]]
</blockquote>
</blockquote>
CSV Input Format: --------------------------------------------------------------------------------
The CSV input format consists in comma separated header names, with optional type and format
information. If not type or format information are specified, the input fields are handled as
strings. The general syntax is:
  <field-name>(<type>[:format])
If a value contains commas, it must be enclosed in double quotes to be parsed correctly, otherwise
the parser will interpret comma as a field separator.
Examples: ----------------------------------------------------------------------------------------
  path - the content of the field be read verbatim and handled as string.
  timestamp(time:yy/MM/dd HH:mm:ss,SSS) - the content of the field will be parsed as time
  information according to the format specified between (time:...). The format follows Java
  SimpleDateFormat conventions. If the content cannot be converted into a time stamp, a FaultEvent
  will be generated and sent down the pipeline. For more details see:
  http://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html
  status-code(int) - the content of the field will be interpreted as integer and parsed
  accordingly. If the content cannot be converted to an integer, a FaultEvent will be generated
  and sent down the pipeline.
  body-size(long) - the content of the field will be interpreted as long and parsed accordingly.
  If the content cannot be converted to a long, a FaultEvent will be generated and sent down the
  pipeline.

Latest revision as of 23:15, 14 February 2017

Internal

Overview

The command configures an events pipeline to process text data arriving at stdout and convert it into a time series at stdout, according to the specified format.

Syntax

events < input-file <input-format-specification>

Input Format Specification

The format of the input stream can be specified either verbatim, in-line in the command line or in a file whose name is specified on the command line, or by a name.

In-Line Input Formats

To specify a verbatim format in line, use: 

-i format-specification

or 

--input-format=format-specification

If the format specification is stored in a file on an accessible filesystem, it can be specified as follows: 

--input-format-file=<file-name-that-contains-input-format>

Formats that can be specified in-line usually apply to line-based logs, such as Apache httpd logs, or CSV files. A httpd log format can be specified in-line, introduced by the -i short option -i (or with the long form equivalent --input-format="...") as follows: 

-i "%h %u [%t] \"%r\" \"%q\" %{c,JSESSIONID} %{i,Some-Request-Header} %s %b %D"

More details about the httpd log format support can be found in the "Apache httpd log format" section. A CSV file format can be specified in-line as follows: 

-i "timestamp, count, status-code"

More details about the CSV format support can be found in the "CSV format" section.

When an in-line format is used, the runtime will apply heuristics and try to figure out what type of format was specified.

Named Formats

A format specified by its logical name can be provided as follows: 

-i format-name

or 

--input-format=format-name

Supported Formats

Apache httpd log format
CSV format
G1 Garbage Collection Log
Retrieved from "https://kb.novaordis.com/index.php?title=Events_User_Manual_-_Parse_Command&oldid=16928"