Events User Manual - Parse Command

Internal

• events User Manual

Overview

The command configures an events pipeline to process text data arriving at stdout and convert it into a time series at stdout, according to the specified format.

Syntax

events < input-file <input-format-specification>

Input Format Specification

The format of the input stream can be specified either verbatim, in-line in the command line or in a file whose name is specified on the command line, or by a name.

In-Line Input Formats

To specify a verbatim format in line, use:

-i format-specification

or

--input-format=format-specification

If the format specification is stored in a file on an accessible filesystem, it can be specified as follows:

--input-format-file=<file-name-that-contains-input-format>

Formats that can be specified in-line usually apply to line-based logs, such as Apache httpd logs, or CSV files. A httpd log format can be specified in-line, introduced by the -i short option -i (or with the long form equivalent --input-format="...") as follows:

-i "%h %u [%t] \"%r\" \"%q\" %{c,JSESSIONID} %{i,Some-Request-Header} %s %b %D"

More details about the httpd log format support can be found in the "Apache httpd log format" section. A CSV file format can be specified in-line as follows:

-i "timestamp, count, status-code"

When an in-line format is used, the runtime will apply heuristics and try to figure out what type of format was specified.

Named Formats

A format specified by its logical name can be provided as follows:

-i format-name

or

--input-format=format-name

Supported Formats

Apache httpd log format

CSV Input Format: --------------------------------------------------------------------------------

The CSV input format consists in comma separated header names, with optional type and format information. If not type or format information are specified, the input fields are handled as strings. The general syntax is:

 <field-name>(<type>[:format])

If a value contains commas, it must be enclosed in double quotes to be parsed correctly, otherwise the parser will interpret comma as a field separator.

Examples: ----------------------------------------------------------------------------------------

 path - the content of the field be read verbatim and handled as string.

 timestamp(time:yy/MM/dd HH:mm:ss,SSS) - the content of the field will be parsed as time
 information according to the format specified between (time:...). The format follows Java
 SimpleDateFormat conventions. If the content cannot be converted into a time stamp, a FaultEvent
 will be generated and sent down the pipeline. For more details see:
 http://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html

 status-code(int) - the content of the field will be interpreted as integer and parsed
 accordingly. If the content cannot be converted to an integer, a FaultEvent will be generated
 and sent down the pipeline.

 body-size(long) - the content of the field will be interpreted as long and parsed accordingly.
 If the content cannot be converted to a long, a FaultEvent will be generated and sent down the
 pipeline.