Events-api Concepts
Internal
Event
An event is an ordered list of properties which describe the state of a system at the time the property values were read. By default, an event is not associated with a specific moment in time. However, for most practical situations, the time when properties have been read is important, and the instance of such an event is modeled as a timed event, which emphasize the event's timestamp. The method Event.isTimed() characterizes the nature of the event. For non-timed events, the method returns false.
All properties of an event are identified by unique names: an event cannot have two properties with the same name. From this perspective, the list of properties mentioned above can be though as an ordered set.
For complex events, a property can contain in turn an event, recursively.
Property Order
The order in which the properties are set on the event instance is relevant, and preserved by the event instance. The order in which getProperties() returns the properties reflects the order in which they were set. The order of the properties is also relevant when properties are referred by their index.
Timed Event
A timed event is an event that has a timestamp. The values of all event's properties are read at the moment in time marked by the timestamp. Event.isTimed() returns true for timed events.
Timestamp Value
For all timed events, the value of the timestamp, expressed in number of milliseconds that have passed since January 1, 1970, 00:00:00 UTC is guaranteed to be available as the "time" (TimedEvent. TIME_PROPERTY_NAME) property, and as the property with index 0.
Property
A name/value pair with additional attributes, such as type, measure unit, presentation format, etc., carried by an event. The value usually represents the snapshot in time of a certain runtime value, so that is why the terms "property" and "reading" are interchangeably used. An event cannot have two properties with the same name. All properties implement the Property interface. Pre-typed properties are available for use:
In case of timed events, only one timestamp property is allowed, and it can always be retrieved by its established name ("time") or index (0). For more details on referring properties by name or index see the "Property Reference" section.
As a common use case, Properties are created based on a metric definition.
Property instances can be instantiate directly using the respective constructors, or they can be created with the static factory PropertyFactory.createInstance(...).
Does the format belong here, or at a different level?
Property Reference
Properties can be referred by their name or by their index. When properties are referred by index, the index is given by the order in which the properties were added to the event. See "Property Order" section. The index is 0-based.
For timed properties, the timestamp can be referred by name as "time" (TimedEvent.TIME_PROPERTY_NAME) or by index as property 0. The timestamp will always be returned wrapped into a TimestampProperty, which carries the timestamp value. If the timestamp was not set on a timed property, getProperty(0) will still return a non-null TimestampProperty instance, but that instance carries a null value.
Metric
Metric Source
A source of metrics. Implies an underlying mechanism that can be used to obtain a set of values given a set of metric definitions, ideally in a single operation, for performance reasons. However, the implementations are free to define what "efficient reading" means. Example of metric sources:
- Local O/S instance can be queried for metrics such as free memory, CPU usage or disk space.
- Remote O/S instance can be queried for metrics such as free memory, CPU usage or disk space over a SSH connections (assuming credentials are available and the system is reachable).
- The JMX bus of a local or remote Java virtual machine.
- The management controller (standalone or domain) of a WildFly instance.
A metric source instance must be started before metrics can be collected from it. The start operation usually implies expensive remote connection creation, initial state verification, etc. so metric source implementations should be designed to be started once and then stay in that state indefinitely. However, the underlying connection may break for various reasons during the metric source life, so it may become necessary to re-start a metric source.
Implementations are NOT expected to maintain their own internal threads. The recommended way to manage sources and collect metrics is to do it with external threads/executors.
The implementations must correctly implement equals() and hashCode(), as metric sources will be used as map keys.
Metric Source Address
OS
Local OS
Remote OS
Valid addresses:
ssh://username@hostaddress:port
"ssh://" protocol implied:
username@hostaddress:port
JBoss Management Controller
jbosscli://admin@1.2.3.4:9999/
"jbosscli://" implied:
admin@1.2.3.4:9999/
The default value, if not specified, is "localhost:9999".
JMX Bus
jmx://admin:adminpasswd@1.2.3.4:2345
Classpath.
Metric Definition
A metric definition is an instance that contains all the information a metric source needs in order to extract a value (reading) for a specific metric. The reading is instantiated as a property, carried by an event. A metric definition instance can be associated with (bound to) a metric source.
Metric Definition ID
The metric definition has an ID which uniquely identifies the metric to a metric source. The ID must not include anything that ties that ID to a specific metric source. The same ID can be given to different metric sources, and each metric source will return a different value of the metric. For example, "PhysicalMemoryTotal" uniquely identifies the total physical memory metric in the context of a local or remote operating system instance, "java.lang:type=Threading.ThreadCount" uniquely identifies the JVM thread count metric in the context of a JVM's platform MBean server and "/subsystem=messaging/hornetq-server=default/jms-queue=DLQ:message-count" uniquely identifies the DLQ queue depth metric for a specific broker node, managed by a JBoss management controller.
Metric Source Address
A specific instance of a metric definition can be associated with (bound to) a metric source via the metric source address. More details on the metric definition/metric source relationship are available in the "Relationship between a Metric Definition and a Metric Source" section.
Relationship between a Metric Definition and a Metric Source
A metric definition must be bound to a metric source to be used to retrieve readings. The binding is done when the metric definition is created, via the constructor, by specifying the metric source address as a constructor argument. A metric definition can be bound to a single metric source. A metric source can have multiple metric definitions bound to it. The binding insures the fact that the metric definition can maintain state on behalf of the metric source. One situation when maintaining state on behalf of the source is useful is when the values for that metric can only be calculated based on previous values - rates, for examples. In situations like this, successive readings must rely on the fact that the previous value (or values) are correctly maintained by the metric definition instance, so they can be relied upon. This is guaranteed once the metric definition instance is bound to a specific source.
A metric source receives a metric defintion and if the source recognizes the definitions, it returns a value (reading) associated with that definition, in form of a property instance:
List<Property> collectMetrics(List<MetricDefinition> metricDefinitions) throws MetricException;
The method collects all metrics whose definitions are present in the argument list.
If a value corresponding to a specific metric definition from the list cannot be successfully collected, a Property instance with the correct id, type and base unit must is returned, but its value is null. Implementations may log as WARN more details on why the collection failed. Implementations may choose - and they are encouraged - to attempt to start the source in-line, if the source is not already started. They, however, may chose to signal the fact that the source is not started and request an external start. The metric source will throw an exception if if finds at least a metric definition that has a null or different source. This indicates a programming error, not a runtime collection failure.
A Property instance returned for a specific metric definition has its name equals with the metric definition ID.
OS Metric Defintion
Local:
PhysicalMemoryFree
Remote:
ssh://test-remote-host:22/PhysicalMemoryFree
More details:
JBoss CLI Metric Defintion
Default (implies the default local jboss instance):
/subsystem=messaging/hornetq-server=default/jms-queue=DLQ/message-count
Remote:
jbosscli://admin:adminpasswd@1.2.3.4:9999/subsystem=messaging/hornetq-server=default/jms-queue=DLQ/message-count
JMX Metric Defintion
jmx://admin:adminpasswd@1.2.3.4:2345/jboss.as:subsystem=messaging,hornetq-server=default,jms-queue=DLQ/messageCount
Type Hierarchy
Parsers
The main parser API is https://github.com/NovaOrdis/events-api/blob/master/src/main/java/io/novaordis/events/api/parser/Parser.java:
package io.novaordis.events.api.parser;
public interface Parser {
List<Event> parse(String line, Query query) throws ParsingException;
List<Event> close() throws ParsingException;
long getLineNumber();
}
where the query is available when the event parsing is performed. This is useful for performance reasons: for example, if query contains a time limit, the parser may use this information to parse the time stamp first and drop the rest of the processing if the time stamp does not match the query, resulting in significant processing speed up for complex events.
Query Once
In order to avoid querying multiple times if the parser used the query when parsing, we use the "pattern" described here: https://github.com/NovaOrdis/events-api/blob/master/src/main/java/io/novaordis/events/api/parser/QueryOnce.java If we ever need to refactor, we only need to track QueryOnce API and refactor.
Query
A query is a combination of free format, structured text and regular expressions that is interpreted by the events runtime to filter a stream of events. An individual event must be selected by the query (match) in order to be considered for further processing.
A query may contain:
- keywords
- event property filters
- regular expressions
- modifiers (command line options that modify the semantics of the query), such as --case-insensitive, etc.
- time boundaries, encapsulated as time queries.
Modifiers
--case-insensitive
Operators
AND
Case insensitive "and", "-a"
OR
Case insensitive "not", "-o"
NOT
Case insensitive "not", "-n"
Keyword Query
A keyword query is a command-line method friendly to filter a stream of events: it consists space-separated, case insensitive words that must be present in the string representation of the event property values, to select the event for further processing or display.
A keyword query is by default case insensitive unless --case-sensitive modifier is specified.
Continue Here
The keyword query is by default case insensitive, unless ... is specified.
When multiple separate keywords are present in query, the query selects the union of events that contain each keyword.
blue red
will return all events that contain "blue" and all events that contain "red". Those events that contain both "blue" and "red" will be represented once in the returned set.
Note that:
blue red
and:
"blue red"
are different.
Keywords are matched against each property.
Event Property Query
Event property values can be queried using the following syntax:
property-name:exact-match
"Property query" and "field query" terms are currently used interchangeably.
Time Query
A time query applies to a timed event's timestamp, and specifies time boundaries for "interesting" events.
... from:'<timestamp>' to:'<timestamp>' ...
Note that the timestamps must be enclosed in single quotes if they contain spaces.
Supported timestamp formats so far are:
- 'MM/dd/yy HH:mm:ss' (example '12/01/16 17:00:00')
Field Query
A field query attempts to match the value of a certain field (property) against a regular expression. Only String properties are matched currently, anything else will not match. "Field query" and "property query" are terms can be used interchangeably.
stack:'Object.wait(.*)'
Regular Expression Metacharacters
Parentheses do not need to be quoted.