Events User Manual
Internal
Overview
"events" is Java library and a set of command line tools that process timed event series.
Its most basic usage is to convert stdout data into a timed event series sent to stdout:
events < input-file [global-options] <input-format-specification>
This syntax exercise the default command, "parse". For more details, see parse Command.
Additional commands can be invoked with the syntax:
events < input-file [global-options] command [command-arguments]
Commands
Everything below this line must be reviewed
Concepts
Faults
A fault is a HTTP request that exhibit some sort of problem, which is not serious enough to indicated corrupted data and stop processing. The faults are shown in statistics.
Common fault types:
- NO_JSESSIONID_COOKIE: the request does not have a JSESSIONID cookie, which means it was sent before authentication and session establishment, or after the session was closed. These requests are reported in summary statistics and usually ignored.
- NO_REQUEST_DURATION_INFO: No request duration information was found in the input data stream. This fault indicates a potentially serious condition, as it won't be possible to calculate request and business scenario durations. A common cause is omitting to set in the WildFly configuration.
- NO_OPEN_BUSINESS_SCENARIO: these faults are generated by the request arriving "between" scenarios. This is a normal condition and they should be usually ignored.
More details:
Business Scenarios
events extracts business scenario statistics and can identify business scenarios in one of these states:
- COMPLETE: the scenario has been closed normally by the occurrence of a STOP marker. If the load was sent sequentially, it is guaranteed that all component HTTP requests have been applied to the scenario.
- INCOMPLETE: we ran out of event data while the scenario was in an OPEN state, usually because we reached the end of the log file, without encountering a STOP marker. This happens when the log generator stops abruptly in the middle of an iteration.
- CLOSED_BY_START_MARKER: we did not encounter the STOP marker, but we did encounter a START marker of a new scenario for the same session.
More details:
null Events
Blocking queues do not accept null elements, so we cannot have null events.
Installation
Download the latest binary release from:
Unzip the binary release file into the directory where third party tools are usually installed on your system (/usr/local, /opt, etc.). Extraction will create a events-<version> directory.
Create an "event" symbolic link to the newly created directory. Assuming the extraction directory was /opt:
cd /opt ln -s events-<version> events
where "<version>" should be replaced with the actual value.
If the link exists and points to an older value, first remove the link and then recreated it.
Add /opt/events/bin to your PATH:
export PATH=/opt/events/bin:${PATH}
Test installation by running:
events version
Business Scenario Data Processing
This section describes the steps required to extract business scenario data from raw httpd access logs. events needs the access log and the httpd log format specification that was used when the log was generated. The format is usually collected automatically and placed in an access_log.def, which is specified to events with the --input-format-file= option. However, the format specification can also be provided directly on command line preceded by --input-format= option.
Assuming that the access log file is named access_log.log and the log specification is stored in access_log.def in the same directory, a short sanity check can be performed by running events without any command:
events < access_log.log --input-format-file=./access_log.def
The output consists of parsed HTTP requests data, and any problem with the log data or the format specification will immediately become obvious, either as an error that stops processing or as faults.
If the HTTP request parsing does not cause errors, extract high level scenario statistics with:
events < access_log.log --input-format-file=./access_log.def business-scenario --stats
The result should be similar to:
Counters
business scenarios: 7743 (3 different states)
NORMAL: 7371, duration min/avg/max: 38/13397/33304 ms, reqs/scenario min/avg/max: 30/36.73/37
INCOMPLETE: 342, duration min/avg/max: 1270/11662/20310 ms, reqs/scenario min/avg/max: 4/20.82/37
CLOSED_BY_START_MARKER: 30, duration min/avg/max: 5343/18836/32691 ms, reqs/scenario min/avg/max: 48/51.03/52
faults: 134355 (3 different types)
NO_JSESSIONID_COOKIE: 1017
NO_ACTIVE_BUSINESS_SCENARIO: 132502
MISSING_ITERATION_ID: 836
other events: 1
HTTP requests: 412922
HTTP sessions: 434
The NO_JSESSIONID_COOKIE and NO_OPEN_BUSINESS_SCENARIO faults are normal, they indicate unauthenticated requests and requests that fall outside marked scenarios.
Per-scenarios statistics can be extracted with:
events < access_log.log --input-format-file=./access_log.def business-scenario --ignore-faults > business-scenarios.csv
Scatter plot the results.
Business Scenario Measurement Procedure
Record the Load Scenarios
OBSOLETE
Record interaction with a load generator. This example was based on NeoLoad.
Mark the start of the target business scenarios.
Design -> Virtual User Profiles -> Style Reader -> Actions -> Style - Seasons - Style - Style Sample - Style Gallery -> First Page -> First request in the page -> Advanced -> Request headers:
Add a new header: "Business-Scenario-Start-Marker" (the exact name is available in BusinessScenario.java). The value of the header should be business scenario type.
Mark the end of the target business scenarios.
Design -> Virtual User Profiles -> Style Reader -> Actions -> Style - Seasons - Style - Style Sample - Style Gallery -> RequestHandler_16 -> First request in page -> Advanced -> Request headers:
Add a new header: "Business-Scenario-Start-Marker" (the exact name is available in BusinessScenario.java). Do Not use an empty value - NeoLoad does not send that or the undertow logging does not log it. Use the same string used to mark the scenario start (the business scenario type).
Save.
Prepare the Target Environment
Configure the Access Log Output
OBSOLETE
This is necessary to be done once.
Stop the server.
cd /c/ovidiu/bin ./environment.sh stop
Update the application server configuration file <access-log> section (if you copy and paste, open this in edit mode to correctly copy quots):
...
<subsystem xmlns="urn:jboss:domain:undertow:1.1">
...
<server name="default-server" >
...
<host name="default-host" alias="localhost">
...
<access-log pattern=""%I" %h %u [%t] "%r" "%q" %s %b %D %{i,Business-Scenario-Start-Marker} %{i,Business-Scenario-Stop-Marker} %{c,JSESSIONID}"/>
...
</host>
</server>
</subsystem>
...
Start the Environment
OBSOLETE
On the target environment, reset the application server node:
cd /c/ovidiu/bin ./environment.sh start
The start script will also clean the previous logs.
Run the Test
Shut Down and Collect Data
OBSOLETE
cd /c/ovidiu/bin ./environment.sh stop
All log data is collected and zipped under /c/ovidiu/data/load-data-yy-mm-dd-hh-mm-ss...zip
Download the test data locally via the QLB-APP02-Redirected-Folder on NOMBP2.
Unzip it in its own separated directory.
The data export script keeps the access log as “access_log.log” and places the definition of the access log fields in access_log.def.