Gld Configuration: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
No edit summary
 
(44 intermediate revisions by the same user not shown)
Line 3: Line 3:
* [[gld User Manual#Subjects|gld User Manual]]
* [[gld User Manual#Subjects|gld User Manual]]


=Configuration=
=Overview=
 
gld reads its configuration from a YAML file. The file name is specified on command line, as follows:


<pre>
<pre>
-c <cofiguration-file.yml>
gld -c <cofiguration-file.yml>
</pre>
</pre>


If the configuration file is not specified on the command line (where it has priority), it is looked up a value of GLD_CONF_FILE environment variable.
If the configuration file is not specified on command line, gld attempts to look up the value of the <tt>GLD_CONF_FILE</tt> environment variable and use the value specified there. If both the command line value and the GLD_CONF_FILE environment variable are present, the command line value will take precedence.


=Service Configuration=
The configuration file contains several main sections:
 
<pre>
service:


"type" is mandatory, it specifies the type of service. Possible values:
  type: cache|jms|http


* "cache" (see [[#Cache_Service_Configuration|Cache Service Configuration]] below for specifics)
  #
* "jms"
  # this is where the target service is configured. The essential piece of configuration
* "http"
  # provided in this area is the URL of the service, and the location of the client
  # libraries. The format depends on the type of the service.
  #


==Implementation==
load:


The implementation section defines the [[Gld_Concepts#Extension|gld extension]] to use. The extension contains implementation-specific details essential to connect to, and interact with the target service.
  #
  # specifies the load characteristics, such as number of concurrent threads,
  # number of operations, etc.
  #


===Implementation Name===
store:


The implementation "name" specifies the name of the [[Gld_Concepts#Extension|gld extension]] that will be instantiated. For more details about extension names, see "[[Gld_Concepts#Extension_Name|Extension Name]]" section.
  #
  # defines the local or remote store for the load that is being generated
  #


<pre>
output:
service:
 
   implementation:
  #
    name: <extension-name>
   # defines how the results are reported
  #
</pre>
</pre>


=Service Configuration=


For testing purposes, each service type ships with an "embedded" implementation.
The service configuration is the section of the configuration file used to specify details of the target service. Essential pieces of configuration that must be present in this section are the type of the service ("cache", "jms", "http", etc.), the service URL and the way gld can locate and load the service's client libraries.


===Implementation Class===
The "type" key is mandatory, it specifies the type of the target service. Available service types are "cache", "jms" and "http".


Optionally, the implementation may be specified by its fully qualified class name:
The type section is followed by the "implementation"and "load-strategy" sections:


<pre>
<pre>
service:
service:
  implementation:
    class: some.fully.qualified.class.Name
</pre>


Note that an implementation may be specified either by name or class, but not both. The configuration parsing will fail both the name and the class are found. The class, and its dependents, must be available on the [[Gld_Concepts#Extension_Classpath|extension class path]].
  type: cache|jms|http


===Client Libraries Configuration===
  implementation:


====Client Libraries Available in the Operational Environment====
    #
    # this is where the extension name, target service URL, a way to
    # locate client libraries  and other target-specific configuration
    # details go
    #


<font color=red>TODO: replace with a real example</font>
  load-strategy:


<pre>
    #
service:
    # implementation-independent load strategy configuration
  implementation:
     #
    name: jboss-datagrid-7
     classpath: ${JDG_HOME}/client/hotrod.jar
...
</pre>
</pre>


Also see:
==Implementation Section==


<blockquote style="background-color: #f9f9f9; border: solid thin lightgrey;">
The configuration details present in the "implementation" section depend on the specific [[Gld_Concepts#Extension|gld extension]] that is used to provide the client-side functionality. This section contains the extension name, the target service URL, a way to locate client libraries and other target-specific configuration. This is an example how a JBoss Datagrid 7 service is configured:
:[[Gld_Concepts#Target_Service_Client_Libraries|Target Service Client Libraries]]
</blockquote>


====Client Libraries Shipped with the gld Distribution====
<pre>
service:


<font color=red>TODO: replace with a real example</font>
  type: cache
 
  ...


<pre>
service:
   implementation:
   implementation:
     name: jboss-datagrid-7
     name: jboss-datagrid-7
     version: 7.0.0
     version: 7.0.0
...
    cache: novaordis
    nodes:
      - localhost:11222
      - localhost:11422
 
  ...
 
</pre>
</pre>


Also see:
The implementation "name" specifies the name of the [[Gld_Concepts#Extension|gld extension]] that will be used to send load into the target service. For more details about extension names, see "[[Gld_Concepts#Extension_Name|Extension Name]]" section. Each type of service comes with a built-in "embedded" extension, that can be used for testing.
 
<blockquote style="background-color: #f9f9f9; border: solid thin lightgrey;">
:[[Gld_Concepts#Target_Service_Client_Libraries|Target Service Client Libraries]]
</blockquote>
 
==Load Strategy Configuration==
 
<pre>
load-strategy:
  name: <load-strategy-name>
 
  # alternatively, the implementation class can be specified
  class: <fully-qualified-class-name>
</pre>


===read-then-write-on-miss Load Strategy Configuration===
Optionally, the implementation can be specified by its fully qualified class name, instead of the extension name. For that, use the "class" key:


<pre>
<pre>
service:
service:
 
   implementation:
   type: cache
    class: some.fully.qualified.class.Name
  ...
 
  load-strategy:
    name: read-then-write-on-miss
    reuse-value: false
   
</pre>
</pre>


====reuse-value====
Note that an implementation may be specified either by name or class, but not both. gld will complain and fail if both the name and the class are found. The class, and its dependents, must be available on the [[Gld_Concepts#Extension_Classpath|extension class path]].


By default, the load strategy randomly generates the first entry value and then keeps reusing it, as a speed optimization (reuse-value: "true"). To change this behavior and configure the load strategy to generate a new random value every times it needs one, set "reuse-value" to "false". Configuring the load strategy to not reuse values will make it somewhat slower.
For an extension to work, the required classes must be available on the classpath. By default, it is sufficient to specify the version of the target service. gld will look into its extension directory and will add the corresponding JARs to the classpath. If the client library for that specific version of the target service were not installed locally, gld it will complain and fail.
 
===write-then-read Load Strategy Configuration===


<pre>
<pre>
service:
service:
  type: cache
   ...
   ...


   load-strategy:
   implementation:
     name: write-then-read
     name: jboss-datagrid-7
     reuse-value: false
     version: 7.0.0
     read-to-write|write-to-read: <integer>
     ...
</pre>
</pre>


====reuse-value====
Alternatively, the class path can be explicitly specified in the configuration file, with the "classpath" key, instead of "version".
 
For more details on target service client libraries, [[Gld_Concepts#Target_Service_Client_Libraries|Target Service Client Libraries]].


See [[#reuse-value|reuse-value]] above.
<span id="extension_configuration"></span>Details on how to configure specific extensions are available here:


====read-to-write====
{{Internal|Jboss-datagrid_Gld_Extension#Configuration_Elements|jboss-datagrid-6, jboss-datagrid-7}}
{{Internal|JBoss EAP JMS gld Extension#Configuration|jboss-eap-7-jms, jboss-eap-6-jms}}


====write-to-read====
==Load Strategy Section==


==Cache Service Configuration==
The load strategy section contains implementation-independent load strategy configuration. For example, this is where we describe the sequence of read and write operation for a cache service, or whether we're sending or receiving messages to a JMS service. An example of configuring a typical load strategy for a cache service follows:


<pre>
<pre>
Line 142: Line 139:


   type: cache
   type: cache
   key-size: 10  # optional, if not specified, the default value is 12 characters
   ...
  value-size: 1024  # optional, if not specified, the default value is 512 characters
 
  implementation:
      ...


   load-strategy:
   load-strategy:
     ...
     name: read-then-write-on-miss
    reuse-value: true
  ...
</pre>
</pre>


For details on how to configure a specific implementation, see [[#Implementation|Implementation]] below.
More details are available below:


Known cache extension implementations:
{{Internal|gld Cache Load Strategy Configuration|Cache Load Strategy Configuration}}
{{Internal|gld JMS Load Strategy Configuration|JMS Load Strategy Configuration}}
{{Internal|gld HTTP Load Strategy Configuration|HTTP Load Strategy Configuration}}


===jboss-datagrid-7===
=Load Configuration=
 
{{Internal|jboss-datagrid-7 Gld Extension|jboss-datagrid-7}}


=Load Configuration=
The "load" section specifies the size of the load. It can be used to configure the key and value size, for cache services, the message payload size for JMS services, the total number of cache operations or messages to be send into the server, the number of concurrent threads to send the load on. The load section must always follow the "service" section because its content will be interpreted differently depending on the type of the service.


<pre>
<pre>
Line 168: Line 163:
   threads: 10
   threads: 10
   requests|operations|messages: 1000 # optional, if not specified load will be generated forever
   requests|operations|messages: 1000 # optional, if not specified load will be generated forever
  key-size: 1024
  value-size|message-size: 2048
</pre>
</pre>


Line 192: Line 189:
         format: csv
         format: csv
         file: test.csv
         file: test.csv
         metrics: TotalPhysicalMemorySize, FreePhysicalMemorySize, SystemCpuLoad, SystemLoadAverage
         metrics:  
          - TotalPhysicalMemorySize
          - FreePhysicalMemorySize
          - SystemCpuLoad
          - SystemLoadAverage
</pre>
</pre>


====format====
Configuration parameters:
 
The output format. By default, if not specified is "csv" (comma-separated value).
 
====file====
 
The name of the file to write the output to. If relative, the path will be considered relative to the location of the configuration file. This is a required value, and there is no default.
 
====sampling-interval====
 
The interval (in milliseconds) between successive samples. The default value is 1,000 ms.
 
====sampling-task-run-interval====
 
The interval (in milliseconds) between successive runs of the sampling task. The default value is 250 ms.
 
====metrics====


The list of metrics to be read ''in addition'' to those configured dynamically.
* <span id="format"></span>'''format''': The output format. By default, if not specified is "csv" (comma-separated value).
* <span id="file"></span>'''file''': The name of the file to write the output to. If relative, the path will be considered relative to the location of the configuration file. This is a required value, and there is no default.
* <span id="sampling-interval"></span>'''sampling-interval''': The interval (in milliseconds) between successive samples. The default value is 1,000 ms.
* <span id="sampling-interval"></span>'''sampling-task-run-interval''': The interval (in milliseconds) between successive runs of the sampling task. The default value is 250 ms.
* <span id="metrics"></span>'''metrics''': The list of metrics to be read ''in addition'' to those configured dynamically. May be not declared, or empty.

Latest revision as of 00:13, 6 September 2017

Internal

Overview

gld reads its configuration from a YAML file. The file name is specified on command line, as follows:

gld -c <cofiguration-file.yml>

If the configuration file is not specified on command line, gld attempts to look up the value of the GLD_CONF_FILE environment variable and use the value specified there. If both the command line value and the GLD_CONF_FILE environment variable are present, the command line value will take precedence.

The configuration file contains several main sections:

service:

  type: cache|jms|http

  #
  # this is where the target service is configured. The essential piece of configuration 
  # provided in this area is the URL of the service, and the location of the client 
  # libraries. The format depends on the type of the service.
  #

load:

  #
  # specifies the load characteristics, such as number of concurrent threads, 
  # number of operations, etc.
  #

store: 

  #
  # defines the local or remote store for the load that is being generated
  #

output:
   
  #
  # defines how the results are reported
  #

Service Configuration

The service configuration is the section of the configuration file used to specify details of the target service. Essential pieces of configuration that must be present in this section are the type of the service ("cache", "jms", "http", etc.), the service URL and the way gld can locate and load the service's client libraries.

The "type" key is mandatory, it specifies the type of the target service. Available service types are "cache", "jms" and "http".

The type section is followed by the "implementation"and "load-strategy" sections:

service:

  type: cache|jms|http

  implementation: 

    #
    # this is where the extension name, target service URL, a way to 
    # locate client libraries  and other target-specific configuration 
    # details go
    #

  load-strategy:

    #
    # implementation-independent load strategy configuration
    #

Implementation Section

The configuration details present in the "implementation" section depend on the specific gld extension that is used to provide the client-side functionality. This section contains the extension name, the target service URL, a way to locate client libraries and other target-specific configuration. This is an example how a JBoss Datagrid 7 service is configured:

service:

  type: cache

  ...

  implementation:
    name: jboss-datagrid-7
    version: 7.0.0
    cache: novaordis
    nodes:
      - localhost:11222
      - localhost:11422
  
  ...

The implementation "name" specifies the name of the gld extension that will be used to send load into the target service. For more details about extension names, see "Extension Name" section. Each type of service comes with a built-in "embedded" extension, that can be used for testing.

Optionally, the implementation can be specified by its fully qualified class name, instead of the extension name. For that, use the "class" key:

service:
  implementation:
    class: some.fully.qualified.class.Name

Note that an implementation may be specified either by name or class, but not both. gld will complain and fail if both the name and the class are found. The class, and its dependents, must be available on the extension class path.

For an extension to work, the required classes must be available on the classpath. By default, it is sufficient to specify the version of the target service. gld will look into its extension directory and will add the corresponding JARs to the classpath. If the client library for that specific version of the target service were not installed locally, gld it will complain and fail.

service:
  ...

  implementation:
    name: jboss-datagrid-7
    version: 7.0.0
 
    ...

Alternatively, the class path can be explicitly specified in the configuration file, with the "classpath" key, instead of "version".

For more details on target service client libraries, Target Service Client Libraries.

Details on how to configure specific extensions are available here:

jboss-datagrid-6, jboss-datagrid-7
jboss-eap-7-jms, jboss-eap-6-jms

Load Strategy Section

The load strategy section contains implementation-independent load strategy configuration. For example, this is where we describe the sequence of read and write operation for a cache service, or whether we're sending or receiving messages to a JMS service. An example of configuring a typical load strategy for a cache service follows:

service:

  type: cache
  ...

  load-strategy:
    name: read-then-write-on-miss
    reuse-value: true
  ...

More details are available below:

Cache Load Strategy Configuration
JMS Load Strategy Configuration
HTTP Load Strategy Configuration

Load Configuration

The "load" section specifies the size of the load. It can be used to configure the key and value size, for cache services, the message payload size for JMS services, the total number of cache operations or messages to be send into the server, the number of concurrent threads to send the load on. The load section must always follow the "service" section because its content will be interpreted differently depending on the type of the service.

service:
  ...
load: 
  threads: 10
  requests|operations|messages: 1000 # optional, if not specified load will be generated forever
  key-size: 1024
  value-size|message-size: 2048

Store Configuration

store:

  type: hierarchical
  directory: ../store # if relative, is relative to the position of the configuration file

Output Configuration

Specifies where various types of outputs go. Can be used to configure logging, stdout, stderr and most importantly, the collected statistics.

Statistics Output Configuration

output:
    statistics:
        sampling-interval: 1000
        sampling-task-run-interval: 250
        format: csv
        file: test.csv
        metrics: 
          - TotalPhysicalMemorySize
          - FreePhysicalMemorySize
          - SystemCpuLoad
          - SystemLoadAverage

Configuration parameters:

  • format: The output format. By default, if not specified is "csv" (comma-separated value).
  • file: The name of the file to write the output to. If relative, the path will be considered relative to the location of the configuration file. This is a required value, and there is no default.
  • sampling-interval: The interval (in milliseconds) between successive samples. The default value is 1,000 ms.
  • sampling-task-run-interval: The interval (in milliseconds) between successive runs of the sampling task. The default value is 250 ms.
  • metrics: The list of metrics to be read in addition to those configured dynamically. May be not declared, or empty.