Gld Configuration: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
No edit summary
 
(77 intermediate revisions by the same user not shown)
Line 3: Line 3:
* [[gld User Manual#Subjects|gld User Manual]]
* [[gld User Manual#Subjects|gld User Manual]]


=Configuration=
=Overview=
 
gld reads its configuration from a YAML file. The file name is specified on command line, as follows:


<pre>
<pre>
-c <cofiguration-file.yml>
gld -c <cofiguration-file.yml>
</pre>
</pre>


If the configuration file is not specified on the command line (where it has priority), it is looked up a value of GLD_CONF_FILE environment variable.
If the configuration file is not specified on command line, gld attempts to look up the value of the <tt>GLD_CONF_FILE</tt> environment variable and use the value specified there. If both the command line value and the GLD_CONF_FILE environment variable are present, the command line value will take precedence.


=Load Configuration=
The configuration file contains several main sections:


<pre>
<pre>
service:
service:
   ...
 
load:  
  type: cache|jms|http
   threads: 10
 
   requests|operations|messages: 1000 # optional, if not specified load will be generated forever
  #
   # this is where the target service is configured. The essential piece of configuration
  # provided in this area is the URL of the service, and the location of the client
  # libraries. The format depends on the type of the service.
  #
 
load:
 
  #
   # specifies the load characteristics, such as number of concurrent threads,
   # number of operations, etc.
  #
 
store:  
 
  #
  # defines the local or remote store for the load that is being generated
  #
 
output:
 
  #
  # defines how the results are reported
  #
</pre>
</pre>


=Cache Service Configuration=
=Service Configuration=
 
The service configuration is the section of the configuration file used to specify details of the target service. Essential pieces of configuration that must be present in this section are the type of the service ("cache", "jms", "http", etc.), the service URL and the way gld can locate and load the service's client libraries.
 
The "type" key is mandatory, it specifies the type of the target service. Available service types are "cache", "jms" and "http".
 
The type section is followed by the "implementation"and "load-strategy" sections:


<pre>
<pre>
service:
service:


   type: cache
   type: cache|jms|http
   implementation: local
 
   implementation:  


  key-size: 10  # optional, if not specified, the default value is 12 characters
    #
  value-size: 1024  # optional, if not specified, the default value is 512 characters
    # this is where the extension name, target service URL, a way to
    # locate client libraries  and other target-specific configuration
    # details go
    #


   load-strategy:
   load-strategy:
     ...
 
     #
    # implementation-independent load strategy configuration
    #
</pre>
</pre>


====implementation====
==Implementation Section==
 
The configuration details present in the "implementation" section depend on the specific [[Gld_Concepts#Extension|gld extension]] that is used to provide the client-side functionality. This section contains the extension name, the target service URL, a way to locate client libraries and other target-specific configuration. This is an example how a JBoss Datagrid 7 service is configured:
 
<pre>
service:
 
  type: cache
 
  ...


Specifies either a conventional service implementation name, such as "local", <font color=red>or ... ?</font>, or a fully qualified class name that can be looked up on the class path and instantiated.
  implementation:
    name: jboss-datagrid-7
    version: 7.0.0
    cache: novaordis
    nodes:
      - localhost:11222
      - localhost:11422
 
  ...


==Generic Load Strategy Configuration==
</pre>
 
The implementation "name" specifies the name of the [[Gld_Concepts#Extension|gld extension]] that will be used to send load into the target service. For more details about extension names, see "[[Gld_Concepts#Extension_Name|Extension Name]]" section. Each type of service comes with a built-in "embedded" extension, that can be used for testing.
 
Optionally, the implementation can be specified by its fully qualified class name, instead of the extension name. For that, use the "class" key:


<pre>
<pre>
load-strategy:
service:
   class: <fully-qualified-class-name>
   implementation:
    class: some.fully.qualified.class.Name
</pre>
</pre>


==read-then-write-on-miss Load Strategy Configuration==
Note that an implementation may be specified either by name or class, but not both. gld will complain and fail if both the name and the class are found. The class, and its dependents, must be available on the [[Gld_Concepts#Extension_Classpath|extension class path]].
 
For an extension to work, the required classes must be available on the classpath. By default, it is sufficient to specify the version of the target service. gld will look into its extension directory and will add the corresponding JARs to the classpath. If the client library for that specific version of the target service were not installed locally, gld it will complain and fail.


<pre>
<pre>
service:
service:
  type: cache
   ...
   ...


   load-strategy:
   implementation:
     name: read-then-write-on-miss
     name: jboss-datagrid-7
     reuse-value: false
     version: 7.0.0
      
     ...
</pre>
</pre>


====reuse-value====
Alternatively, the class path can be explicitly specified in the configuration file, with the "classpath" key, instead of "version".


By default, the load strategy randomly generates the first entry value and then keeps reusing it, as a speed optimization (reuse-value: "true"). To change this behavior and configure the load strategy to generate a new random value every times it needs one, set "reuse-value" to "false". Configuring the load strategy to not reuse values will make it somewhat slower.
For more details on target service client libraries, [[Gld_Concepts#Target_Service_Client_Libraries|Target Service Client Libraries]].


==write-then-read Load Strategy Configuration==
<span id="extension_configuration"></span>Details on how to configure specific extensions are available here:
 
{{Internal|Jboss-datagrid_Gld_Extension#Configuration_Elements|jboss-datagrid-6, jboss-datagrid-7}}
{{Internal|JBoss EAP JMS gld Extension#Configuration|jboss-eap-7-jms, jboss-eap-6-jms}}
 
==Load Strategy Section==
 
The load strategy section contains implementation-independent load strategy configuration. For example, this is where we describe the sequence of read and write operation for a cache service, or whether we're sending or receiving messages to a JMS service. An example of configuring a typical load strategy for a cache service follows:


<pre>
<pre>
Line 74: Line 142:


   load-strategy:
   load-strategy:
     name: write-then-read
     name: read-then-write-on-miss
     reuse-value: false
     reuse-value: true
    read-to-write|write-to-read: <integer>
  ...
</pre>
</pre>


====reuse-value====
More details are available below:
 
{{Internal|gld Cache Load Strategy Configuration|Cache Load Strategy Configuration}}
{{Internal|gld JMS Load Strategy Configuration|JMS Load Strategy Configuration}}
{{Internal|gld HTTP Load Strategy Configuration|HTTP Load Strategy Configuration}}


See [[#reuse-value|reuse-value]] above.
=Load Configuration=


====read-to-write====
The "load" section specifies the size of the load. It can be used to configure the key and value size, for cache services, the message payload size for JMS services, the total number of cache operations or messages to be send into the server, the number of concurrent threads to send the load on. The load section must always follow the "service" section because its content will be interpreted differently depending on the type of the service.


====write-to-read====
<pre>
service:
  ...
load:
  threads: 10
  requests|operations|messages: 1000 # optional, if not specified load will be generated forever
  key-size: 1024
  value-size|message-size: 2048
</pre>


=Store Configuration=
=Store Configuration=
Line 96: Line 176:
</pre>
</pre>


=Output Configuration=


=Extensions=
Specifies where various types of outputs go. Can be used to configure logging, stdout, stderr and most importantly, the collected statistics.
 
==Client Libraries Configuration==
 
===Client Libraries Available in the Operational Environment===


<font color=red>TODO: replace with a real example</font>
==Statistics Output Configuration==


<pre>
<pre>
service:
output:
  implementation:
    statistics:
    name: jboss-datagrid-7
        sampling-interval: 1000
    classpath: {JDG_HOME}/client/hotrod.jar
        sampling-task-run-interval: 250
...
        format: csv
        file: test.csv
        metrics:
          - TotalPhysicalMemorySize
          - FreePhysicalMemorySize
          - SystemCpuLoad
          - SystemLoadAverage
</pre>
</pre>


===Embedded Client Libraries===
Configuration parameters:


<font color=red>TODO: replace with a real example</font>
* <span id="format"></span>'''format''': The output format. By default, if not specified is "csv" (comma-separated value).
 
* <span id="file"></span>'''file''': The name of the file to write the output to. If relative, the path will be considered relative to the location of the configuration file. This is a required value, and there is no default.
<pre>
* <span id="sampling-interval"></span>'''sampling-interval''': The interval (in milliseconds) between successive samples. The default value is 1,000 ms.
service:
* <span id="sampling-interval"></span>'''sampling-task-run-interval''': The interval (in milliseconds) between successive runs of the sampling task. The default value is 250 ms.
  implementation:
* <span id="metrics"></span>'''metrics''': The list of metrics to be read ''in addition'' to those configured dynamically. May be not declared, or empty.
    name: jboss-datagrid-7
    version: 7.0.0
...
</pre>

Latest revision as of 00:13, 6 September 2017

Internal

Overview

gld reads its configuration from a YAML file. The file name is specified on command line, as follows:

gld -c <cofiguration-file.yml>

If the configuration file is not specified on command line, gld attempts to look up the value of the GLD_CONF_FILE environment variable and use the value specified there. If both the command line value and the GLD_CONF_FILE environment variable are present, the command line value will take precedence.

The configuration file contains several main sections:

service:

  type: cache|jms|http

  #
  # this is where the target service is configured. The essential piece of configuration 
  # provided in this area is the URL of the service, and the location of the client 
  # libraries. The format depends on the type of the service.
  #

load:

  #
  # specifies the load characteristics, such as number of concurrent threads, 
  # number of operations, etc.
  #

store: 

  #
  # defines the local or remote store for the load that is being generated
  #

output:
   
  #
  # defines how the results are reported
  #

Service Configuration

The service configuration is the section of the configuration file used to specify details of the target service. Essential pieces of configuration that must be present in this section are the type of the service ("cache", "jms", "http", etc.), the service URL and the way gld can locate and load the service's client libraries.

The "type" key is mandatory, it specifies the type of the target service. Available service types are "cache", "jms" and "http".

The type section is followed by the "implementation"and "load-strategy" sections:

service:

  type: cache|jms|http

  implementation: 

    #
    # this is where the extension name, target service URL, a way to 
    # locate client libraries  and other target-specific configuration 
    # details go
    #

  load-strategy:

    #
    # implementation-independent load strategy configuration
    #

Implementation Section

The configuration details present in the "implementation" section depend on the specific gld extension that is used to provide the client-side functionality. This section contains the extension name, the target service URL, a way to locate client libraries and other target-specific configuration. This is an example how a JBoss Datagrid 7 service is configured:

service:

  type: cache

  ...

  implementation:
    name: jboss-datagrid-7
    version: 7.0.0
    cache: novaordis
    nodes:
      - localhost:11222
      - localhost:11422
  
  ...

The implementation "name" specifies the name of the gld extension that will be used to send load into the target service. For more details about extension names, see "Extension Name" section. Each type of service comes with a built-in "embedded" extension, that can be used for testing.

Optionally, the implementation can be specified by its fully qualified class name, instead of the extension name. For that, use the "class" key:

service:
  implementation:
    class: some.fully.qualified.class.Name

Note that an implementation may be specified either by name or class, but not both. gld will complain and fail if both the name and the class are found. The class, and its dependents, must be available on the extension class path.

For an extension to work, the required classes must be available on the classpath. By default, it is sufficient to specify the version of the target service. gld will look into its extension directory and will add the corresponding JARs to the classpath. If the client library for that specific version of the target service were not installed locally, gld it will complain and fail.

service:
  ...

  implementation:
    name: jboss-datagrid-7
    version: 7.0.0
 
    ...

Alternatively, the class path can be explicitly specified in the configuration file, with the "classpath" key, instead of "version".

For more details on target service client libraries, Target Service Client Libraries.

Details on how to configure specific extensions are available here:

jboss-datagrid-6, jboss-datagrid-7
jboss-eap-7-jms, jboss-eap-6-jms

Load Strategy Section

The load strategy section contains implementation-independent load strategy configuration. For example, this is where we describe the sequence of read and write operation for a cache service, or whether we're sending or receiving messages to a JMS service. An example of configuring a typical load strategy for a cache service follows:

service:

  type: cache
  ...

  load-strategy:
    name: read-then-write-on-miss
    reuse-value: true
  ...

More details are available below:

Cache Load Strategy Configuration
JMS Load Strategy Configuration
HTTP Load Strategy Configuration

Load Configuration

The "load" section specifies the size of the load. It can be used to configure the key and value size, for cache services, the message payload size for JMS services, the total number of cache operations or messages to be send into the server, the number of concurrent threads to send the load on. The load section must always follow the "service" section because its content will be interpreted differently depending on the type of the service.

service:
  ...
load: 
  threads: 10
  requests|operations|messages: 1000 # optional, if not specified load will be generated forever
  key-size: 1024
  value-size|message-size: 2048

Store Configuration

store:

  type: hierarchical
  directory: ../store # if relative, is relative to the position of the configuration file

Output Configuration

Specifies where various types of outputs go. Can be used to configure logging, stdout, stderr and most importantly, the collected statistics.

Statistics Output Configuration

output:
    statistics:
        sampling-interval: 1000
        sampling-task-run-interval: 250
        format: csv
        file: test.csv
        metrics: 
          - TotalPhysicalMemorySize
          - FreePhysicalMemorySize
          - SystemCpuLoad
          - SystemLoadAverage

Configuration parameters:

  • format: The output format. By default, if not specified is "csv" (comma-separated value).
  • file: The name of the file to write the output to. If relative, the path will be considered relative to the location of the configuration file. This is a required value, and there is no default.
  • sampling-interval: The interval (in milliseconds) between successive samples. The default value is 1,000 ms.
  • sampling-task-run-interval: The interval (in milliseconds) between successive runs of the sampling task. The default value is 250 ms.
  • metrics: The list of metrics to be read in addition to those configured dynamically. May be not declared, or empty.