Gld Operations

Internal

• gld User Manual

Overview

DEPLETE PREVIOUS VERSION ON-LINE HELP:

Generic Load Driver ${version}, released on ${release_date}

gld (Generic Load Driver) is a program that sends operations into services (caches, JMS providers,
etc.  and collects client-side statistics.  The program can also be used for  HA testing,  content
queries and in general for any client-side activity simulation.

Usage:

    gld <command> --nodes node1:port1,node2:port2 [command-options]

    gld <command> --conf <configuration-file>

Global Options ___________________________________________________________________________________

    --verbose | -v - configure the runtime to generate verbose output.

    --service <service-type> Defines the service provider type. You can use either one of the
      pre-defined values  ("infinispan",  "activemq", etc.)  or a fully qualified  class name of a
      class available in the class path and that implements io.novaordis.gld.api.Service. The default
      load strategy for caches is "WriteThenRead" and for JMS load is "Send".   More details about
      load strategies can be found below under the "Load Strategies" section.

    --nodes <node1:port1,node2:port2,...>    The list of  host:port values for the backend service
      nodes to send requests into. There is no default, the node list must be explicitly specified.

      If "embedded" is used, gld will mock the service (a cache or a JMS provider) internally. You
      can request the embedded service to initialize itself - for example the embedded cache could
      be  pre-populated  with random content by specifying  "embedded[keyCount]".   The "embedded"
      services are intended for gld testing only.

    --statistics <csv|none>.  By default, the load driver produces CSV statistics.  Statistics can
      be turned off by specifying 'none'.

    --username <username>

    --password <password>

    --conf <configuration-file>  -  the file to read configuration from. If the same configuration
        parameter is specified both  on command linea and the configuration file, the command line
        value takes precedence.

        WARNING - configuration file support not fully implemented yet.

    --background - By default,  gld clients start in an interactive mode,  where the user is given
        access  to  an interactive console  that can be used to control the run,   generate thread
        dumps, annotate the logs, etc.  By specifying  '--background',  the console is not started
        and the gld process simply generates the load, writes statistics in a local file and exits
        when it  reaches  the load limit or it is being told externally to stop.   The command for
        stopping gld processes is "gld stop" (see below).

    --console-output <file> -  this is an optional configuration option that is relevant only when
        --background (see above) is used. When used, the stdout and stderr content produced by the
        background gld process are redirected into the specified file. If not used, the stdout and
        stderr content produced by the background process are redirected into a file named gld.log
        created in the current directory.

     --memory <integer> - the amount of memory, in MB, to be allocated to the driver's java VM. If
        not specified, the default hardcoded value in the startup script is 1024.

Commands _________________________________________________________________________________________

    help

    version

    status - process status.

    stop  - stops gld processes by sending SIGTERM into the corresponding JVMs.  If a list of PIDs
      is given,  it will stop those.  Otherwise it wil stop all gld processes found on the system.
      TODO under development, more to be added here.

                    stop [--force] [pid1 pid2 ...]

        Options:

          --force - sends SIGKILL instead of SIGTERM.


    content - dumps the key-value space into local storage (stdout, single flat file, hierarchical
      directory structure or other format). The default behavior is to dump key-value pairs at stdout.

      Options:

            --key-count-only - returns just the key count, don't retrieve the corresponding values.

            --storage-strategy <local-storage-strategy-name>. The local storage strategy. If not
            specified, the default storage strategy is Stdout - the utility dumps key-value pairs
            at stdout. For available storage strategies, see "Storage Strategies" below.

      Typical Usage Example:

            gld content --nodes localhost:10001 --storage-strategy TextFile --output ./keys.txt

      Storage Strategies:

            stdout

            TextFile - writes the key-value pairs in a local text file. Configuration options:

                --output <file-name>

            Hierarchical - a strategy that stores each key/value pair in its own separated file.
                The files are maintained in a hierarchical directory structure to avoid overloading
                the file table of a single flat directory in case of a large number of keys.

                --root <root-directory>

    generate-keys - generates random keys into a local file. No attempt is made to connect to a cache node.

        Typical Usage Example:

            gld generate-keys 10000 --key-size 70 --key-store ./keys.txt

    load - send load into nodes until it reaches --max-operations or a key is pressed.

      Options:

        --load-strategy <strategy-name> - the name of the load strategy.  The load driver searches
            its classpath for a class  with the same name,  which must implement  the LoadStrategy
            interface.   For a list of available strategies,  see "Load Strategies" section below.
            If not specified the default strategy is  "WriteThenRead".  Different  load strategies
            may have specific configuration options,  which are described in the "Load Strategies"
            section.

        --threads <thread-count>  -  the number of threads to send concurrent operations.  Default
            value 1.

        --max-operations  -  the total number of operations  (cache reads or writes,  JMS messages
          sent or received)  to send into the target service.  If not specifies,  the  default  is
          "unlimited",  meaning  the driver  will continuously send  operations  until  explicitly
          stopped.  The exact meaning of an "operation"  depends on the service  and load strategy
          being used.

        --duration <time-interval-specification> - the duration of the load run.  The duration can
          be specified in seconds (10s), minutes (10m) or hours (10h).

          TODO: when both --max-operations and --duration is used.

        --sleep <ms>   -    the number of milliseconds to sleep  after each operation is completed
            (successfully or not).  The default value is 0,  which means the load driver will send
            operations continuously.

        --key-size <number-of-characters>. Default is 70.

        --value-size|--payload-size <number-of-characters> For JMS text messages, the text message
            size. For cache key-value pairs, the value size. The default is 5000.

        --output <filename>  -  the file  to write  collected statistics into.   If not explicitly
            specified, gld will send statistics at stdout. TODO: make this a global option.

        --max-total <count> - the Apache pool "MaxTotal" parameter. Default value 100.

        --max-wait-millis <timeout> - the Apache pool "MaxWaitMillis" parameter. Default value
              1000 ms (1 second).

        --expiration <seconds>. Key expiration time in seconds. By default, keys don't expire.

        --use-different-values If specified, every time a value is sent it will be generated.
              Otherwise the same value will be reused.

        --exception-file <file-name>. If specified, dump details about exceptions in that file.
              If not specified, the exceptions will only be reflected as failure counters.

        --read-to-write <ratio>. The positive or zero integer read to write ratio. Default is 1:
              for each write there is a read. Zero means only writes, no reads. If you want more writes
              than reads, use --write-to-read (described below). --read-to-write and --write-to-read are
              mutually exclusive (unless they're both 1).

        --write-to-read <ratio>. The positive or zero integer write to read ratio. Default is 1:
              for each read there is a write. Zero means only reads, no writes. If you want more reads
              than writes, use --read-to-write (described above). --read-to-write and --write-to-read are
              mutually exclusive (unless they're both 1).

        --keystore-file <file-name>. Optional. If specified, the name of the file to read/write keys
              from/to. The current implementation works as follows:

              If no key store is specified, nothing is written and nothing is read locally. All keys
              are randomly generated.

              If a key store is specified and the load run writes (all operation are writes or only some
              of the operations are writes), then all the keys to be written are randomly generated,
              written into the cache, then written into the key store, and then used for successive read
              operations.

              If a key store is specified and there are no writes, only reads, then the keys are
              read cyclically from the local key store.

       Typical Usage Example:

            gld load --nodes localhost:10001,localhost:10002 \
                --threads 200 --max-total 192 --value-size 1024 --expiration 2 \
                --output ./stats.csv

    delete - delete keys from the cache.

      Options:

            --key-count <key-count> the number of keys to delete. The keys will be chosen randomly
              from the entire key set maintained by the cache at the moment the command is run. If
              not specified, the default value is 1.


       Typical Usage Example:

            gld delete --key-count 10 \
                --nodes node1.example.com:10001 \
                --threads 10 --output ./delete-stats.csv

Services _________________________________________________________________________________________

  Cache Service

  Cache Service Options:

    --cache <cache-name>  the name of the cache. If not specified, the default cache will be used.

  Implementations:

    --service infinispan
    --service redis

  Cache Load Strategies

    WriteThenRead

      This strategy can be used to populate the cache with random key-value pairs.

    ReadThenWriteOnMiss

      Attempts to read  keys specified in  a local store,   and if it encounters a miss,  writes a
      random value (whose size can be specified) for that key, back into the cluster.

    Delete

      This strategy can be used to read a number of keys from the cache and distribute them to
      concurrent threads  to be deleted.  It is the underlying strategy  used by the "delete"
      command, but it can also be used with the "load" command (and --load-strategy Delete
      qualifier).

    HttpSession

      This load strategy  simulates the interaction between a HTTP session manager and  a back end
      cache.  There are two modes available, currently.  In the default mode, the user selects the
      number of sessions to be simulated  and  the number of threads  that will simulate  the HTTP
      activity (you can think of these threads as typical HTTP/AJP connector threads) and the load
      driver uses the available threads to gradually "establish" the sessions  and then "read" and
      "write" for the entire test duration.   Once the allocated time has passed,  the driver will
      use  its threads  to simulate the invalidation of  all the sessions created during the test,
      leaving an empty cache.  This mode simulates the typical interaction between an application
      server and the back end cache.

      In the "session-per-thread" mode,  there is an one-to-one association  between a load driver
      thread and a HTTP session simulation instance.  A thread creates,  interacts with,  and then
      invalidates a session, and then starts over for the duration of the test.  In this mode, the
      number of HTTP sessions maintained by the back end cache cannot exceed the number of  driver
      threads. Once the allocated time has passed, the driver will invalidate  the active sessions,
      leaving an empty cache.

      Options:

        --http-session-mode <default|session-per-thread>  see above. If not specified, the mode is
          "default".

        --write-count <integer> -  the number of attribute writes to execute after the creation of
          the session and before the session is invalidated. The default value is 10.

        --sessions <integer>  the number of sessions to simulate. This is a required configuration
          parameter in default mode,  and will be ignored,  if specified,  in 'session-per-thread'
          mode,  as in that case,  the number of session is given by the number of driver threads.

        --initial-session-size <bytes>  the initial size,  in bytes,  of the sessions that will be
          simulated during the run.

  JMS

    Common to all JMS load strategies:

            --queue|--topic <destination-name>

        All JMS strategies use a "new endpoint per message but reuse the session" endpoint allocation
        policy (REUSE_SESSION_NEW_ENDPOINT_PER_OPERATION), where a new JMS MessageProducer or MessageConsumer
        are created before each operation and closed after each operation but the JMS sessions are cached and
        reused (on the same threads). The endpoint policy can be changed with:

            --endpoint-policy <REUSE_SESSION_NEW_ENDPOINT_PER_OPERATION|NEW_SESSION_NEW_ENDPOINT_PER_OPERATION>

        Send

            This is the default JMS load strategy.

            It can be explicitly configured with:

                --load-strategy send

            Example:

            gld load --service activemq --threads 10 --load-strategy send \
                 --nodes node1.example.com:61616 --username admin --password admin123 \
                 --queue TEST --endpoint-policy REUSE_SESSION_NEW_ENDPOINT_PER_OPERATION \
                 --max-operations 1

        Receive

            This strategy can be explicitly configured with:

                --load-strategy receive

            Options:

                --timeout <timeout-ms> - receive timeout. Default is 0 (never timeout).

            Example:

            gld load --service activemq --threads 10 --load-strategy receive \
                 --nodes node1.example.com:61616 --username admin --password admin123 \
                 --queue TEST --endpoint-policy REUSE_SESSION_NEW_ENDPOINT_PER_OPERATION \
                 --max-operations 1

  Embedded

    Used for testing:

            gld load --service embedded-generic --nodes embedded