Infinispan HotRod Java Client

Internal

• Infinispan Concepts

Overview

HotRod defines three level of intelligence for the clients:

• basic client, interested in neither cluster nor hash information
• topology-aware client, interested in cluster information
• hash-distribution-aware client, that is interested in both cluster and hash information

The java client implementation supports all three levels of intelligence. It is transparently notified whenever a new server is added/removed from the HotRod cluster. At startup it only needs to know the address of one HotRod server On connection to the server the cluster topology is piggybacked to the client, and all further requests are being dispatched to all available servers. Any further topology change is also piggybacked.

The client is also hash-distribution-aware. This means that, for each operation, the client chooses the most appropriate remote server to go to: the data owner. As an example, for a put(k,v) operation, the client calculates k’s hash value and knows exactly on which server the data resides on. Then it picks up a tcp connection to that particular server and dispatches the operation to it. This means less burden on the server side which would otherwise need to lookup the value based on the key’s hash. It also results in a quicker response from the server, as an additional network roundtrip is skipped. This hash-distribution-aware aspect is only relevant to the distributed HotRod clusters and makes no difference for replicated server deployments.

For a replicated cluster, the client can load balance requests. This aspect is irrelevant with distributed clusters, where, as shown above, the client sends/receives data from the data owner node. The default strategy is round-robin: requests are being dispatched to all existing servers in a circular manner. Custom types of balancing policies can defined by implementing the RequestBalancingStrategy and by specifying it through the infinispan.client.hotrod.request-balancing-strategy configuration property.

In order to avoid creating a TCP connection on each request (which is a costly operation), the client keeps a pool of persistent connections to all the available servers and it reuses these connections whenever it is possible. The validity of the connections is checked using an async thread that iterates over the connections in the pool and sends a HotRod ping command to the server. By using this connection validation process the client is being proactive: there’s a hight chance for broken connections to be found while being idle in the pool and no on actual request from the application. The number of connections per server, total number of connections, how long should a connection be kept idle in the pool before being closed - all these (and more) can be configured. RemoteCacheManager documentation describes configuration options.

Various server usage statistics can be obtained through the RemoteCache.stats() method. This returns a ServerStatistics object.

API

The basic API to interact with an Infinispan server over the HotRod protocol:

import org.infinispan.client.hotrod.RemoteCache;
import org.infinispan.client.hotrod.RemoteCacheManager;
import org.infinispan.client.hotrod.configuration.Configuration;
import org.infinispan.client.hotrod.configuration.ConfigurationBuilder;

...

String host = "localhost";
int port = 11222;

Configuration c = new ConfigurationBuilder().addServer().host(host).port(port).build();

RemoteCacheManager remoteCacheManager = new RemoteCacheManager(c);

RemoteCache defaultCache = emoteCacheManager.getCache();

RemoteCache namedCache = remoteCacheManager.getCache("some-cache");

Relationship between RemoteCacheManager and a Server-Side Cache Container

The RemoteCacheManager is configured with a host name and port. The host:port pair is associated on the server-side with a specific cache container. It starts with the "hotrod" socket binding definition:

<socket-binding-group name="standard-sockets" ...>
    ...
    <socket-binding name="hotrod" port="11222"/>
    ...
</socket-binding-group>

The socket binding definition is associated with the HotRod connector configuration declared in the "infinispan:server:endpoint" subsystem:

...
<subsystem xmlns="urn:infinispan:server:endpoint:6.1">
    <hotrod-connector cache-container="clustered" socket-binding="hotrod">
        <topology-state-transfer lazy-retrieval="false" lock-timeout="1000" replication-timeout="5000"/>
    </hotrod-connector>
</subsystem>
...

More details about the HotRod connector are available here:

The HotRod Connector

The HotRod connector thus declared is bound to a specific cache container.

Interacting with a HotRod Server from within the Same JVM

TODO:

http://infinispan.org/docs/6.0.x/user_guide/user_guide.html#sid-68355104

Factory for {@link org.infinispan.client.hotrod.RemoteCache}s.

Lifecycle:

In order to be able to use an {@link org.infinispan.client.hotrod.RemoteCache}, the {@link org.infinispan.client.hotrod.RemoteCacheManager} must be started first: beside other things, this instantiates connections to Hot Rod server(s). Starting the {@link org.infinispan.client.hotrod.RemoteCacheManager} can be done either at creation by passing start==true to constructor or by using a constructor that does that for you (see C-tor documentation); or after construction by calling {@link #start()}.

This is an "expensive" object, as it manages a set of persistent TCP connections to the Hot Rod servers. It is recommended to only have one instance of this per JVM, and to cache it between calls to the server (i.e. remoteCache operations).

{@link #stop()} needs to be called explicitly in order to release all the resources (e.g. threads, TCP connections).

Configuration:

The cache manager is configured through a {@link Configuration} object passed to the constructor (there are also

• "simplified" constructors that rely on default values). If migrating from a previous version of Infinispan, where {@link Properties}

* were used to configure the RemoteCacheManager, please use the {@link ConfigurationBuilder#withProperties(Properties)} method.

*

* Below is the list of supported configuration elements: *

*
• infinispan.client.hotrod.request_balancing_strategy, default = org.infinispan.client.hotrod.impl.transport.tcp.RoundRobinBalancingStrategy. For replicated (vs distributed) Hot Rod server clusters, the client balances requests to the * servers according to this strategy.
*
• infinispan.client.hotrod.server_list, default = 127.0.0.1:11222. This is the initial list of Hot Rod servers to connect to, specified in the following format: host1:port1;host2:port2... * At least one host:port must be specified.
*
• infinispan.client.hotrod.force_return_values, default = false. Whether or not to implicitly {@link org.infinispan.client.hotrod.Flag#FORCE_RETURN_VALUE} for all calls.
*
• infinispan.client.hotrod.tcp_no_delay, default = true. Affects TCP NODELAY on the TCP stack.
*
• infinispan.client.hotrod.tcp_keep_alive, default = false. Affects TCP KEEPALIVE on the TCP stack.
*
• infinispan.client.hotrod.transport_factory, default = org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory - controls which transport to use. Currently only the TcpTransport is supported.
*
• infinispan.client.hotrod.marshaller, default = org.infinispan.marshall.jboss.GenericJBossMarshaller. Allows you to specify a custom {@link org.infinispan.marshall.Marshaller} implementation to serialize and deserialize user objects. For portable serialization payloads, you should configure the marshaller to be {@link org.infinispan.client.hotrod.marshall.ApacheAvroMarshaller}
*
• infinispan.client.hotrod.async_executor_factory, default = org.infinispan.client.hotrod.impl.async.DefaultAsyncExecutorFactory. Allows you to specify a custom asynchroous executor for async calls.
*
• infinispan.client.hotrod.default_executor_factory.pool_size, default = 99. If the default executor is used, this configures the number of threads to initialize the executor with.
*
• infinispan.client.hotrod.default_executor_factory.queue_size, default = 100000. If the default executor is used, this configures the queue size to initialize the executor with.
*
• infinispan.client.hotrod.hash_function_impl.1, default = It uses the hash function specified by the server in the responses as indicated in {@link org.infinispan.client.hotrod.impl.consistenthash.ConsistentHashFactory}. This specifies the version of the hash function and consistent hash algorithm in use, and is closely tied with the HotRod server version used.
*
• infinispan.client.hotrod.key_size_estimate, default = 64. This hint allows sizing of byte buffers when serializing and deserializing keys, to minimize array resizing.
*
• infinispan.client.hotrod.value_size_estimate, default = 512. This hint allows sizing of byte buffers when serializing and deserializing values, to minimize array resizing.
*
• infinispan.client.hotrod.socket_timeout, default = 60000 (60 seconds). This property defines the maximum socket read timeout before giving up waiting for bytes from the server.
*
• infinispan.client.hotrod.protocol_version, default = 2.0 .This property defines the protocol version that this client should use. Other valid values include 1.0.
*
• infinispan.client.hotrod.connect_timeout, default = 60000 (60 seconds). This property defines the maximum socket connect timeout before giving up connecting to the server.
*
• infinispan.client.hotrod.max_retries, default = 10. This property defines the maximum number of retries in case of a recoverable error. A valid value should be greater or equals to 0 (zero). Zero mean no retry.
*

* 

* The following properties are related to connection pooling:

*

*

*
• maxActive - controls the maximum number of connections per server that are allocated (checked out to client threads, or idle in * the pool) at one time. When non-positive, there is no limit to the number of connections per server. When maxActive * is reached, the connection pool for that server is said to be exhausted. The default setting for this parameter is * -1, i.e. there is no limit.
*
• maxTotal - sets a global limit on the number persistent connections that can be in circulation within the combined set of * servers. When non-positive, there is no limit to the total number of persistent connections in circulation. When * maxTotal is exceeded, all connections pools are exhausted. The default setting for this parameter is -1 (no limit). *
*

*

• maxIdle - controls the maximum number of idle persistent connections, per server, at any time. When negative, there is no limit * to the number of connections that may be idle per server. The default setting for this parameter is -1.
*

*

• * whenExhaustedAction - specifies what happens when asking for a connection from a server's pool, and that pool is exhausted. Possible values: *
  *
  • 0 - an exception will be thrown to the calling user
  *
  • 1 - the caller will block (invoke waits until a new or idle connections is available. *
  • 2 - a new persistent connection will be created and returned (essentially making maxActive meaningless.)
  *

  * The default whenExhaustedAction setting is 1.
  

  *
*

*

• * Optionally, one may configure the pool to examine and possibly evict connections as they sit idle in the pool and to * ensure that a minimum number of idle connections is maintained for each server. This is performed by an "idle connection * eviction" thread, which runs asynchronously. The idle object evictor does not lock the pool * throughout its execution. The idle connection eviction thread may be configured using the following attributes: *
  *
  • timeBetweenEvictionRunsMillis - indicates how long the eviction thread should sleep before "runs" of examining idle * connections. When non-positive, no eviction thread will be launched. The default setting for this parameter is * 2 minutes
  *
  • minEvictableIdleTimeMillis - specifies the minimum amount of time that an connection may sit idle in the pool before it * is eligible for eviction due to idle time. When non-positive, no connection will be dropped from the pool due to * idle time alone. This setting has no effect unless timeBetweenEvictionRunsMillis > 0. The default setting for this * parameter is 1800000(30 minutes).
  *
  • testWhileIdle - indicates whether or not idle connections should be validated by sending an TCP packet to the server, * during idle connection eviction runs. Connections that fail to validate will be dropped from the pool. This setting * has no effect unless timeBetweenEvictionRunsMillis > 0. The default setting for this parameter is true. *
  *
  • minIdle - sets a target value for the minimum number of idle connections (per server) that should always be available. * If this parameter is set to a positive number and timeBetweenEvictionRunsMillis > 0, each time the idle connection * eviction thread runs, it will try to create enough idle instances so that there will be minIdle idle instances * available for each server. The default setting for this parameter is 1.
  *
  *
*
• *

*
* @author Mircea.Markus@jboss.com
* @since 4.1
*/