WildFly Clustering without Multicast: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(2 intermediate revisions by the same user not shown)
Line 22: Line 22:
...
...
</pre>
</pre>


WildFly 10.0 and higher:
WildFly 10.0 and higher:
Line 102: Line 101:


===CLI===
===CLI===
An example of how this procedure is implemented by <tt>em</tt> is available here, search for "function jgroups-swap-MPING-with-TCPPING":
<blockquote style="background-color: AliceBlue; border: solid thin LightSteelBlue;">
:https://github.com/NovaOrdis/em/blob/master/src/main/bash/bin/overlays/lib/jboss.shlib<br>
</blockquote>


Note that we can't simply remove MPING and add TCPING, the CLI API is not expressive enough to allow us to specify the protocol's position in the list. We need to replace MPING with TCPPING as follows:
Note that we can't simply remove MPING and add TCPING, the CLI API is not expressive enough to allow us to specify the protocol's position in the list. We need to replace MPING with TCPPING as follows:
Line 109: Line 114:
</pre>
</pre>


then remove the "socket-binding" node:
<blockquote style="background-color: Gold; border: solid thin Goldenrod;">
:All CLI commands below keep referring to the protocol as "MPING", that won't change until the instance is restarted, so it's not a typo.<br>
</blockquote>
 
Remove the "socket-binding" node:


<pre>
<pre>

Latest revision as of 15:54, 27 May 2016

External

Internal

Procedure

Switch to a "tcp" Default Stack

Configuration File

Locate the "jgroups" subsystem in standalone.xml or domain.xml relevant profile, and set default-stack value to "tcp":

<subsystem xmlns="urn:jboss:domain:jgroups:1.1" default-stack="tcp">
...

WildFly 10.0 and higher:

...
<subsystem xmlns="urn:jboss:domain:jgroups:4.0">
   <channels ...>...</channels>
    <stacks default="tcp">
    ...

Domain CLI

/profile=ha/subsystem=jgroups:write-attribute(name=default-stack,value=tcp)

Note that the operation requires reload, but you should reload the server configuration only after the entire procedure is complete (see reload)

Standalone CLI

/subsystem=jgroups:write-attribute(name=default-stack,value=tcp)

Replace the MPING protocol with TCPPING

Configuration File

Locate the "tcp" stack inside the "jgroups" subsystem and replace the MPING protocol with TCPPING:

   ...
   <stack name="tcp">
      <transport type="TCP" socket-binding="jgroups-tcp"/>
      <protocol type="TCPPING">
          <property name="initial_hosts">1.2.3.4[7600],1.2.3.5[7600]</property>
          <property name="num_initial_members">2</property>
          <property name="port_range">0</property>
          <property name="timeout">2000</property>
       </protocol>
       <!--<protocol type="MPING" socket-binding="jgroups-mping"/>-->
       <protocol type="MERGE2"/>
       ...
   </stack>
  ...

If the domain model is used and the same profile is shared by several server groups, the "initial_hosts" property should be set on the server_group, as follows:

   ...
   <stack name="tcp">
      <transport type="TCP" socket-binding="jgroups-tcp"/>
      <protocol type="TCPPING">
          <property name="initial_hosts">${jboss.cluster.tcp.initial_hosts}</property>
          ...
       </protocol>
       ...
   </stack>
  ...

and the server group-specific values for the system property are set in the <server-group> element as follows:

    ...
    <server-groups>
        <server-group name="something" profile="ha">
            <socket-binding-group ref="ha-sockets"/>
            <system-properties>
              <property name="jboss.cluster.tcp.initial_hosts" value="1.2.3.4[7600],1.2.3.5[7600]" />
            </system-properties>
       </server-group>
      ...
    <server-groups>

CLI

An example of how this procedure is implemented by em is available here, search for "function jgroups-swap-MPING-with-TCPPING":

https://github.com/NovaOrdis/em/blob/master/src/main/bash/bin/overlays/lib/jboss.shlib

Note that we can't simply remove MPING and add TCPING, the CLI API is not expressive enough to allow us to specify the protocol's position in the list. We need to replace MPING with TCPPING as follows:

/profile=ha/subsystem=jgroups/stack=tcp/protocol=MPING/:write-attribute(name=type,value=TCPPING)
All CLI commands below keep referring to the protocol as "MPING", that won't change until the instance is restarted, so it's not a typo.

Remove the "socket-binding" node:

/profile=ha/subsystem=jgroups/stack=tcp/protocol=MPING/:write-attribute(name=socket-binding)
/profile=ha/subsystem=jgroups/stack=tcp/protocol=MPING/property=initial_hosts:add(value="1.2.3.4[7600],1.2.3.5[7600]")
/profile=ha/subsystem=jgroups/stack=tcp/protocol=MPING/property=num_initial_members:add(value="2")
/profile=ha/subsystem=jgroups/stack=tcp/protocol=MPING/property=port_range:add(value="0")
/profile=ha/subsystem=jgroups/stack=tcp/protocol=MPING/property=timeout:add(value="2000")
In domain mode, if the same profile is shared by several server groups, the "initial_hosts" property should be set on the server_group and not in the profile, as follows:
/profile=ha/subsystem=jgroups/stack=tcp/protocol=MPING/property=initial_hosts:add(value="${jboss.cluster.tcp.initial_hosts}")

In this case, the server group-specific values for the system property are set in the <server-group> element as described in manipulating per-server-group properties (note that the value must be set before :reload otherwise the reload will fail:

/server-group=web/system-property=jboss.cluster.tcp.initial_hosts:add(value="1.2.3.4[7600],1.2.3.5[7600]")

Reload

The controllers must be reloaded, first the domain controller and then the host controllers. It is important to reload the domain controller first, otherwise MPING to TCPPING replacement does not propagate to the subordinate host controllers:

reload --host=dc1
reload --host=h1 --restart-servers=true
reload --host=h2 --restart-servers=true

For more details see reload.

Additional Verifications

  • Verify that the cluster members do actually bind to the IP addresses specified in initial_hosts.
  • See port_range recommendations.
  • See num_initial_members recommendations.

Why Doesn't the Cluster Form?

Even if the cluster if correctly configured, the JGroups channels won't be initialized and won't form clusters at boot. This is because the JGroups groups only form if there are services requiring clustering.

One way to start clustering is to deploy a <distributable> servlet.

Another way is to declare cache containers as "eager" starters. For more details see WildFly Infinispan Subsystem Configuration#Caches_Do_Not_Start_at_Boot_Even_if_Declared_Eager.

Other Subsystems that May Require Switching to TCP

WildFly HornetQ-Based Messaging Subsystem - Clustering with TCP