JBoss Clustering without Multicast
Internal
How does JBoss use multicast?
Application Server cluster member management
A JBoss cluster member learns about an existing JBoss cluster by sending multicast queries to a pre-defined multicast address and then listening for existing member replies. IP multicast makes the discovery process environment detail independent - no prior knowledge about the environment, such as specific IP addresses, is necessary. Cluster nodes my come and go without any specific configuration.
Application Server cluster-wide data replication
Data supposed to be seen by all members of the cluster is sent once by originator and distributed across the cluster via a hardware mechanism, instead of being sent individually to each cluster member in a loop. This approach results in performance gains especially if large quantities of data need to be replicated among members.
HornetQ cluster node discovery
A new HornetQ cluster node learns about an existing HornetQ cluster by read discovery and broadcast groups. The mechanism is largely similar with the one employed by the application server cluster; the reason JBoss application server and HornetQ use two different clustering mechanisms is historical: HornetQ clusters were intended to work outside JBoss application server clusters.
HA-JNDI client discovery
What does it need to be configured in order to remove the dependency on multicast?
1. The main application server clustering stack must be reconfigured use a TCP transport instead of the UDP transport and a TCPPING discovery layer instead of MPING.
2. TCPPING requires that some of the IP addresses of the other cluster members be hardcoded in its configuration.
3. HornetQ discovery mechanism must be switched from discovery/broadcast to a statically configured mesh of point-to-point connections.
4. JNDI clients must not use multicast on discovery anymore. They must use specific points of contact instead.
For details, see Procedures below.
Procedures
Impact of Removing Multicast Support
Manual Configuration Required when Adding a Node to the Cluster
No-configuration cluster expansion/contraction is lost. Adding new nodes requires modifying configuration files on all other cluster nodes.
In the application server's case, dynamic discovery can theoretically still be implemented with a TCP stack. The multicast address is just a "well known rendezvous point". Shall that be replaced with something equivalent (shared file system, shared database, etc.), it is possible to achieve dynamic discovery without having to individually configure addresses.
Unfortunately, HornetQ is more restrictive, it does not allow for alternative dynamic discovery, so there is no substitute for modifying configuration files on cluster expansion/contraction.
Decrease in Data Replication Performance
There is no substitute for hardware acceleration on replicating data. If a piece of data needs to be send to all member of the cluster, it will have to be send individually in a loop to all members (opposite to just sending it to a multicast socket that triggers hardware replication over network). This may or may not impact the application depending on how much data is replicated.