WildFly HornetQ-Based Messaging Subsystem - Clustering with TCP: Difference between revisions
Line 171: | Line 171: | ||
=Client-Side= | =Client-Side= | ||
<blockquote style="background-color: #f9f9f9; border: solid thin lightgrey;"> | |||
:[[WildFly HornetQ-Based Messaging Subsystem Concepts#Client-Side_Load_Balancing|Messaging Concepts - Client-Side Load Balancing]] | |||
</blockquote> |
Revision as of 02:00, 7 March 2016
External
- Switch from UDP to TCP for HornetQ clustering in JBoss EAP 6 https://access.redhat.com/solutions/293823
- Administration and Configuration Manual - Switch UDP to TCP for HornetQ Clustering https://access.redhat.com/documentation/en-US/JBoss_Enterprise_Application_Platform/6.4/html/Administration_and_Configuration_Guide/sect-Clustering.html#Switch_UDP_to_TCP_for_HornetQ_Clustering
Internal
Relevance
- EAP 6.4
Overview
This article shows how to configure a network of HornetQ brokers, assuming that the network has no multicast capabilities. For more details on HornetQ Clustering see WildFly HornetQ Clustering Concepts.
Procedure
Make Sure <clustering> is Enabled
The "messaging" subsystem must be configured with <clustered>true</clustered> on all nodes of the cluster:
<subsystem xmlns="urn:jboss:domain:messaging:1.4"> <hornetq-server> ... <clustered>true</clustered> ... </subsystem>
On Each Cluster Node Configure Connectors To All Other Nodes
Assuming the cluster has three nodes ("node1", "node2" and "node3"), the configuration for node1 should declare <connector>s and <outbound-socket-binding>s for node2 and node3:
... <subsystem xmlns="urn:jboss:domain:messaging:1.4"> <hornetq-server> ... <connectors> ... <netty-connector name="node2-connector" socket-binding="node2-hornetq-binding"/> <netty-connector name="node3-connector" socket-binding="node3-hornetq-binding"/> </connectors> ... </subsystem> ... <socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}"> ... <outbound-socket-binding name="node2-hornetq-binding"> <remote-destination host="1.2.3.5" port="5445"/> </outbound-socket-binding> <outbound-socket-binding name="node3-hornetq-binding"> <remote-destination host="1.2.3.6" port="5445"/> </outbound-socket-binding> </socket-binding-group> ...
Note: make sure the nodes at the other end of the connection actually listen on the specified interfaces/ports.
Comment out <broadcast-groups>, <discovery-groups>
They are useless without multicast.
Add a <cluster-connection>
... </address-settings> <cluster-connections> <cluster-connection name="tcp-based-cluster"> <address>jms</address> <connector-ref>netty</connector-ref> <retry-interval>500</retry-interval> <use-duplicate-detection>true</use-duplicate-detection> <forward-when-no-consumers>true</forward-when-no-consumers> <max-hops>1</max-hops> <static-connectors> <connector-ref>node2-connector</connector-ref> <connector-ref>node3-connector</connector-ref> </static-connectors> </cluster-connection> </cluster-connections> ...
Configure Security
If security is enabled (the <securit-enabled> configuration element is "true", and this is the default value even if it is not explicitly declared), then connections authenticate themselves as "HORNETQ.CLUSTER.ADMIN.USER" - the "cluster user". Unless a "cluster password" is specified in the configuration, a random password is used, and the connection establishing will fail. In order to provide known credentials for the cluster connections, configure the following (<cluster-user> is optional and if not specified will default to "HORNETQ.CLUSTER.ADMIN.USER"):
... <subsystem xmlns="urn:jboss:domain:messaging:1.4"> <hornetq-server> ... <cluster-user>hq</cluster-user> <cluster-password>hq123</cluster-password> ... </hornetq-server> </subsystem> ...
Verify this: Note that the user declared as such does not need to be declared in any security repository, the configuration file declaration is sufficient.
Confirmation
You should see this in the logs:
15:11:19,234 INFO [org.hornetq.core.server.cluster.impl.BridgeImpl] (Thread-7 (HornetQ-server-HornetQServerImpl::serverUUID=388611e3-810b-11e2-85c9-694bd06e7c3c-544291442)) Bridge ClusterConnectionBridge@26250f55 [name=sf.o01-o02-tcp-cluster.c07654e0-810b-11e2-a372-f9f6868e86f7, queue=QueueImpl[name=sf.o01-o02-tcp-cluster.c07654e0-810b-11e2-a372-f9f6868e86f7, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=388611e3-810b-11e2-85c9-694bd06e7c3c]]@4b11a903 targetConnector=ServerLocatorImpl (identity=(Cluster-connection-bridge::ClusterConnectionBridge@26250f55 [name=sf.o01-o02-tcp-cluster.c07654e0-810b-11e2-a372-f9f6868e86f7, queue=QueueImpl[name=sf.o01-o02-tcp-cluster.c07654e0-810b-11e2-a372-f9f6868e86f7, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=388611e3-810b-11e2-85c9-694bd06e7c3c]]@4b11a903 targetConnector=ServerLocatorImpl [initialConnectors=[org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=10-76-128-155], discoveryGroupConfiguration=null]]::ClusterConnectionImpl@1267429957[nodeUUID=388611e3-810b-11e2-85c9-694bd06e7c3c, connector=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=10-76-128-154, address=jms, server=HornetQServerImpl::serverUUID=388611e3-810b-11e2-85c9-694bd06e7c3c])) [initialConnectors=[org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=10-76-128-155], discoveryGroupConfiguration=null]] is connected
EAP 6.4
20:37:47,719 INFO [org.hornetq.core.server] (Thread-16 (HornetQ-server-HornetQServerImpl::serverUUID=5032e902-e3fe-11e5-926c-a50d5e1d9a16-46422202)) HQ221027: Bridge ClusterConnectionBridge@20600d51 [name=sf.tcp-based-cluster.c9e7d432-e3fe-11e5-915f-d3dc1740ef3d, queue=QueueImpl[name=sf.tcp-based-cluster.c9e7d432-e3fe-11e5-915f-d3dc1740ef3d, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=5032e902-e3fe-11e5-926c-a50d5e1d9a16]]@298aa2ce targetConnector=ServerLocatorImpl (identity=(Cluster-connection-bridge::ClusterConnectionBridge@20600d51 [name=sf.tcp-based-cluster.c9e7d432-e3fe-11e5-915f-d3dc1740ef3d, queue=QueueImpl[name=sf.tcp-based-cluster.c9e7d432-e3fe-11e5-915f-d3dc1740ef3d, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=5032e902-e3fe-11e5-926c-a50d5e1d9a16]]@298aa2ce targetConnector=ServerLocatorImpl [initialConnectors=[TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5445&host=172-31-17-79], discoveryGroupConfiguration=null]]::ClusterConnectionImpl@1116810330[nodeUUID=5032e902-e3fe-11e5-926c-a50d5e1d9a16, connector=TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5445&host=172-31-28-2, address=jms, server=HornetQServerImpl::serverUUID=5032e902-e3fe-11e5-926c-a50d5e1d9a16])) [initialConnectors=[TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5445&host=172-31-17-79], discoveryGroupConfiguration=null]] is connected
Configure ConnectionFactory for Load Balancing
Add the extra connectors to each ConnectionFactories that are exposed and supposed to be used for load balancing/failover. The connectors are the same that have been added to the cluster connection <static-connectors> list:
... <subsystem xmlns="urn:jboss:domain:messaging:1.4"> <hornetq-server> ... <jms-connection-factories> ... <connection-factory name="RemoteConnectionFactory"> <connectors> <connector-ref connector-name="netty"/> <connector-ref connector-name="node2-connector"/> <connector-ref connector-name="node3-connector"/> </connectors> <entries> <entry name="java:jboss/exported/jms/RemoteConnectionFactory"/> </entries> </connection-factory> ... </jms-connection-factories> </hornetq-server> </subsystem> ...
What Happens if a Cluster Node is Down when Other Starts Up
The node that is starting up attempts to connect ? times.
If it can't, it logs as DEBUG:
2013-02-20 11:39:45,131 DEBUG [org.hornetq.core.client.impl.ServerLocatorImpl] (Thread-1 (HornetQ-server-HornetQServerImpl::serverUUID=afe53124-6e53-11e2-bb90-005056ac34a6-1045936331)) Connector [initialConnector=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=10-153-198-31]::Exception on establish connector initial connection HornetQException[errorCode=2 message=Unable to connect to server using configuration org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=10-153-198-31] at org.hornetq.core.client.impl.ClientSessionFactoryImpl.connect(ClientSessionFactoryImpl.java:233) at org.hornetq.core.client.impl.ServerLocatorImpl$StaticConnector$Connector.tryConnect(ServerLocatorImpl.java:1787) at org.hornetq.core.client.impl.ServerLocatorImpl$StaticConnector.connect(ServerLocatorImpl.java:1624) at org.hornetq.core.client.impl.ServerLocatorImpl.connect(ServerLocatorImpl.java:597) at org.hornetq.core.client.impl.ServerLocatorImpl$3.run(ServerLocatorImpl.java:564) at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662)