Mod cluster Installation

TODO deplete https://home.feodorov.com:9443/wiki/Wiki.jsp?page=Mod_clusterInstallation

Internal

• mod_cluster Subjects

Download

Compiled bundles are available here: http://mod-cluster.jboss.org/downloads

Native Components Installation on Linux

This procedure assumes httpd was already installed, configured, and it works well without mod_cluster.

At the time of the writing, the pre-compiled native components to be installed within an existing httpd are published on the download page as "mod_cluster modules for httpd". For example, the mod_cluster binaries for Linux x86 httpd are listed as "linux-x86_64 mod_cluster binaries". The file name is mod_cluster-1.3.1.Final-linux2-x64-so.tar.gz. It contains the pre-compiled httpd dynamic libraries (mod_advertise.so, mod_cluster_slotmem.so, mod_manager.so and mod_proxy_cluster.so).

Place the Dynamic Libraries in the modules Directory

Identify the httpd instance's module directory, by looking at the existing LoadModule directives specified in the configuration. It is usually /etc/httpd/modules.

It is good practice to include the mod_cluster version in the name of the dynamic library files, to make it obvious what version is used.

cp mod_advertise.so /etc/httpd/modules/mod_advertise-1.3.1.Final.so 
cp mod_cluster_slotmem.so /etc/httpd/modules/mod_cluster_slotmem-1.3.1.Final.so 
cp mod_manager.so /etc/httpd/modules/mod_manager-1.3.1.Final.so 
cp mod_proxy_cluster.so /etc/httpd/modules/mod_proxy_cluster-1.3.1.Final.so 

Make sure the newly copied files have the same ownership and permissions as the existing modules.

Load Modules at Startup

A standard RHEL httpd installation has its modules configured in /etc/httpd/conf.modules.d. All files present in that directory are automatically included from httpd.conf:

...
Include conf.modules.d/*.conf
...

Follow the same pattern to configure mod_cluster.

Add a new 99-mod_cluster.conf file in /etc/httpd/conf.modules.d with the following content:

#
# This file configures mod_cluster
#

LoadModule cluster_slotmem_module modules/mod_cluster_slotmem-1.3.1.Final.so
LoadModule manager_module modules/mod_manager-1.3.1.Final.so 
LoadModule proxy_cluster_module modules/mod_proxy_cluster-1.3.1.Final.so
LoadModule advertise_module modules/mod_advertise-1.3.1.Final.so

Insure mod_proxy Dependencies are Present and Loaded

mod_cluster needs mod_proxy to work. More details about mod_cluster dependency on mod_proxy are found here.

Make sure mod_proxy.so and mod_proxy_ajp.so are loaded. For a standard httpd installation, those modules are loaded in /etc/httpd/conf.modules.d/00-proxy.conf:

...
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
...

Remove undesired mod_proxy Dependencies

mod_cluster's mod_proxy_cluster cannot work correctly if mod_proxy's mod_proxy_balancer, so mod_proxy_balancer must be removed from the httpd configuration.

For a standard httpd deployment, mod_proxy_balancer is loaded in /etc/httpd/conf.modules.d/00-proxy.conf.

SELinux Configuration

If the host has SELinux enabled, we will need to install additional policies to allow httpd to write mod_cluster-related files in /var/log/httpd and also to perform multicast socket operations.

Start with a mod_cluster_1.te policy file:

module mod_cluster_1 1.0;

require {
        type httpd_log_t;
        type httpd_t;
        type unreserved_port_t;
        class file { write unlink };
        class dir remove_name;
        class udp_socket name_bind;
}

#============= httpd_t ==============
allow httpd_t httpd_log_t:dir remove_name;
allow httpd_t httpd_log_t:file unlink;
allow httpd_t httpd_log_t:file write;
allow httpd_t unreserved_port_t:udp_socket name_bind;

Compile the policy:

checkmodule -M -m -o mod_cluster_1.mod mod_cluster_1.te

Create the module package:

semodule_package -o  mod_cluster_1.pp -m mod_cluster_1.mod 

Install the policy:

semodule -i mod_cluster_1.pp

Verify that the policy was installed:

semodule -l | grep mod_cluster_1

More details about managing SELinux policies are available here: SELinux Operations.

Basic httpd.conf Configuration

This is a simple configuration that can be used to conduct smoke tests. It consists in just one mod_cluster-enabled virtual host. Note that access is allowed to all, you may want to configure stricter security in production. Also note that /mod_cluster-manager access is enabled. More complex configurations are of course possible. For more details on how to configure mod_cluster, go to mod_cluster Configuration.

...

<VirtualHost 1.2.3.4:80>

    <Directory />
        Allow from all
    </Directory>

    KeepAliveTimeout 60
    MaxKeepAliveRequests 0
    ManagerBalancerName mycluster
    ServerAdvertise On
    AdvertiseGroup 224.0.1.105:23364
    AdvertiseFrequency 5

    <Location /mod_cluster-manager>
        SetHandler mod_cluster-manager
        Allow from all
    </Location>

</VirtualHost>

...

Important

the default httpd configuration restricts access to the root directory, so you may want to comment out the following. Can do better than that.

#<Directory />
#    AllowOverride none
#    Require all denied
#</Directory>

AdvertiseGroup

The multicast address and port used by AdvertiseGroup must be synchronized with the values used by JBoss nodes. See AdvertiseGroup.

Native Component Installation on Windows

TODO, manual page 6, section 2.2.4 Install on Windows.

iptables Configuration

If iptables is enabled, make sure the multicast, HTTP and AJP traffic are allowed:

HTTP and AJP Traffic

On the JBoss nodes, add this to /etc/sysconfig/iptables:

-A INPUT -i eth0 -p tcp -m tcp --dport 8080 -m state --state NEW -j ACCEPT
-A INPUT -i eth0 -p tcp -m tcp --dport 8009 -m state --state NEW -j ACCEPT

Multicast Traffic

Multicast traffic should propagate from the httpd nodes to the JBoss node and maybe back? Are JBoss nodes multicasting to httpd? Research that.. Normally iptables is configured to leave all traffic out by default, so the httpd nodes should be fine. However, JBoss nodes must be enabled to allow the multicast traffic in, so add this to /etc/sysconfig/iptables:

-A INPUT -m pkttype --pkt-type multicast -j ACCEPT

More details on configuring iptables are available here: iptables configuration examples.

JBoss Components Installation

standard-ha and standard-full-ha WildFly 9 profiles come with mod_cluster enabled.

Make sure the advertise multicast address and port as the same on the httpd nodes and JBoss nodes.

Also make sure multicast propagates between httpd and the JBoss nodes.

Test Initial Installation

Start httpd, it should start without warnings or errors.

mod_cluster will create a set of new files in (by default) /var/log/httpd: manager.*.slotmem and manager.*.lock.