OpenShift 3.5 Installation
External
Internal
Overview
There are two installation methods: quick install, which uses a CLI tool available in the "atomic-openshift-utils" package, which, in turn, uses Ansible in the background, and advanced install. The advanced install assumes familiarity with Ansible. This document covers advance install.
Prerequisites
External DNS Setup
An external DNS server is required.
Procedure to configure a bind server:
Minimum Hardware Requirements
A full RHEL7.3/Master installation requires 121 MB in /boot and 1.7GB in /.
O/S Requirements and Configuration
Install RHEL 7.3 in "minimal" installation mode. A possible way to install it is on top of a VirtualBox virtual machine, as described here.
OpenShift requires NetworkManager on all nodes (see https://docs.openshift.com/container-platform/3.5/install_config/install/prerequisites.html#prereq-networkmanager). Using NetworkManager, assign a static IP address to the interface to be used by the OpenShift cluster, as described here: adding a Static Ethernet Connection with NetworkManager.
Turn off sshd client name DNS verification.
Attach the node to the subscription, using subscription manager, as described here: registering a RHEL System with subscription manager then follow the specific subscription steps required by OpenShift: https://docs.openshift.com/container-platform/3.5/install_config/install/host_preparation.html#host-registration
Sequence of steps - the goal of these steps is to configure the following supported repositories on the system: "rhel-7-server-rpms", "rhel-7-server-extras-rpms", "rhel-7-server-ose-3.5-rpms", "rhel-7-fast-datapath-rpms":
subscription-manager register subscription-manager list --available --matches '*OpenShift*' subscription-manager attach --pool=<pool-id> --quantity=1 subscription-manager repos --disable="*" subscription-manager repos --list-enabled yum repolist yum-config-manager --disable <repo_id> subscription-manager repos --enable="rhel-7-server-rpms" --enable="rhel-7-server-extras-rpms" --enable="rhel-7-server-ose-3.5-rpms" --enable="rhel-7-fast-datapath-rpms" subscription-manager repos --list-enabled yum repolist
Install base packages (https://docs.openshift.com/container-platform/3.5/install_config/install/host_preparation.html#installing-base-packages):
yum install wget git net-tools bind-utils iptables-services bridge-utils bash-completion yum update -y yum install atomic-openshift-utils
Prevent accidental upgrades of OpenShiift and Docker, by installing "excluder" packages. The *-excluder packages add entries to the "exclude" directive in the host’s /etc/yum.conf file when installed. Those entries can be removed later when we explicitly want to upgrade OpenShift or Docker. More details in yum Exclusion.
yum install atomic-openshift-excluder atomic-openshift-docker-excluder
If later we need to upgrade, we must run the following command:
atomic-openshift-excluder unexclude
Reboot the system to make sure it starts correctly after package installation:
systemctl reboot
Configure the DNS client to use the DNS server that was installed as part of the procedure. See Manual /etc/resolv.conf Configuration and https://docs.openshift.com/container-platform/3.5/install_config/install/prerequisites.html#prereq-dns
Make sure SELinux is enabled on all hosts. If is not, enable SELinux and make sure SELINUXTYPE is "targeted" in /etc/selinux/config.
sestatus
Installation User
Create an installation user that can log in remotely from support.openshift35.local, which is the host that will drive installation. Conventionally we name that user "ansible" and it must be able to passwordlessly ssh into itself and all other environment nodes, and do passwordless sudo. The user will be part of the shared image.
groupadd -g 1200 ansible useradd -m -g ansible -u 1200 ansible
Configure public/private key authentication and install the public key into its own authorized_keys file.
Allow passwordless sudo to "ansible".
Firewall Configuration
Turn off firewalld and configure the iptables service.
systemctl stop firewalld systemctl disable firewalld systemctl is-enabled firewalld
OpenShift needs iptables running:
systemctl enable iptables systemctl start iptables
Docker Installation
Install Docker. Docker is technically not required on masters, but it is easier to create a uniform image and only disable docker on masters. The binaries must be installed from the rhel-7-server-ose-3.*-rpms repository and have it running before installing OpenShift.
OpenShift 3.5 requires Docker 1.12.
yum install docker docker version
The advanced installation procedure will update /etc/sysconfig/docker on nodes with OpenShift-specific configuration.
Provision storage for the Docker server. The default loopback storage is not appropriate for production, it should be replaced by a thin-pool logical volume. Follow https://docs.openshift.com/container-platform/3.5/install_config/install/host_preparation.html#configuring-docker-storage. Used Option A) "an additional block device". On VirtualBox, provision a new virtual disk of appropriate size and configure it as Docker storage backend.
The procedure consists in executing /usr/bin/docker-storage-setup with the base configuration read from /usr/lib/docker-storage-setup/docker-storage-setup and custom configuration specified in /etc/sysconfig/docker-storage-setup, similarly to:
STORAGE_DRIVER=devicemapper DEVS=/dev/sdb VG=docker_vg DATA_SIZE=500M MIN_DATA_SIZE=1M
Under some circumstances, /usr/bin/docker-storage-setup fails with:
[...] end of partition 1 has impossible value for cylinders: 65 (should be in 0-64) sfdisk: I don't like these partitions - nothing changed. (If you really want this, use the --force option.)
If that happens, follow the manual procedure of provisioning Docker storage on a dedicated block device:
After the script completes successfully, it creates a logical volume with an XFS filesystem mounted on docker root directory /var/lib/docker and the Docker storage configuration file /etc/sysconfig/docker-storage. The thin pool to be used by Docker should be visible in lvs:
# lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert docker-pool docker_vg twi-a-t--- 500.00m 0.00 0.88 root main_vg -wi-ao---- 7.00g
Enable Docker at boot and start it.
systemctl enable docker systemctl start docker
Reboot the system and then check Docker Server Runtime.
Generic Docker installation instructions Docker Installation.
- Persistent storage support preparation: https://docs.openshift.com/container-platform/3.5/install_config/persistent_storage/persistent_storage_nfs.html#install-config-persistent-storage-persistent-storage-nfs
- Cloud-provider specific configuration
- https://docs.openshift.com/container-platform/3.5/install_config/configuring_aws.html#install-config-configuring-aws
- https://docs.openshift.com/container-platform/3.5/install_config/configuring_openstack.html#install-config-configuring-openstack
- https://docs.openshift.com/container-platform/3.5/install_config/configuring_gce.html#install-config-configuring-gce
- Optionally configure the number of cores used by the master and node OpenShift processes.
- Enable Network Time Protocol (NTP) on all hosts in the cluster. This can be done automatically by Ansible, see "openshift_clock_enabled".
Special Configuration
NFS Server
Run CLI to Install
Post-Install
Deploy the Integrated Docker Registry
Deploy the HAProxy Router
Load Image Streams
Load Templates
Set up NFS
The NFS server is required for persistent volumes.
Advanced Installation
TODO
DNS Configuration
After setup, the DNS server needs to be configured to resolve a public wildcard DNS entry to the public IP address of the node that executes the default router, by adding an A record, with a low TTL. If the environment has multiple routers, an external load balancer is required.
*.myapp.example.com. 300 IN A 1.2.3.4
Post-Installation
- Verify that /etc/sysconfig/docker on all nodes contains --selinux-enabled and --insecure-registry 172.30.0.0/16
Most of the ports will be configured in iptables by the installer, automatically - verify that.