OpenShift 3.6 Installation

From NovaOrdis Knowledge Base
Jump to navigation Jump to search

External

Internal

Overview

This document covers OpenShit Container Platform 3.6 advanced installation procedure. The procedure it is largely based on https://docs.openshift.com/container-platform/3.6/install_config/index.html and contains additional details observed during an actual installation process on KVM guests.

Location

noper430:/root/environments/ocp36
RackStation:/base/NovaOrdis/Archive/yyyy.mm.dd-ocp36

Hardware Requirements

https://docs.openshift.com/container-platform/3.6/install_config/install/prerequisites.html#hardware

System and Environment Prerequisites

The pre-requisites and all preparatory steps presented in the following link are also addressed in this procedure. The links are provided for reference only:

https://docs.openshift.com/container-platform/3.6/install_config/install/prerequisites.html
https://docs.openshift.com/container-platform/3.6/install_config/install/host_preparation.html

Virtualization Host Preparation

Guest Template Preparation

Create VM templates, in this order:

Cloud Provider-Specific Configuration

Guest Configuration

External DNS Setup

An external DNS server is required. Controlling a registrar (such as http://godaddy.com) DNS zone will provide all capabilities needed to set up external DNS address resolution. A valid alternative, though more complex, is to install a dedicated public DNS server. The DNS server must be available to all OpenShift environment nodes, and also to external clients than need to resolve public names such as the master public web console and API URL, the public wildcard name of the environment, the application router public DNS name, etc. A combination of a public DNS server that resolves public addresses and an internal DNS server, deployed as part of the environment, usually on the support node, tasked with resolving internal addresses is also a workable solution. This installation procedure includes installation of the bind DNS server on the support node - see bind DNS server installation section below.

Wildcard Domain for Application Traffic

The DNS server that resolves the public addresses will need to be capable to support wildcard sub-domains and resolve the public wildcard DNS entry to the public IP address of the node that executes the default router, or of the public ingress node that proxies to the default router. If the environment has multiple routers, an external load balancer is required, and the wildcard record must contain the public IP address of the host that runs the load balancer. The name of the wildcard domain will be later specified during the advanced installation procedure in the Ansible inventory file as openshift_master_default_subdomain.

Configuration procedures:

OpenShift Advanced Installation

https://docs.openshift.com/container-platform/3.6/install_config/install/advanced_install.html#install-config-install-advanced-install

The Support Node

Execute the installation as "ansible" from the support node.

As "root" on "support":

cd /usr/share/ansible
chgrp -R ansible /usr/share/ansible
chmod -R g+w /usr/share/ansible
chgrp ansible /etc/ansible/
chmod g+w /etc/ansible/

The support node needs at least 1 GB or RAM to run the installation process.

Create an ansible key pair on "support" and disseminate it to all other nodes. As "ansible":

cd
ssh-keygen -q -b 2048 -f ~/.ssh/id_rsa -t rsa

Configure Ansible Inventory File

OpenShift Ansible hosts Inventory File

Pre-Flight

On the support node, as "root":

cd /tmp
rm -r tmp* yum*
rm /usr/share/ansible/openshift-ansible/playbooks/byo/config.retry

As "ansible":

cd /etc/ansible
ansible all -m ping
ansible nodes -m shell -a "systemctl status docker"
ansible nodes -m shell -a "docker version"
ansible nodes -m shell -a "pvs"
ansible nodes -m shell -a "vgs"
ansible nodes -m shell -a "lvs"
ansible nodes -m shell -a "nslookup something.apps.openshift.novaordis.io"
ansible nodes -m shell -a "mkdir /mnt/tmp"
ansible nodes -m shell -a "mount -t nfs support.ocp36.local:/nfs /mnt/tmp"
ansible nodes -m shell -a "hostname > /mnt/tmp/\$(hostname).txt"
ansible nodes -m shell -a "umount /mnt/tmp"
for i in /nfs/*.ocp36.local.txt; do echo $i; cat $i; done

Snapshot the Guest Images for the Entire Environment

On the virtualization host:

virsh vol-list main-storage-pool --details
./stop-ose36
virsh list --all
qemu-img snapshot
for i in \
 /main-storage-pool/ocp36.ingress.qcow2 \
 /main-storage-pool/ocp36.support.qcow2 \
 /main-storage-pool/ocp36.master.qcow2 \
 /main-storage-pool/ocp36.infranode.qcow2 \
 /main-storage-pool/ocp36.node1.qcow2 \
 /main-storage-pool/ocp36.node2.qcow2; \
 do \
   echo "$i snapshots:"; \
   qemu-img snapshot -l $i; \
 done
for i in \
 /main-storage-pool/ocp36.ingress.qcow2 \
 /main-storage-pool/ocp36.support.qcow2 \
 /main-storage-pool/ocp36.master.qcow2 \
 /main-storage-pool/ocp36.infranode.qcow2 \
 /main-storage-pool/ocp36.node1.qcow2 \
 /main-storage-pool/ocp36.node2.qcow2; \
 do \
   echo "taking snapshot of $i"; \
   qemu-img snapshot -c before_ansible_installation $i; \
 done
./start-ose36

Run the Advanced Installation

May want to run pre-flight one more time after reboot.

As "ansible" on support:

ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/byo/config.yml

For verbose installation use -vvv or -vvvv.

To use a different inventory file than /etc/ansible/hosts, run:

ansible-playbook -vvv -i /custom/path/to/inventory/file /usr/share/ansible/openshift-ansible/playbooks/byo/config.yml

Output of a Successful Run

PLAY RECAP *********************************************************************************************************************************************************************************************************************************************************************************************************************************
api-lb.ocp36.local         : ok=97   changed=2    unreachable=0    failed=0
infranode.ocp36.local      : ok=257  changed=58   unreachable=0    failed=0
localhost                  : ok=14   changed=0    unreachable=0    failed=0
master.ocp36.local         : ok=1039 changed=290  unreachable=0    failed=0
node1.ocp36.local          : ok=256  changed=58   unreachable=0    failed=0
node2.ocp36.local          : ok=256  changed=58   unreachable=0    failed=0
support.ocp36.local        : ok=100  changed=3    unreachable=0    failed=0

Uninstallation

In case the installation procedure runs into problems, troubleshoot and before re-starting the installation procedure, uninstall:

ansible-playbook [-i /custom/path/to/inventory/file] /usr/share/ansible/openshift-ansible/playbooks/adhoc/uninstall.yml

This a relative quick method to iterate over the installation configuration and come up with a stable configuration. However, in order to install a production deployment, you must start from a clean operating system installation. If you are using virtual machines, start from a fresh image. If you are using bare metal machines, run the following on all hosts:

yum -y remove openshift openshift-* etcd docker docker-common
rm -rf /etc/origin /var/lib/openshift /etc/etcd \
   /var/lib/etcd /etc/sysconfig/atomic-openshift* /etc/sysconfig/docker* \
   /root/.kube/config /etc/ansible/facts.d /usr/share/openshift

Installation Validation

OpenShift Installation Validation

Deplete


OSC3.6TODEPLETE