OpenShift 3.6 Generic Guest Template: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
Line 80: Line 80:


{{External|https://docs.openshift.com/container-platform/3.6/install_config/install/host_preparation.html#installing-docker}}
{{External|https://docs.openshift.com/container-platform/3.6/install_config/install/host_preparation.html#installing-docker}}
Install Docker on the template. On a small number of guests, such as the proxies and the support host, it will simply not be activated. Docker is also technically not required on masters, it seems the installation will break if not available (more to comment here later). The binaries must be installed from the rhel-7-server-ose-3.*-rpms repository and have it running before installing OpenShift.
OpenShift 3.5 requires Docker 1.12.
<pre>
yum install docker
docker version
</pre>
The advanced installation procedure is supposed to update [[/etc/sysconfig/docker]] on nodes with OpenShift-specific configuration. The documentation says that the advanced installation procedure will add an "--insecure-registry" option, but that does not seem to be the case, so we add it manually in /etc/sysconfig/docker:
<pre>
INSECURE_REGISTRY='--insecure-registry 172.30.0.0/16'
</pre>
The subnet value used to configure the insecure registry corresponds to the default value of the [[OpenShift_Concepts#The_Services_Subnet|services subnet]].
Provision [[OpenShift_Concepts#Docker_Storage_in_OpenShift|storage for the Docker server]]. The [[Docker_Concepts#Loopback_Storage|default loopback storage]] is not appropriate for production, it should be replaced by a [[Linux_Logical_Volume_Management_Concepts#Thinly-Provisioned_Logical_Volumes_.28Thin_Volumes.29|thin-pool logical volume]]. Follow https://docs.openshift.com/container-platform/3.5/install_config/install/host_preparation.html#configuring-docker-storage. Used Option A) "an additional block device". On VirtualBox or KVM, provision a new virtual disk and install it. At this stage, the size it is not important, as it will replaced with the actual storage when the nodes are built. Use 100 MB for the template.
* [[VirtualBox_Operations#Creating_and_Installing_a_new_Virtual_Disk|Creating and installing a new virtual disk on VirtualBox]]
* [[Virsh_vol-create-as|Creating a new logical volume]] on KVM, followed by attachment to the template. When creating the logical volume, name it "template-docker.storage", following the [[Linux_Virtualization_Naming_Conventions#Storage_Volume_Naming_Convention|storage volume naming conventions]].
KVM example (the template VM must be shut down prior to attaching the storage):
  virsh vol-create-as --pool main-storage-pool --name template-docker.raw --capacity 1024M
  virsh vol-list --pool main-storage-pool
  virsh attach-disk template /main-storage-pool/template-docker.raw vdb --config
Restart the template VM, the new storage should be available as /dev/vdb.
Then execute /usr/bin/docker-storage-setup with the base configuration read from  [[/usr/lib/docker-storage-setup/docker-storage-setup]] and custom configuration specified in /etc/sysconfig/docker-storage-setup, similarly to:
STORAGE_DRIVER=devicemapper
DEVS=/dev/vdb
VG=docker_vg
# set to a little bit less than maximum amount of space available
DATA_SIZE=<b>1023M</b>
MIN_DATA_SIZE=1M
{{Warn|Setting DATA_SIZE too small caused nodes not being able to start and OpenShift [[OpenShift Concepts#OutOfDisk|OutOfDisk events]].}}
Execute:
  /usr/bin/docker-storage-setup
Under some circumstances, /usr/bin/docker-storage-setup fails with:
<pre>
[...]
end of partition 1 has impossible value for cylinders: 65 (should be in 0-64)
sfdisk: I don't like these partitions - nothing changed.
(If you really want this, use the --force option.)
</pre>
If this happens, use the patched docker-storage-setup available here: https://github.com/NovaOrdis/playground/blob/master/openshift/3.5/patches/node/usr/bin/docker-storage-setup
Before running it, remove any logical volume, volume group and physical volume that may have been created and leave an empty /dev/vdb1 partition. Then run
  /usr/bin/docker-storage-setup --force
After the script completes successfully, it creates a logical volume with an XFS filesystem mounted on docker root directory /var/lib/docker and the Docker storage configuration file /etc/sysconfig/docker-storage. The thin pool to be used by Docker should be visible in [[lvs]]:
<pre>
# lvs
  LV          VG        Attr      LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  docker-pool docker_vg twi-a-t--- 500.00m            0.00  0.88
  root        main_vg  -wi-ao----  7.00g
</pre>
<span id='Ny62gV'></span>Alternatively, you can follow the manual procedure of provisioning Docker storage on a dedicated block device:
{{Internal|Provision Docker Storage on a Dedicated Block Device|Provision Docker Storage on a Dedicated Block Device}}
Disable docker-storage-setup, is not needed, storage already setup.
  systemctl disable docker-storage-setup
  systemctl is-enabled docker-storage-setup
Enable Docker at boot and start it.
  systemctl enable docker
  systemctl start docker
 
  systemctl status docker
Reboot the system and then check [[Docker Server Runtime]].
<font color=red>TODO: parse and NOKB this: https://docs.openshift.com/container-platform/3.5/scaling_performance/optimizing_storage.html#optimizing-storage</font>
Generic Docker installation instructions [[Docker_Installation#Prerequisites|Docker Installation]].

Revision as of 04:16, 8 November 2017

Internal

Clone the Basic Template

Use the basic template as a base image, and clone it, as described here:

Cloning a Guest

The particular sequence follows: 

virsh shutdown ocp36.basic-template

virsh dumpxml ocp36.basic-template > /root/environments/ocp36/ocp36.basic-template.xml

cp ocp36.basic-template.xml ocp36.generic-template.xml

Adjust the configuration as described here.

Clone the virtual machine image and create the new disk volumes, as described in Cloning a Guest. 

virsh define /root/environments/ocp36/ocp36.generic-template.xml

virsh start --console ocp36.generic-template

Then reconfigure Linux VM Guest Image.

Install OpenShift Binaries

Attach the Node to the OpenShift Subscription

https://docs.openshift.com/container-platform/3.6/install_config/install/host_preparation.html#host-registration

More details about subscription management:

RedHat Subscription Manager

The node, being based on the basic template, should have been already registered with the subscription manager: 

subscription-manager status

subscription-manager identity

subscription-manager list --available --matches '*OpenShift*'

Identify the OpenShift pool ID: 

subscription-manager attach --pool=<pool-id>

Manage Repositories and Install Utilities

Once attached to the pool, all repositories must be first disabled, and then only select repositories are re-enabled: 

subscription-manager repos --disable="*"

subscription-manager repos --list-enabled
yum repolist

If there are any enabled repositories, disable them with: 

yum-config-manager --disable <repo_id>

Enable only the repositories required by OpenShift Container Platform 3.6: 

subscription-manager repos --enable="rhel-7-server-rpms" --enable="rhel-7-server-extras-rpms" --enable="rhel-7-server-ose-3.6-rpms" --enable="rhel-7-fast-datapath-rpms"

subscription-manager repos --list-enabled
yum repolist

yum update -y

Install atomic-openshift-utils, which contains OpenShift installer utilities and other tools required by the installation process: 

yum install atomic-openshift-utils

At this point, no OpenShift binaries, except installation utilities, are installed. The advanced installer knows how to override this and it will install the binaries as expected, without any further intervention.

Install Docker

https://docs.openshift.com/container-platform/3.6/install_config/install/host_preparation.html#installing-docker







Install Docker on the template. On a small number of guests, such as the proxies and the support host, it will simply not be activated. Docker is also technically not required on masters, it seems the installation will break if not available (more to comment here later). The binaries must be installed from the rhel-7-server-ose-3.*-rpms repository and have it running before installing OpenShift.

OpenShift 3.5 requires Docker 1.12. 

yum install docker
docker version

The advanced installation procedure is supposed to update /etc/sysconfig/docker on nodes with OpenShift-specific configuration. The documentation says that the advanced installation procedure will add an "--insecure-registry" option, but that does not seem to be the case, so we add it manually in /etc/sysconfig/docker: 

INSECURE_REGISTRY='--insecure-registry 172.30.0.0/16'

The subnet value used to configure the insecure registry corresponds to the default value of the services subnet.

Provision storage for the Docker server. The default loopback storage is not appropriate for production, it should be replaced by a thin-pool logical volume. Follow https://docs.openshift.com/container-platform/3.5/install_config/install/host_preparation.html#configuring-docker-storage. Used Option A) "an additional block device". On VirtualBox or KVM, provision a new virtual disk and install it. At this stage, the size it is not important, as it will replaced with the actual storage when the nodes are built. Use 100 MB for the template.

KVM example (the template VM must be shut down prior to attaching the storage): 

 virsh vol-create-as --pool main-storage-pool --name template-docker.raw --capacity 1024M
 virsh vol-list --pool main-storage-pool
 virsh attach-disk template /main-storage-pool/template-docker.raw vdb --config

Restart the template VM, the new storage should be available as /dev/vdb.

Then execute /usr/bin/docker-storage-setup with the base configuration read from /usr/lib/docker-storage-setup/docker-storage-setup and custom configuration specified in /etc/sysconfig/docker-storage-setup, similarly to: 

STORAGE_DRIVER=devicemapper
DEVS=/dev/vdb
VG=docker_vg
# set to a little bit less than maximum amount of space available
DATA_SIZE=1023M
MIN_DATA_SIZE=1M

Setting DATA_SIZE too small caused nodes not being able to start and OpenShift OutOfDisk events.

Execute: 

 /usr/bin/docker-storage-setup

Under some circumstances, /usr/bin/docker-storage-setup fails with: 

[...]
end of partition 1 has impossible value for cylinders: 65 (should be in 0-64)
sfdisk: I don't like these partitions - nothing changed.
(If you really want this, use the --force option.)

If this happens, use the patched docker-storage-setup available here: https://github.com/NovaOrdis/playground/blob/master/openshift/3.5/patches/node/usr/bin/docker-storage-setup

Before running it, remove any logical volume, volume group and physical volume that may have been created and leave an empty /dev/vdb1 partition. Then run 

 /usr/bin/docker-storage-setup --force

After the script completes successfully, it creates a logical volume with an XFS filesystem mounted on docker root directory /var/lib/docker and the Docker storage configuration file /etc/sysconfig/docker-storage. The thin pool to be used by Docker should be visible in lvs: 

# lvs

  LV          VG        Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  docker-pool docker_vg twi-a-t--- 500.00m             0.00   0.88
  root        main_vg   -wi-ao----   7.00g

Alternatively, you can follow the manual procedure of provisioning Docker storage on a dedicated block device:

Provision Docker Storage on a Dedicated Block Device

Disable docker-storage-setup, is not needed, storage already setup. 

 systemctl disable docker-storage-setup
 systemctl is-enabled docker-storage-setup

Enable Docker at boot and start it. 

 systemctl enable docker
 systemctl start docker
 
 systemctl status docker

Reboot the system and then check Docker Server Runtime.

TODO: parse and NOKB this: https://docs.openshift.com/container-platform/3.5/scaling_performance/optimizing_storage.html#optimizing-storage

Generic Docker installation instructions Docker Installation.

Retrieved from "https://kb.novaordis.com/index.php?title=OpenShift_3.6_Generic_Guest_Template&oldid=30593"