OpenShift on Azure

External

• https://docs.microsoft.com/en-us/azure/openshift/?WT.mc_id=Portal-Microsoft_Azure_RedHatOpenShift
• https://docs.openshift.com/aro/4/welcome/index.html
• https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster

Internal

• OpenShift Installation
• Azure Operations

Overview

An OpenShift cluster installed on Azure has an associated service principal.

CLI Support

Azure CLI has an OpenShift extension, with subcommands aimed at managing Azure Red Hat OpenShift clusters.

az aro create|list|delete|list-credentials|show|update|wait

List OpenShift Clusters

az aro list

List Details about a Specific Cluster

az aro show --resource-group <rg-name> --name <cluster-name>

Installation

Prerequisites

• Install Azure CLI, as shown here: Azure CLI | Installation.
• Azure Red Hat OpenShift requires a minimum of 40 cores. May need to increase the quota.

Procedure

Create the Resource Group

The resource group can be created from the console or from command line. The resource group it will encapsulate resources required by, and dedicated to the OpenShift cluster. The name of the resource group should be derived from the name of the cluster by adding the "-rg" postfix. There will be a one-to-one relationship between the resource group, cluster and ancillary resources. Select the appropriate region and set the corresponding LOCATION environment variable.

export CLUSTER=platform-cloud-aro-02
export RESOURCEGROUP="${CLUSTER}-rg"
export LOCATION=eastus2

az group create --name $RESOURCEGROUP --location $LOCATION

Register Resource Providers

export SUBSCRIPTION_ID="..."
az account set --subscription ${SUBSCRIPTION_ID}

Register the Microsoft.RedHatOpenShift, Microsoft.Compute and Microsoft.Storage resource providers:

az provider register -n Microsoft.RedHatOpenShift --wait
az provider register -n Microsoft.Compute --wait
az provider register -n Microsoft.Storage --wait

Get a Red Hat Pull Secret

https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#get-a-red-hat-pull-secret-optional

TODO

Prepare a Custom Domain

https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#prepare-a-custom-domain-for-your-cluster-optional

TODO

Create a Virtual Network and associated Subnets

https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#create-a-virtual-network-containing-two-empty-subnets

Azure Red Hat OpenShift clusters require a virtual network with two empty subnets, for the master and worker nodes. The virtual network can be created as such (for more details about networking operations, see Azure Networking Operations):

az network vnet create \
  --resource-group $RESOURCEGROUP \
  --name ${CLUSTER}-vnet \
  --address-prefixes 10.0.0.0/16

az network vnet subnet create \
  --resource-group $RESOURCEGROUP \
  --vnet-name ${CLUSTER}-vnet \
  --name ${CLUSTER}-master-subnet \
  --address-prefixes 10.0.0.0/17 \
  --service-endpoints Microsoft.ContainerRegistry

az network vnet subnet create \
  --resource-group $RESOURCEGROUP \
  --vnet-name ${CLUSTER}-vnet \
  --name ${CLUSTER}-worker-subnet \
  --address-prefixes 10.0.128.0/17 \
  --service-endpoints Microsoft.ContainerRegistry

Disable subnet private endpoint policies on the master subnet. This is required for the service to be able to connect to and manage the cluster:

az network vnet subnet update \
  --name ${CLUSTER}-master-subnet \
  --resource-group $RESOURCEGROUP \
  --vnet-name ${CLUSTER}-vnet \
  --disable-private-link-service-network-policies true

Create the Cluster

https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#create-the-cluster

az aro create \
  --resource-group $RESOURCEGROUP \
  --name $CLUSTER \
  --vnet ${CLUSTER}-vnet \
  --master-subnet ${CLUSTER}-master-subnet \
  --worker-subnet ${CLUSTER}-worker-subnet \
  --ingress-visibility Public \
  --worker-count 3 \
  --worker-vm-disk-size-gb 200

It normally takes about 35 minutes to create a cluster. A successful execution produces a JSON report that gives essential information about the cluster. The same information can be obtained with 'az aro show'.

TODO:

• Optionally pass Red Hat pull secret by specifying --pull-secret @pull-secret.txt
• Optionally use a custom domain by specifying --domain foo.example.com

Retrieve Credentials

Upon cluster creation, the cluster administrator (kubeadmin) username and password can be retrieved with:

az aro list-credentials --resource-group ${RESOURCEGROUP} --name ${CLUSTER}

To Clarify

• It seems that an OpenShift cluster installation creates an additional resource group (aro-xxxxxxxx, where xxxxxxx is the same random alphanumeric sequence that shows up in the console name). The resource group has storage accounts, disks, load balancers, network interfaces, network security groups, private DNS zones, private link services and public IP addresses.

Concepts

Networking Concepts

https://docs.microsoft.com/en-us/azure/openshift/concepts-networking

Operations

Console

The console URL is exposed as "consoleProfile.url" in the output of 'az aro show' command.

oc

oc Installation

Mac

Go to the Console → Top Level Menu Question Mark → Command Line Tools → Download oc for Mac for x86_64.

cd /usr/local
mkdir openshift
cd openshift
unzip .../oc.zip # this will create an "oc" executable
export PATH=/usr/local/openshift:${PATH}

The first attempt to run will trigger an alert warning you that the oc executable is not from a known developer and cannot be verified. To get around this, navigate to /usr/local/openshift in Finder, right-click and use Open.

oc login

'oc login' should be used upon first login. It updates .kube/config.

apiServer=$(az aro show -g ${RESOURCEGROUP} -n ${CLUSTER} --query apiserverProfile.url -o tsv)
password=$(az aro list-credentials -g ${RESOURCEGROUP} -n ${CLUSTER} --query kubeadminPassword -o tsv)
oc login ${apiServer} -u kubeadmin -p ${password}

Upon execution, it adds the following:

clusters:
  - name: <api-server-url>:<api-server-port>
    cluster:
      server: https://<api-server-url>:<api-server-port>
users:
  - name: kube:admin
    user:
      token: Rt..._Xc
contexts:
  - name: default/<api-server-url>:<api-server-port>/kube:admin
    context:
      cluster: <api-server-url>:<api-server-port>
      namespace: default
      user: kube:admin

and also updates the current context.

For more details on the 'oc login' command, see:

oc login

Storage Operations

• Provision Azure Files ReadWriteMany Persistent Volumes on Azure OpenShift

Security Operations

Obtain the Service Principal

To obtain the service principal associated with the OpenShift cluster:

az aro show -g <openshift-cluster-resource-group> -n <cluster-name> --query servicePrincipalProfile.clientId -o tsv