OpenShift on Azure
External
- https://docs.microsoft.com/en-us/azure/openshift/?WT.mc_id=Portal-Microsoft_Azure_RedHatOpenShift
- https://docs.openshift.com/aro/4/welcome/index.html
- https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster
Internal
Concepts
CLI Support
Azure CLI has an OpenShift extension, with subcommands aimed at managing Azure Red Hat OpenShift clusters.
az aro create|list|delete|list-credentials|show|update|wait
List OpenShift Clusters
az aro list
List Details about a Specific Cluster
az aro show --resource-group <rg-name> --name <cluster-name>
Installation
Prerequisites
- Install Azure CLI, as shown here: Azure CLI | Installation.
- Azure Red Hat OpenShift requires a minimum of 40 cores. May need to increase the quota.
Procedure
Create the Resource Group
The resource group can be created from the console or from command line. The resource group it will encapsulate resources required by, and dedicated to the OpenShift cluster. The name of the resource group should be derived from the name of the cluster by adding the "-rg" postfix. There will be a one-to-one relationship between the resource group, cluster and ancillary resources. Select the appropriate region and set the corresponding LOCATION environment variable.
export CLUSTER=platform-cloud-aro-02
export RESOURCEGROUP="${CLUSTER}-rg"
export LOCATION=eastus2
az group create --name $RESOURCEGROUP --location $LOCATION
Register Resource Providers
export SUBSCRIPTION_ID="..."
az account set --subscription ${SUBSCRIPTION_ID}
Register the Microsoft.RedHatOpenShift, Microsoft.Compute and Microsoft.Storage resource providers:
az provider register -n Microsoft.RedHatOpenShift --wait
az provider register -n Microsoft.Compute --wait
az provider register -n Microsoft.Storage --wait
Get a Red Hat Pull Secret
TODO
Prepare a Custom Domain
TODO
Create a Virtual Network and associated Subnets
Azure Red Hat OpenShift clusters require a virtual network with two empty subnets, for the master and worker nodes. The virtual network can be created as such (for more details about networking operations, see Azure Networking Operations):
az network vnet create \
--resource-group $RESOURCEGROUP \
--name ${CLUSTER}-vnet \
--address-prefixes 10.0.0.0/16
az network vnet subnet create \
--resource-group $RESOURCEGROUP \
--vnet-name ${CLUSTER}-vnet \
--name ${CLUSTER}-master-subnet \
--address-prefixes 10.0.0.0/17 \
--service-endpoints Microsoft.ContainerRegistry
az network vnet subnet create \
--resource-group $RESOURCEGROUP \
--vnet-name ${CLUSTER}-vnet \
--name ${CLUSTER}-worker-subnet \
--address-prefixes 10.0.128.0/17 \
--service-endpoints Microsoft.ContainerRegistry
Disable subnet private endpoint policies on the master subnet. This is required for the service to be able to connect to and manage the cluster:
az network vnet subnet update \
--name ${CLUSTER}-master-subnet \
--resource-group $RESOURCEGROUP \
--vnet-name ${CLUSTER}-vnet \
--disable-private-link-service-network-policies true
Create the Cluster
az aro create \
--resource-group $RESOURCEGROUP \
--name $CLUSTER \
--vnet ${CLUSTER}-vnet \
--master-subnet ${CLUSTER}-master-subnet \
--worker-subnet ${CLUSTER}-worker-subnet \
--ingress-visibility Public \
--worker-count 3 \
--worker-vm-disk-size-gb 200
It normally takes about 35 minutes to create a cluster. A successful execution produces a JSON report that gives essential information about the cluster. The same information can be obtained with 'az aro show'.
TODO:
- Optionally pass Red Hat pull secret by specifying --pull-secret @pull-secret.txt
- Optionally use a custom domain by specifying --domain foo.example.com
Retrieve Credentials
Upon cluster creation, the cluster administrator (kubeadmin) username and password can be retrieved with:
az aro list-credentials --resource-group ${RESOURCEGROUP} --name ${CLUSTER}
To Clarify
- It seems that an OpenShift cluster installation creates an additional resource group (aro-xxxxxxxx, where xxxxxxx is the same random alphanumeric sequence that shows up in the console name). The resource group has storage accounts, disks, load balancers, network interfaces, network security groups, private DNS zones, private link services and public IP addresses.
Operations
Console
The console URL is exposed as "consoleProfile.url" in the output of 'az aro show' command.
oc
oc Installation
Mac
Go to the Console → Top Level Menu Question Mark → Command Line Tools → Download oc for Mac for x86_64.
cd /usr/local
mkdir openshift
cd openshift
unzip .../oc.zip # this will create an "oc" executable
export PATH=/usr/local/openshift:${PATH}
The first attempt to run will trigger an alert warning you that the oc executable is not from a known developer and cannot be verified. To get around this, navigate to /usr/local/openshift in Finder, right-click and use Open.
oc login
'oc login' should be used upon first login. It updates .kube/config.
apiServer=$(az aro show -g ${RESOURCEGROUP} -n ${CLUSTER} --query apiserverProfile.url -o tsv)
password=$(az aro list-credentials -g ${RESOURCEGROUP} -n ${CLUSTER} --query kubeadminPassword -o tsv)
oc login ${apiServer} -u kubeadmin -p ${password}
Upon execution, it adds the following:
clusters:
- name: <api-server-url>:<api-server-port>
cluster:
server: https://<api-server-url>:<api-server-port>
users:
- name: kube:admin
user:
token: Rt..._Xc
contexts:
- name: default/<api-server-url>:<api-server-port>/kube:admin
context:
cluster: <api-server-url>:<api-server-port>
namespace: default
user: kube:admin
and also updates the current context.
For more details on the 'oc login' command, see: