OpenShift on Azure: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
No edit summary
 
(81 intermediate revisions by the same user not shown)
Line 1: Line 1:
=External=
=External=
* https://docs.microsoft.com/en-us/azure/openshift/?WT.mc_id=Portal-Microsoft_Azure_RedHatOpenShift
* https://docs.openshift.com/aro/4/welcome/index.html
* https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster
* https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster


=Internal=
=Internal=
* [[OpenShift#Installation|OpenShift Installation]]
* [[OpenShift#Installation|OpenShift Installation]]
* [[Azure_Compute_Concepts#OpenShift_on_Azure|Azure Compute Concepts]]
* [[Azure Operations#OpenShift_Installation_on_Azure|Azure Operations]]
* [[Azure Operations#OpenShift_Installation_on_Azure|Azure Operations]]
=Overview=
An OpenShift cluster installed on Azure has an associated [[Azure_Security_Concepts#Service_Principal|service principal]].
=CLI Support=
Azure CLI has an OpenShift extension, with subcommands aimed at managing Azure Red Hat OpenShift clusters.
<syntaxhighlight lang='bash'>
az aro create|list|delete|list-credentials|show|update|wait
</syntaxhighlight>
==List OpenShift Clusters==
<syntaxhighlight lang='bash'>
az aro list
</syntaxhighlight>
==List Details about a Specific Cluster==
<syntaxhighlight lang='bash'>
az aro show --resource-group <rg-name> --name <cluster-name>
</syntaxhighlight>


=Installation=
=Installation=
Line 11: Line 31:
* Azure Red Hat OpenShift requires a minimum of 40 cores. May need to [[Azure_Quota_Operations|increase the quota]].
* Azure Red Hat OpenShift requires a minimum of 40 cores. May need to [[Azure_Quota_Operations|increase the quota]].
==Procedure==
==Procedure==
===Create the Resource Group===
The resource group can be created [[Azure_Resource_Group_Operations#Create_a_Resource_Group|from the console or from command line]]. The resource group it will encapsulate resources required by, and dedicated to the OpenShift cluster. The name of the resource group should be derived from the name of the cluster by adding the "-rg" postfix. There will be a one-to-one relationship between the resource group, cluster and ancillary resources. Select the appropriate region and set the corresponding LOCATION environment variable.
<syntaxhighlight lang='bash'>
<syntaxhighlight lang='bash'>
SUBSCRIPTION_ID=...
export CLUSTER=platform-cloud-aro-02
LOCATION=eastus2
export RESOURCEGROUP="${CLUSTER}-rg"
CLUSTER=platform-cloud
export LOCATION=eastus2
CLUSTER_NAME=${platform-cloud}-01
 
az group create --name $RESOURCEGROUP --location $LOCATION
</syntaxhighlight>
</syntaxhighlight>


===Register Resource Providers===
<syntaxhighlight lang='bash'>
<syntaxhighlight lang='bash'>
export SUBSCRIPTION_ID="..."
az account set --subscription ${SUBSCRIPTION_ID}
az account set --subscription ${SUBSCRIPTION_ID}
</syntaxhighlight>
</syntaxhighlight>


=CLI Support=
Register the Microsoft.RedHatOpenShift, Microsoft.Compute and Microsoft.Storage [[Azure_Concepts#Resource_Provider|resource providers]]:
Azure CLI has an OpenShift extension, with subcommands aimed at managing Azure Red Hat OpenShift clusters.
<syntaxhighlight lang='bash'>
az provider register -n Microsoft.RedHatOpenShift --wait
az provider register -n Microsoft.Compute --wait
az provider register -n Microsoft.Storage --wait
</syntaxhighlight>
===Get a Red Hat Pull Secret===
{{External|https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#get-a-red-hat-pull-secret-optional}}
<font color=darkgray>TODO</font>
===Prepare a Custom Domain===
{{External|https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#prepare-a-custom-domain-for-your-cluster-optional}}
<font color=darkgray>TODO</font>
 
===Create a Virtual Network and associated Subnets===
{{External|https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#create-a-virtual-network-containing-two-empty-subnets}}
Azure Red Hat OpenShift clusters require a virtual network with two empty subnets, for the master and worker nodes. The virtual network can be created as such (for more details about networking operations, see [[Azure Networking Operations]]):
<syntaxhighlight lang='bash'>
az network vnet create \
  --resource-group $RESOURCEGROUP \
  --name ${CLUSTER}-vnet \
  --address-prefixes 10.0.0.0/16
 
az network vnet subnet create \
  --resource-group $RESOURCEGROUP \
  --vnet-name ${CLUSTER}-vnet \
  --name ${CLUSTER}-master-subnet \
  --address-prefixes 10.0.0.0/17 \
  --service-endpoints Microsoft.ContainerRegistry
 
az network vnet subnet create \
  --resource-group $RESOURCEGROUP \
  --vnet-name ${CLUSTER}-vnet \
  --name ${CLUSTER}-worker-subnet \
  --address-prefixes 10.0.128.0/17 \
  --service-endpoints Microsoft.ContainerRegistry
</syntaxhighlight>
 
Disable subnet private endpoint policies on the master subnet. This is required for the service to be able to connect to and manage the cluster:
<syntaxhighlight lang='bash'>
az network vnet subnet update \
  --name ${CLUSTER}-master-subnet \
  --resource-group $RESOURCEGROUP \
  --vnet-name ${CLUSTER}-vnet \
  --disable-private-link-service-network-policies true
</syntaxhighlight>
 
===Create the Cluster===
{{External|https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#create-the-cluster}}
 
<syntaxhighlight lang='bash'>
az aro create \
  --resource-group $RESOURCEGROUP \
  --name $CLUSTER \
  --vnet ${CLUSTER}-vnet \
  --master-subnet ${CLUSTER}-master-subnet \
  --worker-subnet ${CLUSTER}-worker-subnet \
  --ingress-visibility Public \
  --worker-count 3 \
  --worker-vm-disk-size-gb 200
</syntaxhighlight>
 
It normally takes about 35 minutes to create a cluster. A successful execution produces a JSON report that gives essential information about the cluster. The same information can be obtained with '[[#List_Details_about_a_Specific_Cluster|az aro show]]'.
 
<font color=darkgray>
TODO:
* Optionally [[#Get_a_Red_Hat_Pull_Secret|pass Red Hat pull secret]] by specifying --pull-secret @pull-secret.txt
* Optionally [[#Prepare_a_Custom_Domain|use a custom domain]] by specifying --domain foo.example.com
</font>
 
===Retrieve Credentials===
 
Upon cluster creation, the cluster administrator (kubeadmin) username and password can be retrieved with:
<syntaxhighlight lang='bash'>
az aro list-credentials --resource-group ${RESOURCEGROUP} --name ${CLUSTER}
</syntaxhighlight>
 
===To Clarify===
 
<font color=darkgray>
* It seems that an OpenShift cluster installation creates an additional resource group (aro-xxxxxxxx, where xxxxxxx is the same random alphanumeric sequence that shows up in the console name). The resource group has [[Azure_Storage_Concepts#Storage_Account|storage accounts]], [[Azure_Storage_Concepts#Disk|disks]], [[Azure_Networking_Concepts#Load_Balancer|load balancers]], [[Azure_Networking_Concepts#Network_Interface|network interfaces]], [[Azure_Networking_Concepts#Network_Security_Group|network security groups]], [[Azure_Networking_Concepts#Private_DNS_Zone|private DNS zones]], [[Azure_Networking_Concepts#Private_Link_Service|private link services]] and [[Azure_Networking_Concepts#Public_IP_Address|public IP addresses]].
</font>
 
=Concepts=
==Networking Concepts==
{{External|https://docs.microsoft.com/en-us/azure/openshift/concepts-networking}}
=Operations=
==Console==
The console URL is exposed as "consoleProfile.url" in the output of '[[#List_Details_about_a_Specific_Cluster|az aro show]]' command.
==oc==
===oc Installation===
====Mac====
Go to the Console → Top Level Menu Question Mark → Command Line Tools → Download oc for Mac for x86_64.
<syntaxhighlight lang='bash'>
cd /usr/local
mkdir openshift
cd openshift
unzip .../oc.zip # this will create an "oc" executable
export PATH=/usr/local/openshift:${PATH}
</syntaxhighlight>
 
The first attempt to run will trigger an alert warning you that the oc executable is not from a known developer and cannot be verified. To get around this, navigate to /usr/local/openshift in Finder, right-click and use Open.
 
===oc login===
'oc login' should be used upon first login. It updates [[.kube_config|.kube/config]].
<syntaxhighlight lang='bash'>
<syntaxhighlight lang='bash'>
az aro create|list|delete|list-credentials|show|update|wait
apiServer=$(az aro show -g ${RESOURCEGROUP} -n ${CLUSTER} --query apiserverProfile.url -o tsv)
password=$(az aro list-credentials -g ${RESOURCEGROUP} -n ${CLUSTER} --query kubeadminPassword -o tsv)
oc login ${apiServer} -u kubeadmin -p ${password}
</syntaxhighlight>
 
Upon execution, it adds the following:
<syntaxhighlight lang='yaml'>
clusters:
  - name: <api-server-url>:<api-server-port>
    cluster:
      server: https://<api-server-url>:<api-server-port>
users:
  - name: kube:admin
    user:
      token: Rt..._Xc
contexts:
  - name: default/<api-server-url>:<api-server-port>/kube:admin
    context:
      cluster: <api-server-url>:<api-server-port>
      namespace: default
      user: kube:admin 
</syntaxhighlight>
</syntaxhighlight>
==List OpenShift Clusters==
and also updates the current context.
 
For more details on the 'oc login' command, see: {{Internal|Oc_login#oc_login_and_Azure_OpenShift|oc login}}
==Storage Operations==
* [[Provision Azure Files ReadWriteMany Persistent Volumes on Azure OpenShift]]
==Security Operations==
===Obtain the Service Principal===
To obtain the service principal associated with the OpenShift cluster:
<syntaxhighlight lang='bash'>
<syntaxhighlight lang='bash'>
az aro list
az aro show -g <openshift-cluster-resource-group> -n <cluster-name> --query servicePrincipalProfile.clientId -o tsv
</syntaxhighlight>
</syntaxhighlight>

Latest revision as of 04:38, 31 December 2021

External

Internal

Overview

An OpenShift cluster installed on Azure has an associated service principal.

CLI Support

Azure CLI has an OpenShift extension, with subcommands aimed at managing Azure Red Hat OpenShift clusters.

az aro create|list|delete|list-credentials|show|update|wait

List OpenShift Clusters

az aro list

List Details about a Specific Cluster

az aro show --resource-group <rg-name> --name <cluster-name>

Installation

Prerequisites

Procedure

Create the Resource Group

The resource group can be created from the console or from command line. The resource group it will encapsulate resources required by, and dedicated to the OpenShift cluster. The name of the resource group should be derived from the name of the cluster by adding the "-rg" postfix. There will be a one-to-one relationship between the resource group, cluster and ancillary resources. Select the appropriate region and set the corresponding LOCATION environment variable.

export CLUSTER=platform-cloud-aro-02
export RESOURCEGROUP="${CLUSTER}-rg"
export LOCATION=eastus2

az group create --name $RESOURCEGROUP --location $LOCATION

Register Resource Providers

export SUBSCRIPTION_ID="..."
az account set --subscription ${SUBSCRIPTION_ID}

Register the Microsoft.RedHatOpenShift, Microsoft.Compute and Microsoft.Storage resource providers:

az provider register -n Microsoft.RedHatOpenShift --wait
az provider register -n Microsoft.Compute --wait
az provider register -n Microsoft.Storage --wait

Get a Red Hat Pull Secret

https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#get-a-red-hat-pull-secret-optional

TODO

Prepare a Custom Domain

https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#prepare-a-custom-domain-for-your-cluster-optional

TODO

Create a Virtual Network and associated Subnets

https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#create-a-virtual-network-containing-two-empty-subnets

Azure Red Hat OpenShift clusters require a virtual network with two empty subnets, for the master and worker nodes. The virtual network can be created as such (for more details about networking operations, see Azure Networking Operations):

az network vnet create \
  --resource-group $RESOURCEGROUP \
  --name ${CLUSTER}-vnet \
  --address-prefixes 10.0.0.0/16

az network vnet subnet create \
  --resource-group $RESOURCEGROUP \
  --vnet-name ${CLUSTER}-vnet \
  --name ${CLUSTER}-master-subnet \
  --address-prefixes 10.0.0.0/17 \
  --service-endpoints Microsoft.ContainerRegistry

az network vnet subnet create \
  --resource-group $RESOURCEGROUP \
  --vnet-name ${CLUSTER}-vnet \
  --name ${CLUSTER}-worker-subnet \
  --address-prefixes 10.0.128.0/17 \
  --service-endpoints Microsoft.ContainerRegistry

Disable subnet private endpoint policies on the master subnet. This is required for the service to be able to connect to and manage the cluster:

az network vnet subnet update \
  --name ${CLUSTER}-master-subnet \
  --resource-group $RESOURCEGROUP \
  --vnet-name ${CLUSTER}-vnet \
  --disable-private-link-service-network-policies true

Create the Cluster

https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#create-the-cluster
az aro create \
  --resource-group $RESOURCEGROUP \
  --name $CLUSTER \
  --vnet ${CLUSTER}-vnet \
  --master-subnet ${CLUSTER}-master-subnet \
  --worker-subnet ${CLUSTER}-worker-subnet \
  --ingress-visibility Public \
  --worker-count 3 \
  --worker-vm-disk-size-gb 200

It normally takes about 35 minutes to create a cluster. A successful execution produces a JSON report that gives essential information about the cluster. The same information can be obtained with 'az aro show'.

TODO:

Retrieve Credentials

Upon cluster creation, the cluster administrator (kubeadmin) username and password can be retrieved with:

az aro list-credentials --resource-group ${RESOURCEGROUP} --name ${CLUSTER}

To Clarify

Concepts

Networking Concepts

https://docs.microsoft.com/en-us/azure/openshift/concepts-networking

Operations

Console

The console URL is exposed as "consoleProfile.url" in the output of 'az aro show' command.

oc

oc Installation

Mac

Go to the Console → Top Level Menu Question Mark → Command Line Tools → Download oc for Mac for x86_64.

cd /usr/local
mkdir openshift
cd openshift
unzip .../oc.zip # this will create an "oc" executable
export PATH=/usr/local/openshift:${PATH}

The first attempt to run will trigger an alert warning you that the oc executable is not from a known developer and cannot be verified. To get around this, navigate to /usr/local/openshift in Finder, right-click and use Open.

oc login

'oc login' should be used upon first login. It updates .kube/config.

apiServer=$(az aro show -g ${RESOURCEGROUP} -n ${CLUSTER} --query apiserverProfile.url -o tsv)
password=$(az aro list-credentials -g ${RESOURCEGROUP} -n ${CLUSTER} --query kubeadminPassword -o tsv)
oc login ${apiServer} -u kubeadmin -p ${password}

Upon execution, it adds the following:

clusters:
  - name: <api-server-url>:<api-server-port>
    cluster:
      server: https://<api-server-url>:<api-server-port>
users:
  - name: kube:admin
    user:
      token: Rt..._Xc
contexts:
  - name: default/<api-server-url>:<api-server-port>/kube:admin
    context:
      cluster: <api-server-url>:<api-server-port>
      namespace: default
      user: kube:admin

and also updates the current context.

For more details on the 'oc login' command, see:

oc login

Storage Operations

Security Operations

Obtain the Service Principal

To obtain the service principal associated with the OpenShift cluster:

az aro show -g <openshift-cluster-resource-group> -n <cluster-name> --query servicePrincipalProfile.clientId -o tsv