OpenShift on Azure: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
No edit summary
 
(40 intermediate revisions by the same user not shown)
Line 1: Line 1:
=External=
=External=
* https://docs.microsoft.com/en-us/azure/openshift/?WT.mc_id=Portal-Microsoft_Azure_RedHatOpenShift
* https://docs.openshift.com/aro/4/welcome/index.html
* https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster
* https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster


=Internal=
=Internal=
* [[OpenShift#Installation|OpenShift Installation]]
* [[OpenShift#Installation|OpenShift Installation]]
* [[Azure_Compute_Concepts#OpenShift_on_Azure|Azure Compute Concepts]]
* [[Azure Operations#OpenShift_Installation_on_Azure|Azure Operations]]
* [[Azure Operations#OpenShift_Installation_on_Azure|Azure Operations]]
=Overview=
An OpenShift cluster installed on Azure has an associated [[Azure_Security_Concepts#Service_Principal|service principal]].
=CLI Support=
Azure CLI has an OpenShift extension, with subcommands aimed at managing Azure Red Hat OpenShift clusters.
<syntaxhighlight lang='bash'>
az aro create|list|delete|list-credentials|show|update|wait
</syntaxhighlight>
==List OpenShift Clusters==
<syntaxhighlight lang='bash'>
az aro list
</syntaxhighlight>
==List Details about a Specific Cluster==
<syntaxhighlight lang='bash'>
az aro show --resource-group <rg-name> --name <cluster-name>
</syntaxhighlight>


=Installation=
=Installation=
Line 14: Line 34:
The resource group can be created [[Azure_Resource_Group_Operations#Create_a_Resource_Group|from the console or from command line]]. The resource group it will encapsulate resources required by, and dedicated to the OpenShift cluster. The name of the resource group should be derived from the name of the cluster by adding the "-rg" postfix. There will be a one-to-one relationship between the resource group, cluster and ancillary resources. Select the appropriate region and set the corresponding LOCATION environment variable.
The resource group can be created [[Azure_Resource_Group_Operations#Create_a_Resource_Group|from the console or from command line]]. The resource group it will encapsulate resources required by, and dedicated to the OpenShift cluster. The name of the resource group should be derived from the name of the cluster by adding the "-rg" postfix. There will be a one-to-one relationship between the resource group, cluster and ancillary resources. Select the appropriate region and set the corresponding LOCATION environment variable.
<syntaxhighlight lang='bash'>
<syntaxhighlight lang='bash'>
export CLUSTER=platform-cloud-aro-01
export CLUSTER=platform-cloud-aro-02
export RESOURCEGROUP="${CLUSTER}-rg"
export RESOURCEGROUP="${CLUSTER}-rg"
export LOCATION=eastus2
export LOCATION=eastus2
az group create --name $RESOURCEGROUP --location $LOCATION
</syntaxhighlight>
</syntaxhighlight>


Line 44: Line 66:
az network vnet create \
az network vnet create \
   --resource-group $RESOURCEGROUP \
   --resource-group $RESOURCEGROUP \
   --name ${CLUSTER}-aro-vnet \
   --name ${CLUSTER}-vnet \
   --address-prefixes 10.0.0.0/16
   --address-prefixes 10.0.0.0/16


az network vnet subnet create \
az network vnet subnet create \
   --resource-group $RESOURCEGROUP \
   --resource-group $RESOURCEGROUP \
   --vnet-name ${CLUSTER}-aro-vnet \
   --vnet-name ${CLUSTER}-vnet \
   --name ${CLUSTER}-master-subnet \
   --name ${CLUSTER}-master-subnet \
   --address-prefixes 10.0.0.0/17 \
   --address-prefixes 10.0.0.0/17 \
Line 56: Line 78:
az network vnet subnet create \
az network vnet subnet create \
   --resource-group $RESOURCEGROUP \
   --resource-group $RESOURCEGROUP \
   --vnet-name ${CLUSTER}-aro-vnet \
   --vnet-name ${CLUSTER}-vnet \
   --name ${CLUSTER}-worker-subnet \
   --name ${CLUSTER}-worker-subnet \
   --address-prefixes 10.0.128.0/17 \
   --address-prefixes 10.0.128.0/17 \
Line 65: Line 87:
<syntaxhighlight lang='bash'>
<syntaxhighlight lang='bash'>
az network vnet subnet update \
az network vnet subnet update \
   --name ${CLUSTER}-worker-subnet \
   --name ${CLUSTER}-master-subnet \
   --resource-group $RESOURCEGROUP \
   --resource-group $RESOURCEGROUP \
   --vnet-name ${CLUSTER}-aro-vnet \
   --vnet-name ${CLUSTER}-vnet \
   --disable-private-link-service-network-policies true
   --disable-private-link-service-network-policies true
</syntaxhighlight>
</syntaxhighlight>
Line 78: Line 100:
   --resource-group $RESOURCEGROUP \
   --resource-group $RESOURCEGROUP \
   --name $CLUSTER \
   --name $CLUSTER \
   --vnet aro-vnet \
   --vnet ${CLUSTER}-vnet \
   --master-subnet master-subnet \
   --master-subnet ${CLUSTER}-master-subnet \
   --worker-subnet worker-subnet
   --worker-subnet ${CLUSTER}-worker-subnet \
  --ingress-visibility Public \
  --worker-count 3 \
  --worker-vm-disk-size-gb 200
</syntaxhighlight>
</syntaxhighlight>
It normally takes about 35 minutes to create a cluster. A successful execution produces a JSON report that gives essential information about the cluster. The same information can be obtained with '[[#List_Details_about_a_Specific_Cluster|az aro show]]'.


<font color=darkgray>
<font color=darkgray>
Line 89: Line 116:
</font>
</font>


===Organizatorium===
===Retrieve Credentials===
 
Upon cluster creation, the cluster administrator (kubeadmin) username and password can be retrieved with:
<syntaxhighlight lang='bash'>
<syntaxhighlight lang='bash'>
az aro list-credentials --name ${CLUSTER_NAME} --resource-group ${RESOURCE_GROUP}
az aro list-credentials --resource-group ${RESOURCEGROUP} --name ${CLUSTER}
</syntaxhighlight>
</syntaxhighlight>


=CLI Support=
===To Clarify===
Azure CLI has an OpenShift extension, with subcommands aimed at managing Azure Red Hat OpenShift clusters.
 
<font color=darkgray>
* It seems that an OpenShift cluster installation creates an additional resource group (aro-xxxxxxxx, where xxxxxxx is the same random alphanumeric sequence that shows up in the console name). The resource group has [[Azure_Storage_Concepts#Storage_Account|storage accounts]], [[Azure_Storage_Concepts#Disk|disks]], [[Azure_Networking_Concepts#Load_Balancer|load balancers]], [[Azure_Networking_Concepts#Network_Interface|network interfaces]], [[Azure_Networking_Concepts#Network_Security_Group|network security groups]], [[Azure_Networking_Concepts#Private_DNS_Zone|private DNS zones]], [[Azure_Networking_Concepts#Private_Link_Service|private link services]] and [[Azure_Networking_Concepts#Public_IP_Address|public IP addresses]].
</font>
 
=Concepts=
==Networking Concepts==
{{External|https://docs.microsoft.com/en-us/azure/openshift/concepts-networking}}
=Operations=
==Console==
The console URL is exposed as "consoleProfile.url" in the output of '[[#List_Details_about_a_Specific_Cluster|az aro show]]' command.
==oc==
===oc Installation===
====Mac====
Go to the Console → Top Level Menu Question Mark → Command Line Tools → Download oc for Mac for x86_64.
<syntaxhighlight lang='bash'>
cd /usr/local
mkdir openshift
cd openshift
unzip .../oc.zip # this will create an "oc" executable
export PATH=/usr/local/openshift:${PATH}
</syntaxhighlight>
 
The first attempt to run will trigger an alert warning you that the oc executable is not from a known developer and cannot be verified. To get around this, navigate to /usr/local/openshift in Finder, right-click and use Open.
 
===oc login===
'oc login' should be used upon first login. It updates [[.kube_config|.kube/config]].
<syntaxhighlight lang='bash'>
<syntaxhighlight lang='bash'>
az aro create|list|delete|list-credentials|show|update|wait
apiServer=$(az aro show -g ${RESOURCEGROUP} -n ${CLUSTER} --query apiserverProfile.url -o tsv)
password=$(az aro list-credentials -g ${RESOURCEGROUP} -n ${CLUSTER} --query kubeadminPassword -o tsv)
oc login ${apiServer} -u kubeadmin -p ${password}
</syntaxhighlight>
 
Upon execution, it adds the following:
<syntaxhighlight lang='yaml'>
clusters:
  - name: <api-server-url>:<api-server-port>
    cluster:
      server: https://<api-server-url>:<api-server-port>
users:
  - name: kube:admin
    user:
      token: Rt..._Xc
contexts:
  - name: default/<api-server-url>:<api-server-port>/kube:admin
    context:
      cluster: <api-server-url>:<api-server-port>
      namespace: default
      user: kube:admin 
</syntaxhighlight>
</syntaxhighlight>
==List OpenShift Clusters==
and also updates the current context.
 
For more details on the 'oc login' command, see: {{Internal|Oc_login#oc_login_and_Azure_OpenShift|oc login}}
==Storage Operations==
* [[Provision Azure Files ReadWriteMany Persistent Volumes on Azure OpenShift]]
==Security Operations==
===Obtain the Service Principal===
To obtain the service principal associated with the OpenShift cluster:
<syntaxhighlight lang='bash'>
<syntaxhighlight lang='bash'>
az aro list
az aro show -g <openshift-cluster-resource-group> -n <cluster-name> --query servicePrincipalProfile.clientId -o tsv
</syntaxhighlight>
</syntaxhighlight>

Latest revision as of 04:38, 31 December 2021

External

Internal

Overview

An OpenShift cluster installed on Azure has an associated service principal.

CLI Support

Azure CLI has an OpenShift extension, with subcommands aimed at managing Azure Red Hat OpenShift clusters.

az aro create|list|delete|list-credentials|show|update|wait

List OpenShift Clusters

az aro list

List Details about a Specific Cluster

az aro show --resource-group <rg-name> --name <cluster-name>

Installation

Prerequisites

Procedure

Create the Resource Group

The resource group can be created from the console or from command line. The resource group it will encapsulate resources required by, and dedicated to the OpenShift cluster. The name of the resource group should be derived from the name of the cluster by adding the "-rg" postfix. There will be a one-to-one relationship between the resource group, cluster and ancillary resources. Select the appropriate region and set the corresponding LOCATION environment variable.

export CLUSTER=platform-cloud-aro-02
export RESOURCEGROUP="${CLUSTER}-rg"
export LOCATION=eastus2

az group create --name $RESOURCEGROUP --location $LOCATION

Register Resource Providers

export SUBSCRIPTION_ID="..."
az account set --subscription ${SUBSCRIPTION_ID}

Register the Microsoft.RedHatOpenShift, Microsoft.Compute and Microsoft.Storage resource providers:

az provider register -n Microsoft.RedHatOpenShift --wait
az provider register -n Microsoft.Compute --wait
az provider register -n Microsoft.Storage --wait

Get a Red Hat Pull Secret

https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#get-a-red-hat-pull-secret-optional

TODO

Prepare a Custom Domain

https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#prepare-a-custom-domain-for-your-cluster-optional

TODO

Create a Virtual Network and associated Subnets

https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#create-a-virtual-network-containing-two-empty-subnets

Azure Red Hat OpenShift clusters require a virtual network with two empty subnets, for the master and worker nodes. The virtual network can be created as such (for more details about networking operations, see Azure Networking Operations):

az network vnet create \
  --resource-group $RESOURCEGROUP \
  --name ${CLUSTER}-vnet \
  --address-prefixes 10.0.0.0/16

az network vnet subnet create \
  --resource-group $RESOURCEGROUP \
  --vnet-name ${CLUSTER}-vnet \
  --name ${CLUSTER}-master-subnet \
  --address-prefixes 10.0.0.0/17 \
  --service-endpoints Microsoft.ContainerRegistry

az network vnet subnet create \
  --resource-group $RESOURCEGROUP \
  --vnet-name ${CLUSTER}-vnet \
  --name ${CLUSTER}-worker-subnet \
  --address-prefixes 10.0.128.0/17 \
  --service-endpoints Microsoft.ContainerRegistry

Disable subnet private endpoint policies on the master subnet. This is required for the service to be able to connect to and manage the cluster:

az network vnet subnet update \
  --name ${CLUSTER}-master-subnet \
  --resource-group $RESOURCEGROUP \
  --vnet-name ${CLUSTER}-vnet \
  --disable-private-link-service-network-policies true

Create the Cluster

https://docs.microsoft.com/en-us/azure/openshift/tutorial-create-cluster#create-the-cluster
az aro create \
  --resource-group $RESOURCEGROUP \
  --name $CLUSTER \
  --vnet ${CLUSTER}-vnet \
  --master-subnet ${CLUSTER}-master-subnet \
  --worker-subnet ${CLUSTER}-worker-subnet \
  --ingress-visibility Public \
  --worker-count 3 \
  --worker-vm-disk-size-gb 200

It normally takes about 35 minutes to create a cluster. A successful execution produces a JSON report that gives essential information about the cluster. The same information can be obtained with 'az aro show'.

TODO:

Retrieve Credentials

Upon cluster creation, the cluster administrator (kubeadmin) username and password can be retrieved with:

az aro list-credentials --resource-group ${RESOURCEGROUP} --name ${CLUSTER}

To Clarify

Concepts

Networking Concepts

https://docs.microsoft.com/en-us/azure/openshift/concepts-networking

Operations

Console

The console URL is exposed as "consoleProfile.url" in the output of 'az aro show' command.

oc

oc Installation

Mac

Go to the Console → Top Level Menu Question Mark → Command Line Tools → Download oc for Mac for x86_64.

cd /usr/local
mkdir openshift
cd openshift
unzip .../oc.zip # this will create an "oc" executable
export PATH=/usr/local/openshift:${PATH}

The first attempt to run will trigger an alert warning you that the oc executable is not from a known developer and cannot be verified. To get around this, navigate to /usr/local/openshift in Finder, right-click and use Open.

oc login

'oc login' should be used upon first login. It updates .kube/config.

apiServer=$(az aro show -g ${RESOURCEGROUP} -n ${CLUSTER} --query apiserverProfile.url -o tsv)
password=$(az aro list-credentials -g ${RESOURCEGROUP} -n ${CLUSTER} --query kubeadminPassword -o tsv)
oc login ${apiServer} -u kubeadmin -p ${password}

Upon execution, it adds the following:

clusters:
  - name: <api-server-url>:<api-server-port>
    cluster:
      server: https://<api-server-url>:<api-server-port>
users:
  - name: kube:admin
    user:
      token: Rt..._Xc
contexts:
  - name: default/<api-server-url>:<api-server-port>/kube:admin
    context:
      cluster: <api-server-url>:<api-server-port>
      namespace: default
      user: kube:admin

and also updates the current context.

For more details on the 'oc login' command, see:

oc login

Storage Operations

Security Operations

Obtain the Service Principal

To obtain the service principal associated with the OpenShift cluster:

az aro show -g <openshift-cluster-resource-group> -n <cluster-name> --query servicePrincipalProfile.clientId -o tsv