Provision Azure Files ReadWriteMany Persistent Volumes on Azure OpenShift

External

• https://docs.microsoft.com/en-us/azure/openshift/howto-create-a-storageclass

Internal

• OpenShift on Azure
• Azure Kubernetes Storage

Overview

This article documents the procedure to configure an Azure OpenShift instance to use Azure Files-backed Persistent Volumes, dynamically provisioned by the kubernetes.io/azure-file plug-in. More details about the kubernetes.io/azure-file provisioner are available here:

Azure Kubernetes Storage | kubernetes.io/azure-file Provisioner

Procedure

Create a Storage Account

Create a storage account that will manage the Azure File shares allocated to the Kubernetes cluster. The storage account may be created within a dedicated resource group, or an already existing resource group can be used for this purpose.

The procedure to create a storage group is available here:

Create Storage Account

Configure OpenShift Service Principal with "listKey" Permissions on the Storage Account Resource Group

The OpenShift service principal must have the "listKey" permission on the resource group associated with the newly created storage account. This is done by assigning the "Contributor" role to the OpenShift service principal for the service account resource group.

The OpenShift service principal can be obtained as described here:

Obtain the OpenShift cluster service principal

To assign the role:

az role assignment create --role Contributor --assignee <openshift-cluster-service-principal> --resource-group <storage-account-resource-group>

Note that --resource-group should get the newly created storage account resource group (or, if no new resource group was created, the resource group the storage account is part of), not the OpenShift cluster resource group, unless they coincide.

For more details about role assignment see:

Azure Security Operations | Assign a Role

Configure system:serviceaccount:kube-system:persistent-volume-binder Service Account with the Ability to Read Secrets

The OpenShift persistent volume binder service account will need the ability to read secrets. This can be achieved by creating and assigning an OpenShift cluster role with the "get" and "create" permissions on secrets. Login into the OpenShift API server as described here: OpenShift on Azure | oc login.

Create the role with:

oc create clusterrole azure-secret-reader --verb=create,get --resource=secrets

Bind the role to system:serviceaccount:kube-system:persistent-volume-binder with:

oc adm policy add-cluster-role-to-user azure-secret-reader system:serviceaccount:kube-system:persistent-volume-binder

Create the Azure Files StorageClass

export LOCATION=...
export STORAGE_ACCOUNT_NAME=...
export STORAGE_RESOURCE_GROUP=...
cat << EOF | oc create -f - 
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: azure-files
provisioner: kubernetes.io/azure-file
parameters:
  location: ${LOCATION}
  skuName: Standard_LRS 
  storageAccount: ${STORAGE_ACCOUNT_NAME}
  resourceGroup: ${STORAGE_RESOURCE_GROUP}
reclaimPolicy: Delete
volumeBindingMode: Immediate
EOF

Change the default StorageClass (Optional)

Optionally, the default StorageClass can be set to be the new StorageClass, but probably in most cases, the existing default storage class is a better choice.

Test

Create a Persistent Volume Claim for a "azure-files" storage class Persistent Volume:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: azure-files-pvc
spec:
  accessModes:
  - ReadWriteMany
  storageClassName: azure-files
  # no volumeName specified, the volume will be auto-provisioned
  #volumeName: azure-files-01
  resources:
    requests:
      storage: 100Mi

Deploy a pod that uses it:

namespace=of
cat << EOF | kubectl -n ${namespace} apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: 'httpd'
spec:
  containers:
  - name: 'httpd'
    image: docker.io/ovidiufeodorov/httpd:latest
    volumeMounts:
      - name: example 
        mountPath: /azure-files
        # subPath: something
  volumes:
    - name: example
      persistentVolumeClaim:
        claimName: azure-files-test-pvc
EOF

Exec into the pod and create a file in /azure-files.

Then deploy the second pod, exec into it, and attempt to read the file from /azure-files:

namespace=of
cat << EOF | kubectl -n ${namespace} apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: 'httpd2'
spec:
  containers:
  - name: 'httpd'
    image: docker.io/ovidiufeodorov/httpd:latest
    volumeMounts:
      - name: example 
        mountPath: /azure-files
        # subPath: something
  volumes:
    - name: example
      persistentVolumeClaim:
        claimName: azure-files-test-pvc
EOF