Provision Azure Files ReadWriteMany Persistent Volumes on Azure OpenShift

External

• https://docs.microsoft.com/en-us/azure/openshift/howto-create-a-storageclass

Internal

• OpenShift on Azure
• Azure Kubernetes Storage

Overview

This article documents the procedure to configure an Azure OpenShift instance to use Azure Files-backed Persistent Volumes, dynamically provisioned by the kubernetes.io/azure-file plug-in. More details about the kubernetes.io/azure-file provisioner are available here:

Azure Kubernetes Storage | kubernetes.io/azure-file Provisioner

Procedure

1. Create a storage account with its dedicated resource group. Why? Why can't we use the OpenShift cluster resource group?. Use this:

Create Storage Account

2. Give the OpenShift service principal "listKey" permission on the new storage account resource group, by assigning the "Contributor" role to the OpenShift service principal for the resource group.

The OpenShift service principal can be obtained as described here:

Obtain the OpenShift cluster service principal

Assign the role:

az role assignment create --role Contributor --assignee <openshift-cluster-service-principal> --resource-group <storage-account-resource-group>

Note that the resource group should be the newly create storage account resource group, not the OpenShift cluster resource group.

For more details about role assignment see:

Azure Security Operations | Assign a Role

3. The OpenShift persistent volume binder service account will need the ability to read secrets. This ability can be given by creating and assigning an OpenShift cluster role to achieve this. Login into the OpenShift API server as described here: OpenShift on Azure | oc login.

Create the role with:

oc create clusterrole azure-secret-reader --verb=create,get --resource=secrets

Bind the role to system:serviceaccount:kube-system:persistent-volume-binder with:

oc adm policy add-cluster-role-to-user azure-secret-reader system:serviceaccount:kube-system:persistent-volume-binder

Create the Azure Files StorageClass

export LOCATION=...
export STORAGE_ACCOUNT_NAME=...
export STORAGE_RESOURCE_GROUP=...
cat << EOF | oc create -f - 
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: azure-files
provisioner: kubernetes.io/azure-file
parameters:
  location: ${LOCATION}
  skuName: Standard_LRS 
  storageAccount: ${STORAGE_ACCOUNT_NAME}
  resourceGroup: ${STORAGE_RESOURCE_GROUP}
reclaimPolicy: Delete
volumeBindingMode: Immediate
EOF

Change the default StorageClass (Optional)

TODO

Test

Create a Persistent Volume Claim for a "azure-files" storage class Persistent Volume:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: azure-files-pvc
spec:
  accessModes:
  - ReadWriteMany
  storageClassName: azure-files
  # no volumeName specified, the volume will be auto-provisioned
  #volumeName: azure-files-01
  resources:
    requests:
      storage: 100Mi

Deploy a pod that uses it:

namespace=of
cat << EOF | kubectl -n ${namespace} apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: 'httpd'
spec:
  containers:
  - name: 'httpd'
    image: docker.io/ovidiufeodorov/httpd:latest
    volumeMounts:
      - name: example 
        mountPath: /azure-files
        # subPath: something
  volumes:
    - name: example
      persistentVolumeClaim:
        claimName: azure-files-test-pvc
EOF

Exec into the pod and create a file in /azure-files.

Then deploy the second pod, exec into it, and attempt to read the file from /azure-files.