Kubernetes Storage - Dynamic NFS Client Provisioner

This article is more like a note to myself, but this may help if you are looking at How to setup Kubernetes Storage using Dynamic NFS Provisioning.

Kubernetes NFS Subdir External Provisioner is an automatic provisioner that use your existing and already configured NFS server to support dynamic provisioning of Kubernetes Persistent Volumes via Persistent Volume Claims.

NFS uses port 2049 and ensures that necessary ports are open to the target.

Prerequisites

Kubernetes Cluster
Helm Installed on Kubernetes Cluster
NFS Server Node (10.20.12.150)

Setup NFS Server

Executed on NFS Server Node

On Ubuntu

apt install nfs-kernel-server
mkdir -p /data/nfs
nano /etc/exports
---
/data/nfs *(rw,sync,no_subtree_check,no_root_squash)
---

sudo exportfs -a

2. On CentOS

dnf install nfs-utils
systemctl enable --now nfs-server.service
mkdir -p /data/nfs
nano /etc/exports
---
/data/nfs *(rw,sync,no_subtree_check,no_root_squash)
---

sudo exportfs -a

firewall-cmd --permanent --add-service=nfs
firewall-cmd --permanent --add-service=rpc-bind
firewall-cmd --permanent --add-service=mountd
firewall-cmd --reload

Deploy NFS Subdir External Provisioner

Executed on Kubernetes Cluster

Install Helm & NFS Client

# Install Helm
curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

helm version

# Install NFS Client
apt install nfs-common -y

2. Add Helm Repo

helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/

3. Install NFS Subdir External Provisioner

helm install nfs-cluster-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
    --set nfs.server=10.20.12.150 \
    --set nfs.path=/data/nfs/ \
    --set storageClass.name=nfs-cluster-client \
    --set storageClass.defaultClass=true \
    --set storageClass.provisionerName=cluster.local/nfs-cluster-provisioner

Or if you want to set reclaimPolicy to Retain (Default is Delete)

helm install nfs-cluster-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
    --set nfs.server=10.20.12.150 \
    --set nfs.path=/data/nfs/ \
    --set nfs.reclaimPolicy=Retain \
    --set storageClass.name=nfs-cluster-client \
    --set storageClass.provisionerName=cluster.local/nfs-cluster-provisioner \
    --set storageClass.defaultClass=true \
    --set storageClass.reclaimPolicy=Retain

4. Verify Installation

kubectl get pod
NAME                                                              READY   STATUS    RESTARTS   AGE
nfs-cluster-provisioner-nfs-subdir-external-provisioner-69fzjmd   1/1     Running   0          24s

kubectl get storageclass
NAME                           PROVISIONER                             RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs-cluster-client (default)   cluster.local/nfs-cluster-provisioner   Retain          Immediate           true                   31s

Operational Test

Deploy Wordpress Application

Download the manifests

curl -LO https://k8s.io/examples/application/wordpress/mysql-deployment.yaml
curl -LO https://k8s.io/examples/application/wordpress/wordpress-deployment.yaml

2. Create secret for mysql password

kubectl create namespace wordpress
kubectl create secret generic mysql-pass --from-literal=password=mysqlpass123 -n wordpress

3. Apply Manifests

kubectl apply -n wordpress -f mysql-deployment.yaml -f wordpress-deployment.yaml

service/wordpress-mysql created
persistentvolumeclaim/mysql-pv-claim created
deployment.apps/wordpress-mysql created
service/wordpress created
persistentvolumeclaim/wp-pv-claim created
deployment.apps/wordpress created

4. Verify The Application

kubectl get all -n wordpress
NAME                                   READY   STATUS    RESTARTS   AGE
pod/wordpress-c5c784d7c-bh89g          1/1     Running   0          72s
pod/wordpress-mysql-6c7ccf489f-mqplr   1/1     Running   0          72s

NAME                      TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)        AGE
service/wordpress         LoadBalancer   10.108.26.92   192.168.1.150   80:32658/TCP   72s
service/wordpress-mysql   ClusterIP      None           <none>          3306/TCP       72s

NAME                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/wordpress         1/1     1            1           72s
deployment.apps/wordpress-mysql   1/1     1            1           72s

NAME                                         DESIRED   CURRENT   READY   AGE
replicaset.apps/wordpress-c5c784d7c          1         1         1       72s
replicaset.apps/wordpress-mysql-6c7ccf489f   1         1         1       72s

5. Verify the PV and PVC Creation

kubectl get pv,pvc -n wordpress
NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                      STORAGECLASS         REASON   AGE
persistentvolume/pvc-1adde9eb-81d3-4643-a58f-a4e3fbc2a5b4   20Gi       RWO            Retain           Bound    wordpress/wp-pv-claim      nfs-cluster-client            101s
persistentvolume/pvc-b8aa3ec1-2e75-436c-99dd-f0b9eeab43e9   20Gi       RWO            Retain           Bound    wordpress/mysql-pv-claim   nfs-cluster-client            101s

NAME                                   STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS         AGE
persistentvolumeclaim/mysql-pv-claim   Bound    pvc-b8aa3ec1-2e75-436c-99dd-f0b9eeab43e9   20Gi       RWO            nfs-cluster-client   101s
persistentvolumeclaim/wp-pv-claim      Bound    pvc-1adde9eb-81d3-4643-a58f-a4e3fbc2a5b4   20Gi       RWO            nfs-cluster-client   101s

SSH to your NFS Server

ls /data/nfs
wordpress-wp-pv-claim-pvc-1adde9eb-81d3-4643-a58f-a4e3fbc2a5b4
wordpress-mysql-pv-claim-pvc-b8aa3ec1-2e75-436c-99dd-f0b9eeab43e9

ls /data/nfs/wordpress-wp-pv-claim-pvc-1adde9eb-81d3-4643-a58f-a4e3fbc2a5b4
index.php    readme.html      wp-admin            wp-comments-post.php  wp-config.php  wp-cron.php  wp-links-opml.php  wp-login.php  wp-settings.php  wp-trackback.php
license.txt  wp-activate.php  wp-blog-header.php  wp-config-sample.php  wp-content     wp-includes  wp-load.php        wp-mail.php   wp-signup.php    xmlrpc.php

ls /data/nfs/wordpress-mysql-pv-claim-pvc-b8aa3ec1-2e75-436c-99dd-f0b9eeab43e9
auto.cnf  ib_logfile0  ib_logfile1  ibdata1  mysql  performance_schema  wordpress

Result!