Vishal Chauhan

Leveraging Amazon FSx as Persistent Volume for EKS Windows Nodes

VISHAL CHAUHAN's photo
VISHAL CHAUHAN
·

4 min read

Leveraging Amazon FSx as Persistent Volume for EKS Windows Nodes

Table of contents

Introduction to Amazon FSx

In the ever-evolving landscape of cloud computing, managing and scaling storage solutions for containerized applications is a critical aspect of modern infrastructure. One of the challenges, particularly in the context of Windows containers orchestrated by Kubernetes, is the efficient provisioning and management of persistent storage. In this blog post, we delve into the solution provided by Amazon FSx for Windows File Server.

What is Amazon FSx?

Amazon FSx is a fully managed file storage service that makes it easy to set up and scale file systems in the AWS Cloud. Specifically designed to address Windows workload requirements, Amazon FSx for Windows File Server provides a shared file storage solution that can be seamlessly integrated with Windows applications and services.

Prerequisites

  1. Amazon EKS cluster with windows nodes configured.

  2. Amazon FSx for Windows File Server deployed (Terraform module for FSx). Make sure to enable inbound port 445 in the Security Group for FSx.

  3. Microsoft Active Directory domain deployed to support Amazon FSx for Windows File Server.

  4. Roles and policies attached to EKS cluster for accessing Amazon FSx.

Using the SMB CSI Driver on Amazon EKS Windows nodes

CSI drivers such as SMB CSI Driver allows Amazon Elastic Kubernetes Service (Amazon EKS) clusters to manage Amazon FSx shares lifecycle for persistent volumes. The Server Message Block (SMB) protocol is a network file sharing protocol that allows applications to access files or other resources at a remote server, here FSx.

  1. Installing the SMB CSI Driver

    Click here to install SMB CSI Driver on your EKS Windows nodes. Apply all these manifest files using kubectl apply command.

  2. Create a Kubernetes secret

    Windows nodes needs Read/Write permissions in the SMB share in order to offer it as local directories to the Windows pod. Create a secret named "smbcreds" that contains an Active Directory username and password with Read/Write privileges on the Amazon FSx for Windows File Server share.

     kubectl create secret generic smbcreds --from-literal domain=DOMAINNAME --from-literal username=USERNAME --from-literal password=PASSWORD

    Replace with the following:
    DOMAINNAME: The Active Directory FQDN domain to which the Amazon FSx for Windows File Server is joined.
    USERNAME: The domain user name with Read/Write access to the Amazon FSx for Windows File Server root share.
    PASSWORD: The password for the specified user.

  3. Mounting Amazon FSx file share

    You can check whether the mount of your Amazon FSx file share on your Windows node is successful or not. Connect to your windows instance and run the command Test-NetConnection FSx_ipaddress -Port 445 in PowerShell to check if the connection works. Replace FSx_ipaddress with your File Server IP address.

  4. Test by mounting local directories in the Windows Pod using SMB CSI Driver on Deployments with PV and PVC.

    1. Create a PersistentVolume manifest. Save it as pv-smb.yaml.

       apiVersion: v1
 kind: PersistentVolume
 metadata:
   name: fsx-pv
   annotations:
     pv.kubernetes.io/provisioned-by: smb.csi.k8s.io
 spec:
   capacity:
     storage: 30Gi  
   accessModes:
     - ReadWriteMany  
   persistentVolumeReclaimPolicy: Retain
   storageClassName: fsx-sc
   mountOptions:
     - dir_mode=0777
     - file_mode=0777
   csi:
     driver: smb.csi.k8s.io
     readOnly: false
     volumeHandle: fs-302kai605a50028b4    #Replace it with your FSx ID
     volumeAttributes:
       source: //10.4.20.19/share    #Replace IP with your File Server IP Address
     nodeStageSecretRef:
       name: smbcreds
       namespace: default

    2. Apply it using kubectl apply -f pv-smb.yaml

    3. Create a PersistentVolumeClaim manifest. Copy the following manifest and save it as pvc-smb.yaml:

       apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
   name: fsx-pvc
 spec:
   accessModes:
     - ReadWriteMany
   resources:
     requests:
       storage: 25Gi 
   volumeName: fsx-pv
   storageClassName: fsx-sc

    4. Apply it using kubectl apply -f pvc-smb.yaml

    5. Deploy a pod that consumes the PersistentVolumeClaim. Copy the following manifest and save it as busybox-smb.yaml

       apiVersion: apps/v1
 kind: Deployment
 metadata:
   name: busybox-smb
   labels:
     app: busybox
 spec:
   replicas: 2
   template:
     metadata:
       name: busybox
       labels:
         app: busybox
     spec:
       nodeSelector:
         "kubernetes.io/os": windows
       containers:
         - name: busybox
           image: e2eteam/busybox:1.29
           command:
             - "powershell.exe"
             - "-Command"
             - "while (1) { Add-Content -Encoding Ascii C:\\mnt\\smb\\data.txt  $(Get-Date -Format u); sleep 1 }"
           volumeMounts:
             - name: smb
               mountPath: "/pv/fsx-pv"
       volumes:
         - name: smb
           persistentVolumeClaim:
             claimName: fsx-pvc
   selector:
     matchLabels:
       app: busybox

    6. Apply it using kubectl apply -f busybox-smb.yaml

    7. To validate if SMB CSI Driver was correctly setup, lets proceed with a simple test of writing a simple “Hello” file to the local directory “C:\pv\fsx-pv” inside Pod1 and accessing the file from Pod2.

      1. Identify busybox pods name -> kubectl get pods

      2. Go inside the Pod1 -> kubectl exec -it busybox-smb-POD-NAME-1 -- powershell

      3. Create a text file -> Write-Output "Hello, world!" | Out-File -FilePath "C:\pv\fsx-pv\hello.txt"

      4. Exit Pod1 and access Pod2 -> kubectl exec -it busybox-smb-POD-NAME-2 -- powershell

      5. You will see hello.txt at “C:\pv\fsx-pv” inside Pod2 -> Get-Content -Path "C:\pv\fsx-pv\hello.txt"

Conclusion

In this post, we have outlined how to configure your Amazon EKS Windows cluster pods to mount a volume through an SMB share hosted in Amazon FSx for Windows File Server. We then used a third-party CSI SMB driver to deploy a sample application to validate the cross-access of the file share. The CSI driver is a free, open-source Kubernetes CSI driver and with no additional cost on the environment.

This approach of data persistence demonstrates the capabilities of extending the use of persistent storage hosted on SMB shares with Amazon FSx using Container Storage Interface (CSI) to access Windows shares storage.

EKSWindowsNodeSMB CSI DriverEKSfsx-for-windows