s3fs with sidecar

what

• mount s3 bucket with s3fs
• use sidecar to initialize s3fs
• share sub path of mounted fs to other containers

assumptions

• s3
  • endpoint: http://minio.storage:9000
  • credentials can be found in the secret named minio-credentials

    • #MINIO_ROOT_PASSWORD=your_minio_root_password
      kubectl create secret generic minio-credentials \
          --from-literal=access-key=admin \
          --from-literal=access-secret=$MINIO_ROOT_PASSWORD
      

  • bucket: bucket-to-mount
    • tree of files
      • /
        • file1
        • file2
        • project-foo
          • file3
          • file4
          • dir-foo
            • file7
        • project-bar
          • file5
          • file6

sharing /project-foo to other containers

1. prepare s3fs-with-sidecar.yaml

   • apiVersion: apps/v1
     kind: Deployment
     metadata:
       name: s3fs-client-deployment
       labels:
         app: s3fs-client
     spec:
       replicas: 1
       selector:
         matchLabels:
           app: s3fs-client
       template:
         metadata:
           labels:
             app: s3fs-client
         spec:
           containers:
           - name: s3fs-mount-container
             image: docker.io/efrecon/s3fs:1.94
             securityContext:
               privileged: true
               capabilities:
                 add:
                   - SYS_ADMIN
             env:
             - name: AWS_S3_URL
               value: http://minio.storage:9000/
             - name: AWS_S3_BUCKET
               value: bucket-to-mount
             - name: AWS_S3_ACCESS_KEY_ID
               valueFrom:
                 secretKeyRef:
                   name: minio-credentials
                   key: access-key
             - name: AWS_S3_SECRET_ACCESS_KEY
               valueFrom:
                 secretKeyRef:
                   name: minio-credentials
                   key: access-secret
             volumeMounts:
             - name: s3-volume
               mountPath: /opt/s3fs/bucket
               mountPropagation: Bidirectional
           - name: volume-using-container
             image: busybox
             command: ["/bin/sh", "-c", "while true; do ls -l /mnt/s3; sleep 10; done"]
             volumeMounts:
             - name: s3-volume
               mountPath: /mnt/s3
               subPath: project-foo
               mountPropagation: HostToContainer
           volumes:
           - name: s3-volume
             emptyDir: {}
     
     

2. apply to k8s

   • kubectl apply -f s3fs-with-sidecar.yaml
     

3. check logs

   • kubectl logs -f deployment/s3fs-client-deployment -c volume-using-container