container-image-mirror

prepare

1. k8s is ready
   • in this article, the k8s cluster is created by minikube
2. argocd is ready and logged in
3. minio is ready

logic

• inspired by wzshiming
• 
  logic-of-container-image-mirror.png
• references
  • https://github.com/DaoCloud/crproxy
  • https://docs.docker.com/docker-hub/mirror/
  • https://github.com/twuni/docker-registry.helm

initialization

1. prepare secret named s3-credentials-for-registry to store the minio credentials

   • kubectl -n basic-components create secret generic s3-credentials-for-registry \
       --from-literal=s3AccessKey=$(kubectl -n storage get secret minio-credentials -o jsonpath='{.data.rootUser}' | base64 -d) \
       --from-literal=s3SecretKey=$(kubectl -n storage get secret minio-credentials -o jsonpath='{.data.rootPassword}' | base64 -d)
     

2. create bucket named registry in minio

   • # change K8S_MASTER_IP to your k8s master ip
     K8S_MASTER_IP=$(kubectl get node -l node-role.kubernetes.io/control-plane -o jsonpath='{.items[0].status.addresses[?(@.type=="InternalIP")].address}')
     ACCESS_SECRET=$(kubectl -n storage get secret minio-credentials -o jsonpath='{.data.rootPassword}' | base64 -d)
     podman run --rm \
         --entrypoint bash \
         --add-host=minio-api.dev.geekcity.tech:${K8S_MASTER_IP} \
         -it docker.io/minio/mc:latest \
         -c "mc alias set minio http://minio-api.dev.geekcity.tech:32080 admin ${ACCESS_SECRET} \
             && mc mb --ignore-existing minio/mirrors"
     

installation

1. prepare crproxy.yaml

   • apiVersion: argoproj.io/v1alpha1
     kind: Application
     metadata:
       name: crproxy
     spec:
       syncPolicy:
         syncOptions:
         - CreateNamespace=true
       project: default
       source:
         repoURL: https://charts.bitnami.com/bitnami
         chart: nginx
         targetRevision: 15.10.4
         helm:
           releaseName: crproxy
           values: |
             image:
               registry: ghcr.io
               repository: daocloud/crproxy/crproxy
               tag: v0.8.0
               pullPolicy: IfNotPresent
             containerPorts:
               http: 8080
             service:
               type: ClusterIP
               ports:
                 http: 8080
             ingress:
               enabled: false
       destination:
         server: https://kubernetes.default.svc
         namespace: basic-components
     
     

2. apply to k8s

   • kubectl -n argocd apply -f crproxy.yaml
     

3. sync by argocd

   • argocd app sync argocd/crproxy
     

4. prepare registry.yaml

   • apiVersion: argoproj.io/v1alpha1
     kind: Application
     metadata:
       name: registry
     spec:
       syncPolicy:
         syncOptions:
         - CreateNamespace=true
       project: default
       source:
         repoURL: https://helm.twun.io
         chart: docker-registry
         targetRevision: 2.2.3
         helm:
           releaseName: registry
           values: |
             image:
               repository: docker.io/library/registry
             storage: s3
             secrets:
               s3:
                 secretRef: s3-credentials-for-registry
             s3:
               region: us-east-1
               regionEndpoint: http://minio-api.dev.geekcity.tech:32080
               bucket: mirrors
               rootdirectory: /registry/pull-through-cache
               encrypt: false
               secure: true
             proxy:
               enabled: true
               remoteurl: http://crproxy-nginx.basic-components:8080
             ingress:
               enabled: true
               className: nginx
               annotations:
                 cert-manager.io/cluster-issuer: self-signed-ca-issuer
               hosts:
                 - container-image-mirror.dev.geekcity.tech
               tls:
                 - secretName: container-image-mirror.dev.geekcity.tech-tls
                   hosts:
                     - container-image-mirror.dev.geekcity.tech
       destination:
         server: https://kubernetes.default.svc
         namespace: basic-components
     
     

5. apply to k8s

   • kubectl -n argocd apply -f registry.yaml
     

6. sync by argocd

   • argocd app sync argocd/registry
     

7. if you can't control dns to point minio-api.dev.geekcity.tech to 192.168.49.2
   • patch the deployment by hostAliases

     • kubectl -n basic-components patch deployment registry-docker-registry --patch '
       spec:
         template:
           spec:
             hostAliases:
             - ip: 192.168.49.2
               hostnames:
               - minio-api.dev.geekcity.tech
       '
       

tests

1. • container-image-mirror.dev.geekcity.tech and minio-api.dev.geekcity.tech can be resolved
     • for example
       • add $K8S_MASTER_IP container-image-mirror.dev.geekcity.tech to /etc/hosts
       • add $K8S_MASTER_IP minio-api.dev.geekcity.tech to /etc/hosts
     • with k8s with minikube, $K8S_MASTER_IP is the ip of the minikube vm, usually 192.168.49.2
2. pull image

   • podman pull --tls-verify=false container-image-mirror.dev.geekcity.tech:32443/docker.io/library/alpine:3.20.1
     

extensions

1. storage
   • replace minio with oss
   • use pvc
   • without persistent storage
2. network of crproxy
   • set http_proxy and https_proxy for crproxy
3. apply ssl with let's encrypt