backup and restore

reference

• https://docs.pingcap.com/zh/tidb-in-kubernetes/stable/backup-restore-cr
• https://github.com/pingcap/tidb-operator/blob/master/pkg/apis/pingcap/v1alpha1/types.go#L1912
• https://asktug.com/t/topic/1042360

introduction

• two types of backup
  • full backup: backup full data of a cluster at a specific point in time
  • log backup(log refers to KV changes in TiKV): backup data changes in TiDB
• restore
  • restore the full backup
  • restore the target cluster to any specific point in time of the source backup cluster by integrating full backup with log backup, a capability formally referred to as Point-in-Time Recovery (PITR)

prepare

0. optional: only for local storage
1. requirements
   • for backup/restore by br, local:// protocol requires a local path in tikv to store the backup/restore files
   • therefore, we need to create a pvc to mount the local path in tikv
2. prepare backup.pvc.yaml

   • apiVersion: v1
     kind: PersistentVolumeClaim
     metadata:
       name: tidb-cluster-backup-pvc
       namespace: tidb-cluster
     spec:
       accessModes:
         - ReadWriteMany
       resources:
         requests:
           storage: 20Gi
       storageClassName: 
     
     

   • apply to the cluster

     • kubectl -n tidb-cluster apply -f backup.pvc.yaml
       

3. patch the tidb cluster

   • kubectl -n tidb-cluster patch tidbcluster basic --type=merge -p '{"spec":{"tikv":{"additionalVolumes":[{"name":"backup","persistentVolumeClaim":{"claimName":"tidb-cluster-backup-pvc"}}],"additionalVolumeMounts":[{"name":"backup","mountPath":"/backup"}]}}}'
     

   • check and waiting for the cluster status to be ready

     • kubectl -n tidb-cluster get tidbcluster basic -w
       

4. perpare default service account tidb-backup-manager

   • apiVersion: v1
     kind: ServiceAccount
     metadata:
       name: tidb-backup-manager
       namespace: tidb-cluster
     
     ---
     apiVersion: rbac.authorization.k8s.io/v1
     kind: Role
     metadata:
       name: tidb-backup-manager-role
       namespace: tidb-cluster
     rules:
     - apiGroups: ["pingcap.com"]
       resources: 
         - backups
         - restores
       verbs:
         - list
         - get
         - update
         - watch
         - patch
         - create
         - delete
     
     ---
     apiVersion: rbac.authorization.k8s.io/v1
     kind: RoleBinding
     metadata:
       name: tidb-backup-manager-rolebinding
       namespace: tidb-cluster
     subjects:
     - kind: ServiceAccount
       name: tidb-backup-manager
       namespace: tidb-cluster
     roleRef:
       kind: Role
       name: tidb-backup-manager-role
       apiGroup: rbac.authorization.k8s.io
     
     

backup and restore full database via pvc

1. apply backup
   • prepare backup-full-pvc.yaml

     • apiVersion: pingcap.com/v1alpha1
       kind: Backup
       metadata:
         name: backup-tidb-cluster-basic-full-1748228371
         namespace: tidb-cluster
       spec:
         cleanPolicy: Retain
         resources:
           requests:
             cpu: "500m"
             memory: "512Mi"
           limits:
             cpu: "1000m"
             memory: "1Gi"
         backupMode: snapshot
         backupType: full
         toolImage: m.daocloud.io/docker.io/pingcap/br:v8.5.0
         br:
           cluster: basic
           clusterNamespace: tidb-cluster
           logLevel: info
           concurrency: 4
           # rateLimit: 0
           # options:
           # - --lastbackupts=420134118382108673
         local:
           prefix: tidb-cluster/basic/full/1748228371
           volume:
             name: backup
             persistentVolumeClaim:
               claimName: tidb-cluster-backup-pvc
           volumeMount:
             name: backup
             mountPath: /backup
       
       

   • apply to the cluster

     • kubectl -n tidb-cluster apply -f backup-full-pvc.yaml
       

2. delete database
   • prepare delete-database.job.yaml

     • apiVersion: batch/v1
       kind: Job
       metadata:
         name: mysql-delete-database-job
       spec:
         template:
           spec:
             containers:
             - name: mysql-client
               image: m.daocloud.io/docker.io/library/mysql:9.3.0
               command: ['sh', '-c']
               env:
                 - name: MYSQL_SERVICE_IP
                   value: basic-tidb.tidb-cluster.svc.cluster.local
                 - name: MYSQL_SERVICE_PORT
                   value: "4000"
                 - name: MYSQL_ROOT_PASSWORD
                   valueFrom:
                     secretKeyRef:
                       name: basic-tidb-credentials
                       key: root
                       optional: false
                 - name: TARGET_DATABASE
                   value: "shopping"
               args:
                 - |
                   export MYSQL_PWD=$MYSQL_ROOT_PASSWORD
                   mysql -h $MYSQL_SERVICE_IP -P $MYSQL_SERVICE_PORT -u root -e "DROP DATABASE IF EXISTS $TARGET_DATABASE;"
             restartPolicy: Never
         backoffLimit: 4
       
       

   • apply to the cluster

     • kubectl -n tidb-cluster apply -f delete-database.job.yaml
       kubectl -n tidb-cluster wait --for=condition=complete job/mysql-delete-database-job
       kubectl -n tidb-cluster logs -l job-name=mysql-delete-database-job
       

3. restore the database
   • prepare restore-full-pvc.yaml

     • apiVersion: pingcap.com/v1alpha1
       kind: Restore
       metadata:
         name: restore-tidb-cluster-basic-full-1748228371
         namespace: tidb-cluster
       spec:
         resources:
           requests:
             cpu: "500m"
             memory: "512Mi"
           limits:
             cpu: "1000m"
             memory: "1Gi"
         restoreMode: snapshot
         backupType: full
         toolImage: m.daocloud.io/docker.io/pingcap/br:v8.5.0
         br:
           cluster: basic
           clusterNamespace: tidb-cluster
           logLevel: info
           concurrency: 4
           # rateLimit: 0
           # options:
           # - --lastbackupts=420134118382108673
         local:
           prefix: tidb-cluster/basic/full/1748228371
           volume:
             name: backup
             persistentVolumeClaim:
               claimName: tidb-cluster-backup-pvc
           volumeMount:
             name: backup
             mountPath: /backup
       
       

   • apply to the cluster

     • kubectl -n tidb-cluster apply -f restore-full-pvc.yaml
       

backup and resstore specific database/table via pvc

backup and restore specific table via s3

backup schedule and restore full database via pvc

1. apply backup schedule
   • prepare backup-schedule-full-pvc.yaml

     • apiVersion: pingcap.com/v1alpha1
       kind: BackupSchedule
       metadata:
         name: basic-backup-schedule-full-pvc
         namespace: tidb-cluster
       spec:
         maxBackups: 3
         pause: false
         schedule: "*/30 * * * *"
         backupTemplate:
           cleanPolicy: Retain
           resources:
             requests:
               cpu: "500m"
               memory: "512Mi"
             limits:
               cpu: "1000m"
               memory: "1Gi"
           backupMode: snapshot
           backupType: full
           toolImage: m.daocloud.io/docker.io/pingcap/br:v8.5.0
           br:
             cluster: basic
             clusterNamespace: tidb-cluster
             logLevel: info
             concurrency: 4
             # rateLimit: 0
             # checksum: true
           local:
             prefix: tidb-cluster/basic/full/scheduled
             volume:
               name: backup
               persistentVolumeClaim:
                 claimName: tidb-cluster-backup-pvc
             volumeMount:
               name: backup
               mountPath: /backup
       
       

   • apply to the cluster

     • kubectl -n tidb-cluster apply -f backup-schedule-full-pvc.yaml
       

2. pause the backup schedule
3. delete database
4. restore the database

backup log and PITR (Point-in-time recovery)