Backup and restore data using NebulaGraph Operator¶
This article introduces how to back up and restore data of the NebulaGraph cluster on Kubernetes.
Enterpriseonly
This feature is only for the enterprise edition NebulaGraph clusters on Kubernetes.
Overview¶
NebulaGraph BR (Enterprise Edition) is a command line tool for data backup and recovery of NebulaGraph enterprise edition. NebulaGraph Operator is based on the BR tool to achieve data backup and recovery for NebulaGraph clusters on Kubernetes.
When backing up data, NebulaGraph Operator creates a Job to back up the data in the NebulaGraph cluster to the specified storage service.
When restoring data, NebulaGraph Operator checks the specified backup NebulaGraph cluster for existence, and whether the access to remote storage is normally based on the information defined in the NebulaRestore resource object. It then creates a new cluster and restores the backup data to the new NebulaGraph cluster. For more information, see restore flowchart.
Prerequisites¶
To backup and restore data using NebulaGraph Operator, the following conditions must be met:
- Nebula Operator version >= 1.4.0.
- The enterprise edition NebulaGraph cluster deployed on Kubernetes is running.
-
In the YAML file used to create the cluster,
spec.enableBRis set to true.// Partial content of a sample cluster YAML file. apiVersion: apps.nebula-graph.io/v1alpha1 kind: NebulaCluster metadata: name: nebula spec: enableBR: true // Set to true to enable the backup and restore function. ...
- Only storage services that use the S3 protocol (such as AWS S3, Minio, etc.) can be used to back up and restore data.
- Sufficient computing resources are available in the cluster to restore data.
Backup¶
Notes¶
- NebulaGraph Operator supports full and incremental backups.
- During data backup, DDL and DML statements in the specified graph space will be blocked. We recommend performing the operation during off-peak hours, such as from 2:00 am to 5:00 am.
- The cluster executing incremental backups and the cluster specified for the last backup must be the same, and the (storage bucket) path for the last backup must be the same.
- Ensure that the time between each incremental backup and the last backup is less than a
wal_ttl. - Specifying the backup data of a specified graph space is not supported.
-
Before backing up data, you need to create a Secret to restore the credential for pulling the image of the BR-ent tool.
kubectl - <nebula> create secret docker-registry <br-ent-secret> \ --docker-server=REGISTRY_SERVER \ --docker-username=REGISTRY_USERNAME \ --docker-password=REGISTRY_PASSWORD \<nebula>: The namespace where the Secret is located.<br-ent-secret>: The name of the Secret.REGISTRY_SERVER: The address of the private image repository server from which the image is pulled, for example,reg.example-inc.com.REGISTRY_USERNAME: The username for logging in to the private image repository server.REGISTRY_PASSWORD: The password for logging in to the private image repository server.
Full backup¶
When backing up data to a storage service compatible with the S3 protocol, you need to create a backup Job, which will back up the full NebulaGraph data to the specified storage location.
Here is an example of the YAML file for a full backup Job:
apiVersion: batch/v1
kind: Job
metadata:
name: nebula-full-backup
spec:
parallelism: 1
ttlSecondsAfterFinished: 600
template:
spec:
restartPolicy: OnFailure
imagePullSecrets:
- name: br-ent-secret # The name of the Secret for pulling the image of the BR-ent tool.
containers:
- image: reg.vesoft-inc.com/cloud-dev/br-ent:v3.5.1
imagePullPolicy: Always
name: backup
command:
- /bin/sh
- -ecx
- 'exec /usr/local/bin/br-ent backup full
--meta nebula-metad-0.nebula-metad-headless.nebula.svc.cluster.local:9559 # The address of the Metad service.
--storage s3://BUCKET # The storage location of the backup file.
--s3.access_key ACCESS_KEY # The AccessKey for accessing the S3 protocol-compatible storage service.
--s3.secret_key SECRET_KEY # The SecretKey for accessing the S3 protocol-compatible storage service.
--s3.region REGION # The region of the S3 protocol-compatible storage service.
--s3.endpoint https://s3.REGION.amazonaws.com' # The endpoint of the S3 protocol-compatible storage service.
Incremental backup¶
Except for the name of the Job and the command specified in spec.template.spec.containers[0].command, the YAML file for incremental backup is the same as that for a full backup. Here is an example of the YAML file for incremental backup:
apiVersion: batch/v1
kind: Job
metadata:
name: nebula-incr-backup
spec:
parallelism: 1
ttlSecondsAfterFinished: 60
template:
spec:
restartPolicy: OnFailure
imagePullSecrets:
- name: br-ent-secret
containers:
- image: reg.vesoft-inc.com/cloud-dev/br-ent:v3.5.1
imagePullPolicy: Always
name: backup
command:
- /bin/sh
- -ecx
- 'exec /usr/local/bin/br-ent backup incr
--meta nebula-metad-0.nebula-metad-headless.nebula.svc.cluster.local:9559 # The address of the Metad service.
--storage s3://BUCKET # The storage location of the backup file.
--s3.access_key ACCESS_KEY # The AccessKey for accessing the S3 protocol-compatible storage service.
--s3.secret_key SECRET_KEY # The SecretKey for accessing the S3 protocol-compatible storage service.
--s3.region REGION # The region of the S3 protocol-compatible storage service.
--s3.endpoint https://s3.REGION.amazonaws.com' # The endpoint of the S3 protocol-compatible storage service.
Parameter description¶
The main parameters are described as follows:
| Parameter | Default value | Description |
|---|---|---|
spec.parallelism |
1 | The number of tasks executed in parallel. |
spec.ttlSecondsAfterFinished |
60 | The time to keep task information after the task is completed. |
spec.template.spec.containers[0].image |
vesoft/br-ent:3.5.1 |
The image address of the NebulaGraph BR Enterprise Edition tool. |
spec.template.spec.containers[0].command |
- | The command for backing up data to the storage service compatible with the S3 protocol. For descriptions of the options in the command, see Parametr description. |
For more settings of the Job, see Kubernetes Jobs.
After the YAML file for the backup Job is set, run the following command to start the backup Job:
kubectl apply -f <backup_file_name>.yaml
When the data backup succeeds, a backup file is generated in the specified storage location. For example, the backup file name is BACKUP_2023_02_12_10_04_16.
Restore¶
Notes¶
- After the data recovery is successful, a new cluster will be created and the old cluster will not be deleted by default. You can decide whether to delete the old cluster themselves. The name of the new cluster is automatically generated by the Operator.
- There will be a period of service unavailability during the data recovery process, so it is recommended to perform the operation during a low period of business activity.
Process¶
When restoring data from a compatible S3 protocol service, you need to create a Secret to store the credentials for accessing the compatible S3 protocol service. Then create a resource object (NebulaRestore) for restoring the data, which will instruct the Operator to create a new NebulaGraph cluster based on the information defined in this resource object and restore the backup data to the newly created cluster.
Here is an example YAML for restoring data based on the backup file BACKUP_2023_02_12_10_04_16:
apiVersion: v1
kind: Secret
metadata:
name: aws-s3-secret
type: Opaque
data:
access-key: QVNJQVE0WFlxxx
secret-key: ZFJ6OEdNcDdxenMwVGxxx
---
apiVersion: apps.nebula-graph.io/v1alpha1
kind: NebulaRestore
metadata:
name: restore1
spec:
br:
clusterName: nebula
backupName: "BACKUP_2023_02_12_10_04_16"
concurrency: 5
s3:
region: "us-west-2"
bucket: "nebula-br-test"
endpoint: "https://s3.us-west-2.amazonaws.com"
secretName: "aws-s3-secret"
Parameter Description¶
-
Secret
Parameter Default Value Description metadata.name- The name of the Secret. typeOpaqueThe type of the Secret. See Types of Secret for more information. data.access-key- The AccessKey for accessing the S3 protocol-compatible storage service. data.secret-key- The SecretKey for accessing the S3 protocol-compatible storage service.
-
NebulaRestore
Parameter Default Value Description metadata.name- The name of the resource object NebulaRestore. spec.br.clusterName- The name of the backup cluster. Backup based on this cluster, then create a new cluster with a name automatically generated. spec.br.backupName- The name of the backup file. Restore data based on this backup file. spec.br.concurrency5The number of concurrent downloads when restoring data. The default value is 5.spec.br.s3.region- The geographical region where the S3 storage bucket is located. spec.br.s3.bucket- The path of the S3 storage bucket where backup data is stored. spec.br.s3.endpoint- The access address of the S3 storage bucket. spec.br.s3.secretName- The name of the Secret that is used to access the S3 storage bucket.
After setting up the YAML file for restoring the data, run the following command to start the restore job:
kubectl apply -f <restore_file_name>.yaml
Run the following command to check the status of the NebulaRestore object.
kubectl get rt <NebulaRestore_name> -n <namespace>
# Output example:
NAME STATUS STARTED COMPLETED AGE
restore1 Complete 67m 59m 67m
After the restore job is completed, a new NebulaGraph cluster is created with the name automatically generated by the Operator. To check the status of the new cluster:
kubectl get nc -n <namespace>
# Output example:
NAME GRAPHD-DESIRED GRAPHD-READY METAD-DESIRED METAD-READY STORAGED-DESIRED STORAGED-READY AGE
nebula 1 1 1 1 3 3 2d3h
ngxvsm 1 1 1 1 3 3 92m # The newly created cluster.