Backup and restore data with snapshots

NebulaGraph supports using snapshots to back up and restore data. When data loss or misoperation occurs, the data will be restored through the snapshot.

Prerequisites

NebulaGraph authentication is disabled by default. In this case, all users can use the snapshot feature.

If authentication is enabled, only the GOD role user can use the snapshot feature. For more information about roles, see Roles and privileges.

Precautions

• To prevent data loss, create a snapshot as soon as the system structure changes, for example, after operations such as ADD HOST, DROP HOST, CREATE SPACE, DROP SPACE, and BALANCE are performed.

• NebulaGraph cannot automatically delete the invalid files created by a failed snapshot task. You have to manually delete them by using DROP SNAPSHOT.

• Customizing the storage path for snapshots is not supported for now.

Create snapshots

Run CREATE SNAPSHOT to create a snapshot for all the graph spaces based on the current time for NebulaGraph. Creating a snapshot for a specific graph space is not supported yet.

Note

If the creation fails, refer to the later section to delete the corrupted snapshot and then recreate the snapshot.

nebula> CREATE SNAPSHOT;

View snapshots

To view all existing snapshots, run SHOW SNAPSHOTS.

nebula> SHOW SNAPSHOTS;
+--------------------------------+---------+------------------+
| Name                           | Status  | Hosts            |
+--------------------------------+---------+------------------+
| "SNAPSHOT_2021_03_09_08_43_12" | "VALID" | "127.0.0.1:9779" |
| "SNAPSHOT_2021_03_09_09_10_52" | "VALID" | "127.0.0.1:9779" |
+--------------------------------+---------+------------------+

The parameters in the return information are described as follows.

Parameter	Description
Name	The name of the snapshot directory. The prefix SNAPSHOT indicates that the file is a snapshot file, and the suffix indicates the time the snapshot was created (UTC).
Status	The status of the snapshot. VALID indicates that the creation succeeded, while INVALID indicates that it failed.
Hosts	The IPs (or hostnames) and ports of all Storage servers at the time the snapshot was created.

Snapshot path

Snapshots are stored in the path specified by the data_path parameter in the Meta and Storage configuration files. When a snapshot is created, the checkpoints directory is checked in the datastore path of the leader Meta service and all Storage services for the existence, and if it is not there, it is automatically created. The newly created snapshot is stored as a subdirectory within the checkpoints directory. For example, SNAPSHOT_2021_03_09_08_43_12. The suffix 2021_03_09_08_43_12 is generated automatically based on the creation time (UTC).

To fast locate the path where the snapshots are stored, you can use the Linux command find in the datastore path. For example:

$ cd /usr/local/nebula-graph-ent-3.6.0/data
$ find |grep 'SNAPSHOT_2021_03_09_08_43_12'
./data/meta2/nebula/0/checkpoints/SNAPSHOT_2021_03_09_08_43_12
./data/meta2/nebula/0/checkpoints/SNAPSHOT_2021_03_09_08_43_12/data
./data/meta2/nebula/0/checkpoints/SNAPSHOT_2021_03_09_08_43_12/data/000081.sst
...

Delete snapshots

To delete a snapshot with the given name, run DROP SNAPSHOT.

DROP SNAPSHOT <snapshot_name>;

Example:

nebula> DROP SNAPSHOT SNAPSHOT_2021_03_09_08_43_12;
nebula> SHOW SNAPSHOTS;
+--------------------------------+---------+------------------+
| Name                           | Status  | Hosts            |
+--------------------------------+---------+------------------+
| "SNAPSHOT_2021_03_09_09_10_52" | "VALID" | "127.0.0.1:9779" |
+--------------------------------+---------+------------------+

Note

Deleting the only snapshot within the checkpoints directory also deletes the checkpoints directory.

Restore data with snapshots

Warning

When you restore data with snapshots, make sure that the graph spaces backed up in the snapshot have not been dropped. Otherwise, the data of the graph spaces cannot be restored.

Currently, there is no command to restore data with snapshots. You need to manually copy the snapshot file to the corresponding folder, or you can make it by using a shell script. The logic implements as follows:

1. After the snapshot is created, the checkpoints directory is generated in the installation directory of the leader Meta service and all Storage services, and saves the created snapshot. Taking this topic as an example, when there are two graph spaces, the snapshots created are saved in /usr/local/nebula/data/meta/nebula/0/checkpoints, /usr/local/nebula/data/storage/ nebula/3/checkpoints and /usr/local/nebula/data/storage/nebula/4/checkpoints.

   $ ls /usr/local/nebula/data/meta/nebula/0/checkpoints/
   SNAPSHOT_2021_03_09_09_10_52
   $ ls /usr/local/nebula/data/storage/nebula/3/checkpoints/
   SNAPSHOT_2021_03_09_09_10_52
   $ ls /usr/local/nebula/data/storage/nebula/4/checkpoints/
   SNAPSHOT_2021_03_09_09_10_52
   

2. To restore the lost data through snapshots, you can take a snapshot at an appropriate time, copy the folders data and wal in the corresponding snapshot directory to its parent directory (at the same level with checkpoints) to overwrite the previous data and wal, and then restart the cluster.

   Warning

   The data and wal directories of all Meta services should be overwritten at the same time. Otherwise, the new leader Meta service will use the latest Meta data after a cluster is restarted.

---

Last update: November 22, 2023