Backup and restore data with snapshots

NebulaGraph supports using snapshots to back up and restore data. When data loss or misoperation occurs, the data will be restored through the snapshot.

Prerequisites

NebulaGraph authentication is disabled by default. In this case, all users can use the snapshot feature.

If authentication is enabled, only the GOD role user can use the snapshot feature. For more information about roles, see Roles and privileges.

Precautions

• To prevent data loss, create a snapshot as soon as the system structure changes, for example, after operations such as ADD HOST, DROP HOST, CREATE SPACE, DROP SPACE, and BALANCE are performed.

• NebulaGraph cannot automatically delete the invalid files created by a failed snapshot task. You have to manually delete them by using DROP SNAPSHOT.

• Customizing the storage path for snapshots is not supported for now. The default path is /usr/local/nebula/data.

Snapshot form and path

NebulaGraph snapshots are stored in the form of directories with names like SNAPSHOT_2021_03_09_08_43_12. The suffix 2021_03_09_08_43_12 is generated automatically based on the creation time (UTC).

When a snapshot is created, snapshot directories will be automatically created in the checkpoints directory on the leader Meta server and each Storage server.

To fast locate the path where the snapshots are stored, you can use the Linux command find. For example:

$ find |grep 'SNAPSHOT_2021_03_09_08_43_12'
./data/meta2/nebula/0/checkpoints/SNAPSHOT_2021_03_09_08_43_12
./data/meta2/nebula/0/checkpoints/SNAPSHOT_2021_03_09_08_43_12/data
./data/meta2/nebula/0/checkpoints/SNAPSHOT_2021_03_09_08_43_12/data/000081.sst
...

Create snapshots

Run CREATE SNAPSHOT to create a snapshot for all the graph spaces based on the current time for NebulaGraph. Creating a snapshot for a specific graph space is not supported yet.

Note

If the creation fails, delete the snapshot and try again.

nebula> CREATE SNAPSHOT;

View snapshots

To view all existing snapshots, run SHOW SNAPSHOTS.

nebula> SHOW SNAPSHOTS;
+--------------------------------+---------+------------------+
| Name                           | Status  | Hosts            |
+--------------------------------+---------+------------------+
| "SNAPSHOT_2021_03_09_08_43_12" | "VALID" | "127.0.0.1:9779" |
| "SNAPSHOT_2021_03_09_09_10_52" | "VALID" | "127.0.0.1:9779" |
+--------------------------------+---------+------------------+

The parameters in the return information are described as follows.

| Parameter | Description | |-----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Name | The name of the snapshot directory. The prefix SNAPSHOT indicates that the file is a snapshot file, and the suffix indicates the time the snapshot was created (UTC). | | Status | The status of the snapshot. VALID indicates that the creation succeeded, while INVALID indicates that it failed. | | Hosts | IP addresses and ports of all Storage servers at the time the snapshot was created. |

Delete snapshots

To delete a snapshot with the given name, run DROP SNAPSHOT.

DROP SNAPSHOT <snapshot_name>;

Example:

nebula> DROP SNAPSHOT SNAPSHOT_2021_03_09_08_43_12;
nebula> SHOW SNAPSHOTS;
+--------------------------------+---------+------------------+
| Name                           | Status  | Hosts            |
+--------------------------------+---------+------------------+
| "SNAPSHOT_2021_03_09_09_10_52" | "VALID" | "127.0.0.1:9779" |
+--------------------------------+---------+------------------+

Restore data with snapshots

Currently, there is no command to restore data with snapshots. You need to manually copy the snapshot file to the corresponding folder, or you can make it by using a shell script. The logic implements as follows:

1. After the snapshot is created, the checkpoints directory is generated in the installation directory of the leader Meta server and all Storage servers, and saves the created snapshot. Taking this topic as an example, when there are two graph spaces, the snapshots created are saved in /usr/local/nebula/data/meta/nebula/0/checkpoints, /usr/local/nebula/data/storage/ nebula/3/checkpoints and /usr/local/nebula/data/storage/nebula/4/checkpoints.

   $ ls /usr/local/nebula/data/meta/nebula/0/checkpoints/
   SNAPSHOT_2021_03_09_09_10_52
   $ ls /usr/local/nebula/data/storage/nebula/3/checkpoints/
   SNAPSHOT_2021_03_09_09_10_52
   $ ls /usr/local/nebula/data/storage/nebula/4/checkpoints/
   SNAPSHOT_2021_03_09_09_10_52
   

2. To restore the lost data through snapshots, you can take a snapshot at an appropriate time, copy the folders data and wal in the corresponding snapshot directory to its parent directory (at the same level with checkpoints) to overwrite the previous data and wal, and then restart the cluster.

   Caution

   The data and wal directories of all Meta servers should be overwritten at the same time. Otherwise, the new leader Meta server will use the latest Meta data after a cluster is restarted.

---

Last update: March 13, 2023