Skip to content

Deploy Raft Listener for Nebula Storage service

Full-text index data is written to the Elasticsearch cluster asynchronously. The Raft Listener (Listener for short) is a separate process that fetches data from the Storage Service and writes them into the Elasticsearch cluster.

Prerequisites

  • You have read and fully understood the restrictions for using full-text indexes.
  • You have prepared at least one extra Storage Server. To use the full-text search, you must run one or more Storage Server as the Raft Listener.

Precautions

  • The Storage Service that you want to run as the Listener must have the same or later release with all the other Nebula Graph services in the cluster.
  • For now, you can only add all Listeners to a graph space once and for all. Trying to add a new Listener to a graph space that already has a Listener will fail. To add all Listeners, set them in one statement.

Deployment process

Step 1: Install the Storage service

The Listener process and the storaged process use the same binary file. However, their configuration files and using ports are different. You can install NebulaGraph on all servers that need to deploy a Listener, but only the Storage service can be used. For details, see Install NebulaGraph by RPM or DEB Package.

Step 2: Prepare the configuration file for the Listener

You have to prepare a corresponding configuration file on the machine that you want to deploy a Listener. The file must be named as nebula-storaged-listener.conf and stored in the etc directory. A template is provided for your reference. Note that the file suffix .production should be removed.

Most configurations are the same as the configurations of Storage Service. This topic only introduces the differences.

Name Default value Description
daemonize true When set to true, the process is a daemon process.
pid_file pids_listener/nebula-storaged.pid The file that records the process ID.
meta_server_addrs - IP addresses and ports of all Meta services. Multiple Meta services are separated by commas.
local_ip - The local IP address of the Listener service.
port - The listening port of the RPC daemon of the Listener service.
heartbeat_interval_secs 10 The heartbeat interval of the Meta service. The unit is second (s).
listener_path data/listener The WAL directory of the Listener. Only one directory is allowed.
data_path data For compatibility reasons, this parameter can be ignored. Fill in the default value data.
part_man_type memory The type of the part manager. Optional values ​​are memory and meta.
rocksdb_batch_size 4096 The default reserved bytes for batch operations.
rocksdb_block_cache 4 The default block cache size of BlockBasedTable. The unit is Megabyte (MB).
engine_type rocksdb The type of the Storage engine, such as rocksdb, memory, etc.
part_type simple The type of the part, such as simple, consensus, etc.

Note

Use real IP addresses in the configuration file instead of domain names or loopback IP addresses such as 127.0.0.1.

Step 3: Start Listeners

Run the following command to start the Listener.

./bin/nebula-storaged --flagfile <listener_config_path>/nebula-storaged-listener.conf

${listener_config_path} is the path where you store the Listener configuration file.

Step 4: Add Listeners to NebulaGraph

Connect to NebulaGraph and run USE <space> to enter the graph space that you want to create full-text indexes for. Then run the following statement to add a Listener into NebulaGraph.

ADD LISTENER ELASTICSEARCH <listener_ip:port> [,<listener_ip:port>, ...]

Warning

You must use real IPs for a Listener.

Add all Listeners in one statement completely.

nebula> ADD LISTENER ELASTICSEARCH 192.168.8.5:9789,192.168.8.6:9789;

Show Listeners

Run the SHOW LISTENER statement to list all Listeners.

Example

nebula> SHOW LISTENER;
+--------+-----------------+-----------------------+----------+
| PartId | Type            | Host                  | Status   |
+--------+-----------------+-----------------------+----------+
| 1      | "ELASTICSEARCH" | "[192.168.8.5:46780]" | "ONLINE" |
| 2      | "ELASTICSEARCH" | "[192.168.8.5:46780]" | "ONLINE" |
| 3      | "ELASTICSEARCH" | "[192.168.8.5:46780]" | "ONLINE" |
+--------+-----------------+-----------------------+----------+

Remove Listeners

Run the REMOVE LISTENER ELASTICSEARCH statement to remove all Listeners in a graph space.

Example

nebula> REMOVE LISTENER ELASTICSEARCH;

Danger

After the Listener is deleted, it cannot be added again. Therefore, the synchronization to the ES cluster cannot be continued and the text index data will be incomplete. If needed, you can only recreate the graph space.

Next

After deploying the Elasticsearch cluster and the Listener, full-text indexes are created automatically on the Elasticsearch cluster. Users can do full-text search now. For more information, see Full-Text search.

Last update: March 13, 2023
Back to top