Deploy Raft Listener for NebulaGraph Storage service¶

Full-text index data is written to the Elasticsearch cluster asynchronously. The Raft Listener (Listener for short) is a separate process that fetches data from the Storage Service and writes them into the Elasticsearch cluster.

Prerequisites¶

You have read and fully understood the restrictions for using full-text indexes.

You have deployed a NebulaGraph cluster.

You have deploy a Elasticsearch cluster.

You have prepared at least one extra Storage Server. To use the full-text search, you must run one or more Storage Server as the Raft Listener.

Precautions¶

The Storage Service that you want to run as the Listener must have the same or later release with all the other Nebula Graph services in the cluster.

For now, you can only add all Listeners to a graph space once and for all. Trying to add a new Listener to a graph space that already has a Listener will fail. To add all Listeners, set them in one statement.

Deployment process¶

Step 1: Install the Storage service¶

The Listener process and the storaged process use the same binary file. However, their configuration files and using ports are different. You can install NebulaGraph on all servers that need to deploy a Listener, but only the Storage service can be used. For details, see Install NebulaGraph by RPM or DEB Package.

Step 2: Prepare the configuration file for the Listener¶

You have to prepare a corresponding configuration file on the machine that you want to deploy a Listener. The file must be named as nebula-storaged-listener.conf and stored in the etc directory. A template is provided for your reference. Note that the file suffix .production should be removed.

Most configurations are the same as the configurations of Storage Service. This topic only introduces the differences.

Name	Default value	Description
`daemonize`	`true`	When set to `true`, the process is a daemon process.
`pid_file`	`pids_listener/nebula-storaged.pid`	The file that records the process ID.
`meta_server_addrs`	-	IP addresses and ports of all Meta services. Multiple Meta services are separated by commas.
`local_ip`	-	The local IP address of the Listener service.
`port`	-	The listening port of the RPC daemon of the Listener service.
`heartbeat_interval_secs`	`10`	The heartbeat interval of the Meta service. The unit is second (s).
`listener_path`	`data/listener`	The WAL directory of the Listener. Only one directory is allowed.
`data_path`	`data`	For compatibility reasons, this parameter can be ignored. Fill in the default value `data`.
`part_man_type`	`memory`	The type of the part manager. Optional values are `memory` and `meta`.
`rocksdb_batch_size`	`4096`	The default reserved bytes for batch operations.
`rocksdb_block_cache`	`4`	The default block cache size of BlockBasedTable. The unit is Megabyte (MB).
`engine_type`	`rocksdb`	The type of the Storage engine, such as `rocksdb`, `memory`, etc.
`part_type`	`simple`	The type of the part, such as `simple`, `consensus`, etc.

Note

Use real IP addresses in the configuration file instead of domain names or loopback IP addresses such as 127.0.0.1.

Step 3: Start Listeners¶

Run the following command to start the Listener.

./bin/nebula-storaged --flagfile <listener_config_path>/nebula-storaged-listener.conf

${listener_config_path} is the path where you store the Listener configuration file.

Step 4: Add Listeners to NebulaGraph¶

Connect to NebulaGraph and run USE <space> to enter the graph space that you want to create full-text indexes for. Then run the following statement to add a Listener into NebulaGraph.

ADD LISTENER ELASTICSEARCH <listener_ip:port> [,<listener_ip:port>, ...]

Warning

You must use real IPs for a Listener.

Add all Listeners in one statement completely.

nebula> ADD LISTENER ELASTICSEARCH 192.168.8.5:9789,192.168.8.6:9789;

Show Listeners¶

Run the SHOW LISTENER statement to list all Listeners.

Example¶

nebula> SHOW LISTENER;
+--------+-----------------+-----------------------+----------+
| PartId | Type            | Host                  | Status   |
+--------+-----------------+-----------------------+----------+
| 1      | "ELASTICSEARCH" | "[192.168.8.5:46780]" | "ONLINE" |
| 2      | "ELASTICSEARCH" | "[192.168.8.5:46780]" | "ONLINE" |
| 3      | "ELASTICSEARCH" | "[192.168.8.5:46780]" | "ONLINE" |
+--------+-----------------+-----------------------+----------+

Remove Listeners¶

Run the REMOVE LISTENER ELASTICSEARCH statement to remove all Listeners in a graph space.

Example¶

nebula> REMOVE LISTENER ELASTICSEARCH;

Danger

After the Listener is deleted, it cannot be added again. Therefore, the synchronization to the ES cluster cannot be continued and the text index data will be incomplete. If needed, you can only recreate the graph space.

Next¶

After deploying the Elasticsearch cluster and the Listener, full-text indexes are created automatically on the Elasticsearch cluster. Users can do full-text search now. For more information, see Full-Text search.

Last update: February 1, 2023