Storage Service configurations¶

NebulaGraph provides two initial configuration files for the Storage Service, nebula-storaged.conf.default and nebula-storaged.conf.production. Users can use them in different scenarios conveniently. The default file path is /usr/local/nebula/etc/.

Caution

It is not recommended to modify the value of local_config to false. If modified, the NebulaGraph service will first read the cached configurations, which may cause configuration inconsistencies between clusters and cause unknown risks.
It is not recommended to modify the configurations that are not introduced in this topic, unless you are familiar with the source code and fully understand the function of configurations.

How to use the configuration files¶

To use the initial configuration file, choose one of the above two files and delete the suffix .default or .production from the initial configuration file for the Meta Service to apply the configurations defined in it.

About parameter values¶

If a parameter is not set in the configuration file, NebulaGraph uses the default value. Not all parameters are predefined. And the predefined parameters in the two initial configuration files are different. This topic uses the parameters in nebula-metad.conf.default. For parameters that are not included in nebula-metad.conf.default, see nebula-storaged.conf.production.

Note

The configurations of the Raft Listener and the Storage service are different. For details, see Deploy Raft listener.

For all parameters and their current values, see Configurations.

Basics configurations¶

Name	Predefined value	Description
`daemonize`	`true`	When set to `true`, the process is a daemon process.
`pid_file`	`pids/nebula-storaged.pid`	The file that records the process ID.
`timezone_name`	-	Specifies the NebulaGraph time zone. This parameter is not predefined in the initial configuration files. The system default value is `UTC+00:00:00`. For the format of the parameter value, see Specifying the Time Zone with TZ. For example, `--timezone_name=UTC+08:00` represents the GMT+8 time zone.
`local_config`	`true`	When set to `true`, the process gets configurations from the configuration files.

Note

While inserting property values of time types, NebulaGraph transforms time types (except TIMESTAMP) to the corresponding UTC according to the time zone specified by timezone_name. The time-type values returned by nGQL queries are all UTC.
timezone_name is only used to transform the data stored in NebulaGraph. Other time-related data of the NebulaGraph processes still uses the default time zone of the host, such as the log printing time.

Logging configurations¶

Name	Predefined value	Description
`log_dir`	`logs`	The directory that stores the Meta Service log. It is recommended to put logs on a different hard disk from the data.
`minloglevel`	`0`	Specifies the minimum level of the log. That is, no logs below this level will be printed. Optional values are `0` (INFO), `1` (WARNING), `2` (ERROR), `3` (FATAL). It is recommended to set it to `0` during debugging and `1` in a production environment. If it is set to `4`, NebulaGraph will not print any logs.
`v`	`0`	Specifies the detailed level of the log. The larger the value, the more detailed the log is. Optional values are `0`, `1`, `2`, `3`.
`logbufsecs`	`0`	Specifies the maximum time to buffer the logs. If there is a timeout, it will output the buffered log to the log file. `0` means real-time output. This configuration is measured in seconds.
`redirect_stdout`	`true`	When set to `true`, the process redirects the`stdout` and `stderr` to separate output files.
`stdout_log_file`	`graphd-stdout.log`	Specifies the filename for the `stdout` log.
`stderr_log_file`	`graphd-stderr.log`	Specifies the filename for the `stderr` log.
`stderrthreshold`	`2`	Specifies the `minloglevel` to be copied to the `stderr` log.
`timestamp_in_logfile_name`	`true`	Specifies if the log file name contains a timestamp. `true` indicates yes, `false` indicates no.

Networking configurations¶

Name	Predefined value	Description
`meta_server_addrs`	`127.0.0.1:9559`	Specifies the IP addresses and ports of all Meta Services. Multiple addresses are separated with commas.
`local_ip`	`127.0.0.1`	Specifies the local IP for the Storage Service. The local IP address is used to identify the nebula-storaged process. If it is a distributed cluster or requires remote access, modify it to the corresponding address.
`port`	`9779`	Specifies RPC daemon listening port of the Storage service. The external port for the Meta Service is predefined to `9779`. The internal port is predefined to `9777`, `9778`, and `9780`. Nebula Graph uses the internal port for multi-replica interactions.
`ws_ip`	`0.0.0.0`	Specifies the IP address for the HTTP service.
`ws_http_port`	`19779`	Specifies the port for the HTTP service.
`heartbeat_interval_secs`	`10`	Specifies the default heartbeat interval. Make sure the `heartbeat_interval_secs` values for all services are the same, otherwise NebulaGraph CANNOT work normally. This configuration is measured in seconds.

Caution

The real IP address must be used in the configuration file. Otherwise, 127.0.0.1/0.0.0.0 cannot be parsed correctly in some cases.

Raft configurations¶

Name	Predefined value	Description
`raft_heartbeat_interval_secs`	`30`	Specifies the time to expire the Raft election. The configuration is measured in seconds.
`raft_rpc_timeout_ms`	`500`	Specifies the time to expire the Raft RPC. The configuration is measured in milliseconds.
`wal_ttl`	`14400`	Specifies the lifetime of the RAFT WAL. The configuration is measured in seconds.

Disk configurations¶

Name	Predefined value	Description
`data_path`	`data/storage`	Specifies the data storage path. Multiple paths are separated with commas. One RocksDB example corresponds to one path.
`minimum_reserved_bytes`	`268435456`	Specifies the minimum remaining space of each data storage path. When the value is lower than this standard, the cluster data writing may fail. This configuration is measured in bytes.
`rocksdb_batch_size`	`4096`	Specifies the block cache for a batch operation. The configuration is measured in bytes.
`rocksdb_block_cache`	`4`	Specifies the block cache for BlockBasedTable. The configuration is measured in megabytes.
`disable_page_cache`	`false`	Enables or disables the operating system's page cache for NebulaGraph. By default, the parameter value is `false` and page cache is enabled. If the value is set to `true`, page cache is disabled and sufficient block cache space must be configured for NebulaGraph.
`engine_type`	`rocksdb`	Specifies the engine type.
`rocksdb_compression`	`lz4`	Specifies the compression algorithm for RocksDB. Optional values are `no`, `snappy`, `lz4`, `lz4hc`, `zlib`, `bzip2`, and `zstd`.
`rocksdb_compression_per_level`	\	Specifies the compression algorithm for each level.
`enable_rocksdb_statistics`	`false`	When set to `false`, RocksDB statistics is disabled.
`rocksdb_stats_level`	`kExceptHistogramOrTimers`	Specifies the stats level for RocksDB. Optional values are `kExceptHistogramOrTimers`, `kExceptTimers`, `kExceptDetailedTimers`, `kExceptTimeForMutex`, and `kAll`.
`enable_rocksdb_prefix_filtering`	`true`	When set to `true`, the prefix bloom filter for RocksDB is enabled. Enabling prefix bloom filter makes the graph traversal faster but occupies more memory.
`enable_rocksdb_whole_key_filtering`	`false`	When set to `true`, the whole key bloom filter for RocksDB is enabled.
`rocksdb_filtering_prefix_length`	`12`	Specifies the prefix length for each key. Optional values are `12` and `16`. The configuration is measured in bytes.
`enable_partitioned_index_filter`	-	When set to `true`, it reduces the amount of memory used by the bloom filter. But in some random-seek situations, it may reduce the read performance.

Key-Value separation configurations¶

Name	Predefined value	Description
`rocksdb_enable_kv_separation`	`false`	Whether or not to enable BlobDB (RocksDB key-value separation support). This function improves query performance.
`rocksdb_kv_separation_threshold`	`100`	RocksDB key value separation threshold. Values at or above this threshold will be written to blob files during flush or compaction. Unit: bytes.
`rocksdb_blob_compression`	`lz4`	Compression algorithm for BlobDB. Optional values are `no`, `snappy`, `lz4`, `lz4hc`, `zlib`, `bzip2`, and `zstd`.
`rocksdb_enable_blob_garbage_collection`	`true`	Whether to perform BlobDB garbage collection during compaction.

misc configurations¶

Caution

The configuration snapshot in the following table is different from the snapshot in NebulaGraph. The snapshot here refers to the stock data on the leader when synchronizing Raft.

Name	Predefined value	Description
`auto_remove_invalid_space`	`true`	After executing `DROP SPACE`, the specified graph space will be deleted. This parameter sets whether to delete all the data in the specified graph space at the same time. When the value is `true`, all the data in the specified graph space will be deleted at the same time.
`num_io_threads`	`16`	The number of network I/O threads used to send RPC requests and receive responses.
`num_worker_threads`	`32`	The number of worker threads for one RPC-based Storage service.
`max_concurrent_subtasks`	`10`	The maximum number of concurrent subtasks to be executed by the task manager.
`snapshot_part_rate_limit`	`10485760`	The rate limit when the Raft leader synchronizes the stock data with other members of the Raft group. Unit: bytes/s.
`snapshot_batch_size`	`1048576`	The amount of data sent in each batch when the Raft leader synchronizes the stock data with other members of the Raft group. Unit: bytes.
`rebuild_index_part_rate_limit`	`4194304`	The rate limit when the Raft leader synchronizes the index data rate with other members of the Raft group during the index rebuilding process. Unit: bytes/s.
`rebuild_index_batch_size`	`1048576`	The amount of data sent in each batch when the Raft leader synchronizes the index data with other members of the Raft group during the index rebuilding process. Unit: bytes.

RocksDB options¶

Name	Predefined value	Description
`rocksdb_db_options`	`{}`	Specifies the RocksDB database options.
`rocksdb_column_family_options`	`{"write_buffer_size":"67108864",` `"max_write_buffer_number":"4",` `"max_bytes_for_level_base":"268435456"}`	Specifies the RocksDB column family options.
`rocksdb_block_based_table_options`	`{"block_size":"8192"}`	Specifies the RocksDB block based table options.

The format of the RocksDB option is {"<option_name>":"<option_value>"}. Multiple options are separated with commas.

Supported options of rocksdb_db_options and rocksdb_column_family_options are listed as follows.

rocksdb_db_options

max_total_wal_size
delete_obsolete_files_period_micros
max_background_jobs
stats_dump_period_sec
compaction_readahead_size
writable_file_max_buffer_size
bytes_per_sync
wal_bytes_per_sync
delayed_write_rate
avoid_flush_during_shutdown
max_open_files
stats_persist_period_sec
stats_history_buffer_size
strict_bytes_per_sync
enable_rocksdb_prefix_filtering
enable_rocksdb_whole_key_filtering
rocksdb_filtering_prefix_length
num_compaction_threads
rate_limit

rocksdb_column_family_options

write_buffer_size
max_write_buffer_number
level0_file_num_compaction_trigger
level0_slowdown_writes_trigger
level0_stop_writes_trigger
target_file_size_base
target_file_size_multiplier
max_bytes_for_level_base
max_bytes_for_level_multiplier
disable_auto_compactions

For more information, see RocksDB official documentation.

Storage cache configurations¶

Enterpriseonly

Only available for the NebulaGraph Enterprise Edition.

Name	Predefined value	Description
`enable_storage_cache`	`false`	Whether or not to cache Storage data.
`storage_cache_capacity`	`0`	The size of memory reserved for Storage caches. The value must be slightly greater than the sum of `vertex_pool_capacity` and `empty_key_pool_capacity`. The configuration is measured in MB.
`storage_cache_buckets_power`	`20`	The number of buckets. The value is a logarithm with a base of 2. Optional values are 0~32. For example, the value 20 indicates that the number of buckets is 2\(^{20}\). The recommended value is `ceil(log2(cacheEntries * 1.6))`. `cacheEntries` indicates the total number of cache items.
`storage_cache_locks_power`	`10`	The number of locks. The value is a logarithm with a base of 2. Optional values are 0~32. For example, the value 10 indicates that the number of locks is 2\(^{10}\). The recommended value is `max(1, storage_cache_buckets_power - 10)`.
`enable_vertex_pool`	`false`	Whether or not to add a vertex cache pool. Only valid when the storage cache feature is enabled.
`vertex_pool_capacity`	`50`	The size of the vertex cache pool. The configuration is measured in MB.
`vertex_item_ttl`	`300`	The TTL of vertex cache pool items. The configuration is measured in seconds.
`enable_empty_key_pool`	`false`	Whether or not to add an empty_key pool in the cache. Only valid when the storage cache is enabled. The empty_key indicates a key that was queried but does not actually exist.
`empty_key_pool_capacity`	`50`	The size of the empty_key cache pool. The configuration is measured in MB.
`empty_key_item_ttl`	`300`	The TTL of the empty_key cache pool items. The configuration is measured in seconds.

For super-Large vertices¶

When the query starting from each vertex gets an edge, truncate it directly to avoid too many neighboring edges on the super-large vertex, because a single query occupies too much hard disk and memory. Or you can truncate a certain number of edges specified in the Max_edge_returned_per_vertex parameter. Excess edges will not be returned. This parameter applies to all spaces.

Property name	Default value	Description
max_edge_returned_per_vertex	2147483647	Specifies the maximum number of edges returned for each dense vertex. Excess edges are truncated and not returned. This parameter is not predefined in the configuration files.

Storage configurations for large dataset¶

When you have a large dataset (in the RocksDB directory) and your memory is tight, we suggest that you set the enable_partitioned_index_filter parameter to true. The performance is affected because RocksDB indexes are cached.

Last update: February 1, 2023