Skip to content

Storage Service configurations

NebulaGraph provides two initial configuration files for the Storage Service, nebula-storaged.conf.default and nebula-storaged.conf.production. Users can use them in different scenarios conveniently. The default file path is /usr/local/nebula/etc/.

Caution

  • It is not recommended to modify the value of local_config to false. If modified, the NebulaGraph service will first read the cached configurations, which may cause configuration inconsistencies between clusters and cause unknown risks.
  • It is not recommended to modify the configurations that are not introduced in this topic, unless you are familiar with the source code and fully understand the function of configurations.

How to use the configuration files

To use the initial configuration file, choose one of the above two files and delete the suffix .default or .production from the initial configuration file for the Meta Service to apply the configurations defined in it.

About parameter values

If a parameter is not set in the configuration file, NebulaGraph uses the default value. Not all parameters are predefined. And the predefined parameters in the two initial configuration files are different. This topic uses the parameters in nebula-metad.conf.default. For parameters that are not included in nebula-metad.conf.default, see nebula-storaged.conf.production.

Note

The configurations of the Raft Listener and the Storage service are different. For details, see Deploy Raft listener.

For all parameters and their current values, see Configurations.

Basics configurations

Name Predefined value Description
daemonize true When set to true, the process is a daemon process.
pid_file pids/nebula-storaged.pid The file that records the process ID.
timezone_name - Specifies the NebulaGraph time zone. This parameter is not predefined in the initial configuration files. The system default value is UTC+00:00:00. For the format of the parameter value, see Specifying the Time Zone with TZ. For example, --timezone_name=UTC+08:00 represents the GMT+8 time zone.
local_config true When set to true, the process gets configurations from the configuration files.

Note

  • While inserting property values of time types, NebulaGraph transforms time types (except TIMESTAMP) to the corresponding UTC according to the time zone specified by timezone_name. The time-type values returned by nGQL queries are all UTC.
  • timezone_name is only used to transform the data stored in NebulaGraph. Other time-related data of the NebulaGraph processes still uses the default time zone of the host, such as the log printing time.

Logging configurations

Name Predefined value Description
log_dir logs The directory that stores the Meta Service log. It is recommended to put logs on a different hard disk from the data.
minloglevel 0 Specifies the minimum level of the log. That is, no logs below this level will be printed. Optional values are 0 (INFO), 1 (WARNING), 2 (ERROR), 3 (FATAL). It is recommended to set it to 0 during debugging and 1 in a production environment. If it is set to 4, NebulaGraph will not print any logs.
v 0 Specifies the detailed level of the log. The larger the value, the more detailed the log is. Optional values are 0, 1, 2, 3.
logbufsecs 0 Specifies the maximum time to buffer the logs. If there is a timeout, it will output the buffered log to the log file. 0 means real-time output. This configuration is measured in seconds.
redirect_stdout true When set to true, the process redirects thestdout and stderr to separate output files.
stdout_log_file graphd-stdout.log Specifies the filename for the stdout log.
stderr_log_file graphd-stderr.log Specifies the filename for the stderr log.
stderrthreshold 2 Specifies the minloglevel to be copied to the stderr log.

Networking configurations

Name Predefined value Description
meta_server_addrs 127.0.0.1:9559 Specifies the IP addresses and ports of all Meta Services. Multiple addresses are separated with commas.
local_ip 127.0.0.1 Specifies the local IP for the Storage Service. The local IP address is used to identify the nebula-storaged process. If it is a distributed cluster or requires remote access, modify it to the corresponding address.
port 9779 Specifies RPC daemon listening port of the Storage service. The external port for the Meta Service is predefined to 9779. The internal port is predefined to 9777, 9778, and 9780. Nebula Graph uses the internal port for multi-replica interactions.
ws_ip 0.0.0.0 Specifies the IP address for the HTTP service.
ws_http_port 19779 Specifies the port for the HTTP service.
ws_h2_port 19780 Specifies the port for the HTTP2 service.
heartbeat_interval_secs 10 Specifies the default heartbeat interval. Make sure the heartbeat_interval_secs values for all services are the same, otherwise NebulaGraph CANNOT work normally. This configuration is measured in seconds.

Caution

The real IP address must be used in the configuration file. Otherwise, 127.0.0.1/0.0.0.0 cannot be parsed correctly in some cases.

Raft configurations

Name Predefined value Description
raft_heartbeat_interval_secs 30 Specifies the time to expire the Raft election. The configuration is measured in seconds.
raft_rpc_timeout_ms 500 Specifies the time to expire the Raft RPC. The configuration is measured in milliseconds.
wal_ttl 14400 Specifies the lifetime of the RAFT WAL. The configuration is measured in seconds.

Disk configurations

Name Predefined value Description
data_path data/storage Specifies the data storage path. Multiple paths are separated with commas. One RocksDB example corresponds to one path.
minimum_reserved_bytes 268435456 Specifies the minimum remaining space of each data storage path. When the value is lower than this standard, the cluster data writing may fail. This configuration is measured in bytes. The default value is 1073741824, namely, 1GB.
rocksdb_batch_size 4096 Specifies the block cache for a batch operation. The configuration is measured in bytes.
rocksdb_block_cache 4 Specifies the block cache for BlockBasedTable. The configuration is measured in megabytes.
engine_type rocksdb Specifies the engine type.
rocksdb_compression lz4 Specifies the compression algorithm for RocksDB. Optional values are no, snappy, lz4, lz4hc, zlib, bzip2, and zstd.
rocksdb_compression_per_level \ Specifies the compression algorithm for each level.
enable_rocksdb_statistics false When set to false, RocksDB statistics is disabled.
rocksdb_stats_level kExceptHistogramOrTimers Specifies the stats level for RocksDB. Optional values are kExceptHistogramOrTimers, kExceptTimers, kExceptDetailedTimers, kExceptTimeForMutex, and kAll.
enable_rocksdb_prefix_filtering true When set to true, the prefix bloom filter for RocksDB is enabled. Enabling prefix bloom filter makes the graph traversal faster but occupies more memory.
enable_rocksdb_whole_key_filtering false When set to true, the whole key bloom filter for RocksDB is enabled.
rocksdb_filtering_prefix_length 12 Specifies the prefix length for each key. Optional values are 12 and 16. The configuration is measured in bytes.
enable_partitioned_index_filter - When set to true, it reduces the amount of memory used by the bloom filter. But in some random-seek situations, it may reduce the read performance.

misc configurations

Caution

The configuration snapshot in the following table is different from the snapshot in NebulaGraph. The snapshot here refers to the stock data on the leader when synchronizing Raft.

Name Predefined value Description
snapshot_part_rate_limit 8388608 The rate limit when the Raft leader synchronizes the stock data with other members of the Raft group. Unit: bytes/s.
snapshot_batch_size 1048576 The amount of data sent in each batch when the Raft leader synchronizes the stock data with other members of the Raft group. Unit: bytes.
rebuild_index_part_rate_limit 4194304 The rate limit when the Raft leader synchronizes the index data rate with other members of the Raft group during the index rebuilding process. Unit: bytes/s.
rebuild_index_batch_size 1048576 The amount of data sent in each batch when the Raft leader synchronizes the index data with other members of the Raft group during the index rebuilding process. Unit: bytes.

RocksDB options

Name Predefined value Description
rocksdb_db_options {} Specifies the RocksDB database options.
rocksdb_column_family_options {"write_buffer_size":"67108864",
"max_write_buffer_number":"4",
"max_bytes_for_level_base":"268435456"}
Specifies the RocksDB column family options.
rocksdb_block_based_table_options {"block_size":"8192"} Specifies the RocksDB block based table options.

The format of the RocksDB option is {"<option_name>":"<option_value>"}. Multiple options are separated with commas.

Supported options of rocksdb_db_options and rocksdb_column_family_options are listed as follows.

  • rocksdb_db_options
    max_total_wal_size
    delete_obsolete_files_period_micros
    max_background_jobs
    stats_dump_period_sec
    compaction_readahead_size
    writable_file_max_buffer_size
    bytes_per_sync
    wal_bytes_per_sync
    delayed_write_rate
    avoid_flush_during_shutdown
    max_open_files
    stats_persist_period_sec
    stats_history_buffer_size
    strict_bytes_per_sync
    enable_rocksdb_prefix_filtering
    enable_rocksdb_whole_key_filtering
    rocksdb_filtering_prefix_length
    num_compaction_threads
    rate_limit
    
  • rocksdb_column_family_options
    write_buffer_size
    max_write_buffer_number
    level0_file_num_compaction_trigger
    level0_slowdown_writes_trigger
    level0_stop_writes_trigger
    target_file_size_base
    target_file_size_multiplier
    max_bytes_for_level_base
    max_bytes_for_level_multiplier
    disable_auto_compactions 
    

For more information, see RocksDB official documentation.

For super-Large vertices

When the query starting from each vertex gets an edge, truncate it directly to avoid too many neighboring edges on the super-large vertex, because a single query occupies too much hard disk and memory. Or you can truncate a certain number of edges specified in the Max_edge_returned_per_vertex parameter. Excess edges will not be returned. This parameter applies to all spaces.

Property name Default value Description
max_edge_returned_per_vertex 2147483647 Specifies the maximum number of edges returned for each dense vertex. Excess edges are truncated and not returned. This parameter is not predefined in the configuration files.

Compatibility

The reservoir sampling algorithm in NebulaGraph 1.x is no longer supported in NebulaGraph 2.6.2.

Storage configurations for large dataset

When you have a large dataset (in the RocksDB directory) and your memory is tight, we suggest that you set the enable_partitioned_index_filter parameter to true. The performance is affected because RocksDB indexes are cached.


Last update: March 13, 2023