Kernel Configuration Reference¶

This section contains a reference of Nebula Graph system configuration settings.

ulimit¶

ulimit -c¶

It limits the size of core dumps. The recommended setting is unlimited, i.e. ulimit -c unlimited.

ulimit -n¶

It limits a user by number of opened files (file descriptors). The recommended setting is greater than 100,000. For example, ulimit -n 130000.

Memory¶

vm.swappiness¶

It controls the relative weight given to swapping out of runtime memory, as opposed to dropping memory pages from the system page cache. The lower the value, the less swapping is used and the more memory pages are kept in physical memory. The recommended setting is 0. Note that 0 does not mean that there is no swapping out.

vm.min_free_kbytes¶

It is used to force the Linux VM to keep a minimum number of kilobytes free. If the system physical memory is large, it is recommended to increase this value (for example, if your physical memory 128GB, set it to 5GB). Otherwise, when there is not enough memory, the system cannot apply for large continuous physical memory.

vm.max_map_count¶

It contains the maximum number of vma (virtual memory area) a process may have. The default value is 65530. If the memory application fails when the memory consumption is large, you can increase it appropriately.

vm.overcommit_memory¶

This value contains a flag that enables memory overcommitment. It is recommended to set it to the default value 0 or 1. Do not set it to 2.

vm.dirty_*¶

These values contain the the aggressiveness of the dirty page cache that controls the system. For write-intensive scenarios, you can make adjustments based on your needs (throughput priority or delay priority). It is recommended to use the system default.

Transparent Huge Page¶

Transparent Huge Pages (THP) must be disabled for better delay performance. The options are /sys/kernel/mm/transparent_hugepage/enabled and/sys/kernel/mm/transparent_hugepage/defrag. For example:

echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag
swapoff -a && swapon -a

Networking¶

net.ipv4.tcp_slow_start_after_idle¶

If set to the default value 1, it will time out the congestion window after an idle period. It is recommended to set to 0, especially for long fat links (high latency and large bandwidth).

net.core.somaxconn¶

This option, together with the backlog parameter called by the listen system, specifies the upper limit on how many connections the kernel will accept. For scenarios where a large number of burst connections are established, it is recommended to set it above 1024 (the default value is 128).

net.ipv4.tcp_max_syn_backlog¶

Maximum number of remembered connection requests. The setting rule is the same as net.core.somaxconn.

net.core.netdev_max_backlog¶

It determines the maximum number of packets. It is suggested raising this value to over 10,000 for high-throughput scenarios, especially 10G network adapters. The default value is 1,000.

net.ipv4.tcp_keepalive_*¶

Parameters to enable the TCP keep alive feature. For applications that use a 4-layer transparent load balancer, if the idle connection is unexpectedly disconnected, you can decrease tcp_keepalive_time and tcp_keepalive_intvl.

net.ipv4.tcp_rmem/wmem¶

They represent the minimum, default and maximum size of the TCP socket receive buffer. For long fat links, it is recommended to increase the default value to bandwidth * RTT.

scheduler¶

For SSD devices, it is recommended to set /sys/block/DEV_NAME/queue/scheduler to noop or none.

Other Parameters¶

kernel.core_pattern¶

It is recommended to set it to core and set kernel.core_uses_pid to 1.

Parameter Usage Guide¶

sysctl¶

sysctl conf_name checks the current parameter value
sysctl -w conf_name=value modifies the parameter value and takes effect immediately
sysctl -p loads parameter values from relevant configuration files

Introduction to ulimit¶

ulimit provides control over the resources available to the shell and to processes started by it, on systems that allow such control. Please note that:

Changes made by the ulimit command are only valid for the current session (and child processes).
ulimit cannot adjust the (soft) threshold of a resource to be greater than the current hard value.
Ordinary users cannot adjust the hard threshold (even using sudo ) through this command.
To modify on the system level, or adjust the hard threshold, you need to edit the /etc/security/limits.conf file. But this method needs to log in again to take effect.

prlimit¶

prlimit tries to retrieve and/or modify the limits to a specified process. Together with the sudo command, the hard threshold can be modified. For example, prlimit --nofile = 130000 --pid = $$ can adjust the maximum number of open files allowed by the current process to 14000 and take effect immediately. Please note that this command is only available in RedHat 7u or later OS versions.