Kernel configurations¶

This topic introduces the Kernel configurations in Nebula Graph.

Resource control¶

ulimit precautions¶

The ulimit command specifies the resource threshold for the current shell session. The precautions are as follows:

• The changes made by ulimit only take effect for the current session or child process.

• The resource threshold (soft threshold) cannot exceed the hard threshold.

• Common users cannot use commands to adjust the hard threshold, even with sudo.

• To modify the system level or adjust the hard threshold, edit the file /etc/security/limits.conf. This method requires re-login to take effect.

ulimit -c¶

ulimit -c limits the size of the core dumps. We recommend that you set it to unlimited. The command is:

ulimit -c unlimited

ulimit -n¶

ulimit -n limits the number of open files. We recommend that you set it to more than 100,000. For example:

ulimit -n 130000

Memory¶

vm.swappiness¶

vm.swappiness specifies the percentage of the available memory before starting swap. The greater the value, the more likely the swap occurs. We recommend that you set it to 0. When set to 0, the page cache is removed first. Note that when vm.swappiness is 0, it does not mean that there is no swap.

vm.min_free_kbytes¶

vm.min_free_kbytes specifies the minimum number of kilobytes available kept by Linux VM. If you have a large system memory, we recommend that you increase this value. For example, if your physical memory 128GB, set it to 5GB. If the value is not big enough, the system cannot apply for enough continuous physical memory.

vm.max_map_count¶

vm.max_map_count limits the maximum number of vma (virtual memory area) for a process. The default value is 65530. It is enough for most applications. If your memory application fails because the memory consumption is large, increase the vm.max_map_count value.

vm.dirty_*¶

These values control the dirty data cache for the system. For write-intensive scenarios, you can make adjustments based on your needs (throughput priority or delay priority). We recommend that you use the system default value.

Transparent huge page¶

For better delay performance, you must disable the transparent huge pages (THP). The command is:

root# echo never > /sys/kernel/mm/transparent_hugepage/enabled
root# echo never > /sys/kernel/mm/transparent_hugepage/defrag
root# swapoff -a && swapon -a

Networking¶

net.ipv4.tcp_slow_start_after_idle¶

The default value of net.ipv4.tcp_slow_start_after_idle is 1. If set, the congestion window is timed out after an idle period. We recommend that you set it to 0, especially for long fat scenarios (high latency and large bandwidth).

net.core.somaxconn¶

net.core.somaxconn specifies the maximum number of connection queues listened by the socket. The default value is 128. For scenarios with a large number of burst connections, we recommend that you set it to greater than 1024.

net.ipv4.tcp_max_syn_backlog¶

net.ipv4.tcp_max_syn_backlog specifies the maximum number of TCP connections in the SYN_RECV (semi-connected) state. The setting rule for this parameter is the same as that of net.core.somaxconn.

net.core.netdev_max_backlog¶

net.core.netdev_max_backlog specifies the maximum number of packets. The default value is 1000. We recommend that you increase it to greater than 10,000, especially for 10G network adapters.

net.ipv4.tcp_keepalive_*¶

These values keep parameters alive for TCP connections. For applications that use a 4-layer transparent load balancer, if the idle connection is disconnected unexpectedly, decrease the values of tcp_keepalive_time and tcp_keepalive_intvl.

net.ipv4.tcp_rmem/wmem¶

net.ipv4.tcp_wmem/rmem specifies the minimum, default, and maximum size of the buffer pool sent/received by the TCP socket. For long fat links, we recommend that you increase the default value to bandwidth (GB) * RTT (ms).

scheduler¶

For SSD devices, we recommend that you set scheduler to noop or none. The path is /sys/block/DEV_NAME/queue/scheduler.

Other parameters¶

kernel.core_pattern¶

we recommend that you set it to core and set kernel.core_uses_pid to 1.

Modify parameters¶

sysctl¶

• sysctl <conf_name>

  Checks the current parameter value.

• sysctl -w <conf_name>=<value>

  Modifies the parameter value. The modification takes effect immediately. The original value is restored after restarting.

• sysctl -p [<file_path>] 

  Loads Linux parameter values ​​from the specified configuration file. The default path is /etc/sysctl.conf.

prlimit¶

The prlimit command gets and sets process resource limits. You can modify the hard threshold by using it and the sudo command. For example, prlimit --nofile = 130000 --pid = $$ adjusts the maximum number of open files permitted by the current process to 14000. And the modification takes effect immediately. Note that this command is only available in RedHat 7u or higher versions.