Kernel configurations¶

This document gives some introductions to the Kernel configurations in Nebula Graph.

ulimit¶

ulimit -c¶

ulimit -c limits the size of the core dumps. We recommend that you set it to unlimited. The command is:

ulimit -c unlimited

ulimit -n¶

ulimit -n limits the number of open files. We recommend that you set it to more than 100,000. For example:

ulimit -n 130000

Memory¶

vm.swappiness¶

vm.swappiness is the percentage of the free memory before starting swap. The greater the value, the more likely the swap occurs. We recommend that you set it to 0. When set to 0, the page cache is removed first. Note that when vm.swappiness is 0, it does not mean that there is no swap.

vm.min_free_kbytes¶

vm.min_free_kbytes is used to force the Linux VM to keep a minimum number of kilobytes free. If you have a large system memory, we recommend that you increase this value. For example, if your physical memory 128GB, set it to 5GB. If the value is not big enough, the system cannot apply for enough continuous physical memory.

vm.max_map_count¶

vm.max_map_count limits the maximum number of vma (virtual memory area) for a process. The default value is 65530. It is enough for most applications. If your memory application fails because the memory consumption is large, increase the vm.max_map_count value.

vm.overcommit_memory¶

vm.overcommit_memory contains a flag that enables memory overcommitment. We recommend that you set the default value 0 or 1. DO NOT set it to 2.

vm.dirty_*¶

These values control the aggressiveness of the dirty page cache for the system. For write-intensive scenarios, you can make adjustments based on your needs (throughput priority or delay priority). We recommend that you use the system default value.

Transparent huge page¶

For better delay performance, you must delete the transparent huge pages (THP). The options are /sys/kernel/mm/transparent_hugepage/enabled and/sys/kernel/mm/transparent_hugepage/defrag. For example:

echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag
swapoff -a && swapon -a

Networking¶

net.ipv4.tcp_slow_start_after_idle¶

The default value for this parameter is 1. If set, the congestion window is timed out after an idle period. We recommend that you set it to 0, especially for long fat links (high latency and large bandwidth).

net.core.somaxconn¶

net.core.somaxconn is the maximum number of the backlogged sockets. The default value is 128. For scenarios with a large number of burst connections, we recommend that you set it to greater than 1024.

net.ipv4.tcp_max_syn_backlog¶

The maximum number of remembered connection requests. The setting rule for this parameter is the same as that of net.core.somaxconn.

net.core.netdev_max_backlog¶

It determines the maximum number of packets. We recommend that you increase it to greater than 10,000, especially for 10G network adapters. The default value is 1000.

net.ipv4.tcp_keepalive_*¶

Keep alive parameters for the TCP connections. For applications that use a 4-layer transparent load balancer, if the idle connection is disconnected unexpectedly, decrease tcp_keepalive_time and tcp_keepalive_intvl.

net.ipv4.tcp_rmem/wmem¶

The minimum, default, and maximum size of the TCP socket receive buffer. For long fat links, we recommend that you increase the default value to bandwidth * RTT.

scheduler¶

For SSD devices, we recommend that you set /sys/block/DEV_NAME/queue/scheduler to noop or none.

Other parameters¶

kernel.core_pattern¶

we recommend that you set it to core and set kernel.core_uses_pid to 1.

Parameter usage guide¶

sysctl¶

• sysctl conf_name checks the current parameter value.
• sysctl -w conf_name=value modifies the parameter value. And your modification takes effect immediately.
• sysctl -p  loads parameter values ​​from related configuration files.

Introduction to ulimit¶

ulimit sets the resource threshold for the current shell session. Please note that:

• Changes made by the ulimit command are valid only for the current session (and child processes).
• ulimit cannot adjust the (soft) threshold of a resource to a value greater than the current hard value.
• Ordinary users cannot adjust the hard threshold (even by using sudo ) through this command.
• To modify on the system level, or adjust the hard threshold, edit the /etc/security/limits.conf file. But this method needs to re-log in to take effect.

prlimit¶

prlimit gets and sets process resource limits. You can modify the hard threshold by using it and the sudo command. Together with the sudo command, the hard threshold can be modified. For example, prlimit --nofile = 130000 --pid = $$ adjusts the maximum number of open files permitted by the current process to 14000. And the modification takes effect immediately. Note that this command is only available in RedHat 7u or later OS versions.