Skip to content

Kernel configurations

This topic introduces the Kernel configurations in Nebula Graph.

Resource control

ulimit precautions

The ulimit command specifies the resource threshold for the current shell session. The precautions are as follows:

  • The changes made by ulimit only take effect for the current session or child process.
  • The resource threshold (soft threshold) cannot exceed the hard threshold.
  • Common users cannot use commands to adjust the hard threshold, even with sudo.
  • To modify the system level or adjust the hard threshold, edit the file /etc/security/limits.conf. This method requires re-login to take effect.

ulimit -c

ulimit -c limits the size of the core dumps. We recommend that you set it to unlimited. The command is:

ulimit -c unlimited

ulimit -n

ulimit -n limits the number of open files. We recommend that you set it to more than 100,000. For example:

ulimit -n 130000

Memory

vm.swappiness

vm.swappiness specifies the percentage of the available memory before starting swap. The greater the value, the more likely the swap occurs. We recommend that you set it to 0. When set to 0, the page cache is removed first. Note that when vm.swappiness is 0, it does not mean that there is no swap.

vm.min_free_kbytes

vm.min_free_kbytes specifies the minimum number of kilobytes available kept by Linux VM. If you have a large system memory, we recommend that you increase this value. For example, if your physical memory 128GB, set it to 5GB. If the value is not big enough, the system cannot apply for enough continuous physical memory.

vm.max_map_count

vm.max_map_count limits the maximum number of vma (virtual memory area) for a process. The default value is 65530. It is enough for most applications. If your memory application fails because the memory consumption is large, increase the vm.max_map_count value.

vm.dirty_*

These values control the dirty data cache for the system. For write-intensive scenarios, you can make adjustments based on your needs (throughput priority or delay priority). We recommend that you use the system default value.

Transparent huge page

For better delay performance, you must run the following commands to disable the transparent huge pages (THP).

root# echo never > /sys/kernel/mm/transparent_hugepage/enabled
root# echo never > /sys/kernel/mm/transparent_hugepage/defrag
root# swapoff -a && swapon -a

To prevent THP from being enabled again after the system restarts, you can modify the GRUB configuration file or /etc/rc.local to disable THP automatically upon system startup.

Networking

net.ipv4.tcp_slow_start_after_idle

The default value of net.ipv4.tcp_slow_start_after_idle is 1. If set, the congestion window is timed out after an idle period. We recommend that you set it to 0, especially for long fat scenarios (high latency and large bandwidth).

net.core.somaxconn

net.core.somaxconn specifies the maximum number of connection queues listened by the socket. The default value is 128. For scenarios with a large number of burst connections, we recommend that you set it to greater than 1024.

net.ipv4.tcp_max_syn_backlog

net.ipv4.tcp_max_syn_backlog specifies the maximum number of TCP connections in the SYN_RECV (semi-connected) state. The setting rule for this parameter is the same as that of net.core.somaxconn.

net.core.netdev_max_backlog

net.core.netdev_max_backlog specifies the maximum number of packets. The default value is 1000. We recommend that you increase it to greater than 10,000, especially for 10G network adapters.

net.ipv4.tcp_keepalive_*

These values keep parameters alive for TCP connections. For applications that use a 4-layer transparent load balancer, if the idle connection is disconnected unexpectedly, decrease the values of tcp_keepalive_time and tcp_keepalive_intvl.

net.ipv4.tcp_rmem/wmem

net.ipv4.tcp_wmem/rmem specifies the minimum, default, and maximum size of the buffer pool sent/received by the TCP socket. For long fat links, we recommend that you increase the default value to bandwidth (GB) * RTT (ms).

scheduler

For SSD devices, we recommend that you set scheduler to noop or none. The path is /sys/block/DEV_NAME/queue/scheduler.

Other parameters

kernel.core_pattern

we recommend that you set it to core and set kernel.core_uses_pid to 1.

Modify parameters

sysctl

  • sysctl <conf_name>

    Checks the current parameter value.

  • sysctl -w <conf_name>=<value>

    Modifies the parameter value. The modification takes effect immediately. The original value is restored after restarting.

  • sysctl -p [<file_path>] 

    Loads Linux parameter values ​​from the specified configuration file. The default path is /etc/sysctl.conf.

prlimit

The prlimit command gets and sets process resource limits. You can modify the hard threshold by using it and the sudo command. For example, prlimit --nofile = 130000 --pid = $$ adjusts the maximum number of open files permitted by the current process to 14000. And the modification takes effect immediately. Note that this command is only available in RedHat 7u or higher versions.


Last update: May 13, 2022