Kernel configurations¶

This topic introduces the Kernel configurations in Nebula Graph.

Resource control¶

You may run the ulimit command to control the resource threshold. However, the changes made only take effect for the current session or sub-process. To make permanent changes, edit file /etc/security/limits.conf. The configuration is as follows:

# <domain>     <type>      <item>          <value>
*              soft         core           unlimited    
*              hard         core           unlimited   
*              soft         nofile          130000   
*              hard         nofile          130000

Note

The configuration modification takes effect for new sessions.

The parameter descriptions are as follows.

Parameter	Description
`domain`	Control Domain. This parameter can be a user name, a user group name (starting with `@`), or `*` to indicate all users.
`type`	Control type. This parameter can be `soft` or `hard`. `soft` indicates a soft threshold (the default threshold) for the resource and `hard` indicates a maximum value that can be set by the user. The `ulimit` command can be used to increase `soft`, but not to exceed `hard`.
`item`	Resource types. For example, `core` limits the size of the core dump file, and `nofile` limits the maximum number of file descriptors a process can open.
`value`	Resource limit value. This parameter can be a number, or `unlimited` to indicate that there is no limit.

You can run man limits.conf for more helpful information.

Memory¶

vm.swappiness¶

vm.swappiness specifies the percentage of the available memory before starting swap. The greater the value, the more likely the swap occurs. We recommend that you set it to 0. When set to 0, the page cache is removed first. Note that when vm.swappiness is 0, it does not mean that there is no swap.

vm.min_free_kbytes¶

vm.min_free_kbytes specifies the minimum number of kilobytes available kept by Linux VM. If you have a large system memory, we recommend that you increase this value. For example, if your physical memory 128GB, set it to 5GB. If the value is not big enough, the system cannot apply for enough continuous physical memory.

vm.max_map_count¶

vm.max_map_count limits the maximum number of vma (virtual memory area) for a process. The default value is 65530. It is enough for most applications. If your memory application fails because the memory consumption is large, increase the vm.max_map_count value.

vm.dirty_*¶

These values control the dirty data cache for the system. For write-intensive scenarios, you can make adjustments based on your needs (throughput priority or delay priority). We recommend that you use the system default value.

Transparent Huge Pages¶

Transparent Huge Pages (THP) is a memory management feature of the Linux kernel, which enhances the system's ability to use large pages. In most database systems, Transparent Huge Pages can degrade performance, so it is recommended to disable it.

Perform the following steps:

Edit the GRUB configuration file /etc/default/grub.
```
sudo vi /etc/default/grub
```
Add transparent_hugepage=never to the GRUB_CMDLINE_LINUX option, and then save and exit.
```
GRUB_CMDLINE_LINUX="... transparent_hugepage=never"
```

Update the GRUB configuration.

For CentOS:

sudo grub2-mkconfig -o /boot/grub2/grub.cfg

For Ubuntu:
```
sudo update-grub
```

Reboot the computer.

sudo reboot

If you don't want to reboot, you can run the following commands to temporarily disable THP until the next reboot.

echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled
echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag

Networking¶

net.ipv4.tcp_slow_start_after_idle¶

The default value of net.ipv4.tcp_slow_start_after_idle is 1. If set, the congestion window is timed out after an idle period. We recommend that you set it to 0, especially for long fat scenarios (high latency and large bandwidth).

net.core.somaxconn¶

net.core.somaxconn specifies the maximum number of connection queues listened by the socket. The default value is 128. For scenarios with a large number of burst connections, we recommend that you set it to greater than 1024.

net.ipv4.tcp_max_syn_backlog¶

net.ipv4.tcp_max_syn_backlog specifies the maximum number of TCP connections in the SYN_RECV (semi-connected) state. The setting rule for this parameter is the same as that of net.core.somaxconn.

net.core.netdev_max_backlog¶

net.core.netdev_max_backlog specifies the maximum number of packets. The default value is 1000. We recommend that you increase it to greater than 10,000, especially for 10G network adapters.

net.ipv4.tcp_keepalive_*¶

These values keep parameters alive for TCP connections. For applications that use a 4-layer transparent load balancer, if the idle connection is disconnected unexpectedly, decrease the values of tcp_keepalive_time and tcp_keepalive_intvl.

net.ipv4.tcp_rmem/wmem¶

net.ipv4.tcp_wmem/rmem specifies the minimum, default, and maximum size of the buffer pool sent/received by the TCP socket. For long fat links, we recommend that you increase the default value to bandwidth (GB) * RTT (ms).

scheduler¶

For SSD devices, we recommend that you set scheduler to noop or none. The path is /sys/block/DEV_NAME/queue/scheduler.

Other parameters¶

kernel.core_pattern¶

we recommend that you set it to core and set kernel.core_uses_pid to 1.

Modify parameters¶

sysctl¶

sysctl <conf_name>

Checks the current parameter value.

sysctl -w <conf_name>=<value>

Modifies the parameter value. The modification takes effect immediately. The original value is restored after restarting.

sysctl -p [<file_path>]

Loads Linux parameter values from the specified configuration file. The default path is /etc/sysctl.conf.

prlimit¶

The prlimit command gets and sets process resource limits. You can modify the hard threshold by using it and the sudo command. For example, prlimit --nofile = 130000 --pid = $$ adjusts the maximum number of open files permitted by the current process to 14000. And the modification takes effect immediately. Note that this command is only available in RedHat 7u or higher versions.

Last update: January 30, 2024