Skip to content

Build From Source Code

Overview

We have tested building on various environments, including CentOS 6 to 8, Ubuntu 16.04 to 19.04, Fedora 28 to 30, GCC 7.1.0 to 9.2.0 and recent Clang++ and the devtoolset of Red Hat and CentOS. But due to the complexity of building environments, we still cannot guarantee that we have covered all kinds of situations. If any problem encountered, please fire an issue or open a pull request to let us know.

Requirements

The following are the configuration requirements for compiling Nebula Graph. For the configuration requirements of the operating environment, see here.

  • CPU: x86_64
  • Memory: 4GB at least
  • Disk space: 10GB at least
  • Linux: 2.6.32 or higher, check with uname -r
  • glibc: 2.12 or higher, check with ldd --version
  • GCC: 7.1.0 or higher, check with g++ --version
  • CMake: 3.5.0 or higher, check with cmake --version
  • Access to the Internet

Quick Steps to Build

Installing Dependencies

Please note that it requires root privileges to install packages.

For CentOS, RedHat and Fedora users:

$ yum update
$ yum install -y make \
                 m4 \
                 git \
                 wget \
                 unzip \
                 xz \
                 readline-devel \
                 ncurses-devel \
                 zlib-devel \
                 gcc \
                 gcc-c++ \
                 cmake \
                 gettext \
                 curl \
                 redhat-lsb-core

# For CentOS 8+, RedHat 8+, and Fedora, you need to install libstdc++-static, libasan
$ yum install -y libstdc++-static libasan

For Debian and Ubuntu users:

$ apt-get update
$ apt-get install -y make \
                     m4 \
                     git \
                     wget \
                     unzip \
                     xz-utils \
                     curl \
                     lsb-core \
                     build-essential \
                     libreadline-dev \
                     ncurses-dev \
                     cmake \
                     gettext

For Arch and Gentoo users, you can definitely handle all of these on your own, right?

To make sure your GCC and CMake are in the right version:

$ g++ --version
$ cmake --version

If not, please refer to the following sections: Install an Applicable CMake and Install an Applicable GCC.

Cloning the Repo

$ git clone https://github.com/vesoft-inc/nebula.git

If you don't care about the commit history of the repo, and to make the cloning faster, you could perform a shallow clone:

$ git clone --depth=1 https://github.com/vesoft-inc/nebula.git

Configuring and Building

$ cd nebula
$ mkdir build
$ cd build
$ cmake -DENABLE_TESTING=OFF -DCMAKE_BUILD_TYPE=Release ..
# Assuming cores is the number of cores and mem_gb is the memory size (in GB), the value of N is recommended to select the smaller one from cores and mem_gb / 2
# We suggest choosing release build type to speed up compilation
$ make -jN
# The default installation directory is /usr/local/nebula
$ sudo make install
# If you want to start the services, copy the configuration files under the etc/ directory
# For production
$ cd /usr/local/nebula
$ sudo cp etc/nebula-storaged.conf.production etc/nebula-storaged.conf
$ sudo cp etc/nebula-metad.conf.production etc/nebula-metad.conf
$ sudo cp etc/nebula-graphd.conf.production etc/nebula-graphd.conf
# For trial
$ cd /usr/local/nebula
$ sudo cp etc/nebula-storaged.conf.default etc/nebula-storaged.conf
$ sudo cp etc/nebula-metad.conf.default etc/nebula-metad.conf
$ sudo cp etc/nebula-graphd.conf.default etc/nebula-graphd.conf

See the Start and Stop Nebula Graph Services Doc for details.

Since C++ templates are heavily used by Nebula Graph and its third party dependencies, especially Folly, fbthrift and boost, the building is very time-consuming.

For your reference, it is expected to take about 35 minutes in CPU time(less than 4 minutes with -j16), given an Intel E5-2697 v3 processor and unit tests are disabled.

Packing Your Source Code (Optional)

  • Package Nebula Graph to one package with the following command:
cd nebula/package
./package.sh -v <version>
  • Package Nebula Graph to multiple packages with the following command:
cd nebula/package
./package.sh -v <version> -n OFF

Ways to Tweak the Building

Until now, you might already have built Nebula Graph successfully. If so or not, we also provide ways to tweak the building process.

CMake Arguments/Variables

We provide several options to make one tweak the building, while some of these are builtins from CMake. These arguments are used at the configure(cmake) stage, like cmake -DArgument=Value ...

ENABLE_WERROR

By default, Nebula Graph turns on the -Werror compiler option to regard any warnings as errors. If the building fails with such errors, you could still continue the building by set ENABLE_WERROR to OFF.

ENABLE_TESTING

This option allows to enable or disable the build of unit tests. We suggest to turn it off if you just need the service modules of Nebula Graph. Options are ON and OFF, and the default value is ON.

ENABLE_ASAN

This option enables or disables the ASan building, a.k.a AddressSanitizer, which is a memory error detector. It is meant to be used by Nebula Graph developers. This option is OFF by default.

CMAKE_BUILD_TYPE

There are a few building types supported:

  • Debug, to build with debug info but without optimization, which is by default
  • Release, to build with optimization but without debug info
  • RelWithDebInfo, to build with optimization AND debug info
  • MinSizeRel, to build with optimizations for code size

CMAKE_INSTALL_PREFIX

This option is to specify the location where the service modules, scripts, configuration files and tools are installed when make install is performed. It is set to /usr/local/nebula by default.

CMAKE_CXX_COMPILER

Normally, CMake will figure out and locate an applicable C++ compiler for us. But if your compiler installation is not at the standard location, or if you want to use a different one, you have to specify it explicitly as follows:

$ cmake -DCMAKE_C_COMPILER=/path/to/gcc/bin/gcc -DCMAKE_CXX_COMPILER=/path/to/gcc/bin/g++ ..
$ cmake -DCMAKE_C_COMPILER=/path/to/clang/bin/clang -DCMAKE_CXX_COMPILER=/path/to/clang/bin/clang++ ..

ENABLE_CCACHE

This option is to enable the use of ccache, which is for speeding up the compilation during daily development.

By default, Nebula Graph will take advantage of ccache if it's found. So you don't have to enable it for yourself.

If you want to disable ccache, it might be not enough to just turn ENABLE_CCACHE off. Since on some platforms, the ccache installation hooks up or precedes the compiler. For such a case, you have to set an environment variable export CCACHE_DISABLE=true, or add a line disable=true to ~/.ccache/ccache.conf. We may do this for you automatically in future.

Please see the official documentation for more details.

NEBULA_USE_LINKER

This option allows users to use an alternative linker, e.g. gold. Options are bfd, lld and gold for now. Among them, bfd and gold belong to GNU binutils, while lld needs to install LLVM / Clang. In addition, you can use this parameter to specify the absolute path of the linker when needed.

NEBULA_THIRDPARTY_ROOT

This option is to explicitly specify the location of the third party.

Installing Third Party Manually

By default, at the configure(cmake) stage, a prebuilt third party will be downloaded and installed to the current build directory. If you would like to install it into another location for some reason, e.g. to rebuild by removing the whole build directory without downloading the third party again, you could perform the installation manually. Assume you are now at the build directory, run:

# To install third party to /opt requires root privilege, you could change it to another location with --prefix.
$ ../third-party/install-third-party.sh --prefix=/opt/vesoft/third-party

If the third party is installed to /opt/vesoft/third-party, which is by default if no --prefix given, the building system of Nebula Graph would find it automatically. Otherwise, you need to specify the location with the CMake argument NEBULA_THIRDPARTY_ROOT as mentioned above, or set an environment variable to the location and export it. The precedence for Nebula Graph to find and choose the third party is:

  1. The CMake argument NEBULA_THIRDPARTY_ROOT
  2. third-party/install in the current build directory
  3. The NEBULA_THIRDPARTY_ROOT environment variable
  4. /opt/vesoft/third-party

Install an Applicable CMake

For users who don't have a usable CMake installation, we provide a script to automatically download and install one for you. Assuming you are now at the build directory, run:

$ ../third-party/install-cmake.sh cmake-install
CMake has been installed to prefix=cmake-install
Run 'source cmake-install/bin/enable-cmake.sh' to make it ready to use.
Run 'source cmake-install/bin/disable-cmake.sh' to disable it.

$ source cmake-install/bin/enable-cmake.sh
$ cmake --version
cmake version 3.15.5

Now you have an applicable CMake ready to use. At any time, you could run the command source cmake-install/bin/disable-cmake.sh to disable it.

Install an Applicable GCC

For users who don't have a usable GCC installation, we provide a prebuilt GCC and a script to automatically download and install it. Assuming you are now at the build directory, run:

# To install GCC to /opt requires root privilege, you could change it to other locations
$ ../third-party/install-gcc.sh --prefix=/opt
GCC-7.5.0 has been installed to /opt/vesoft/toolset/gcc/7.5.0
Performing usability tests
Performing regular C++14 tests...OK
Performing LeakSanitizer tests...OK
Run 'source /opt/vesoft/toolset/gcc/7.5.0/enable' to start using.
Run 'source /opt/vesoft/toolset/gcc/7.5.0/disable' to stop using.

# Please note that the path and specific version might be different from your environment
$ source /opt/vesoft/toolset/gcc/7.5.0/enable
# Only PATH was setup so as not to pollute your library path
# You could run 'export LD_LIBRARY_PATH=/opt/vesoft/toolset/gcc/7.5.0/lib64:$LD_LIBRARY_PATH' if needed

$ g++ --version
g++ (Nebula Graph Build) 7.5.0
Copyright (C) 2017 Free Software Foundation, Inc.

Now you have an applicable GCC compiler ready to use. At any time, you could run the command source /opt/vesoft/toolset/gcc/7.5.0/disable to disable it.

Building Without Internet Access

For those who don't have an Internet access in the building environment, you have to download the above tools and dependencies manually, including the repo of Nebula Graph, GCC compiler, third party and CMake. Then you copy all of these to your building host. Following is a quick guide. Refer to the steps above for more details.

First, in the downloading host:

# Please note that although we use command line to illustrate the process, you could perform all the downloading via a browser.

# Download GCC
# For RedHat or CentOS users
$ wget https://oss-cdn.nebula-graph.com.cn/toolset/vesoft-gcc-7.5.0-CentOS-x86_64-glibc-2.12.sh
# For Debian or Ubuntu users
$ wget https://oss-cdn.nebula-graph.com.cn/toolset/vesoft-gcc-7.5.0-Debian-x86_64-glibc-2.13.sh

# Download CMake
$ wget https://cmake.org/files/v3.15/cmake-3.15.5-Linux-x86_64.sh

# Download third party
$ wget https://oss-cdn.nebula-graph.com.cn/third-party/vesoft-third-party-x86_64-libc-2.12-gcc-7.5.0-abi-11.sh

Then, copy these packages to the building host, and:

# Install GCC
# For RedHat or CentOS users
$ sudo bash vesoft-gcc-7.5.0-CentOS-x86_64-glibc-2.12.sh
# For Debian or Ubuntu users
$ sudo bash vesoft-gcc-7.5.0-Debian-x86_64-glibc-2.13.sh

# Enable the GCC installation
$ source /opt/vesoft/toolset/gcc/7.5.0/enable

# Install CMake
$ sudo bash cmake-3.15.5-Linux-x86_64.sh --skip-license --prefix=/opt/vesoft/toolset/cmake

# Enable CMake by appending its bin directory to PATH
$ export PATH=/opt/vesoft/toolset/cmake:$PATH

# Install third party
$ sudo bash vesoft-third-party-x86_64-libc-2.12-gcc-7.5.0-abi-11.sh

Now you are ready to download and build the Nebula Graph project.

Uninstalling

Before uninstalling, please stop the services. Use the make uninstall command in the make directory to uninstall Nebula Graph. Please note that the configuration file will not be deleted after uninstalling.

FAQ

error: invalid argument type 'auto' to unary expression

This error happens when building with Clang 9.0, as shown below:

[  5%] Building CXX object src/common/fs/CMakeFiles/fs_obj.dir/FileUtils.cpp.o
In file included from src/common/fs/FileUtils.cpp:8:
In file included from src/common/fs/FileUtils.h:12:
src/common/base/StatusOr.h:57:19: error: invalid argument type 'auto' to unary expression
    static_assert(!is_status_v<T>, "`T' must not be of type `Status'");
                  ^~~~~~~~~~~~~~~
src/common/fs/FileUtils.cpp:90:34: note: in instantiation of template class 'nebula::StatusOr<std::__cxx11::basic_string<char> >' requested here
StatusOr<std::string> FileUtils::readLink(const char *path) {
...

It is due to a known bug of Clang 9.0 to deal with auto template variables, which has not been fixed by Clang 10.0 until now(2020-05-25).

Last update: April 8, 2021