System design suggestions¶

QPS or low-latency first¶

Nebula Graph is good at handling small requests with high concurrency. In such scenarios, the whole graph is huge, containing maybe trillions of vertices, but the subgraphs accessed by each request are not large (containing millions of vertices or edges), and the latency of a single request is low. The concurrent number of such requests, i.e., the QPS, can be huge.

On the other hand, in interactive analysis scenarios, the request concurrency is usually not high, but the subgraphs accessed by each request are large, with thousands of millions of vertices or edges. To lower the latency of big requests in such scenarios, you can split big requests into multiple small requests in the application, and send them to multiple Graph servers. This can decrease the memory used by each Graph server as well. Besides, you can use Nebula Algorithm for such scenarios.

Horizontal or vertical scaling¶

Nebula Graph 2.5.1 supports horizontal scaling.

• The horizontal scaling of the Storage Service:

  Increasing the number of Storage machines increases the overall capability of the cluster linearly, including increasing overall QPS and reducing latency.

  However, the number of partitions is fixed when creating a graph space. The service capability of a single partition is determined by a single server. The operations depending on a single partition include fetching properties of a single vertex (FETCH), a breadth-first traversal from a single vertex (GO), etc.

• The horizontal scaling of the Graph Service:

  Each request from the client is handled by one and only one Graph server, with no other Graph servers participating in the processing of the request. Therefore, increasing the number of Graph machines can increase the overall QPS of the cluster, but cannot lower the latency of a single request.

• Metad does not support horizontal scaling.

Vertical scaling usually has higher hardware costs, but relatively simple operations. Nebula Graph 2.5.1 can also be scaled vertically.

Data transmission and optimization¶

• Read/write balance. Nebula Graph fits into OLTP scenarios with balanced read/write, i.e., concurrent write and read. It is not suitable for OLAP scenarios that usually need to write once and read many times.
• Select different write methods. Write large batches of data with SST files, and small batches of data with INSERT statements.
• Run COMPACTION and BALANCE jobs to optimize data format and storage distribution at the right time.
• Nebula Graph 2.5.1 N Does not support transactions and isolation in the relational database sense and is closer to NoSQL.

Query preheating and data preheating¶

Preheat on the application side:

• The Graph Service does not support pre-compiling queries and generating corresponding query plans, nor can it cache previous query results.
• The Storage Service does not support preheating data, and only the LSM-Tree and BloomFilter of RocksDB are loaded into memory at startup.
• Once accessed, vertices and edges are cached respectively in two types of LRU cache of the Storage Service.