Enable mTLS in NebulaGraph¶

Transport Layer Security (TLS) is an encryption protocol in wide use on the Internet. TLS, which was formerly called SSL, authenticates the server in a client-server connection and encrypts communications between client and server so that external parties cannot spy on the communications. Its working principle is mainly to protect data transmitted over the network by using encryption algorithms to prevent data interception or tampering during transmission. During the TLS connection establishment process, the server sends a digital certificate containing a public key and some identity information to the client. This certificate is issued by a trusted third-party Certification Authority (CA). The client verifies this digital certificate to confirm the identity of the server.

In the NebulaGraph environment running in Kubernetes, mutual TLS (mTLS) is used to encrypt the communication between the client and server by default, which means both the client and server need to verify their identities. This article explains how to enable mTLS encryption in NebulaGraph running in K8s.

Prerequisites¶

NebulaGraph Operator has been installed.
A NebulaGraph cluster has been created. For details, see Create a NebulaGraph cluster with kubectl or Create a NebulaGraph cluster with Helm.
Certificates and their corresponding private keys have been generated for the client and server, and the CA certificate has been generated. For details, see Generate Certificates Manually.

Note

In the cluster created using Operator, the client and server use the same CA root certificate by default.

Create a TLS-type Secret¶

In a K8s cluster, you can create Secrets to store sensitive information, such as passwords, OAuth tokens, and TLS certificates. In NebulaGraph, you can create a Secret to store TLS certificates and private keys. When creating a Secret, the type tls should be specified. A tls Secret is used to store TLS certificates.

For example, to create a Secret for storing server certificates and private keys:

kubectl create secret tls <server-cert> --key=<path/to/tls.key> --cert=<path/to/tls.cert> --namespace=<namespace>

<server-cert>: The name of the Secret storing the server certificate and private key.
<path/to/tls.key>: The path to the server private key file.
<path/to/tls.cert>: The path to the server certificate file.
<namespace>: The namespace where the Secret is located. If --namespace is not specified, it defaults to the default namespace.

You can follow the above steps to create Secrets for the client certificate and private key, and the CA certificate.

To view the created Secrets:

kubectl get secret --namespace=<namespace>

Configure certifications¶

Operator provides the sslCerts field to specify the encrypted certificates. The sslCerts field contains three subfields: serverSecret, clientSecret, and caSecret. These three fields are used to specify the Secret names of the NebulaGraph server certificate, client certificate, and CA certificate, respectively. When you specify these three fields, Operator reads the certificate content from the corresponding Secret and mounts it into the cluster's Pod.

sslCerts:
  serverSecret: "server-cert"     # The name of the server certificate Secret.
  serverCert: ""                  # The key name of the certificate in the server certificate Secret, default is tls.crt.
  serverKey: ""                   # The key name of the private key in the server certificate Secret, default is tls.key.
  clientSecret: "client-cert"     # The name of the client certificate Secret.
  clientCert: ""                  # The key name of the certificate in the client certificate Secret, default is tls.crt.
  clientKey: ""                   # The key name of the private key in the client certificate Secret, default is tls.key.
  caSecret: "ca-cert"             # The name of the CA certificate Secret.
  caCert: ""                      # The key name of the certificate in the CA certificate Secret, default is ca.crt.

The serverCert and serverKey, clientCert and clientKey, and caCert are used to specify the key names of the certificate and private key of the server Secret, the key names of the certificate and private key of the client Secret, and the key name of the CA Secret certificate. If you do not customize these field values, Operator defaults serverCert and clientCert to tls.crt, serverKey and clientKey to tls.key, and caCert to ca.crt. However, in the K8s cluster, the TLS type Secret uses tls.crt and tls.key as the default key names for the certificate and private key. Therefore, after creating the NebulaGraph cluster, you need to manually change the caCert field from ca.crt to tls.crt in the cluster configuration, so that the Operator can correctly read the content of the CA certificate. Before you customize these field values, you need to specify the key names of the certificate and private key in the Secret when creating it. For how to create a Secret with the key name specified, run the kubectl create secret generic -h command for help.

You can use the insecureSkipVerify field to decide whether the client will verify the server's certificate chain and hostname. In production environments, it is recommended to set this to false to ensure the security of communication. If set to true, the client will not verify the server's certificate chain and hostname.

sslCerts:
  # Determines whether the client needs to verify the server's certificate chain and hostname when establishing an SSL connection. It defaults to true.
  insecureSkipVerify: false

When the certificates are approaching expiration, they can be automatically updated by installing cert-manager. NebulaGraph will monitor changes to the certificate directory files, and once a change is detected, it will load the new certificate content into memory.

Encryption strategies¶

NebulaGraph offers three encryption strategies that you can choose and configure according to your needs.

Encryption of client-graph and all inter-service communications

If you want to encrypt all data transmission between the client, Graph service, Meta service, and Storage service, you need to add the enable_ssl = true field to each service.

Here is an example configuration:

apiVersion: apps.nebula-graph.io/v1alpha1
kind: NebulaCluster
metadata:
  name: nebula
  namespace: default
spec:
  sslCerts:
    serverSecret: "server-cert"   # The Secret name of the server certificate and private key.
    clientSecret: "client-cert"   # The Secret name of the client certificate and private key.
    caSecret: "ca-cert"           # The Secret name of the CA certificate.
  graphd:
    config:
      enable_ssl: "true"
  metad:
    config:
      enable_ssl: "true"
  storaged:
    config:
      enable_ssl: "true"

Encryption of only Graph service communication

If the K8s cluster is deployed in the same data center and only the port of the Graph service is exposed externally, you can choose to encrypt only data transmission between the client and the Graph service. In this case, other services can communicate internally without encryption. Just add the enable_graph_ssl = true field to the Graph service.

Here is an example configuration:
```
apiVersion: apps.nebula-graph.io/v1alpha1
kind: NebulaCluster
metadata:
  name: nebula
  namespace: default
spec:
  sslCerts:
    serverSecret: "server-cert"
    caSecret: "ca-cert"
  graphd:
    config:
      enable_graph_ssl: "true"
```
Note

Because Operator doesn't need to call the Graph service through an interface, it's not necessary to set clientSecret in sslCerts.

Encryption of only Meta service communication

If you need to transmit confidential information to the Meta service, you can choose to encrypt data transmission related to the Meta service. In this case, you need to add the enable_meta_ssl = true configuration to each component.

Here is an example configuration:

apiVersion: apps.nebula-graph.io/v1alpha1
kind: NebulaCluster
metadata:
  name: nebula
  namespace: default
spec:
  sslCerts:
    serverSecret: "server-cert"
    clientSecret: "client-cert"
    caSecret: "ca-cert"
  graphd:
    config:
      enable_meta_ssl: "true"
  metad:
    config:
      enable_meta_ssl: "true"
  storaged:
    config:
      enable_meta_ssl: "true"

After setting up the encryption policy, when an external client needs to connect to the Graph service with mutual TLS, you still need to set the relevant TLS parameters according to the different clients. See the Use NebulaGraph Console to connect to Graph service section below for examples.

Steps¶

Use the pre-generated server and client certificates and private keys, and the CA certificate to create a Secret for each.
```
kubectl create secret tls <client/server/ca-cert-secret> --key=<client/server/ca.key> --cert=<client/server/ca.crt>
```
- tls: Indicates that the type of secret being created is TLS, which is used to store TLS certificates.
- <client/server/ca-cert-secret>: Specifies the name of the new secret being created, which can be customized.
- --key=<client/server/ca.key>: Specifies the path to the private key file of the TLS certificate to be stored in the secret.
- --cert=<client/server/ca.crt>: Specifies the path to the public key certificate file of the TLS certificate to be stored in the secret.

Add server certificate, client certificate, CA certificate configuration, and encryption policy configuration in the corresponding cluster instance YAML file. For details, see Encryption strategies.

For example, add encryption configuration for transmission data between client, Graph service, Meta service, and Storage service.

apiVersion: apps.nebula-graph.io/v1alpha1
kind: NebulaCluster
metadata:
  name: nebula
  namespace: default
spec:
  sslCerts:
    serverSecret: "server-cert"  // The name of the server Certificate Secret.
    clientSecret: "client-cert"  // The name of the client Certificate Secret.
    caSecret: "ca-cert"          // The name of the CA Certificate Secret.
  graphd:
    config:
      enable_ssl: "true"
  metad:
    config:
      enable_ssl: "true"
  storaged:
    config:
      enable_ssl: "true"

Use kubectl apply -f to apply the file to the Kubernetes cluster.
Verify that the values of serverCert, serverKey, clientCert, clientKey, caCert under the sslCerts field in the cluster configuration match the key names of the certificates and private keys stored in the created Secret.
```
# Check the key names of the certificate and private key stored in the Secret. For example, check the key name of the CA certificate stored in the Secret.
kubectl get secret ca-cert -o yaml
```
```
# Check the cluster configuration file.
kubectl get nebulacluster nebula -o yaml
```
Example output:
```
...
spec:
  sslCerts:
    serverSecret: server-cert
    serverCert: tls.crt
    serverKey: tls.key
    clientSecret: client-cert
    clientCert: tls.crt
    clientKey: tls.key
    caSecret: ca-cert
    caCert: ca.crt    
...
```
If the key names of the certificates and private keys stored in the Secret are different from the values of serverCert, serverKey, clientCert, clientKey, caCert under the sslCerts field in the cluster configuration, you need to execute kubectl edit nebulacluster <cluster_name> to manually modify the cluster configuration file.

In the example output, the key name of the CA certificate in the TLS-type Secret is tls.crt, so you need to change the value of caCert from ca.crt to tls.crt.
Use NebulaGraph Console to connect to the Graph service and establish a secure TLS connection.

Example:
```
kubectl run -ti --image vesoft/nebula-console:v3.5.0 --restart=Never -- nebula-console -addr 10.98.xxx.xx  -port 9669 -u root -p nebula -enable_ssl -ssl_root_ca_path /path/to/cert/root.crt -ssl_cert_path /path/to/cert/client.crt -ssl_private_key_path /path/to/cert/client.key
```
- -enable_ssl: Use mTLS when connecting to NebulaGraph.
- -ssl_root_ca_path: Specify the storage path of the CA root certificate.
- -ssl_cert_path: Specify the storage path of the TLS public key certificate.
- -ssl_private_key_path: Specify the storage path of the TLS private key.
- For details on using NebulaGraph Console to connect to the Graph service, see Connect to NebulaGraph.

At this point, you can enable mTLS in NebulaGraph.

Last update: July 27, 2023