Manage cluster logs¶
Running logs of NebulaGraph cluster services (graphd, metad, storaged) are generated and stored in the
/usr/local/nebula/logs directory of each service container by default.
To view the running logs of a NebulaGraph cluster, you can use the
kubectl logs command.
For example, to view the running logs of the Storage service:
// View the name of the Storage service Pod, nebula-storaged-0. $ kubectl get pods NAME READY STATUS RESTARTS AGE nebula-exporter-84b6974497-cr54d 1/1 Running 0 43h nebula-graphd-0 1/1 Running 0 22h nebula-metad-0 1/1 Running 0 45h nebula-storaged-0 1/1 Running 0 45h ... // Enter the container storaged of the Storage service. $ kubectl exec -it nebula-storaged-0 -c storaged -- /bin/bash // View the running logs of the Storage service. $ cd /usr/local/nebula/logs
Running logs generated by cluster services during runtime will occupy disk space. To avoid occupying too much disk space, the Operator uses a sidecar container to periodically clean and archive logs.
To facilitate log collection and management, each NebulaGraph service deploys a sidecar container responsible for collecting logs generated by the service container and sending them to the specified log disk. The sidecar container automatically cleans and archives logs using the logrotate tool.
In the YAML configuration file of the cluster instance, you can configure log rotation to automatically clean and archive logs through the
spec.logRotate field. By default, the log rotation feature is turned off. Here is an example of enabling log rotation:
... spec: graphd: config: # Whether to include a timestamp in the log file name. "true" means yes, "false" means no. "timestamp_in_logfile_name": "false" metad: config: "timestamp_in_logfile_name": "false" storaged: config: "timestamp_in_logfile_name": "false" logRotate: # Log rotation configuration # The number of times a log file is rotated before being deleted. The default value is 5, and 0 means the log file will not be rotated before being deleted. rotate: 5 # The log file is rotated only if it grows larger than the specified size. The default value is 200M. size: "200M"
If you don't want to mount additional log disks to back up log files, or if you want to collect logs and send them to a log center using services like fluent-bit, you can configure logs to be output to standard error. The Operator uses the glog tool to log to standard error output.
Currently, NebulaGraph Operator only collects standard error logs.。
In the YAML configuration file of the cluster instance, you can configure logging to standard error output in the
env fields of each service.
... spec: graphd: config: # Whether to redirect standard error to a separate output file. The default value is false, which means it is not redirected. redirect_stdout: "false" # The severity level of log content: INFO, WARNING, ERROR, and FATAL. The corresponding values are 0, 1, 2, and 3. stderrthreshold: "0" env: - name: GLOG_logtostderr # Write log to standard error output instead of a separate file. value: "1" # 1 represents writing to standard error output, and 0 represents writing to a file. image: vesoft/nebula-graphd replicas: 1 resources: requests: cpu: 500m memory: 500Mi service: externalTrafficPolicy: Local type: NodePort version: v3.4.0 metad: config: redirect_stdout: "false" stderrthreshold: "0" dataVolumeClaim: resources: requests: storage: 1Gi storageClassName: ebs-sc env: - name: GLOG_logtostderr value: "1" image: vesoft/nebula-metad ...