Manage cluster logs¶
Running logs of NebulaGraph cluster services (graphd, metad, storaged) are generated and stored in the /usr/local/nebula/logs directory of each service container by default.
View logs¶
To view the running logs of a NebulaGraph cluster, you can use the kubectl logs command.
For example, to view the running logs of the Storage service:
// View the name of the Storage service Pod, nebula-storaged-0.
$ kubectl get pods -l app.kubernetes.io/component=storaged
NAME READY STATUS RESTARTS AGE
nebula-storaged-0 1/1 Running 0 45h
...
// Enter the container storaged of the Storage service.
$ kubectl exec -it nebula-storaged-0 -c storaged -- /bin/bash
// View the running logs of the Storage service.
$ cd /usr/local/nebula/logs
Clean logs¶
Running logs generated by cluster services during runtime will occupy disk space. To avoid occupying too much disk space, the NebulaGraph Operator uses a sidecar container to periodically clean and archive logs.
To facilitate log collection and management, each NebulaGraph service deploys a sidecar container responsible for collecting logs generated by the service container and sending them to the specified log disk. The sidecar container automatically cleans and archives logs using the logrotate tool.
In the YAML configuration file of the cluster instance, set spec.logRotate to enable log rotation and set timestamp_in_logfile_name to false to disable the timestamp in the log file name to implement log rotation for the target service. The timestamp_in_logfile_name parameter is configured under the spec.<graphd|metad|storaged>.config field. By default, the log rotation feature is turned off. Here is an example of enabling log rotation for all services:
...
spec:
graphd:
config:
# Whether to include a timestamp in the log file name.
# You must set this parameter to false to enable log rotation.
# It is set to true by default.
"timestamp_in_logfile_name": "false"
metad:
config:
"timestamp_in_logfile_name": "false"
storaged:
config:
"timestamp_in_logfile_name": "false"
logRotate: # Log rotation configuration
# The number of times a log file is rotated before being deleted.
# The default value is 5, and 0 means the log file will not be rotated before being deleted.
rotate: 5
# The log file is rotated only if it grows larger than the specified size. The default value is 200M.
size: "200M"
Collect logs¶
If you don't want to mount additional log disks to back up log files, or if you want to collect logs and send them to a log center using services like fluent-bit, you can configure logs to be output to standard error. The Operator uses the glog tool to log to standard error output.
Note
Currently, NebulaGraph Operator only collects standard error logs.
In the YAML configuration file of the cluster instance, you can configure logging to standard error output in the config and env fields of each service.
...
spec:
graphd:
config:
# Whether to redirect standard error to a separate output file. The default value is false, which means it is not redirected.
redirect_stdout: "false"
# The severity level of log content: INFO, WARNING, ERROR, and FATAL. The corresponding values are 0, 1, 2, and 3.
stderrthreshold: "0"
env:
- name: GLOG_logtostderr # Write log to standard error output instead of a separate file.
value: "1" # 1 represents writing to standard error output, and 0 represents writing to a file.
image: vesoft/nebula-graphd
replicas: 1
resources:
requests:
cpu: 500m
memory: 500Mi
service:
externalTrafficPolicy: Local
type: NodePort
version: v3.8.0
metad:
config:
redirect_stdout: "false"
stderrthreshold: "0"
dataVolumeClaim:
resources:
requests:
storage: 1Gi
storageClassName: ebs-sc
env:
- name: GLOG_logtostderr
value: "1"
image: vesoft/nebula-metad
...