Skip to content

Commit

Permalink
Create 8.4.manage-running-logs.md (#2000)
Browse files Browse the repository at this point in the history
* Create 8.4.manage-running-logs.md

* Update 8.4.manage-running-logs.md
  • Loading branch information
abby-cyber authored Mar 13, 2023
1 parent d95354e commit 150211a
Showing 1 changed file with 103 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# Manage cluster logs

Running logs of NebulaGraph cluster services (graphd, metad, storaged) are generated and stored in the `/usr/local/nebula/logs` directory of each service container by default.

## View logs

To view the running logs of a NebulaGraph cluster, you can use the `kubectl logs` command.

For example, to view the running logs of the Storage service:

```bash
// View the name of the Storage service Pod, nebula-storaged-0.
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
nebula-exporter-84b6974497-cr54d 1/1 Running 0 43h
nebula-graphd-0 1/1 Running 0 22h
nebula-metad-0 1/1 Running 0 45h
nebula-storaged-0 1/1 Running 0 45h
...

// Enter the container storaged of the Storage service.
$ kubectl exec -it nebula-storaged-0 -c storaged -- /bin/bash

// View the running logs of the Storage service.
$ cd /usr/local/nebula/logs
```

## Clean logs

Running logs generated by cluster services during runtime will occupy disk space. To avoid occupying too much disk space, the Operator uses a sidecar container to periodically clean and archive logs.

To facilitate log collection and management, each NebulaGraph service deploys a sidecar container responsible for collecting logs generated by the service container and sending them to the specified log disk. The sidecar container automatically cleans and archives logs using the [logrotate](https://linux.die.net/man/8/logrotate) tool.

In the YAML configuration file of the cluster instance, you can configure log rotation to automatically clean and archive logs through the `spec.logRotate` field. By default, the log rotation feature is turned off. Here is an example of enabling log rotation:

```yaml
...
spec:
graphd:
config:
# Whether to include a timestamp in the log file name. "true" means yes, "false" means no.
"timestamp_in_logfile_name": "false"
metad:
config:
"timestamp_in_logfile_name": "false"
storaged:
config:
"timestamp_in_logfile_name": "false"
logRotate: # Log rotation configuration
# The number of times a log file is rotated before being deleted. The default value is 5, and 0 means the log file will not be rotated before being deleted.
rotate: 5
# The log file is rotated only if it grows larger than the specified size. The default value is 200M.
size: "200M"
```
## Collect logs
If you don't want to mount additional log disks to back up log files, or if you want to collect logs and send them to a log center using services like [fluent-bit](https://fluentbit.io/), you can configure logs to be output to standard error. The Operator uses the [glog](https://github.com/google/glog) tool to log to standard error output.
!!! caution
Currently, NebulaGraph Operator only collects standard error logs.。
In the YAML configuration file of the cluster instance, you can configure logging to standard error output in the `config` and `env` fields of each service.


```yaml
...
spec:
graphd:
config:
# Whether to redirect standard error to a separate output file. The default value is false, which means it is not redirected.
redirect_stdout: "false"
# The severity level of log content: INFO, WARNING, ERROR, and FATAL. The corresponding values are 0, 1, 2, and 3.
stderrthreshold: "0"
env:
- name: GLOG_logtostderr # Write log to standard error output instead of a separate file.
value: "1" # 1 represents writing to standard error output, and 0 represents writing to a file.
image: vesoft/nebula-graphd
replicas: 1
resources:
requests:
cpu: 500m
memory: 500Mi
service:
externalTrafficPolicy: Local
type: NodePort
version: v{{nebula.release}}
metad:
config:
redirect_stdout: "false"
stderrthreshold: "0"
dataVolumeClaim:
resources:
requests:
storage: 1Gi
storageClassName: ebs-sc
env:
- name: GLOG_logtostderr
value: "1"
image: vesoft/nebula-metad
...
```

0 comments on commit 150211a

Please sign in to comment.