log_processor/data_sources/victorialogs: add new datasource (#727)

Co-authored-by: Laurence Jones <[email protected]>

crowdsec-docs/docs/log_processor/data_sources/introduction.md

@@ -23,6 +23,7 @@ Name | Type | Stream | One-shot
 [Kafka](/log_processor/data_sources/kafka.md)| read logs from kafka topic | yes | no
 [Kubernetes Audit](/log_processor/data_sources/kubernetes_audit.md) | expose a webhook to receive audit logs from a Kubernetes cluster  | yes | no
 [Loki](/log_processor/data_sources/loki.md) | read logs from loki | yes | yes
+[VictoriaLogs](/log_processor/data_sources/victorialogs.md) | read logs from VictoriaLogs | yes | yes
 [syslog service](/log_processor/data_sources/syslog_service.md) | read logs received via syslog protocol | yes | no
 [Windows Event](/log_processor/data_sources/windows_event_log.md)| read logs from windows event log | yes | yes
 

crowdsec-docs/docs/log_processor/data_sources/victorialogs.md

@@ -0,0 +1,118 @@
+---
+id: victorialogs
+title: VictoriaLogs
+---
+
+This module allows the `Security Engine` to acquire logs from VictoriaLogs query.
+
+## Configuration example
+
+This will allow to read logs from VictoriaLogs, using the query `app:nginx`.
+```yaml
+source: victorialogs
+mode: tail
+log_level: info
+url: http://localhost:9428/
+limit: 1000
+query: |
+  app:nginx
+auth:
+  username: something
+  password: secret
+labels:
+ type: nginx
+```
+
+:::info
+The reader will always start at "now" for `tail` mode.
+:::
+
+Look at the `configuration parameters` to view all supported options.
+
+## Parameters
+
+### `mode`
+
+Mode to fetch the logs, supported values: `tail` and `cat`.
+
+Defaults to `tail`.
+
+### `url`
+
+The VictoriaLogs URL to connect to.
+
+Required.
+
+### `prefix`
+
+The VictoriaLogs prefix (present in http path, useful if VictoriaLogs is behind a reverse-proxy).
+
+Defaults to `/`.
+
+### `query`
+
+The [VictoriaLogs query](https://docs.victoriametrics.com/victorialogs/logsql/).
+
+Required.
+
+Note that `tail` requests have limitations for operators used query. See [this doc](https://docs.victoriametrics.com/victorialogs/querying/#live-tailing) for the details.
+
+### `limit`
+
+The maximum number of messages to be retried from VictoriaLogs at once.
+
+### `headers`
+
+Allows you to specify headers to be sent to VictoriaLogs, in the format:
+
+```yaml
+headers:
+  foo: bar
+  AccountID: 0
+  ProjectID: 0
+```
+
+See this doc for more information: [VictoriaLogs headers](https://docs.victoriametrics.com/victorialogs/querying/#http-api)
+
+### `wait_for_ready`
+
+The retry interval at startup before giving on VictoriaLogs.
+
+Defaults to `10 seconds`.
+
+### `auth`
+
+Login/password authentication for VictoriaLogs, in the format:
+
+```yaml
+auth:
+  username: someone
+  password: something
+```
+
+### `max_failure_duration`
+
+The maximum duration VictoriaLogs is allowed to be unavailable (once startup is successful) before giving up on the data source.
+
+Default to `30 seconds`.
+
+
+## DSN and command-line
+
+All the parameters above are available via DNS (one-shot mode), plus the following ones:
+
+### `ssl`
+
+if present, scheme will be set to `https`
+
+### `since`
+
+Allows to set the "start" duration for VictoriaLogs query.
+
+### `log_level`
+
+Set the `log_level` for VictoriaLogs datasource.
+
+```bash
+crowdsec -type foobar -dsn 'victorialogs://login:password@localhost:9428/?query=server:"demoVictoriaLogsVictoriaLogs"'
+```

crowdsec-docs/sidebars.js

@@ -122,6 +122,7 @@ module.exports = {
                         "log_processor/data_sources/kafka",
                         "log_processor/data_sources/kubernetes_audit",
                         "log_processor/data_sources/loki",
+                        "log_processor/data_sources/victorialogs",
                         "log_processor/data_sources/syslog",
                         "log_processor/data_sources/windows_evt_log",
                         "log_processor/data_sources/troubleshoot",