Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

crowdsec explain ooms #3469

Open
bjo81 opened this issue Feb 15, 2025 · 2 comments
Open

crowdsec explain ooms #3469

bjo81 opened this issue Feb 15, 2025 · 2 comments
Labels
kind/bug Something isn't working needs/triage os/linux version/1.6.5

Comments

@bjo81
Copy link

bjo81 commented Feb 15, 2025

What happened?

Unfortunately, the dovecot and postfix scenarios didn't seem to work as expected, so I wanted to try

cscli explain --dsn "journalctl://filters=_SYSTEMD_UNIT=dovecot.service" --type syslog

But crowdsec ate 6GB of memory (the system has 8GB memory and 4GB swap) and got oomkilled.

[1274714.165203] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/session-116.scope,task=crowdsec,pid=208475,uid=0
[1274714.165254] Out of memory: Killed process 208475 (crowdsec) total-vm:9781416kB, anon-rss:6006508kB, file-rss:192kB, shmem-rss:0kB, UID:0 pgtables:15276kB oom_score_adj:0

What did you expect to happen?

Getting an explaination of the log parser. Maybe some restriction regarding the time period would be helpful.

How can we reproduce it (as minimally and precisely as possible)?

Try to use a journalctl journal of 14 days.

Anything else we need to know?

No response

Crowdsec version

version: v1.6.5-d8dcdc913
Codename: alphaga
BuildDate: 2025-02-14_15:49:33
GoVersion: 1.24.0
Platform: linux
libre2: C++
User-Agent: crowdsec/v1.6.5-d8dcdc913-linux
Constraint_parser: >= 1.0, <= 3.0
Constraint_scenario: >= 1.0, <= 3.0
Constraint_api: v1
Constraint_acquis: >= 1.0, < 2.0
Built-in optional components: cscli_setup, datasource_appsec, datasource_cloudwatch, datasource_docker, datasource_file, datasource_http, datasource_journalctl, datasource_k8s-audit, datasource_kafka, datasource_kinesis, datasource_loki, datasource_s3, datasource_syslog, datasource_victorialogs, datasource_wineventlog

OS version

NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
ANSI_COLOR="38;2;23;147;209"
HOME_URL="https://archlinux.org/"
DOCUMENTATION_URL="https://wiki.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://gitlab.archlinux.org/groups/archlinux/-/issues"
PRIVACY_POLICY_URL="https://terms.archlinux.org/docs/privacy-policy/"
LOGO=archlinux-logo

Linux host.domain.org 6.6.72-1-lts #1 SMP PREEMPT_DYNAMIC Fri, 17 Jan 2025 14:04:26 +0000 x86_64 GNU/Linux

Enabled collections and parsers

update for collection crowdsecurity/postfix available (currently:0.2, latest:0.4)
crowdsecurity/nextcloud is outdated because of parsers:crowdsecurity/nextcloud-logs
crowdsecurity/nextcloud is outdated because of parsers:crowdsecurity/nextcloud-whitelist
Loaded: 133 parsers, 10 postoverflows, 751 scenarios, 8 contexts, 4 appsec-configs, 93 appsec-rules, 131 collections
Unmanaged items: 1 local, 0 tainted
name,status,version,description,type
crowdsecurity/dateparse-enrich,enabled,0.2,,parsers
crowdsecurity/dovecot-logs,enabled,0.9,Parse dovecot logs,parsers
crowdsecurity/geoip-enrich,enabled,0.5,"Populate event with geoloc info : as, country, coords, source range.",parsers
crowdsecurity/http-logs,enabled,1.3,"Parse more Specifically HTTP logs, such as HTTP Code, HTTP path, HTTP args and if its a static ressource",parsers
crowdsecurity/nextcloud-logs,"enabled,update-available",0.2,Parse nextcloud logs,parsers
crowdsecurity/nextcloud-whitelist,"enabled,update-available",0.3,Whitelist events from nextcloud,parsers
crowdsecurity/nginx-logs,enabled,1.7,Parse nginx access and error logs,parsers
crowdsecurity/postfix-logs,enabled,0.8,Parse postfix logs,parsers
crowdsecurity/postscreen-logs,"enabled,update-available",0.2,Parse postscreen logs,parsers
crowdsecurity/sshd-logs,enabled,2.9,Parse openSSH logs,parsers
crowdsecurity/syslog-logs,enabled,0.8,,parsers
crowdsecurity/whitelists,"enabled,local",,,parsers
crowdsecurity/apache_log4j2_cve-2021-44228,enabled,0.6,Detect cve-2021-44228 exploitation attemps,scenarios
crowdsecurity/CVE-2017-9841,enabled,0.2,Detect CVE-2017-9841 exploits,scenarios
crowdsecurity/CVE-2019-18935,enabled,0.2,Detect Telerik CVE-2019-18935 exploitation attempts,scenarios
crowdsecurity/CVE-2022-26134,enabled,0.2,Detect CVE-2022-26134 exploits,scenarios
crowdsecurity/CVE-2022-35914,enabled,0.2,Detect CVE-2022-35914 exploits,scenarios
crowdsecurity/CVE-2022-37042,enabled,0.2,Detect CVE-2022-37042 exploits,scenarios
crowdsecurity/CVE-2022-40684,enabled,0.3,Detect cve-2022-40684 exploitation attempts,scenarios
crowdsecurity/CVE-2022-41082,enabled,0.4,Detect CVE-2022-41082 exploits,scenarios
crowdsecurity/CVE-2022-41697,enabled,0.2,Detect CVE-2022-41697 enumeration,scenarios
crowdsecurity/CVE-2022-42889,enabled,0.3,Detect CVE-2022-42889 exploits (Text4Shell),scenarios
crowdsecurity/CVE-2022-44877,enabled,0.3,Detect CVE-2022-44877 exploits,scenarios
crowdsecurity/CVE-2022-46169,enabled,0.2,Detect CVE-2022-46169 brute forcing,scenarios
crowdsecurity/CVE-2023-22515,enabled,0.1,Detect CVE-2023-22515 exploitation,scenarios
crowdsecurity/CVE-2023-22518,enabled,0.2,Detect CVE-2023-22518 exploits,scenarios
crowdsecurity/CVE-2023-49103,enabled,0.3,Detect owncloud CVE-2023-49103 exploitation attempts,scenarios
crowdsecurity/CVE-2024-0012,enabled,0.1,Detect CVE-2024-0012 exploitation attempts,scenarios
crowdsecurity/CVE-2024-38475,enabled,0.1,Detect CVE-2024-38475 exploitation attempts,scenarios
crowdsecurity/CVE-2024-9474,enabled,0.1,Detect CVE-2024-9474 exploitation attempts,scenarios
crowdsecurity/dovecot-spam,enabled,0.5,detect errors on dovecot,scenarios
crowdsecurity/f5-big-ip-cve-2020-5902,enabled,0.2,Detect cve-2020-5902 exploitation attemps,scenarios
crowdsecurity/fortinet-cve-2018-13379,enabled,0.3,Detect cve-2018-13379 exploitation attemps,scenarios
crowdsecurity/grafana-cve-2021-43798,enabled,0.2,Detect cve-2021-43798 exploitation attemps,scenarios
crowdsecurity/http-admin-interface-probing,enabled,0.4,Detect generic HTTP admin interface probing,scenarios
crowdsecurity/http-backdoors-attempts,enabled,0.6,Detect attempt to common backdoors,scenarios
crowdsecurity/http-bad-user-agent,enabled,1.2,Detect usage of bad User Agent,scenarios
crowdsecurity/http-crawl-non_statics,enabled,0.7,Detect aggressive crawl on non static resources,scenarios
crowdsecurity/http-cve-2021-41773,enabled,0.3,Apache - Path Traversal (CVE-2021-41773),scenarios
crowdsecurity/http-cve-2021-42013,enabled,0.3,Apache - Path Traversal (CVE-2021-42013),scenarios
crowdsecurity/http-cve-probing,enabled,0.2,Detect generic HTTP cve probing,scenarios
crowdsecurity/http-generic-bf,enabled,0.9,Detect generic http brute force,scenarios
crowdsecurity/http-open-proxy,enabled,0.5,Detect scan for open proxy,scenarios
crowdsecurity/http-path-traversal-probing,enabled,0.4,Detect path traversal attempt,scenarios
crowdsecurity/http-probing,enabled,0.4,Detect site scanning/probing from a single ip,scenarios
crowdsecurity/http-sensitive-files,enabled,0.4,"Detect attempt to access to sensitive files (.log, .db ..) or folders (.git)",scenarios
crowdsecurity/http-sqli-probing,enabled,0.4,A scenario that detects SQL injection probing with minimal false positives,scenarios
crowdsecurity/http-wordpress-scan,enabled,0.2,Detect WordPress scan: vuln hunting,scenarios
crowdsecurity/http-xss-probing,enabled,0.4,A scenario that detects XSS probing with minimal false positives,scenarios
crowdsecurity/jira_cve-2021-26086,enabled,0.3,Detect Atlassian Jira CVE-2021-26086 exploitation attemps,scenarios
crowdsecurity/netgear_rce,enabled,0.4,Detect Netgear RCE DGN1000/DGN220 exploitation attempts,scenarios
crowdsecurity/nextcloud-bf,enabled,0.3,Detect Nextcloud bruteforce,scenarios
crowdsecurity/nginx-req-limit-exceeded,enabled,0.3,Detects IPs which violate nginx's user set request limit.,scenarios
crowdsecurity/postfix-spam,enabled,0.4,Detect spammers,scenarios
crowdsecurity/pulse-secure-sslvpn-cve-2019-11510,enabled,0.3,Detect cve-2019-11510 exploitation attemps,scenarios
crowdsecurity/spring4shell_cve-2022-22965,enabled,0.3,Detect cve-2022-22965 probing,scenarios
crowdsecurity/ssh-bf,enabled,0.3,Detect ssh bruteforce,scenarios
crowdsecurity/ssh-cve-2024-6387,enabled,0.2,Detect exploitation attempt of CVE-2024-6387,scenarios
crowdsecurity/ssh-slow-bf,enabled,0.4,Detect slow ssh bruteforce,scenarios
crowdsecurity/thinkphp-cve-2018-20062,enabled,0.6,Detect ThinkPHP CVE-2018-20062 exploitation attemps,scenarios
crowdsecurity/vmware-cve-2022-22954,enabled,0.3,Detect Vmware CVE-2022-22954 exploitation attempts,scenarios
crowdsecurity/vmware-vcenter-vmsa-2021-0027,enabled,0.2,Detect VMSA-2021-0027 exploitation attemps,scenarios
ltsich/http-w00tw00t,enabled,0.2,detect w00tw00t,scenarios
crowdsecurity/bf_base,enabled,0.1,,contexts
crowdsecurity/http_base,enabled,0.2,,contexts
crowdsecurity/base-http-scenarios,enabled,1.0,http common : scanners detection,collections
crowdsecurity/dovecot,enabled,0.1,dovecot support : parser and spammer detection,collections
crowdsecurity/http-cve,enabled,2.9,Detect CVE exploitation in http logs,collections
crowdsecurity/linux,enabled,0.2,core linux support : syslog+geoip+ssh,collections
crowdsecurity/nextcloud,"enabled,update-available",0.3,Nextcloud support : parser and brute-force detection,collections
crowdsecurity/nginx,enabled,0.2,nginx support : parser and generic http scenarios,collections
crowdsecurity/postfix,"enabled,update-available",0.2,postfix support : parser and spammer d

Acquisition config

``` #Generated acquisition file - wizard.sh (service: nginx) / files : /var/log/nginx/access.log /var/log/nginx/error.log filenames: - /var/log/nginx/access.log - /var/log/nginx/error.log labels: type: nginx --- #Generated acquisition file - wizard.sh (service: sshd) / files : journalctl_filter: - _SYSTEMD_UNIT=sshd.service labels: type: syslog --- journalctl_filter: - _SYSTEMD_UNIT=postfix.service labels: type: syslog --- journalctl_filter: - _SYSTEMD_UNIT=dovecot.service labels: type: syslog --- #Generated acquisition file - wizard.sh (service: nginx) / files : /var/log/nginx/access.log /var/log/nginx/error.log filenames: - /var/log/nginx/access.log - /var/log/nginx/error.log labels: type: nginx --- #Generated acquisition file - wizard.sh (service: sshd) / files : journalctl_filter: - _SYSTEMD_UNIT=sshd.service labels: type: syslog --- 

</details>


### Config show

<details>

Global:

  • Configuration Folder : /etc/crowdsec

  • Data Folder : /var/lib/crowdsec/data

  • Hub Folder : /etc/crowdsec/hub

  • Simulation File : /etc/crowdsec/simulation.yaml

  • Log Folder : /var/log

  • Log level : info

  • Log Media : file
    Crowdsec:

  • Acquisition File : /etc/crowdsec/acquis.yaml

  • Parsers routines : 1

  • Acquisition Folder : /etc/crowdsec/acquis.d
    cscli:

  • Output : human

  • Hub Branch :
    API Client:

  • URL : http://127.0.0.1:8080/

  • Login : c067eae1ebe243b29bc7d3ba65520ef95otiH8edD5H18qr3

  • Credentials File : /etc/crowdsec/local_api_credentials.yaml
    Local API Server:

  • Listen URL : 127.0.0.1:8080

  • Listen Socket :

  • Profile File : /etc/crowdsec/profiles.yaml

  • Trusted IPs:

    • 127.0.0.1
    • ::1

  • Database:

    • Type : sqlite
    • Path : /var/lib/crowdsec/data/crowdsec.db
    • Flush age : 7d
    • Flush size : 5000

</details>


### Prometheus metrics

<details>

╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Acquisition Metrics │
├───────────────────────────────────────────────────────┬────────────┬──────────────┬────────────────┬────────────────────────┬───────────────────┤
│ Source │ Lines read │ Lines parsed │ Lines unparsed │ Lines poured to bucket │ Lines whitelisted │
├───────────────────────────────────────────────────────┼────────────┼──────────────┼────────────────┼────────────────────────┼───────────────────┤
│ file:/var/log/nginx/access.log │ 440.42k │ 440.42k │ - │ 58.06k │ 24.93k │
│ file:/var/log/nginx/error.log │ 27.55k │ 27.55k │ - │ 13.82k │ 6 │
│ journalctl:journalctl-%s_SYSTEMD_UNIT=dovecot.service │ 2.04k │ 593 │ 1.45k │ 7 │ 128 │
│ journalctl:journalctl-%s_SYSTEMD_UNIT=postfix.service │ 7.76k │ 1.95k │ 5.81k │ 1.95k │ - │
│ journalctl:journalctl-%s_SYSTEMD_UNIT=sshd.service │ 832 │ 460 │ 372 │ 1.13k │ - │
╰───────────────────────────────────────────────────────┴────────────┴──────────────┴────────────────┴────────────────────────┴───────────────────╯
╭────────────────────────────────────────────────────╮
│ Local API Alerts │
├────────────────────────────────────────────┬───────┤
│ Reason │ Count │
├────────────────────────────────────────────┼───────┤
│ crowdsecurity/ssh-slow-bf │ 1 │
│ crowdsecurity/CVE-2022-41082 │ 1 │
│ crowdsecurity/apache_log4j2_cve-2021-44228 │ 1 │
│ crowdsecurity/http-bad-user-agent │ 9 │
│ crowdsecurity/netgear_rce │ 1 │
│ crowdsecurity/postfix-spam │ 1 │
│ crowdsecurity/ssh-bf │ 71 │
╰────────────────────────────────────────────┴───────╯
╭───────────────────────────────────────────────────────────────────────────────────────────╮
│ Bouncer Metrics (crowdsec-firewall-bouncer) since 2025-02-14 17:14:18 +0000 UTC │
├────────────────────────────┬──────────────────┬───────────────────┬───────────────────────┤
│ Origin │ active_decisions │ dropped │ processed │
│ │ IPs │ bytes │ packets │ bytes │ packets │
├────────────────────────────┼──────────────────┼─────────┼─────────┼───────────┼───────────┤
│ CAPI (community blocklist) │ 23.89k │ 938.36k │ 15.76k │ - │ - │
│ crowdsec (security engine) │ 18 │ 132.92k │ 2.09k │ - │ - │
├────────────────────────────┼──────────────────┼─────────┼─────────┼───────────┼───────────┤
│ Total │ 23.91k │ 1.07M │ 17.85k │ 16.48G │ 35.48M │
╰────────────────────────────┴──────────────────┴─────────┴─────────┴───────────┴───────────╯
╭─────────────────────────────────────────────────────────────────────────╮
│ Local API Decisions │
├─────────────────────────────────────────────┬──────────┬────────┬───────┤
│ Reason │ Origin │ Action │ Count │
├─────────────────────────────────────────────┼──────────┼────────┼───────┤
│ crowdsecurity/apache_log4j2_cve-2021-44228 │ crowdsec │ ban │ 1 │
│ crowdsecurity/apache_log4j2_cve-2021-44228 │ CAPI │ ban │ 31 │
│ crowdsecurity/fortinet-cve-2018-13379 │ CAPI │ ban │ 15 │
│ crowdsecurity/http-generic-bf │ CAPI │ ban │ 19 │
│ crowdsecurity/http-path-traversal-probing │ CAPI │ ban │ 158 │
│ crowdsecurity/jira_cve-2021-26086 │ CAPI │ ban │ 8 │
│ crowdsecurity/CVE-2024-9474 │ CAPI │ ban │ 1 │
│ crowdsecurity/http-crawl-non_statics │ CAPI │ ban │ 636 │
│ crowdsecurity/nextcloud-bf │ CAPI │ ban │ 2 │
│ crowdsecurity/ssh-slow-bf │ CAPI │ ban │ 4304 │
│ crowdsecurity/CVE-2023-22515 │ CAPI │ ban │ 3 │
│ crowdsecurity/http-backdoors-attempts │ CAPI │ ban │ 480 │
│ crowdsecurity/netgear_rce │ CAPI │ ban │ 210 │
│ crowdsecurity/spring4shell_cve-2022-22965 │ CAPI │ ban │ 3 │
│ crowdsecurity/CVE-2022-37042 │ CAPI │ ban │ 4 │
│ crowdsecurity/nginx-req-limit-exceeded │ CAPI │ ban │ 438 │
│ crowdsecurity/CVE-2022-35914 │ CAPI │ ban │ 4 │
│ crowdsecurity/http-bad-user-agent │ CAPI │ ban │ 4407 │
│ crowdsecurity/http-bad-user-agent │ crowdsec │ ban │ 1 │
│ crowdsecurity/ssh-cve-2024-6387 │ CAPI │ ban │ 91 │
│ crowdsecurity/CVE-2024-0012 │ CAPI │ ban │ 3 │
│ crowdsecurity/dovecot-spam │ CAPI │ ban │ 770 │
│ crowdsecurity/http-admin-interface-probing │ CAPI │ ban │ 600 │
│ crowdsecurity/http-cve-2021-41773 │ CAPI │ ban │ 115 │
│ crowdsecurity/http-cve-probing │ CAPI │ ban │ 10 │
│ crowdsecurity/http-wordpress-scan │ CAPI │ ban │ 970 │
│ crowdsecurity/ssh-bf │ CAPI │ ban │ 4577 │
│ crowdsecurity/ssh-bf │ crowdsec │ ban │ 14 │
│ crowdsecurity/thinkphp-cve-2018-20062 │ CAPI │ ban │ 65 │
│ crowdsecurity/CVE-2022-26134 │ CAPI │ ban │ 15 │
│ crowdsecurity/CVE-2023-49103 │ CAPI │ ban │ 76 │
│ crowdsecurity/grafana-cve-2021-43798 │ CAPI │ ban │ 5 │
│ crowdsecurity/http-open-proxy │ CAPI │ ban │ 1764 │
│ crowdsecurity/postfix-spam │ CAPI │ ban │ 237 │
│ crowdsecurity/postfix-spam │ crowdsec │ ban │ 1 │
│ crowdsecurity/vmware-vcenter-vmsa-2021-0027 │ CAPI │ ban │ 2 │
│ ltsich/http-w00tw00t │ CAPI │ ban │ 2 │
│ crowdsecurity/CVE-2017-9841 │ CAPI │ ban │ 267 │
│ crowdsecurity/CVE-2023-22518 │ CAPI │ ban │ 2 │
│ crowdsecurity/f5-big-ip-cve-2020-5902 │ CAPI │ ban │ 4 │
│ crowdsecurity/http-cve-2021-42013 │ CAPI │ ban │ 27 │
│ crowdsecurity/http-probing │ CAPI │ ban │ 3273 │
│ crowdsecurity/http-sensitive-files │ CAPI │ ban │ 272 │
│ crowdsecurity/CVE-2019-18935 │ CAPI │ ban │ 24 │
╰─────────────────────────────────────────────┴──────────┴────────┴───────╯
╭──────────────────────────────────────╮
│ Local API Metrics │
├──────────────────────┬────────┬──────┤
│ Route │ Method │ Hits │
├──────────────────────┼────────┼──────┤
│ /v1/alerts │ POST │ 69 │
│ /v1/decisions/stream │ GET │ 5446 │
│ /v1/heartbeat │ GET │ 907 │
│ /v1/usage-metrics │ POST │ 91 │
│ /v1/watchers/login │ POST │ 16 │
╰──────────────────────┴────────┴──────╯
╭──────────────────────────────────────────────────────────────────╮
│ Local API Bouncers Metrics │
├───────────────────────────┬──────────────────────┬────────┬──────┤
│ Bouncer │ Route │ Method │ Hits │
├───────────────────────────┼──────────────────────┼────────┼──────┤
│ crowdsec-firewall-bouncer │ /v1/decisions/stream │ GET │ 5446 │
╰───────────────────────────┴──────────────────────┴────────┴──────╯
╭──────────────────────────────────────────────────────────────────────────────────╮
│ Local API Machines Metrics │
├──────────────────────────────────────────────────┬───────────────┬────────┬──────┤
│ Machine │ Route │ Method │ Hits │
├──────────────────────────────────────────────────┼───────────────┼────────┼──────┤
│ c067eae1ebe243b29bc7d3ba65520ef95otiH8edD5H18qr3 │ /v1/heartbeat │ GET │ 907 │
│ c067eae1ebe243b29bc7d3ba65520ef95otiH8edD5H18qr3 │ /v1/alerts │ POST │ 69 │
╰──────────────────────────────────────────────────┴───────────────┴────────┴──────╯
╭───────────────────────────────────────────────────────────────────────╮
│ Parser Metrics │
├────────────────────────────────────────┬─────────┬─────────┬──────────┤
│ Parsers │ Hits │ Parsed │ Unparsed │
├────────────────────────────────────────┼─────────┼─────────┼──────────┤
│ child-child-crowdsecurity/postfix-logs │ 4.33k │ 10 │ 4.32k │
│ child-crowdsecurity/dovecot-logs │ 6.39k │ 593 │ 5.80k │
│ child-crowdsecurity/http-logs │ 1.40M │ 951.42k │ 452.46k │
│ child-crowdsecurity/nginx-logs │ 495.55k │ 467.96k │ 27.59k │
│ child-crowdsecurity/postfix-logs │ 19.94k │ 1.95k │ 17.99k │
│ child-crowdsecurity/sshd-logs │ 8.08k │ 460 │ 7.62k │
│ child-crowdsecurity/syslog-logs │ 10.63k │ 10.63k │ - │
│ crowdsecurity/dateparse-enrich │ 470.96k │ 470.96k │ - │
│ crowdsecurity/dovecot-logs │ 2.04k │ 593 │ 1.45k │
│ crowdsecurity/geoip-enrich │ 470.96k │ 470.96k │ - │
│ crowdsecurity/http-logs │ 467.96k │ 467.95k │ 12 │
│ crowdsecurity/nextcloud-whitelist │ 467.96k │ 467.96k │ - │
│ crowdsecurity/nginx-logs │ 467.96k │ 467.96k │ - │
│ crowdsecurity/non-syslog │ 467.96k │ 467.96k │ - │
│ crowdsecurity/postfix-logs │ 6.27k │ 1.95k │ 4.32k │
│ crowdsecurity/sshd-logs │ 828 │ 460 │ 368 │
│ crowdsecurity/syslog-logs │ 10.63k │ 10.63k │ - │
│ crowdsecurity/whitelists │ 470.96k │ 470.96k │ - │
╰────────────────────────────────────────┴─────────┴─────────┴──────────╯
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Scenario Metrics │
├────────────────────────────────────────────┬───────────────┬───────────┬──────────────┬────────┬─────────┤
│ Scenario │ Current Count │ Overflows │ Instantiated │ Poured │ Expired │
├────────────────────────────────────────────┼───────────────┼───────────┼──────────────┼────────┼─────────┤
│ LePresidente/http-generic-403-bf │ - │ - │ 10 │ 20 │ 10 │
│ crowdsecurity/CVE-2022-41082 │ - │ 2 │ 2 │ - │ - │
│ crowdsecurity/apache_log4j2_cve-2021-44228 │ - │ 2 │ 2 │ - │ - │
│ crowdsecurity/dovecot-spam │ - │ - │ 7 │ 7 │ 7 │
│ crowdsecurity/http-admin-interface-probing │ - │ - │ 1 │ 2 │ 1 │
│ crowdsecurity/http-backdoors-attempts │ - │ - │ 1 │ 1 │ 1 │
│ crowdsecurity/http-bad-user-agent │ - │ 11 │ 11 │ 22 │ - │
│ crowdsecurity/http-crawl-non_statics │ 26 │ - │ 48.64k │ 50.34k │ 48.61k │
│ crowdsecurity/http-probing │ 44 │ - │ 19.57k │ 21.48k │ 19.53k │
│ crowdsecurity/http-sensitive-files │ - │ - │ 18 │ 18 │ 18 │
│ crowdsecurity/netgear_rce │ - │ 2 │ 2 │ - │ - │
│ crowdsecurity/postfix-spam │ - │ 2 │ 1.06k │ 1.95k │ 1.06k │
│ crowdsecurity/ssh-bf │ - │ 59 │ 108 │ 456 │ 49 │
│ crowdsecurity/ssh-bf_user-enum │ - │ - │ 65 │ 108 │ 65 │
│ crowdsecurity/ssh-slow-bf │ - │ 1 │ 66 │ 456 │ 65 │
│ crowdsecurity/ssh-slow-bf_user-enum │ - │ - │ 65 │ 108 │ 65 │
╰────────────────────────────────────────────┴───────────────┴───────────┴──────────────┴────────┴─────────╯
╭────────────────────────────────────────────────────────────────────────────────╮
│ Whitelist Metrics │
├───────────────────────────────────┬─────────────────────┬────────┬─────────────┤
│ Whitelist │ Reason │ Hits │ Whitelisted │
├───────────────────────────────────┼─────────────────────┼────────┼─────────────┤
│ crowdsecurity/nextcloud-whitelist │ Nextcloud Whitelist │ 467960 │ - │
│ crowdsecurity/whitelists │ my ip ranges │ 470960 │ 25066 │
╰───────────────────────────────────┴─────────────────────┴────────┴─────────────╯


</details>


### Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

<details>

</details>
@bjo81 bjo81 added the kind/bug Something isn't working label Feb 15, 2025
@github-actions GitHub Actions
Copy link

@bjo81: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

  1. Check Crowdsec Documentation to see if your issue can be self resolved.
  2. You can also join our Discord.
  3. Check Releases to make sure your agent is on the latest version.
Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

@LaurenceJJones
Copy link
Contributor

LaurenceJJones commented Feb 15, 2025 •
edited
Loading

What happened?

Unfortunately, the dovecot and postfix scenarios didn't seem to work as expected, so I wanted to try

cscli explain --dsn "journalctl://filters=_SYSTEMD_UNIT=dovecot.service" --type syslog

But crowdsec ate 6GB of memory (the system has 8GB memory and 4GB swap) and got oomkilled.

[1274714.165203] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/session-116.scope,task=crowdsec,pid=208475,uid=0
[1274714.165254] Out of memory: Killed process 208475 (crowdsec) total-vm:9781416kB, anon-rss:6006508kB, file-rss:192kB, shmem-rss:0kB, UID:0 pgtables:15276kB oom_score_adj:0

What did you expect to happen?

Getting an explaination of the log parser. Maybe some restriction regarding the time period would be helpful.

How can we reproduce it (as minimally and precisely as possible)?

Try to use a journalctl journal of 14 days.

Anything else we need to know?

No response

Crowdsec version

version: v1.6.5-d8dcdc913
Codename: alphaga
BuildDate: 2025-02-14_15:49:33
GoVersion: 1.24.0
Platform: linux
libre2: C++
User-Agent: crowdsec/v1.6.5-d8dcdc913-linux
Constraint_parser: >= 1.0, <= 3.0
Constraint_scenario: >= 1.0, <= 3.0
Constraint_api: v1
Constraint_acquis: >= 1.0, < 2.0
Built-in optional components: cscli_setup, datasource_appsec, datasource_cloudwatch, datasource_docker, datasource_file, datasource_http, datasource_journalctl, datasource_k8s-audit, datasource_kafka, datasource_kinesis, datasource_loki, datasource_s3, datasource_syslog, datasource_victorialogs, datasource_wineventlog

OS version

NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
ANSI_COLOR="38;2;23;147;209"
HOME_URL="https://archlinux.org/"
DOCUMENTATION_URL="https://wiki.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://gitlab.archlinux.org/groups/archlinux/-/issues"
PRIVACY_POLICY_URL="https://terms.archlinux.org/docs/privacy-policy/"
LOGO=archlinux-logo

Linux host.domain.org 6.6.72-1-lts #1 SMP PREEMPT_DYNAMIC Fri, 17 Jan 2025 14:04:26 +0000 x86_64 GNU/Linux

Enabled collections and parsers

update for collection crowdsecurity/postfix available (currently:0.2, latest:0.4)
crowdsecurity/nextcloud is outdated because of parsers:crowdsecurity/nextcloud-logs
crowdsecurity/nextcloud is outdated because of parsers:crowdsecurity/nextcloud-whitelist
Loaded: 133 parsers, 10 postoverflows, 751 scenarios, 8 contexts, 4 appsec-configs, 93 appsec-rules, 131 collections
Unmanaged items: 1 local, 0 tainted
name,status,version,description,type
crowdsecurity/dateparse-enrich,enabled,0.2,,parsers
crowdsecurity/dovecot-logs,enabled,0.9,Parse dovecot logs,parsers
crowdsecurity/geoip-enrich,enabled,0.5,"Populate event with geoloc info : as, country, coords, source range.",parsers
crowdsecurity/http-logs,enabled,1.3,"Parse more Specifically HTTP logs, such as HTTP Code, HTTP path, HTTP args and if its a static ressource",parsers
crowdsecurity/nextcloud-logs,"enabled,update-available",0.2,Parse nextcloud logs,parsers
crowdsecurity/nextcloud-whitelist,"enabled,update-available",0.3,Whitelist events from nextcloud,parsers
crowdsecurity/nginx-logs,enabled,1.7,Parse nginx access and error logs,parsers
crowdsecurity/postfix-logs,enabled,0.8,Parse postfix logs,parsers
crowdsecurity/postscreen-logs,"enabled,update-available",0.2,Parse postscreen logs,parsers
crowdsecurity/sshd-logs,enabled,2.9,Parse openSSH logs,parsers
crowdsecurity/syslog-logs,enabled,0.8,,parsers
crowdsecurity/whitelists,"enabled,local",,,parsers
crowdsecurity/apache_log4j2_cve-2021-44228,enabled,0.6,Detect cve-2021-44228 exploitation attemps,scenarios
crowdsecurity/CVE-2017-9841,enabled,0.2,Detect CVE-2017-9841 exploits,scenarios
crowdsecurity/CVE-2019-18935,enabled,0.2,Detect Telerik CVE-2019-18935 exploitation attempts,scenarios
crowdsecurity/CVE-2022-26134,enabled,0.2,Detect CVE-2022-26134 exploits,scenarios
crowdsecurity/CVE-2022-35914,enabled,0.2,Detect CVE-2022-35914 exploits,scenarios
crowdsecurity/CVE-2022-37042,enabled,0.2,Detect CVE-2022-37042 exploits,scenarios
crowdsecurity/CVE-2022-40684,enabled,0.3,Detect cve-2022-40684 exploitation attempts,scenarios
crowdsecurity/CVE-2022-41082,enabled,0.4,Detect CVE-2022-41082 exploits,scenarios
crowdsecurity/CVE-2022-41697,enabled,0.2,Detect CVE-2022-41697 enumeration,scenarios
crowdsecurity/CVE-2022-42889,enabled,0.3,Detect CVE-2022-42889 exploits (Text4Shell),scenarios
crowdsecurity/CVE-2022-44877,enabled,0.3,Detect CVE-2022-44877 exploits,scenarios
crowdsecurity/CVE-2022-46169,enabled,0.2,Detect CVE-2022-46169 brute forcing,scenarios
crowdsecurity/CVE-2023-22515,enabled,0.1,Detect CVE-2023-22515 exploitation,scenarios
crowdsecurity/CVE-2023-22518,enabled,0.2,Detect CVE-2023-22518 exploits,scenarios
crowdsecurity/CVE-2023-49103,enabled,0.3,Detect owncloud CVE-2023-49103 exploitation attempts,scenarios
crowdsecurity/CVE-2024-0012,enabled,0.1,Detect CVE-2024-0012 exploitation attempts,scenarios
crowdsecurity/CVE-2024-38475,enabled,0.1,Detect CVE-2024-38475 exploitation attempts,scenarios
crowdsecurity/CVE-2024-9474,enabled,0.1,Detect CVE-2024-9474 exploitation attempts,scenarios
crowdsecurity/dovecot-spam,enabled,0.5,detect errors on dovecot,scenarios
crowdsecurity/f5-big-ip-cve-2020-5902,enabled,0.2,Detect cve-2020-5902 exploitation attemps,scenarios
crowdsecurity/fortinet-cve-2018-13379,enabled,0.3,Detect cve-2018-13379 exploitation attemps,scenarios
crowdsecurity/grafana-cve-2021-43798,enabled,0.2,Detect cve-2021-43798 exploitation attemps,scenarios
crowdsecurity/http-admin-interface-probing,enabled,0.4,Detect generic HTTP admin interface probing,scenarios
crowdsecurity/http-backdoors-attempts,enabled,0.6,Detect attempt to common backdoors,scenarios
crowdsecurity/http-bad-user-agent,enabled,1.2,Detect usage of bad User Agent,scenarios
crowdsecurity/http-crawl-non_statics,enabled,0.7,Detect aggressive crawl on non static resources,scenarios
crowdsecurity/http-cve-2021-41773,enabled,0.3,Apache - Path Traversal (CVE-2021-41773),scenarios
crowdsecurity/http-cve-2021-42013,enabled,0.3,Apache - Path Traversal (CVE-2021-42013),scenarios
crowdsecurity/http-cve-probing,enabled,0.2,Detect generic HTTP cve probing,scenarios
crowdsecurity/http-generic-bf,enabled,0.9,Detect generic http brute force,scenarios
crowdsecurity/http-open-proxy,enabled,0.5,Detect scan for open proxy,scenarios
crowdsecurity/http-path-traversal-probing,enabled,0.4,Detect path traversal attempt,scenarios
crowdsecurity/http-probing,enabled,0.4,Detect site scanning/probing from a single ip,scenarios
crowdsecurity/http-sensitive-files,enabled,0.4,"Detect attempt to access to sensitive files (.log, .db ..) or folders (.git)",scenarios
crowdsecurity/http-sqli-probing,enabled,0.4,A scenario that detects SQL injection probing with minimal false positives,scenarios
crowdsecurity/http-wordpress-scan,enabled,0.2,Detect WordPress scan: vuln hunting,scenarios
crowdsecurity/http-xss-probing,enabled,0.4,A scenario that detects XSS probing with minimal false positives,scenarios
crowdsecurity/jira_cve-2021-26086,enabled,0.3,Detect Atlassian Jira CVE-2021-26086 exploitation attemps,scenarios
crowdsecurity/netgear_rce,enabled,0.4,Detect Netgear RCE DGN1000/DGN220 exploitation attempts,scenarios
crowdsecurity/nextcloud-bf,enabled,0.3,Detect Nextcloud bruteforce,scenarios
crowdsecurity/nginx-req-limit-exceeded,enabled,0.3,Detects IPs which violate nginx's user set request limit.,scenarios
crowdsecurity/postfix-spam,enabled,0.4,Detect spammers,scenarios
crowdsecurity/pulse-secure-sslvpn-cve-2019-11510,enabled,0.3,Detect cve-2019-11510 exploitation attemps,scenarios
crowdsecurity/spring4shell_cve-2022-22965,enabled,0.3,Detect cve-2022-22965 probing,scenarios
crowdsecurity/ssh-bf,enabled,0.3,Detect ssh bruteforce,scenarios
crowdsecurity/ssh-cve-2024-6387,enabled,0.2,Detect exploitation attempt of CVE-2024-6387,scenarios
crowdsecurity/ssh-slow-bf,enabled,0.4,Detect slow ssh bruteforce,scenarios
crowdsecurity/thinkphp-cve-2018-20062,enabled,0.6,Detect ThinkPHP CVE-2018-20062 exploitation attemps,scenarios
crowdsecurity/vmware-cve-2022-22954,enabled,0.3,Detect Vmware CVE-2022-22954 exploitation attempts,scenarios
crowdsecurity/vmware-vcenter-vmsa-2021-0027,enabled,0.2,Detect VMSA-2021-0027 exploitation attemps,scenarios
ltsich/http-w00tw00t,enabled,0.2,detect w00tw00t,scenarios
crowdsecurity/bf_base,enabled,0.1,,contexts
crowdsecurity/http_base,enabled,0.2,,contexts
crowdsecurity/base-http-scenarios,enabled,1.0,http common : scanners detection,collections
crowdsecurity/dovecot,enabled,0.1,dovecot support : parser and spammer detection,collections
crowdsecurity/http-cve,enabled,2.9,Detect CVE exploitation in http logs,collections
crowdsecurity/linux,enabled,0.2,core linux support : syslog+geoip+ssh,collections
crowdsecurity/nextcloud,"enabled,update-available",0.3,Nextcloud support : parser and brute-force detection,collections
crowdsecurity/nginx,enabled,0.2,nginx support : parser and generic http scenarios,collections
crowdsecurity/postfix,"enabled,update-available",0.2,postfix support : parser and spammer d

Acquisition config

``` #Generated acquisition file - wizard.sh (service: nginx) / files : /var/log/nginx/access.log /var/log/nginx/error.log filenames: - /var/log/nginx/access.log - /var/log/nginx/error.log labels: type: nginx --- #Generated acquisition file - wizard.sh (service: sshd) / files : journalctl_filter: - _SYSTEMD_UNIT=sshd.service labels: type: syslog --- journalctl_filter: - _SYSTEMD_UNIT=postfix.service labels: type: syslog --- journalctl_filter: - _SYSTEMD_UNIT=dovecot.service labels: type: syslog --- #Generated acquisition file - wizard.sh (service: nginx) / files : /var/log/nginx/access.log /var/log/nginx/error.log filenames: - /var/log/nginx/access.log - /var/log/nginx/error.log labels: type: nginx --- #Generated acquisition file - wizard.sh (service: sshd) / files : journalctl_filter: - _SYSTEMD_UNIT=sshd.service labels: type: syslog --- 

</details>


### Config show

<details>

Global:

  • Configuration Folder : /etc/crowdsec

  • Data Folder : /var/lib/crowdsec/data

  • Hub Folder : /etc/crowdsec/hub

  • Simulation File : /etc/crowdsec/simulation.yaml

  • Log Folder : /var/log

  • Log level : info

  • Log Media : file
    Crowdsec:

  • Acquisition File : /etc/crowdsec/acquis.yaml

  • Parsers routines : 1

  • Acquisition Folder : /etc/crowdsec/acquis.d
    cscli:

  • Output : human

  • Hub Branch :
    API Client:

  • URL : http://127.0.0.1:8080/

  • Login : c067eae1ebe243b29bc7d3ba65520ef95otiH8edD5H18qr3

  • Credentials File : /etc/crowdsec/local_api_credentials.yaml
    Local API Server:

  • Listen URL : 127.0.0.1:8080

  • Listen Socket :

  • Profile File : /etc/crowdsec/profiles.yaml

  • Trusted IPs:

    • 127.0.0.1
    • ::1

  • Database:

    • Type : sqlite
    • Path : /var/lib/crowdsec/data/crowdsec.db
    • Flush age : 7d
    • Flush size : 5000

</details>


### Prometheus metrics

<details>

╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Acquisition Metrics │
├───────────────────────────────────────────────────────┬────────────┬──────────────┬────────────────┬────────────────────────┬───────────────────┤
│ Source │ Lines read │ Lines parsed │ Lines unparsed │ Lines poured to bucket │ Lines whitelisted │
├───────────────────────────────────────────────────────┼────────────┼──────────────┼────────────────┼────────────────────────┼───────────────────┤
│ file:/var/log/nginx/access.log │ 440.42k │ 440.42k │ - │ 58.06k │ 24.93k │
│ file:/var/log/nginx/error.log │ 27.55k │ 27.55k │ - │ 13.82k │ 6 │
│ journalctl:journalctl-%s_SYSTEMD_UNIT=dovecot.service │ 2.04k │ 593 │ 1.45k │ 7 │ 128 │
│ journalctl:journalctl-%s_SYSTEMD_UNIT=postfix.service │ 7.76k │ 1.95k │ 5.81k │ 1.95k │ - │
│ journalctl:journalctl-%s_SYSTEMD_UNIT=sshd.service │ 832 │ 460 │ 372 │ 1.13k │ - │
╰───────────────────────────────────────────────────────┴────────────┴──────────────┴────────────────┴────────────────────────┴───────────────────╯
╭────────────────────────────────────────────────────╮
│ Local API Alerts │
├────────────────────────────────────────────┬───────┤
│ Reason │ Count │
├────────────────────────────────────────────┼───────┤
│ crowdsecurity/ssh-slow-bf │ 1 │
│ crowdsecurity/CVE-2022-41082 │ 1 │
│ crowdsecurity/apache_log4j2_cve-2021-44228 │ 1 │
│ crowdsecurity/http-bad-user-agent │ 9 │
│ crowdsecurity/netgear_rce │ 1 │
│ crowdsecurity/postfix-spam │ 1 │
│ crowdsecurity/ssh-bf │ 71 │
╰────────────────────────────────────────────┴───────╯
╭───────────────────────────────────────────────────────────────────────────────────────────╮
│ Bouncer Metrics (crowdsec-firewall-bouncer) since 2025-02-14 17:14:18 +0000 UTC │
├────────────────────────────┬──────────────────┬───────────────────┬───────────────────────┤
│ Origin │ active_decisions │ dropped │ processed │
│ │ IPs │ bytes │ packets │ bytes │ packets │
├────────────────────────────┼──────────────────┼─────────┼─────────┼───────────┼───────────┤
│ CAPI (community blocklist) │ 23.89k │ 938.36k │ 15.76k │ - │ - │
│ crowdsec (security engine) │ 18 │ 132.92k │ 2.09k │ - │ - │
├────────────────────────────┼──────────────────┼─────────┼─────────┼───────────┼───────────┤
│ Total │ 23.91k │ 1.07M │ 17.85k │ 16.48G │ 35.48M │
╰────────────────────────────┴──────────────────┴─────────┴─────────┴───────────┴───────────╯
╭─────────────────────────────────────────────────────────────────────────╮
│ Local API Decisions │
├─────────────────────────────────────────────┬──────────┬────────┬───────┤
│ Reason │ Origin │ Action │ Count │
├─────────────────────────────────────────────┼──────────┼────────┼───────┤
│ crowdsecurity/apache_log4j2_cve-2021-44228 │ crowdsec │ ban │ 1 │
│ crowdsecurity/apache_log4j2_cve-2021-44228 │ CAPI │ ban │ 31 │
│ crowdsecurity/fortinet-cve-2018-13379 │ CAPI │ ban │ 15 │
│ crowdsecurity/http-generic-bf │ CAPI │ ban │ 19 │
│ crowdsecurity/http-path-traversal-probing │ CAPI │ ban │ 158 │
│ crowdsecurity/jira_cve-2021-26086 │ CAPI │ ban │ 8 │
│ crowdsecurity/CVE-2024-9474 │ CAPI │ ban │ 1 │
│ crowdsecurity/http-crawl-non_statics │ CAPI │ ban │ 636 │
│ crowdsecurity/nextcloud-bf │ CAPI │ ban │ 2 │
│ crowdsecurity/ssh-slow-bf │ CAPI │ ban │ 4304 │
│ crowdsecurity/CVE-2023-22515 │ CAPI │ ban │ 3 │
│ crowdsecurity/http-backdoors-attempts │ CAPI │ ban │ 480 │
│ crowdsecurity/netgear_rce │ CAPI │ ban │ 210 │
│ crowdsecurity/spring4shell_cve-2022-22965 │ CAPI │ ban │ 3 │
│ crowdsecurity/CVE-2022-37042 │ CAPI │ ban │ 4 │
│ crowdsecurity/nginx-req-limit-exceeded │ CAPI │ ban │ 438 │
│ crowdsecurity/CVE-2022-35914 │ CAPI │ ban │ 4 │
│ crowdsecurity/http-bad-user-agent │ CAPI │ ban │ 4407 │
│ crowdsecurity/http-bad-user-agent │ crowdsec │ ban │ 1 │
│ crowdsecurity/ssh-cve-2024-6387 │ CAPI │ ban │ 91 │
│ crowdsecurity/CVE-2024-0012 │ CAPI │ ban │ 3 │
│ crowdsecurity/dovecot-spam │ CAPI │ ban │ 770 │
│ crowdsecurity/http-admin-interface-probing │ CAPI │ ban │ 600 │
│ crowdsecurity/http-cve-2021-41773 │ CAPI │ ban │ 115 │
│ crowdsecurity/http-cve-probing │ CAPI │ ban │ 10 │
│ crowdsecurity/http-wordpress-scan │ CAPI │ ban │ 970 │
│ crowdsecurity/ssh-bf │ CAPI │ ban │ 4577 │
│ crowdsecurity/ssh-bf │ crowdsec │ ban │ 14 │
│ crowdsecurity/thinkphp-cve-2018-20062 │ CAPI │ ban │ 65 │
│ crowdsecurity/CVE-2022-26134 │ CAPI │ ban │ 15 │
│ crowdsecurity/CVE-2023-49103 │ CAPI │ ban │ 76 │
│ crowdsecurity/grafana-cve-2021-43798 │ CAPI │ ban │ 5 │
│ crowdsecurity/http-open-proxy │ CAPI │ ban │ 1764 │
│ crowdsecurity/postfix-spam │ CAPI │ ban │ 237 │
│ crowdsecurity/postfix-spam │ crowdsec │ ban │ 1 │
│ crowdsecurity/vmware-vcenter-vmsa-2021-0027 │ CAPI │ ban │ 2 │
│ ltsich/http-w00tw00t │ CAPI │ ban │ 2 │
│ crowdsecurity/CVE-2017-9841 │ CAPI │ ban │ 267 │
│ crowdsecurity/CVE-2023-22518 │ CAPI │ ban │ 2 │
│ crowdsecurity/f5-big-ip-cve-2020-5902 │ CAPI │ ban │ 4 │
│ crowdsecurity/http-cve-2021-42013 │ CAPI │ ban │ 27 │
│ crowdsecurity/http-probing │ CAPI │ ban │ 3273 │
│ crowdsecurity/http-sensitive-files │ CAPI │ ban │ 272 │
│ crowdsecurity/CVE-2019-18935 │ CAPI │ ban │ 24 │
╰─────────────────────────────────────────────┴──────────┴────────┴───────╯
╭──────────────────────────────────────╮
│ Local API Metrics │
├──────────────────────┬────────┬──────┤
│ Route │ Method │ Hits │
├──────────────────────┼────────┼──────┤
│ /v1/alerts │ POST │ 69 │
│ /v1/decisions/stream │ GET │ 5446 │
│ /v1/heartbeat │ GET │ 907 │
│ /v1/usage-metrics │ POST │ 91 │
│ /v1/watchers/login │ POST │ 16 │
╰──────────────────────┴────────┴──────╯
╭──────────────────────────────────────────────────────────────────╮
│ Local API Bouncers Metrics │
├───────────────────────────┬──────────────────────┬────────┬──────┤
│ Bouncer │ Route │ Method │ Hits │
├───────────────────────────┼──────────────────────┼────────┼──────┤
│ crowdsec-firewall-bouncer │ /v1/decisions/stream │ GET │ 5446 │
╰───────────────────────────┴──────────────────────┴────────┴──────╯
╭──────────────────────────────────────────────────────────────────────────────────╮
│ Local API Machines Metrics │
├──────────────────────────────────────────────────┬───────────────┬────────┬──────┤
│ Machine │ Route │ Method │ Hits │
├──────────────────────────────────────────────────┼───────────────┼────────┼──────┤
│ c067eae1ebe243b29bc7d3ba65520ef95otiH8edD5H18qr3 │ /v1/heartbeat │ GET │ 907 │
│ c067eae1ebe243b29bc7d3ba65520ef95otiH8edD5H18qr3 │ /v1/alerts │ POST │ 69 │
╰──────────────────────────────────────────────────┴───────────────┴────────┴──────╯
╭───────────────────────────────────────────────────────────────────────╮
│ Parser Metrics │
├────────────────────────────────────────┬─────────┬─────────┬──────────┤
│ Parsers │ Hits │ Parsed │ Unparsed │
├────────────────────────────────────────┼─────────┼─────────┼──────────┤
│ child-child-crowdsecurity/postfix-logs │ 4.33k │ 10 │ 4.32k │
│ child-crowdsecurity/dovecot-logs │ 6.39k │ 593 │ 5.80k │
│ child-crowdsecurity/http-logs │ 1.40M │ 951.42k │ 452.46k │
│ child-crowdsecurity/nginx-logs │ 495.55k │ 467.96k │ 27.59k │
│ child-crowdsecurity/postfix-logs │ 19.94k │ 1.95k │ 17.99k │
│ child-crowdsecurity/sshd-logs │ 8.08k │ 460 │ 7.62k │
│ child-crowdsecurity/syslog-logs │ 10.63k │ 10.63k │ - │
│ crowdsecurity/dateparse-enrich │ 470.96k │ 470.96k │ - │
│ crowdsecurity/dovecot-logs │ 2.04k │ 593 │ 1.45k │
│ crowdsecurity/geoip-enrich │ 470.96k │ 470.96k │ - │
│ crowdsecurity/http-logs │ 467.96k │ 467.95k │ 12 │
│ crowdsecurity/nextcloud-whitelist │ 467.96k │ 467.96k │ - │
│ crowdsecurity/nginx-logs │ 467.96k │ 467.96k │ - │
│ crowdsecurity/non-syslog │ 467.96k │ 467.96k │ - │
│ crowdsecurity/postfix-logs │ 6.27k │ 1.95k │ 4.32k │
│ crowdsecurity/sshd-logs │ 828 │ 460 │ 368 │
│ crowdsecurity/syslog-logs │ 10.63k │ 10.63k │ - │
│ crowdsecurity/whitelists │ 470.96k │ 470.96k │ - │
╰────────────────────────────────────────┴─────────┴─────────┴──────────╯
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Scenario Metrics │
├────────────────────────────────────────────┬───────────────┬───────────┬──────────────┬────────┬─────────┤
│ Scenario │ Current Count │ Overflows │ Instantiated │ Poured │ Expired │
├────────────────────────────────────────────┼───────────────┼───────────┼──────────────┼────────┼─────────┤
│ LePresidente/http-generic-403-bf │ - │ - │ 10 │ 20 │ 10 │
│ crowdsecurity/CVE-2022-41082 │ - │ 2 │ 2 │ - │ - │
│ crowdsecurity/apache_log4j2_cve-2021-44228 │ - │ 2 │ 2 │ - │ - │
│ crowdsecurity/dovecot-spam │ - │ - │ 7 │ 7 │ 7 │
│ crowdsecurity/http-admin-interface-probing │ - │ - │ 1 │ 2 │ 1 │
│ crowdsecurity/http-backdoors-attempts │ - │ - │ 1 │ 1 │ 1 │
│ crowdsecurity/http-bad-user-agent │ - │ 11 │ 11 │ 22 │ - │
│ crowdsecurity/http-crawl-non_statics │ 26 │ - │ 48.64k │ 50.34k │ 48.61k │
│ crowdsecurity/http-probing │ 44 │ - │ 19.57k │ 21.48k │ 19.53k │
│ crowdsecurity/http-sensitive-files │ - │ - │ 18 │ 18 │ 18 │
│ crowdsecurity/netgear_rce │ - │ 2 │ 2 │ - │ - │
│ crowdsecurity/postfix-spam │ - │ 2 │ 1.06k │ 1.95k │ 1.06k │
│ crowdsecurity/ssh-bf │ - │ 59 │ 108 │ 456 │ 49 │
│ crowdsecurity/ssh-bf_user-enum │ - │ - │ 65 │ 108 │ 65 │
│ crowdsecurity/ssh-slow-bf │ - │ 1 │ 66 │ 456 │ 65 │
│ crowdsecurity/ssh-slow-bf_user-enum │ - │ - │ 65 │ 108 │ 65 │
╰────────────────────────────────────────────┴───────────────┴───────────┴──────────────┴────────┴─────────╯
╭────────────────────────────────────────────────────────────────────────────────╮
│ Whitelist Metrics │
├───────────────────────────────────┬─────────────────────┬────────┬─────────────┤
│ Whitelist │ Reason │ Hits │ Whitelisted │
├───────────────────────────────────┼─────────────────────┼────────┼─────────────┤
│ crowdsecurity/nextcloud-whitelist │ Nextcloud Whitelist │ 467960 │ - │
│ crowdsecurity/whitelists │ my ip ranges │ 470960 │ 25066 │
╰───────────────────────────────────┴─────────────────────┴────────┴─────────────╯


</details>


### Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

<details>

</details>

Since the DSN is invoking the journalctl binary you should be able to pass the since parameter as a query for example.

&since="15 minutes ago"

But yeah any options you can pass to journalctl you can pass them as query opts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug Something isn't working needs/triage os/linux version/1.6.5
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bjo81 @LaurenceJJones