Skip to content

Commit

Permalink
enhance: Remove as much WAN ips from items that may cause CAPI signals
Browse files Browse the repository at this point in the history
  • Loading branch information
@LaurenceJJones
LaurenceJJones committed Feb 3, 2025
1 parent c08bbe2 commit 2fecde0
Show file tree
Hide file tree
Showing 43 changed files with 273 additions and 273 deletions.
6 changes: 3 additions & 3 deletions crowdsec-docs/docs/appsec/hooks.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ inband_rules:
- crowdsecurity/base-config
- crowdsecurity/vpatch-*
pre_eval:
- filter: IsInBand == true && req.RemoteAddr == "42.42.42.42"
- filter: IsInBand == true && req.RemoteAddr == "192.168.1.1"
apply:
- RemoveInBandRuleByName("my_rule")
```
Expand Down Expand Up @@ -188,13 +188,13 @@ inband_rules:
- crowdsecurity/base-config
- crowdsecurity/vpatch-*
post_eval:
- filter: IsInBand == true && req.RemoteAddr == "42.42.42.42"
- filter: IsInBand == true && req.RemoteAddr == "192.168.1.1"
apply:
- CancelAlert()
- CancelEvent()
- filter: |
any( evt.Appsec.MatchedRules, #.name == "crowdsecurity/vpatch-env-access") and
req.RemoteAddr = "42.42.42.42"
req.RemoteAddr = "192.168.1.1"
apply:
- SetRemediation("allow")
- filter: evt.Appsec.MatchedRules.GetURI() contains "/foobar/"
Expand Down
4 changes: 2 additions & 2 deletions crowdsec-docs/docs/appsec/quickstart/nginxopenresty.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ Please keep this key since you will not be able to retrieve it!
2. Emit a legitimate request to the AppSec Component:

```bash
curl -X POST localhost:7422/ -i -H 'x-crowdsec-appsec-uri: /test' -H 'x-crowdsec-appsec-ip: 42.42.42.42' -H 'x-crowdsec-appsec-host: foobar.com' -H 'x-crowdsec-appsec-verb: POST' -H 'x-crowdsec-appsec-api-key: this_is_a_bad_password'
curl -X POST localhost:7422/ -i -H 'x-crowdsec-appsec-uri: /test' -H 'x-crowdsec-appsec-ip: 192.168.1.1' -H 'x-crowdsec-appsec-host: foobar.com' -H 'x-crowdsec-appsec-verb: POST' -H 'x-crowdsec-appsec-api-key: this_is_a_bad_password'
```

Which will give us an answer such as:
Expand All @@ -181,7 +181,7 @@ We're trying to access a `.env` file, a [common way to get access to some creden
:::

```bash
curl -X POST localhost:7422/ -i -H 'x-crowdsec-appsec-uri: /.env' -H 'x-crowdsec-appsec-ip: 42.42.42.42' -H 'x-crowdsec-appsec-host: foobar.com' -H 'x-crowdsec-appsec-verb: POST' -H 'x-crowdsec-appsec-api-key: this_is_a_bad_password'
curl -X POST localhost:7422/ -i -H 'x-crowdsec-appsec-uri: /.env' -H 'x-crowdsec-appsec-ip: 192.168.1.1' -H 'x-crowdsec-appsec-host: foobar.com' -H 'x-crowdsec-appsec-verb: POST' -H 'x-crowdsec-appsec-api-key: this_is_a_bad_password'

```

Expand Down
10 changes: 5 additions & 5 deletions crowdsec-docs/docs/appsec/troubleshooting.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ cscli bouncers add appsec_test -k this_is_a_bad_password
> Emit a request to the AppSec Component
```bash
curl -I -X POST localhost:7422/ -i -H 'x-crowdsec-appsec-api-key: this_is_a_bad_password' -H 'x-crowdsec-appsec-ip: 42.42.42.42' -H 'x-crowdsec-appsec-uri: /test' -H 'x-crowdsec-appsec-host: test.com' -H 'x-crowdsec-appsec-verb: GET'
curl -I -X POST localhost:7422/ -i -H 'x-crowdsec-appsec-api-key: this_is_a_bad_password' -H 'x-crowdsec-appsec-ip: 192.168.1.1' -H 'x-crowdsec-appsec-uri: /test' -H 'x-crowdsec-appsec-host: test.com' -H 'x-crowdsec-appsec-verb: GET'
HTTP/1.1 200 OK
Date: Tue, 05 Dec 2023 19:37:56 GMT
Content-Length: 18
Expand All @@ -85,7 +85,7 @@ Content-Type: text/plain; charset=utf-8
If you receive a `200 OK`, you can authenticate to the AppSec Component. If the component is misconfigured or your API key is invalid, you will receive a `401 Unauthorized`:

```bash
curl -I -X POST localhost:7422/ -i -H 'x-crowdsec-appsec-api-key: meeh' -H 'x-crowdsec-appsec-ip: 42.42.42.42' -H 'x-crowdsec-appsec-uri: /test' -H 'x-crowdsec-appsec-host: test.com' -H 'x-crowdsec-appsec-verb: GET'
curl -I -X POST localhost:7422/ -i -H 'x-crowdsec-appsec-api-key: meeh' -H 'x-crowdsec-appsec-ip: 192.168.1.1' -H 'x-crowdsec-appsec-uri: /test' -H 'x-crowdsec-appsec-host: test.com' -H 'x-crowdsec-appsec-verb: GET'
HTTP/1.1 401 Unauthorized
Date: Tue, 05 Dec 2023 19:38:51 GMT
Content-Length: 0
Expand Down Expand Up @@ -212,7 +212,7 @@ cscli bouncers add appsec_test -k this_is_a_bad_password
We can now query our AppSec Component (we're assuming here that it runs on the default `127.0.0.1:7422`, see the `listen_addr` parameter of the acquisition config):

```bash
▶ curl -X POST localhost:7422/ -i -H 'x-crowdsec-appsec-ip: 42.42.42.42' -H 'x-crowdsec-appsec-uri: /rpc2' -H 'x-crowdsec-appsec-host: google.com' -H 'x-crowdsec-appsec-verb: POST' -H 'x-crowdsec-appsec-api-key: this_is_a_bad_password'
▶ curl -X POST localhost:7422/ -i -H 'x-crowdsec-appsec-ip: 192.168.1.1' -H 'x-crowdsec-appsec-uri: /rpc2' -H 'x-crowdsec-appsec-host: google.com' -H 'x-crowdsec-appsec-verb: POST' -H 'x-crowdsec-appsec-api-key: this_is_a_bad_password'
HTTP/1.1 403 Forbidden
Date: Tue, 05 Dec 2023 11:17:51 GMT
Content-Length: 16
Expand All @@ -225,7 +225,7 @@ And we see the alert appearing in `crowdsec.log` :

```
...
INFO[2023-12-05 12:17:52] (test) alert : crowdsecurity/vpatch-CVE-2023-42793 by ip 42.42.42.42
INFO[2023-12-05 12:17:52] (test) alert : crowdsecurity/vpatch-CVE-2023-42793 by ip 192.168.1.1
...
```

Expand All @@ -235,7 +235,7 @@ And in `cscli alerts list` :
╭────┬────────────────┬─────────────────────────────────────┬─────────┬────┬───────────┬───────────────────────────────╮
│ ID │ value │ reason │ country │ as │ decisions │ created_at │
├────┼────────────────┼─────────────────────────────────────┼─────────┼────┼───────────┼───────────────────────────────┤
│ 1 │ Ip:42.42.42.42 │ crowdsecurity/vpatch-CVE-2023-42793 │ │ │ │ 2023-12-05 11:17:51 +0000 UTC │
│ 1 │ Ip:192.168.1.1 │ crowdsecurity/vpatch-CVE-2023-42793 │ │ │ │ 2023-12-05 11:17:51 +0000 UTC │
╰────┴────────────────┴─────────────────────────────────────┴─────────┴────┴───────────┴───────────────────────────────╯
```
4 changes: 2 additions & 2 deletions crowdsec-docs/docs/expr/other_helpers.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ An empty string if the key doesn't exist (or has been evicted), and error is rai

Returns true if it's a valid IPv4.

> `IsIPV4("1.2.3.4")`
> `IsIPV4("192.168.1.1")`
> `IsIPV4(Alert.GetValue())`
Expand All @@ -44,7 +44,7 @@ Returns true if it's a valid IP (v4 or v6).

> `IsIP("2001:0db8:85a3:0000:0000:8a2e:0370:7334")`
> `IsIP("1.2.3.4")`
> `IsIP("192.168.1.1")`
> `IsIP(Alert.GetValue())`
Expand Down
8 changes: 4 additions & 4 deletions crowdsec-docs/docs/getting_started/crowdsec_tour.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ sudo cscli decisions list
+-----+-----------+-------------+------------------------------------+--------+---------+----+--------+--------------------+----------+
| 802 | cscli | Ip:1.2.3.5 | manual 'ban' from | ban | | | 1 | 3h50m58.10039043s | 802 |
| | | | 'b76cc7b1bbdc489e93909d2043031de8' | | | | | | |
| 801 | crowdsec | Ip:1.2.3.4 | crowdsecurity/ssh-bf | ban | | | 6 | 3h59m45.100387557s | 801 |
| 801 | crowdsec | Ip:192.168.1.1 | crowdsecurity/ssh-bf | ban | | | 6 | 3h59m45.100387557s | 801 |
+-----+-----------+-------------+------------------------------------+--------+---------+----+--------+--------------------+----------+
```
</details>
Expand All @@ -130,11 +130,11 @@ There are different decisions `SOURCE`:
## Add/Remove decisions

```bash
cscli decisions add -i 1.2.3.4
cscli decisions delete -i 1.2.3.4
cscli decisions add -i 192.168.1.1
cscli decisions delete -i 192.168.1.1
```

Those commands will respectively add a manual decision for ip `1.2.3.4` (with default parameters such as duration and such), and remove all active decisions for ip `1.2.3.4`.
Those commands will respectively add a manual decision for ip `192.168.1.1` (with default parameters such as duration and such), and remove all active decisions for ip `192.168.1.1`.



Expand Down
14 changes: 7 additions & 7 deletions crowdsec-docs/docs/local_api/bouncers-api.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ As stated in the [swagger documentation](https://crowdsecurity.github.io/api_doc
To have some data to query for, let's add two decisions to our local API

```bash
▶ sudo cscli decisions add -i 1.2.3.4
▶ sudo cscli decisions add -i 192.168.1.1
INFO[0000] Decision successfully added
▶ sudo cscli decisions add -r 2.2.3.0/24
INFO[0000] Decision successfully added
Expand All @@ -82,7 +82,7 @@ INFO[0000] Decision successfully added
+------+--------+------------------+----------------------------------------------------+--------+---------+----+--------+--------------------+----------+
| 2337 | cscli | Range:2.2.3.0/24 | manual 'ban' from | ban | | | 1 | 3h59m18.079301785s | 1164 |
| | | | '939972095cf1459c8b22cc608eff85daEb4yoi2wiTD7Y3fA' | | | | | | |
| 2336 | cscli | Ip:1.2.3.4 | manual 'ban' from | ban | | | 1 | 3h59m11.079297437s | 1163 |
| 2336 | cscli | Ip:192.168.1.1 | manual 'ban' from | ban | | | 1 | 3h59m11.079297437s | 1163 |
| | | | '939972095cf1459c8b22cc608eff85daEb4yoi2wiTD7Y3fA' | | | | | | |
+------+--------+------------------+----------------------------------------------------+--------+---------+----+--------+--------------------+----------+

Expand All @@ -92,8 +92,8 @@ INFO[0000] Decision successfully added


```bash title="Query a single banned IP"
▶ curl -H "X-Api-Key: 837be58e22a28738066de1be8f53636b" http://localhost:8080/v1/decisions\?ip=1.2.3.4
[{"duration":"3h51m57.363171728s","id":2336,"origin":"cscli","scenario":"manual 'ban' from '939972095cf1459c8b22cc608eff85daEb4yoi2wiTD7Y3fA'","scope":"Ip","type":"ban","value":"1.2.3.4"}]
▶ curl -H "X-Api-Key: 837be58e22a28738066de1be8f53636b" http://localhost:8080/v1/decisions\?ip=192.168.1.1
[{"duration":"3h51m57.363171728s","id":2336,"origin":"cscli","scenario":"manual 'ban' from '939972095cf1459c8b22cc608eff85daEb4yoi2wiTD7Y3fA'","scope":"Ip","type":"ban","value":"192.168.1.1"}]
```

```bash title="Query a single IP"
Expand All @@ -112,7 +112,7 @@ _note: notice that the decision returned is the range that we banned earlier and

```bash title="Query a range in which one of the ban is contained"
▶ curl -H "X-Api-Key: 837be58e22a28738066de1be8f53636b" http://localhost:8080/v1/decisions\?range=1.2.3.0/24\&contains\=false
[{"duration":"3h48m7.676653651s","id":2336,"origin":"cscli","scenario":"manual 'ban' from '939972095cf1459c8b22cc608eff85daEb4yoi2wiTD7Y3fA'","scope":"Ip","type":"ban","value":"1.2.3.4"}]
[{"duration":"3h48m7.676653651s","id":2336,"origin":"cscli","scenario":"manual 'ban' from '939972095cf1459c8b22cc608eff85daEb4yoi2wiTD7Y3fA'","scope":"Ip","type":"ban","value":"192.168.1.1"}]
```
_note: notice the `contains` flag that is set to false_

Expand Down Expand Up @@ -141,7 +141,7 @@ INFO[0000] Decision successfully added
| | | | '939972095cf1459c8b22cc608eff85daEb4yoi2wiTD7Y3fA' | | | | | | |
| 2337 | cscli | Range:2.2.3.0/24 | manual 'ban' from | ban | | | 1 | 3h27m1.384972861s | 1164 |
| | | | '939972095cf1459c8b22cc608eff85daEb4yoi2wiTD7Y3fA' | | | | | | |
| 2336 | cscli | Ip:1.2.3.4 | manual 'ban' from | ban | | | 1 | 3h26m54.384971268s | 1163 |
| 2336 | cscli | Ip:192.168.1.1 | manual 'ban' from | ban | | | 1 | 3h26m54.384971268s | 1163 |
| | | | '939972095cf1459c8b22cc608eff85daEb4yoi2wiTD7Y3fA' | | | | | | |
+------+--------+------------------+----------------------------------------------------+-------------+---------+----+--------+--------------------+----------+
```
Expand Down Expand Up @@ -174,7 +174,7 @@ Given the our state looks like :
+------+--------+------------------+----------------------------------------------------+--------+---------+----+--------+--------------------+----------+
| 2337 | cscli | Range:2.2.3.0/24 | manual 'ban' from | ban | | | 1 | 2h55m26.05271136s | 1164 |
| | | | '939972095cf1459c8b22cc608eff85daEb4yoi2wiTD7Y3fA' | | | | | | |
| 2336 | cscli | Ip:1.2.3.4 | manual 'ban' from | ban | | | 1 | 2h55m19.052706441s | 1163 |
| 2336 | cscli | Ip:192.168.1.1 | manual 'ban' from | ban | | | 1 | 2h55m19.052706441s | 1163 |
| | | | '939972095cf1459c8b22cc608eff85daEb4yoi2wiTD7Y3fA' | | | | | | |
+------+--------+------------------+----------------------------------------------------+--------+---------+----+--------+--------------------+----------+

Expand Down
2 changes: 1 addition & 1 deletion crowdsec-docs/docs/local_api/profiles/format.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ It is a list of `models.Decision` objects. The following fields, when present, a
- `scope` : defines the scope of the resulting decision
- `duration` : defines for how long will the decision be valid. The format must comply with [golang's ParseDuration](https://pkg.go.dev/time#ParseDuration)
- `type` : defines the type of the remediation that will be applied by available bouncers, for example `ban`, `captcha`
- `value` : define a hardcoded value for the decision (ie. `1.2.3.4`)
- `value` : define a hardcoded value for the decision (ie. `192.168.1.1`)

### `duration_expr`

Expand Down
28 changes: 14 additions & 14 deletions crowdsec-docs/docs/log_processor/parsers/create.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ The creation of said functional testing will guide our process and will make it
We're going to create a parser for the imaginary service "myservice" that produce three types of logs via syslog :

```
Dec 8 06:28:43 mymachine myservice[2806]: bad password for user 'toto' from '1.2.3.4'
Dec 8 06:28:43 mymachine myservice[2806]: unknown user 'toto' from '1.2.3.4'
Dec 8 06:28:43 mymachine myservice[2806]: accepted connection for user 'toto' from '1.2.3.4'
Dec 8 06:28:43 mymachine myservice[2806]: bad password for user 'toto' from '192.168.1.1'
Dec 8 06:28:43 mymachine myservice[2806]: unknown user 'toto' from '192.168.1.1'
Dec 8 06:28:43 mymachine myservice[2806]: accepted connection for user 'toto' from '192.168.1.1'
```

As we are going to parse those logs to further detect bruteforce and user-enumeration attacks, we're simply going to "discard" the last type of logs.
Expand Down Expand Up @@ -144,8 +144,8 @@ results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Parsed["program"] ==
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Parsed["timestamp"] == "Dec 8 06:28:43"
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Parsed["is_my_service"] == "yes"
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Parsed["logsource"] == "syslog"
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Parsed["message"] == "bad password for user 'toto' from '1.2.3.4'"
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Parsed["some_data"] == "bad password for user 'toto' from '1.2.3.4'"
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Parsed["message"] == "bad password for user 'toto' from '192.168.1.1'"
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Parsed["some_data"] == "bad password for user 'toto' from '192.168.1.1'"
...
Expand All @@ -162,19 +162,19 @@ Further inspection can be seen with `cscli hubtest explain` :

```bash
▶ cscli hubtest explain myservice-logs
line: Dec 8 06:28:43 mymachine myservice[2806]: bad password for user 'toto' from '1.2.3.4'
line: Dec 8 06:28:43 mymachine myservice[2806]: bad password for user 'toto' from '192.168.1.1'
├ s00-raw
| └ 🟢 crowdsecurity/syslog-logs
└ s01-parse
└ 🟢 crowdsecurity/myservice-logs
line: Dec 8 06:28:43 mymachine myservice[2806]: unknown user 'toto' from '1.2.3.4'
line: Dec 8 06:28:43 mymachine myservice[2806]: unknown user 'toto' from '192.168.1.1'
├ s00-raw
| └ 🟢 crowdsecurity/syslog-logs
└ s01-parse
└ 🟢 crowdsecurity/myservice-logs
line: Dec 8 06:28:43 mymachine myservice[2806]: accepted connection for user 'toto' from '1.2.3.4'
line: Dec 8 06:28:43 mymachine myservice[2806]: accepted connection for user 'toto' from '192.168.1.1'
├ s00-raw
| └ 🟢 crowdsecurity/syslog-logs
└ s01-parse
Expand Down Expand Up @@ -250,12 +250,12 @@ results["s01-parse"]["crowdsecurity/myservice-logs"][0].Success == true
...
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Parsed["timestamp"] == "Dec 8 06:28:43"
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Parsed["program"] == "myservice"
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Parsed["source_ip"] == "1.2.3.4"
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Parsed["source_ip"] == "192.168.1.1"
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Parsed["user"] == "toto"
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Meta["log_subtype"] == "myservice_bad_password"
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Meta["log_type"] == "myservice_failed_auth"
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Meta["service"] == "myservice"
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Meta["source_ip"] == "1.2.3.4"
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Meta["source_ip"] == "192.168.1.1"
results["s01-parse"]["crowdsecurity/myservice-logs"][0].Evt.Meta["username"] == "toto"
...
results["s01-parse"]["crowdsecurity/myservice-logs"][1].Evt.Meta["log_subtype"] == "myservice_bad_user"
Expand All @@ -272,26 +272,26 @@ Again, further inspection with `cscli hubtest explain` will show us more about w

```bash
▶ cscli hubtest explain myservice-logs
line: Dec 8 06:28:43 mymachine myservice[2806]: bad password for user 'toto' from '1.2.3.4'
line: Dec 8 06:28:43 mymachine myservice[2806]: bad password for user 'toto' from '192.168.1.1'
├ s00-raw
| └ 🟢 crowdsecurity/syslog-logs
└ s01-parse
└ 🟢 crowdsecurity/myservice-logs
line: Dec 8 06:28:43 mymachine myservice[2806]: unknown user 'toto' from '1.2.3.4'
line: Dec 8 06:28:43 mymachine myservice[2806]: unknown user 'toto' from '192.168.1.1'
├ s00-raw
| └ 🟢 crowdsecurity/syslog-logs
└ s01-parse
└ 🟢 crowdsecurity/myservice-logs
line: Dec 8 06:28:43 mymachine myservice[2806]: accepted connection for user 'toto' from '1.2.3.4'
line: Dec 8 06:28:43 mymachine myservice[2806]: accepted connection for user 'toto' from '192.168.1.1'
├ s00-raw
| └ 🟢 crowdsecurity/syslog-logs
└ s01-parse
└ 🔴 crowdsecurity/myservice-logs
```

__note: we can see that our log line `accepted connection for user 'toto' from '1.2.3.4'` wasn't parsed by `crowdsecurity/myservice-logs` as we have no pattern for it__
__note: we can see that our log line `accepted connection for user 'toto' from '192.168.1.1'` wasn't parsed by `crowdsecurity/myservice-logs` as we have no pattern for it__


## Closing word
Expand Down
Loading

0 comments on commit 2fecde0

Please sign in to comment.