Skip to content
This repository has been archived by the owner on Feb 25, 2023. It is now read-only.

Commit

Permalink
Prometheus: log public network availability
Browse files Browse the repository at this point in the history
https://github.com/shiftdevices/bitbox-base-internal/issues/370

Prometheus should log the availability of public internet without
leaking privacy information.

This pull requests queries an external host with a lot of general
traffic, Cloudflare, over Tor:

```
curl --socks5-hostname localhost:9050 1.1.1.1
```

If Tor is not active, it's impossible to ping an external host without
revealing the own ip address, but the solution can just "mingle in the
crowd", e.g. by `ping`ing Cloudflare, which should not be suspicious.

```
ping -c 1 1.1.1.1
```

These queries are run regularly from `prometheus-base.py`, so that the
result is collected by Prometheus and stored in its time-series database.
This helps with analyzing/debugging incidents, as the public network
availability can also be queried after the fact.

This commit:
* provides the new Prometheus metric 'base_internet_connectivity' that
  is 0 when OK, or an error code when NOT OK
  • Loading branch information
Stadicus committed Dec 9, 2019
1 parent d8ec60b commit f668792
Showing 1 changed file with 23 additions and 0 deletions.
23 changes: 23 additions & 0 deletions armbian/base/scripts/prometheus-base.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
BASE_SYSTEMD_LIGHTNINGD = Gauge("base_systemd_lightningd", "Systemd unit status for c-lightning")
BASE_SYSTEMD_PROMETHEUS = Gauge("base_systemd_prometheus", "Systemd unit status for Prometheus")
BASE_SYSTEMD_GRAFANA = Gauge("base_systemd_grafana", "Systemd unit status for Grafana")
BASE_INTERNET_CONNECTIVITY = Gauge("base_internet_connectivity", "Connectivity to public internet")

r = redis.Redis(
host='127.0.0.1',
Expand Down Expand Up @@ -80,6 +81,27 @@ def getSystemdStatus(unit):
print(unit, e.returncode, e.output)
return e.returncode

def getInternetConnectivity():
torEnabled = int(r.get('tor:base:enabled').decode("utf-8"))

try:
if torEnabled == 1:
print("Tor ok")
subprocess.check_output(["curl", "--socks5-hostname", "localhost:9050", "1.1.1.1"], shell=False, timeout=5, stderr=subprocess.STDOUT)
else:
print("Tor not ok")
subprocess.check_output(["ping", "-c", "1", "1.1.1.1"], shell=False, timeout=5, stderr=subprocess.STDOUT)

return 0

except subprocess.TimeoutExpired as e:
print("getInternetConnectivity(): subprocess.TimeoutExpired; torEnabled", torEnabled)
return 1

except subprocess.CalledProcessError as e:
print("getInternetConnectivity(): subprocess.CalledProcessError (", e.returncode, "); torEnabled", torEnabled, e.output)
return e.returncode

def main():
# Start up the server to expose the metrics.
start_http_server(8400)
Expand All @@ -91,6 +113,7 @@ def main():
BASE_SYSTEMD_LIGHTNINGD.set(int(getSystemdStatus("lightningd")))
BASE_SYSTEMD_PROMETHEUS.set(int(getSystemdStatus("prometheus")))
BASE_SYSTEMD_GRAFANA.set(int(getSystemdStatus("grafana-server")))
BASE_INTERNET_CONNECTIVITY.set(int(getInternetConnectivity()))

try:
BASE_CPU_TEMP.set(readFile("/sys/class/thermal/thermal_zone0/temp"))
Expand Down

0 comments on commit f668792

Please sign in to comment.