Benchmark ingestion

[mariusandra]

Changes

• New version of Ingestion and e2e benchmarks #122 (to avoid a bad merge)
• Adds a simple "ingestion only" benchmark (measures await processEvent)
• Adds Kafka & Celery E2E ingestion benchmarks (from "posthog.capture" to event in postgres or clickhouse)
• The E2E benchmarks run one simple sync plugin and then add the result to the database. Benchmarks for plugins that e.g. wait for 20sec before doing anything will look wildly different and should be tested next
• Also adds some scripts to package.json to wipe the local kafka/clickhouse/zookeeper docker images

Results on my 16" 2.4GHz i9 MBP for the E2E benchmarks with 4 workers:

    [Celery & Postgres] Ingested 3000 events in 6.94s (432.18 events/sec, 2.31ms per event)
    [Celery & Postgres] Ingested 3000 events in 7.09s (423.41 events/sec, 2.36ms per event)
    [Celery & Postgres] Ingested 10000 events in 26.09s (383.24 events/sec, 2.61ms per event)
    [Celery & Postgres] Ingested 10000 events in 26.04s (384.08 events/sec, 2.6ms per event)

    [Kafka & ClickHouse] Ingested 3000 events in 25.62s (117.09 events/sec, 8.54ms per event)
    [Kafka & ClickHouse] Ingested 3000 events in 20.68s (145.06 events/sec, 6.89ms per event)
    [Kafka & ClickHouse] Ingested 3000 events in 19.85s (151.13 events/sec, 6.62ms per event)
    [Kafka & ClickHouse] Ingested 3000 events in 21.00s (142.83 events/sec, 7.00ms per event)
    [Kafka & ClickHouse] Ingested 10000 events in 143.94s (69.47 events/sec, 14.39ms per event)
    [Kafka & ClickHouse] Ingested 10000 events in 136.75s (73.12 events/sec, 13.68ms per event)

There's definitely room for improvement!

Checklist

• Updated Settings section in README.md, if settings are affected
• Jest tests

[Twixes]

I don't think there are any results shown in the ClickHouse benchmark action. As a nit, the Postgres one's results are super noisy on the other hand

[mariusandra]

It's there:

    [Kafka & ClickHouse] Ingested 3000 events in 17.89s (167.66 events/sec, 5.96ms per event)

but yeah, noisy