Optimization for PG2 ETS insertion

[gabiz]

ETS :bag insertion for elements with the same key grows linearly (on the number of elements with same key). On the other hand :duplicate_bag insertion is constant.
This change does not increase the size of the ETS and there is no performance degradation for lookups and deletions.
I was able to replicate insertion timeouts when there are lots of elements with same key using :bag but it does not happen using :duplicate_bag.

[Gazler]

Before:

After:

[chrismccord]

@gabiz You are my hero! This increased our arrival rate by 10x, as well as solved the mysterious contention causing our subscribe timeouts! Thanks so much!!! 😍🚀🚀

[bryansray]

Awesome commit @gabiz

[josevalim]

@gabiz this is so awesome!

[josevalim]

@chrismccord just to document, it would be really nice to move the writes back to Local (both inserts and deletes) and see how they affect performance now that we are using the proper ETS type! Once you do, remove the write_concurrency: true and :public flags. It may help performance (or not at all), that's why we need to measure. :)

[chrismccord]

That's the first thing on my todo list tomorrow

Sent from my iPhone

On Oct 25, 2015, at 8:20 PM, José Valim [email protected] wrote:

@chrismccord just to document, it would be really nice to move the writes back to Local (both inserts and deletes) and see how they affect performance now that we are using the proper ETS type! Once you do, remove the write_concurrency: true and :public flags. It may help performance (or not at all), that's why we need to measure. :)

—
Reply to this email directly or view it on GitHub.

[gabiz]

@josevalim the keyless match_delete is very expensive. Needs to traverse the whole table.
https://github.com/phoenixframework/phoenix/blob/master/lib/phoenix/pubsub/gc.ex#L59

Have you considered using a reverse ETS table to store the pid -> topic info. Using that the delete is several order of magnitude faster and avoids blocking other gen_server operations.

[gabiz]

@josevalim I just noticed that we used to have a reverse HashDict pid to topic back in time. We certainly need something like that if we plan to move the deletes back to the Local. Otherwise a bunch of disconnects are going to trigger the expensive deletes and block incoming subscriptions.

[indera]

@gabiz - what tools do you recommend to profile Elixir code... To fix this kind of issues there must be a better way than looking at :observer.start
Thank you.