Optimize lookups with ets pids table #31
Conversation
Add pids table for reverse lookup against values table for efficient matching of get_by_pid. Optimize ets select for get_by_topic by or'ing downed node comparisons.
| @@ -51,7 +54,8 @@ defmodule Phoenix.Tracker.State do | |||
| reset_delta(%State{ | |||
| replica: replica, | |||
| mode: :normal, | |||
| values: :ets.new(:values, [:duplicate_bag]), | |||
| values: :ets.new(:values, [:ordered_set]), | |||
| pids: :ets.new(:pids, [:duplicate_bag]), | |||
There was a problem hiding this comment.
@chrismccord I know this is an old commit, but do you remember why did you use duplicate bag here instead of just bag which still allows duplicate keys?
There was a problem hiding this comment.
duplicate bag will not check for existing keys on insert. It gave us a 10x speedup when we profiled phoenix pubsub back in 2015 for the 2M connection benchmark
There was a problem hiding this comment.
Phoenix channels*. More info on the original PR:
phoenixframework/phoenix#1311
Insertion time is constant for dup bag, where the insertion grows linearly with the bag size for an ordered set. I just realized you are asking :bag vs :duplicate_bag, not vs :ordered_set. I'm not sure of the differences, if any, for bag vs dup bag inserts in the above context.
Add pids table for reverse lookup against
values table for efficient matching of
get_by_pid.
Optimize ets select for get_by_topic
by or'ing downed node comparisons.