Switching to Qlobber.Dedup

[behrad]

This eliminates match/add pairs when adding topics to tries, and mosca is behaving more performant, Fixes #428, and I believe it also should fix #426

[mcollina]

Are you running this in production? Can you confirm it fixes #428 and maybe #426?

This is not passing on node v0.10 and v0.12, we should add a polyfill for those. I think Array.from is not supported.

[behrad]

Yes, It improves #428 , However there are still another optimizations available discussed on that topic 👍
I double checked #426 and it seems related to ascoltatori, May we introduce QlobberDedup in there?

Let me see what I can do for the Array.from for nodes < 4.0

[behrad]

Not to add a new npm dependency, I can use https://gist.github.com/brettz9/4212262, Where dya want me to put it @mcollina ?

[mcollina]

You can define it as a function somewhere, but don't polyfill array.

[behrad]

Why a fail? Can't see travis logs... :\

[behrad]

@davedoesdev What happens to QlobberDedup on node < 4 with no ES6 Set?

[davedoesdev]

david@david-Latitude-E6440:~$ nvm use 0.12
Now using node v0.12.11 (npm v2.14.9)
david@david-Latitude-E6440:~$ node --version
v0.12.11
david@david-Latitude-E6440:~$ node
> Set
[Function: Set]

[behrad]

Mosca has 10, 12, 4 & 5: https://github.com/mcollina/mosca/blob/master/.travis.yml
there should be a problem on 0.10? or even with 5?

[davedoesdev]

qlobber already polyfills Array.from: https://github.com/davedoesdev/qlobber/blob/master/test/dedup_spec.js#L26

[davedoesdev]

QlobberDedup won't work on Node 0.10.
Node 5 will be fine.

[behrad]

QlobberDedup won't work on Node 0.10

Will drop 0.10 support @mcollina ?

[davedoesdev]

Why does this PR need to use Array.from?

[behrad]

Seems async.eachdoesn't support ES6 Sets... you can find the code in commit log of this PR.
The problem seems to be ES6 which is not compatible with Mosca on 0.10

[davedoesdev]

https://github.com/caolan/async#collections-1

Collection methods can iterate over Arrays, Objects, Maps, Sets, and any object that implements the ES2015 iterator protocol.

[behrad]

hmm... I should check why I was unable to iterate over them with async.each

[behrad]

I should check why I was unable to iterate over them with async.each

async 1.5.x doesn't support them, updated async to latest

[davedoesdev]

Hi @behrad is performance acceptable now or are there still problems?

[behrad]

Startup really improved @davedoesdev, I believe it was an effective change, Thank you for your nice and quick feedbacks.

However we have stills performance issues in Mosca with lots of connecting/disconnecting clients (~10/seconds)

I'm trying to check timeout management, sub trie add/removes and the way Mosca is working at last, in isolated tests.

[behrad]

I'm using this in my production, Will it be merged in @mcollina ?

[davedoesdev]

@behrad FYI I just published qlobber 0.7.0, which now uses ES6 Maps internally for the trie. It seems to give roughly 25% speedup from the benchmarks.

[behrad]

@behrad FYI I just published qlobber 0.7.0, which now uses ES6 Maps internally for the trie. It seems to give roughly 25% speedup from the benchmarks.

Is there any upgrade notes I should care about? Replaced Sets with Maps?

[davedoesdev]

No - changes are all internal.

[behrad]

this is ready @mcollina

[mcollina]

Awesome, I will assemble a new major as soon as I have a bit of time.

[mcollina]

Already merged in v2, about to be released.