Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Engine interface should change to take batches of points #294

Closed
pauldix opened this issue Mar 4, 2014 · 4 comments
Closed

Engine interface should change to take batches of points #294

pauldix opened this issue Mar 4, 2014 · 4 comments

Comments

@pauldix
Copy link
Member

pauldix commented Mar 4, 2014

Performance on some queries would be better if the engine took a batch of points instead of yielding each individual point. Refactor so that it takes a YieldSeries instead of YieldPoint

@pauldix pauldix added this to the 0.5.0 milestone Mar 4, 2014
elcct added a commit to elcct/influxdb that referenced this issue Mar 7, 2014
@elcct
Copy link
Contributor

elcct commented Mar 7, 2014

I started some work on refactoring and now it yields in series rather than individual points. This introduced new methods YieldSeries and AggregateSeries.

I only optimized Count and Timestamp aggregator for now, other aggegators fall back to use AggregatePoint instead.

I noticed that this gives about 2x faster execution of select count(value) from some_series; query and other queries seem to have improved.

Series are being collected in datastore/leveldb_shard.go and there is POINT_BATCH_SIZE constant that defines how large batch should be. This probably should be in config file

Would be great if you could take a look

@pauldix
Copy link
Member Author

pauldix commented Mar 9, 2014

Would love to take a look, can you push to a branch and submit a PR?

@jvshahid
Copy link
Contributor

jvshahid commented Apr 3, 2014

Moved this story to 0.5.5, we should also get rid of the YieldPoint method

@jvshahid jvshahid self-assigned this Apr 4, 2014
@jvshahid jvshahid modified the milestones: 0.6.0, 0.5.5, 0.5.6 Apr 4, 2014
@jvshahid jvshahid modified the milestones: 0.5.7, 0.5.6 Apr 9, 2014
@pauldix pauldix modified the milestones: 0.5.9, 0.5.7, 0.5.10, 0.5.11 Apr 17, 2014
@jvshahid jvshahid modified the milestones: 0.5.12, 0.5.11 Apr 25, 2014
@jvshahid jvshahid modified the milestones: 0.5.13, 0.5.12, 0.6.0, 0.6.1 Apr 29, 2014
@jvshahid jvshahid modified the milestones: 0.6.2, 0.6.1, Next release May 7, 2014
@jvshahid
Copy link
Contributor

I think this is already done.

@jvshahid jvshahid modified the milestones: 0.6.4, Next release May 15, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants