Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BTreeMap: comment why drain_filter's size_hint is somewhat pessimistic #77449

Merged
merged 2 commits into from
Oct 9, 2020

Conversation

ssomers
Copy link
Contributor

@ssomers ssomers commented Oct 2, 2020

The size_hint of the DrainFilter iterator doesn't adjust as you iterate. This hardly seems important to me, but there has been a comparable PR #64383 in the past. I guess a scenario is that you first iterate half the map manually and keep most of the key/value pairs in the map, and then tell the predicate to drain most of the key/value pairs and .collect the iterator over the remaining half of the map.

I am totally ambivalent whether this is better or not.

r? @Mark-Simulacrum

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 2, 2020
@jyn514 jyn514 added the T-libs Relevant to the library team, which will review and decide on the PR/issue. label Oct 2, 2020
@Mark-Simulacrum
Copy link
Member

I am a bit confused -- we're updating the length live, so why do we need a new remaining field?

My guess is that there's something subtle going on here that would be good to document on the remaining field (as well as clarify here).

If we do need the additional field I think we shouldn't do this, it's not worth the extra field and more complicated logic.

@Mark-Simulacrum Mark-Simulacrum added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 3, 2020
@ssomers
Copy link
Contributor Author

ssomers commented Oct 3, 2020

The length includes the part of the map that the predicate saw and decided not to drain. It overestimates how many elements are yet to be returned by the iterator.

I agree this is rather futile. If the predicate keeps very many elements, then the size hint is always overestimated. If the predicate drains very many elements, then you're better of with into_iter and collecting a new map. #64383 didn't require any new data to improve the estimate, but here we need some number in addition to the length - the number drained, the number kept, or the number remaining.

@Mark-Simulacrum
Copy link
Member

I think it's probably fine for us to include that extra in the size hint -- it's a hint, and I'd prefer to avoid the complication introduced by the logic here (I am surprised tracking the information we need requires so many changes).

I am leaning towards saying that we shouldn't do this.

@ssomers
Copy link
Contributor Author

ssomers commented Oct 3, 2020

(I am surprised tracking the information we need requires so many changes).

Well it doesn't require changes in navigate.rs, but it allows getting rid of some Err values and saving a total of 5 lines there. But without really reducing complexity.

I'll write a comment in the size_hint function instead.

@ssomers ssomers force-pushed the btree_drain_filter_size_hint branch from ab4cb00 to 3b051d0 Compare October 3, 2020 19:18
@ssomers ssomers changed the title BTreeMap: make drain_filter's size_hint more accurate @ssomers BTreeMap: comment why drain_filter's size_hint is somewhat pessimistic Oct 3, 2020
@ssomers
Copy link
Contributor Author

ssomers commented Oct 5, 2020

@rustbot modify labels: +S-waiting-on-review -S-waiting-on-author

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 5, 2020
@Mark-Simulacrum Mark-Simulacrum changed the title @ssomers BTreeMap: comment why drain_filter's size_hint is somewhat pessimistic BTreeMap: comment why drain_filter's size_hint is somewhat pessimistic Oct 8, 2020
@Mark-Simulacrum
Copy link
Member

@bors r+ rollup

@bors
Copy link
Contributor

bors commented Oct 8, 2020

📌 Commit 3b051d0 has been approved by Mark-Simulacrum

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 8, 2020
Dylan-DPC-zz pushed a commit to Dylan-DPC-zz/rust that referenced this pull request Oct 8, 2020
…t, r=Mark-Simulacrum

BTreeMap: comment why drain_filter's size_hint is somewhat pessimistic

The `size_hint` of the `DrainFilter` iterator doesn't adjust as you iterate. This hardly seems important to me, but there has been a comparable PR rust-lang#64383 in the past. I guess a scenario is that you first iterate half the map manually and keep most of the key/value pairs in the map, and then tell the predicate to drain most of the key/value pairs and `.collect` the iterator over the remaining half of the map.

I am totally ambivalent whether this is better or not.

r? @Mark-Simulacrum
bors added a commit to rust-lang-ci/rust that referenced this pull request Oct 8, 2020
…as-schievink

Rollup of 8 pull requests

Successful merges:

 - rust-lang#76750 (Don't discourage implementing `core::fmt::Write`)
 - rust-lang#77449 (BTreeMap: comment why drain_filter's size_hint is somewhat pessimistic)
 - rust-lang#77660 ((docs): make mutex error comment consistent with codebase)
 - rust-lang#77663 (Add compile fail test for issue 27675)
 - rust-lang#77673 (Remove unnecessary lamda on emitter map.)
 - rust-lang#77701 (Make `max_log_info` easily greppable (for figuring out why debug logging is disabled))
 - rust-lang#77702 (Remove not needed lambda.)
 - rust-lang#77710 (Update submodule llvm to get LVI bugfix)

Failed merges:

r? `@ghost`
@bors bors merged commit 738a41b into rust-lang:master Oct 9, 2020
@rustbot rustbot added this to the 1.49.0 milestone Oct 9, 2020
@ssomers ssomers deleted the btree_drain_filter_size_hint branch October 9, 2020 09:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants