-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
release-24.3: opt: add new optimizer session settings #140255
release-24.3: opt: add new optimizer session settings #140255
Conversation
This is a mechanical change that will set up the ability to use CostFlags in a future commit. Release note: None
Thanks for opening a backport. Please check the backport criteria before merging:
If your backport adds new functionality, please ensure that the following additional criteria are satisfied:
Also, please add a brief release justification to the body of your PR to justify this |
Your pull request contains more than 1000 changes. It is strongly encouraged to split big PRs into smaller chunks. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
4bf23a1
to
6033c3d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 53 of 53 files at r1, 35 of 35 files at r2, 15 of 15 files at r3, 20 of 20 files at r4, all commit messages.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @mgartner)
blathers backport 24.3.4-rc |
Added a hint to avoid full scans (see release note below for details). To support this change, added a field to memo.Cost with a new type memo.CostFlags, which contains a number of boolean flags and supports "multi-dimensional costing". This allows the optimizer to compare plans based on the flags set in addition to the single-dimensional float64 cost. For example, plans with the new FullScanPenalty cost flag enabled will always be more expensive than plans without any cost flags, even if the base float64 cost is lower. The new CostFlags type also includes a flag for HugeCostPenalty, which must be set for plans with "hugeCost". This ensures that existing hints that use hugeCost still work if some other cost flags are set, since HugeCostPenalty takes precedence over other cost flags. This new CostFlags field is needed to support hints that do not cause an error if the optimizer cannot find a plan complying with the hint. This is needed because the previous approach of simply using "hugeCost" to avoid certain plans meant that if such plans were unavoidable, we could not effectively compare plans with cost greater than hugeCost due to loss of floating point precision. Informs cockroachdb#79683 Release note (sql change): Added support for a new index hint, AVOID_FULL_SCAN, which will prevent the optimizer from planning a full scan for the specified table if any other plan is possible. The hint can be used in the same way as other existing index hints. For example, SELECT * FROM table_name@{AVOID_FULL_SCAN};. This hint is similar to NO_FULL_SCAN, but will not error if a full scan cannot be avoided. Note that normally a full scan of a partial index would not be considered a "full scan" for the purposes of the NO_FULL_SCAN and AVOID_FULL_SCAN hints, but if the user has explicitly forced the partial index via FORCE_INDEX=index_name, we do consider it a full scan.
Release note (sql change): The `optimizer_prefer_bounded_cardinality` session setting has been added which instructs the optimizer to prefer query plans where every expression has a guaranteed upper-bound on the number of rows it will process. This may help the optimizer produce better query plans in some cases. This setting is disabled by default.
6033c3d
to
9e1288a
Compare
Informs cockroachdb#64570 Informs cockroachdb#130201 Release note (sql change): The `optimizer_min_row_count` session setting has been added which sets a lower bound on row count estimates for relational expressions during query planning. A value of zero, which is the default, indicates no lower bound. Note that if this is set to a value greater than zero, a row count of zero can still be estimated for expressions with a cardinality of zero, e.g., for a contradictory filter. Setting this to a value higher than 0, such as 1, may yield better query plans in some cases, such as when statistics are frequently stale and inaccurate.
9e1288a
to
279d22f
Compare
Backport:
optimizer_prefer_bounded_cardinality
session setting" (opt: addoptimizer_prefer_bounded_cardinality
session setting #139985)optimizer_min_row_count
session setting" (opt: addoptimizer_min_row_count
session setting #140065)Please see individual PRs for details.
/cc @cockroachdb/release
Release justification: Optimizer changes gated behind session settings.