Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql: add estimated and actual statistics to telemetry logging #85169

Merged
merged 1 commit into from
Jul 29, 2022

Conversation

rytaft
Copy link
Collaborator

@rytaft rytaft commented Jul 27, 2022

This commit adds several new fields to the SampledQuery structure used for
telemetry logging.

Closes #71666

Release note (sql change): The structured payloads used for telemetry
logs now include the following new fields:
MaxFullScanRowsEstimate: Maximum number of rows scanned by a full scan,
as estimated by the optimizer.
TotalScanRowsEstimate: Total number of rows read by all scans in the query,
as estimated by the optimizer.
OutputRowsEstimate: The number of rows output by the query, as estimated
by the optimizer.
StatsAvailable: Whether table statistics were available to the optimizer
when planning the query.
NanosSinceStatsCollected: The maximum number of nanoseconds that have
passed since stats were collected on any table scanned by this query.
BytesRead: The number of bytes read from disk.
RowsRead: The number of rows read from disk.
RowsWritten: The number of rows written.

@rytaft rytaft requested review from yuzefovich, vy-ton, michae2, kevin-v-ngo and a team July 27, 2022 22:22
@rytaft rytaft requested a review from a team as a code owner July 27, 2022 22:22
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 11 of 11 files at r1, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @kevin-v-ngo, @michae2, and @vy-ton)

Copy link

@kevin-v-ngo kevin-v-ngo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! Excited to see the insights we're going to pull from this.

@@ -2465,6 +2465,13 @@ contains common SQL event/execution details.
| `TransactionID` | Transaction ID of the query. | no |
| `DatabaseID` | Database ID of the query. | no |
| `StatementFingerprintID` | Statement fingerprint ID of the query. | no |
| `MaxFullScanRowsEstimate` | Maximum number of rows scanned by a full scan, as estimated by the optimizer. | no |

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the maximum full scan over both full index and full table scans correct?

Copy link
Collaborator Author

@rytaft rytaft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTRs!

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @kevin-v-ngo, @michae2, and @vy-ton)


docs/generated/eventlog.md line 2468 at r1 (raw file):

Previously, kevin-v-ngo wrote…

This is the maximum full scan over both full index and full table scans correct?

Yep, that's correct

This commit adds several new fields to the SampledQuery structure used for
telemetry logging.

Closes cockroachdb#71666

Release note (sql change): The structured payloads used for telemetry
logs now include the following new fields:
MaxFullScanRowsEstimate: Maximum number of rows scanned by a full scan,
as estimated by the optimizer.
TotalScanRowsEstimate: Total number of rows read by all scans in the query,
as estimated by the optimizer.
OutputRowsEstimate: The number of rows output by the query, as estimated
by the optimizer.
StatsAvailable: Whether table statistics were available to the optimizer
when planning the query.
NanosSinceStatsCollected: The maximum number of nanoseconds that have
passed since stats were collected on any table scanned by this query.
BytesRead: The number of bytes read from disk.
RowsRead: The number of rows read from disk.
RowsWritten: The number of rows written.
@rytaft rytaft force-pushed the stats-telemetry branch from 2df5c53 to e1f2459 Compare July 28, 2022 13:03
@rytaft
Copy link
Collaborator Author

rytaft commented Jul 28, 2022

bors r+

@craig
Copy link
Contributor

craig bot commented Jul 28, 2022

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Jul 29, 2022

Build succeeded:

@craig craig bot merged commit 1129fbc into cockroachdb:master Jul 29, 2022
@blathers-crl
Copy link

blathers-crl bot commented Jul 29, 2022

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


error creating merge commit from e1f2459 to blathers/backport-release-22.1-85169: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 22.1.x failed. See errors above.


🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sql: Telemetry ideas for query planning/execution
4 participants