Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql: translate SQL increments into kv IncrementRequests #22778

Closed
nvanbenschoten opened this issue Feb 16, 2018 · 4 comments
Closed

sql: translate SQL increments into kv IncrementRequests #22778

nvanbenschoten opened this issue Feb 16, 2018 · 4 comments
Labels
A-sql-execution Relating to SQL execution. A-sql-mutations Mutation statements: UPDATE/INSERT/UPSERT/DELETE. A-sql-optimizer SQL logical planning and optimizations. C-performance Perf of queries or internals. Solution not expected to change functional behavior.
Milestone

Comments

@nvanbenschoten
Copy link
Member

When performing an UPDATE we usually split it into two phases: a read phase and a write phase. During the read phase, we gather all rows necessary to perform the update using kv scans. During the write phase, we then send kv puts to perform the actual update.

These two distinct phases are usually necessary but could be optimized for SQL queries like:

UPDATE t SET v = v + 1

or

UPDATE t SET v = v +10 RETURNING v

Here, we could rely on our kv increment primitive (roachpb.IncrementRequests) to avoid a separate read phase. This is important because we often see these kinds of increments on hotly contested keys, as is the case in TPCC. In these situations, we should do whatever we can to avoid contention which may result in transaction restarts. Speeding up the increment would help alleviate this issue. More importantly, reducing this to a single kv operation would allow these UPDATEs to use 1PC transaction commits if they are performed in isolation, which would avoid it completely.

@nvanbenschoten nvanbenschoten added the C-performance Perf of queries or internals. Solution not expected to change functional behavior. label Feb 16, 2018
@nvanbenschoten nvanbenschoten added this to the Later milestone Feb 16, 2018
@petermattis
Copy link
Collaborator

This is moderately challenging because IncrementRequest relies on a particular format for the value which isn't usually true for SQL row data. We'd have to extend IncrementRequest to be told the format of the row and column it is incrementing.

@nvanbenschoten
Copy link
Member Author

Yeah, that's a good point. We'd need to teach IncrementRequest about SQL value encoding. This would probably also only be possible if the value was in its own column family.

@bdarnell
Copy link
Contributor

This would probably also only be possible if the value was in its own column family.

We could add a column identifier to the request to make it work when there are other columns in the family.

IIRC Increment currently has some issues with replays when used outside of an explicit transaction. That's fine for its current uses (allocating range/node/store ids), but we'd need to harden this for use in SQL.

Increments give us a lot of other opportunities for optimization too. Non-transactional increments to the same key can be batched. They're commutative, which we may be able to take advantage of by assigning timestamps as late as possible to minimize contention.

@knz knz added A-sql-optimizer SQL logical planning and optimizations. A-sql-execution Relating to SQL execution. A-sql-mutations Mutation statements: UPDATE/INSERT/UPSERT/DELETE. labels May 14, 2018
@nvanbenschoten
Copy link
Member Author

Closing in favor of the more general #29431.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-sql-execution Relating to SQL execution. A-sql-mutations Mutation statements: UPDATE/INSERT/UPSERT/DELETE. A-sql-optimizer SQL logical planning and optimizations. C-performance Perf of queries or internals. Solution not expected to change functional behavior.
Projects
None yet
Development

No branches or pull requests

4 participants