[extension/dbstorage] Add DB Transactions to dbstorage.Batch() method #37805
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fiery-Fenix
requested review from
dmitryax,
atoulme and
a team
as code owners
github-actions
bot
requested review from
djaglowski,
swiatekm and
VihasMakwana
Fiery-Fenix
force-pushed
the
feat/dbstorage-transaction-support
branch
from
c57c90e to
3129f39
Compare
Fiery-Fenix
force-pushed
the
feat/dbstorage-transaction-support
branch
from
3129f39 to
c6641c6
Compare
atoulme
reviewed
Feb 10, 2025
| if err != nil { | ||
| return err | ||
| } | ||
| //nolint:errcheck |
There was a problem hiding this comment.
do we not want to do something on error here?
There was a problem hiding this comment.
Agreed, I'd at least log the error.
There was a problem hiding this comment.
Added logging here
Unfortunately there was no logger instance in this object, so have to inject it, I believe it might be useful in this object for future improvements
atoulme
approved these changes
Feb 10, 2025
| return err | ||
| } | ||
| //nolint:errcheck | ||
| defer tx.Rollback() |
There was a problem hiding this comment.
This might be confusing for other developers. We commit the tx on return and a rollback call is also made due to this defer.
There are two scenarios:
- when any error is encoutered
- when no error is encountered
First scenario is straightforward. We return the error and a rollback is called.
For the second scenario, we commit the transaction and then rollback is called due to the defer. The rollback call in this scenario is a no-op because tx is already committed.
Can we add a comment here to mention this nuance?
There was a problem hiding this comment.
Added few comments to clarify commit/rollback flow, as well as for tx.Rollback() error handling
Added few comments to clarify commit/rollback flow
Fiery-Fenix
force-pushed
the
feat/dbstorage-transaction-support
branch
from
57c2d9d to
d65307c
Compare
swiatekm
approved these changes
Feb 11, 2025
atoulme
added
extension/storage/dbstorage
ready to merge
MovieStoreGuy
approved these changes
Feb 12, 2025
khushijain21
pushed a commit
to khushijain21/opentelemetry-collector-contrib
that referenced
this pull request
Feb 14, 2025
…open-telemetry#37805) <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description According to [Storage API documentation](https://github.com/open-telemetry/opentelemetry-collector/blob/main/extension/xextension/storage/README.md) `Batch` is expected to "execute several operations in a single transaction" Moreover, Persistent Queue in `exporterhelper` is actively using `storage.Batch()` with read+write/write+delete operations in single call, which is really needs to be in a single transaction to avoid accidental data inconsistency in Persistent Queue in case of unexpected service shutdown. For example [here](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/exporterqueue/persistent_queue.go#L140) or [here](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/exporterqueue/persistent_queue.go#L257) or [here](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/exporterqueue/persistent_queue.go#L463) As currently supported SQlite and PostgreSQL driver natively support Transaction - this PR implements single transaction inside `storage.Batch()` call Also, I've added bunch of unit tests that were missing to ensure that show Storage API works as expected in `dbstorage` extension <!--Describe what testing was performed and which tests were added.--> #### Testing Respective Unit Tests were added --------- Co-authored-by: Sean Marciniak <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
According to Storage API documentation
Batchis expected to "execute several operations in a single transaction"Moreover, Persistent Queue in
exporterhelperis actively usingstorage.Batch()with read+write/write+delete operations in single call, which is really needs to be in a single transaction to avoid accidental data inconsistency in Persistent Queue in case of unexpected service shutdown. For example here or here or hereAs currently supported SQlite and PostgreSQL driver natively support Transaction - this PR implements single transaction inside
storage.Batch()callAlso, I've added bunch of unit tests that were missing to ensure that show Storage API works as expected in
dbstorageextensionTesting
Respective Unit Tests were added