Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed segment upload error return #7957

Merged
merged 2 commits into from
Dec 24, 2021

Conversation

mcvsubbu
Copy link
Contributor

Description

If a segment is uploaded that is not in tar.gz format, we just get back an internal error message. It is useful
to return the actual message.
Unfortunately, we return a 5xx error code when this should be a 4xx. The only way we can distinguish the invalid
file format case in pinot is by comparing the error message string. The apache commons lib raises the same exception
(java.io.IOException) whether the input is bad or we had trouble accessing the file system. Will let that be for now

Upgrade Notes

Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion)

  • Yes (Please label as backward-incompat, and complete the section below on Release Notes)

Does this PR fix a zero-downtime upgrade introduced earlier?

  • Yes (Please label this as backward-incompat, and complete the section below on Release Notes)

Does this PR otherwise need attention when creating release notes? Things to consider:

  • New configuration options
  • Deprecation of configurations
  • Signature changes to public methods/interfaces
  • New plugins added or old plugins removed
  • Yes (Please label this PR as release-notes and complete the section on Release Notes)

Release Notes

Documentation

@codecov-commenter
Copy link

codecov-commenter commented Dec 23, 2021

Codecov Report

Merging #7957 (ff7025c) into master (b2b2a14) will increase coverage by 0.26%.
The diff coverage is 0.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #7957      +/-   ##
============================================
+ Coverage     71.11%   71.38%   +0.26%     
- Complexity     4115     4194      +79     
============================================
  Files          1593     1594       +1     
  Lines         82397    82562     +165     
  Branches      12276    12321      +45     
============================================
+ Hits          58600    58936     +336     
+ Misses        19842    19645     -197     
- Partials       3955     3981      +26     
Flag Coverage Δ
integration1 29.07% <0.00%> (+<0.01%) ⬆️
integration2 27.70% <0.00%> (+0.24%) ⬆️
unittests1 68.24% <ø> (+0.19%) ⬆️
unittests2 14.29% <0.00%> (-0.06%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ces/PinotSegmentUploadDownloadRestletResource.java 58.26% <0.00%> (ø)
...a/manager/realtime/RealtimeSegmentDataManager.java 50.00% <0.00%> (-25.00%) ⬇️
...in/stream/kafka20/KafkaPartitionLevelConsumer.java 66.66% <0.00%> (-22.81%) ⬇️
...lix/core/realtime/PinotRealtimeSegmentManager.java 78.53% <0.00%> (-6.29%) ⬇️
...ment/index/readers/DefaultIndexReaderProvider.java 70.00% <0.00%> (-3.69%) ⬇️
...riter/impl/FixedByteChunkSVForwardIndexWriter.java 96.55% <0.00%> (-3.45%) ⬇️
...gregation/function/StUnionAggregationFunction.java 73.52% <0.00%> (-2.95%) ⬇️
...controller/helix/core/minion/PinotTaskManager.java 65.51% <0.00%> (-1.92%) ⬇️
...nMaxValueBasedSelectionOrderByCombineOperator.java 70.45% <0.00%> (-1.52%) ⬇️
...t/index/loader/invertedindex/JsonIndexHandler.java 72.58% <0.00%> (-0.76%) ⬇️
... and 50 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b2b2a14...ff7025c. Read the comment docs.

@mcvsubbu mcvsubbu merged commit 3aa4f5d into apache:master Dec 24, 2021
@mcvsubbu mcvsubbu deleted the segment-upload-error-handling branch December 24, 2021 21:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants