Supporting Consistent Data Push and Data Rollback on Pinot

[snleee]

@jackjlli and I have come up with the design document for supporting the consistent push on Pinot. The main idea is to reuse the "segment replacement protocol" added as part of the merge-rollup service to switch old segments with new segments.

The existing protocol has some limitations since it was originally designed in the context of merge/rollup. Here are some main changes:

• Add "reverted" state for segment lineage entry & revert API
• Add "select" API that returns the list of target segments to be replaced given the time range.
• Improve the segment name generator for the unique segment name
• Improve the code to cover some failure cases

https://docs.google.com/document/d/1PUy4wSUPFyEWEW3a88Mipdug3cPj4EpV__lx-BVUTYk/edit#

[dang-stripe]

@snleee are there plans to still support this for APPEND tables?