[feat][broker] Segmented transaction buffer snapshot segment and index system topic

[liangyepianzhou]

Master Issue: #16913

Motivation

Implement system topic client for snapshot segment topic and index topic to send segment snapshots or indexes.
The configuration transactionBufferSegmentedSnapshotEnabled is used in the Transaction Buffer to determine which AbortedTxnProcessor is adopted by this TB.

Modification

In the new implementation of the Transaction Buffer Snapshot System topic, because the system topic that needs to be processed has changed from the original one to three with different schemes, we have added generics to the TransactionBufferSnapshotBaseSystemTopicClient class and the SystemTopicTxnBufferSnapshotService class.
And Pulsar Service maintains a factory class TransactionBufferSnapshotServiceFactory used to obtain SystemTopicTxnBufferSnapshotService.
This way, we can obtain the required System topic client through pulsarService to read and send snapshots.

Verifying this change

• Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

• Added integration tests for end-to-end deployment with large payloads (10MB)
• Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

• Dependencies (does it add or upgrade a dependency): (yes / no)
• The public API: (yes / no)
• The schema: (yes / no / don't know)
• The default values of configurations: (yes / no)
• The wire protocol: (yes / no)
• The rest endpoints: (yes / no)
• The admin cli options: (yes / no)
• Anything that affects deployment: (yes / no / don't know)

Documentation

Check the box below or label this PR directly.

Need to update docs?

• doc-required
  (Your PR needs to update docs and you will update later)

• doc-not-needed
  (Please explain why)

• doc
  (Your PR contains doc changes)

• doc-complete
  (Docs have been already added)

[congbobo184]

You can retain write(T t), just add write(T t, String key)

[codelipenghui]

+1, why mark this one as resolved, I don't see any conclusion for this one.
And it's better to use MessageId write(String key, T value).

[liangyepianzhou]

Because MessageId write(T t) throws PulsarClientException; can not be implement in the TransactionBufferSnapshotBaseSystemTopicClient.
We can not get the key from the value of type T.

[codelipenghui]

+1, why mark this one as resolved, I don't see any conclusion for this one.
And it's better to use MessageId write(String key, T value).

[codelipenghui]

@poorbarcode @congbobo184 Please help review the PR again, thanks.

[codecov-commenter]

Codecov Report

Merging #16931 (39da371) into master (6c65ca0) will increase coverage by 10.99%.
The diff coverage is 60.42%.

@@              Coverage Diff              @@
##             master   #16931       +/-   ##
=============================================
+ Coverage     34.91%   45.91%   +10.99%     
- Complexity     5707    17672    +11965     
=============================================
  Files           607     1579      +972     
  Lines         53396   128705    +75309     
  Branches       5712    14161     +8449     
=============================================
+ Hits          18644    59093    +40449     
- Misses        32119    63498    +31379     
- Partials       2633     6114     +3481     

Flag	Coverage Δ
unittests	45.91% <60.42%> (+10.99%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...org/apache/pulsar/broker/ServiceConfiguration.java	52.06% <ø> (ø)
.../pulsar/broker/stats/BrokerOperabilityMetrics.java	98.21% <ø> (+5.56%)	⬆️
...ache/pulsar/broker/systopic/SystemTopicClient.java	0.00% <ø> (ø)
.../transaction/buffer/metadata/AbortTxnMetadata.java	28.57% <ø> (ø)
...ion/buffer/metadata/TransactionBufferSnapshot.java	42.85% <ø> (ø)
...er/metadata/v2/TransactionBufferSnapshotIndex.java	0.00% <0.00%> (ø)
.../metadata/v2/TransactionBufferSnapshotIndexes.java	0.00% <0.00%> (ø)
.../metadata/v2/TransactionBufferSnapshotSegment.java	0.00% <0.00%> (ø)
...oker/transaction/buffer/metadata/v2/TxnIDData.java	0.00% <0.00%> (ø)
...g/apache/pulsar/compaction/CompactedTopicImpl.java	69.28% <0.00%> (+58.57%)	⬆️
... and 1149 more

[congbobo184]

please modify the PR description for the new commit, then can be merged. @liangyepianzhou