[improve][misc] Sync commits from apache into 3.1_ds (19 Feb) #367
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(cherry picked from commit df737f7)
…pache#23929) ### Motivation When the producer's maximum count is reached, the broker will log the following error message: ``` 2025-02-05T18:31:37,996+0800 [pulsar-io-18-16] ERROR org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:57684] Failed to create topic persistent://public/default/test2asgasgaw, producerId=1 java.util.concurrent.CompletionException: org.apache.pulsar.broker.service.BrokerServiceException$ProducerBusyException: Topic 'persistent://public/default/test2asgasgaw' reached max producers limit ``` These errors are related to the client side. The client can handle the error, so we don't need to print it in the broker log. ### Modifications - Avoid printing the error log for the ProducerBusyException (cherry picked from commit 144fe2e) (cherry picked from commit 92982bb)
…tadata not found (apache#20979) (cherry picked from commit 75d4d82) (cherry picked from commit c269e38)
…rors when using the Admin API (apache#23928) ### Motivation Currently, when there is a 404 or 409 error in the Admin API call, the broker prints the error logs. ``` ERROR org.apache.pulsar.broker.admin.v2.PersistentTopics - [xxx] Failed to get partitioned metadata topic persistent://xxx: Namespace not found ``` ``` ERROR org.apache.pulsar.broker.admin.v2.Namespaces - Failed to get policies for namespace xxx: Namespace does not exist ``` ``` ERROR org.apache.pulsar.broker.admin.v2.PersistentTopics - [xxx] Failed to create non-partitioned topic persistent:/xxx: This topic already exists ``` ``` [pulsar-web-44-1] ERROR org.apache.pulsar.broker.admin.AdminResource - [admin] Failed to create partitioned topic persistent://xxx java.util.concurrent.CompletionException: org.apache.pulsar.broker.web.RestException: This topic already exists ``` These errors are related to the client side. The client can handle the error, so we don't need to print it in the broker log. ### Modifications - Print a warning log for NotFound or Conflict errors in the Admin API. (cherry picked from commit 99dc74b) (cherry picked from commit 560d120)
…ue during subscription. (apache#23939) (cherry picked from commit 5e5d514) (cherry picked from commit 35c97fc)
…in ServerCnx (apache#23938) ### Motivation If the producer is created with some schema error, the broker will print many error logs like this: ``` ERROR org.apache.pulsar.broker.service.ServerCnx - Try add schema failed, remote address xxx java.util.concurrent.CompletionException: org.apache.pulsar.broker.service.schema.exceptions.IncompatibleSchemaException: Producers cannot connect or send message without a schema to topics with a schemawhen SchemaValidationEnforced is enabled ``` This error can be reported to the client and not need to print it in the broker. ### Modifications - Avoid printing log for IncompatibleSchemaException in ServerCnx (cherry picked from commit 3c0bbee) (cherry picked from commit 0de057a)
Signed-off-by: Zixuan Liu <[email protected]> (cherry picked from commit e6cfd2b) (cherry picked from commit a5512c3)
…ent-auth-athenz (apache#23947) (cherry picked from commit 54e9eb1) (cherry picked from commit bf2ff38)
… it exists - In branch-3.0, -Ddocker.squash=true is used and it relies on experimental key set to true in daemon.json. The docker image build fails with "connection reset by peer" if squash is requested and the daemon doesn't support it. - The daemon.json file was overridden with apache#23909 changes and the experimental setting got removed. (cherry picked from commit fab143b)
…perations (apache#23955) Co-authored-by: Lari Hotari <[email protected]> (cherry picked from commit 20b3b22) (cherry picked from commit 224320e)
… trimming ledger and non-durable cursor creation (apache#23951) Co-authored-by: Yunze Xu <[email protected]> (cherry picked from commit cc7b381) (cherry picked from commit 16bfb1a)
(cherry picked from commit 0776b37)
(cherry picked from commit 92527df)
… decompression and compression (apache#23960) (cherry picked from commit 7f595cd) (cherry picked from commit 77dc34d)
…pute backlog twice (apache#23957) (cherry picked from commit 6db275c) (cherry picked from commit 921b75e)
Signed-off-by: Zixuan Liu <[email protected]> (cherry picked from commit 3cc2206) (cherry picked from commit 2557d9d)
… cache (apache#23970) (cherry picked from commit 0a95976) (cherry picked from commit d717da1)
…io/solr (apache#23899) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Matteo Merli <[email protected]> (cherry picked from commit 014f0a8)
…ManyRequestsException (apache#23971) (cherry picked from commit 9d767b3)
…ize and batching is enabled (apache#23967) (cherry picked from commit 2620871)
Signed-off-by: Zixuan Liu <[email protected]> (cherry picked from commit d3ea0ee) (cherry picked from commit 10ae34c)
apache#23981) (cherry picked from commit eb1391a) (cherry picked from commit ebcbfff)
…axUnackedMessagesPerConsumer is 1 (apache#23796) (cherry picked from commit 5443c69) (cherry picked from commit b098772)
…ode (apache#23984) (cherry picked from commit 40d5af4) (cherry picked from commit 2cf8389)
Co-authored-by: fanjianye <[email protected]> (cherry picked from commit a7d9d8f)
…apache#23986) (cherry picked from commit eb7a4f3) (cherry picked from commit e729c90)
…ath expression (apache#23985) Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> (cherry picked from commit 5812084) (cherry picked from commit 8ee80ff)
Signed-off-by: Zixuan Liu <[email protected]> (cherry picked from commit 40b96de) (cherry picked from commit 9cdf9ad)
…ateOrCreate` (apache#23686) The method `MetadataCache#readModifyUpdateOrCreate` should handle the BadVersionException by retrying the modification process, as already noted in the Java documentation: "The modify function can potentially be called multiple times due to concurrent updates." Currently, `MetadataCache#readModifyUpdateOrCreate` does not catch the BadVersionException on the second attempt, allowing the exception to be passed to the caller. This issue can be easily reproduced by increasing concurrent futures in the test `MetadataCacheTest#readModifyUpdateBadVersionRetry`. The current retry implementation is incorrect and lacks a backoff mechanism, which could lead to too many requests to the metadata store. - Correct the retry process in `MetadataCache#readModifyUpdateOrCreate` to ensure BadVersionException is caught during each retry. - Implement a retry backoff mechanism in `MetadataCache#readModifyUpdateOrCreate` to manage the frequency of retries effectively. - Add new config `retryBackoff` to the MetadataCacheConfig to control the MetadataCache retry backoff. - Respective the `metadataStoreOperationTimeoutSeconds` for the MetadataCache retry (cherry picked from commit 0ae3f9d) (cherry picked from commit b56dfc7)
…ephemeral and ZK stat handling (apache#23988) (cherry picked from commit df51972) (cherry picked from commit 67ae209)
…ut using java.io.File (apache#23996) (cherry picked from commit 4bfdcd8) (cherry picked from commit f5d7309)
(cherry picked from commit ee834f8)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
This PR sync all commits from apache/branch-3.0 into 3.1_ds which are not present.
Modifications
Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:
If
yeswas chosen, please highlight the changesDocumentation
Check the box below or label this PR directly (if you have committer privilege).
Need to update docs?
doc-required(If you need help on updating docs, create a doc issue)
no-need-doc(Please explain why)
doc(If this PR contains doc changes)