Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish message on specific partition #166

Merged
merged 8 commits into from
Sep 21, 2022
Merged

Publish message on specific partition #166

merged 8 commits into from
Sep 21, 2022

Conversation

ChrisKujawa
Copy link
Member

@ChrisKujawa ChrisKujawa commented Aug 26, 2022

Adds a new feature to publish a message with a printable correlation key to an specific (given) partition. This is useful if we want to experiment with message correlation, for example disconnect certain partitions etc.

// The message publish partition distribution is based
// on the following hashcode and modulo by the partition count
//
// Since this calculation is used not only on publish, but also for
// opening the message and instance subscriptions we must support for
// backwards compatibility, otherwise message will not be correlated
//
// This allows us to make certain assumptions, and shortcuts.
//
// static int getSubscriptionHashCode(final DirectBuffer correlationKey) {
//   is equal to java.lang.String#hashCode
//   int hashCode = 0;
//
//   for (int i = 0, length = correlationKey.capacity(); i < length; i++) {
//      	hashCode = 31 * hashCode + correlationKey.getByte(i);
//   }
//   return hashCode;
// }
//
// and
//   public static int getSubscriptionPartitionId(
//      final DirectBuffer correlationKey, final int partitionCount) {
//    final int hashCode = getSubscriptionHashCode(correlationKey);
//    // partition ids range from START_PARTITION_ID .. START_PARTITION_ID + partitionCount
//    return Math.abs(hashCode % partitionCount) + START_PARTITION_ID;
//  }
// Based on the hash code function, we now that if the string has a length of one byte
// no multiplication will happen.
//
// Since we need a printable (for variables) character we start with ASCII code 48, until 126.
// Formulas:
//
// expectedPartition = (hashcode mod partitionCount) + partitionStartId
// hashcode = i ( 48 >= i >= 126) == ASCII CODE
//

Usage

$ ./zbchaos publish -v --partitionId 1
Connecting to zell-chaos
Successfully created port forwarding tunnel
Send message 'msg', with correaltion key '0' (ASCII: 48) 
Message was sent and returned key 2251799813685346, which corresponds to partition: 1

@ChrisKujawa ChrisKujawa requested a review from npepinpe August 26, 2022 19:55
Previously control-characters have been used for simplicity, but this makes it hard to create variables in PI in order to correlate the message. For that we need printable characters. this commit adjustes the calculation to find correct and suitable correlation keys.
Copy link
Member

@npepinpe npepinpe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Just one comment about verbose logging

partitionIdFromKey := internal.ExtractPartitionIdFromKey(messageResponse.Key)

if Verbose {
fmt.Printf("Message was sent and returned key %d, which corresponds to partition: %d\n", messageResponse.Key, partitionIdFromKey)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Very nice. I'm wondering if it shouldn't always be printed regardless, as if a test were to fail due to a partition mismatch, it would be pretty important info

@ChrisKujawa ChrisKujawa merged commit af7cba9 into main Sep 21, 2022
@ChrisKujawa ChrisKujawa deleted the zell-publish-msg branch September 21, 2022 06:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants