Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change update namespace to upsert custom search attributes #4080

Merged
merged 3 commits into from
Mar 25, 2023

Conversation

rodrigozhou
Copy link
Contributor

What changed?

  • Changed operator handler and admin handler to get namespace metadata using DescribeNamespace API: it fetches from persistence instead of cache.
  • Changed UpdateNamespace to upsert the custom search attributes field to alias map instead of simply overwriting the entire map. If the value is an empty string (ie., the alias is an empty string), then remove the alias.

Why?

  • Get the up-to-date namespace metadata, and therefore, the most up-to-date existing search attributes. Using the cached value might lead to incorrect validation of new search attributes.
  • Doing upsert in UpdateNamespace ensures that concurrent calls to add search attributes are handled correctly without racing condition.

How did you test it?
Started Server, and sent multiple calls to create search attribute using tctl and Temporal CLI. Verified that all search attributes were created correctly.

Potential risks
No risks.

Is hotfix candidate?
No.

@rodrigozhou rodrigozhou force-pushed the upsert-csa-update-ns branch 2 times, most recently from 4892c3e to ed674da Compare March 23, 2023 00:53
@yiminc yiminc requested a review from pdoerner March 23, 2023 15:52
@rodrigozhou rodrigozhou force-pushed the upsert-csa-update-ns branch from ed674da to 7926cf4 Compare March 23, 2023 17:30
yycptt
yycptt approved these changes Mar 24, 2023
} else if _, ok := current[key]; !ok {
result[key] = value
} else {
return nil, serviceerror.NewInvalidArgument(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm ask user to retry invalid argument looks weird.
Currently if there're concurrent namespace update, persistence layer will return Unavailable, which will be retried and customer most likely won't see the error.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this Unavailable error retry is in the grpc level, which won't work because the args will be the same from the operator handler.
The retry from the user will make operator handler to generate a new map to be upsert based on newer version of the namespace data.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If user must change agrs, then InvalidArgument is the right error. Message should be different though. Don't use word "retry" there and explicitly say that specific argument must be changed.

@@ -280,18 +277,22 @@ func (h *OperatorHandlerImpl) addSearchAttributesSQL(
if nsName == "" {
return errNamespaceNotSet
}
ns, err := h.namespaceRegistry.GetNamespace(namespace.Name(nsName))
resp, err := client.DescribeNamespace(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this to avoid cache? Why not to call DescribeNamesapce from namespace handler directly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's to avoid cache. Namespace handler is not available in this context.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be added, but ok, let's leave it this way.

} else if _, ok := current[key]; !ok {
result[key] = value
} else {
return nil, serviceerror.NewInvalidArgument(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If user must change agrs, then InvalidArgument is the right error. Message should be different though. Don't use word "retry" there and explicitly say that specific argument must be changed.

@rodrigozhou rodrigozhou force-pushed the upsert-csa-update-ns branch from 7926cf4 to db45583 Compare March 24, 2023 23:34
@rodrigozhou rodrigozhou force-pushed the upsert-csa-update-ns branch from db45583 to 8e47773 Compare March 24, 2023 23:59
@rodrigozhou
Copy link
Contributor Author

Updated the operator/admin handler to return servicererror.Unavailable when the error is related to conflicting field names. This should avoid showing InvalidArgument error to the user, trigger retry from operator/admin handler side.

@rodrigozhou rodrigozhou force-pushed the upsert-csa-update-ns branch from 8e47773 to 84c4045 Compare March 25, 2023 00:25
@yycptt yycptt enabled auto-merge (squash) March 25, 2023 00:55
},
})
if err.Error() == errCustomSearchAttributeFieldAlreadyAllocated.Error() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is ugly. But I don't see a better way :-(

@@ -280,18 +277,22 @@ func (h *OperatorHandlerImpl) addSearchAttributesSQL(
if nsName == "" {
return errNamespaceNotSet
}
ns, err := h.namespaceRegistry.GetNamespace(namespace.Name(nsName))
resp, err := client.DescribeNamespace(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be added, but ok, let's leave it this way.

@yycptt yycptt merged commit f1ef4db into temporalio:master Mar 25, 2023
@rodrigozhou rodrigozhou deleted the upsert-csa-update-ns branch March 27, 2023 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants