Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: resolve issue handling protobuf responses in rest streaming #604

Merged
merged 5 commits into from
Feb 13, 2024
Merged

fix: resolve issue handling protobuf responses in rest streaming #604

merged 5 commits into from
Feb 13, 2024

Conversation

parthea
Copy link
Collaborator

@parthea parthea commented Feb 12, 2024

Towards b/311671723

@product-auto-label product-auto-label bot added the size: m Pull request size is medium. label Feb 12, 2024
@parthea parthea marked this pull request as ready for review February 12, 2024 21:01
@parthea parthea requested review from a team as code owners February 12, 2024 21:01
vchudnov-g
vchudnov-g approved these changes Feb 13, 2024
View reviewed changes
Copy link
Contributor

@vchudnov-g vchudnov-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor comments, but LGTM.

google/api_core/rest_streaming.py Outdated
return self._response_message_cls.from_json(self._ready_objs.popleft())
if issubclass(self._response_message_cls, proto.Message):
return self._response_message_cls.from_json(self._ready_objs.popleft())
else:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: for safety and future-proofing, maybe this should be an elif and we check for google.protobuf.Message
(I realize we don't expect to ever have the second check fail if we get here....but "we don't expect" == "famous last words")

(not a blocker)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9f5b22f

tests/unit/test_rest_streaming.py Outdated
responses = [EchoResponse(content="hello world"), EchoResponse(content="yes")]
@pytest.mark.parametrize(
"random_split,resp_message_is_proto_plus,response_type",
[(False, True, EchoResponse), (False, False, httpbody_pb2.HttpBody)],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that you're not testing different combinations of resp_message_is_proto_plus and response_type, and that you're already switching on the first to construct the responses (which you have to because they take different parameters): I suggest you don't parametrize on response_type and instead, in the function, do

if resp_message_is_proto_plus:
  response_type = EchoResponse
  responses = [EchoResponse(content="hello world"), EchoResponse(content="yes")]
else:
  response_type = httpbody_pb2.HttpBody
  responses = [
            httpbody_pb2.HttpBody(content_type="hello world"),
            httpbody_pb2.HttpBody(content_type="yes"),
        ]

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5b7a4ce

tests/unit/test_rest_streaming.py Outdated
Song(title="another song", date_added=datetime.datetime(2021, 12, 17)),
]
@pytest.mark.parametrize(
"random_split,resp_message_is_proto_plus,response_type",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above: since you have the if below already, don't parametrize on response_type

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5b7a4ce

tests/unit/test_rest_streaming.py Outdated
@pytest.mark.parametrize("random_split", [True, False])
def test_next_stress(random_split):
@pytest.mark.parametrize(
"random_split,resp_message_is_proto_plus,response_type",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto on parametrization

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5b7a4ce

tests/unit/test_rest_streaming.py
),
Song(title='\\{"key": ["value",]}\\', composer=composer_with_relateds),
]
@pytest.mark.parametrize(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto on parametrizing

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5b7a4ce

@product-auto-label product-auto-label bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Feb 13, 2024
@parthea parthea merged commit bcebc92 into main Feb 13, 2024
27 checks passed
@parthea parthea deleted the fix-rest-streaming branch February 13, 2024 14:59
@release-please release-please bot mentioned this pull request Feb 13, 2024
Merged
This was referenced Feb 13, 2024
Closed
Merged
@parthea
Copy link
Collaborator Author

parthea commented Feb 13, 2024 •
edited
Loading

/cherry-pick v1

parthea added a commit that referenced this pull request Feb 13, 2024
@parthea @gcf-owl-bot
fix: resolve issue handling protobuf responses in rest streaming (#604)
65b760e 
* fix: resolve issue handling protobuf responses in rest streaming

* raise ValueError if response_message_cls is not a subclass of proto.Message or google.protobuf.message.Message

* remove response_type from pytest.mark.parametrize

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* add test for ValueError in response_iterator._grab()

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
@parthea parthea restored the fix-rest-streaming branch February 13, 2024 16:50
@parthea parthea deleted the fix-rest-streaming branch February 13, 2024 17:02
@parthea
Copy link
Collaborator Author

parthea commented Feb 13, 2024

/cherry-pick v1

gcp-cherry-pick-bot bot pushed a commit that referenced this pull request Feb 13, 2024
@parthea @gcf-owl-bot
fix: resolve issue handling protobuf responses in rest streaming (#604)
fde7316 
* fix: resolve issue handling protobuf responses in rest streaming

* raise ValueError if response_message_cls is not a subclass of proto.Message or google.protobuf.message.Message

* remove response_type from pytest.mark.parametrize

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* add test for ValueError in response_iterator._grab()

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
@gcp-cherry-pick-bot gcp-cherry-pick-bot bot mentioned this pull request Feb 13, 2024
Merged
@parthea parthea mentioned this pull request Feb 29, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vchudnov-g vchudnov-g vchudnov-g approved these changes

Assignees

@vchudnov-g vchudnov-g

@ohmayr ohmayr

Labels
size: l Pull request size is large.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@parthea @vchudnov-g @ohmayr