Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the issue gRPC notify thread blocked #965

Merged
merged 6 commits into from
Aug 19, 2020

Conversation

xin-hedera
Copy link
Contributor

@xin-hedera xin-hedera commented Aug 17, 2020

Detailed description:

Add backpressure buffer with ERROR strategy to notifying and shared polling topic listeners. When overflow happens, the controller will send a gRPC error after all buffered messages and disconnect the client.

Which issue(s) this PR fixes:
Fixes #945

Special notes for your reviewer:

Checklist

  • Documentation added
  • Tests updated

Xin Li added 2 commits August 15, 2020 13:24
@xin-hedera xin-hedera added bug Type: Something isn't working P1 grpc Area: GRPC API labels Aug 17, 2020
@xin-hedera xin-hedera added this to the Mirror 0.18.0 milestone Aug 17, 2020
@xin-hedera xin-hedera requested a review from a team August 17, 2020 14:05
@xin-hedera xin-hedera self-assigned this Aug 17, 2020
@codecov
Copy link

codecov bot commented Aug 17, 2020

Codecov Report

Merging #965 into master will increase coverage by 0.04%.
The diff coverage is 100.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #965      +/-   ##
============================================
+ Coverage     86.81%   86.86%   +0.04%     
  Complexity      210      210              
============================================
  Files           168      168              
  Lines          4233     4239       +6     
  Branches        466      466              
============================================
+ Hits           3675     3682       +7     
+ Misses          411      409       -2     
- Partials        147      148       +1     
Impacted Files Coverage Δ Complexity Δ
...ra/mirror/grpc/controller/ConsensusController.java 86.84% <100.00%> (+0.35%) 13.00 <0.00> (ø)
...edera/mirror/grpc/listener/ListenerProperties.java 92.85% <100.00%> (+1.19%) 5.00 <1.00> (+1.00)
...a/mirror/grpc/listener/NotifyingTopicListener.java 90.00% <100.00%> (+6.66%) 14.00 <1.00> (ø)
...rror/grpc/listener/SharedPollingTopicListener.java 92.98% <100.00%> (-3.45%) 10.00 <1.00> (-1.00)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 49fd703...449ce2b. Read the comment docs.

@xin-hedera xin-hedera marked this pull request as ready for review August 17, 2020 19:58
Nana-EC
Nana-EC previously approved these changes Aug 18, 2020
Copy link
Contributor

@Nana-EC Nana-EC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xin-hedera xin-hedera merged commit e301647 into master Aug 19, 2020
@xin-hedera xin-hedera deleted the grpc-notify-thread-blocked branch August 19, 2020 14:58
Daniel-K-Ivanov added a commit to LimeChain/hedera-mirror-node that referenced this pull request Aug 24, 2020
* Fix the issue gRPC notify thread blocked (hiero-ledger#965)

Add backpressure buffer with ERROR strategy to notifying and shared polling topic listeners. When overflow happens, the controller will send a gRPC error after all buffered messages and disconnect the client.

Signed-off-by: Xin Li <[email protected]>

* Performance test HighTPS HCS publish improvements  (hiero-ledger#970)

* Performance updates to high tps module

- Adds additional stats
- Allow publish of stats interval to be configured
- Does some reorganizing of code
- Sets JMeter and java flags to allow for improved publish performance and prevent memory exhaustion

Signed-off-by: Nana-EC <[email protected]>

* Bump spring-boot-starter-parent from 2.2.7.RELEASE to 2.3.3.RELEASE (hiero-ledger#961)

Bumps [spring-boot-starter-parent](https://github.com/spring-projects/spring-boot) from 2.2.7.RELEASE to 2.3.3.RELEASE.
- [Release notes](https://github.com/spring-projects/spring-boot/releases)
- [Commits](spring-projects/spring-boot@v2.2.7.RELEASE...v2.3.3.RELEASE)

Signed-off-by: Nana-EC <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: Xin Li <[email protected]>
Co-authored-by: Nana-EC <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Nana-EC pushed a commit that referenced this pull request Aug 31, 2020
Add backpressure buffer with ERROR strategy to notifying and shared polling topic listeners. When overflow happens, the controller will send a gRPC error after all buffered messages and disconnect the client.

Signed-off-by: Xin Li <[email protected]>
Nana-EC pushed a commit that referenced this pull request Aug 31, 2020
Add backpressure buffer with ERROR strategy to notifying and shared polling topic listeners. When overflow happens, the controller will send a gRPC error after all buffered messages and disconnect the client.

Signed-off-by: Xin Li <[email protected]>
Signed-off-by: Nana-EC <[email protected]>
xin-hedera pushed a commit that referenced this pull request Sep 11, 2020
xin-hedera pushed a commit that referenced this pull request Sep 11, 2020
xin-hedera added a commit that referenced this pull request Sep 12, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Type: Something isn't working grpc Area: GRPC API P1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

gRPC API notify thread blocks and cancels subscribers
3 participants