Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize Application Server worker pools #4607

Merged
merged 3 commits into from
Sep 10, 2021

Conversation

adriansmares
Copy link
Contributor

Summary

Closes #4606

Changes

  • Add pkg/workerpool, which contains the generic worker pool that spawns workers on demand
  • Refactor the application packages frontend to use the pkg/workerpool pool instead of the manually spawned tasks
    • A small deviation that you will see here is that there is a worker pool added between the io.Subscription and the actual worker pools of each package. This is intentional and allows us to decide to which package we need to send a particular uplink to in parallel.
  • Refactor the webhooks frontend to use the pkg/workerpool pool instead of the manually spawned goroutines

Testing

Unit testing and local testing.

Regressions

This changes the piping used by the Application Server worker pools. The unit tests cover this.

Checklist

  • Scope: The referenced issue is addressed, there are no unrelated changes.
  • Compatibility: The changes are backwards compatible with existing API, storage, configuration and CLI, according to the compatibility commitments in README.md for the chosen target branch.
  • Documentation: Relevant documentation is added or updated.
  • Changelog: Significant features, behavior changes, deprecations and fixes are added to CHANGELOG.md.
  • Commits: Commit messages follow guidelines in CONTRIBUTING.md, there are no fixup commits left.

@adriansmares adriansmares added this to the v3.15.0 milestone Sep 9, 2021
@adriansmares adriansmares self-assigned this Sep 9, 2021
@adriansmares adriansmares force-pushed the feature/global-worker-pool branch from 0774bb6 to 98e91bd Compare September 9, 2021 15:31
@github-actions github-actions bot added c/application server This is related to the Application Server compat/config This could affect Configuration compatibility labels Sep 9, 2021
@adriansmares adriansmares force-pushed the feature/global-worker-pool branch 5 times, most recently from 3d43128 to 56c51c9 Compare September 9, 2021 16:14
@adriansmares adriansmares marked this pull request as ready for review September 9, 2021 16:34
@adriansmares adriansmares force-pushed the feature/global-worker-pool branch from 56c51c9 to 86240b5 Compare September 9, 2021 17:03
Copy link
Member

@KrishnaIyer KrishnaIyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic. Looks good to me.

@adriansmares adriansmares merged commit 04759c0 into v3.15 Sep 10, 2021
@adriansmares adriansmares deleted the feature/global-worker-pool branch September 10, 2021 10:38
@adriansmares adriansmares mentioned this pull request Sep 25, 2021
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c/application server This is related to the Application Server compat/config This could affect Configuration compatibility
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Optimize Application Server worker pools
2 participants