Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Do not start and end the stream prematurely (#135) #136

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

pavelhoral
Copy link

This PR fixes premature start of the glob stream which can lead to closing the resulting Transform stream before it is being properly processed by upstream components. This can lead to very strange race conditions and unfinished pipelines.

@phated
Copy link
Member

phated commented Jun 14, 2024

This is awesome. Thanks for debugging and adding a regression test! I think we can actually leverage the open function in streamx for this.

@pavelhoral
Copy link
Author

pavelhoral commented Jun 15, 2024

This is awesome. Thanks for debugging and adding a regression test! I think we can actually leverage the open function in streamx for this.

Not sure why, but moving the code to open function breaks the pipeline completely. Seems like open state is too soon for data being produced by the stream.

UPDATE: I am dumb... forgot about the callback.

@pavelhoral pavelhoral force-pushed the fix-premature-start branch from f022a1c to 81552cc Compare June 15, 2024 07:18
@pavelhoral
Copy link
Author

Pushed new changes (switch to open method).

@pavelhoral
Copy link
Author

Any update on this? Is there anything else I can do? Ty.

@pavelhoral
Copy link
Author

Any update on this? Is there anything else I can do? Ty.

Would love to have this merged and released. If there is anything I can do to help that, let me know.

@yocontra yocontra requested review from yocontra and phated December 3, 2024 02:22
Copy link
Member

@yocontra yocontra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable to me, will let @phated weigh in here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants