Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#1296] Swinging Door Trending (SDT) Filter Processor #1306

Merged
merged 6 commits into from
Feb 20, 2023

Conversation

SteveYurongSu
Copy link
Member

Purpose

The Swinging Door Trending (SDT) algorithm is a linear trend compression algorithm. In essence, it replaces a series of continuous (timestamp, value) points with a straight line determined by the start and end points.

The Swinging Door Trending (SDT) Filter Processor can extract and forward the characteristic events of the original stream. In general, this filter can also be used to reduce the frequency of original data in a lossy way.

This PR is related to:

Remarks

PR introduces (a) breaking change(s): <yes/no>
no

PR introduces (a) deprecation(s): <yes/no>
no

Tests

I setup a workflow like the following:

image

Then I checked the consumed count / produced count of the SDT filter:
image
image

The SDT filter worked as expected.

@SteveYurongSu
Copy link
Member Author

PTAL, thanks! 🙏 @dominikriemer @tenthe @bossenti

Copy link
Contributor

@bossenti bossenti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you so much for this high-class contribution @SteveYurongSu 🤩
I'm really amazed how easily you get into StreamPipes and provide valuable contributions.
I skimmed through the PR and it really looks great.
In the upcoming days, I will take some time to understand the implementation in more detail.
Until then I only have one question: Where did to get the icon from? Did you create it by your own?

@bossenti bossenti added enhancement New feature or request java Pull requests that update Java code pipeline elements Relates to pipeline elements labels Feb 17, 2023
@bossenti bossenti linked an issue Feb 17, 2023 that may be closed by this pull request
@bossenti
Copy link
Contributor

PS: You can ignore the failed pr labeling run this simply does not work for fork-based PRs (in case you didn't already realized) 🙂

@SteveYurongSu
Copy link
Member Author

SteveYurongSu commented Feb 17, 2023

thank you so much for this high-class contribution @SteveYurongSu 🤩
I'm really amazed how easily you get into StreamPipes and provide valuable contributions.
I skimmed through the PR and it really looks great.
In the upcoming days, I will take some time to understand the implementation in more detail.
Until then I only have one question: Where did to get the icon from? Did you create it by your own?

@bossenti Thanks for your reply! 😀

A friend of mine drew the swinging door for me on an iPad. I think it only took her about 5 minutes, but the picture looks really good to me haha :)

@SteveYurongSu
Copy link
Member Author

PS: You can ignore the failed pr labeling run this simply does not work for fork-based PRs (in case you didn't already realized) 🙂

@bossenti Thanks for your warm reminder! Maybe I can also take a look at it and help fix the failure on forks.

@bossenti
Copy link
Contributor

A friend of mine drew the swinging door for me on an iPad. I think it only took her about 5 minutes, but the picture looks really good to me haha

Ah that's great!
I'm just asking because we avoid using icons from public internet sources to simply exclude the risk of violating any copyright or quotation rules
So your approach is just ideal 🙂

@bossenti
Copy link
Contributor

Thanks for your warm reminder! Maybe I can also take a look at it and help fix the failure on forks.

That would be great 🙂

@SteveYurongSu
Copy link
Member Author

Thanks for your warm reminder! Maybe I can also take a look at it and help fix the failure on forks.

That would be great 🙂

@bossenti I have submitted a PR to fix the issue. See #1309 :).

@bossenti
Copy link
Contributor

@SteveYurongSu The PR looks really good
From my point of view, it's ready to merge
It would be great if you could address the two outstanding, cosmetic issues but I would not insist on them

@github-actions github-actions bot added backend Everything that is related to the StreamPipes backend documentation Everything related to documentation labels Feb 18, 2023
@SteveYurongSu
Copy link
Member Author

@bossenti Thanks for your careful review, which is great!
I have summarized the description of parameters and added Javadoc to the code. Please take a look again :)

@dominikriemer
Copy link
Member

Thank you @SteveYurongSu for this awesome contribution!

@bossenti bossenti merged commit 62037ab into apache:dev Feb 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Everything that is related to the StreamPipes backend documentation Everything related to documentation enhancement New feature or request java Pull requests that update Java code pipeline elements Relates to pipeline elements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Swinging Door Trending (SDT) Filter Processor
3 participants