Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streamlining PR pipeline #1378

Merged
merged 15 commits into from
Feb 14, 2025

Conversation

payalcha
Copy link
Collaborator

@payalcha payalcha commented Feb 13, 2025

There are multiple duplicate jobs present in the pipeline.
Removing the duplicate jobs from pipeline and adding E2E tests instead.

https://wiki.ith.intel.com/display/TPE/Existing+Tests+in+OpenFL - Can discuss.

Changes done -

  1. Removed double_ws_export.yml as Task Runner E2E covers it
  2. For Trivy fix added sudo apt update and upgrade and in .trivyignore added CVE-2024-3596. Suggested by Rajith.
  3. Removed ubuntu.yml as Pytest coverage is happening for every Pipeline and rest is covered in Task runner E2E.
  4. Moved all windows related tests in windows.yml and also added it in the pipeline.
  5. Straggler and Eden Compression added to task runner and their direct calls removed from PR pipeline.
  6. PR pipeline is updated with above changes.
  7. Made Resiliency and Task runner E2E runs mandatorily without any specific labels.
  8. Eden compression and workflow interface labels will work as is.

IMP - Resiliency testing is added in the PR pipeline it performs 30 rounds with torch/mnist and also performs multiple restarts while checking tensor.db for rounds. This test takes around ~12-15 mins. So PR pipeline will take little extra time overall.

Signed-off-by: Chaurasiya, Payal <[email protected]>
Signed-off-by: Chaurasiya, Payal <[email protected]>
Signed-off-by: Chaurasiya, Payal <[email protected]>
Signed-off-by: Chaurasiya, Payal <[email protected]>
@payalcha payalcha marked this pull request as draft February 13, 2025 11:16
@payalcha payalcha marked this pull request as ready for review February 13, 2025 11:18
Signed-off-by: Chaurasiya, Payal <[email protected]>
@payalcha payalcha changed the title WIP: Streamlining PR pipeline Streamlining PR pipeline Feb 14, 2025
Copy link
Collaborator

@teoparvanov teoparvanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, thank you @payalcha and @noopurintel ! Hopefully those improvements will help us catch potential regressions early, during the PR pipeline.

Signed-off-by: Chaurasiya, Payal <[email protected]>
@payalcha payalcha merged commit fb2094e into securefederatedai:develop Feb 14, 2025
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants