Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cli and provide tests are largely flaky #1935

Closed
flub opened this issue Jan 8, 2024 · 4 comments
Closed

cli and provide tests are largely flaky #1935

flub opened this issue Jan 8, 2024 · 4 comments
Assignees
@rklaehn
Labels
flaky-test

Comments

@flub
Copy link
Contributor

flub commented Jan 8, 2024

There are a whole bunch of flaky integration tests in the following files:

  • iroh/tests/cli.rs
  • iroh/tests/provide.rs

I suspect all of these are related to filesystem latencies. We probably need a reliable way to run sync(1)/fsync(2) on the filesystem to fix all of these.
@flub flub added the flaky-test label Jan 8, 2024
@github-project-automation github-project-automation bot moved this to 📋 Backlog in iroh Jan 8, 2024
flub added a commit that referenced this issue Jan 8, 2024
@flub
tests: Mark a bunch of flaky tests
d51f547 
I think all of these are filesystem related.

Issue filed in #1935
@flub flub mentioned this issue Jan 8, 2024
Merged
2 tasks
flub added a commit that referenced this issue Jan 8, 2024
@flub
tests: Mark a bunch of flaky tests (#1936)
6aa77ae 
## Description

I think all of these are filesystem related.

Issue filed in #1935

## Notes & open questions

## Change checklist

- [x] Self-review.
- [x] Tests if relevant.
@flub flub mentioned this issue Jan 11, 2024
Merged
1 task
flub added a commit that referenced this issue Jan 11, 2024
@flub
chore(ci): run non-flaky tests again (#1948)
41f056b 
## Description

We had a bug in the github actions job that resulted in not running
any tests.  Oops.

## Notes & open questions

Running 248 tests again!

Newly marked flaky test already covered by #1935 

## Change checklist

- [x] Self-review.
@flub flub mentioned this issue Feb 8, 2024
Merged
1 task
@flub
Copy link
Contributor Author

flub commented Feb 8, 2024

https://github.com/n0-computer/iroh/actions/runs/7829002247/job/21360042881?pr=2009

github-merge-queue bot pushed a commit that referenced this issue Feb 8, 2024
@flub
refactor(iroh-net): Remove manual struct logging (#2009)
315032a 
## Description

This was essentially re-implementing the Rust Debug trait.  Now that
DerpUrl has a nice Debug log there really is no need to manually
implement this, instead we rely on Debug.


## Notes & open questions

This is splitting off some peripheral stuff from
#1984

New flaky test marked: #2010
and #1935

## Change checklist

- [x] Self-review.
@flub
Copy link
Contributor Author

flub commented Feb 15, 2024

Maybe we should just sprinkle shelling out to the sync(1) command around to see if this helps quickly. As ugly as it may be, we'd get these tests back. If there's an equivalent for this on Windows.

@flub flub mentioned this issue Feb 15, 2024
Merged
3 tasks
fubuloubu pushed a commit to ApeWorX/iroh that referenced this issue Feb 21, 2024
@flub @fubuloubu
tests: Mark a bunch of flaky tests (n0-computer#1936)
1ab9dab 
## Description

I think all of these are filesystem related.

Issue filed in n0-computer#1935

## Notes & open questions

## Change checklist

- [x] Self-review.
- [x] Tests if relevant.
fubuloubu pushed a commit to ApeWorX/iroh that referenced this issue Feb 21, 2024
@flub @fubuloubu
chore(ci): run non-flaky tests again (n0-computer#1948)
36a6157 
## Description

We had a bug in the github actions job that resulted in not running
any tests.  Oops.

## Notes & open questions

Running 248 tests again!

Newly marked flaky test already covered by n0-computer#1935 

## Change checklist

- [x] Self-review.
fubuloubu pushed a commit to ApeWorX/iroh that referenced this issue Feb 21, 2024
@flub @fubuloubu
refactor(iroh-net): Remove manual struct logging (n0-computer#2009)
fbc0c4b 
## Description

This was essentially re-implementing the Rust Debug trait.  Now that
DerpUrl has a nice Debug log there really is no need to manually
implement this, instead we rely on Debug.


## Notes & open questions

This is splitting off some peripheral stuff from
n0-computer#1984

New flaky test marked: n0-computer#2010
and n0-computer#1935

## Change checklist

- [x] Self-review.
@matheus23
Copy link
Member

https://github.com/n0-computer/iroh/actions/runs/7829002247/job/21360042881?pr=2009

The next time we should download the logs & upload them here.
They've expired since, and they'd be really valuable actually.

@divagant-martian
Copy link
Contributor

most of these tests are no longer flaky, and grouping them all together gives us a poor view of our understanding of what's wrong with each. That being said, I'll close this and open new issue for the two remaining cli flaky tests

@github-project-automation github-project-automation bot moved this from 📋 Backlog to ✅ Done in iroh Aug 6, 2024
matheus23 pushed a commit that referenced this issue Nov 14, 2024
@flub
chore(ci): run non-flaky tests again (#1948)
c707011 
## Description

We had a bug in the github actions job that resulted in not running
any tests.  Oops.

## Notes & open questions

Running 248 tests again!

Newly marked flaky test already covered by #1935 

## Change checklist

- [x] Self-review.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rklaehn rklaehn

Labels
flaky-test
Projects
iroh
Archived in project
Milestone
No milestone
Development

No branches or pull requests
4 participants
@flub @rklaehn @matheus23 @divagant-martian