Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor(issuegenerator): Create issues based on JUnit test results #672

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

ArthurSens
Copy link
Member

@ArthurSens ArthurSens commented Jan 28, 2025

One of our goals for the collector is to be more proactive when detecting flaky tests on main, as mentioned in open-telemetry/opentelemetry-collector-contrib#36761. Since tests must pass before getting merged to main, it's safe to assume that the test is flaky if it fails on the main branch.

Thanks to open-telemetry/opentelemetry-collector#11963, we can use JUnit test results to populate this tool.

@ArthurSens
Copy link
Member Author

I'm opening it as a draft first so people can chime in on the approach :)

This is still not working 100% because I'm failing to correlate module names with the appropriate filepaths that are present in the codeowners file.

A module name has the full name that is present in the go.mod file, while the pattern used in the codeowners files uses only the relative path of files. 😕

issuegenerator/main.go Outdated Show resolved Hide resolved
@mx-psi mx-psi self-requested a review January 29, 2025 12:13
Signed-off-by: Arthur Silva Sens <[email protected]>
Signed-off-by: Arthur Silva Sens <[email protected]>
Copy link

codecov bot commented Jan 29, 2025

Codecov Report

Attention: Patch coverage is 13.18681% with 79 lines in your changes missing coverage. Please review.

Project coverage is 52.43%. Comparing base (14f89b4) to head (f393dd2).

Files with missing lines Patch % Lines
issuegenerator/main.go 13.18% 77 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #672      +/-   ##
==========================================
- Coverage   52.49%   52.43%   -0.07%     
==========================================
  Files          57       57              
  Lines        3444     3477      +33     
==========================================
+ Hits         1808     1823      +15     
- Misses       1466     1480      +14     
- Partials      170      174       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Arthur Silva Sens <[email protected]>
Signed-off-by: Arthur Silva Sens <[email protected]>
@ArthurSens ArthurSens marked this pull request as ready for review January 29, 2025 17:57
@ArthurSens ArthurSens requested review from a team as code owners January 29, 2025 17:57
@ArthurSens ArthurSens changed the title Refactor(issuegenerator): Ping codeowners on test failures Refactor(issuegenerator): Create issues based on JUnit test results Jan 29, 2025
Copy link
Member

@mx-psi mx-psi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks reasonable to me, have you done any tests with real contrib data? I would be willing to try directly on contrib, if we can restrict it to a just a few modules to begin with (e.g. we could begin with just extensions)

@ArthurSens
Copy link
Member Author

ArthurSens commented Jan 31, 2025

I haven't tried with contrib, but with my fork of collector-core. I've just added one commit updating the codeowners (This test was done before removing the codeowners bit from this PR).

The issue was opened without problems: ArthurSens/opentelemetry-collector#1 . This comment was after removing the codeowners bit.

if we can restrict it to a just a few modules to begin with (e.g. we could begin with just extensions)

I can see two ways forward here:

  • Figure out a way to only generate JUnit files for the modules we want
  • Add another flag here, e.g. a regex matcher and only generate reports for modules that match this regex

@mx-psi
Copy link
Member

mx-psi commented Jan 31, 2025

Figure out a way to only generate JUnit files for the modules we want

I think we can do that using some conditionals on Github Actions, no need to modify this tool

issuegenerator/main.go Outdated Show resolved Hide resolved
@mx-psi mx-psi requested a review from mowies January 31, 2025 12:08
issuegenerator/go.mod Outdated Show resolved Hide resolved
Copy link
Member

@mowies mowies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • please either use methods that always have the value receiver or the object receiver :) right now, it's a mix of both
  • i like it more when types/consts/vars etc. are at the top of the file and then the funcs follow that

rg.logger.Info("No existing Issues found, creating a new one.")
createdIssue := rg.createIssue(report)
rg.logger.Info("New GitHub Issue created", zap.String("html_url", *createdIssue.HTMLURL))
rg.reportIterator++
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can be moved out of the if block

@@ -115,9 +179,9 @@ type reportGenerator struct {
func (rg *reportGenerator) getRequiredEnv() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think the doc string for this method needs an update

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants