Jmj/jan 2025 problemlist

[jmjaffe37]

Updated msft problemlists for the jan 2025 psu, as mentioned in: #4714

I worked with @dhanalla to compile this problemlist :)

[karianna]

I assume this is distinct from the general temurin ones? Was wondering if there was more overlap

[jmjaffe37]

@karianna you are correct, here is a link to the directory where the general adoptium problemlists reside: https://github.com/adoptium/aqa-tests/tree/master/openjdk/excludes

[jmjaffe37]

@gdams, are you able to take a look at this please? Looks like we need 2 reviewers to approve

[gdams]

Temurin also publishes a Windows aarch64 release for JDK21 so I'd be surprised if there's additional Msft-specific tests that need skipping for this platform? @ShelleyLambert is there a default problemlist that the Microsoft one overrides?

[jmjaffe37]

@gdams, if you view the whole jdk21 problemlist file then you will see that there are other tests here that we have problemlisted in the past for either windows-aarch64 or windows-all

[smlambert]

Will be worth comparing against failures in the Temurin runs and moving common excludes to the main problemlist file instead of MSFT specific one.

[karianna]

Will be worth comparing against failures in the Temurin runs and moving common excludes to the main problemlist file instead of MSFT specific one.

Agreed (I should have stated that in my initial review).

[jmjaffe37]

@karianna, @smlambert, are there any progress updates on the work on creating a main problemlist? Is there anything I can do to help?

I have a work item that I can only close after this gets merged, so I would also be open to the idea of merging this for now and then I can go through and remove common excludes in a new MSFT PR when the General Adoptium PR is made :)

[smlambert]

are there any progress updates on the work on creating a main problemlist? Is there anything I can do to help?

Sorry, I thought you were actively looking at it.

For any of the entries that are already reported upstream (JBS issues), can be put in common problemlists (found in this folder https://github.com/adoptium/aqa-tests/tree/master/openjdk/excludes).

For the aqa-tests reported issues, it will be good to check if those testcases are running and failing in our weekly test jobs, If they are, those can also go into the common problemlists.

For anything that you know to be MSFT specific and where we see it running and passing against Temurin, those excludes can stay in the vendor problemlist.

I see references to some of the issues that @andrew-m-leonard reported because they were failing against Temurin and thought to be common problems, such as https://bugs.openjdk.org/browse/JDK-8348862, so I expect we already have them excluded in the common problemlists (see https://github.com/adoptium/aqa-tests/blob/master/openjdk/excludes/ProblemList_openjdk21.txt#L478 for example).

If you are running tests via TKG and our playlists, then you will already be using both the common and vendor problemlists passed to jtreg on the commandline (example: https://github.com/adoptium/aqa-tests/blob/master/openjdk/playlist.xml#L354-L357)

Noting that for JDK11, there is no Temurin windows-aarch64 build so it won't be possible to check the test jobs for that version.

[karianna]

I'm happy to merge this one if we then open a new one and tidy the whole lot up so to speak. Probably the right time to think about only hosting that Problem list at Microsoft (unless we need it for marketplace compliance reasons).

[smlambert]

Sounds good, we will merge and then clean up later.

[karianna]

@smlambert I've created a new issue but can't assign it to jmjaffe37 - do we need to add him?

[smlambert]

@smlambert I've created a new issue but can't assign it to jmjaffe37 - do we need to add him?

Unfortunately, the GitHub repo settings under Eclipse Foundation managed orgs are such that only committers are shortlisted as candidates to assign issues to. When we want to assign issues to contributors or new folks, we ask them to add a comment in the issue, at which point their profile ID shows up in the list of potential assignees. All @jmjaffe37 will need to do is add a comment in #6012 and you will then be able to assign it to him.