Migration from JUnit4 to JUnit5

[bauersimon]

Goal: benchmark LLMs on migrating code.

In our concrete case it should be from JUnit 4 to JUnit 5.

Iteration 1:

• add a new task for migrating code
  • testdata layout
    • implementation sample files are in one folder
    • JUnit 4 test sample files are in one test folder (1:1 mapping)
    • pom.xml already contains JUnit 5 dependencies (cause migrating a pom file successfully is a different task!)
    • (maybe) pom_4.xml contains JUnit 4 dependencies to manually verify that the tests work in practice
  • for every test file, give to LLM, request migration to JUnit 5
  • execute only that test file
  • score should be if the coverage is still the same (but that shouldn't matter during the benchmark anyways)
• apply symflower fix as a separate task version
• plain example to showcase it works

Iteration 2: Actual real world testdata -> proprietary

Iteration 3: Testdrive with a few models - > done

[bauersimon]

done