Test optimization levels #1317
Test optimization levels #1317
Conversation
|
Could you elaborate a bit on the purpose of this PR? |
|
The idea is to make sure we get the same result when executing js stuff with different optimization levels. |
|
@p-bakker ok for you? |
|
Mmm, not sure this is the right approach, adding the logic to run the tests at different optimization levels to the test itself. We have all the logic to run tests at the different optimization levels in the TestRunners I think, see https://github.com/mozilla/rhino/blob/master/testsrc/README.md#optimization-levels. And at least when running Test262 with the option enabled to update the test262.properties file, the tests are run at optimization levels -1, 0 and 1 and for a test to pass it must pass at all three tested optimization levels. So the approach to code inside the test itself to run against multiple optLevels seems not correct to me and will cause the tests to run at the tested optLevels multiple times when the MozillaSuiteTest is used, as that already runs the tests at multiple optLevels. Instead, for our own tests (MozillaSuiteTest) maybe we want to have an option to somehow combine the results of the three runs at different optLevels and pass or fail it once. |
|
Ah now i see you point, but this is more a general problem with the test suite (i noted parts of the problem also while working on this). Maybe i will try to cleanup the test suit regarding this (step by step). |
|
Haven't double-checked it now, but from what I recall running |
|
And I know for sure that the update test262.properties option only marks a particular test as passed in the .properties file if it passes all three tested optLevels (and if it doesn't pass on a particular optLevel but passes on other optLevels, it'll update the comment on the .properties file accordingly) |
|
ok, will have a look |
Yes that's also my understanding, same for all test annotated @rhinotest (but we have to at least sync the test levels used) |
use Utils.runWithAllOptimizationLevels at more places
|
Sorry it's been such a long time -- some of the big frameworks like test262 do indeed run bunches of tests at multiple optimization levels but many more tests do things differently and it's inconsistent. If this PR makes more tests work at different optimization levels then it seems like a pretty good idea to fix! |
…0917 * commit '6387bac4281ba1c4a8050cd055071615593d214c': (78 commits) More consistent test optimization levels (mozilla#1317) ci: set minimal permissions on GitHub Workflows Compare all primitive value type wrappers by value Fixing formatting issues Treat String, ConsString, Boolean, and Double as value types setter function (from property descriptor) has to convert the args (see HtmlUnit#7) also use try-with-resource FIX: Wrap key and value for NativeJavaMap iterator Added more tests FIX: Wrap result of iterator BUG: for X of javaList does not work properly in strict mode removed obsolete LineSeparator Spotless Preserving cause, when a JavaException is rethrown in JavaScript fix ScriptException when bound functions are called inside Promise.then() test cases from mozilla#1242 fix taken from p-bakker's sugestion fix name property for bound functions (see issue mozilla#1297) Fix the condition for isResourceChanged Code Cleanup (mozilla#1295) ignore %c styling but count to not disturb further replacements ... # Conflicts: # src/org/mozilla/javascript/NativeObject.java # testsrc/org/mozilla/javascript/tests/json/JsonParserTest.java
No description provided.