Skip to content

Commit

Permalink
Several changes to runtests.py
Browse files Browse the repository at this point in the history 
Remove the flag-based test ignoring of -simdjs as it has
caused confusion on whether or not tests were running.
Add test tagging for simd, asmjs, and wasm functionality.
Add the ability to read --sanitize flags and exclude known
failing tests.

Edit: Allow all of the asmjs tests, since they work now.

@obastemur fixed the issues with them in chakra-core#2716, so I've gone and
enabled them here.

Edit: Add --static to test flags.

Not using it currently to mask out any tests; this is just to make
the test and build flags line up better.

Edit: Add a -j flag to runtests.py for num processes

This makes it line up better with the build flags for CI

Edit: Add a flag to warn instead of fail on test timeout

This is in an attempt to reduce noise a bit; a small number of tests
fail to run quickly sometimes enough on one channel of our test infra.

Edit: Specify type for -j test flag.

Edit: Disable tests that do not currently work with ASAN
  • Loading branch information
@Penguinwizzard
Penguinwizzard committed Apr 1, 2017
1 parent 6c5914f commit ec5ee75
Show file tree
Hide file tree
Showing 5 changed files with 91 additions and 48 deletions.
14 changes: 7 additions & 7 deletions test/Bugs/rlexe.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@
<default>
<files>SIMDExtendArgCSE.js</files>
<baseline>SIMDExtendArgCSE.baseline</baseline>
<tags>exclude_dynapogo,exclude_ship,require_backend</tags>
<tags>exclude_dynapogo,exclude_ship,require_backend,require_simd</tags>
<compile-flags>-bgjit- -on:asmjs -simdjs -testtrace:asmjs -AsmJsStopOnError</compile-flags>
</default>
</test>
Expand Down Expand Up @@ -280,7 +280,7 @@
<default>
<files>b208.js</files>
<baseline>b208.baseline</baseline>
<tags>require_backend</tags>
<tags>require_backend,require_simd</tags>
<compile-flags> -bgjit- -simdjs -simd128typespec -mic:1 -lic:1 -off:simplejit</compile-flags>
</default>
</test>
Expand All @@ -307,39 +307,39 @@
<default>
<files>b208_asmjs.js</files>
<baseline>b208_asmjs.baseline</baseline>
<tags>exclude_dynapogo,exclude_ship,require_backend</tags>
<tags>exclude_dynapogo,exclude_ship,require_backend,require_simd</tags>
<compile-flags>-simdjs -AsmJsStopOnError</compile-flags>
</default>
</test>
<test>
<default>
<files>b208_asmjs.js</files>
<baseline>b208_asmjs.baseline</baseline>
<tags>exclude_dynapogo,exclude_ship,require_backend</tags>
<tags>exclude_dynapogo,exclude_ship,require_backend,require_simd</tags>
<compile-flags> -bgjit- -simdjs -simd128typespec -asmjs- -mic:1 -lic:1 -off:simplejit</compile-flags>
</default>
</test>
<test>
<default>
<files>testSimdManyVars.js</files>
<baseline>testSimdManyVars.baseline</baseline>
<tags>exclude_dynapogo,exclude_ship,require_backend</tags>
<tags>exclude_dynapogo,exclude_ship,require_backend,require_simd</tags>
<compile-flags> -simdjs -asmjs -off:backend -testtrace:asmjs</compile-flags>
</default>
</test>
<test>
<default>
<files>b95.js</files>
<baseline>b95.baseline</baseline>
<tags>exclude_dynapogo,exclude_ship,require_backend</tags>
<tags>exclude_dynapogo,exclude_ship,require_backend,require_simd</tags>
<compile-flags>-simdjs -testtrace:asmjs -asmjs</compile-flags>
</default>
</test>
<test>
<default>
<files>b108.js</files>
<baseline>b108.baseline</baseline>
<tags>exclude_dynapogo,exclude_ship,require_backend</tags>
<tags>exclude_dynapogo,exclude_ship,require_backend,require_simd</tags>
<compile-flags>-simdjs -asmjs -testtrace:asmjs -asmjsstoponerror</compile-flags>
</default>
</test>
Expand Down
13 changes: 9 additions & 4 deletions test/es6/rlexe.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1330,59 +1330,64 @@
<default>
<files>moduletest1.js</files>
<compile-flags>-ES6Module</compile-flags>
<tags>exclude_dynapogo</tags>
<tags>exclude_dynapogo,exclude_sanitize_address</tags>
</default>
</test>
<test>
<default>
<files>moduletest2.js</files>
<compile-flags>-ES6Module</compile-flags>
<tags>exclude_dynapogo</tags>
<tags>exclude_dynapogo,exclude_sanitize_address</tags>
</default>
</test>
<test>
<default>
<files>module-syntax.js</files>
<compile-flags>-ES6Module -args summary -endargs</compile-flags>
<tags>exclude_dynapogo</tags>
<tags>exclude_dynapogo,exclude_sanitize_address</tags>
</default>
</test>
<test>
<default>
<files>module-syntax1.js</files>
<compile-flags>-ES6Module -args summary -endargs</compile-flags>
<tags>exclude_sanitize_address</tags>
</default>
</test>
<test>
<default>
<files>module-functionality.js</files>
<compile-flags>-ES6Module -args summary -endargs</compile-flags>
<tags>exclude_dynapogo</tags>
<tags>exclude_dynapogo,exclude_sanitize_address</tags>
</default>
</test>
<test>
<default>
<files>module-syntax.js</files>
<compile-flags>-ES6Module -force:deferparse -args summary -endargs</compile-flags>
<tags>exclude_sanitize_address</tags>
</default>
</test>
<test>
<default>
<files>module-syntax1.js</files>
<compile-flags>-ES6Module -force:deferparse -args summary -endargs</compile-flags>
<tags>exclude_sanitize_address</tags>
</default>
</test>
<test>
<default>
<files>module-namespace.js</files>
<compile-flags>-ES6Module -Es6ToStringTag</compile-flags>
<baseline>module-namespace.baseline</baseline>
<tags>exclude_sanitize_address</tags>
</default>
</test>
<test>
<default>
<files>test_bug_2645.js</files>
<compile-flags>-ES6Module</compile-flags>
<tags>exclude_sanitize_address</tags>
</default>
</test>
<test>
Expand Down
46 changes: 23 additions & 23 deletions test/rlexedirs.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -265,140 +265,140 @@
<dir>
<default>
<files>AsmJs</files>
<tags>exclude_serialized,exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_serialized,exclude_arm,exclude_arm64,require_backend,require_asmjs</tags>
</default>
</dir>
<dir>
<default>
<files>AsmJsFloat</files>
<tags>exclude_serialized,exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_serialized,exclude_arm,exclude_arm64,require_backend,require_asmjs</tags>
</default>
</dir>
<dir>
<default>
<files>AsmJsParser</files>
<tags>exclude_serialized,exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_serialized,exclude_arm,exclude_arm64,require_backend,require_asmjs</tags>
</default>
</dir>
<dir>
<default>
<files>SIMD.float32x4</files>
<tags>exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_arm,exclude_arm64,require_backend,require_simd</tags>
</default>
</dir>
<dir>
<default>
<files>SIMD.int32x4</files>
<tags>exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_arm,exclude_arm64,require_backend,require_simd</tags>
</default>
</dir>
<dir>
<default>
<files>SIMD.int8x16</files>
<tags>exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_arm,exclude_arm64,require_backend,require_simd</tags>
</default>
</dir>
<dir>
<default>
<files>SIMD.float32x4.asmjs</files>
<tags>exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_arm,exclude_arm64,require_backend,require_simd</tags>
</default>
</dir>
<dir>
<default>
<files>SIMD.int32x4.asmjs</files>
<tags>exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_arm,exclude_arm64,require_backend,require_simd</tags>
</default>
</dir>
<dir>
<default>
<files>SIMD.int16x8.asmjs</files>
<tags>exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_arm,exclude_arm64,require_backend,require_simd</tags>
</default>
</dir>
<dir>
<default>
<files>SIMD.int8x16.asmjs</files>
<tags>exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_arm,exclude_arm64,require_backend,require_simd</tags>
</default>
</dir>
<dir>
<default>
<files>SIMD.uint32x4.asmjs</files>
<tags>exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_arm,exclude_arm64,require_backend,require_simd</tags>
</default>
</dir>
<dir>
<default>
<files>SIMD.uint16x8.asmjs</files>
<tags>exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_arm,exclude_arm64,require_backend,require_simd</tags>
</default>
</dir>
<dir>
<default>
<files>SIMD.uint8x16.asmjs</files>
<tags>exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_arm,exclude_arm64,require_backend,require_simd</tags>
</default>
</dir>
<dir>
<default>
<files>SIMD.bool32x4.asmjs</files>
<tags>exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_arm,exclude_arm64,require_backend,require_simd</tags>
</default>
</dir>
<dir>
<default>
<files>SIMD.bool16x8.asmjs</files>
<tags>exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_arm,exclude_arm64,require_backend,require_simd</tags>
</default>
</dir>
<dir>
<default>
<files>SIMD.bool8x16.asmjs</files>
<tags>exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_arm,exclude_arm64,require_backend,require_simd</tags>
</default>
</dir>
<dir>
<default>
<files>SIMD.workloads.asmjs</files>
<tags>exclude_serialized,exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_serialized,exclude_arm,exclude_arm64,require_backend,require_simd</tags>
</default>
</dir>
<dir>
<default>
<files>SIMD.wrappers</files>
<tags>exclude_serialized,exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_serialized,exclude_arm,exclude_arm64,require_backend,require_simd</tags>
</default>
</dir>
<dir>
<default>
<files>SIMD.TypeSpec</files>
<tags>exclude_serialized,exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_serialized,exclude_arm,exclude_arm64,require_backend,require_simd</tags>
</default>
</dir>
<dir>
<default>
<files>WasmSpec</files>
<tags>exclude_serialized,exclude_arm,exclude_arm64,require_backend,exclude_jshost,exclude_win7</tags>
<tags>exclude_serialized,exclude_arm,exclude_arm64,require_backend,exclude_jshost,exclude_win7,require_wasm</tags>
</default>
</dir>
<dir>
<default>
<files>wasm</files>
<tags>exclude_serialized,exclude_arm,exclude_arm64,require_backend</tags>
<tags>exclude_serialized,exclude_arm,exclude_arm64,require_backend,require_wasm</tags>
<tags>exclude_arm,exclude_arm64</tags>
</default>
</dir>
<dir>
<default>
<files>Debugger</files>
<tags>exclude_serialized,exclude_jshost,exclude_snap,require_debugger</tags>
<tags>exclude_serialized,exclude_jshost,exclude_snap,require_debugger,exclude_sanitize_address</tags>
</default>
</dir>
<dir>
<default>
<files>DebuggerCommon</files>
<tags>exclude_serialized,exclude_jshost,exclude_snap,require_debugger</tags>
<tags>exclude_serialized,exclude_jshost,exclude_snap,require_debugger,exclude_sanitize_address</tags>
</default>
</dir>
<dir>
Expand Down
Loading

0 comments on commit ec5ee75

Please sign in to comment.