Version-sensitive plutus-benchmark tests (PLT 7544) #5587
Conversation
| @@ -12,6 +12,10 @@ | |||
| {-# OPTIONS_GHC -fno-omit-interface-pragmas #-} | |||
| {-# OPTIONS_GHC -fno-spec-constr #-} | |||
| {-# OPTIONS_GHC -fno-specialise #-} | |||
| {-# OPTIONS_GHC -fexpose-all-unfoldings #-} | |||
There was a problem hiding this comment.
We should remember to get someone to look at this. Like it says in the comment underneath, there's a ticket at PLT-7976.
|
|
||
| -- | Run a 'TestTree' of tests with a given name prefix. | ||
| runTestNestedIn :: [String] -> TestNested -> TestTree |
There was a problem hiding this comment.
I''m not sure if I like this name. It sounds like it runs a test, but really it's running a computation in the Reader monad.
| testNestedGhc folderName = testNested (folderName </> ghcVersion) | ||
|
|
||
| -- Create a TestTree which runs in the directory 'path/<ghc-version>/dir' | ||
| runTestGroupNestedGhcIn :: [FilePath] -> FilePath -> [TestNested] -> TestTree |
There was a problem hiding this comment.
I'm conflicted about whether to represent paths like ["dir1", "dir2", "dir3"] or "dir1/dir2/dir3". We use both of these in the codebase (and even in this file), but in theory we might be in trouble with the latter on a system that doesn't use "/" as the path separator. That's presumably very unlikely though. Anyway, I didn't want to embark on a massive irrelevant-to-this-PR refactoring to make everything consistent.
There was a problem hiding this comment.
The first options seems strictly better. I'm not sure whether or not the second option works on Windows.
There was a problem hiding this comment.
The first options seems strictly better.
Maybe we can do that in a later PR. You can work around it though, like this.
There was a problem hiding this comment.
I found some tests in plutus-ir that had paths like ["a/b/c"] so I changed them to ["a", "b", "c"] while I was at it.
| testNestedGhc folderName = testNested (folderName </> ghcVersion) | ||
|
|
||
| -- Create a TestTree which runs in the directory 'path/<ghc-version>/dir' | ||
| runTestGroupNestedGhcIn :: [FilePath] -> FilePath -> [TestNested] -> TestTree |
There was a problem hiding this comment.
I don't like this name very much, but I couldn't think of anything better.
There was a problem hiding this comment.
I'm not sure why this is needed. There's doesn't seem to be a use case where dir isn't []?
There was a problem hiding this comment.
Oh, that's wrong. It says it puts it in path/version/dir, but in fact it puts it in path/dir/version.
This gets used in marlowe/test and lists/test. See here for example, where the directory is semantics. You end up with golden files in plutus-benchmark/marlowe/test in the following directories:
role-payout/9.2role-payout/9.6semantics/9.2semantics/9.6
What I meant to do in that function was to move the version directories further up, so you'd have
9.2/role-payout9.2/semantics9.6/role-payout9.6/semantics
I don't know if that's any better. Maybe we should just leave the directory layout as it is and get rid of the final dir argument in that function.
There was a problem hiding this comment.
I've fixed this now: all of the 9.2/9.6/etc directories should be at the leaves of the directory hierarchy (so we're using the first option above).
|
|
||
| -- Make a set of golden tests with results stored in subdirectories determined | ||
| -- by the GHC version. | ||
| testGroupGhc :: [TestNested] -> TestTree |
There was a problem hiding this comment.
... or maybe versionedTestGroup or something. I feel that it should mention versions somewhere.
| testNestedGhc folderName = testNested (folderName </> ghcVersion) | ||
|
|
||
| -- Create a TestTree which runs in the directory 'path/<ghc-version>/dir' | ||
| runTestGroupNestedGhcIn :: [FilePath] -> FilePath -> [TestNested] -> TestTree |
There was a problem hiding this comment.
The first options seems strictly better. I'm not sure whether or not the second option works on Windows.
| testNestedGhc folderName = testNested (folderName </> ghcVersion) | ||
|
|
||
| -- Create a TestTree which runs in the directory 'path/<ghc-version>/dir' | ||
| runTestGroupNestedGhcIn :: [FilePath] -> FilePath -> [TestNested] -> TestTree |
There was a problem hiding this comment.
I'm not sure why this is needed. There's doesn't seem to be a use case where dir isn't []?
kwxm
changed the title
This release contains backport of the change to PlutusLedgerApi.V1.Value (adding `-fexpose-all-unfoldings`) from #5587.
* Reorganise nofib tests * Versioned golden tests for nofib * Realign comment * Versioned golden tests for marlowe * Versioned golden tests for lists and script contexts * Versioned golden tests for ed25519 and bls12-381 costs * Tidying up * Update some golden files * Update some golden files * Tidy up directory path usage * Comment about directory structure
This is PLT-7544: "Run plutus-benchmark tests on both 9.2 and 9.6". The golden tests in
plutus-benchmarknow all have two lots of results, one for GHC 9.2 and one for GHC 9.6.