-
Notifications
You must be signed in to change notification settings - Fork 45
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tribits_add[_advanced]_test(): Address handling of semi-colons in ARGS and ENVIRONMENT (#299) #464
tribits_add[_advanced]_test(): Address handling of semi-colons in ARGS and ENVIRONMENT (#299) #464
Conversation
…ning() (TriBITSPub#299) With a newer version of CMake, we can use DEPRECATION instead of WARNING.
TriBITSPub#299) This now allows arguments to 'ARGS' to contain semi-colons ';'. To do this, it adds a new function with unit tests tribits_parse_arguments_for_list() to deal with these cases. A number of other changes in this commit include: * Fixed tribits_add_advanced_test() to actually check and report unused args (this did not work before) and added more tests to show this this. (I had to fix one TriBITS test TriBITS_CTestScripts_cmnd_3_args_1_overall_working_directory defined with tribits_add_advanced_test() that was using MESSAGE incorrectly.) * Fixed setting and testing with <Project>_CHECK_FOR_UNPARSED_ARGUMENTS CUSTOM_FAILURE_MODE and added more automated tests to show this. * Renamed the test TriBITS_TribitsExampleProject_PkgWithUserErrors_UNPARSED_ARGUMENTS and the the name _UNPARSED_ARGS_ to _UNPARSED_ARGUMENTS_ for consistancy. * Added tribits_parse_arguments_in_list() * Added tribits_copy_list() * Added unittest_compare_list_ele_const()
…c args with semi-colons (TriBITSPub#299) This is a better solution that is consistent with how ExternalProject_Add() gets around this problem. What I did: * Added explicit LIST_SEPARATOR to tribits_add_advanced_test() * Removed usage of tribits_parse_arguments_in_list() (it is no longer needed) * Moved top-level arguments to cmake_parse_arguments() into their own self-documenting variables that will be easier to maintain.
…#299) There is no need for these anymore now that an explicit LIST_SEPARATOR is used from the top down. Parsing and everything happens just fine.
f02f4a1
to
47bc34f
Compare
TriBITSPub#299) Now I will be able to set env vars that have semi-colons, like CMAKE_PREFIX_PATH. NOTE: I had to use the: cmake_parse_arguments(PARSE_ARGV 0 FWD "" "" "") tick to get the arugments forwarded correctly that I learned from "Professional CMake".
47bc34f
to
bd550e6
Compare
Hello @MikolajZuzek and @mperrinel. Can one of you review this PR in detail? I don't think both of you need to review the entire PR. But you both might look at changes to these three files: |
FYI: More discussion about how to handle semi-colons |
@MikolajZuzek and @mperrinel, BTW, please let me know (in a comment in this PR) when you have started the review. I am seeing a few things that need addressed so I will be adding a new commit or two to address the issues. |
…NT (TriBITSPub#299) This is to complement the same changes made to tribits_add_advanced_test(). I also added some missing tests for tribits_add_advanced_test() and updated CHANGELOG.txt for more info.
…EPARATOR (TriBITSPub#299) NOTE: I reversed the arguments to tribits_add_executable_and_test() in the unit test since when I first added LIST_SEPARATOR <sep> after TIMEOUT 1.5, it passed LIST_SEPARATOR <sep> as elements in TIMEOUT and passed my check at the end. By reversing the argments, this did not happen and the test failed again.
@MikolajZuzek and @mperrinel, FYI, I added some new commits to round this out. I don't think I will be adding any more new commits (unless typos or other problems are found). |
@bartlettroscoe I am starting the review on it |
Reviewing this as well |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It was a technical one. The code seems good for me. I found a lot of TYPO mistakes. Will create a PR for that.
@mperrinel, thanks for the review. Yes, please put in a PR with the fixes. (Looks like I forgot to 'flyspell' when I edited these files.) |
Is I was able to get rid of it and make all [new] It's more of a PoC than complete fix - there is still handful of tests failing. |
…po_fixes Fixed typo mistake seen in PR 464
Thanks for the review @mperrinel and fixing the typos. No problems with any of the source code or tests? @MikolajZuzek, note the final conclusions on the handling of semicolon
@bradking did not refute these conclusions and added some CMake documentation MRs to officially document the workaround However, it just occurred to me that we could consider using the generator expression @MikolajZuzek, please don't just jump in and try to implement this. Let's wait for the response back from @bradking in that CMake GitLab Issue. Then, trying this should be fairly easy to try out given the tip of this existing branch (but adapting for the unit tests will be a little tricky since those need to generate the |
@bartlettroscoe Thank you for the feedback. Here's the fixed version that greens out all relevant tests without changing their content (except for If you see any break in backward compatibility or any other issue, please expose it via unit test. |
@bartlettroscoe |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bartlettroscoe @keitat @mperrinel
Looks good. The fix is consistent with semicolon handling in CMake and the test coverage is great!
It could be worthy to also explore handling problematic lists as strings to avoid complicating TriBITS API with LIST_SEPARATOR
- see the discussion above starting at #464 (comment).
Of course. Let me know when the PR is ready for re-review. |
@MikolajZuzek, can you please post that as a WIP PR so it is easier to see the changes? However, Kitware's feedback up to this point (from multiple CMake maintainers) is that trying to keep raw semi-colons so my idea to try out Also, see recent comments:
The only robust way to deal with semi-colon And now the As noted by @mathstuf in:
Getting even the internals of CMake to deal with raw semi-colon |
Yes. That's why CMake's own ExternalProject module uses
CMake interfaces that don't document support for semicolons likely don't support them. We've now updated the documentation to clarify this point. It's unfortunate that the language is like this, but it wasn't designed to be as general purpose as today's use cases would like it to be. In some cases one can add quoting or backslashes near semicolons until things seem to work locally, but the resulting code may be relying on assumptions about implementation details that might change. |
@bradking, right, and CMake needs to write tests and documentation that pin down existing behaviors so at least it is possible to pass data containing semi-colon |
Without further objection, I am going to merge this to 'master'. There is just no other robust alternative (short of creating a new CMake scripting language that treats lists as first-class entities and has the notation to support that). |
This PR relies on Please correct me if I get your discussion wrong: |
Of course - see bartlettroscoe#2. Please note that there is no difference in how we feed CMake commands with |
@MikolajZuzek, yes Kitware knows about that and has promised to address this. See: CC: @bradking |
@bartlettroscoe Thank you for the confirmation. PS. Year ago I was happily using CMake 3.19+ to pass around JSON structures with recursively embedded lists and dictionaries. Now I'm a bit scarred of passing plain strings because they may eventually contain semicolons, which can lead to undefined behavior on client platforms... |
Right. The only solution is to develop a new CMake language and parser. I am working with Kitware to start to advertise the idea for a proposal for a new CMake language parser (with both a new declarative language and a new Turing-complete imperative scripting language). |
@MikolajZuzek passing around plain strings containing JSON is fine so long as nothing tries to interpret it as a list or anything else besides a single string. The problem with semicolons is only when one wants to store that plain string as an element of another list. Cleanly handling lists with arbitrary content in their elements, such as nested lists, will require the new language Roscoe mentioned. |
Thank you @bradking: that's an excellent summary of my key takeaway and lesson learned from this PR, bartlettroscoe#2 and related discussions on CMake issues. |
…/TriBITS#464) Before, this was passing in: FAIL_REGULAR_EXPRESSION "FAIL;BUMMER" which just happened to be added as a list to the test property with the old TriBITS. But with the new TriBITS in TriBITSPub/TriBITS#464 which more correctly deals with semi-colons and list arguments, this was being set as a single property value "FAIL\\;BUMMER" which did not match the output. Passing in the list with: FAIL_REGULAR_EXPRESSION FAIL BUMMER works both with old and new TriBITS and is more logical given how CMake/CTest is supposed to be used. (The old TriBITS documentation was incorrect.)
…BITS#464) Tt turns out that these tests really did not need to pass a list of values to FAIL_REGULAR_EXPRESSION. Each test was really only looking for a single value of 'FAILED' or 'BUMMER'. And you can argue that the tests are stronger and better by taking out the other value that is not supposed to be printed in that test case.
…ESSION (TriBITSPub#464) Turns out that the tribits_add_advanced_test() test-block arguments PASS_REGULAR_EXPRESSION and FAIL_REGULAR_EXPRESSION did not support taking in a list of regexes like the built-in CMake CTest properties of the same names. This commit updates TAAT() to allow a list of arguments and they behave the exact same way as the built-in properties (i.e. matching any). I also updated the documentation to show how to pass in a list of regexes correctly for PASS_REGULAR_EXPRESSION, FAIL_REGULAR_EXPRESSION, FINAL_PASS_REGULAR_EXPRESSION and FINAL_FAIL_REGULAR_EXPRESSION. I also fixed the documentation for tribits_add_test() as well for PASS_REGULAR_EXPRESSION and FAIL_REGULAR_EXPRESSION. Basically, a well constructed CMake function should almost never take in a list of arguments expecting an explicit semi-colon. This was part of the learning in PR TriBITSPub#464 with how to treat semi-colons in CMake. This commit was driven by a usage of FAIL_REGULAR_EXPRESSION in Trilinos while testing updated TriBITS as part of TriBITSPub#299 when I realized the documentation was wrong for how to deal with these lists of regexes.
…expressions Support list for TAAT() PASS_REGULAR_EXPRESSION and FAIL_REGULAR_EXPRESSION (#464)
…/TriBITS#464) Before, this was passing in: FAIL_REGULAR_EXPRESSION "FAIL;BUMMER" which just happened to be added as a list to the test property with the old TriBITS. But with the new TriBITS in TriBITSPub/TriBITS#464 which more correctly deals with semi-colons and list arguments, this was being set as a single property value "FAIL\\;BUMMER" which did not match the output. Passing in the list with: FAIL_REGULAR_EXPRESSION FAIL BUMMER works both with old and new TriBITS and is more logical given how CMake/CTest is supposed to be used. (The old TriBITS documentation was incorrect.)
…BITS#464) Tt turns out that these tests really did not need to pass a list of values to FAIL_REGULAR_EXPRESSION. Each test was really only looking for a single value of 'FAILED' or 'BUMMER'. And you can argue that the tests are stronger and better by taking out the other value that is not supposed to be printed in that test case.
…s:develop' (f7c3706). * trilinos-develop: FindTPLNetcdf.cmake: Lower-case function names (TriBITSPub/TriBITS#274) FindTPLLAPACK.cmake: Lower-case function names (TriBITSPub/TriBITS#274) FindTPLBLAS.cmake: Lower-case function names (TriBITSPub/TriBITS#274) Isorropria: Unconditionally disable tests that depend on EpetraExt (trilinos#10534) Zoltan2: Use single value for FAIL_REGULAR_EXPRESSION (TriBITSPub/TriBITS#464) Zoltan2: Properly pass in list to FAIL_REGULAR_EXPRESSION (TriBITSPub/TriBITS#464) Sacado: Call target_include_directories() for new TriBITS (TriBITSPub/TriBITS#299) ATDM: Strip leading and trailing whitespace from flags for newer CMakes
…s:develop' (f7c3706). * trilinos-develop: FindTPLNetcdf.cmake: Lower-case function names (TriBITSPub/TriBITS#274) FindTPLLAPACK.cmake: Lower-case function names (TriBITSPub/TriBITS#274) FindTPLBLAS.cmake: Lower-case function names (TriBITSPub/TriBITS#274) Isorropria: Unconditionally disable tests that depend on EpetraExt (trilinos#10534) Zoltan2: Use single value for FAIL_REGULAR_EXPRESSION (TriBITSPub/TriBITS#464) Zoltan2: Properly pass in list to FAIL_REGULAR_EXPRESSION (TriBITSPub/TriBITS#464) Sacado: Call target_include_directories() for new TriBITS (TriBITSPub/TriBITS#299) ATDM: Strip leading and trailing whitespace from flags for newer CMakes
…Trilinos:develop' (7d907aa). * trilinos/develop: (31 commits) MiniEM: Fix lambda capture issues in evaluators Phalanx: fix for non-cuda spaces Cmake: Tpetra deprecation removal nightlies Phalanx: cleanup Phalanx: add new ctor to v_of_v Phalanx: another view-of-view utility FindTPLNetcdf.cmake: Lower-case function names (TriBITSPub/TriBITS#274) FindTPLLAPACK.cmake: Lower-case function names (TriBITSPub/TriBITS#274) FindTPLBLAS.cmake: Lower-case function names (TriBITSPub/TriBITS#274) Isorropria: Unconditionally disable tests that depend on EpetraExt (trilinos#10534) Panzer MiniEM: Disable tests for 2nd order elements MueLu Maxwell1: Small fix MiniEM: Add "p coarsen schedule" Zoltan2: Use single value for FAIL_REGULAR_EXPRESSION (TriBITSPub/TriBITS#464) Zoltan2: Properly pass in list to FAIL_REGULAR_EXPRESSION (TriBITSPub/TriBITS#464) Sacado: Call target_include_directories() for new TriBITS (TriBITSPub/TriBITS#299) Phalanx: Array of Views acceptance test. ATDM: Strip leading and trailing whitespace from flags for newer CMakes Ifpack2 Hiptmair: Add parameter "hiptmair: implicit transpose" Ifpack2 Hiptmair: timers ...
This PR makes the
tribits_add_advanced_test()
function handle arguments that need to include a semi-colon';'
without messing up the list handling in CMake. This introduces a new top-level argumentLIST_SEPARATOR <sep>
used exactly the same way as theLIST_SEPARATOR
argument withExternalProject_Add()
. Apparently, this is an accepted workaround for the problem of lists elements containing semi-colons with CMake.I need to be able to handle semi-colons with
tribits_add_advanced_test()
for the new tests I am developing for #299.Note that this PR contains an earlier commit that allowed the user to use quoted args that contained
';'
and silently replaced that with<semicolon>
but I think it is better to follow the convention fromExternalProject_Add()
and make this explicit (even if it is a bit of a hack). I spent many hours experimenting with ways to deal with these arguments with failure after failure but I wanted to document this one way I found that worked with a commit. (It was after I asked Brad King that he mentioned theExternalProject_Add()
way of usingLIST_SEPARATOR <sep>
to workaround this problem and I refactored to use that approach. The core CMake language is really just broken when it comes to dealing with general lists with spaces and semi-colons.)NOTE: I also fixed checking for unparsed args to
tribits_add_advanced_test()
that were not being checked before and this technically breaks backward compatibility. (I had to fix one of the TriBITS tests that had an unparsed/ignored argument so I am sure this is going to break Trilinos once this gets merged to Trilinos 'develop'.)See the individual commit log messages for more details.