-
Notifications
You must be signed in to change notification settings - Fork 273
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Option --full-slice is unsound #260
Comments
@kroening: can you also provide some starting point for this particular task? For example, which test cases should I consider (or if I have to create a test suite)? What is actually wrong with --full-slice and --all-properties? Which additional options should I use to test --full-slice and --all-properties (e.g., cbmc file.c --full-slice, cbmc file.c --all-properties)? |
@peterschrammel: which test suite can I add regression tests related to this issue? It seems that the --all-properties option is related to cbmc-incr. Is that correct? However, --full-slice option is not clear which test suite I can add test cases to. |
Just to keep the history. @tautschnig: This is the example with pointers that I mentioned in your visit at DiffBlue.
I was getting a different verification result when verifying it with --full-slice. Additionally, the full-slice option gives incorrect results for the following C programs from cbmc regression suite: Running Fixedbv3/test.desc [FAILED] We believe that issue #327 should resolve part of these problems. |
We have made some progress with the full-slice option, but we are still facing issues about the functional call. Here is the current status of the CBMC regression suite when using the --full-slice option:
|
@tautschnig: After integrating your work from #327 into my branch, most benchmarks fail due to "failed to find state". In particular, this part of the code in the ai.h:
It must be some minor issue to solve, but I'm still unable to find the root cause. Do you have any suggestion? |
Some additional information about the debug process. It seems that we correctly build the CFG, fill the queue according to slicing criterion, and then compute the program dependence graph (and post-dominators). However, when we try computing the fixedpoint, we simply get "failed to find state" after adding the data / control dependencies for a given node. |
If we check whether a given state exists before we add data / control dependencies, then apparently it solves the problem; however, I'm unsure whether this fix is correct. This really needs further investigation. Here is the current status if I run CBMC regression suite using the full-slice option:
|
Thanks lot for all the updates! Unfortunately I won't find time to properly look into this before Saturday, but any notes posted by then will be of help. |
@tautschnig: I was just wondering whether you had a chance to look at this issue? I'm going to create some issue in github so that I can get assistance to fix further bugs that I have found in the dependence graph (e.g., C programs with unions are not fully supported). |
@tautschnig: If we check for array_copy during the implicit call in the full slicer, then we can fix the issues related to the test cases realloc1 and realloc2.
However, we still have 6 issues to fix in the CBMC regression suite as follows:
I have open a ticket in github to further investigate the test case equality_through_union3 (#380). I'll take a look at the other test cases that are failing. |
@kroening and @tautschnig: I have updated my branch with the latest version of the CBMC master branch. This particular commit starts breaking the full-slice option of CBMC: fca6cd6 (make_top, make_bottom, make_entry are now are required) In order to reproduce the problem, you can just compile and run the CBMC master branch as follows:
I'm struggling for some days to find a solution for this particular problem as I don't understand very well this AI code. In the full slicer code, I'm unsure whether I'm incorrectly instantiating the program dependence graph after all those changes to the CBMC master:
I've also talked to @danpoe to get some help on the AI code, but this particular part of the code (i.e., the dependence graph) is also new for him. I would appreciate any support from @kroening and @tautschnig since I have no idea about the root cause of the problem. |
I think I know what the problem is. |
I have integrated the fix from #420 into my branch https://github.com/lucasccordeiro/cbmc/tree/dependence-graph-fix and also fixed more 3 test cases. The current results of running the full-slice option over the CBMC regression suite is:
|
SEC-105 Cleanup LVSA code (nothing functional)
…in-methods-for-vs-dependence-graph Specialised AI domain methods for variable sensitivity dependence graph
An update on this: Tracing the tests here is hard (because some of them have been deleted in subsequent PRs, and others are running as part of our However, the There exist some cases however (when you change the An example of that would be the test
with the execution log being this:
Notice the last assertion, which is the only assertion present in the code ( If we ask for a trace for that we get:
If we ask to see the BEFORE:
AFTER:
As you can notice, |
@NlightNFotis thank you so much for digging deeper here! It won't be the last problem that needs fixing, but on this occasion it seems that the lack of handling |
@tautschnig Not a problem at all. Anything I can do, just let me know - happy to help with getting to the bottom of this. |
To ensure --full-slice keeps working on all of CBMC's regression tests, run them in CI via an additional test profile. Some tests have to be excluded for they rely on program statements not being removed. Fixes: diffblue#260
To ensure --full-slice keeps working on all of CBMC's regression tests, run them in CI via an additional test profile. Some tests have to be excluded for they rely on program statements not being removed. Fixes: diffblue#260
To ensure --full-slice keeps working on all of CBMC's regression tests, run them in CI via an additional test profile. Some tests have to be excluded for they rely on program statements not being removed. Fixes: diffblue#260
It won't be the last bug to resolve, but this is currently blocked by a bug in how we handle recursion in the abstract interpreter, see also #7041 (comment). |
add regression tests for C
@kroening, any observations what goes wrong with --full-slice at the moment?
The text was updated successfully, but these errors were encountered: