Coverage: support instrumenting one function at a time #1744
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
smowton
requested review from
cesaro,
chrisr-diffblue,
kroening,
martin-cs,
mgudemann,
peterschrammel,
tautschnig and
thk123
as code owners
src/goto-instrument/cover.cpp
Outdated
| instrument_cover_goals( | ||
| function.body, config.cover_instrumenters, message_handler); | ||
|
|
||
| if(config.traces_must_terminate && |
There was a problem hiding this comment.
This might not (and will not) be reached if the function_filter filters out __CPROVER prefixed functions. But it must be called after instrument_cover_goals() (or requires a fix to is_non_coverage_assert()).
src/goto-instrument/cover.h
Outdated
| void instrument_cover_goals( | ||
| const cover_configt &, | ||
| goto_model_functiont &, | ||
| message_handlert &message_handler); |
There was a problem hiding this comment.
not sure why the param name is needed here
| @@ -44,6 +55,14 @@ void instrument_cover_goals( | |||
| coverage_criteriont, | |||
| message_handlert &message_handler); | |||
|
|
|||
| std::unique_ptr<cover_configt> get_cover_config( | |||
There was a problem hiding this comment.
What about wrapping everything into a covert class to keep the configuration as state? This would avoid exposing the implementation details to the user of the interface.
smowton
force-pushed
the
smowton/feature/instrument_cover_per_function
branch
from
e90ddd9 to
6021281
Compare
|
@peterschrammel updated fixing the issue you pointed out. Re: exposing implementation details, if I wrap it all into one big class I'll have to expose that, so I think exposure happens either way. |
smowton
force-pushed
the
smowton/feature/instrument_cover_per_function
branch
from
6021281 to
44da863
Compare
This is necessary because not all functions work with goto_programt::get_function_id, e.g. those with empty bodies.
smowton
force-pushed
the
smowton/feature/instrument_cover_per_function
branch
from
44da863 to
35286ed
Compare
|
@tautschnig @peterschrammel this one is now ready. Peter, I addressed your comments above. |
smowton
force-pushed
the
smowton/feature/instrument_cover_per_function
branch
from
35286ed to
94c462b
Compare
|
Pinging @thk123 @chrisr-diffblue @mgudemann this is ready for review |
|
Made a test-gen bump: https://github.com/diffblue/test-gen/pull/1428 |
chrisr-diffblue
approved these changes
Jan 29, 2018
thk123
approved these changes
Jan 29, 2018
| const auto sf_it= | ||
| goto_functions.function_map.find(goto_functions.entry_point()); | ||
| if(sf_it==goto_functions.function_map.end()) | ||
| Forall_goto_program_instructions(i_it, function.body) |
There was a problem hiding this comment.
Can we here swap to using range based for loop? Or perhaps even a std::foreach since the body is so simple
There was a problem hiding this comment.
I'm going to plead "moved code" plus slim benefit and leave this alone :)
|
@thk123 I argue that the existing coverage tests will do this for us when this comes into use. Those tests will appear with the symex-driven lazy loading PR, which will be this code's first user and will introduce a variant of the cbmc-java test directory amongst others, running all tests with symex-driven lazy loading (and therefore per-function GOTO program passes) enabled. |
smowton
force-pushed
the
smowton/feature/instrument_cover_per_function
branch
from
94c462b to
47bc490
Compare
tautschnig
reviewed
Jan 29, 2018
| message_handlert &message_handler) | ||
| { | ||
| messaget msg(message_handler); | ||
| std::unique_ptr<cover_configt> config = util_make_unique<cover_configt>(); |
There was a problem hiding this comment.
Why does this need to be a pointer?
There was a problem hiding this comment.
I'd rather not implement move constructors for all the filter kinds -- simpler to let them not support copy/move and pass a handle to the config object instead. Unique-ptr really takes the pain out of managing its lifespan in any case.
There was a problem hiding this comment.
It just seems a lot of machinery for a very simple problem where performance doesn't matter either. Why not just return cover_configt by value?
There was a problem hiding this comment.
Same thing -- needing copyable and movable everything in the config object. So much simpler to just allocate one and pass it around.
There was a problem hiding this comment.
Ok. So while I would have done it differently, you actually did it, so you win :-)
There was a problem hiding this comment.
Are you happy to approve in that case? This is still blocked on code-owner approval
src/goto-instrument/cover.cpp
Outdated
| /// \param symbol_table: global symbol table | ||
| /// \param message_handler: used to log incorrect option specifications | ||
| /// \return a cover_configt on success, or optionalt() otherwise. | ||
| std::unique_ptr<cover_configt> get_cover_config( |
There was a problem hiding this comment.
Can't this be an optional instead?
This was partially supported already, but didn't apply exactly the same rules as the whole-program instrument_cover_goals function. This factors the whole-program version to use the per-function version, and allows the configuration to be parsed up-front and passed into the per-function version.
smowton
force-pushed
the
smowton/feature/instrument_cover_per_function
branch
from
47bc490 to
311af6d
Compare
peterschrammel
approved these changes
Jan 30, 2018
smowton
pushed a commit
to smowton/cbmc
that referenced
this pull request
May 9, 2018
f7602af Merge commit 'bb88574aaa4043f0ebf0ad6881ccaaeb1f0413ff' into merge-develop-20180327 906aeb3 Merge pull request diffblue#349 from diffblue/owen-jones-diffblue/fix-compilation-for-release 3d8423c Merge pull request diffblue#350 from diffblue/owen-jones-diffblue/skip-duplicate-callsites-in-lazy-methods 73fb488 bugfix from upstream repo for generic crash fd76555 Speed up resolution of virtual callsites in lazy loading 3fd28f3 Replace assert(X) by UNREACHABLE/INVARIANT(X) 557158e Merge pull request diffblue#334 from diffblue/pull-support-20180216 1e48132 Merge from master, 20180216 ad7b28e Updates requsted in the PR: mostly rename 'size -> length'. e3fcb9b Introducing MAX_FILE_NAME_SIZE constant. bb88574 Merge pull request diffblue#1806 from thk123/refactor/address-review-comments-from-1796 db9c214 Merge pull request diffblue#1850 from tautschnig/include-cleanup 78fbf08 Merge pull request diffblue#1844 from smowton/smowton/feature/prepare-symex-for-lazy-loading 4098ed5 Merge pull request diffblue#1849 from smowton/smowton/cleanup/java-main-function-types 06f3e83 Use C++ headers instead of their C variants e918a91 Goto-symex: add support for general function accessor 9e31303 Symex: switch to incrementally populating address-taken locals ac5af68 Address-taken locals analysis: support incremental analysis fe775f3 Merge pull request diffblue#1843 from peterschrammel/instructions-function a6fd729 Cleanup tests with anomalous main functions 5df3fca Clean up get_function_id hacks 552b100 Set function member of each goto instruction in goto program passes 38e6e4a Merge pull request diffblue#1813 from smowton/smowton/fix/cleanup-unused-clinits 278e4e6 Merge pull request diffblue#1826 from smowton/smowton/fix/java-inherited-static-fields a2ebb33 Merge pull request diffblue#1713 from karkhaz/kk-debug-timestamps 7b5dd17 Merge pull request diffblue#1834 from diffblue/library-preconditions 1da5be1 Add tests for inherited static fields 19d622b Add tests to verify that synthetic method removal is performed adc9fd4 Java frontend: clean up unused clinit symbols f15c312 Exclude j.l.O from possible static field hosts. afa443c US spelling of initialize d4d4a9a Tolerate stub static fields defined on non-stub types 873e1f6 Guess public access for stub globals d6783d8 Java method converter: look for inherited static fields 32cc538 Insert stub static globals onto any incomplete ancestor class b2d3d61 Search static fields inherited from interfaces 045ac05 Create stub globals on the first parent incomplete class 5b3cde5 Use a common instance of class_hierarchyt in get_inherited_component e73e756 Create stub globals: check for inherited globals bea6371 Annotate static fields with accessibility 168c2a8 Generalise get_inherited_method f3160e1 resolve_concrete_function_callt -> resolve_inherited_componentt 82549de Emit timestamps on each line of output 3f6965b Replace util/timer* with std::chrono ef08ae2 Merge pull request diffblue#1820 from smowton/smowton/fix/remove-string-solver-iteration-limit 0f20482 Merge pull request diffblue#1836 from karkhaz/kk-remove-unused-lambda-capture e8105bd Merge pull request diffblue#1833 from diffblue/symex_class_cleanup f6f45fc turn some assertions in the stdlib.h models into preconditions 9ea0cc6 pre-conditions for strings fbd54df Remove unused lambda capture 9620802 Merge pull request diffblue#1815 from smowton/smowton/feature/replace-clinit-unwinder 9b59631 Merge pull request diffblue#1828 from smowton/smowton/cleanup/remove-recreate-initialize 1ac9abe Remove string refinement iteration limit c94548c preconditions for delete and delete[] 9ba7fe2 cleanup of some noise (mostly obvious declarators) in the goto_symext class bb64ea6 clean up symex_assign vs. symex_assign_rec 932a38f Merge pull request diffblue#1827 from karkhaz/kk-symex-operator-tidy 968d97e Remove __CPROVER_initialize recreation 06a220a Reimplement remove-static-init-loops to avoid need to inspect GOTO program 2bb98d9 Merge pull request diffblue#1819 from romainbrenguier/refactor/coverage-instrumentation 6492b3a Rearrange cover_basic_blocks header a9549e7 Define constants as const fa35ccd Pull continuation_of_block function out 560d712 Declare constants as const 35422f3 Make update_covered_lines a static function b4cadf8 [path explore 1/8] Tidy up symext top-level funs 0f3ae1a Make representative_inst an optional dc696a4 Make format_number_range function instead of class 678218a Merge pull request diffblue#1825 from thk123/refactor/corrected-path-of-language-file ba76a8f Merge pull request diffblue#1751 from tautschnig/fix-1748 b665269 Correcting path to a file d0889a8 Merge pull request diffblue#1822 from diffblue/legacy-language 441f706 Merge pull request diffblue#1823 from diffblue/cleanup 11714b2 use constant_exprt version of to_integer 733f3b8 remove old-style constructor for member_exprt 63f09ac remove unused function make_next_state 77c8b9c remove translation for certain boolean program constructs 1bac484 cleanout decision_proceduret::in_core d8967f5 moving language.h and language_file.h to langapi folder f9b9599 Merge pull request diffblue#1761 from diffblue/function_typet dd040e5 Added function_typet. bcd88a0 Merge pull request diffblue#1821 from smowton/smowton/feature/test-pl-tags 45f0939 test.pl: add support for tagging test-cases 40b8c03 Updates requested in PR - mainly rename of functions. 7f868e2 Reused private code in 'remove_virtual_functions.cpp' by making it public. ae6775a Merge pull request diffblue#1790 from martin-cs/fix/correct-domain-interface d7bb937 Catch the case when a GOTO instruction is effectively a SKIP. b2fba97 Correct domain transformers so that they meet the preconditions. d447c26 Document the invariants on iterators arguments to transform and merge. e3db794 Whitespace changes to keep clang-format happy. 1990994 Revert "Add edge type parameter to ai transform method" 3ca91bc Revert "Fix iterator comparison bug in reaching_definitions.cpp" ac036fd Revert "Fix iterator equality check bug in dependence_graph.cpp" 86cadcd Revert "Fix iterator equality check bug in custom_bitvector_analysis.cpp" db925de Revert "Fix iterator equality check bug in constant_propagator.cpp" 2c69364 Merge pull request diffblue#1811 from cesaro/iterator-fix 807268e Fixes the symbol_base_tablet iterator 0df054c Merge pull request diffblue#1781 from smowton/smowton/feature/java-create-stub-globals-earlier e163ab6 Java frontend: create synthetic static initialisers for stub globals fbcb423 Merge pull request diffblue#1802 from NathanJPhillips/feature/symbol_iterator e106cf8 Merge pull request diffblue#1793 from smowton/smowton/cleanup/remove-java-new-lowering-pass 52dfc36 Merge pull request diffblue#1731 from diffblue/bugfix/all_resolved_calls f123ae9 Adding comment referencing where the invariant comes from f7c89e1 Add iterator for symbol_table_baset 150f826 Merge pull request diffblue#1801 from hannes-steffenhagen-diffblue/add-idea-gitignore 745afbc Merge pull request diffblue#1796 from thk123/refactor/bytecode-parsing-tidy 6362295 Add .idea (CLion) directory to .gitignore 31da890 Revert "Do lowering of java_new as a function-level pass" 6f6fda7 Merge pull request diffblue#1794 from smowton/smowton/fix/goto-diff-test-escapes 4a538d2 Adding comments on the non-standard patternt 9bfe177 Adding an early guard for correctly parsed exception table 6fe1808 Improved error reporting on invalid constant pool index 93dab4c Escape curly braces in regexes 814cfcc Adapt failing unit test for value set analysis 3ff90bc Add unit test e67a96e Add regression test 3df5348 Adapt regression tests for virtual functions. 09efc90 Re-Resolve function calls if only java.lang.Object was found. a619e48 Merge pull request diffblue#1763 from jeannielynnmoulton/base_class_info_tg1287 0b8dd57 Merge pull request diffblue#1785 from smowton/smowton/fix/core-models-cmake-script 50dcec8 Adding unit tests for extracting generic bases' info 54df3a1 Correcting generic parameters in bases for implicitly generic classes 6d691d7 Parsing generic bases' information into the class symbol 7d041f0 Defining a new type for generic bases f10eb71 Fix Java core-models build script 8d66028 Merge pull request diffblue#1774 from smowton/smowton/feature/java-create-clinit-state-earlier 679d9b8 Java frontend: create static initialiser globals ahead of time 4a93a29 Merge pull request diffblue#1788 from smowton/smowton/fix/java_tcmp_nan 6ad8ffd Fix Java ternary-compare against NaN 1e0ac30 Turn get_may, set_may, etc into irep_ids b0cb1ee Merge pull request diffblue#1766 from smowton/smowton/feature/java-frontend-create-literal-globals-early a2e3af5 Merge pull request diffblue#1744 from smowton/smowton/feature/instrument_cover_per_function 22ae7aa Merge pull request diffblue#1637 from tautschnig/bswap a1a972f Merge pull request diffblue#1776 from smowton/smowton/feature/class-hierarchy-grapht ef3c598 Merge pull request diffblue#1775 from diffblue/refactor/set_classpath 45dd840 Merge pull request diffblue#1728 from romainbrenguier/refactor/split-axiom-vectors 8a27950 Java frontend: create String an Class literals earlier d95cb12 Move string literal initialisation into separate file 515ebdd CI lazy methods: scan global initialisers for global references cab7b52 C front-end: fix promotion order for types ranking lower than int c450328 Support for --16 on Visual Studio, no _WIN64 in 32-bit mode b2c4188 Do not use non-trivial system headers with --32 80b972b Use split_string in set_classpath fdb2ebc Merge pull request diffblue#1773 from smowton/smowton/feature/string-solver-ensure-class-graph-consistency a6eed7c Add class-hierarchy variant based on grapht 311af6d Coverage: fully support instrumenting one function at a time ceafd85 Java string solver: ensure base types are loaded 1e17db6 Merge pull request diffblue#1735 from cesaro/core-models 6844760 Merge pull request diffblue#1769 from smowton/smowton/fix/nondet-initialize-after-initialize a8e659c Fixed CMake linker ODR violations caused by a regression-test f66288b Internalize core models of the Java Class Library 34216f5 Refactor jar loading ed008f9 Add constructors for having memory-loaded jar files This allows the jar_file class to load from a buffer (c array) as opposed to a file 86a34c9 Merge pull request diffblue#1765 from smowton/smowton/fix/ci-lazy-methods-array-element-types 1e11f6d Add test for multiple array types in single method 5009cbb CI lazy methods: re-explore array types with different element types 857fcf9 Cleanup unused fields in string refinement 51d86f5 Adapt unit tests for splitted axiom vectors 1843e44 Split string generator axioms into separate vectors 5669d9b Java: run nondet-initialize method *after* initialization 0b5a5c3 Rename test case 3440018 Provide function name in goto_model_functiont f17e2c8 Merge pull request diffblue#1741 from smowton/smowton/feature/add_failed_symbols_per_function f65f0fd Merge pull request diffblue#1764 from smowton/smowton/feature/java-infer-opaque-type-fields-earlier dbc00a7 Add doxygen to add-failed-symbols 3788467 JBMC: add failed symbols on a per-function basis e934867 Provide a journalling symbol table to process-goto-function e86e2a0 Java: infer opaque type fields before method conversion f0f50e3 Journalling symbol table: enable nesting 58d5980 Merge pull request diffblue#1740 from smowton/smowton/feature/adjust_float_expressions_per_function c91ff69 JBMC: adjust float expressions per function eed983a JBMC: add property checks on a per-function basis db3bc99 JBMC: run convert-nondet on a per-function basis 99ea8fe JBMC: run replace-Java-nondet on function-by-function basis bfd4f50 Merge pull request diffblue#1730 from smowton/smowton/feature/remove_returns_per_function 96569c3 JBMC: remove return values on a per-function basis a7595c1 Remove returns: support running per-function fd6e195 Merge pull request diffblue#1718 from cesaro/concurrency-team-small-fixes e6fe617 Merge pull request diffblue#1705 from jgwilson42/goto-diff-tests 22afc5c Fixes wrong invocation order for static initializers 5c3997d Refectors how CBMC interprets a codet thread-block 001c1a2 ireps of type "ID_atomic_begin" and "ID_atomic_end" will now be properly displayed when the "show-symbol-table" flag is specified. d978ef9 Folder build/ ignored. bc145fd Merge pull request diffblue#1756 from romainbrenguier/tests/index-of-corrections#TG-2246 47b4ee9 Merge pull request diffblue#1725 from cesaro/exception-handlig-fixes d397d6a Merge pull request diffblue#1726 from diffblue/multi_ary_expr2c bd95317 Merge pull request diffblue#1753 from diffblue/xor_exprt 1d4af6d Merge pull request diffblue#1747 from NathanJPhillips/feature/upstream-cleanup 9c7debb Merge pull request diffblue#1750 from pkesseli/feature/sat-interrupt f11c995 Merge pull request diffblue#1749 from pkesseli/ci/remove-unapproved 981c8e0 Merge pull request diffblue#1743 from tautschnig/dump-c-fix bcb076b Correct tests for String.indexOf ef5c6f0 Merge pull request diffblue#1742 from owen-jones-diffblue/owen-jones-diffblue/small-shared-ptr 6c9f05e Fixes to exception handling behaviour 80dd48a added multi-ary xor_exprt 703e4a3 Remove unapproved C++11 header warning. bf7ed1a Merge pull request diffblue#313 from diffblue/owen-jones-diffblue/add-structured-lhs-to-value-set cc9398d Expose MiniSAT's `interrupt()` 8360233 Merge pull request diffblue#1646 from peterschrammel/list-goto-functions e4a2763 Tests for scope changes for variables and functions 8ee1956 goto-diff tests for package name changes ce3a5e9 Basic tests for java goto-diff 3bf9987 Compare access qualifiers in goto-diff f71cc7f Attach class name to method symbol 1f06d35 Merge pull request diffblue#312 from diffblue/pull-support-20180112 fda9daa Cleanup of create-and-swap to emplace e42e97a Merge commit '23666e3af35673c734c9816ffc131b6b9a379e86' into pull-support-20180112 53f1a41 Populate structured_lhs in all `entryt`s d7121f2 dump-c: fix support of use-system-headers eb5ec24 Merge pull request diffblue#1736 from hannes-steffenhagen-diffblue/develop_fix-bitfield-pretty-printing 7a0de46 Add comment suggested by @owen-jones-diffblue b741d4b Use small intrusive pointer in irep 434cc99 Merge pull request diffblue#1732 from peterschrammel/catch-sat-memout 8ae53bb Merge pull request diffblue#1733 from peterschrammel/mem-limit 574101c Add `structured_lhs` field to entryt 4f1a67a Uses alternatives to get_string in type2name when possible b46149d Merge pull request diffblue#1719 from smowton/smowton/cleanup/remove_exceptions_single_global 82a7ec6 Adds regression test for bitfield naming bug 651d8d1 Fixes use of wrong identifier when pretty printing bitfield types 638937a Merge pull request diffblue#1709 from romainbrenguier/doc/string-solver-intro 1d1be4c Move non-string-specific part of doc to solvers 549eb57 Delete trailing whitespaces db3e044 Add introduction to string solver documentation 74be7fb Merge pull request diffblue#1729 from romainbrenguier/refactor/unused-nonempty-option d101b22 Set memory limit utility ef45a1d Replace assertions by invariants 84e04a7 Catch Glucose::OutOfMemoryException 89fc48d Replace assertions by invariants 5e85701 Catch Minisat::OutOfMemoryException b8cee29 Enable list-goto-functions in clobber d902ec8 Replace cout by message stream in show-goto-functions d970673 Move show-loops in the right place in goto-diff e1227ef Enable list-goto-functions in goto diff 7e1110c Enable list-goto-functions in goto-instrument 0fb4868 Enable list-goto-functions in goto-analyzer e67abfa Remove exceptions: switch to single inflight exception global 2fabbd4 Enable list-goto-functions in JBMC 9e1705f Enable list-goto-functions in CBMC ebd8248 Add list-goto-functions command line option 2fe43a9 Add parameter to list goto functions without printing bodies 3d492fe Add documentation of return values 5a8eea5 Remove the string-non-empty option 9810f92 Drop string_non_empty field for string refinement fec16d7 expr2c now distinguishes binary and multi-ary expressions d16a918 C library: network byteorder functions 05bc9ed Implement bswap in SAT back-end 4f37035 Introduce bswap_exprt git-subtree-dir: cbmc git-subtree-split: f7602af
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This factors the coverage driver to pull apart parsing configuration and generating a coverage driver structure from actually performing the coverage, and factors the whole-program version to use a single-function version that operates under the same rules. This sets us up for an easy transition to applying coverage instrumentation per-function in JBMC.
Note I don't actually use this in JBMC yet as it does not yet use remove-exceptions in per-function mode. This PR is for reviewing the coverage alterations; I will bring in remove-exceptions and coverage-instrumentation together with the upcoming symex-driven function loading PR.