Merge branch 'devel' into feature/respect-dlpaths-order

.github/workflows/mac-builds.yml

@@ -4,15 +4,10 @@ on: [push, pull_request]
 
 jobs:
   build:
-    # macos-12 updated to a toolchain that crashes when linking the
-    # test binary. This seems to be a known bug in that version,
-    # and will eventually get fixed in an update. After that, we can go
-    # back to newer macos images.
-    runs-on: macos-11
+    runs-on: macos-12
     strategy:
       matrix:
         cxx:
-          - g++-11
           - clang++
         build_type: [Debug, Release]
         std: [14, 17]
@@ -29,8 +24,6 @@ jobs:
       env:
         CXX: ${{matrix.cxx}}
         CXXFLAGS: ${{matrix.cxxflags}}
-      # Note: $GITHUB_WORKSPACE is distinct from ${{runner.workspace}}.
-      #       This is important
       run: |
         cmake -Bbuild -H$GITHUB_WORKSPACE \
               -DCMAKE_BUILD_TYPE=${{matrix.build_type}} \

.gitignore

@@ -25,7 +25,7 @@ Build
 cmake-build-*
 benchmark-dir
 .conan/test_package/build
-.conan/test_package/CMakeUserPresets.json
+**/CMakeUserPresets.json
 bazel-*
 MODULE.bazel.lock
 build-fuzzers
@@ -37,3 +37,4 @@ msvc-sln*
 docs/doxygen
 *.cache
 compile_commands.json
+**/*.unapproved.txt

MODULE.bazel

@@ -1,3 +1,3 @@
 module(name = "catch2")
 
-bazel_dep(name = "bazel_skylib", version = "1.5.0")
+bazel_dep(name = "bazel_skylib", version = "1.7.1")

WORKSPACE.bazel

@@ -4,10 +4,10 @@ load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
 
 http_archive(
     name = "bazel_skylib",
-    sha256 = "cd55a062e763b9349921f0f5db8c3933288dc8ba4f76dd9416aac68acee3cb94",
+    sha256 = "bc283cdfcd526a52c3201279cda4bc298652efa898b10b4db0837dc51652756f",
     urls = [
-        "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.5.0/bazel-skylib-1.5.0.tar.gz",
-        "https://github.com/bazelbuild/bazel-skylib/releases/download/1.5.0/bazel-skylib-1.5.0.tar.gz",
+        "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.7.1/bazel-skylib-1.7.1.tar.gz",
+        "https://github.com/bazelbuild/bazel-skylib/releases/download/1.7.1/bazel-skylib-1.7.1.tar.gz",
     ],
 )
 

docs/assertions.md

@@ -110,7 +110,7 @@ Expects that an exception is thrown that, when converted to a string, matches th
 
 e.g.
 ```cpp
-REQUIRE_THROWS_WITH( openThePodBayDoors(), Contains( "afraid" ) && Contains( "can't do that" ) );
+REQUIRE_THROWS_WITH( openThePodBayDoors(), ContainsSubstring( "afraid" ) && ContainsSubstring( "can't do that" ) );
 REQUIRE_THROWS_WITH( dismantleHal(), "My mind is going" );
 ```
 

docs/cmake-integration.md

@@ -8,6 +8,7 @@
 [`CATCH_CONFIG_*` customization options in CMake](#catch_config_-customization-options-in-cmake)<br>
 [Installing Catch2 from git repository](#installing-catch2-from-git-repository)<br>
 [Installing Catch2 from vcpkg](#installing-catch2-from-vcpkg)<br>
+[Installing Catch2 from Bazel](#installing-catch2-from-bazel)<br>
 
 Because we use CMake to build Catch2, we also provide a couple of
 integration points for our users.

docs/deprecations.md

@@ -35,6 +35,19 @@ being aborted (when using `--abort` or `--abortx`). It is however
 **NOT** invoked for test cases that are [explicitly skipped using the `SKIP`
 macro](skipping-passing-failing.md#top).
 
+
+### Non-const function for `TEST_CASE_METHOD`
+
+> Deprecated in Catch2 vX.Y.Z
+
+Currently, the member function generated for `TEST_CASE_METHOD` is
+not `const` qualified. In the future, the generated member function will
+be `const` qualified, just as `TEST_CASE_PERSISTENT_FIXTURE` does.
+
+If you are mutating the fixture instance from within the test case, and
+want to keep doing so in the future, mark the mutated members as `mutable`.
+
+
 ---
 
 [Home](Readme.md#top)

docs/list-of-examples.md

@@ -8,6 +8,7 @@
 - Assertion: [REQUIRE, CHECK](../examples/030-Asn-Require-Check.cpp)
 - Fixture: [Sections](../examples/100-Fix-Section.cpp)
 - Fixture: [Class-based fixtures](../examples/110-Fix-ClassFixture.cpp)
+- Fixture: [Persistent fixtures](../examples/111-Fix-PersistentFixture.cpp)
 - BDD: [SCENARIO, GIVEN, WHEN, THEN](../examples/120-Bdd-ScenarioGivenWhenThen.cpp)
 - Listener: [Listeners](../examples/210-Evt-EventListeners.cpp)
 - Configuration: [Provide your own output streams](../examples/231-Cfg-OutputStreams.cpp)

docs/matchers.md

@@ -210,15 +210,36 @@ The other miscellaneous matcher utility is exception matching.
 
 #### Matching exceptions
 
-Catch2 provides a utility macro for asserting that an expression
-throws exception of specific type, and that the exception has desired
-properties. The macro is `REQUIRE_THROWS_MATCHES(expr, ExceptionType, Matcher)`.
+Because exceptions are a bit special, Catch2 has a separate macro for them.
+
+
+The basic form is
+
+```
+REQUIRE_THROWS_MATCHES(expr, ExceptionType, Matcher)
+```
+
+and it checks that the `expr` throws an exception, that exception is derived
+from the `ExceptionType` type, and then `Matcher::match` is called on
+the caught exception.
 
 > `REQUIRE_THROWS_MATCHES` macro lives in `catch2/matchers/catch_matchers.hpp`
 
+For one-off checks you can use the `Predicate` matcher above, e.g.
 
-Catch2 currently provides two matchers for exceptions.
-These are:
+```cpp
+REQUIRE_THROWS_MATCHES(parse(...),
+                       parse_error,
+                       Predicate<parse_error>([] (parse_error const& err) -> bool { return err.line() == 1; })
+);
+```
+
+but if you intend to thoroughly test your error reporting, I recommend
+defining a specialized matcher.
+
+
+Catch2 also provides 2 built-in matchers for checking the error message
+inside an exception (it must be derived from `std::exception`):
 * `Message(std::string message)`.
 * `MessageMatches(Matcher matcher)`.
 
@@ -236,10 +257,7 @@ REQUIRE_THROWS_MATCHES(throwsDerivedException(),  DerivedException,  Message("De
 REQUIRE_THROWS_MATCHES(throwsDerivedException(),  DerivedException,  MessageMatches(StartsWith("DerivedException")));
 ```
 
-Note that `DerivedException` in the example above has to derive from
-`std::exception` for the example to work.
-
-> the exception message matcher lives in `catch2/matchers/catch_matchers_exception.hpp`
+> the exception message matchers live in `catch2/matchers/catch_matchers_exception.hpp`
 
 
 ### Generic range Matchers

docs/other-macros.md

@@ -93,30 +93,6 @@ TEST_CASE("STATIC_CHECK showcase", "[traits]") {
 
 ## Test case related macros
 
-* `METHOD_AS_TEST_CASE`
-
-`METHOD_AS_TEST_CASE( member-function-pointer, description )` lets you
-register a member function of a class as a Catch2 test case. The class
-will be separately instantiated for each method registered in this way.
-
-```cpp
-class TestClass {
-    std::string s;
-
-public:
-    TestClass()
-        :s( "hello" )
-    {}
-
-    void testCase() {
-        REQUIRE( s == "hello" );
-    }
-};
-
-
-METHOD_AS_TEST_CASE( TestClass::testCase, "Use class's method as a test case", "[class]" )
-```
-
 * `REGISTER_TEST_CASE`
 
 `REGISTER_TEST_CASE( function, description )` let's you register

docs/test-cases-and-sections.md

@@ -48,7 +48,7 @@ For more detail on command line selection see [the command line docs](command-li
 Tag names are not case sensitive and can contain any ASCII characters.
 This means that tags `[tag with spaces]` and `[I said "good day"]`
 are both allowed tags and can be filtered on. However, escapes are not
-supported however and `[\]]` is not a valid tag.
+supported and `[\]]` is not a valid tag.
 
 The same tag can be specified multiple times for a single test case,
 but only one of the instances of identical tags will be kept. Which one

docs/test-fixtures.md

@@ -1,9 +1,30 @@
 <a id="top"></a>
 # Test fixtures
 
-## Defining test fixtures
+**Contents**<br>
+[Non-Templated test fixtures](#non-templated-test-fixtures)<br>
+[Templated test fixtures](#templated-test-fixtures)<br>
+[Signature-based parameterised test fixtures](#signature-based-parametrised-test-fixtures)<br>
+[Template fixtures with types specified in template type lists](#template-fixtures-with-types-specified-in-template-type-lists)<br>
 
-Although Catch allows you to group tests together as [sections within a test case](test-cases-and-sections.md), it can still be convenient, sometimes, to group them using a more traditional test fixture. Catch fully supports this too. You define the test fixture as a simple structure:
+## Non-Templated test fixtures
+
+Although Catch2 allows you to group tests together as 
+[sections within a test case](test-cases-and-sections.md), it can still 
+be convenient, sometimes, to group them using a more traditional test. 
+Catch2 fully supports this too with 3 different macros for 
+non-templated test fixtures. They are: 
+
+| Macro    | Description |
+|----------|-------------|
+|1. `TEST_CASE_METHOD(className, ...)`| Creates a uniquely named class which inherits from the class specified by `className`. The test function will be a member of this derived class. An instance of the derived class will be created for every partial run of the test case. |
+|2. `METHOD_AS_TEST_CASE(member-function, ...)`| Uses `member-function` as the test function. An instance of the class will be created for each partial run of the test case. |
+|3. `TEST_CASE_PERSISTENT_FIXTURE(className, ...)`| Creates a uniquely named class which inherits from the class specified by `className`. The test function will be a member of this derived class. An instance of the derived class will be created at the start of the test run. That instance will be destroyed once the entire test case has ended. |
+
+### 1. `TEST_CASE_METHOD`
+
+
+You define a `TEST_CASE_METHOD` test fixture as a simple structure:
 
 ```c++
 class UniqueTestsFixture {
@@ -30,8 +51,116 @@ class UniqueTestsFixture {
  }
 ```
 
-The two test cases here will create uniquely-named derived classes of UniqueTestsFixture and thus can access the `getID()` protected method and `conn` member variables. This ensures that both the test cases are able to create a DBConnection using the same method (DRY principle) and that any ID's created are unique such that the order that tests are executed does not matter.
+The two test cases here will create uniquely-named derived classes of 
+UniqueTestsFixture and thus can access the `getID()` protected method 
+and `conn` member variables. This ensures that both the test cases 
+are able to create a DBConnection using the same method 
+(DRY principle) and that any ID's created are unique such that the 
+order that tests are executed does not matter. 
+
+### 2. `METHOD_AS_TEST_CASE`
+
+`METHOD_AS_TEST_CASE` lets you register a member function of a class 
+as a Catch2 test case. The class will be separately instantiated 
+for each method registered in this way.
+
+```cpp
+class TestClass {
+    std::string s;
+
+public:
+    TestClass()
+        :s( "hello" )
+    {}
+
+    void testCase() {
+        REQUIRE( s == "hello" );
+    }
+};
+
+
+METHOD_AS_TEST_CASE( TestClass::testCase, "Use class's method as a test case", "[class]" )
+```
+
+This type of fixture is similar to [TEST_CASE_METHOD](#1-test_case_method) except in this 
+case it will directly use the provided class to create an object rather than a derived 
+class.
+
+### 3. `TEST_CASE_PERSISTENT_FIXTURE`
+
+> [Introduced](https://github.com/catchorg/Catch2/pull/2885) in Catch2 X.Y.Z
+
+`TEST_CASE_PERSISTENT_FIXTURE` behaves in the same way as
+[TEST_CASE_METHOD](#1-test_case_method) except that there will only be
+one instance created throughout the entire run of a test case. To 
+demonstrate this have a look at the following example:
+
+```cpp
+class ClassWithExpensiveSetup {
+public:
+    ClassWithExpensiveSetup() {
+        // expensive construction
+        std::this_thread::sleep_for( std::chrono::seconds( 2 ) );
+    }
+
+    ~ClassWithExpensiveSetup() noexcept {
+        // expensive destruction
+        std::this_thread::sleep_for( std::chrono::seconds( 1 ) );
+    }
+
+    int getInt() const { return 42; }
+};
+
+struct MyFixture {
+    mutable int myInt = 0;
+    ClassWithExpensiveSetup expensive;
+};
+
+TEST_CASE_PERSISTENT_FIXTURE( MyFixture, "Tests with MyFixture" ) {
+
+    const int val = myInt++;
+
+    SECTION( "First partial run" ) {
+        const auto otherValue = expensive.getInt();
+        REQUIRE( val == 0 );
+        REQUIRE( otherValue == 42 );
+    }
+
+    SECTION( "Second partial run" ) { REQUIRE( val == 1 ); }
+}
+```
+
+This example demonstates two possible use-cases of this fixture type:
+1. Improve test run times by reducing the amount of expensive and 
+redundant setup and tear-down required.
+2. Reusing results from the previous partial run, in the current
+partial run.
+
+This test case will be executed twice as there are two leaf sections.
+On the first run `val` will be `0` and on the second run `val` will be 
+`1`. This demonstrates that we were able to use the results of the
+previous partial run in subsequent partial runs.
+
+Additionally, we are simulating an expensive object using 
+`std::this_thread::sleep_for`, but real world use-cases could be:
+1. Creating a D3D12/Vulkan device
+2. Connecting to a database
+3. Loading a file.
+
+The fixture object (`MyFixture`) will be constructed just before the
+test case begins, and it will be destroyed just after the test case 
+ends. Therefore, this expensive object will only be created and 
+destroyed once during the execution of this test case. If we had used 
+`TEST_CASE_METHOD`, `MyFixture` would have been created and destroyed 
+twice during the execution of this test case.
+
+NOTE: The member function which runs the test case is `const`. Therefore 
+if you want to mutate any member of the fixture it must be marked as
+`mutable` as shown in this example. This is to make it clear that
+the initial state of the fixture is intended to mutate during the
+execution of the test case.
 
+## Templated test fixtures
 
 Catch2 also provides `TEMPLATE_TEST_CASE_METHOD` and
 `TEMPLATE_PRODUCT_TEST_CASE_METHOD` that can be used together
@@ -93,7 +222,7 @@ _While there is an upper limit on the number of types you can specify
 in single `TEMPLATE_TEST_CASE_METHOD` or `TEMPLATE_PRODUCT_TEST_CASE_METHOD`,
 the limit is very high and should not be encountered in practice._
 
-## Signature-based parametrised test fixtures
+## Signature-based parameterised test fixtures
 
 > [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch2 2.8.0.