Upstream prep

rust/pact_matching_ffi/ARCHITECTURE.md

@@ -0,0 +1,156 @@
+
+# Architecture
+
+The `pact_matching_ffi` crate is built with one goal and a couple constraints.
+The goal is that it expose all of `pact_matching` for use by other languages.
+
+The constraints are:
+
+1. It should be non-invasive (meaning it doesn't require changes to
+   `pact_matching` to, for example, use FFI-friendly types which could be
+   exposed directly to C).
+2. It should introduce limited performance overhead.
+3. It should be easy to call from C.
+4. It should be easy for other languages to wrap.
+5. It should provide strong support for error handling and logging.
+6. It should preserve and document any safety guarantees or requirements. 
+
+The design of the crate is done with these goals in mind. Before getting into
+how the wrapping of `pact_matching` is performed, it's worthwhile to describe
+the mechanisms around it.
+
+## Error Handling
+
+Error handling in Rust is robust, using the `Option` and `Result` types to
+indicate when data may be missing or errors may occur. Error handling in C
+is less powerful, usually using some sentinel values in the return value of
+a function to indicate an error has occured. There is also the `errno` API
+from the C standard library, which permits reporting of a standardized set
+of error codes, but that mechanism doesn't permit custom error information,
+and would require us to create a mapping from all possible `pact_matching`
+errors into `errno` error codes.
+
+Instead, the error handling system in `pact_matching_ffi` is based around
+1) signaling of errors through sentinel values, and 2) permitting C-side
+collection of error messages through the `get_error_message` function.
+This design is based on the work of Michael F. Bryan in the
+[unofficial Rust FFI Guide][ffi_guide], which was based on the error handling
+in `libgit2`.
+
+In short: errors are collected from all FFI code, and stored in a
+thread-local variable called `LAST_ERROR`. The `get_error_message` function
+pulls the latest error message from `LAST_ERROR` and reports it to the
+user. All the mechanisms in `pact_matching_ffi/src/error` exist to handle
+this error collection and reporting.
+
+## Logging
+
+Logging is a crucial part of any application, and just because `pact_matching`
+is being called from another language is no reason to render logging info
+from it unavailable. For this reason, `pact_matching_ffi` provides a way for
+C callers to initialize a logger and direct logging output to a location of
+their choice.
+
+First, they call `logger_init`, which begins the logging setup by creating
+a log dispatcher with no sinks. Then they call `logger_attach_sink` to add
+sinks, to which any logs matching the filtering level will be sent. Finally,
+after all `logger_attach_sink` calls are done, they call `logger_apply` to
+apply the logger and complete setup.
+
+The remainder of the code in `pact_matching_ffi/src/log` is plumbing for this
+logging setup process.
+
+## The `ffi_fn` macro
+
+FFI functions have to be written with some boilerplate. First, they must be
+marked `#[no_mangle]` so their names are exported as-written for C to call.
+Second, they must be marked `pub extern` so they're exported and picked up
+by `cbindgen`, the tool we use for generating the C header which users of
+`pact_matching_ffi` will need.
+
+Then, within the functions themselves, we want to be sure that certain FFI
+information is logged, and that errors are captured and reported consistently
+and correctly.
+
+For this purpose, we use the `ffi_fn` macro, found in
+`pact_matching_ffi/src/util/ffi.rs`. This macro is inspired by a
+[macro of the same name][hyper_macro] from the `hyper` crate's C API
+(created for use of Hyper in `curl`). Our macro does considerably more work
+to capture errors and ensure complete logging, but the idea is the same. The
+macro also ensures panics are captured and packaged up like regular errors.
+This includes panics due to allocation failure, although we don't do anything
+to handle this case specifically.
+
+One thing to note is that most functions using the macro will be written
+slightly differently from normal Rust functions. If the function being
+wrapped is fallible, then a second block is needed after the function body
+block showing what to return in the case of any unexpected failure (usually
+either a null pointer of the appropriate `const`-ness and type, or a
+sentinel integer value signalling an error occurred).
+
+Here's an example of the macro in use.
+
+```rust
+ffi_fn! {
+    /// Get a mutable pointer to a newly-created default message on the heap.
+    fn message_new() -> *mut Message {
+        let message = Message::default();
+        ptr::raw_to(message)
+    } {
+        ptr::null_mut_to::<Message>()
+    }
+}
+```
+
+For destructors, or other functions which have no return value, no second
+block is required.
+
+```rust
+ffi_fn! {
+    /// Destroy the `Message` being pointed to.
+    fn message_delete(message: *mut Message) {
+        ptr::drop_raw(message);
+    }
+}
+```
+
+## Opaque Pointers
+
+The `pact_matching_ffi` crate is [_non-invasive_][non_invasive], meaning it
+doesn't involve the modification of types in `pact_matching` to be exposable
+over the FFI boundary (which would at minimum mean marking them `#[repr(C)]`).
+Instead, we expose types in almost all cases as an "opaque pointer," meaning
+C knows the _name_ of the type, but not its _contents_, and works only with
+pointers to that type.
+
+On the one hand, this is convenient, because it keeps the FFI boundary clear
+and separate. On the other hand, it means that C code has to make function
+calls to the Rust side rather than accessing fields directly, and that certain
+optimizations may be missed because of the indirection. This can be addressed
+at least in part through use of cross-language [Link Time Optimization (LTO)][lto]
+
+## Strings
+
+For the most part, to avoid issues with buffer sizing and fallible operations
+against output parameters, the `pact_matching_ffi` crate is designed, when
+returning strings, to return them as `const char *`, pointing to a
+heap-allocated string buffer which must be deleted by calling the provided
+`string_delete` function when finished.
+
+One exception is returning the error message, where the user provides a buffer
+and the size of that buffer, and the operation to fill the buffer with the
+error message may fail. This is because error handling is expected to be very
+common, so buffer reuse is ideal.
+
+Additionally, `pact_matching_ffi` returns exclusively UTF-8-encoded strings,
+and expects all strings it receives to be UTF-8 encoded.
+
+## Buffers
+
+Whenever `pact_matching_ffi` writes to a buffer, it zeroes out any excess
+capacity in the buffer for security reasons.
+
+[ffi_guide]: https://michael-f-bryan.github.io/rust-ffi-guide/errors/index.html
+[hyper_macro]: https://github.com/hyperium/hyper/blob/master/src/ffi/macros.rs
+[non_invasive]: https://www.possiblerust.com/guide/inbound-outbound-ffi#non-invasive-outbound-ffi
+[lto]: http://blog.llvm.org/2019/09/closing-gap-cross-language-lto-between.html

rust/pact_matching_ffi/README.md

@@ -5,6 +5,15 @@ This crate provides a Foreign Function Interface (FFI) to the `pact_matching` cr
 with the intent of enabling Pact's core matching mechanisms to be used by implementations
 in other languages.
 
+## Dependencies
+
+This crates requires:
+
+- `cbindgen`, a tool for automatically generating the header file needed for C users of the crate.
+- A nightly-channel version of Cargo (needed for an unstable flag used by `cbindgen` to get the macro-expanded contents of the crate source).
+
+It will additionally attempt to find and use `Doxygen` to generate C-friendly documentation (you can of course alternatively use `cargo doc` to get Rustdoc documentation).
+
 ## Building
 
 For convenience, this tool integrates with CMake, which is setup to:
@@ -47,3 +56,8 @@ $ cd build
 $ cmake ..
 $ cmake --build .
 ```
+
+## Architecture
+
+You can read about the architecture and design choices of this crate in
+[ARCHITECTURE.md](./ARCHITECTURE.md).

rust/pact_matching_ffi/examples/.gitignore

@@ -0,0 +1,3 @@
+build/
+*/build/
+

rust/pact_matching_ffi/examples/error_handling/CMakeLists.txt

@@ -0,0 +1,86 @@
+#################################################################################################
+# CMAKE VERSION
+#################################################################################################
+
+# Set the minimum to 3.15. This is arbitrary and we should probably try to
+# test everything with older CMake versions once this is all written, to
+# figure out an actual lower-bound.
+cmake_minimum_required(VERSION 3.15...3.17)
+
+# Set policies appropriately, so it knows when to warn about policy
+# violations.
+if(${CMAKE_VERSION} VERSION_LESS 3.17)
+    cmake_policy(VERSION ${CMAKE_MAJOR_VERSION}.${CMAKE_MINOR_VERSION})
+else()
+    cmake_policy(VERSION 3.17)
+endif()
+
+#################################################################################################
+# PROJECT DECLARATION
+#################################################################################################
+
+project(PMFFI_ERROR_HANDLING
+        VERSION "0.1.0"
+        DESCRIPTION "A basic example of C consumer error handling for the pact matching FFI"
+        LANGUAGES C)
+
+#################################################################################################
+# OUT OF SOURCE BUILDS
+#
+# Require out-of-source builds for this project. It keeps things much simpler
+# and cleaner.
+#################################################################################################
+
+# Set a path to the CMake config (this file)
+file(TO_CMAKE_PATH "${PROJECT_BINARY_DIR}/CMakeLists.txt" LOC_PATH)
+
+# Define the error message to potentially be printed.
+set(OOS_MSG "\
+You cannot build in a source directory (or any directory with a CMakeLists.txt file). \
+Please make a build subdirectory. \
+Feel free to remove CMakeCache.txt and CMakeFiles.
+")
+
+# If that file path exists, we're doing an in-source build, so we should exit with a fatal
+# error complaining only out-of-source builds are supported.
+if(EXISTS ${LOC_PATH})
+    message(FATAL_ERROR ${OOS_MSG})
+endif()
+
+#################################################################################################
+# DEFAULT BUILD TYPE
+#
+# Make release the default build type
+#################################################################################################
+
+set(default_build_type "Release")
+if(NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES)
+  message(STATUS "Setting build type to '${default_build_type}' as none was specified.")
+  set(CMAKE_BUILD_TYPE "${default_build_type}" CACHE STRING "Choose the type of build." FORCE)
+  # Set the possible values of build type
+  set_property(CACHE CMAKE_BUILD_TYPE PROPERTY STRINGS "Debug" "Release")
+endif()
+
+#################################################################################################
+# FIND PACT MATCHING FFI
+#
+# This ensures CMake can find the pact matching FFI library file
+#################################################################################################
+
+# Sets the search path to the location of the package config
+get_filename_component(REAL_ROOT "${CMAKE_CURRENT_SOURCE_DIR}/../.." ABSOLUTE)
+set(SEARCH_PATH "${REAL_ROOT}/build/install/lib/cmake")
+
+# Find the pact matching FFI package and load the imported target
+find_package(PactMatchingFfi REQUIRED CONFIG PATHS ${SEARCH_PATH})
+
+#################################################################################################
+# BUILD
+#################################################################################################
+
+# Define the executable
+add_executable(example src/main.c)
+
+# Link to pact matching FFI
+target_link_libraries(example PRIVATE PactMatchingFfi)
+

rust/pact_matching_ffi/example/src/main.c → rust/pact_matching_ffi/examples/error_handling/src/main.c

@@ -6,36 +6,42 @@
 #define ERROR_MSG_LEN 256
 
 int main(void) {
-    logger_init();
-    logger_attach_sink("stdout", LevelFilter_Trace);
-    logger_apply();
+    /*=======================================================================
+     * Simple empty message creation.
+     *---------------------------------------------------------------------*/
 
     Message *msg = message_new();
-    int error = message_delete(msg);
 
-    if (error == EXIT_FAILURE) {
+    if (msg == NULL) {
         char error_msg[ERROR_MSG_LEN];
-
         int error = get_error_message(error_msg, ERROR_MSG_LEN);
-
         printf("%s\n", error_msg);
-
         return EXIT_FAILURE;
     }
 
+    message_delete(msg);
+
+
+    /*=======================================================================
+     * Creating a message from a JSON string.
+     *---------------------------------------------------------------------*/
+
     char *json_str = "{\
         \"description\": \"String\",\
         \"providerState\": \"provider state\",\
         \"matchingRules\": {}\
     }";
-    Message *msg_json = message_from_json(0, json_str, PactSpecification_V3);
+    Message *msg_json = message_new_from_json(0, json_str, PactSpecification_V3);
 
-    if (NULL == msg_json) {
+    if (msg_json == NULL) {
         char error_msg[ERROR_MSG_LEN];
         int error = get_error_message(error_msg, ERROR_MSG_LEN);
         printf("%s\n", error_msg);
         return EXIT_FAILURE;
     }
 
+    message_delete(msg_json);
+
     return EXIT_SUCCESS;
 }
+