[Serde reflection] improve tracing

README.md

@@ -54,7 +54,7 @@ One difficulty often associated with Serde is that small modifications in Rust m
 
 ### Language Interoperability
 
-The data formats extracted by `serde-reflection` also serve as basis for code generation with the tool [`serde-generate`](serde-generate). Specifically, `serde-generate` takes a registry of Serde data formats as input and translates them into type definitions for other programming languages such as Python and C++.
+The data formats extracted by `serde-reflection` also serve as basis for code generation with the library and tool [`serde-generate`](serde-generate).
 
 For instance, the definition of `Foo` above translates into C++ as follows: (omitting methods)
 ```
@@ -67,9 +67,19 @@ struct Foo {
     std::variant<A, B, C> value;
 };
 ```
-
-To provide (de)serialization, the code generated by `serde-generate` is completed by runtime libraries in each target language and for each supported binary encoding.
-
+To provide (de)serialization, the code generated by `serde-generate` is completed by
+runtime libraries in each target language and for each supported binary encoding.
+
+Currently, `serde-generate` generates type definitions and supports [Bincode](https://docs.rs/bincode/1.3.1/bincode/) and
+ [BCS](https://github.com/diem/bcs) serialization in the following programming languages:
+
+* C++
+* Java
+* Python
+* Rust
+* Go
+* C#
+* Typescript (in progress)
 
 ## Benefits
 

serde-generate/Cargo.toml

@@ -16,24 +16,24 @@ exclude = [
 ]
 
 [dependencies]
-heck = "0.3.1"
-include_dir = "0.6"
+heck = "0.3.2"
+include_dir = "0.6.0"
 maplit = "1.0.2"
-serde = { version = "1.0.116", features = ["derive"] }
-serde_bytes = "0.11.3"
-serde_yaml = "0.8"
-structopt = "0.3.12"
-textwrap = "0.13.3"
+serde = { version = "1.0.126", features = ["derive"] }
+serde_bytes = "0.11.5"
+serde_yaml = "0.8.17"
+structopt = "0.3.21"
+textwrap = "0.13.4"
 
-serde-reflection = { path = "../serde-reflection", version = "0.3.0" }
-bincode = { version = "1.3.1" }
-bcs = { version = "0.1.1" }
+serde-reflection = { path = "../serde-reflection", version = "0.3.3" }
+bincode = "1.3.3"
+bcs = "0.1.3"
 
 [dev-dependencies]
-lazy_static = "1"
-tempfile = "3.1"
-hex = "0.4.2"
-which = "4.0.2"
+lazy_static = "1.4.0"
+tempfile = "3.2.0"
+hex = "0.4.3"
+which = "4.1.0"
 
 [[bin]]
 name = "serdegen"

serde-generate/README.md

@@ -7,11 +7,13 @@
 [![License](https://img.shields.io/badge/license-MIT-green.svg)](../LICENSE-MIT)
 
 This crate aims to compile the data formats extracted from Rust by [`serde_reflection`](https://crates.io/crates/serde_reflection)
-into type definitions for other programming languages.
+into type definitions and (de)serialization methods for other programming languages.
+
+It can be used as a library or as a command-line tool (see `serdegen` below).
 
 ### Supported Languages
 
-The following target languages are currently supported:
+The following programming languages are fully supported as target languages:
 
 * C++ 17
 * Java 8
@@ -20,7 +22,7 @@ The following target languages are currently supported:
 * Go >= 1.14
 * C# (NetCoreApp >= 2.1)
 
-### Work in progress
+The following languages are partially supported and still considered under development:
 
 * TypeScript > 3.2 (make sure to enable `esnext.BigInt` and `dom` at tsconfig.json -> lib)
 
@@ -31,15 +33,16 @@ provides (de)serialization in a particular [Serde encoding format](https://serde
 
 This crate provides easy-to-deploy runtime libraries for the following binary formats, in all supported languages:
 
-* [Bincode](https://docs.rs/bincode/1.3.1/bincode/),
-* [BCS](https://github.com/diem/bcs) (short for "Binary Canonical Serialization" -- formerly known as "LCS" or "Libra Canonical Serialization").
+* [Bincode](https://docs.rs/bincode/1.3.1/bincode/) (default configuration only),
+* [BCS](https://github.com/diem/bcs) (short for Binary Canonical Serialization, the main format used
+  in the [Diem blockchain](https://github.com/diem/diem)).
 
 ### Quick Start with Python and Bincode
 
 In the following example, we transfer a `Test` value from Rust to Python using [`bincode`](https://docs.rs/bincode/1.3.1/bincode/).
 ```rust
 use serde::{Deserialize, Serialize};
-use serde_reflection::{Registry, Samples, Tracer, TracerConfig};
+use serde_reflection::{Registry, Tracer, TracerConfig};
 use std::io::Write;
 
 #[derive(Serialize, Deserialize)]
@@ -48,9 +51,9 @@ struct Test {
     b: (u32, u32),
 }
 
-// Obtain the Serde format of `Test`. (In practice, formats are more often read from a file.)
+// Obtain the Serde format of `Test`. (In practice, formats are more likely read from a file.)
 let mut tracer = Tracer::new(TracerConfig::default());
-tracer.trace_type::<Test>(&Samples::new()).unwrap();
+tracer.trace_simple_type::<Test>().unwrap();
 let registry = tracer.registry().unwrap();
 
 // Create Python class definitions.

serde-generate/src/lib.rs

@@ -2,11 +2,13 @@
 // SPDX-License-Identifier: MIT OR Apache-2.0
 
 //! This crate aims to compile the data formats extracted from Rust by [`serde_reflection`](https://crates.io/crates/serde_reflection)
-//! into type definitions for other programming languages.
+//! into type definitions and (de)serialization methods for other programming languages.
+//!
+//! It can be used as a library or as a command-line tool (see `serdegen` below).
 //!
 //! ## Supported Languages
 //!
-//! The following target languages are currently supported:
+//! The following programming languages are fully supported as target languages:
 //!
 //! * C++ 17
 //! * Java 8
@@ -15,7 +17,7 @@
 //! * Go >= 1.14
 //! * C# (NetCoreApp >= 2.1)
 //!
-//! ## Work in progress
+//! The following languages are partially supported and still considered under development:
 //!
 //! * TypeScript > 3.2 (make sure to enable `esnext.BigInt` and `dom` at tsconfig.json -> lib)
 //!
@@ -26,15 +28,16 @@
 //!
 //! This crate provides easy-to-deploy runtime libraries for the following binary formats, in all supported languages:
 //!
-//! * [Bincode](https://docs.rs/bincode/1.3.1/bincode/),
-//! * [BCS](https://github.com/diem/bcs) (short for "Binary Canonical Serialization" -- formerly known as "LCS" or "Libra Canonical Serialization").
+//! * [Bincode](https://docs.rs/bincode/1.3.1/bincode/) (default configuration only),
+//! * [BCS](https://github.com/diem/bcs) (short for Binary Canonical Serialization, the main format used
+//!   in the [Diem blockchain](https://github.com/diem/diem)).
 //!
 //! ## Quick Start with Python and Bincode
 //!
 //! In the following example, we transfer a `Test` value from Rust to Python using [`bincode`](https://docs.rs/bincode/1.3.1/bincode/).
 //! ```
 //! use serde::{Deserialize, Serialize};
-//! use serde_reflection::{Registry, Samples, Tracer, TracerConfig};
+//! use serde_reflection::{Registry, Tracer, TracerConfig};
 //! use std::io::Write;
 //!
 //! #[derive(Serialize, Deserialize)]
@@ -44,9 +47,9 @@
 //! }
 //!
 //! # fn main() -> Result<(), std::io::Error> {
-//! // Obtain the Serde format of `Test`. (In practice, formats are more often read from a file.)
+//! // Obtain the Serde format of `Test`. (In practice, formats are more likely read from a file.)
 //! let mut tracer = Tracer::new(TracerConfig::default());
-//! tracer.trace_type::<Test>(&Samples::new()).unwrap();
+//! tracer.trace_simple_type::<Test>().unwrap();
 //! let registry = tracer.registry().unwrap();
 //!
 //! // Create Python class definitions.
@@ -111,14 +114,15 @@
 
 /// Dependency analysis and topological sort for Serde formats.
 pub mod analyzer;
+/// Utility function to generate indented text
+pub mod indent;
+
 /// Support for code-generation in C++
 pub mod cpp;
 /// Support for code-generation in C#
 pub mod csharp;
-/// Utility function to generate indented text
-pub mod golang;
 /// Support for code-generation in Go
-pub mod indent;
+pub mod golang;
 /// Support for code-generation in Java
 pub mod java;
 /// Support for code-generation in Python 3

serde-generate/src/test_utils.rs

@@ -605,7 +605,7 @@ fn test_get_sample_values() {
 fn test_get_simple_registry() {
     let registry = get_simple_registry().unwrap();
     assert_eq!(
-        serde_yaml::to_string(&registry).unwrap() + "\n",
+        serde_yaml::to_string(&registry).unwrap(),
         r#"---
 Choice:
   ENUM:

serde-name/Cargo.toml

@@ -16,8 +16,8 @@ exclude = [
 ]
 
 [dependencies]
-thiserror = "1.0.22"
-serde = { version = "1.0", features = ["derive"] }
+thiserror = "1.0.25"
+serde = { version = "1.0.126", features = ["derive"] }
 
 [dev-dependencies]
-serde-reflection = { path = "../serde-reflection", version = "0.3.0" }
+serde-reflection = { path = "../serde-reflection", version = "0.3.3" }

serde-reflection/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "serde-reflection"
-version = "0.3.2"
+version = "0.3.3"
 description = "Extract representations of Serde data formats"
 documentation = "https://docs.rs/serde-reflection"
 repository = "https://github.com/novifinancial/serde-reflection"
@@ -16,11 +16,12 @@ exclude = [
 ]
 
 [dependencies]
-thiserror = "1.0.22"
-serde = { version = "1.0", features = ["derive"] }
+thiserror = "1.0.25"
+serde = { version = "1.0.126", features = ["derive"] }
+once_cell = "1.7.2"
 
 [dev-dependencies]
-bincode = "1.3.1"
-serde_json = "1.0"
-serde_yaml = "0.8"
-serde_bytes = "0.11.3"
+bincode = "1.3.3"
+serde_json = "1.0.64"
+serde_yaml = "0.8.17"
+serde_bytes = "0.11.5"

serde-reflection/README.md

@@ -6,15 +6,22 @@
 [![License](https://img.shields.io/badge/license-Apache-green.svg)](../LICENSE-APACHE)
 [![License](https://img.shields.io/badge/license-MIT-green.svg)](../LICENSE-MIT)
 
-This crate provides a way to extract format descriptions for Rust containers
-that implement the Serialize and/or Deserialize trait(s) of Serde.
+This crate provides a way to extract format descriptions for Rust containers that
+implement the Serialize and/or Deserialize trait(s) of Serde.
+
+Format descriptions are useful in several ways:
+* Stored under version control, formats can be tested to prevent unintended modifications
+of binary serialization formats (e.g. by changing variant order).
+* Formats can be passed to [`serde-generate`](https://docs.rs/serde-generate/0.19.2/serde_generate/)
+in order to generate class definitions and provide Serde-compatible binary
+serialization in other languages (C++, python, Java, etc).
 
 ## Quick Start
 
-Very often, Serde traits are only implemented using Serde derive macros.
-In this case, simply
-* call `trace_type` on the desired top-level definitions, then
-* add a call to `trace_type` for each `enum` type. (This will fix any `MissingVariants` error.)
+Very often, Serde traits are simply implemented using Serde derive macros. In this case,
+you may obtain format descriptions as follows:
+* call `trace_simple_type` on the desired top-level container definition(s), then
+* add a call to `trace_simple_type` for each `enum` type. (This will fix any `MissingVariants` error.)
 
 ```rust
 #[derive(Deserialize)]
@@ -31,17 +38,16 @@ enum Choice { A, B, C }
 
 // Start the tracing session.
 let mut tracer = Tracer::new(TracerConfig::default());
-let samples = Samples::new();
 
 // Trace the desired top-level type(s).
-tracer.trace_type::<Foo>(&samples)?;
+tracer.trace_simple_type::<Foo>()?;
 
 // Also trace each enum type separately to fix any `MissingVariants` error.
-tracer.trace_type::<Choice>(&samples)?;
+tracer.trace_simple_type::<Choice>()?;
 
 // Obtain the registry of Serde formats and serialize it in YAML (for instance).
 let registry = tracer.registry()?;
-let data = serde_yaml::to_string(&registry).unwrap() + "\n";
+let data = serde_yaml::to_string(&registry).unwrap();
 assert_eq!(&data, r#"---
 Bar:
   NEWTYPESTRUCT: U64
@@ -124,7 +130,7 @@ match registry.get("Person").unwrap() {
 };
 
 // Export the registry in YAML.
-let data = serde_yaml::to_string(&registry).unwrap() + "\n";
+let data = serde_yaml::to_string(&registry).unwrap();
 assert_eq!(&data, r#"---
 Name:
   NEWTYPESTRUCT: STR