Docs: Integrating with other cddl-codegen gen'd libs

docs/docs/command_line_flags.mdx

@@ -149,7 +149,7 @@ cddl-codegen --input=example --output=export --common-import-override=cml_core
 
 <br/><br/>
 
-:::info `--package-json`
+:::info `--wasm-cbor-json-api-macro`
 If it is passed in, it will call the supplied externally defined macro on each exported type, instead of manually exporting the functions for to/from CBOR bytes + to/from JSON API.
 
 The external macro is assumed to exist at the specified path and will be imported if there are module prefixes.

docs/docs/comment_dsl.mdx

@@ -98,7 +98,7 @@ This can also be useful when you have a spec that is either very awkward to use
 This can also be useful when you have a spec that is either very awkward to use (so you hand-write or hand-modify after generation) in some type so you don't generate those types and instead manually merge those hand-written/hand-modified structs back in to the code afterwards. This saves you from having to manually remove all code that is generated regarding `Foo` first before merging in your own.
 
 
-#### _CDDL_CODEGEN_RAW_BYTES_TYPE_
+## _CDDL_CODEGEN_RAW_BYTES_TYPE_
 
 Allows encoding as `bytes` but imposing hand-written constraints defined elsewhere.
 ```cddl

docs/docs/integration-other.mdx

@@ -0,0 +1,32 @@
+---
+sidebar_position: 7
+---
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+
+# Integration with other cddl-codegen libraries
+
+This guide is written in general for integrating with other libraries generated by cddl-codegen, but in particular references CML (cardano-multiplatform-lib) for examples. Most things referencing CML will be relevant to other common cddl-codegen generated libraries used as dependencies.
+
+## Common cddl-codegen traits
+
+When generating a library that has as a dependency another cddl-codegen-generated library you can share the common cddl-codegen types/traits like `Deserialize`, `RawBytesEncoding`, etc. Remember to pass in `--common-import-override` tag. For CML we pass in `--common-import-override=cml_core`. This is where all the common cddl-codegen traits are located so we can avoid having duplicate incompatible traits in other libraries.
+
+## CML macros
+
+In CML we have macros for implementing WASM conversions and JSON/bytes. We pass in `--wasm-cbor-json-api-macro=cml_core_wasm::impl_wasm_cbor_json_api` and `--wasm-conversions-macro=cml_core_wasm::impl_wasm_conversions` which are both located in `cml_core_wasm`. This drastically reduces WASM wrapper boilerplate.
+
+## Externally defined types
+
+### `_CDDL_CODEGEN_EXTERN_TYPE_` vs `_CDDL_CODEGEN_RAW_BYTES_TYPE_`
+
+There are two ways to have explicitly externally-defined types in cddl-codegen: `_CDDL_CODEGEN_EXTERN_TYPE_` and `_CDDL_CODEGEN_RAW_BYTES_TYPE_`. It is important to choose the appropriate one. If the type was defined originally as `_CDDL_CODEGEN_RAW_BYTES_TYPE_` in CML (or whatever library) then it is important to define it using this so it will be encoded correctly. If the type was either defined using `_CDDL_CODEGEN_EXTERN_TYPE_` (hand-written) or was explicitly defined normally in the dependency lib (e.g. CML) then use `_CDDL_CODEGEN_EXTERN_TYPE_`.
+
+### Import pathing
+
+In order to make imports easier it's recommended to make a directory corresponding to the dependency and put the `_CDDL_CODEGEN_RAW_BYTES_TYPE_` and `_CDDL_CODEGEN_EXTERN_TYPE_` external types inside of there and then later delete the output directories containing those modules. For an example see the `cml_chain` directory inside of the [`specs/multiera`](https://github.com/dcSpark/cardano-multiplatform-lib/tree/develop/specs/multiera).
+
+### Non-black-box types
+
+Another important detail, demonstrated in the above `multiera` CDDL spec, is that when using external types that aren't 100% self-contained (i.e. can't be treated as a black box that implements `Serialize` + `Deserialize`, nor as CBOR bytes implementing `RawBytesEncoding`) like `uint` aliases should be explicitly defined and then removed afterwards. Using the above directory/pathing tip makes this trivial to remove after.

src/cli.rs

@@ -67,13 +67,8 @@ pub struct Cli {
     /// Location override for default common types (error, serialization, etc)
     /// This is useful for integrating into an exisitng project that is based on
     /// these types.
-    #[clap(
-        long,
-        value_parser,
-        value_name = "COMMON_IMPORT_OVERRIDE",
-        default_value = "crate"
-    )]
-    pub common_import_override: String,
+    #[clap(long, value_parser, value_name = "COMMON_IMPORT_OVERRIDE")]
+    common_import_override: Option<String>,
 
     /// An external macro to be called instead of manually emitting functions for
     /// conversions to/from CBOR bytes or JSON.
@@ -97,4 +92,19 @@ impl Cli {
     pub fn lib_name_code(&self) -> String {
         self.lib_name.replace('-', "_")
     }
+
+    /// If someone override the common imports, we don't want to export them
+    pub fn export_static_files(&self) -> bool {
+        self.common_import_override.is_none()
+    }
+
+    pub fn common_import_rust(&self) -> &str {
+        self.common_import_override.as_deref().unwrap_or("crate")
+    }
+
+    pub fn common_import_wasm(&self) -> String {
+        self.common_import_override
+            .clone()
+            .unwrap_or_else(|| self.lib_name_code())
+    }
 }

src/generation.rs

@@ -745,11 +745,14 @@ impl GenerationScope {
         }
 
         // declare modules (root lib specific)
-        self.rust_lib().raw("pub mod error;");
+        if cli.export_static_files() {
+            self.rust_lib().raw("pub mod error;");
+            if cli.preserve_encodings {
+                self.rust_lib().raw("pub mod ordered_hash_map;");
+            }
+        }
         if cli.preserve_encodings {
-            self.rust_lib()
-                .raw("pub mod ordered_hash_map;")
-                .raw("extern crate derivative;");
+            self.rust_lib().raw("extern crate derivative;");
         }
         let scope_names = self
             .rust_scopes
@@ -778,17 +781,17 @@ impl GenerationScope {
             // needed if there's any params that can fail
             content
                 .push_import("std::convert", "TryFrom", None)
-                .push_import(format!("{}::error", cli.common_import_override), "*", None);
+                .push_import(format!("{}::error", cli.common_import_rust()), "*", None);
             // in case we store these in enums we're just going to dump them in everywhere
             if cli.preserve_encodings {
                 content
                     .push_import(
-                        format!("{}::serialization", cli.common_import_override),
+                        format!("{}::serialization", cli.common_import_rust()),
                         "LenEncoding",
                         None,
                     )
                     .push_import(
-                        format!("{}::serialization", cli.common_import_override),
+                        format!("{}::serialization", cli.common_import_rust()),
                         "StringEncoding",
                         None,
                     );
@@ -802,12 +805,12 @@ impl GenerationScope {
                 content
                     .push_import("std::collections", "BTreeMap", None)
                     .push_import(
-                        format!("{}::serialization", cli.common_import_override),
+                        format!("{}::serialization", cli.common_import_rust()),
                         "LenEncoding",
                         None,
                     )
                     .push_import(
-                        format!("{}::serialization", cli.common_import_override),
+                        format!("{}::serialization", cli.common_import_rust()),
                         "StringEncoding",
                         None,
                     );
@@ -880,15 +883,11 @@ impl GenerationScope {
             // Issue (general - not just here): https://github.com/dcSpark/cddl-codegen/issues/139
             content.push_import("std::collections", "BTreeMap", None);
             if cli.preserve_encodings {
-                if *scope == *ROOT_SCOPE {
-                    content.push_import("ordered_hash_map", "OrderedHashMap", None);
-                } else {
-                    content.push_import(
-                        format!("{}::ordered_hash_map", cli.common_import_override),
-                        "OrderedHashMap",
-                        None,
-                    );
-                }
+                content.push_import(
+                    format!("{}::ordered_hash_map", cli.common_import_rust()),
+                    "OrderedHashMap",
+                    None,
+                );
             }
         }
 
@@ -903,7 +902,7 @@ impl GenerationScope {
                 .push_import("std::io", "Write", None)
                 .push_import("cbor_event::de", "Deserializer", None)
                 .push_import("cbor_event::se", "Serializer", None)
-                .push_import(format!("{}::error", cli.common_import_override), "*", None);
+                .push_import(format!("{}::error", cli.common_import_rust()), "*", None);
             if cli.preserve_encodings {
                 content.push_import("super::cbor_encodings", "*", None);
             }
@@ -912,7 +911,7 @@ impl GenerationScope {
             }
             if *scope != *ROOT_SCOPE {
                 content.push_import(
-                    format!("{}::serialization", cli.common_import_override),
+                    format!("{}::serialization", cli.common_import_rust()),
                     "*",
                     None,
                 );
@@ -962,7 +961,7 @@ impl GenerationScope {
                     .push_import("wasm_bindgen::prelude", "JsValue", None);
                 if cli.preserve_encodings {
                     content.push_import(
-                        format!("{}::ordered_hash_map", cli.lib_name_code()),
+                        format!("{}::ordered_hash_map", cli.common_import_wasm()),
                         "OrderedHashMap",
                         None,
                     );
@@ -1069,31 +1068,34 @@ impl GenerationScope {
         )?;
 
         // serialiation.rs / {module}/serialization.rs files (if input is a directory)
-        let mut serialize_paths = vec![cli.static_dir.join("serialization.rs")];
-        if cli.preserve_encodings {
-            serialize_paths.push(cli.static_dir.join("serialization_preserve.rs"));
-            if cli.canonical_form {
-                serialize_paths.push(
-                    cli.static_dir
-                        .join("serialization_preserve_force_canonical.rs"),
-                );
+        let mut merged_rust_serialize_scope = codegen::Scope::new();
+        if cli.export_static_files() {
+            let mut serialize_paths = vec![cli.static_dir.join("serialization.rs")];
+            if cli.preserve_encodings {
+                serialize_paths.push(cli.static_dir.join("serialization_preserve.rs"));
+                if cli.canonical_form {
+                    serialize_paths.push(
+                        cli.static_dir
+                            .join("serialization_preserve_force_canonical.rs"),
+                    );
+                } else {
+                    serialize_paths.push(
+                        cli.static_dir
+                            .join("serialization_preserve_non_force_canonical.rs"),
+                    );
+                    serialize_paths
+                        .push(cli.static_dir.join("serialization_non_force_canonical.rs"));
+                }
             } else {
-                serialize_paths.push(
-                    cli.static_dir
-                        .join("serialization_preserve_non_force_canonical.rs"),
-                );
+                serialize_paths.push(cli.static_dir.join("serialization_non_preserve.rs"));
                 serialize_paths.push(cli.static_dir.join("serialization_non_force_canonical.rs"));
             }
-        } else {
-            serialize_paths.push(cli.static_dir.join("serialization_non_preserve.rs"));
-            serialize_paths.push(cli.static_dir.join("serialization_non_force_canonical.rs"));
-        }
-        // raw_bytes_encoding in serialization too
-        if export_raw_bytes_encoding_trait {
-            serialize_paths.push(cli.static_dir.join("raw_bytes_encoding.rs"));
+            // raw_bytes_encoding in serialization too
+            if export_raw_bytes_encoding_trait {
+                serialize_paths.push(cli.static_dir.join("raw_bytes_encoding.rs"));
+            }
+            merged_rust_serialize_scope.raw(concat_files(&serialize_paths)?);
         }
-        let mut merged_rust_serialize_scope = codegen::Scope::new();
-        merged_rust_serialize_scope.raw(concat_files(&serialize_paths)?);
         merged_rust_serialize_scope.append(&self.rust_serialize_lib_scope);
         merge_scopes_and_export(
             rust_dir.join("rust/src"),
@@ -1153,30 +1155,32 @@ impl GenerationScope {
             rust_cargo_toml.replace("cddl-lib", &cli.lib_name),
         )?;
 
-        // error.rs
-        std::fs::copy(
-            cli.static_dir.join("error.rs"),
-            rust_dir.join("rust/src/error.rs"),
-        )?;
+        if cli.export_static_files() {
+            // error.rs
+            std::fs::copy(
+                cli.static_dir.join("error.rs"),
+                rust_dir.join("rust/src/error.rs"),
+            )?;
 
-        // ordered_hash_map.rs
-        if cli.preserve_encodings {
-            let mut ordered_hash_map_rs =
-                std::fs::read_to_string(cli.static_dir.join("ordered_hash_map.rs"))?;
-            if cli.json_serde_derives {
-                ordered_hash_map_rs.push_str(&std::fs::read_to_string(
-                    cli.static_dir.join("ordered_hash_map_json.rs"),
-                )?);
-            }
-            if cli.json_schema_export {
-                ordered_hash_map_rs.push_str(&std::fs::read_to_string(
-                    cli.static_dir.join("ordered_hash_map_schemars.rs"),
-                )?);
+            // ordered_hash_map.rs
+            if cli.preserve_encodings {
+                let mut ordered_hash_map_rs =
+                    std::fs::read_to_string(cli.static_dir.join("ordered_hash_map.rs"))?;
+                if cli.json_serde_derives {
+                    ordered_hash_map_rs.push_str(&std::fs::read_to_string(
+                        cli.static_dir.join("ordered_hash_map_json.rs"),
+                    )?);
+                }
+                if cli.json_schema_export {
+                    ordered_hash_map_rs.push_str(&std::fs::read_to_string(
+                        cli.static_dir.join("ordered_hash_map_schemars.rs"),
+                    )?);
+                }
+                std::fs::write(
+                    rust_dir.join("rust/src/ordered_hash_map.rs"),
+                    rustfmt_generated_string(&ordered_hash_map_rs)?.as_ref(),
+                )?;
             }
-            std::fs::write(
-                rust_dir.join("rust/src/ordered_hash_map.rs"),
-                rustfmt_generated_string(&ordered_hash_map_rs)?.as_ref(),
-            )?;
         }
 
         // wasm crate
@@ -3409,7 +3413,7 @@ fn create_base_wasm_struct<'a>(
                     if cli.preserve_encodings && cli.canonical_form {
                         to_bytes.line(format!(
                             "{}::serialization::Serialize::to_cbor_bytes(&self.0)",
-                            cli.lib_name_code()
+                            cli.common_import_wasm()
                         ));
                         let mut to_canonical_bytes =
                             codegen::Function::new("to_canonical_cbor_bytes");
@@ -3421,7 +3425,7 @@ fn create_base_wasm_struct<'a>(
                     } else {
                         to_bytes.line(format!(
                             "{}::serialization::ToCBORBytes::to_cbor_bytes(&self.0)",
-                            cli.lib_name_code()
+                            cli.common_import_wasm()
                         ));
                     }
                     s_impl.push_fn(to_bytes);
@@ -3433,7 +3437,7 @@ fn create_base_wasm_struct<'a>(
                             .vis("pub")
                             .line(format!(
                                 "{}::serialization::Deserialize::from_cbor_bytes(cbor_bytes).map(Self).map_err(|e| JsValue::from_str(&format!(\"from_bytes: {{}}\", e)))",
-                                cli.lib_name_code()));
+                                cli.common_import_wasm()));
                     }
                 }
                 if cli.json_serde_derives {