cargo publish/package ignores include section and fails when subdirectory contains a Cargo.toml

[yaahc]

Problem

I'm trying to publish https://github.com/ZcashFoundation/zcashconsensus which compiles and re-exports a c++ library as a rust crate. This library is defined in another repository we do not control and we just include it as a git submodule. However, when attempting to publish the build.rs file fails because the c++ files it needs to build aren't pulled in with the rest of the files before packaging.

I tried to fix this by explicitly specifying an include list in the [package] section of my Cargo.toml but cargo seems to completely ignore the include section for files in the git submodule. I believe this is the same issue as seen in this older issue #2657

However, I cannot fix this the same way they did because the way brson fixed the issue was by using a previous version of cargo from before the feature to filter out files in subdirectories was introduced, and presumably such a version of cargo is 4 years old now...

Steps

1. checkout the zcashconsensus repo https://github.com/ZcashFoundation/zcashconsensus
2. add the following include section to the Cargo.toml

include = [
    ".github/workflows/ci.yml",
    ".gitignore",
    ".gitmodules",
    "Cargo.toml",
    "Cargo.toml.orig",
    "LICENSE",
    "README.md",
    "build.rs",
    "depend/check_uint128_t.c",
    "src/blake2b.rs",
    "src/lib.rs",
    "zcash/src/script/zcashconsensus.h"
]

3. run cargo package --list --allow-dirty to show that the "zcash/src/script/zcashconsensus.h" file has not been included.

Notes

Output of cargo version:

❯ cargo version
cargo 1.45.0 (744bd1f 2020-06-15)

[scouten]

Following: I'm encountering what appears to be a similar issue.

[yaahc]

@scouten fwiw I ended up fixing this by removing the Cargo.toml in the subdirectory that was causing cargo to filter out all files from that directory.

[scouten]

@yaahc thanks. In my case, it turned out to be a false alarm. It looked like the dir wasn't there, but in fact it was an unrelated issue.

[emilio]

#9017 is another instance of this.

[weihanglo]

From #9643 (comment) by @tmplt:

Problem

I am trying to publish a crate which builds adhoc crates during run-time. For this reason, I include_dir::include_dir!("assets/libadhoc"). This assets/libadhoc/ contains a Cargo.toml which is then written to disk and modified during run-time. However, on a cargo publish this directory is not copied to target/package/, even with package.include = ["*"] in the parent Cargo.toml.

Steps

$ cargo init --bin project && cd project
$ mkdir assets
$ touch assets/Cargo.toml
$ cargo package --list
Cargo.toml
src/main.rs
Possible Solution(s)

I figure **/Cargo.toml are ignored for a reason, but this is not documented in cargo-package(1). I suggest an amend.

Further, I cannot find a way to override this behavior. I suggest making that possible. Alternatively/additionally, I suggest adding an environmental variable that points to the generated target/package/project-0.1.0/ so that I can copy the files myself.

For now, I'll rename the file to not-Cargo.toml and rename it during run-time so that I can publish my crate.

[weihanglo]

From #9643 (comment) by @gdesmott:

I'm having a similar problem with system-deps. My tests use a bunch of sub-directories containing a Cargo.toml which are not included in the package.

I think it's because cargo package does not recurse in dir containing a Cargo.toml. While it makes sense for the default, it should be overridable using the package.include field but it's not at the moment.

[Systemcluster]

The workaround I used for wrappe is to tar the inner crate before publishing, and using the build.rs to unpack it during builds.