rust-analyzer fails to load miri workspace when toolchain is on read-only file system

[RalfJung]

I am running vscode in a sandbox which leads to `~/.rustup/toolchains' being read-only from the POV of vscode and RA. This usually works fine, except when I open the Miri workspace; then I get the following error:

rust-analyzer failed to load workspace: Failed to read Cargo metadata for Rust sources: Failed to run `cargo metadata --manifest-path /home/r/.rustup/toolchains/miri/lib/rustlib/rustc-src/rust/compiler/rustc/Cargo.toml`: `cargo metadata` exited with an error:     Updating crates.io index
error: failed to write /home/r/.rustup/toolchains/miri/lib/rustlib/rustc-src/rust/compiler/rustc/Cargo.lock

Caused by:
  failed to open: /home/r/.rustup/toolchains/miri/lib/rustlib/rustc-src/rust/compiler/rustc/Cargo.lock

Caused by:
  Read-only file system (os error 30)
rust-analyzer failed to load workspace: Failed to read Cargo metadata for Rust sources: Failed to run `cargo metadata --manifest-path /home/r/.rustup/toolchains/miri/lib/rustlib/rustc-src/rust/compiler/rustc/Cargo.toml`: `cargo metadata` exited with an error:     Updating crates.io index
error: failed to write /home/r/.rustup/toolchains/miri/lib/rustlib/rustc-src/rust/compiler/rustc/Cargo.lock

Caused by:
  failed to open: /home/r/.rustup/toolchains/miri/lib/rustlib/rustc-src/rust/compiler/rustc/Cargo.lock

Caused by:
  Read-only file system (os error 30)

Indeed /home/r/.rustup/toolchains/miri/lib/rustlib/rustc-src/rust/compiler/rustc/Cargo.lock is read-only, but I also don't think RA has any business mutating that file (it is managed by rustup after all), so it doesn't seem like making that directory read-write would be the proper fix. Maybe cargo metadata should be called with --offline or --frozen?

This is probably related to my workspace settings:

// Place your settings in this file to overwrite default and user settings.
{
    "rust-analyzer.rustfmt.extraArgs": [
        "+nightly"
    ],
    "rust-analyzer.checkOnSave.overrideCommand": [
        "./miri",
        "check",
        "--message-format=json",
    ],
    "rust-analyzer.linkedProjects": [
        "./Cargo.toml",
        "./cargo-miri/Cargo.toml"
    ],
    "rust-analyzer.rustcSource": "discover"
}

Specifically I suspect the rustcSource part is relevant here.

[RalfJung]

Indeed commenting out that rustcSource option works around the problem, but of course then auto-completion of rustc types and methods stops working.

[bjorn3]

I think Cargo.lock is the literal Cargo.lock from the rust repo. When you do cargo metadata it will update it to remove all references to the standard library as the standard library is not included in rustc-src. There is no way around it AFAIK. --locked would error with an outdated lockfile error. You can try cargo metadata outside of the sandbox inside the rustc-src dir to update Cargo.lock.

[RalfJung]

I think Cargo.lock is the literal Cargo.lock from the rust repo.

Indeed. And it is part of the rustc distribution, so RA should not modify it.

There is no way around it AFAIK.

That sounds like a cargo metadata bug then?

I should also note that this is a new problem, everything used to work fine a few months ago -- though I cannot exclude the possibility that some system update changed the behavior of the sandbox.

[RalfJung]

I think Cargo.lock is the literal Cargo.lock from the rust repo.

Actually, no, it is not. That one lives at /home/r/.rustup/toolchains/miri/lib/rustlib/rustc-src/rust/Cargo.lock.

/home/r/.rustup/toolchains/miri/lib/rustlib/rustc-src/rust/compiler/rustc/Cargo.lock is a file that does not exist.

[bjorn3]

I see. rustlib/rustc-src/rust/Cargo.toml doesn't exist, so cargo doesn't know rustlib/rustc-src/rust should be a workspace and thus rustlib/rustc-src/rust/Cargo.lock is unused and instead rustlib/rustc-src/rust/compiler/rustc/Cargo.lock is written by cargo metadata

There is no way around it AFAIK.

That sounds like a cargo metadata bug then?

All cargo commands that resolve packages update Cargo.lock if it is outdated or non-existent.

[RalfJung]

I see -- but that doesn't mean that is a good idea. cargo metadata is often used as a way to query information about a crate/workspace and that should really be a read-only process.

I understand RA cannot do much about that -- so moving to cargo: rust-lang/cargo#10096.

[RalfJung]

As per this comment, cargo now has a feature that can be used to solve this problem: --lockfile-path. Reopening to track possibly using this on the RA side. :)

[Veykril]

I fail to see how that flag is useful to us, if I see that right if we were to use that we would not necessarily be getting the exact versions of dependencies that the project is using which seems suboptimal. Or am I misunderstanding things here?

[RalfJung]

You can copy the Cargo.lock from the sysroot into a location that's writeable, and then set --lockfile-path to that location. Then RA is guaranteed to use the same versions as the sysroot, but also keeps working when the sysroot is read-only.

[Veykril]

I see, that sounds like an odd way to solve this. Remind me (if you can), how can passing --locked fail? As in what scenario could occur that requires the lock file to be updated?

[RalfJung]

AFAIK --locked fails because the Cargo.lock file needs to be updated to remove all the things that aren't needed -- all the things that were only included because in rustc, the lockfile covers the library and the compiler.

However, @bjorn3 recently changed that, so it's also possible that --locked will work on recent nightlies.

[Veykril]

Ah I see, too many entries in the lock file can make it fail with --locked, then us using --locked here should suffice for this I believe (we already do this now since the workspace split)

[RalfJung]

Ah if you're setting --locked now then indeed I assume this will work now.

[RalfJung]

Hm, I am still getting errors in Miri though when the sysroot is read-only.

024-08-17T09:14:56.995879Z ERROR Failed to read Cargo metadata from rustc source at /home/r/.rustup/toolchains/miri/lib/rustlib/rustc-src/rust/compiler/rustc/Cargo.toml e=Failed to run `cd "/home/r/src/rust/miri/miri-script" && RUSTUP_TOOLCHAIN="/home/r/.rustup/toolchains/miri" "/home/r/.cargo/bin/cargo" "metadata" "--format-version" "1" "--manifest-path" "/home/r/.rustup/toolchains/miri/lib/rustlib/rustc-src/rust/compiler/rustc/Cargo.toml" "--filter-platform" "x86_64-unknown-linux-gnu"`

I think this is about the rustc sources, not the sysroot sources?

[RalfJung]

Ah indeed, the rustc sources just don't have a lockfile at all. So when RA invokes cargo metadata, it tries to create a lockfile, and that will fail when the sysroot is read-only.

Or more precisely, there is a lockfile in lib/rustlib/rustc-src/rust, but no corresponding Cargo.toml, so cargo doesn't even recognize all these crates as a workspace and ignores that lockfile.

[Veykril]

Oh no, have we been creating a lockfile in the rustc crate in the rustc sources this entire time? 😨

[RalfJung]

Yes, yes you have. ;)

[bjorn3]

Simply adding Cargo.toml to the rustc-dev component doesn't work as the root workspace contains a fair amount of tools that are not included in the rustc-dev component.