What is needed to implement a custom file system?

[mikelsr]

Let me start by apologizing in advance if this is not the place to ask this, and asking you to point me to the correct place.

I am creating a custom FileSystem by implementing the FileSystem, FileOpener and VirtualFile traits. The custom file system is configured in this line.

The file system is just a wrapper that allows WASM guest programs to read (and only read) from IPFS. Here is the guest program I am using for testing.

When examining the calls made in Wasmer, I find I am getting to this error branch in path_open, but I am unsure of why.

Is there anything I am not implementing, configuring properly...? I'm attaching a segment of the logs at the end of this post.

Thank you in advance!

2024-11-14T14:54:56.515147Z DEBUG wasmer_wasix::fs: Initializing WASI filesystem
2024-11-14T14:54:56.517502Z TRACE wasmer_wasix::state::func_env: initializing with layout WasiMemoryLayout { stack_upper: 1048576, stack_lower: 0, guard_size: 0, stack_size: 1048576 }
2024-11-14T14:54:56.517941Z TRACE fd_prestat_get: wasmer_wasix::syscalls::wasi::fd_prestat_get: return=Errno::success fd=3
2024-11-14T14:54:56.518209Z TRACE fd_prestat_dir_name: wasmer_wasix::syscalls::wasi::fd_prestat_dir_name: return=Errno::success fd=3 path="/"
2024-11-14T14:54:56.518270Z TRACE fd_prestat_get: wasmer_wasix::syscalls::wasi::fd_prestat_get: return=Errno::badf fd=4
2024-11-14T14:54:56.518486Z TRACE path_open:new_open_options: fs: return=OpenOptionsConfig { read: true, write: false, create_new: false, create: false, append: false, truncate: false } dirfd=3 follow_symlinks=true path="ipfs/QmeeD4LBwMxMkboCNvsoJ2aDwJhtsjFyoS1B9iMXiDcEqH"
2024-11-14T14:54:56.518563Z TRACE path_open: wasmer_wasix::syscalls::wasi::path_open: return=Ok(Errno::notcapable) dirfd=3 follow_symlinks=true path="ipfs/QmeeD4LBwMxMkboCNvsoJ2aDwJhtsjFyoS1B9iMXiDcEqH"
Error reading from file: Capabilities insufficient (os error 76)

[Arshia001]

Hey @mikelsr!

Lots of code in WASIX assumes the file system is always created through the use of RootFileSystemBuilder. Can you try mounting your custom file system in a sub-directory? You can look at the WASIX code in the Wasmer CLI as a sample.

[mikelsr]

Hi @Arshia001, thank you very much for your response and apologies for the late reply. Unfortunately, following the recommendation and examples yielded the same result. This is how I build the file system now:

    let ipfs_fs = IpfsFs::new(ipfs_client);
    let ipfs_path = ipfs_fs.path();  # equivalent to "/ipfs"
    let shared_ipfs_fs = Arc::new(ipfs_fs) as Arc<dyn virtual_fs::FileSystem + Send + Sync>;
    let root_fs = RootFileSystemBuilder::new().build();
    root_fs.mount(ipfs_path.clone(), &shared_ipfs_fs, ipfs_path)?;
    let mut wasm_process = wasm_runtime.build(bytecode, root_fs)?;
    wasm_process.run(wasm_runtime.store_mut())?;

And then mount it with WasiEnv::builder(uuid).fs(Box::new(fs))....

Funnily enough, I only get this error when opening a file from a WASM guest program. I wrote this test to check if the FileSystem implementation was correct, and it does indeed work:

#[cfg(test)]
mod tests {
    use virtual_fs::FileSystem;

    use super::*;

    #[tokio::test(flavor = "multi_thread", worker_threads = 1)]
    async fn read_file() {
        let ipfs_fs = IpfsFs::new(net::ipfs::Client::new(
            "/ip4/127.0.0.1/tcp/5001".parse().unwrap(),
        ));
        let ipfs_path = ipfs_fs.path();
        let shared_ipfs_fs = Arc::new(ipfs_fs) as Arc<dyn virtual_fs::FileSystem + Send + Sync>;
        let root_fs = RootFileSystemBuilder::new().build();
        root_fs
            .mount(ipfs_path.clone(), &shared_ipfs_fs, ipfs_path)
            .unwrap();
        let file_res = root_fs
            .new_open_options()
            .read(true)
            .open("/ipfs/QmSJPwM15H5uM29dqh4aKt3i1FyVzsgWfz2ahMU9RY3rjm");
        assert!(file_res.is_ok());
        let file = file_res.unwrap();
        assert_eq!(file.size(), 20);
    }
}

The code above works fine, but trying to read the same file from a WASM guest does not. This makes me think the issue lies somewhere in how I am mounting the custom filesystem into the Wasmer runtime. When I run the following code compiled to WASM, I get the same log/error as before:

use std::fs;

fn main() {
    let file_content = fs::read_to_string("/ipfs/QmSdCbA1YZeoTC7qJigXUHYhBw5nUjM6rkKCzXrdrA3JF7");
    match file_content {
        Ok(s) => println!("{}", s),
        Err(e) => println!("Error reading from file: {}", e),
    }
}

Which is run with:

    let mut store = wasmer::Store::default();
    let ipfs_fs = IpfsFs::new(ipfs_client);
    let ipfs_path = ipfs_fs.path();  // equivalent to "/ipfs"
    let shared_ipfs_fs = Arc::new(ipfs_fs) as Arc<dyn virtual_fs::FileSystem + Send + Sync>;
    let root_fs = RootFileSystemBuilder::new().build();
    root_fs.mount(ipfs_path.clone(), &shared_ipfs_fs, ipfs_path)?;

    let module = wasmer::Module::new(&store, bytecode).expect("couldn't load WASM module");
    let uuid = Uuid::new_v4();
    let mut wasi_env = WasiEnv::builder(uuid)
        //.fs(Box::new(root_fs))
        .sandbox_fs(root_fs)
        .finalize(self.store_mut())?;
    let import_object = wasi_env.import_object(store_mut(), &module)?;
    let instance = wasmer::Instance::new(store_mut(), &module, &import_object)?;
    wasi_env.initialize(self.store, instance.clone())?;
    
    let function = instance.exports.get_function("_start")?;
    function.call(store, &[])?;

Do you have any pointers as to what I might be doing wrong? As an additional note, my OpenOptions have read set to true.

Thank you very much for your time, your help is really appreciated!

[Arshia001]

Sorry, I can't go through this in detail right now, but are you using WasiEnvBuilder to build and run the module? It seems like you're calling _start directly. Without going through WasiEnvBuilder, there's going to be a lot of missing required setup.

[mikelsr]

Hi! I've updated the last code snippet to show more context :) thank you for the quick reply @Arshia001

[mikelsr]

Hi!

Turns out all I was missing was adding a call to preopen_vfs_dirs!

Thank you for your time, I'm closing the discussion.