What about `--target web` _with_ a bundler? Needed for threads. #3720

kalzoo · 2023-11-25T21:42:51Z

kalzoo
Nov 25, 2023

This is half a question and half a possible answer for others. It addresses a use case that is missed by the docs (AFAICT).

Backdrop: I've had a react + rust-via-wasm app happily running for a long time using the default wasm-pack build [--target bundler]. I'd like to add threading support to the WASM binary which explicitly needs --target web per this. I'd like to keep using Webpack (and CRA) for everything else; I don't want to move DOM manipulation into Rust (yet), something that seems to be implied by the --target web example. I'd like the bundled JS app to still be able to call the WASM bindings.

Rather than init()ing the wasm module in the <script> tag like the example shows, I found I had to do the following:

pin my library, libfoo, to the window, along with its init function
Call that function within the React context which does initialization work of its own
Use copy-webpack-plugin and a webpack-dev-server override to copy the files from the crate's pkg directory (whereas before, in bundler mode, I was able to rely on yarn link)

Like so:

index.html

<!DOCTYPE html>
<html lang="en">
  <!-- omitted -->
  <body>
    <!-- from https://rustwasm.github.io/docs/wasm-bindgen/examples/without-a-bundler.html -->
    <!-- Note the usage of `type=module` here as this is an ES6 module -->
    <script type="module">
      import init, * as libfoo from './static/pkg/libfoo.js';
      window.init_libfoo = init;
      window.libfoo = libfoo;
    </script>

   <!-- standard react root element: -->
    <div id="root"></div>
  </body>
</html>

wasm_context.tsx

useEffect(() => {
    // @ts-ignore
    window.init_libfoo()
      .then(() => {
        // this works
        // @ts-ignore
        return window.libfoo.FooBackend.new(input);
      })
      .then((foo: FooBackend) => {
        // this also works (even with the thread-supporting .cargo/config.toml described below)
        foo.add(1,2);
        setInnerContext(foo);
      })
      .catch((err: any) => {
        console.error(
          `Unexpected error loading wasm binary. [Message: ${err.message}]`, err
        );
      });
    // eslint-disable-next-line
  }, [input]);

config-overrides.js

module.exports = {
  devServer: function (configFunction) {
    return function (proxy, allowedHost) {
      const config = configFunction(proxy, allowedHost);
      config.devMiddleware.writeToDisk = true;
      return config;
    };
  },
  webpack: function override(config, env) {
   // Some existing WASM config omitted for clarity...

    config.externals = {
      ...config.externals,
      sched: "window.sched",
    }

    config.plugins.push(
      new CopyPlugin({
        patterns: [
          {
            from: path.join(__dirname, "../lib/pkg"),
            to: path.join(__dirname, "build/static/pkg"),
          },
        ],
      }),
    );

    return config;
  },
};

This setup prevents a race condition between the await init() call in the <script> and the import call in the bundled app. By letting the app do the init, I can ensure that init_libfoo runs first.

And it does indeed work like the --target bundler version did - most of the time. There appears to still be a race condition in initialization, and some page loads break with TypeError: window.init_sched is not a function. That is not an issue for wasm-bindgen to handle, it's a browser/react thing for me to investigate. But it does reinforce that the above is not the "right" way to mix --target web with a bundler.

But...

That feels messy. I don't want to be hanging my module on the global window if I can avoid it. And unfortunately, threading support still doesn't work. When I configure cargo.toml as directed, I see this:

ptr::write requires that the pointer argument is aligned and non-null

which, in this particular case, seems to emanate from serde-yaml. Simple functions like add are callable without error.

I have not yet investigated that one well enough to know if that is a distinct issue or related to how I'm initializing this. But it means that --target web is so far just slower (because it requires dev-server to write to disk) and more clumsy than --target bundler without benefits.

tl;dr: is there a better way to make a module built with --target web still usable from code bundled by webpack? And is there something I did above that prevents thread support from working?

FYI, I also used the docs for wasm-bindgen-rayon as a reference, since they seem to be more recent than the relevant wasm-bindgen docs, but I haven't used that package directly myself because I don't use or need rayon itself. I'll try using that library next, just in case its init functions handle the errors that I'm seeing.

daxpedda · 2023-11-26T10:08:01Z

daxpedda
Nov 26, 2023
Maintainer

I'd like to add threading support to the WASM binary which explicitly needs --target web per this.

This comes down to threading support requiring wasm_bindgen::module(), which isn't available on the bundler target. I don't know why it isn't available on the bundler target, but I guess this can be fixed in wasm-bindgen.

I'd like to keep using Webpack (and CRA) for everything else; I don't want to move DOM manipulation into Rust (yet), something that seems to be implied by the --target web example. I'd like the bundled JS app to still be able to call the WASM bindings.

Unfortunately I know nothing about webpack, but you can call WASM bindings from JS when using the web target just fine.

And is there something I did above that prevents thread support from working?

Generally speaking, even though threading support in Rust+Wasm+wasm-bindgen has some edges, its working, I'm using it on a daily basis without issues. I can't say exactly what it is you are doing that is breaking that, but I'm happy to look at any minimal example if you can provide one.

6 replies

kmdouglass Jan 10, 2024

Hi @kalzoo ,

Did you have any luck with this? I asked a similar question in #3769 before I saw your question.

kalzoo Jan 10, 2024
Author

@kmdouglass - I did! It turns out that the only blocking issue - the panic - was indeed a bug and was fixed quickly (in the issue linked).

as I described here, this works just fine with a bundler. —target web is not as different as the docs make it seem. the key part for me was not following the example in the docs and not modifying index.html, but instead:

importing the init function as the default export from my wasm module (alongside anything else already imported)
Awaiting the call to that init function before trying to do anything else with the module

(as illustrated in my example here)

Like you, and as I described above, I also tried to use wasm-bindgen-rayon but rolled it back as unnecessary. One thing there that did help was the tip to use the right target-feature RUSTFLAGS; there are 3, rather than 2.

that’s it! Pretty easy in retrospect.

matthiasgeihs May 8, 2024

@kalzoo Thanks for sharing your insights here.

Got a related problem, wondering if you had the same.

Im trying to compile into something that can be used on Node.js and Web. With --target web this is almost possible, but there is one instance in the glue code where window is used, which is not accessible in Node.js.

Did you run into similar issues? Any suggestions how to resolve this?

kalzoo May 8, 2024
Author

I haven’t, I’ve only run this in the browser, no SSR etc. From what you mentioned, I’d be curious if a polyfill could help, although it sounds like compiling to a different target for browser vs node might be the way to go if it’s feasible for you.

matthiasgeihs May 8, 2024

Turns out we were using wasm_bindgen_test::wasm_bindgen_test_configure!(run_in_browser); in one place by mistake. Removing that from the production build made the usage of window go away. Haven't tested in Node.js yet, but might just work like that.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

What about `--target web` _with_ a bundler? Needed for threads. #3720

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 1 comment 6 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

What about --target web _with_ a bundler? Needed for threads. #3720

kalzoo Nov 25, 2023

But...

Replies: 1 comment · 6 replies

daxpedda Nov 26, 2023 Maintainer

kmdouglass Jan 10, 2024

kalzoo Jan 10, 2024 Author

matthiasgeihs May 8, 2024

kalzoo May 8, 2024 Author

matthiasgeihs May 8, 2024

What about `--target web` _with_ a bundler? Needed for threads. #3720

kalzoo
Nov 25, 2023

Replies: 1 comment 6 replies

daxpedda
Nov 26, 2023
Maintainer

kalzoo Jan 10, 2024
Author

kalzoo May 8, 2024
Author