[Web] Error when using Web Workers on Next.js

[illbexyz]

Describe the issue

I get the following error when using onnxruntime-web on Next.js with web workers:

• Error: no available backend found. ERR: [wasm] TypeError: Failed to fetch, [cpu] Error: multiple calls to 'initWasm()' detected.

I had no issues with [email protected].

I believe the issue is that Next.js is trying to load the file from the filesystem:

• See Request URL in the following screenshot file:///Users/alberto.nicoletti/node_modules/onnxruntime-web/dist/ort.bundle.min.mjs

Additional notes:

• I have no issues when not using web workers
• I obtain a slightly different error when disabling the proxy but setting onnx.env.wasm.numThreads = 2
  • Error: no available backend found. ERR: [wasm] SecurityError: Failed to construct 'Worker': Script at 'file:///Users/alberto.nicoletti/node_modules/onnxruntime-web/dist/ort.bundle.min.mjs' cannot be accessed from origin 'http://localhost:3000'., [cpu] Error: previous call to 'initWasm()' failed..
  • In this case there are no network requests being made but I believe these issues are likely related.

Network requests screenshot:

To reproduce

The following reproductions are a bare Next.js 14 starter repository with [email protected].

• Proxy error reproduction here: https://github.com/illbexyz/nextjs-onnxruntime/tree/wasm-proxy
  • See https://github.com/illbexyz/nextjs-onnxruntime/blob/wasm-proxy/src/pages/components/OnnxButton.tsx where I use a web worker by enabling onnx.env.wasm.proxy = true.
• Multi-thread reproduction here: https://github.com/illbexyz/nextjs-onnxruntime/tree/wasm-threads
  • See https://github.com/illbexyz/nextjs-onnxruntime/blob/wasm-threads/src/pages/components/OnnxButton.tsx where I use onnx.env.wasm.numThreads = 2.
  • In this reproduction I also added crossOriginIsolation headers here: https://github.com/illbexyz/nextjs-onnxruntime/blob/wasm-threads/next.config.mjs

Urgency

Kinda urgent: every Next.js project is impacted.

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.19.2

Execution Provider

'wasm'/'cpu' (WebAssembly CPU)

[satyajandhyala]

@illbexyz Can you try using onnxruntime 1.19 in a clean environment?

[illbexyz]

@satyajandhyala This is already the most minimal reproduction I could give. It's just onnxruntime-web added to a bare Next.js project initialized with npx create-next-app@latest.

[fs-eire]

Script at 'file:///Users/alberto.nicoletti/node_modules/onnxruntime-web/dist/ort.bundle.min.mjs' cannot be accessed from origin 'http://localhost:3000'.

This is a known issue. If you use webpack loading onnxruntime-web as ESM, you need this line in your webpack config:

module: { parser: { javascript: { importMeta: false } } },

(see https://github.com/Microsoft/onnxruntime/blob/abdc31de401262bcb03f538423389c2eb264a0ce/js/web/test/e2e/webpack.config.esm-js.js)

Explaination: onnxruntime-web use import.meta.url at runtime to determine the script path so that it is able to create worker with correct path. However the default behavior of Webpack rewrites import.meta.url into a static local file path ( in your case file:///Users/alberto.nicoletti/node_modules/onnxruntime-web/dist/ort.bundle.min.mjs) and it won't work at runtime. the highlighted line in the config disabled this specific behavior of webpack.

May be a good idea to update our document/examples.

[illbexyz]

Hey @fs-eire, thanks for your time!

I tried adding the following to next.config.mjs:

webpack: (config) => {
  config.module.parser.javascript.importMeta = false;

  return config;
},

But now I get a different error:

SyntaxError: Cannot use 'import.meta' outside a module

I updated the reproductions if you wanna have a look.

[fs-eire]

I made a PR #22196. Hope to workaround this issue without having to modify webpack config from user side.

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

[fs-eire]

The issue is not resolved. I am still working on it.

[arkniazi]

@fs-eire I'm also facing similar issue. Were you able to create a fix for it?

[fs-eire]

It is still on my list. However I have multiple tasks to do so it may take a while. #22196 is an attempt but testing result shows that it does not fix everything. #22615 is the issue that tracks and explains why it is hard to fix.

I would be appreciated if anyone can help contribute or provider detailed information/clues to the issue.

[illbexyz]

@fs-eire until the issue is fixed for good, it would great to have a way to patch onnxruntime-web to work for the specific case of Next.js/Webpack. Can you please point me in the right direction?

[fs-eire]

@illbexyz if you are OK with loading the .mjs file and .wasm file dynamically from CDN, here is the steps:

• try to import the non .bundle file. you can either patch the package.json of onnxruntime-web, or add a resolve rule in your webpack config to achieve this. Or you can import onnxruntime-web as UMD instead of ESM.
• set ort.env.wasm.wasmPaths:

  ort.env.wasm.wasmPaths = {
    mjs: 'https://cdn.jsdelivr.net/npm/[email protected]/dist/ort-wasm-simd-threaded.mjs',
    wasm: 'https://cdn.jsdelivr.net/npm/[email protected]/dist/ort-wasm-simd-threaded.wasm'
  };

[364199013]

Hello, have you solved this problem? I encountered the same issue when setting up the web demo for Segment Anything.

[fs-eire]

@364199013 which version of onnxruntime-web are you using? A recent change (#23257) should have fixed major use cases with nextjs so please let me know:

• the version of onnxruntime-web
• the setup (reproduce) steps that get into the issue

[364199013]

@fs-eire 1.20.1，At web demo for Segment Anything. this project url is https://github.com/facebookresearch/segment-anything.git.

[364199013]

I resolved the issue after building the project. @fs-eire

[sehy0803]

@364199013 Hello, I also had the same issue when setting up the web demo for Segment Anything, so I built the project. However, the build failed with this error message 'Failed to load./.env.' Do you happen to know a solution? Please help. Thank you.

[ysh-art]

I am getting this error for onnxruntime-web: ^1.20.1

[fs-eire]

Please use a dev version including change (#23257). For example, "1.21.0-dev.20250114-228dd16893"

[ysh-art]

Please use a dev version including change (#23257). For example, "1.21.0-dev.20250114-228dd16893"

Now I'm gettting the following error
"Uncaught (in promise) DataCloneError: Failed to execute 'postMessage' on 'Worker': ArrayBuffer at index 3 is already detached."

[fs-eire]

This error message is likely caused by accessing a detached ArrayBuffer. For example, you have an instance of ArrayBuffer that transferred from main thread to worker thread using postMessage() with the array buffer in parameter transfer, and then use the ArrayBuffer in main thread. This will trigger the error.

[kevinjosethomas]

hey, I was running into a similar issue a few weeks ago and recently ported to 1.21.0-dev.20250114-228dd16893, which fixed the issue in development. However, when building my NextJS project, I run into another issue:

% npm run build

> build
> next build


./src/pages/analysis/[...id].tsx
455:45  Warning: Unexpected any. Specify a different type.  @typescript-eslint/no-explicit-any

info  - Need to disable some ESLint rules? Learn more here: https://nextjs.org/docs/basic-features/eslint#disabling-rules
 ✓ Linting and checking validity of types    
Failed to compile.

static/media/ort.bundle.min.69c3a2ff.mjs from Terser
  x 'import.meta' cannot be used outside of module code.
   ,-[3:1]
 3 |  * Copyright (c) Microsoft Corporation. All rights reserved.
 4 |  * Licensed under the MIT License.
 5 |  */
 6 | var Un=Object.defineProperty;var Vp=Object.getOwnPropertyDescriptor;var Wp=

This is followed with the script and quite a long traceback. Any idea how I can tackle this? I'm using [email protected] and [email protected]

[fs-eire]

@kevinjosethomas could you share the repo with this issue? I can take a look

[kevinjosethomas]

Of course, thank you. The codebase is quite large but this is where all my onnx model inference occurs: https://github.com/CSSLab/maia-platform-frontend/blob/v1/src/utils/maia2/model.ts

[fs-eire]

Of course, thank you. The codebase is quite large but this is where all my onnx model inference occurs: https://github.com/CSSLab/maia-platform-frontend/blob/v1/src/utils/maia2/model.ts

I cloned the repo, checked out the branch v1, and I can reproduce the build failure.

To fix the failure, I updated 3 packages to latest:

• next: v15 fixes the failure.
• typescript: v5 is required, otherwise compile will fail with next v15
• onnxruntime-web: include fixes related to package export.

There are other errors, but I think they are not related to onnxruntime.

ReferenceError: window is not defined
    at window (src\hooks\useLocalStorage\useLocalStorage.ts:10:19)
    at useState (src\hooks\useLocalStorage\useLocalStorage.ts:8:48)
    at THEME (src\hooks\useTheme\useTheme.ts:8:51)
    at useTheme (src\providers\ThemeContextProvider\ThemeContextProvider.tsx:10:39)
   8 |   const [storedValue, setStoredValue] = useState<T>(() => {
   9 |     try {
> 10 |       const item = window.localStorage.getItem(key)
     |                   ^