fix package export #1161

fs-eire · 2025-01-20T06:49:44Z

Summary

This PR includes a few changes that helps to fix the package export. Now transformers.js has the following exports:

usage	file	description
CDN `import {...} from "https://cdn.jsdelivr.net/npm/@huggingface/transformers@{version}"`	`dist/transformers[.min].js`	This is a JavaScript file that bundled with all its JS dependencies.
Node.js/Deno (ESM/CJS) `import {...} from "@huggingface/transformers"` `const {...} = require("@huggingface/transformers")`	`dist/transformers.node[.min].mjs` `dist/transformers.node[.min].cjs`	Treat `onnxruntime-common` and `onnxruntime-node` as external; Ignore `onnxruntime-web`
Web (ESM) `import {...} from "@huggingface/transformers"`	`dist/transformers.web[.min].js`	Treat `onnxruntime-common` and `onnxruntime-web` as external; Ignore `onnxruntime-node`

Problem

There are complains about different kinds of failures when users try to import transformers.js and/or onnxruntime-web in their web applications.

It took me quite a while to fix most of the problems in onnxruntime-web. The latest dev version of onnxruntime-web should have worked for all known major use cases.

When things come to transformers.js, the situation become more complicated. After some investigation, I found that some import/export problems that resolved in onnxruntime-web occurred in transformers.js again. This is because when building transformers.js, webpack generated some code that will not work for using as input for bundler again.

Solution

There are 2 major changes in this PR:

remove the override of env.wasm.wasmPaths
make onnxruntime-web as external dependency of transformers.js

Why treat `onnxruntime-web` as external dependency

ONNX Runtime Web has a few workarounds for webpack to ensure out-of-the-box user experience. Those workarounds requires the code as-is when webpack starts to process it. The workarounds cannot keep in the bundled JS file.

To make sure of backward compatibility, this change still keeps the file dist/transformers[.min].js unchanged. Instead, a new file dist/transformers.web[.min].js is added and replaced in the "exports" property of the package.json.

Tests

The following tests are passed when applying this PR:

CDN
https://github.com/huggingface/transformers.js-examples/tree/migrate-examples/segment-anything-webgpu
Next.js (Server, onnxruntime-node)
https://github.com/huggingface/transformers.js-examples/tree/migrate-examples/next-server
* some issue with the "standalone" mode but not related to package export.
* need workaround for Nextjs. See [1] below
Next.js (Client, onnxruntime-web)
* need workaround for Nextjs. See [1] below
Vite
https://github.com/huggingface/transformers.js-examples/tree/migrate-examples/remove-background-webgpu
* need workaround for Vite. See [2] below
Vite + worker
https://github.com/huggingface/transformers.js-examples/tree/migrate-examples/phi-3.5-webgpu
* need workaround for Vite. See [2] below
Service worker
[js/web] ESM: use the bundled target as default export microsoft/onnxruntime#20991

** [1] - For Next.js based app, a workaround is needed if any code statically imports @huggingface/transformers. This is because Next.js tries to bundle the code for server even if those code starts with "use client";. This happens to both dev mode and prod mode, and to both default and turbopack bundler.
** [2] - For Vite 5.x based app, a workaround is needed according to vitejs/vite#8427. This is a known issue affecting all Vite app depending on WebAssembly based packages, and may be fixed in Vite 6.x

examples/extension/src/background.js

src/backends/onnx.js

fs-eire · 2025-01-20T06:53:59Z

webpack.config.js

@@ -42,7 +42,7 @@ function buildConfig({
        type,
      },
      assetModuleFilename: "[name][ext]",
-      chunkFormat: "module",
+      chunkFormat: false,


This is why the final bundle includes new URL("./", import.meta.url).

Setting it to false to disable the chunks feature of webpack (which is not used anyway)

Good find! Thanks!

webpack.config.js

src/backends/onnx.js

fs-eire · 2025-01-24T21:35:07Z

@xenova what do you think of the proposed changes?

fs-eire · 2025-01-29T21:44:30Z

@xenova what do you think of the proposed changes?

PR is ready for review.

xenova · 2025-01-30T20:12:04Z

Thanks @fs-eire! I will review this fully soon!

sroussey · 2025-02-04T02:05:34Z

src/backends/onnx.js

-
-    // https://developer.mozilla.org/en-US/docs/Web/API/crossOriginIsolated
-    if (typeof crossOriginIsolated === 'undefined' || !crossOriginIsolated) {
-        ONNX_ENV.wasm.numThreads = 1;
-    }


why is this included?

there was a bug that in a non-cross origin isolated environment, onnxruntime-web will use multiple threads and then fail the initialization. This code snippet was a workaround for that bug.

Now in latest onnxruntime-web, this check is included so no necessary to do it again in transformerjs

Ah nice. I’ve run into this error so was curious.

Good to know! Thanks @fs-eire!

xenova

Thanks for this PR! I'm doing some more testing now.

xenova · 2025-02-05T13:34:11Z

src/backends/onnx.js

+    // Currently, this behavior is disabled. To enable it, uncomment the following code.
+
+    // // Override the wasm search path if:
+    // // 1. We are not in a service worker.
+    // // 2. ONNX Runtime Web version is defined.
+    // // 3. The wasmPaths are not already set.
+    // //
+    // // @ts-ignore Cannot find name 'ServiceWorkerGlobalScope'.ts(2304)
+    // if (!(typeof ServiceWorkerGlobalScope !== 'undefined' && self instanceof ServiceWorkerGlobalScope)
+    //     && typeof ONNX?.versions?.web === 'string'
+    //     && !ONNX_ENV.wasm.wasmPaths) {
+    //     ONNX_ENV.wasm.wasmPaths = `https://cdn.jsdelivr.net/npm/onnxruntime-web@${ONNX.versions.web}/dist/`;
+    // }


We've run into several issues in the past where the wasm files weren't able to be loaded locally, so we default to the CDN. Maybe in future we can switch to this, but I think we can keep it for now.

Another reason is that the raw .wasm file is ~21MB, but the CDN (gzipped) version is ~4MB. It might be a good idea to find a way to compress the raw file too. cc @fs-eire @guschmue

No problem. I will make a change to keep them.

We've run into several issues in the past where the wasm files weren't able to be loaded locally

This is indeed the case back to a few versions ago. I made it much better now but I still cannot say it's working for 100% cases.

the CDN (gzipped) version is ~4MB.

I think it is already the default setting for all modern server to enable gzip. A dev server usually does not enable this feature. However I didn't check the file size.

xenova · 2025-02-05T13:34:29Z

src/backends/onnx.js

-
-    // https://developer.mozilla.org/en-US/docs/Web/API/crossOriginIsolated
-    if (typeof crossOriginIsolated === 'undefined' || !crossOriginIsolated) {
-        ONNX_ENV.wasm.numThreads = 1;
-    }


Good to know! Thanks @fs-eire!

xenova · 2025-02-05T13:34:55Z

webpack.config.js

@@ -42,7 +42,7 @@ function buildConfig({
        type,
      },
      assetModuleFilename: "[name][ext]",
-      chunkFormat: "module",
+      chunkFormat: false,


Good find! Thanks!

fs-eire commented Jan 20, 2025

View reviewed changes

examples/extension/src/background.js Outdated Show resolved Hide resolved

fs-eire commented Jan 20, 2025

View reviewed changes

src/backends/onnx.js Outdated Show resolved Hide resolved

fs-eire commented Jan 20, 2025

View reviewed changes

webpack.config.js Outdated Show resolved Hide resolved

fs-eire commented Jan 20, 2025

View reviewed changes

src/backends/onnx.js Outdated Show resolved Hide resolved

fs-eire mentioned this pull request Jan 21, 2025

[js/web] ESM: use the bundled target as default export microsoft/onnxruntime#20991

Merged

fs-eire force-pushed the fix-export branch from a6873bc to 5447d28 Compare January 29, 2025 19:54

fs-eire changed the title ~~[DO NOT MERGE] fix package export~~ fix package export Jan 29, 2025

fs-eire marked this pull request as ready for review January 29, 2025 21:44

fs-eire mentioned this pull request Jan 29, 2025

[Web] Feature Request: workaround for bundlers when using onnxruntime-web microsoft/onnxruntime#22615

Closed

Upgrade package export

b4b0931

fs-eire force-pushed the fix-export branch from 5447d28 to b4b0931 Compare January 29, 2025 23:50

fs-eire mentioned this pull request Jan 30, 2025

Next.js example breaks with v3 #911

Closed

sroussey reviewed Feb 4, 2025

View reviewed changes

xenova reviewed Feb 5, 2025

View reviewed changes

set wasm path

85bde32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix package export #1161

fix package export #1161

fs-eire commented Jan 20, 2025 •

edited

Loading

fs-eire Jan 20, 2025

xenova Feb 5, 2025

fs-eire commented Jan 24, 2025

fs-eire commented Jan 29, 2025

xenova commented Jan 30, 2025

sroussey Feb 4, 2025

fs-eire Feb 4, 2025

sroussey Feb 5, 2025

xenova Feb 5, 2025

xenova left a comment

xenova Feb 5, 2025

fs-eire Feb 6, 2025

xenova Feb 5, 2025

xenova Feb 5, 2025

fix package export #1161

Are you sure you want to change the base?

fix package export #1161

Conversation

fs-eire commented Jan 20, 2025 • edited Loading

Summary

Problem

Solution

Why treat onnxruntime-web as external dependency

Tests

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fs-eire commented Jan 24, 2025

fs-eire commented Jan 29, 2025

xenova commented Jan 30, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xenova left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fs-eire commented Jan 20, 2025 •

edited

Loading

Why treat `onnxruntime-web` as external dependency