V3 - WebGPU Whisper in Chrome Extention

[chandeldivyam]

Question

Can webGPU accelerated whisper run in a chrome extension?

I checked the space and found the dependency "@xenova/transformers": "github:xenova/transformers.js#v3" which I imported in a chrome extension. When I tried to import it, it didn't work.

Module not found: Error: Can't resolve '@xenova/transformers' in 'D:\projects\mosaic8\browser-extension\src\utils'  
resolve '@xenova/transformers' in 'D:\projects\mosaic8\browser-extension\src\utils'
  Parsed request is a module
  using description file: D:\projects\mosaic8\browser-extension\package.json (relative path: ./src/utils)
    Field 'browser' doesn't contain a valid alias configuration
    resolve as module
      D:\projects\mosaic8\browser-extension\src\utils\node_modules doesn't exist or is not a directory
      D:\projects\mosaic8\browser-extension\src\node_modules doesn't exist or is not a directory
      D:\projects\mosaic8\browser-extension\node_modules doesn't exist or is not a directory
      looking for modules in D:\projects\mosaic8\node_modules
        single file module
          using description file: D:\projects\mosaic8\package.json (relative path: ./node_modules/@xenova/transformers)
            no extension
              Field 'browser' doesn't contain a valid alias configuration
              D:\projects\mosaic8\node_modules\@xenova\transformers is not a file
            .ts
              Field 'browser' doesn't contain a valid alias configuration
              D:\projects\mosaic8\node_modules\@xenova\transformers.ts doesn't exist
            .tsx
              Field 'browser' doesn't contain a valid alias configuration
              D:\projects\mosaic8\node_modules\@xenova\transformers.tsx doesn't exist
            .js
              Field 'browser' doesn't contain a valid alias configuration
              D:\projects\mosaic8\node_modules\@xenova\transformers.js doesn't exist
            .jsx
              Field 'browser' doesn't contain a valid alias configuration
              D:\projects\mosaic8\node_modules\@xenova\transformers.jsx doesn't exist
        existing directory D:\projects\mosaic8\node_modules\@xenova\transformers
          using description file: D:\projects\mosaic8\node_modules\@xenova\transformers\package.json (relative path: .)
            using exports field: ./dist/transformers.js
              using description file: D:\projects\mosaic8\node_modules\@xenova\transformers\package.json (relative path: ./dist/transformers.js)
                no extension
                  D:\projects\mosaic8\node_modules\@xenova\transformers\dist\transformers.js doesn't exist
                .ts
                  D:\projects\mosaic8\node_modules\@xenova\transformers\dist\transformers.js.ts doesn't exist       
                .tsx
                  D:\projects\mosaic8\node_modules\@xenova\transformers\dist\transformers.js.tsx doesn't exist      
                .js
                  D:\projects\mosaic8\node_modules\@xenova\transformers\dist\transformers.js.js doesn't exist       
                .jsx
                  D:\projects\mosaic8\node_modules\@xenova\transformers\dist\transformers.js.jsx doesn't exist      
                as directory
                  D:\projects\mosaic8\node_modules\@xenova\transformers\dist\transformers.js doesn't exist

I might be doing something I don't know maybe. What could the issue here be?

What I can understand is that it is trying to search for a ts/tsx/js/jsx file (as specified in the webpack.config.js and it is unable to get it.

[chandeldivyam]

Successfully able to import. Realized that we manually have to build it inside node modules. Link

Will check if this works as expected or not. Thanks!

[xenova]

Can you try npm i @huggingface/transformers and then replace the imports to @huggingface/transformers?

[chandeldivyam]

Tried it, that worked. But when we use 3.0.0-alpha.14 version, it shows similar error.

Currently, I was able to build and import without errors by going to node modules and installing packages and then building it.

Trying to implement an open source application for meeting note taker. Where we can record the meetings (with audio) and then later transcribe it and ask questions all locally in the extension. I think it would be possible with v2 with CPU maybe. Trying to figure out if we can utilize GPU. It would be very helpful as meetings could be 1-2 hours and on CPU it might be super long wait.

Realized 1 thing that we can not import @xenova/transformers in background service worker. It need document. maybe helpful for someone in future.

Thank you so much for such fast response @xenova. If you have any thoughts / ideas / suggestions please let me know.

[ggaabe]

@chandeldivyam did you ever figure out an import structure to make this work in the extensions? Struggling with this myself

[xenova]

I recently updated the browser extension example here: https://github.com/huggingface/transformers.js-examples/tree/main/browser-extension

Could you try that out and see if that fixes your issue?