Using google-cloud/vertexai in edge environments

[lgrammel]

Environment details

• Programming language: JavaScript / TypeScript
• OS: Targeting edge environments (cloudflare, vercel)
• Package version: 0.1.3

Background

I'm implementing Gemini support for the Vercel AI SDK ( https://github.com/vercel/ai/pull/863/files ). However, when I set the runtime to 'edge', I run into the error below. My investigation showed that the auth library has several dependencies that are not supported in edge environments. Are there any guides on how to use the Vertex library in an edge environment, e.g. for streaming AI responses?

Steps to reproduce

1. Try to use the library in a edge environment (cloudflare, vercel)
2. Observe the following error:

[sararob]

You can use a bundler to remove https-proxy-agent, but depending on the environment where you're running this you may need to reimplement the crypto interface. If you do need to reimplement that piece, feel free to open a PR on google-auth-library.

[lgrammel]

@sararob thanks for the quick answer. I've tried removing https-proxy-agent and configured various other related settings in the next config, but it seems that the proxy agent is a hard requirement. I'll continue to try to get it to work, was just curious if there are some working examples or guides for this setup.

next.config.js (for reference)

/** @type {import('next').NextConfig} */
const nextConfig = {
  webpack: (config, { isServer, webpack }) => {
    config.plugins.push(
      new webpack.IgnorePlugin({ resourceRegExp: /https-proxy-agent/ }),
    );

    config.resolve.fallback = {
      buffer: false,
      crypto: false,
      events: false,
      fs: false,
      http: false,
      https: false,
      net: false,
      os: false,
      path: false,
      querystring: false,
      stream: false,
      tty: false,
      util: false,
      zlib: false,
      child_process: false,
      tls: false,
    };

    return config;
  },
};

module.exports = nextConfig;

error (for reference)

 ⨯ ../node_modules/.pnpm/[email protected]/node_modules/gaxios/build/src/gaxios.js (27:0) @ <unknown>
 ⨯ Cannot find module 'https-proxy-agent'
null

[alexander-fenster]

Try using null-loader as we do in https://github.com/googleapis/google-auth-library-nodejs/blob/7ae4aaea0311de1a3e10f4e86a1ef93ff00656b5/webpack.config.js#L43-L46. It will replace the module with an empty thing, and will not fail unless it's called (and it won't be called). We don't have a good test coverage for browsers (it's not an officially supported use case), but we run some tests in browser with webpack and karma in some other repos (e.g. in google-gax) and they work with this config.

[lgrammel]

@alexander-fenster thanks - I've given it a try, but ran into other issues as well and ended up needing to disable more items, which eventually led to the route not being available. We have implemented a solution with the generative ai sdk in the meantime, which works on the edge.

Abandoned PR for reference: https://github.com/vercel/ai/pull/863/files