proxy issues with accept-encoding (zstd)

[wug-ge]

Environment

Node v22.1.0
[email protected]

Reproduction

Create new Project, add the following to nitro.config.ts:
routeRules: { "/api/**": { proxy: "https://jsonplaceholder.typicode.com/**" }, },

Describe the bug

I'm not sure what reproduces this, but it seems to be affected by some kind of race condition given the result is not always the same.

When I call http://localhost:3000/api/todos/1 with this setting, I get back

502 Bad Gateway
Bad Gateway

at createError (./node_modules/h3/dist/index.mjs:78:15)
...

or a 200 response with some kind of encoded string/binary data? (see attached screenshot)

In my tests, it seems to be happening ONLY with https requests, http requests work fine.

Additional context

Let me know if you need any more information.

Logs

No response

[gxres042]

This link can only get a vaild JSON data by ofetch:

// server/api/todos/[slug]/index.ts
export default defineEventHandler(async (event) => {
    const slug = getRouterParam(event, "slug");
    return await $fetch(
        `https://jsonplaceholder.typicode.com/todos/${slug}`
    );
});

I think it looks like the target link is blocking the proxy.

[pi0]

Issues is probably related to proxied encoding headers. using curl (to the same nitro route) also things work.

[wug-ge]

I did some debugging, interestingly requests to https://jsonplaceholder.typicode.com/todos are successful while those to https://jsonplaceholder.typicode.com/todos/1 (specific todo) fail.

I found out that removing "zstd" in the Accept-Encoding header in the proxy solves this issue, maybe this is helpful.

[pi0]

Came to the same result same time haha, yes we might need to filter accept-encoding in h3 sendProxy util. I'm wondering if it is limitation of Undici fetch (node.js fetch)...

[wug-ge]

Yes it seems like it: nodejs/node#48412

Looks like zstd is already added but not yet released.
Maybe it's best to just remove zstd from the header if it exists for now? (and when released check if the current NodeJS version supports it?)

[pi0]

I think safest is to ignore it altogether, similar cases might happen in the future / for other runtimes: unjs/h3#914

thanks for report ❤️