modifyGoogle.js from the examples fails with "Content Encoding Error"

[OnkelTem]

I'm using Firefox 99.0 (64-bit) on Linux Kubuntu 20.04

Running the example fails with this error:

when I open an URL like this one

Ideas?

[OnkelTem]

I added console.log() in the onResponseData() handler:

 ctx.onResponseData(function (ctx, chunk, callback) {
      console.log(chunk.toString());
      ...

and it prints this:

Y͘��J�#���1�8�!�:
�&�[��>�h����ɩm3w����|�m�N���Jc`}<}�~�X�P�Q�@���`FpE�Xr���zLf��q�}��fߖ8~ջij�X4�*�� ����...

So maybe it's the Proxy.gunzip middleware that doesn't work properly?

[OnkelTem]

I should say I installed only ca.pem certificate.
Not sure what else should I install.

[wawahuy]

try https://www.npmjs.com/package/pms-proxy

[OnkelTem]

Thanks @wawahuy
I switched to https://www.npmjs.com/package/mockttp

[vtulin]

Thanks @wawahuy I switched to https://www.npmjs.com/package/mockttp

Could you please give a short piece of code how to use it? I have the same issue

[OnkelTem]

Could you please give a short piece of code how to use it? I have the same issue

Sure! I had the same question :)

httptoolkit/mockttp#76

And also, I've almost finished a standalone proxying script which is using mockttp. I'm gonna upload a fresh version in an hour or two.

[vtulin]

httptoolkit/mockttp#76

It looks like mockttp depends on a module called vm2 which has a binary, this is not equal replacement as http-mitm-proxy is script only. For me that's important as I'm using webvpack.

[vtulin]

So maybe it's the Proxy.gunzip middleware that doesn't work properly?

I found the problem, that was so simple, we should take a look on content-encoding in response, there we may see that Google not using gzip, it uses br (brotli). So the partial solution is:

import zlib from 'zlib';
...
ctx.addResponseFilter(zlib.createBrotliDecompress());

instead of:

ctx.use(Proxy.gunzip);

Check node_modules\http-mitm-proxy\lib\middleware\gunzip.js.
The best solution would be to check content-encoding and add required decompressor if needed.

[OnkelTem]

Ok, I see you're solving a different task.
But since I promised to share, here is my proxy code: https://github.com/OnkelTem/wmod-proxy/blob/master/src/proxy.ts
(not finished or well tested)

[vtulin]

Ok, I see you're solving a different task. But since I promised to share, here is my proxy code: https://github.com/OnkelTem/wmod-proxy/blob/master/src/proxy.ts (not finished or well tested)

Thank you