Still empty content-encoding when it is not set

[Maxim-Filimonov]

We are experiencing empty content-type with v0.4.0 when using with middleman-s3_sync needed to downgrade all the way to 0.1.2 to fix it.

[rhomeister]

I can confirm this. Using:

• asset_sync version `1.1.0'
• fog version 1.30

Thanks @Maxim-Filimonov for suggesting a (temporary) fix to this problem. Really has giving me headaches with users complaining about botched CSS and JS.

[geemus]

@fcheung I know you have looked at this previously, any ideas where we might be going off the rails still? Thanks!

[fcheung]

Not really. I had a look at boto and it seems to implement as we did, and the aws ruby sdk doesn't implement the streaming upload stuff at all (it looks like it would read the entire file into memory in order to compute its sha). I opened a support ticket with AWS about it but haven't heard back yet.

On 16 Jun 2015, at 19:08, Wesley Beary [email protected] wrote:

@fcheung I know you have looked at this previously, any ideas where we might be going off the rails still? Thanks!

—
Reply to this email directly or view it on GitHub.

[fcheung]

AWS have confirmed to me that they can reproduce this, ie it isn't an issue with the fog implementation of this. No workarounds for now other than avoid this code path (ie use signature v2, or don't use the stream upload feature (in fog this is triggered if the body responds to :read)). They have asked me for more details of issues caused by the blank header - can someone chime in with that (exact symptoms, browsers affected etc)

[geemus]

@fcheung got it, thanks again for digging in on this.

[fcheung]

@rhomeister @Maxim-Filimonov if you have details on what symptoms end users are seeing that would be helpful to AWS. In the mean time passing aws_signature_version: 2 should fix things (unless connecting to a region that doesn't support v2)

[Maxim-Filimonov]

@fcheung It affects CDN we use - Cloudflare as it cannot determine encoding when content-encoding is set to empty string. I heard about other CDN's but I don't remember exact names

[rhomeister]

I'm also using Cloudflare with S3, but I'm not sure if Cloudflare is the cause of this problem. The symptoms are that the gzipped version gets served as the plain one, resulting in garbled CSS or JS. This happens in certain browsers, among which IE.

On 19 jun. 2015, at 10:32, tpaktop [email protected] wrote:

@fcheung It affects CDN we use - Cloudflare as it cannot determine encoding when content-encoding is set to empty string. I heard about other CDN's but I don't remember exact names

—
Reply to this email directly or view it on GitHub.

[fcheung]

Thanks for the info - I have passed it on

[oskarpearson]

I can also confirm this.

Is there any way to force aws_signature_version to 2 when using asset sync?

Gems:

Fog - 1.31.0
Fog-aws - 0.5.0

I am syncing assets with Gzip compression disabled. After doing this, the AWS console S3 page shows Content-Encoding with a blank value:

Interestingly, I can't save things on the AWS console with a blank value. After clicking save:

The URLS are then served with a blank content-encoding string:

$ curl -v 'https://s3-eu-west-1.amazonaws.com/whatever.css'
< HTTP/1.1 200 OK
...
< Content-Encoding:
< Last-Modified: Thu, 18 Jun 2015 17:16:10 GMT

[fcheung]

I had a quick look through the asset sync source and it didn't look like there was a way to change how the fog::storage instance is created, beyond the settings asset sync has provided. What effect is the blank content encoding having on the served assets?

[oskarpearson]

The main problem occurs when combined with something proxying the assets (specifically nginx).

(We store all assets in S3, but run our own proxy to respond with the assets from s3, rather than cloudfront or similar.)

As you can see from http://forum.nginx.org/read.php?2,257672,257672#msg-257672 nginx doesn't handle cases where the Content-Encoding is supplied but is blank. When nginx does dynamic compression of the asset when serving it, the resulting headers then look like this:

Content-Encoding:
Content-Encoding: whatever

This duplication breaks Safari terribly, since the content-encoding is now gzipped, and Safari uses the first header, and believes the original content to not be gzipped. Most other browsers appear to do the right thing.

It's possible to strip out the offending blank headers with nginx, as per followups to that thread. However, I don't think that returning a blank header is the right thing to do in the first place :)

[fcheung]

Thanks for the extra info!

On 23 Jun 2015, at 16:12, Oskar Pearson [email protected] wrote:

The main problem occurs when combined with something proxying the assets (specifically nginx).

(We store all assets in S3, but run our own proxy to respond with the assets from s3, rather than cloudfront or similar.)

As you can see from http://forum.nginx.org/read.php?2,257672,257672#msg-257672 nginx doesn't handle cases where the Content-Encoding is supplied but is blank. When nginx does dynamic compression of the asset when serving it, the resulting headers then look like this:

Content-Encoding:
Content-Encoding: whatever
This duplication breaks Safari terribly, since the content-encoding is now gzipped, and Safari uses the first header, and believes the original content to not be gzipped. Most other browsers appear to do the right thing.

It's possible to strip out the offending blank headers with nginx, as per followups to that thread. However, I don't think that returning a blank header is the right thing to do in the first place :)

—
Reply to this email directly or view it on GitHub.

[fcheung]

This fix should be in 0.7.1 and above if you want to give that version a whirl.

[github-actions]

This issue has been marked inactive and will be closed if no further activity occurs.

[geemus]

Closing as it sounds like it was fixed some time ago.