Compression: Add support for Deflate64

[ianhays]

We allow decompression of Zip64 zips but not if they contain entries compressed with Deflate64. This is somewhat problematic, as zips created by the Windows explorer utility that are >2gb are compressed using Deflate64.

ZLib doesn't support deflate64, so it would need to be our own implementation.

[ianhays]

Windows 7/8/10 all use Deflate64. The cutoff in zip size where it switches from Deflate to Deflate64 appears to be 2 gigs. The number of files within the zip is irrelevant to the compression type chosen e.g. one 2.5 gig file will be Deflate64’ed the same as 5 .5 gig files.

I’m having trouble finding any good information on the details of the actual format since it’s proprietary by PKWare and not frequently used. What I’ve read makes it sound like it’s nearly the exact same as deflate minus a few tweaks. We might be able to modify our old managed Deflate implementation to also work with Deflate64. That would cause it to (somewhat ironically) be inferior to vanilla deflate in decompress time, but at least we would have some kind of support for decompression.

[ianhays]

One of our other options would be to add Adler's unsupported Deflate64 implementation to ClrCompression. That way we could avoid a non-performant managed implementation

The issue with that is it's a Windows-specific solution.

[stephentoub]

@ianhays, this can be closed now, right?

[ianhays]

@stephentoub yup!