7zip backups

[freitagdavid]

Is your feature request related to a problem? Please describe.
I would love to be able to use lzma compression for my backups. I tend to find I get much better compression ratios and the compression is just way faster.

Describe the solution you'd like
There are two ways I see to solve this. You could use an lzma library to support lzma or you could give us an option to use a custom archiver and just list out the functionalities you need for us to input the needed commands. In a perfect world both would be great but I would very much be happy with either outcome.

Additional context
It never hurts to save a bit more space and I think this would be very beneficial in that goal.

[Dimensional]

For this, I don't know how hard or easy it would be, but at least 7zip provides an LZMA SDK on their website, which has CS libraries. From the looks of it, LZMA is just an optimized version of the LZ77 algorithm, which LZX is also a version of.

The unfortunate thing is there are no CS sources for LZMA2 in the SDK. So it might either require creating them from scratch, which might not be to difficult for someone with experience, or have to include a setting/option to allow the running of an external program such as 7zip. If the libraries can be made for CS, there would be added benefits of multithreading to speed compression. If it could be integrated into the program, would be nice, but supporting an external and packaged DLL/exe might be the only way to make it work.

[RevoLand]

From what i have know, the available LZMA SDK is only for compressing single file, not making archives.

The only useful library i could find for C# is SharpCompress. And we can only use Zip/Tar formats with None/Deflate/LZMA/PPMd compression formats. And Tar can only do 'None'.

So what you guys think? Switching to SharpCompress from default Windows zip would be better or not?

Also i have to mention that LZMA is really slow and PPMd is even slower as we can't get the full advantage of the compression format as @Dimensional mentioned.

For this, I don't know how hard or easy it would be, but at least 7zip provides an LZMA SDK on their website, which has CS libraries. From the looks of it, LZMA is just an optimized version of the LZ77 algorithm, which LZX is also a version of.

The unfortunate thing is there are no CS sources for LZMA2 in the SDK. So it might either require creating them from scratch, which might not be to difficult for someone with experience, or have to include a setting/option to allow the running of an external program such as 7zip. If the libraries can be made for CS, there would be added benefits of multithreading to speed compression. If it could be integrated into the program, would be nice, but supporting an external and packaged DLL/exe might be the only way to make it work.

This is doable but however we may limit this to advanced users and let them select 7z DLL/exe path so we will only be shipping the SLM exe, no license stuff.

New Task Manager UI

Yes, it is still compressing for more than 21 minutes...

Supported Format Table from SharpCompress

[RevoLand]

Just did some benchmarking with Human: Fall Flat - 1.61 GB

I would personally use Optimal.

[RevoLand]

How about LZ4 compression?

That looks like a much better option than having 1%~5% smaller archive size?

[Dimensional]

From what i have know, the available LZMA SDK is only for compressing single file, not making archives.

The only useful library i could find for C# is SharpCompress. And we can only use Zip/Tar formats with None/Deflate/LZMA/PPMd compression formats. And Tar can only do 'None'.

That's the reason many archive software around, including 7Zip, are supported on top of Tar. Tar's only purpose is to combine multiple files and folders into a single file, which is commonly known as a Tape ARchive, which another program, like Gzip, BZip2, 7Zip, etc, would all then compress on top of. Tar files aren't really meant to be used independently when making compressed archives. They only serve as a means to combine files into a single one for things like LZMA compression afterwards. One of the reasons you see the sources files in github releases, for example SLM's release page, has it recorded as source.tar.gz.

GZ in this example is used only to compress individual files, but can't package them all together. Tar handles that part first, followed by GZ.

Using an external program can possibly be done via auto detection, but again it would be an advanced option for experienced users or those who know how to read instructions to manually specify where the program is located. It's a good thing 7Zip supports argument passthru, and unless the LZMA2 sdk can be updated for C#, then it would be necessary to use 7zip externally to take full advantage of LZMA2.

At the moment the only C# program I can find that seems to be able to use LZMA2 is another Steam library program called Steam Backup Tool, but it packages a 7zip.dll, SevinZip++Lib.dll, and 7za.exe files with it. Either it managed to do some linking to the program or it's doing the same of just calling 7za externally and passing the files to compress as an argument. Another program that might be useful, if packaged together, is PeaZip. It's Open Source, under the GPLv3 license, so packaging it to use might not be a problem, and apparently it's CLI interface is a lot more intuitive than it's GUI.

How about LZ4 compression?

That looks like a much better option than having 1%~5% smaller archive size?

That looks like something that can be integrated, however it has been noted that LZ4 is slightly worse in compression to LZO compression, despite being the same speed in compression but faster decompressing. And LZO is worse compression to DEFLATE, in terms of output size. https://github.com/lz4/lz4

At the moment it seems most compression has the same issue, trading speed for compression size. The faster they compress, the larger the file will be, and if someone is compressing to a file they don't expect to be updated regularly, speed isn't really the matter here. But if you plan to update the archives a lot, or even access and extract from the archives frequently, then (de)compression speed is more important than the archive's size. Having the options for the user to pick from is better than not having the options.