61

I want to compress a 16GB folder, but what's the best method? tar.gz? tar.bz2 rar? 7z? Would the archive be smaller if I first compressed in a method, then copy the compressed archive to a new folder, then re-compressed in some other method? I need to make it fit on a DVD (output maybe 8.5GB, don't remember) but putting "4370 MB" makes the compressed file be 2.5GB part.

BTW, what's the default compression method on Ubuntu?

3 Answers 3

80

The default is gz. The best results I get with 7z though.

Here is the results for a 1.4 Gb virtualbox container:

enter image description here

Best compression – size in MB:

7z 493
rar 523
bz2 592
lzh 607
gz 614
Z 614
zip 614
.arj 615
lzo 737
zoo 890

Source

enter image description here

Install

 sudo apt-get install p7zip-full
5
  • Thanks, I already installed 7z and rar via Software Center. I'll give a try with 7z.
    – Amanda
    Jan 5, 2013 at 16:32
  • 2
    Where do these results come from? For every algorithm, the compression ratio and speed depend very much on what you are trying to compress. Try compressing some data coming from /dev/urandom: you'll get different results in each try. Or try /dev/zero: bzip2 is the winner (for the ratio). Jan 5, 2013 at 16:46
  • 1
    @AndreaCorbellini the link is in the answer so click it!? and like I said: it's based on a 1.4 virtualbox container. I agree that compression depends a lot on what files it are but from a few years of experience 7z seems to be better for the files I tend to have(mostly software and binary data files)
    – Rinzwind
    Jan 5, 2013 at 18:00
  • 1
    @Rinzwind: I'm really sorry, I didn't see the Source link. Jan 5, 2013 at 18:06
  • 1
    @AndreaCorbellini it's ok ;) let's hope amanda reports back the results :+
    – Rinzwind
    Jan 5, 2013 at 18:10
15

This question is very old, but perhaps somebody finds this solution useful:

Use rzip, after tar. It first compresses 900 MB large data blocks using a dictionary method, and then it hands the cleaned-up data over to bzip2. It is much faster than the other strong compression tools (bzip2, lzma), and some files it compresses even better than bzip2 or lzma.

Yes, gz is the default compression tool on Linux. It is fast, and despite its age it gives still very good results in compressing text files like source code. Another standard tool is bzip2, though it is much slower.

Addition: lrzip is newer and extends the principle of rzip. It even supports unlimited block sizes, and a choice of compression methods (LZMA, Bzip2, Gzip, LZO, ZPAQ or none). LZMA is the standard. For backup or if you share much data with other Linux/BSD users, it can come in really handy.

2
3

I opt for a LZMA. It has smallest byte overhead and has strong compression ratio. Comparison between ZIP and LZMA: I've generated two files seq.txt with PHP code

$s = '0123456789'; $str = ''; for ($i=0; $i < 1000000; $i++) $str .= $s[$i%10].($i%10==9 ? "\n":""); file_put_contents('seq.txt', $str);

which holds repeating blocks of 0..9 digits ~ 1Mb of data and rnd.txt with PHP code

$s = '0123456789'; $str = ''; for ($i=0; $i < 1000000; $i++) $str .= $s[rand(0,9)].($i%10==9 ? "\n":""); file_put_contents('rnd.txt', $str);

which holds random blocks of 0..9 digits ~ 1Mb of data.

Compression results:

  • seq.txt, rnd.txt - 1100000 bytes
  • seq.txt.zip - 2502 bytes
  • rnd.txt.zip - 515957 bytes
  • seq.txt.lzma - 257 bytes
  • rnd.txt.lzma - 484939 bytes

Compression ratio:

  • ZIP       -> "seq.txt" -> 99.772%
  • ZIP       -> "rnd.txt" -> 53.094%
  • LZMA  -> "seq.txt" -> 99.976%
  • LZMA  -> "rnd.txt" -> 55.914%

So LZMA has compressed sequential data by 0.2% more effectively than ZIP
and random data 2.8% more effectively than ZIP.

For sure LZMA wins !

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .