There are two distinct but related tasks. Packing a tree of files
(including filenames, directory structure, filesystem permissions,
ownership and any other metadata) into a byte stream is called
archiving. Removing redundancy in a byte stream to produce a
smaller byte stream is called compression.
On Unix, the two operations are separated, with distinct tools for
each. On most other platforms (current and historical) combined tools
perform both archiving and compression.
(gzip and other programs that mimic gzip's interface often have the
option to store the original filename in the compressed output, but
this, along with a CRC or other check to detect corruption, is the
only metadata they can store.)
There are advantages to separating compression from archiving.
Archiving is platform-specific (the filesystem metadata needing
preserving varies widely), but the implementation is straightforward,
largely I/O-bound, and changes little over time. Compression is platform-independent, but implementations are CPU-bound
and algorithms are constantly improving to take advantage of the
increased resources that modern hardware can bring to bear on the
problem.
The most popular Unix archiver is tar
, although there exist others
such as cpio
and ar
. (Debian packages are ar
archives, while
cpio
is often used for inital ramdisks.) tar
is or has often been
combined with compression tools such as compress
(.Z), gzip
(.gz),
bzip2
(.bz2) and xz
(.xz), from oldest to youngest, and not
coincidentally from worst to best compression.
Making a tar
archive and compressing it are distinct steps: the
compressor knows nothing about the tar
file format. This means that
extracting a single file from a compressed tar
archive requires
decompressing all of the preceding files. This is often called a
"solid" archive.
Equally, since tar is a "streaming" format--required for it to be useful in a
pipeline--there is no global index in a tar archive, and listing the
contents of a tar archive is just as expensive as extracting it.
By contrast, Zip and RAR and 7-zip (the most popular archivers on
modern Windows platforms) usually compress each file separately, and
compress metadata lightly if at all. This allows for cheap listing of
the files in an archive and extraction of individual files, but
means that redundancy between multiple files in the same archive
cannot be exploited to increase compression. While in general
compressing an already-compressed file does not reduce file size
further, occasionally you might see a zip file within a zip file: the
first zipping turned lots of small files into one big file (probably
with compression disabled), which the second zipping then compressed
as a single entity.
There is cross-pollination between the differing platforms and
philosophies: gzip
is essentially zip
's compressor without its
archiver, and xz
is essentially 7-zip
's compressor without its
archiver.
There are other, specialized compressors. PPM variants and their
successor ZPAQ
are optimized for maximum compression without regard to
resource consumption. They can easily chew up as much CPU and RAM as
you can throw at them, and decompression is just as taxing as
compression (for contrast, most widely-used compression tools are
asymmetric: decompressing is cheaper than compressing).
On the other end of the spectrum, lzo
, snappy
and LZ4
are "light"
compressors designed for maximum speed and minimum resource
consumption, at the cost of compression. They're widely used within
filesystems and other object stores, but less so as standalone tools.
So which should you pick?
Archiving:
Since you're on Ubuntu there's no real reason to use anything other
than tar
for archiving, unless you're trying to make files that are
easily readable elsewhere.
zip
is hard to beat for ubiquity, but it's not Unix-centric and will
not keep your filesystem permissions and ownership information, and
its baked-in compression is antiquated. 7-zip and RAR (and ZPAQ) have
more modern compression but are equally unsuited to archiving Unix
filesystems (although there's nothing stopping you using them just as
compressors); RAR is also proprietary.
Compression:
For maximum compression you can have a look at a benchmark, such as the
enormous one at http://mattmahoney.net/dc/text.html. This should give
you a better idea of the tradeoffs involved.
You probably don't want maximum compression, though. It's way too
expensive.
xz
is the most popular general-purpose compression tool on modern Unix
systems. I believe 7-zip can read xz files too, as they are closely
related.
Finally: if you're archiving data for anything other than short-term
storage you should pick something open-source and preferably
widespread, to minimize headaches later on.