Tell me more ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.
up vote 11 down vote favorite
2

I have a rather large file (~50GB) and it takes some time to run

tar xvf file.tar.bz2

on it. I'm aware of programs that can do parallel compression for bzip2 files but unaware of programs that can do parallel decompression for bzip2 files.

Are there any programs that can achieve this? What is the exact syntax of the command to use to extract from the file?

I'm using ubuntu 12.04

share|improve this question

2 Answers

up vote 9 down vote accepted

lbzip2 and pbzip2 are the tools which you can use for parallel compression and decompression.

Usage:

lbzip2 -d <file.tar.bz2> 
pbzip2 -d <file.tar.bz2>

-d option is used for decompression.

To install these packages:

lbzip2 Install lbzip2type:

sudo apt-get install pbzip2

pbzip2 Install pbzip2type:

sudo apt-get install pbzip2
share|improve this answer
So if I understand correctly, I need to decompress and then untar? Like 2 commands as opposed to tar xvf? – user784637 Nov 8 '12 at 7:21
1  
Yes when you run lbzip2 -d -n 2 file.tar.bz2 it will give a tar file. Which needs to be untarred. – devav2 Nov 8 '12 at 9:16
From the man page of pbzip2 (lbzip2 tells a similar story): "Files that are compressed with pbzip2 will also gain considerable speedup when decompressed using pbzip2. Files that were compressed using bzip2 will not see speedup since bzip2 packages the data into a single chunk that cannot be split between processors." – Tapio Nov 8 '12 at 10:34
1  
@Tapio Here is the Description for lbzip2 "Compress or decompress FILE operands or standard input to regular files or standard output, by calling Julian Seward's libbz2 from multiple threads. The lbzip2 utility employs multiple threads and an input-bound splitter even when decompressing .bz2 files created by standard bzip2 (but see BUGS below)." – devav2 Nov 8 '12 at 11:06
up vote 4 down vote

you can use pbzip2 with the -d flag to "decompress",

from the manpage: 

  pbzip2 -d myfile.tar.bz2

This example will decompress the file "myfile.tar.bz2" into the decompressed file "myfile.tar". It will use the autodetected # of processors (or 2 processors if autodetect not supported).

After decompressing, you need to untar the file with 

 tar xf myfile.tar

A tar file is just a container, to which you can apply multiple compression algorithms, for example, you can have a ".tar.gz" or a ".tar.bz2" which both have different compression algorithms applied. So pbzip2 will only uncompress the archive but it will not extract the files, use tar to extract the files. Tar shouldn't take long since the archive is already uncompressed and it will just extract the files. (note that we are Not using the 'z' flag or the 'j' flag in the tar command, which they indicate that we also want to decompress the file)

share|improve this answer
the manual page has some useful examples: manpages.ubuntu.com/pbzip2 – medigeek Nov 8 '12 at 6:07
@Sam Thanks for the answer - would you be able to answer the comment I left on the other answer – user784637 Nov 8 '12 at 8:53

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.