I have a folder with over 3000 images, I want to compress 50 of them individually and store them in tar files.
The images are not serially named.
How to use write a bash script which does this job?
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It only takes a minute to sign up.
Sign up to join this communityI have a folder with over 3000 images, I want to compress 50 of them individually and store them in tar files.
The images are not serially named.
How to use write a bash script which does this job?
Let's assume your images are stored in ~/Pictures
which is shorthand for /home/USER_ID/Pictures
. Place the following commands in your home directory in a script file called CompressImages
. Make the file executable using:
chmod a+x ~/CompressImages
Call the script using:
~/CompressImages
Copy the following script into the file ~/CompressImages
:
#!/bin/bash
# Where to store Tars? We don't want in Images directory
COMPRESSED_DIR="$HOME/Downloads"
# TARs created with name below + "_999.tar"
TAR_NAME="CompressedImages"
# Directory Images are located in
FILES="$HOME/Pictures/*"
FileCount=50 # Number of files compressed into one Tar
TarCount=0 # Count of Tars created
i=0
for f in $FILES
do
if [[ $(( i % FileCount )) -eq 0 ]] ; then
let TarCount++
TarName="$COMPRESSED_DIR/$TAR_NAME$TarCount.tar"
tar -cvpf "$TarName" "$f" # create .tar & add first file
else
tar -rvpf "$TarName" "$f" # Add to existing tar
fi
let i++
done
After running CompressImages
script check the results:
$ ll -h ~/Downloads/*.tar
-rw-rw-r-- 1 rick rick 14M Jun 18 16:55 Downloads/CompressedImages10.tar
-rw-rw-r-- 1 rick rick 48M Jun 18 16:55 Downloads/CompressedImages11.tar
-rw-rw-r-- 1 rick rick 16M Jun 18 16:55 Downloads/CompressedImages12.tar
-rw-rw-r-- 1 rick rick 55M Jun 18 16:55 Downloads/CompressedImages13.tar
-rw-rw-r-- 1 rick rick 45M Jun 18 16:55 Downloads/CompressedImages14.tar
-rw-rw-r-- 1 rick rick 43M Jun 18 16:55 Downloads/CompressedImages15.tar
-rw-rw-r-- 1 rick rick 37M Jun 18 16:55 Downloads/CompressedImages16.tar
-rw-rw-r-- 1 rick rick 38M Jun 18 16:55 Downloads/CompressedImages17.tar
-rw-rw-r-- 1 rick rick 44M Jun 18 16:55 Downloads/CompressedImages18.tar
-rw-rw-r-- 1 rick rick 47M Jun 18 16:55 Downloads/CompressedImages19.tar
-rw-rw-r-- 1 rick rick 180M Jun 18 16:55 Downloads/CompressedImages1.tar
-rw-rw-r-- 1 rick rick 53M Jun 18 16:55 Downloads/CompressedImages20.tar
-rw-rw-r-- 1 rick rick 33M Jun 18 16:55 Downloads/CompressedImages21.tar
-rw-rw-r-- 1 rick rick 13M Jun 18 16:55 Downloads/CompressedImages22.tar
-rw-rw-r-- 1 rick rick 26M Jun 18 16:55 Downloads/CompressedImages23.tar
-rw-rw-r-- 1 rick rick 1.4M Jun 18 16:55 Downloads/CompressedImages24.tar
-rw-rw-r-- 1 rick rick 13M Jun 18 16:55 Downloads/CompressedImages2.tar
-rw-rw-r-- 1 rick rick 15M Jun 18 16:55 Downloads/CompressedImages3.tar
-rw-rw-r-- 1 rick rick 23M Jun 18 16:55 Downloads/CompressedImages4.tar
-rw-rw-r-- 1 rick rick 11M Jun 18 16:55 Downloads/CompressedImages5.tar
-rw-rw-r-- 1 rick rick 12M Jun 18 16:55 Downloads/CompressedImages6.tar
-rw-rw-r-- 1 rick rick 20M Jun 18 16:55 Downloads/CompressedImages7.tar
-rw-rw-r-- 1 rick rick 7.4M Jun 18 16:55 Downloads/CompressedImages8.tar
-rw-rw-r-- 1 rick rick 21M Jun 18 16:55 Downloads/CompressedImages9.tar
$/home/user/catkin_ws/src/
to /home/user/catkin_ws/src/
Jun 19, 2019 at 10:33
You can do that using parallel -l 50
:
cd /path/to/images
find . -maxdepth 1 -type f -print0 \
| parallel -0 -l 50 -j 1 tar cvpzf {#}.tar.gz {}
Add -iname "*.jpg"
or similar to the find command if there are other files, too.
Leave out z
option from tar
and name it .tar
instead of tar.gz
if your images are already compressed (which would be the case with e.g. jpg
).
Explanation:
find -print0
sends null-delimited dataparallel -0
reads null-delimited data-l 50
read 50 lines for each execution-j 1
executes one task in parallel, you can change it to what works best for you (depends on how many CPU Threads you have and how fast your I/O is).{#}
will be replaced by the sequence number (0-n){}
wil be replaced by the 50 file names.You can then move/rename *.tar.gz
or *.tar
to your liking.
This will be way faster than calling tar
for every file adding it to an existing tar.