2

I have a folder with over 3000 images, I want to compress 50 of them individually and store them in tar files.

The images are not serially named.

How to use write a bash script which does this job?

Improve this question
2
  • A bash script would do the job (eventually other types of solutions exist, like python). As it is difficult here to give a full course of bash programming, could you summarize your efforts and ask for more specific help?
    – Marc Vanhoomissen
    Jun 18, 2019 at 9:50
  • Having read the question and written an answer I think the question is clear and shouldn't be closed as unclear.
    – WinEunuuchs2Unix
    Jun 18, 2019 at 11:35

2 Answers 2

Reset to default
1

Let's assume your images are stored in ~/Pictures which is shorthand for /home/USER_ID/Pictures. Place the following commands in your home directory in a script file called CompressImages. Make the file executable using:

chmod a+x ~/CompressImages

Call the script using:

~/CompressImages

Copy the following script into the file ~/CompressImages:

#!/bin/bash

# Where to store Tars? We don't want in Images directory
COMPRESSED_DIR="$HOME/Downloads"
# TARs created with name below + "_999.tar"
TAR_NAME="CompressedImages"
# Directory Images are located in
FILES="$HOME/Pictures/*"

FileCount=50     # Number of files compressed into one Tar
TarCount=0       # Count of Tars created

i=0
for f in $FILES
do
    if [[ $(( i % FileCount )) -eq 0 ]] ; then
        let TarCount++
        TarName="$COMPRESSED_DIR/$TAR_NAME$TarCount.tar"
        tar -cvpf "$TarName" "$f"   # create .tar & add first file
    else
        tar -rvpf "$TarName" "$f"   # Add to existing tar
    fi
    let i++
done

After running CompressImages script check the results:

$ ll -h ~/Downloads/*.tar

-rw-rw-r-- 1 rick rick  14M Jun 18 16:55 Downloads/CompressedImages10.tar
-rw-rw-r-- 1 rick rick  48M Jun 18 16:55 Downloads/CompressedImages11.tar
-rw-rw-r-- 1 rick rick  16M Jun 18 16:55 Downloads/CompressedImages12.tar
-rw-rw-r-- 1 rick rick  55M Jun 18 16:55 Downloads/CompressedImages13.tar
-rw-rw-r-- 1 rick rick  45M Jun 18 16:55 Downloads/CompressedImages14.tar
-rw-rw-r-- 1 rick rick  43M Jun 18 16:55 Downloads/CompressedImages15.tar
-rw-rw-r-- 1 rick rick  37M Jun 18 16:55 Downloads/CompressedImages16.tar
-rw-rw-r-- 1 rick rick  38M Jun 18 16:55 Downloads/CompressedImages17.tar
-rw-rw-r-- 1 rick rick  44M Jun 18 16:55 Downloads/CompressedImages18.tar
-rw-rw-r-- 1 rick rick  47M Jun 18 16:55 Downloads/CompressedImages19.tar
-rw-rw-r-- 1 rick rick 180M Jun 18 16:55 Downloads/CompressedImages1.tar
-rw-rw-r-- 1 rick rick  53M Jun 18 16:55 Downloads/CompressedImages20.tar
-rw-rw-r-- 1 rick rick  33M Jun 18 16:55 Downloads/CompressedImages21.tar
-rw-rw-r-- 1 rick rick  13M Jun 18 16:55 Downloads/CompressedImages22.tar
-rw-rw-r-- 1 rick rick  26M Jun 18 16:55 Downloads/CompressedImages23.tar
-rw-rw-r-- 1 rick rick 1.4M Jun 18 16:55 Downloads/CompressedImages24.tar
-rw-rw-r-- 1 rick rick  13M Jun 18 16:55 Downloads/CompressedImages2.tar
-rw-rw-r-- 1 rick rick  15M Jun 18 16:55 Downloads/CompressedImages3.tar
-rw-rw-r-- 1 rick rick  23M Jun 18 16:55 Downloads/CompressedImages4.tar
-rw-rw-r-- 1 rick rick  11M Jun 18 16:55 Downloads/CompressedImages5.tar
-rw-rw-r-- 1 rick rick  12M Jun 18 16:55 Downloads/CompressedImages6.tar
-rw-rw-r-- 1 rick rick  20M Jun 18 16:55 Downloads/CompressedImages7.tar
-rw-rw-r-- 1 rick rick 7.4M Jun 18 16:55 Downloads/CompressedImages8.tar
-rw-rw-r-- 1 rick rick  21M Jun 18 16:55 Downloads/CompressedImages9.tar
Improve this answer
4
  • After creating a bash file and editing the COMPRESSED_DIR, TAR_NAME and FILES variables. I encounter the following error user:~/catkin_ws/src$ bash CompressImages.sh tar: $/home/user/catkin_ws/src/CompressedImages1.tar: Cannot open: No such file or directory tar: Error is not recoverable: exiting now
    – Pradeep BV
    Jun 19, 2019 at 7:52
  • Change your variable from $/home/user/catkin_ws/src/ to /home/user/catkin_ws/src/
    – WinEunuuchs2Unix
    Jun 19, 2019 at 10:33
  • Working Good, thank you for the Prompt reply
    – Pradeep BV
    Jun 19, 2019 at 10:38
  • You're welcome. As a new user you may not know it is customary to click the grey check mark next to the answer to accept it. Thanks.
    – WinEunuuchs2Unix
    Jun 19, 2019 at 10:40
0

You can do that using parallel -l 50:

cd /path/to/images

find . -maxdepth 1 -type f -print0 \
| parallel -0 -l 50 -j 1 tar cvpzf {#}.tar.gz {}

Add -iname "*.jpg" or similar to the find command if there are other files, too.

Leave out z option from tar and name it .tar instead of tar.gz if your images are already compressed (which would be the case with e.g. jpg).

Explanation:

  • find -print0 sends null-delimited data
  • parallel -0 reads null-delimited data
  • -l 50 read 50 lines for each execution
  • -j 1 executes one task in parallel, you can change it to what works best for you (depends on how many CPU Threads you have and how fast your I/O is).
  • {#} will be replaced by the sequence number (0-n)
  • {} wil be replaced by the 50 file names.

You can then move/rename *.tar.gz or *.tar to your liking.

This will be way faster than calling tar for every file adding it to an existing tar.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.