Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I have a script which searches all files in multiple subfolders and archives to tar. My script is

for FILE in `find . -type f  -name '*.*'`
  do
if [[ ! -f archive.tar ]]; then

  tar -cpf archive.tar $FILE
else 
  tar -upf archive.tar $FILE 
fi
done

The find command gives me the following output

find . -type f  -iname '*.*'
./F1/F1-2013-03-19 160413.csv
./F1/F1-2013-03-19 164411.csv
./F1-FAILED/F2/F1-2013-03-19 154412.csv
./F1-FAILED/F3/F1-2011-10-02 212910.csv
./F1-ARCHIVE/F1-2012-06-30 004408.csv
./F1-ARCHIVE/F1-2012-05-08 190408.csv

But the FILE variable only stores first part of the path ./F1/F1-2013-03-19 and then the next part 160413.csv.

I tried using read with a while loop,

while read `find . -type f  -iname '*.*'`;   do ls $REPLY; done

but I get the following error

bash: read: `./F1/F1-2013-03-19': not a valid identifier

Can anyone suggest an alternative way?

Update

As suggested in the answers below I updated the scripts

#!/bin/bash

INPUT_DIR=/usr/local/F1
cd $INPUT_DIR
for FILE in "$(find  . -type f -iname '*.*')"
do
archive=archive.tar

        if [ -f $archive ]; then
        tar uvf $archive "$FILE"
        else
        tar -cvf $archive "$FILE"
        fi
done

The output that i get is

./test.sh
tar: ./F1/F1-2013-03-19 160413.csv\n./F1/F1-2013-03-19 164411.csv\n./F1/F1-2013-03-19 153413.csv\n./F1/F1-2013-03-19 154412.csv\n./F1/F1-2012-09-10 113409.csv\n./F1/F1-2013-03-19 152411.csv\n./.tar\n./F1-FAILED/F3/F1-2013-03-19 154412.csv\n./F1-FAILED/F3/F1-2013-03-19 170411.csv\n./F1-FAILED/F3/F1-2012-09-10 113409.csv\n./F1-FAILED/F2/F1-2011-10-03 113911.csv\n./F1-FAILED/F2/F1-2011-10-02 165908.csv\n./F1-FAILED/F2/F1-2011-10-02 212910.csv\n./F1-ARCHIVE/F1-2012-06-30 004408.csv\n./F1-ARCHIVE/F1-2011-08-17 133905.csv\n./F1-ARCHIVE/F1-2012-10-21 154410.csv\n./F1-ARCHIVE/F1-2012-05-08 190408.csv: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
share|improve this question
2  
It looks you should set IFS=$'\n' before the `for loop to make it parse by each line –  minerz029 Sep 10 '13 at 11:03

7 Answers 7

up vote 6 down vote accepted

Using for with find is the wrong approach here, see for example this writeup about the can of worms you are opening.

The recommended approach is to use find, while and read as described here. Below is an example that should work for you:

find . -type f -name '*.*' -print0 | while IFS= read -r -d '' file; do
    printf '%s\n' "$file"
done

This way you delimit the filenames with null (\0) characters, this means that variation in space and other special characters will not cause problems.

In order to update an archive with the files that find locates, you can pass its output directly to tar:

find . -type f -name '*.*' -printf '%p\0' | tar --null -uf archive.tar -T -

Note that you do not have to differentiate between if the archive exists or not, tar will handle it sensibly. Also note the use of -printf here to avoid including the ./ bit in the archive.

share|improve this answer
    
Thanks, it almost works. The only thing is its archiving the ./ as tar. ./.tar tar: ./archive.tar: file is the archive; not dumped –  Ubuntuser Sep 10 '13 at 11:07
    
@Ubuntuser You could add a simple check to see if [[ "$FILE" == "./" ]]; then continue –  minerz029 Sep 10 '13 at 11:17
    
@Ubuntuser: You can avoid the ./ bit with -printf see updated answer. However it should not make any difference if it is included or not as it just references the current directory. I also included an alternative find/tar combination that you might want to use. –  Thor Sep 10 '13 at 12:41
    
Thank you so much! it works perfectly. –  Ubuntuser Sep 10 '13 at 17:50

In addition to proper quoting, you can tell find to use a NULL separator, and then read and process the results in a while loop

while read -rd $'\0' file; do
    something with "$file"
done < <(find  . -type f -name '*.*' -print0)

This should handle any filenames that are POSIX-compliant - see man find

   -print0
          True; print the full file name on the standard output, followed by a null character (instead of the newline character that  -print  uses).   This  allows  file
          names that contain newlines or other types of white space to be correctly interpreted by programs that process the find output.  This option corresponds to the
          -0 option of xargs.
share|improve this answer
    
this is only solution that worked for me. Thanks. –  codefreak Nov 8 at 7:58

Try quoting the for loop like this:

for FILE in "`find . -type f  -name '*.*'`"   # note the quotation marks

Without quotes, bash doesn't handle spaces and newlines (\n) well at all...

Also try setting

IFS=$'\n'
share|improve this answer
    
+1 for $IFS. That descibes the separator character. –  Ray yesterday

This works and is simpler:

find . -name '<pattern>' | while read LINE; do echo "$LINE" ; done

Credit to Rupa (https://github.com/rupa/z) for this answer.

share|improve this answer

As minerz029 suggested, you need to quote the expansion of the find command. You also need to quote all the substitutions of $FILE in your loop.

for FILE in "$(find . -type f  -name '*.*')"
do
    if [ ! -f archive.tar ]; then
        tar -cpf archive.tar "$FILE"
    else 
        tar -upf archive.tar "$FILE" 
    fi
done

Note that the $() syntax should be preferred to the use of backticks; see this U & L question. I also removed the [[ keyword and replaced it by the [ command because it's POSIX.

share|improve this answer
    
About [[ and [, it seems that [[ is newer and supports more features like globbing and regex matching. [[ is only in bash, though, not sh –  minerz029 Sep 10 '13 at 10:01
    
@minerz029 Yes. That's what I'm saying. I don't know what you mean by [[ supports globbing. According to Greg's wiki, no globbing takes place inside [[. –  Joseph R. Sep 10 '13 at 10:05
    
Try [ "ab" == a? ] && echo "true" then [[ "ab" == a? ]] && echo "true" –  minerz029 Sep 10 '13 at 10:09
    
@minerz029 That's not globbing. These are regular expressions (loosely interpreted). This is not a glob because a* means "a followed by any number of characters" rather than "all files whose names start with a and have any number of characters afterward". Try [ ab = a* ] && echo true vs. [[ ab == a* ]] && echo true. –  Joseph R. Sep 10 '13 at 10:16
    
Ah well, [[ still does regular expressions whilst [ does not. Must have gotten confused –  minerz029 Sep 10 '13 at 10:18

Most answers here break if there is a newline character in the filename. I use the bash more then 15 years, but only interactive.

In Python you can us os.walk(): http://docs.python.org/2/library/os.html#os.walk

And the tarfile module: http://docs.python.org/2/library/tarfile.html#tar-examples

share|improve this answer

I think you may be better off using find's -exec option.

if [[ -f archive.tar ]]
then
   find . -type f -name '*.*' -exec tar -cpf archive.tar \{\} +
else
   find . -type f -name '*.*' -exec tar -upf archive.tar \{\} +
fi

Find then handles the quoting necessary on a per file basis for you, for spaces and newlines

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.