7

Searching for files using the bash find command, I would like to concatenate the resulting files into a new file. For example, if the find command yields:

find . -name "configuration_dev.txt"

./tmp/configuration1/configuration_dev.txt
./tmp/configuration2/configuration_dev.txt

I would like to concatenate the contents of the two files into a new file directly as a bash command.

1
  • I don't know really know how to do it, but you could try to use cat command which take find output, and do a redirect to a .txt file with > operand
    – damadam
    Oct 22, 2019 at 11:57

4 Answers 4

5

One possible way to do this is by piping the output of find to xargs and concatenate the content of the files by cat, then you can redirect the output to a new file:

find . -name '*file*' | xargs -I{} cat "{}" > output

The above command will call cat for each file and then the entire output of the xargs statement will be redirected to the output file. The more effective way is to use null delimiter - thanks to @pLumo for this update:

find . -name '*file*' -print0 | xargs -0 cat > output
2
  • 2
    This will call cat for each file, you could just use xargs -0 cat.
    – pLumo
    Oct 22, 2019 at 13:20
  • 4
    Don't forget xarg's -r flag. Running a command when no files were found is often Not Good.
    – studog
    Oct 22, 2019 at 22:29
4

The command to achieve the desired result is:

find . -name "configuration_dev.txt" -exec cat > testing.txt {} +

A good explanation of the above line is provided here: What is meaning of {} + in find's -exec command?

4
  • 1
    The > cat testing.txt part can be put anywhere in the command. I'd put it in the beginning or the end, feels more natural, but maybe it's just me... But anyways, this is the way to do it! +1
    – pLumo
    Oct 22, 2019 at 13:12
  • 1
    Add -type f to avoid error messages if your find includes directories.
    – pLumo
    Oct 22, 2019 at 13:22
  • This probably fails with a large number of files because cat > testing.txt {} will be called more than once, overriding the previous contents of the file each time.
    – danzel
    Oct 22, 2019 at 18:46
  • 3
    @danzel That's not a problem, it just looks like it would be, due to the confusing placement of > cat testing.txt. As pLumo says, it's clearer to put it at the beginning or end of the whole command. The key is, > cat testing.txt is not actually part of any command run by find. find never sees > cat testing.txt. The shell applies that redirection to the command find . -name "configuration_dev.txt" -exec cat {} +. No matter how many times find runs cat--as you say, with enough files this happens multiple times--testing.txt is opened only once, when the shell opens it. Oct 22, 2019 at 18:54
0

You can also open multiple files in cat and redirect the output to a new file. Since the names of multiple files would be returned by find, you can use that inside cat using `(backtick):

cat `find . -name "configuration_dev.txt"` > testing.txt

Command between backticks would be executed and replaced by the output of the command removing the trailing new line character and inserting a whitespace.

You can confirm this by running

A=`find . -name "configuration_dev.txt"`
echo $A

As you'll find that the output would be something like:

./tmp/configuration/configuration_dev.txt ./tmp/configuration/configuration_dev.txt

Thus the actual cat command under the hood will be:

cat ./tmp/configuration/configuration_dev.txt ./tmp/configuration/configuration_dev.txt > testing.txt
2
  • 2
    This fails with spaces somewhere along the path ...
    – pLumo
    Oct 22, 2019 at 13:20
  • @pLumo I understand. Thank you for the comment. As of now I have no idea to remove this limitation using the current approach. :)
    – Kulfy
    Oct 23, 2019 at 14:54
0

The Rust tool fd, which is a user-friendly alternative to find, makes this quite easy.

fd -X cat \; "configuration_dev.txt" . > output

-X is a batch executation of a command on all search results at once. The nice thing about this (since v8.4.0) is that it will not fail with the error Argument list too long.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .