I am trying to extract two text strings from multiple data files in a directory. I want to separate the text string using a ~ character and send the output to one single file.

Example directory name: /xmldocs AND the filenames are name of files: 1.xml, 2.xml, 3.xml

Expected Output:

TEXT_STRING_ONE~TEXT_STRING_TWO   ------- This is from 1.xml
TEXT_STRING_ONE~TEXT_STRING_TWO   ------- This is from 2.xml
TEXT_STRING_ONE~TEXT_STRING_TWO   ------- This is from 3.xml

SCRIPT used kind of works with only one grep command

for i in `ls`
grep "Document ID:" $i
echo "~" 
echo "\n"
done >> /xmldocs/1.txt

When I insert the 2nd grep command; grep 'CI[^"]' the script hangs and there is no output.

Current Output is

<KMS_data id="KMS_documentId" name="Document ID:">12</KMS_data>
<KMS_data id="KMS_documentId" name="Document ID:">19</KMS_data>
<KMS_data id="KMS_documentId" name="Document ID:">6</KMS_data>

grep "Document ID:" $i

grep "Document ID:" $i

grep "Document ID:" $i > /xmldocs/1.txt

Any help is appreciative. I am very very new with LINUX/UNIX commands and I need to extract the information from 20,000 xml files. Thanks.

  • 1
    Could you please post a sample data file ? Commented Jul 10, 2015 at 3:37
  • As @Serg said, we need an input file example.
    – A.B.
    Commented Jul 10, 2015 at 4:54

3 Answers 3


The idea is process the result from grep and append it explicitly to the output file. This way you can use the console to write debug messages.


# Save output to this file
rm -f $outputFile

# List only *.xml files and iterate
for i in `ls *.xml`
    # Echo which file is being processed (only printed to console )
    echo 'Processing :'$i
    # Grep, remove trailing newline and append to $outputFile
    grep "Document ID:" -s $i | tr -d '\n'  >> $outputFile
    # Add char to separate
    printf "~" >> $outputFile
    # Grep, remove trailing newline and append to $outputFile
    grep 'CI[^"]' -s $i | tr -d '\n' >> $outputFile
    # Print newline to separate results
    printf "\n" >> $outputFile

echo '!! done'

If this don't work please post the other line that you want to grep to test it.

  • Hi bistoco. Thanks very much. Your code worked very well. I have tweak a bit on the second grep statement. The idea is to extract text starting from CI and end till the next word is Summary. I'll do research on the net on how to do this. But as far as the code goes, it is working fine. Thanks again.
    – Kays Kay
    Commented Jul 10, 2015 at 1:11

What you want is paste:

for f in *.xml
    paste -d '~' <(grep 'Document ID:' "$f") <(grep 'CI[\^"]' "$f")
done > /xmldocs/1.txt

Regarding why the script may be hanging with your use of grep 'CI[^"]', you should be escaping ^. Using grep 'CI[\^"]' fixed the issue for me. This is because the carrot symbol, even within the scope of brackets, is interpreted as negation.

EDIT: Steeldriver's correction

  • Hi aJetHorn. Thanks for the tip. I now realized why it hung. Cheers!
    – Kays Kay
    Commented Jul 10, 2015 at 1:12
  • 1
    Actually a leading caret within a character set or range acts as a negation operator (rather than a line anchor) i.e. [^"] means 'any character except ": you can avoid that interpretation just by reversing the order ["^] Commented Jul 10, 2015 at 2:20
  • Hi steeldriver, Thanks for the explanation. Cheers!
    – Kays Kay
    Commented Jul 10, 2015 at 3:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .