Script to extract text using grep

Question

I am trying to extract two text strings from multiple data files in a directory. I want to separate the text string using a ~ character and send the output to one single file.

Example directory name: /xmldocs AND the filenames are name of files: 1.xml, 2.xml, 3.xml

Expected Output:

TEXT_STRING_ONE~TEXT_STRING_TWO   ------- This is from 1.xml
TEXT_STRING_ONE~TEXT_STRING_TWO   ------- This is from 2.xml
TEXT_STRING_ONE~TEXT_STRING_TWO   ------- This is from 3.xml

SCRIPT used kind of works with only one grep command

for i in `ls`
do
grep "Document ID:" $i
echo "~" 
echo "\n"
done >> /xmldocs/1.txt

When I insert the 2nd grep command; grep 'CI[^"]' the script hangs and there is no output.

Current Output is

<KMS_data id="KMS_documentId" name="Document ID:">12</KMS_data>
~
<KMS_data id="KMS_documentId" name="Document ID:">19</KMS_data>
~
<KMS_data id="KMS_documentId" name="Document ID:">6</KMS_data>
~


grep "Document ID:" $i
~


grep "Document ID:" $i
~


grep "Document ID:" $i > /xmldocs/1.txt
~

Any help is appreciative. I am very very new with LINUX/UNIX commands and I need to extract the information from 20,000 xml files. Thanks.

Answer 1

The idea is process the result from grep and append it explicitly to the output file. This way you can use the console to write debug messages.

#/bin/bash

# Save output to this file
outputFile='./xmldocs/1.txt'
rm -f $outputFile

# List only *.xml files and iterate
for i in `ls *.xml`
do
    # Echo which file is being processed (only printed to console )
    echo 'Processing :'$i
    # Grep, remove trailing newline and append to $outputFile
    grep "Document ID:" -s $i | tr -d '\n'  >> $outputFile
    # Add char to separate
    printf "~" >> $outputFile
    # Grep, remove trailing newline and append to $outputFile
    grep 'CI[^"]' -s $i | tr -d '\n' >> $outputFile
    # Print newline to separate results
    printf "\n" >> $outputFile
done 

echo '!! done'

If this don't work please post the other line that you want to grep to test it.

Answer 2

What you want is paste:

#!/bin/bash
for f in *.xml
do
    paste -d '~' <(grep 'Document ID:' "$f") <(grep 'CI[\^"]' "$f")
done > /xmldocs/1.txt

Answer 3

Regarding why the script may be hanging with your use of grep 'CI[^"]', you should be escaping ^. Using grep 'CI[\^"]' fixed the issue for me. This is because the carrot symbol, even within the scope of brackets, is interpreted as negation.

EDIT: Steeldriver's correction