0

I am trying to extract two text strings from multiple data files in a directory. I want to separate the text string using a ~ character and send the output to one single file.

Example directory name: /xmldocs AND the filenames are name of files: 1.xml, 2.xml, 3.xml

Expected Output:

TEXT_STRING_ONE~TEXT_STRING_TWO   ------- This is from 1.xml
TEXT_STRING_ONE~TEXT_STRING_TWO   ------- This is from 2.xml
TEXT_STRING_ONE~TEXT_STRING_TWO   ------- This is from 3.xml

SCRIPT used kind of works with only one grep command

for i in `ls`
do
grep "Document ID:" $i
echo "~" 
echo "\n"
done >> /xmldocs/1.txt

When I insert the 2nd grep command; grep 'CI[^"]' the script hangs and there is no output.

Current Output is

<KMS_data id="KMS_documentId" name="Document ID:">12</KMS_data>
~
<KMS_data id="KMS_documentId" name="Document ID:">19</KMS_data>
~
<KMS_data id="KMS_documentId" name="Document ID:">6</KMS_data>
~


grep "Document ID:" $i
~


grep "Document ID:" $i
~


grep "Document ID:" $i > /xmldocs/1.txt
~

Any help is appreciative. I am very very new with LINUX/UNIX commands and I need to extract the information from 20,000 xml files. Thanks.

  • 1
    Could you please post a sample data file ? – Sergiy Kolodyazhnyy Jul 10 '15 at 3:37
  • As @Serg said, we need an input file example. – A.B. Jul 10 '15 at 4:54
1

The idea is process the result from grep and append it explicitly to the output file. This way you can use the console to write debug messages.

#/bin/bash

# Save output to this file
outputFile='./xmldocs/1.txt'
rm -f $outputFile

# List only *.xml files and iterate
for i in `ls *.xml`
do
    # Echo which file is being processed (only printed to console )
    echo 'Processing :'$i
    # Grep, remove trailing newline and append to $outputFile
    grep "Document ID:" -s $i | tr -d '\n'  >> $outputFile
    # Add char to separate
    printf "~" >> $outputFile
    # Grep, remove trailing newline and append to $outputFile
    grep 'CI[^"]' -s $i | tr -d '\n' >> $outputFile
    # Print newline to separate results
    printf "\n" >> $outputFile
done 

echo '!! done'

If this don't work please post the other line that you want to grep to test it.

  • Hi bistoco. Thanks very much. Your code worked very well. I have tweak a bit on the second grep statement. The idea is to extract text starting from CI and end till the next word is Summary. I'll do research on the net on how to do this. But as far as the code goes, it is working fine. Thanks again. – Kays Kay Jul 10 '15 at 1:11
0

What you want is paste:

#!/bin/bash
for f in *.xml
do
    paste -d '~' <(grep 'Document ID:' "$f") <(grep 'CI[\^"]' "$f")
done > /xmldocs/1.txt
0

Regarding why the script may be hanging with your use of grep 'CI[^"]', you should be escaping ^. Using grep 'CI[\^"]' fixed the issue for me. This is because the carrot symbol, even within the scope of brackets, is interpreted as negation.

EDIT: Steeldriver's correction

  • Hi aJetHorn. Thanks for the tip. I now realized why it hung. Cheers! – Kays Kay Jul 10 '15 at 1:12
  • 1
    Actually a leading caret within a character set or range acts as a negation operator (rather than a line anchor) i.e. [^"] means 'any character except ": you can avoid that interpretation just by reversing the order ["^] – steeldriver Jul 10 '15 at 2:20
  • Hi steeldriver, Thanks for the explanation. Cheers! – Kays Kay Jul 10 '15 at 3:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.