Condition that matches all occurrences of a word in a file ad does something

Question

I am to do something like this

for (( every occurrence of the word TRAP-TYPE in a file )) 
do 
desc="$(< inputfile awk '/DESCRIPTION/ {getline; gsub(/^\s*"/, ""); gsub(/"\s*$/, ""); print}')"

casenum="$(< inputfile awk '/::=/ {gsub(/^\s*::=\s*/, ""); gsub(/\s*$/, ""); print}')"

echo $desc $numvar $casenum 
done

It will search for every occurrence of TRAP-TYPE. Any language will do!

INPUT

sCSISmart20 TRAP-TYPE
ENTERPRISE cyclone
DESCRIPTION
"Aspi: unable to read the file server hard disk might have problems"
--#TYPE "Aspi: unable to read the database file"
--#SUMMARY "ASPI: unable to read the file, server hard disk may have problems"
--#ARGUMENTS {}
--#SEVERITY WARNING
--#TIMEINDEX 100
--#STATE OPERATIONAL
--#HELP "scsismrt.hlp"
--#HELPTAG 124
::= 124

sCSISmart21 TRAP-TYPE
ENTERPRISE cyclone
DESCRIPTION
"Aspi: database is corrupted"
--#TYPE "Aspi: database is corrupted"
--#SUMMARY "ASPI: database file is corrupted"
--#ARGUMENTS {}
--#SEVERITY WARNING
--#TIMEINDEX 100
--#STATE OPERATIONAL
--#HELP "scsismrt.hlp"
--#HELPTAG 125 ::= 125

sCSISmart12 TRAP-TYPE
ENTERPRISE cyclone
VARIABLES {cycHostAdapterNumber, cycScsiTargetID, cycLun, cycVendor, cycProduct, cycSenseInfo}
DESCRIPTION
"The HostAdapter# %d, TargetID %d, Lun# %d has Predictive Failure Condition on vendor %s product %s with sense info MSB(sense code), next 8 bits (sense code Qual) next 8 bits (Add sense code Qual) LSB (0000) %d"
--#TYPE "Device has SMART/Predicictive failure event"
--#SUMMARY "The HostAdapter# %d , TargetID %d, Lun# %d has Predictive Failure Condition on vendor %s product %s with senseinfo %d"
--#ARGUMENTS {0,1,2,3,4,5}
--#SEVERITY INFORMATIONAL
--#TIMEINDEX 100
--#STATE OPERATIONAL
--#HELP "scsismrt.hlp"
--#HELPTAG 116
::= 116

OUTPUT

Aspi: unable to read the file server hard disk might have problems             
124
Aspi: database is corrupted            
125

Sorry for the misunderstanding. I am plagued by bad internet connectivity.

So you want to store the total number of occurences of the word and execute N times a statment right? — kos, Jun 15, 2015 at 12:44
what command do you want to execute? Im writing the program. — oOpSgEo, Jun 15, 2015 at 12:53
@SamFlynn Can we have more information on how the output and input should look like ? Is TRAP-TYPE exact string you want to search for, or it's just placeholder for some other things ? please clarify your specifications so that we can adjust answers accordingly — Sergiy Kolodyazhnyy, Jun 15, 2015 at 13:37
@Serg and kos I have made changes. Sorry for the misunderstanding. — SamFlynn, Jun 15, 2015 at 14:19
One thing, I posted an answer to this however when processing text files whitespaces matter. So if you want an accurate and working answer make sure that the sample input reflects exactly the file you're going to process, i.e. it has the same number of spaces / tabs at the start, in the middle and at the end of each line, otherwise answers may become useless. — kos, Jun 15, 2015 at 16:05

Sergiy Kolodyazhnyy · Accepted Answer · 2015-06-16 16:35:35Z

WORK IN PROGRESS

Extracting data itself is easy:

awk '{ if($0~/DESCRIPTION/){getline;print $0}; if($0~/::=/) print $2}' testfile

Running that with testfile that contains input text you've posted , gives this output:

$ awk '{if($0~/DESCRIPTION/){getline;print $0}; if($0~/::=/) print $2}' testfile                      
    "Aspi: unable to read the file server hard disk might have problems"
124
    "Aspi: database is corrupted"
125

If we're gonna have multiple files there, the code can be edited like so:

$ awk 'FNR==1{print FILENAME"\n========"} { if($0~/DESCRIPTION/){getline;print $0}; if($0~/::=/) print $2}' *.test                  
file1.test
========
    "Aspi: unable to read the file server hard disk might have problems"
124
    "Aspi: database is corrupted"
125
file2.test
========
    "Aspi: second file"
134
    "Aspi: i love awk"
135

That's just for extracting data. I will keep on editing this answer to include how you can assign exctracted data to variables.

One approach is to use awk's system function, that will allow running shell commands with variables passed by awk. In this function, the command has to be within double quotes, and awk's internal variables outside the quotes. For example something like this :

awk '{ if($0~/DESCRIPTION/){getline;printf $0"|"}; if($0~/::=/) printf $2"\n"}' *.test | awk -F'|' '{ STRING=$1;NUM=$2; system("echo this is the NUMBER "NUM" and this is the TEXT  "STRING)   }'

Output:

this is the NUMBER 124 and this is the TEXT Aspi: unable to read the file server hard disk might have problems
this is the NUMBER 125 and this is the TEXT Aspi: database is corrupted
this is the NUMBER 134 and this is the TEXT Aspi: second file
this is the NUMBER 135 and this is the TEXT Aspi: i love awk

One possible way of assigning output to variables would be with two parralel arrays.

$   IFS="|"; STRING_ARRAY=($(awk ' /DESCRIPTION/ {getline;printf "%s|",$0}; /::=/ { printf $2"\n" }' trapfile.txt | awk -F'|' '{printf "%s|",$1}'))

$ echo ${STRING_ARRAY[*]}
    "Aspi: unable to read the file server hard disk might have problems"     "Aspi: database is corrupted"     "Aspi: second file"     "Aspi: i love awk"

There, I've used internal field separator IFS | and created strings array as processed by the two awk commands. Now that STRING_ARRAY can be used in a for loop. To output the appropriate numbers to an array:

$ IFS="|"; NUMS_ARRAY=($(awk ' /DESCRIPTION/ {getline;printf "%s|",$0}; /::=/ { printf $2"\n" }' trapfile.txt | awk -F'|' '{printf "%s|",$2}'))

$ echo ${NUMS_ARRAY[*]}
124 125 134 135

Now we have two parrallel arrays, with each index matching string and number from each occurrence in any file

Notice that the code in the first pipe is the same, so we can simplify this, by making an awk script:

#!/usr/bin/awk -f
# Author: SergKolo
# Date: June 16,2015
# Written for: http://askubuntu.com/q/636705/295286
# Awk script to extract text
# between two specific strings
# in a file


{

          if($0~/DESCRIPTION/)
          {
            getline;printf "%s|",$0
          }; 

          if($0~/::=/) { printf $2"\n" }

}

Save that script in a file with some name , chmod +x scriptname.awk. Now those long commands simplify as:

$ IFS="|"; LINES_ARRAY=($(trap-script.awk trapfile.txt | awk -F'|' '{printf "%s|",$1}' ))

And

$ IFS="|"; NUMBERS_ARRAY=($(trap-script.awk trapfile.txt | awk -F'|' '{printf "%s|",$2}' ))

Note: because out of our conversation in the comments it became apparent that some of your files contain lines that have %d and %s format , characters, I've included printf "%s|",$0 likes in the code for a reason. When printf function expands a line $0 and sees those format characters it assumes that there must be input for them, instead of treating it as one long string. printf "%s|",$0 allows treating those %d characters as text, not as something that needs input.

The first comand you gave me works perfectly. I was planning on storing these values line by line in a 2-D array. I just need to know all of this being printed how can I store them in a variable? — SamFlynn, Jun 15, 2015 at 15:31
The problem is that because of the multiple occurences of the TRAP-TYPE in every single file, you can store the one string and one number in a variable, but I dont know how to approach storing second occurence. I'd say the last command that pipes from awk to awk can be used to store everything into an arrray with the say system command. I just need to think how to approach this. That's definitely feasible though — Sergiy Kolodyazhnyy, Jun 15, 2015 at 15:37
Speaking of 2d arrays, bash does not support multi-dimensional arrays. They can be simulated, but that's considered not quite an optimal solution.I suggest using two , parallel arrays. — Sergiy Kolodyazhnyy, Jun 15, 2015 at 15:43
@SamFlynn I've added some corrections to the last two pieces of code that store output into array. I've tested them with the examples you've posted, and everything works. The issue was that we needed printf "%s" on both first and second awk command (before and after pipe). Please review and test, let me know if everything works as expected. — Sergiy Kolodyazhnyy, Jun 16, 2015 at 15:31
I've also simplified the code by a lot. There were some unnecessary bits. Let me know if I can remove the WORK IN PROGRESS tag, if you find this answer as working and if this answer satisfies your requirements, please accept it. Also, let me know if I may shorten the text only up to the script and the last two lines — Sergiy Kolodyazhnyy, Jun 16, 2015 at 16:04

muru · Accepted Answer · 2015-06-15 17:48:17Z

Another awk solution:

$ gawk -F '\n *' -v RS="::=[^\n]*\n*" '{gsub(/[^0-9]/,"", RT); printf "%s\n%s\n", $4, RT}' foo
"Aspi: unable to read the file server hard disk might have problems"                           
124
"Aspi: database is corrupted"                                                  
125

The text naturally divides into records, but the record separator is not very straightforward. It could be two consecutive newlines. I decided to use the last field in the record (::= ...), followed by arbitrarily many newlines, as the record separator.

Then, for splitting on fields, I used a newline followed by arbitrarily many spaces. After that, it's just a matter of printing the fourth field, and extracting the numbers from the record separator text (RT). Because this solution uses RT, it is GNU awk-only.

oOpSgEo · Accepted Answer · 2015-06-15 13:12:12Z

In your scenario, the occurences of a word could be found with awk, grep, and sed; but you would have to incase those in a bash script with some a case statement and an iteration. It can be done. Writing a Perl, Python, Java, or C++ program seems like a better way to go about it.

I just wrote this for you. This is a c++ program. Copy and paste the program below into gedit. Save it as findword.cpp. NOTE: you need to build-essential to compile it.

:~$ sudo apt-get install build-essential -y

:~$ g++ -o findword findword.cpp

The program:

#include <iostream>
#include <fstream>
#include <string>
#include <cstdlib>
#include <locale>

using namespace std;

int main()
{
    string wordhold = "";
    int index;
    string line;
    string trptype = "TRAP-TYPE";

    ifstream infile;
    infile.open("Yourfile.txt"); // change Yourfile.txt to the name of the file

    while ( getline(infile, line) ) {

        index = line.find('\n');
        string holdword = line.substr(0,index);
        wordhold = holdword;

        if ( wordhold.compare(wordhold.size(),9,trptype) == 0 ) {

            system("Execute the command you want inside the quotes");
            wordhold = "";

        }
        else {

            wordhold = "";

        }

    }
    infile.close();

    cout << "The file is closed\nDone" << endl;

    return 0;   
}

By the you can run multiple commands by duplicating the system function that is in the condition statement (if).

To run the program enter the directory you compiled in and execute it from command line.

:~$ ./findword

Good luck.

I'm no C++ expert, does this cater for multiple occurences on the same line? However something that jumps to the eye is the fact that you're assigning an empty string to wordhold both in the if and in the else body, and that the else body is empty, so perhaps you can drop the else body and make an unique assignment outside of the if body — kos, Jun 15, 2015 at 13:31

kos · Accepted Answer · 2015-06-15 18:01:50Z

Using awk:

< inputfile awk '/DESCRIPTION/ {getline; sub(/^ *"/,""); sub(/"$/,""); print}; /::=/ {sub(/::= */,""); print}'

Expanded version:

< inputfile awk '
    /DESCRIPTION/ {
        getline;
        sub(/^ *"/,"");
        sub(/"$/,"");
        print
    };
    /::=/ {
        sub(/^::= */,"");
        print
    }
'

/DESCRIPTION/: selects only the records containing the string DESCRIPTION
getline: skips the first record
sub(/^ *"/,""): substitute a string composed by any number of spaces followed by a " character, at the start of the line, with an empty string
sub(/"$/,""): substitute a " character, at the end of the line, with an empty string
print: prints the record
/::=/: selects only the records containing the string ::=
sub(/^::= */,""): substitute a string composed by ::= followed by any number of spaces, at the start of the line, with an empty string
print: prints the record

A.B. · Accepted Answer · 2015-06-16 11:56:11Z

My awk version:

awk '/TRAP-TYPE/ {traptype=1}; traptype && /DESCRIPTION/ {getline; gsub(/^ +/, "", $0); gsub(/^\"/, "", $0); gsub(/\"\s+$/, "", $0); print}; traptype && /::=/ {print $2}' testfile

or more specific

awk '/sCSISmart.*TRAP-TYPE/ {traptype=1}; traptype && /DESCRIPTION/ {getline; gsub(/^ +/, "", $0); gsub(/^\"/, "", $0); gsub(/\"\s+$/, "", $0); print}; traptype && /::=/ {print $2}' testfile

Input

testfile

sCSISmart20 TRAP-TYPE                                  
    ENTERPRISE  cyclone                                                  
    DESCRIPTION                                          
    "Aspi: unable to read the file server hard disk might have problems"                           
    --#TYPE "Aspi: unable to read the database file"                                 
    --#SUMMARY "ASPI: unable to read the file, server hard disk may have                                                              problems"                                     
    --#ARGUMENTS {}                                 
    --#SEVERITY WARNING                                         
    --#TIMEINDEX 100                                                    
    --#STATE OPERATIONAL                                          
    --#HELP "scsismrt.hlp"                                                      
    --#HELPTAG 124                         
::=  124                                                                                            

sCSISmart21 TRAP-TYPE                                     
    ENTERPRISE  cyclone                                                
    DESCRIPTION                                                      
    "Aspi: database is corrupted"                                                  
    --#TYPE "Aspi: database is corrupted"                                          
    --#SUMMARY "ASPI: database file is corrupted"                                              
    --#ARGUMENTS {}                                                             
    --#SEVERITY WARNING                   
    --#TIMEINDEX 100                           
    --#STATE OPERATIONAL                              
    --#HELP "scsismrt.hlp"                        
    --#HELPTAG 125
::=  125

Output

Aspi: unable to read the file server hard disk might have problems
124
Aspi: database is corrupted
125

Separate outputs

awk '/TRAP-TYPE/ {traptype=1}; traptype && /DESCRIPTION/ {getline; gsub(/^ +/, "", $0); gsub(/^\"/, "", $0); gsub(/\"\s+$/, "", $0); print;}' testfile

Aspi: unable to read the file server hard disk might have problems
Aspi: database is corrupted

awk '/sCSISmart.*TRAP-TYPE/ {traptype=1}; traptype && /::=/ {print $2}' testfile

124
125

Okay printing part is alright, I am still to store it in a array, ie, in one array the description and in other array the number. — SamFlynn, Jun 16, 2015 at 3:26

Stack Exchange Network

Condition that matches all occurrences of a word in a file ad does something

5 Answers 5

Your Answer

Not the answer you're looking for? Browse other questions tagged
bash
scripts
grep
sed
awk
or ask your own question.

Hot Network Questions

Condition that matches all occurrences of a word in a file ad does something

5 Answers 5

Your Answer

Sign up or log in

Post as a guest

Not the answer you're looking for? Browse other questions tagged bashscriptsgrepsedawk or ask your own question.

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
bash
scripts
grep
sed
awk
or ask your own question.