2

I was wondering if there is an easy way to use awk or any other command to parse a text file such that a file of something like:

Step Temp Enthalpy
0    0    -368
100  1    -369
200  2    -372
300  6    -362
400  9    -365
SHAKE stats (type/ave/delta) on step 500
1 1.09  8.71362e-08
500  13   -358
600  15   -339
.
.
900  25   -306
SHAKE stats (type/ave/delta) on step 1000
1 1.09 7.06858e-08
1000 28  -306
.
.
.

could print an output of only the specific column of numbers I want such as only the temperature values. I know I could just do something like awk '{print $2}' for the temperature values but my particular data file has other lines of various data before and after the 'Step Temp Enthalpy' table that makes this impractical so I'd like to ideally cut out everything before and after this 'Step Temp Enthalpy' information and print out only the particular column of this section of the data file I need. The data file also has that 'SHAKE stats' line every 5 steps in addition to a line after it '1 1.09 ....etc' that I'd like removed. If I were to print only the temperature column I'd like it to output:

0
1
2
6
9
13
15
.
.
25
28
  • So... to sum up, there's a one-line header Step Temp Enthalpy followed by 5 lines of temps, then 2 lines to discard, then 5 lines of temps, then 2 lines to discard and so on? – steeldriver Jul 26 '18 at 3:04
2

The simple pattern that you have is to take out column 2 in lines containing/starting with digits. The usual structure of awk commands is /Pattern in current line/ { commands};. The commands in braces are executed only if pattern is found in current line. Therefore we can do:

$ awk '/^[[:digit:]]/{print $2}' input.txt
0
1
2
6
9
1.09
13
15
25
1.09
28

To also remove the floating point numbers, add extra pattern with logical AND operator &&:

$ awk '/^[[:digit:]]/ && $0 !~ /[.]/ {print $2}' input.txt
0
1
2
6
9
13
15
25
28

Alternatively, just use a negation pattern to exclude shake stats:

awk '!/^SHAKE/ && $0 !~ /[.]/ {print $2}' input.txt 

To address the question in the comment, you can combine range pattern ( something like /Pattern1/,/Pattern2/ {commands} ) with and if statement. The range pattern will perform commands within curly braces only for lines fitting the range, and then if statement can do extra filtering. In this particular case, you can simply combine it with previous solution like so:

$ awk '$0 == "Step Temp Enthalpy",0 { if( $0 ~ /^[[:digit:]]/ && $0 !~ /[.]/  ) print $2  }' input.txt

The pattern $0 == "Step Temp Enthalpy",0 signifies processing of the exact line Step Temp Enthalpy to 0, that is end of file.

  • To follow up, since my data file has a bunch of stuff before the table with "step Temp Enthalpy" printing the second column like that prints out a lot of stuff both before and after that table I don't want. Is there a way I can make awk start and stop searching for the pattern at the beginning of the table and after it. – Jonathan Tran Jul 26 '18 at 3:18
  • @JonathanTran Yeah, there are ways to do that. Give me a minute or two, I'll update the answer. – Sergiy Kolodyazhnyy Jul 26 '18 at 3:19
  • 1
    @JonathanTran See the update – Sergiy Kolodyazhnyy Jul 26 '18 at 3:26
  • Appreciate your help awk seems like it can get very complicated very quickly haha still have a lot to read up on. – Jonathan Tran Jul 26 '18 at 3:29
  • @JonathanTran You're very welcome. awk can get complex, but as long as you keep in mind the simple /Pattern/ {command;} structure, it works just fine. Familiarity with C syntax and some SQL vocabulary helps, but not a requirement. I've learned awk with just whatever I knew about C at the time. Python syntax can be much more readable. If you're interested, check out some of my other answers - I have a lot of text processing answers that use Python for all sorts of things. – Sergiy Kolodyazhnyy Jul 26 '18 at 3:33
2

Here's a (possibly) simpler approach based on the structure of the data rather than on matching the content:

  • use modulo arithmetic to test whether we are on one of the SHAKE stats lines
  • if so, slurp up the next line with getline and move on
  • otherwise, print the second field

So

$ awk '!(NR%7) {getline; next} {print $2}' data
Temp
0
1
2
6
9
13
15


25
28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.