I have a number of pdb files and I want to keep only those lines that starts with ^FORMUL and if line has C followed by a number that is larger than (C3,C4,C5,C6..100 etc) then I should not print it. Second condition is that within every line sum of C, H and N should be at least 6.

So overall delete the lines in which C is followed by number more then 2> and sum of C+O+N is >= 6.

FORMUL   3  HOH   *207(H2 O)    (print it)                                     
FORMUL   2  SF4    FE4 S4       (print it)                                                
FORMUL   3  NIC    C5 H7 N O7   (don't print, there is C5)                                               
FORMUL   4  HOH   *321(H2 O)    (print it)                                                
FORMUL   3  HEM    2(C34 H32 FE N4 O4)  (don't print, there is C34)

I have tried to do it in perl but lines are so diverse from each other so I'm not sure if it is possible to do.

  • I have been trying to do in perl few days ago but the thing is that this lines are so diverse, so I would have to consider few loops, I did it (splitted lines according to the space and brackets) and pushed the elements into an array, but it doesnt make sense..cant do it this way (at least me). I have used this command in bash grep '^FORMUL' *pdb | grep -vE 'C([3-9]|[12][0-9])' because I wanted to delete these lines that after C have number larger then 3. Both conditions should be included but not together, meaning that if I have C2 N5 02 it is still ok although C is followed by 1 – djordje Jan 10 '18 at 8:07
  • So, for example C3 N1 01 should not be deleted because overall its less then 5, C2 N4 02 should be deleted although C2 exist, overall its more then 6. Overall delete all lines in which C>2 and sum is 6. – djordje Jan 10 '18 at 8:09
  • pa4080: Actually, yes these conditions have to be put together. My mistake – djordje Jan 10 '18 at 8:15

Here is my suggestion, unfortunately I don't have a time to write an explanation right now:

$ cat script.bash


[[ -z "${2}" ]] && OUTPUT_FILE="output-file" || OUTPUT_FILE="${2}"


while read -a line; do

        C_VALUE="$(echo ${line[@]} | grep -Po 'C\d+' | sed 's/C//')"
        O_VALUE="$(echo ${line[@]} | grep -Po 'O\d+' | sed 's/O//')"
        H_VALUE="$(echo ${line[@]} | grep -Po 'H\d+' | sed 's/H//')"
        COH_SUM=$(( C_VALUE + O_VALUE + H_VALUE  ))
        printf '\nLINE=%s: C=%s, O=%s, H=%s, SUM=%s' "$LINE_COUNTER" "$C_VALUE" "$O_VALUE" "$H_VALUE" "$COH_SUM"

        if [[ "$COH_SUM" -ge "6" ]] || [[ "$C_VALUE" -gt "2" ]]; then


done < "$INPUT_FILE"

printf '\n\nLines to be removed: %s\n\nResult:\n\n' "${LINES_TO_BE_REMOVED#';'}"

sed -e "${LINES_TO_BE_REMOVED#';'}" "$INPUT_FILE" | tee "$OUTPUT_FILE"

How to use it:

$ cat input-file
FORMUL   3  HOH   *207(H2 O)
FORMUL   2  SF4    FE4 S4
FORMUL   3  NIC    C5 H7 N O7
FORMUL   4  HOH   *321(H2 O)
FORMUL   3  HEM    2(C34 H32 FE N4 O4)

$ ./script.bash input-file output-file

LINE=1: C=, O=, H=2, SUM=2
LINE=2: C=, O=, H=, SUM=0
Lines to be removed: 3d;5d

FORMUL   3  HOH   *207(H2 O)

$ cat output-file
FORMUL   3  HOH   *207(H2 O)
FORMUL   2  SF4    FE4 S4
FORMUL   4  HOH   *321(H2 O)

Here is much simple script that just output the lines to be kept:


while read -a line; do
        C_VALUE="$(echo "${line[@]}" | grep -Po 'C\d+')"
        if [[ ! "${C_VALUE#C}" -gt '2' ]]; then echo "${line[@]}"
                O_VALUE="$(echo "${line[@]}" | grep -Po 'O\d+')"
                H_VALUE="$(echo "${line[@]}" | grep -Po 'H\d+')"
                C_VALUE="${C_VALUE#C}"; O_VALUE="${O_VALUE#O}"; H_VALUE="${H_VALUE#H}"
                if (( (( C_VALUE + O_VALUE + H_VALUE )) <= 6 )); then echo "${line[@]}"; fi
done < "${@}"

How to use it:

./script.bash input-file > output-file
  • Merely seeing the start of the answer told me it was you, nice! New year wishes... – George Udosen Jan 10 '18 at 9:10
  • Hvala ti Spas, pozdrav is Srbije. Just one question, when I run this script on my data I also get these C34 H32 FE N4 O4 which is a bit strange. – djordje Jan 10 '18 at 9:31
  • 1
    Hi, @djordje, could you upload somewhere your data file, to make some tests. Поздрави! – pa4080 Jan 10 '18 at 10:38
  • 1
    Hey Spas I have dealed with this one by writing a script in perl. I can upload it if you want for your personal satisfaction, but I guess you have more important things to do :D – djordje Jan 10 '18 at 11:24
  • 1
    @djordje, no, I not needed it if you have solved your problem. Happy to help! – pa4080 Jan 10 '18 at 11:33

Although it is not written as a shell script, if somebody will make use of it in the future here is the answer how to do it in Perl.


use strict;
use warnings;

#open ($file, '<', '5PCZ.pdb') or die $!; 

while (<>) { # read from STDIN a line at a time
  # Split data on whitespace, but only into three columns
  my @cols = split /\s+/, $_, 3;

  next unless $cols[0] eq 'FORMUL';

  # Now extract the letter stuff into a hash for easy access.
  my %letters = m/([A-Z])(\d+)/g;

  # Give the values we're interested in, a default of 0
  $letters{$_} //= 0 for (qw[C O N]);

  next if $letters{C} > 2
    and $letters{C} + $letters{O} + $letters{N} >= 6;

  # I think we can then print the line;
  • 3
    It would be polite to mention that this is the literal answer @DaveCross posted on Stack Overflow. – PerlDuck Jan 10 '18 at 15:06
  • 1
    This is really powerful solution! It process a file with two million lines for less than 20 seconds. – pa4080 Jan 10 '18 at 15:59
  • 1
    @djordje: Posting someone else's code without giving them credit is a bit rude, isn't it? – Dave Cross Jan 10 '18 at 16:35
  • 1
    @djordje: And you've pasted an earlier version of the code. I fixed a bug in a later edit. – Dave Cross Jan 10 '18 at 16:42
  • 2
    Yes, I forgot to say that all credits are given to @Dave Cross. I was in a bit rush so I really forgot, just wanted to help others who encounter similar problem. Really didn't mean to put some credits on me, sorry. It is very powerful solution in just few lines of code. Many thanks Dave once again! Best! – djordje Jan 11 '18 at 1:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.