Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free.

I have a few hundreds of files that has this pattern

@<TRIPOS>ATOM
  2 H18 65.2220 Du 1 RES1 0.0000
@<TRIPOS>BOND
 1  3  5  ar
@<TRIPOS>SUBSTRUCTURE

among them, some of the files are missing the line after the @<TRIPOS>BOND and they look like

@<TRIPOS>ATOM
  2 H18 65.2220 Du 1 RES1 0.0000
@<TRIPOS>BOND
@<TRIPOS>SUBSTRUCTURE

I'm trying to find all the files in my working directory that are missing the numeric line after the @<TRIPOS>BOND and move them to another directory. I know this is a simple task, but I am quite new to Linux.

Note: the files vary in length and line numbers, this is why I am "grepping" the line after the @<TRIPOS>BOND string.

Here's one of my codes, which I was planning to write in a for loop. It doesn't do the job, but I am showing it to show one of my trials.

cat file | grep -A1 '@<TRIPOS>BOND' | awk 'FNR == 2 {print}'

Thank you

share|improve this question

6 Answers 6

up vote 6 down vote accepted

If your version of grep supports PCRE mode (-P) you could try a multiline match that finds instances of @<TRIPOS>BOND that are followed (after only a newline) by @<TRIPOS>SUBSTRUCTURE e.g.

grep -lzP '\Q@<TRIPOS>BOND\E\n\Q@<TRIPOS>SUBSTRUCTURE\E' *

The \Q and \E may be unnecessary in this case but are intended to force literal matching (in case @, >, < have special meaning in the Perl regex syntax). The -l tells grep to list the matching files instead of printing the match. You can then use the list of files as an input to the mv command e.g.

grep -lzP '\Q@<TRIPOS>BOND\E\n\Q@<TRIPOS>SUBSTRUCTURE\E' * | xargs mv -t /path/to/newdir/


Additional info

You could express the second part of the match as a lookahead but I don't think it has any advantage in this case

grep -lzP '\Q@<TRIPOS>BOND\E\n(?=\Q@<TRIPOS>SUBSTRUCTURE\E)' *

Equivalent expressions in pcregrep (which is not part of the standard Ubuntu system, but obtainable from the repository) would be something like

pcregrep -lM '\Q@<TRIPOS>BOND\E\n\Q@<TRIPOS>SUBSTRUCTURE\E' *

and

pcregrep -lM '\Q@<TRIPOS>BOND\E\n(?=\Q@<TRIPOS>SUBSTRUCTURE\E)' *
share|improve this answer

How about

for file in *.txt; do 
    grep -A1 "@<TRIPOS>BOND" "$file" | grep -q SUBSTR && mv "$file" bad_files/
done

Explanation:

This will loop through all .txt files in the current directory (change to the blob to whatever matches your files) and save each as $file. It will then search $file for @<TRIPOS>BOND and print that and the next line. This is passed through the next grep which silently (-q) looks for SUBSTR, if it finds it, it means that the line after BOND is SUBSTRUCTURE and not the numeric line you want so it will move the current file to the folder bad_files.

share|improve this answer
    
+1 Seems much simpler than mine ^^ –  chaos May 7 '14 at 11:28

The task is quite simple using awk. Here's my example. I've created two files file-nm (for not-missing) and file-m(for missing), and moved directory for the files that we want to move.

awk '/@<TRIPOS>BOND/ {getline; if ($0 == "@<TRIPOS>SUBSTRUCTURE" ) system("mv \""FILENAME"\" moved")}' file-nm file-m

Here we find the @<TRIPOS>BOND string, step into the next line, and check if that line is @<TRIPOS>SUBSTRUCTURE. If it is, we make system call with "mv " the FILENAME of the file that has been found, and " moved" as destination. Here's the result:

$ ls
file-m  file-nm  moved


$ awk '/@<TRIPOS>BOND/ {getline; if ($0 == "@<TRIPOS>SUBSTRUCTURE" ) system("mv \""FILENAME"\" moved")}' file-nm file-m      


$ ls                                                                                                                     
file-nm  moved


$ ls moved                                                                                                               
file-m
share|improve this answer
    
You should use double quotes for your filenames in system (…) –  A.B. Jul 3 at 21:59
    
@A.B. If I use "FILENAME" , system command will literally treat that as mv FILENAME moved . Unquoted FILENAME gets substituted by awk , from the actual file's filename –  Serg Jul 3 at 22:03
    
"mv \""FILENAME"\" moved" –  A.B. Jul 3 at 22:05
    
@A.B. That works :) Thank you –  Serg Jul 3 at 22:07

Using awk or gawk

awk '/@<TRIPOS>BOND/,/@/ {getline; if ($_ ~ /^@/) {printf "%s:%s\n",$_,FILENAME; system ("mv \""FILENAME"\" <bad_files>/$(basename \""FILENAME"\")")} exit}' <file_name>

Explanation

  • /@<TRIPOS>BOND/,/@/

    We need only the block between @<TRIPOS>BOND and the next line starting with @

  • getline

    Read the next line after @<TRIPOS>BOND

  • if ($_ ~ /^@/)

    Check, if the line starts with a @

    • true

      Print a message

      printf "%s:%s\n",$_,FILENAME
      

      Move the file

      system ("mv \""FILENAME"\" <bad_files>/$(basename \""FILENAME"\")")
      
    • false

      Leave the script

      exit
      

Example

$ cat foo
@<TRIPOS>ATOM
  2 H18 65.2220 Du 1 RES1 0.0000
@<TRIPOS>BOND
@<TRIPOS>SUBSTRUCTURE

$ cat bar
@<TRIPOS>ATOM
  2 H18 65.2220 Du 1 RES1 0.0000
@<TRIPOS>BOND
 1  3  5  ar
@<TRIPOS>SUBSTRUCTURE

$ awk '/@<TRIPOS>BOND/,/@/ {getline; if ($_ ~ /^@/) {printf "%s:%s\n",$_,FILENAME; system ("mv \""FILENAME"\" \""FILENAME"\"_moved")} exit}' foo
@<TRIPOS>SUBSTRUCTURE:foo

$ awk '/@<TRIPOS>BOND/,/@/ {getline; if ($_ ~ /^@/) {printf "%s:%s\n",$_,FILENAME; system ("mv \""FILENAME"\" \""FILENAME"\"_moved")} exit}' bar

$ cat foo_moved 
@<TRIPOS>ATOM
  2 H18 65.2220 Du 1 RES1 0.0000
@<TRIPOS>BOND
@<TRIPOS>SUBSTRUCTURE

$ cat bar_moved
cat: bar_moved: No such file or directory
share|improve this answer

It's not that simple:

find -type f -exec \
 awk '/@<TRIPOS>BOND/{getline; \
  if ($0 !~ /1  3  5  ar/){\
  printf "mv %s /path/to/move/%s\n", FILENAME, FILENAME}}' {} \; \
| bash

Explanation:

  • find -type f: Find all files in the current working directory
  • awk '/@<TRIPOS>BOND/{getline; \: find the line inside the file and move to the next line
  • if ($0 !~ /1 3 5 ar/){\: If the next line is NOT (!~) your desired "numeric line"
  • printf "mv %s /path/to/move/%s\n", FILENAME, FILENAME}}' {} \; \: build an mv command and pipe it to...
  • | bash: ...bash and execute it.

So the command will mv all file that are not contianing the numeric line to a directory called /path/to/move/.

share|improve this answer
1  
I think the OP's numeric line can change. I doubt it will be the same for all bonds. –  terdon May 7 '14 at 10:55
    
Then he can replace if ($0 !~ /1 3 5 ar/) with if ($0 ~ /@<TRIPOS>SUBSTRUCTURE/) –  chaos May 7 '14 at 11:24
    
Indeed. Or, simpler if ($1 !~ /^[0-9]$/), that way you don't need to worry about multiple spaces or tabs. –  terdon May 7 '14 at 11:27
    
You should change this part if ($0 !~ /1 3 5 ar/) –  A.B. Jul 3 at 22:32
nawk '/^@<TRIPOS>BOND/{getline;if( $0 ~ /^@/){print "mv", FILENAME, "../NewLoc/"}}' * | bash
share|improve this answer
1  
piping to bash is unnecessary, we can use system() function. Please check my answer –  Serg Jul 3 at 18:11
    
-1 you have a BIG problem, if the file name contains spaces and other funny characters. –  A.B. Jul 3 at 22:29

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.