70

How to remove all the lines from the text file containing the words "cat" and "rat"?

  • This sounds suspiciously like a homework assignment. Please remember to attribute your answer to the nice folks over at Askubuntu. – zwets Oct 7 '13 at 19:59
  • That a part of the big project, I am new to Linux environment. – PersonX Oct 7 '13 at 20:05
95

grep approach

To create a copy of the file without lines matching "cat" or "rat", one can use grep in reverse (-v) and with the whole-word option (-w).

grep -vwE "(cat|rat)" sourcefile > destinationfile

The whole-word option makes sure it won't match cats or grateful for example. Output redirection of your shell is used (>) to write it to a new file. We need the -E option to enable the extended regular expressions for the (one|other) syntax.

sed approach

Alternatively, to remove the lines in-place one can use sed -i:

sed -i "/\b\(cat\|rat\)\b/d" filename

The \b sets word boundaries and the d operation deletes the line matching the expression between the forward slashes. cat and rat are both being matched by the (one|other) syntax we apparently need to escape with backslashes.

Tip: use sed without the -i operator to test the output of the command before overwriting the file.

(Based on Sed - Delete a line containing a specific string)

  • I wonder if there's a way to achieve both the removal from the source file AND generate the file with matches. Probably not, but it would be useful (e.g. when you get a file that is growing too large, you are splitting it based on content). – Sridhar Sarnobat Nov 14 '16 at 20:29
  • 1
    @Sridhar-Sarnobat Oh, you can. Use tee and subshells to copy stdout. In one you filter, in the other the reverse. Use of tee and subshells demonstrated in an unrelated usecase demonstrated here: blog.g3rt.nl/… – gertvdijk Nov 15 '16 at 9:38
15

To test in terminal only, use:

sed '/[cr]at/d' file_name

To really remove those lines from the file, use:

sed -i '/[cr]at/d' file_name
5

Try the vim-way:

ex +"g/[cr]at/d" -scwq file.txt
0

Consider if you have file with file_name and you want to search for mouse but on the same time few rows from mouse having other words like cat and rat and you don't want to see those in your output, so the one way to do it is -

grep -r mouse file_name | grep -vE "(cat|rat)"
0

portable shell way

Works in /bin/sh, which is dash on Ubuntu, as well as ksh, and bash. Slightly awkward that you have to write multiple test cases for each word in case statement but portable. Works with cases where word appears alone on the line, in the beginning, end of the line, or middle of the line, and ignores where it might be part of another word.

#!/bin/sh
line_handler(){
   # $1 is line read, prints to stdout
    case "$1" in
        cat|cat\ *|*\ cat\ *|*\ cat) true;; # do nothing if cat or rat in line
        rat|rat\ *|*\ rat\ *|*\ rat) true;; 
        *) printf "%s\n" "$1"
    esac
}

readlines(){
    # $1 is input file, the rest is words we want to remove
    inputfile="$1"
    shift

    while IFS= read -r line;
    do
        line_handler "$line" "$@"
    done < "$inputfile"
    [ -n "$line" ] && line_handler "$line" 
}

readlines "$@"

And this is how it works:

$ cat input.txt                                                                                                                                                        
the big big fat cat
the cat who likes milk 
jumped over gray rat
concat 
this is catchy
rat
rational
irrational
$ ./dellines.sh input.txt                                                                                                                                              
concat 
this is catchy
rational
irrational

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.