18

In a file with lots of lines I want to delete lines that starts with HERE IT IS.

How can I do this using only command-line tools?

2
  • 2
    Although a little unconventional, you could use vim like so: vim '+g/^HERE IT IS/d' +wq test.txt ;)
    – Doorknob
    Feb 20, 2015 at 3:33
  • @Doorknob, thank you for pointing this out. Actually I'm on my way to using vim Feb 20, 2015 at 8:31

7 Answers 7

32

Try sed:

sed -i '/^HERE IT IS/d' <file>

WARNING: Its better to take a backup when using -i switch of sed:

sed -i.bak '/^HERE IT IS/d' <file>

The original file will remain as <file>.bak and the modified file will be <file>.

7
  • How can I put newline character after matched string and write to newline ? Feb 19, 2015 at 9:55
  • 2
    sed -i 's/^HERE IT IS/HERE IT IS\n/' <file>
    – heemayl
    Feb 19, 2015 at 10:02
  • 1
    @micgeronimo: happy to help. Check my edits please.
    – heemayl
    Feb 19, 2015 at 10:16
  • 6
    @micgeronimo try to ask the question you really want answered in your original question (remember you can edit it), rather than through comments that may get cleaned up / deleted. You can use sed '/^HERE IT IS/G' file. Feb 19, 2015 at 13:36
  • 1
    Sed usage is so pro.
    – lakshyaag
    Feb 19, 2015 at 15:04
20

In addition to the very good grep and sed answers you've received, here are some other tools that can do the same thing:

  • A few Perl ways:

    perl -ne '/^HERE IT IS/ || print' file > newfile
    perl -ne 'print if !/^HERE IT IS/' file > newfile
    perl -ne 'print unless /^HERE IT IS/' file > newfile
    

    You can add the -i switch to any of the examples to edit the file in place:

    perl -i.bak -ne '/^HERE IT IS/ || print' file        
    
  • (g)awk

    awk '!/^HERE IT IS/' file > newfile
    

    Newer versions (4.1.1 and later) of GNU awk (the default awk on Linux) can also edit the file in place:

    gawk -i inplace  '!/^HERE IT IS/' file
    
  • Shell (bash, zsh, ksh, probably others). This is kind of silly though, it can be done but other tools are better.

    while IFS= read -r line; do 
      [[ $line =~ ^"HERE IT IS" ]] || printf "%s\n" "$line"
    done < file > newfile
    
5
  • 1
    You're just showing off! ;-) (but you got an upvote, because it's smart and I learnt a lot and the bash one made me LOL)
    – Fabby
    Feb 20, 2015 at 12:09
  • the bash one should use printf "%s\n" "$line" : quoting $line to preserve whitespaces, and avoiding some echo problems (interpreting special chars, etc). and avoids the need to add -- too. Feb 20, 2015 at 18:04
  • @OlivierDulac fair enough. I didn't want to complicate things for fringe cases but since Cuanglm has added IFS= and -r, I may as well go all the way and make it robust.
    – terdon
    Feb 20, 2015 at 18:19
  • @terdon : it s all for the better good ^^ (and I did +1 already, though, as it's very informative for beginners) Feb 20, 2015 at 18:21
  • 2
    @OlivierDulac I can assure you that if I were posting on Unix & Linux I'd have used printf, IFS=, -r and quoting :). I often simplify things for the AU audience who are often less comfortable with the command line.
    – terdon
    Feb 20, 2015 at 18:23
13

I would use grep to filter them out. For example :

grep -v "^HERE IT IS" infile > outfile

Then move outfile back to infile.

1
  • Clever thinking
    – Anwar
    Feb 24, 2015 at 5:26
5

sed is definitely the way to go.

This slight modification of the command @heemayl gave you will delete the line whether the same case is used in the pattern or not, due to the I in the pattern reference.

sed -i '/HERE IT IS/Id' <file>

If you had several files in a directory that you wanted to do this on, you could combine it with find like so.

find . -maxdepth 1 -type f -exec sed -i.bak '/HERE IT IS/Id' {} +

The maxdepth option means this won't recurse into directories.

4

Another python option:

#!/usr/bin/env python3
[print(l, end = "") for l in open(f).readlines() if not l.startswith("HERE IT IS")]

Where f is the path to the file, between quotes.

0
4

Grep

grep -P '^(?!HERE IT IS)' file

(?!HERE IT IS) negative lookahead assertion which makes the regex engine to match all the line starting boundary (which is usually matched by ^) only if it's not followed by the string HERE IT IS

python

#!/usr/bin/python3
import sys
fil = sys.argv[1]
with open(fil) as f:
    for line in f:
        if not line.startswith('HERE IT IS'):
            print(line, end="")

Save the script in a file, say script.py and then run it through the below command on the terminal.

python3 script.py infile
3
  • you could use regex there, like [print(l, end = "") for l in open(fil).readlines() if not re.match("HERE IT IS", l)], but it's not much more efficient than startswith. I wondered how [print(l, end = "") for l in open(f).readlines() if not l.startswith("HERE IT IS")] won't produce the output in a list. Feb 20, 2015 at 12:54
  • The first time I ran into it, it looked strange to me to. It generates a print command (or whatever action you'd like to to with it) for all items in the defined list. Feb 20, 2015 at 12:57
  • Undeleting it, just for fun :) Feb 20, 2015 at 12:58
1

You can use Vim in Ex mode:

ex -sc 'g/^HERE IT IS/d' -cx file
  1. g global search

  2. d delete

  3. x save and close

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .