14

In a file with lots of lines I want to delete lines that starts with HERE IT IS.

How can I do this using only command-line tools?

  • 2
    Although a little unconventional, you could use vim like so: vim '+g/^HERE IT IS/d' +wq test.txt ;) – Doorknob Feb 20 '15 at 3:33
  • @Doorknob, thank you for pointing this out. Actually I'm on my way to using vim – micgeronimo Feb 20 '15 at 8:31
28

Try sed:

sed -i '/^HERE IT IS/d' <file>

WARNING: Its better to take a backup when using -i switch of sed:

sed -i.bak '/^HERE IT IS/d' <file>

The original file will remain as <file>.bak and the modified file will be <file>.

  • How can I put newline character after matched string and write to newline ? – micgeronimo Feb 19 '15 at 9:55
  • 2
    sed -i 's/^HERE IT IS/HERE IT IS\n/' <file> – heemayl Feb 19 '15 at 10:02
  • 1
    @micgeronimo: happy to help. Check my edits please. – heemayl Feb 19 '15 at 10:16
  • 6
    @micgeronimo try to ask the question you really want answered in your original question (remember you can edit it), rather than through comments that may get cleaned up / deleted. You can use sed '/^HERE IT IS/G' file. – steeldriver Feb 19 '15 at 13:36
  • 1
    Sed usage is so pro. – LakshyaAg Feb 19 '15 at 15:04
17

In addition to the very good grep and sed answers you've received, here are some other tools that can do the same thing:

  • A few Perl ways:

    perl -ne '/^HERE IT IS/ || print' file > newfile
    perl -ne 'print if !/^HERE IT IS/' file > newfile
    perl -ne 'print unless /^HERE IT IS/' file > newfile
    

    You can add the -i switch to any of the examples to edit the file in place:

    perl -i.bak -ne '/^HERE IT IS/ || print' file        
    
  • (g)awk

    awk '!/^HERE IT IS/' file > newfile
    

    Newer versions (4.1.1 and later) of GNU awk (the default awk on Linux) can also edit the file in place:

    gawk -i inplace  '!/^HERE IT IS/' file
    
  • Shell (bash, zsh, ksh, probably others). This is kind of silly though, it can be done but other tools are better.

    while IFS= read -r line; do 
      [[ $line =~ ^"HERE IT IS" ]] || printf "%s\n" "$line"
    done < file > newfile
    
  • 1
    You're just showing off! ;-) (but you got an upvote, because it's smart and I learnt a lot and the bash one made me LOL) – Fabby Feb 20 '15 at 12:09
  • the bash one should use printf "%s\n" "$line" : quoting $line to preserve whitespaces, and avoiding some echo problems (interpreting special chars, etc). and avoids the need to add -- too. – Olivier Dulac Feb 20 '15 at 18:04
  • @OlivierDulac fair enough. I didn't want to complicate things for fringe cases but since Cuanglm has added IFS= and -r, I may as well go all the way and make it robust. – terdon Feb 20 '15 at 18:19
  • @terdon : it s all for the better good ^^ (and I did +1 already, though, as it's very informative for beginners) – Olivier Dulac Feb 20 '15 at 18:21
  • 2
    @OlivierDulac I can assure you that if I were posting on Unix & Linux I'd have used printf, IFS=, -r and quoting :). I often simplify things for the AU audience who are often less comfortable with the command line. – terdon Feb 20 '15 at 18:23
13

I would use grep to filter them out. For example :

grep -v "^HERE IT IS" infile > outfile

Then move outfile back to infile.

  • Clever thinking – Anwar Feb 24 '15 at 5:26
5

sed is definitely the way to go.

This slight modification of the command @heemayl gave you will delete the line whether the same case is used in the pattern or not, due to the I in the pattern reference.

sed -i '/HERE IT IS/Id' <file>

If you had several files in a directory that you wanted to do this on, you could combine it with find like so.

find . -maxdepth 1 -type f -exec sed -i.bak '/HERE IT IS/Id' {} +

The maxdepth option means this won't recurse into directories.

4

Another python option:

#!/usr/bin/env python3
[print(l, end = "") for l in open(f).readlines() if not l.startswith("HERE IT IS")]

Where f is the path to the file, between quotes.

4

Grep

grep -P '^(?!HERE IT IS)' file

(?!HERE IT IS) negative lookahead assertion which makes the regex engine to match all the line starting boundary (which is usually matched by ^) only if it's not followed by the string HERE IT IS

python

#!/usr/bin/python3
import sys
fil = sys.argv[1]
with open(fil) as f:
    for line in f:
        if not line.startswith('HERE IT IS'):
            print(line, end="")

Save the script in a file, say script.py and then run it through the below command on the terminal.

python3 script.py infile
  • you could use regex there, like [print(l, end = "") for l in open(fil).readlines() if not re.match("HERE IT IS", l)], but it's not much more efficient than startswith. I wondered how [print(l, end = "") for l in open(f).readlines() if not l.startswith("HERE IT IS")] won't produce the output in a list. – Avinash Raj Feb 20 '15 at 12:54
  • The first time I ran into it, it looked strange to me to. It generates a print command (or whatever action you'd like to to with it) for all items in the defined list. – Jacob Vlijm Feb 20 '15 at 12:57
  • Undeleting it, just for fun :) – Jacob Vlijm Feb 20 '15 at 12:58
1

You can use Vim in Ex mode:

ex -sc 'g/^HERE IT IS/d' -cx file
  1. g global search

  2. d delete

  3. x save and close

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.