How do I remove lines of a file (hosts) which contain "adf.ly" string?


using sed

Run this command:

sed -i '/adf\.ly/d' inputfile 

man sed

   -i[SUFFIX], --in-place[=SUFFIX]
      edit files in place (makes backup if extension supplied)

Using grep

Thanks for @kos notes:

grep -v "ad\.fly" inputFile  > outputfile
  • 2
    You're doing two different things in the sed and in the grep version: in the sed version you're deleting all the lines containing an ad.fly substring while in the grep version you're deleting only lines containing an ad.fly substring surrounded by a non-word character (i.e. not surrounded by [A-Za-z0-9_]); in the grep command also the . matches any character (i.e. it will exclude lines containing e.g. adxfly) and you're not using any of the EREs' syntax in the pattern, so the -E option is not really needed. – kos Jun 16 '15 at 9:11
  • However I didn't downvote this, drop me a comment if you ever correct this and I'll upvote it since the sed version is fine – kos Jun 16 '15 at 9:35
  • @kos Even you downvote it's legitimated :). I removed the grep for now – Maythux Jun 16 '15 at 9:35
  • 1
    I will fix it before adding it again – Maythux Jun 16 '15 at 9:39
  • 1
    +1 for grep, which is IMO the most straightforward way to do this. – Gaurav Jun 16 '15 at 18:33

The following will remove the lines containing "adf.ly" in filename.txt in-place:

sed -i '/adf\.ly/d' filename.txt

Use the above command without -i to test it before removing lines.


Using awk (thanks to terdon for the shortened version):

< inputfile awk '!/adf\.ly/' > outputfile
  • < inputfile: redirects the content of inputfile to awk's stdin
  • > outputfile: redirects the content of awk's stdout to outputfile

awk command breakdown:

  • !/adf\.ly/: prints the record if not matching the adf\.ly regex

Using Perl (thanks to terdon for the shortened version):

< inputfile perl -ne '/adf\.ly/||print' > outputfile
  • -n: places a while (<>) {[...]} loop around the script
  • -e: reads the script from the arguments

Perl command breakdown:

  • /: starts the pattern
  • adf\.ly: matches an adf\.ly string
  • /stops the pattern
  • ||: executes the following command only if the pattern didn't match the line
  • print: prints the line
  • 1
    Why not just awk '!/ad\.fly/' in > out? You could also shorten the Perl to perl -ne '/ad\.fly/||print' file if you're into golfing :) – terdon Jun 16 '15 at 12:03
  • @terdon Because I'm silly (awk) and anyway not that proficient in both languages. Thanks for your suggestions (by the way I didn't get the golfing reference!) – kos Jun 16 '15 at 12:19
  • 1
    Code golfing is the game of making code as concise as possible, usually at the price of sacrificing legibility. My point is that the perl alternative I gave is indeed shorter but yours was much clearer and more understandable to a non-expert. – terdon Jun 16 '15 at 13:11
  • @terdon Ah I see, yes I'd say I am then. After all it's sill explained right after – kos Jun 16 '15 at 13:29

Here is a bash one-liner:

while IFS= read -r i; do [[ ! $i =~ .*adf\.ly.* ]] && echo "$i"; done <file.txt
  • Variable i will contain each line while iterating

  • [[ $i =~ .*adf\.ly.* ]] checks if the line has the string adf.ly, ! negates the check so [[ ! $i =~ .*adf\.ly.* ]] will check if the line does not contain adf.ly

  • If yes (&&), then the line will be printed.

To save the output to another file (out.txt):

while IFS= read -r i; do [[ ! $i =~ .*adf\.ly.* ]] && echo "$i"; done <file.txt >out.txt
  • 1
    +1, just a little suggestion, you can directly pattern match against adf\.ly – kos Jun 16 '15 at 12:47

You could use the standard text editor, ed:

printf '%s\n' 'g/adf\.ly/d' w q | ed file.txt
  • Great, a +1 for ed =) – A.B. Jun 16 '15 at 17:50

You can use Vim in Ex mode:

ex -sc 'g/adf\.ly/d' -cx hosts
  1. g global search

  2. d delete

  3. x save and close

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.