-2

From a file, containing one email per line, I have tried to remove the domains in the emails using this command:

grep -viwE "(gmail.com|hotmail|yahoo|aol|rediffmail.com|msn|outlook|inbox.com|icloud.com|mail.com|zoho.com|yandex|live.)" * > dest.txt

It is however also removing the emails containing particular terms, similar to what I have given in the brackets.
Say for example the email: example@gmail.com & examplegmail.com@ff.com
The email example@gmail.com should get deleted, but the another emails should not. The above command is deleting both the instances which is not good. How I can avoid it?

Additional complication is that possibly encoding errors will occur.

  • 2
    Try adding the @ character before the opening parenthesis in your pattern. – glenn jackman Feb 10 '17 at 15:52
  • How is your input formatted? One email address per line? Space separated list, comma separated list, etc? – Byte Commander Feb 10 '17 at 15:53
  • @ByteCommander Yes single line single email. – Jaffer Wilson Feb 10 '17 at 16:03
  • @JacobVlijm It is nice. But I want to delete only the lines containing specific domains in them. You can check in the grep command I have mentioned some domains whose emails I need to eliminate. I want the lines to be deleted containing those domains in them. – Jaffer Wilson Feb 10 '17 at 16:28
  • 1
    @JacobVlijm no... oops question editing .. may framed wrongly... – Jaffer Wilson Feb 10 '17 at 16:50
1

In a tiny script:

#!/usr/bin/env python3
import sys

# list domains to be removed
rm = [
    'gmail.com', 'hotmail', 'yahoo', 'aol', 'rediffmail.com',
    'msn', 'outlook', 'inbox.com', 'icloud.com', 'mail.com',
    'zoho.com', 'yandex', 'live'
    ]
# read the file per line
for l in open(sys.argv[1]):
    # see if not any of the @domains is in the line
    if not any([d in l for d in ["@"+d for d in rm]]):
        # then print the line
        print(l.strip())

To use

  • Copy the script into an empty file as filter_doms.py
  • Run it with the input file as argument:

    python3 /path/to/filter_doms.py input_file > output_file 
    

Explanation

I believe the code and the comments explains itself :)


As requested in a comment, a version to ignore decoding errors:

#!/usr/bin/env python3
import sys
import codecs

rm = [
    'gmail.com', 'hotmail', 'yahoo', 'aol', 'rediffmail.com',
    'msn', 'outlook', 'inbox.com', 'icloud.com', 'mail.com',
    'zoho.com', 'yandex', 'live'
    ]

with codecs.open(sys.argv[1], "r", encoding="utf-8", errors="ignore") as read:
    for l in read:
        if not any([d in l for d in ["@"+d for d in rm]]):
            print(l.strip())

Usage is exactly similar

  • may I know the meaning of this error: ` Traceback (most recent call last): File "RemaovedomainGMAIL.py", line 13, in <module> for l in open(file): File "/usr/lib/python3.5/codecs.py", line 321, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc4 in position 4245: invalid continuation byte ` – Jaffer Wilson Feb 10 '17 at 17:15
  • @JafferWilson What is the origin of the file? Obviously an encoding error. – Jacob Vlijm Feb 10 '17 at 17:18
  • Its a text file. also the code leaves space in between the two lines which not good. How I can resolve those encoding error. Kindly suggest me. – Jaffer Wilson Feb 10 '17 at 17:20
  • @JafferWilson added version, ignoring decoding errors. – Jacob Vlijm Feb 10 '17 at 17:30
  • Is it possibe Jacob, to eliminate or delete those lines which are not in UTF-8 encoded, so that I get a pure file with UTF-8 encoded charcters/text only? – Jaffer Wilson Feb 11 '17 at 5:06
0

You could use grep's -f option something like this.

grep -vhf patternfile file file1

Put all your pattern in patternfile line by line like

@gmail.com
@hotmail
@yahoo
@aol
@rediffmail.com
..

The option -h is supress prefixing filename when more than one file is given in input.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.