Ask Ubuntu is a question and answer site for Ubuntu users and developers. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've got a ton of duplicate contacts in my Thunderbird addressbook. I don't want to share them with Google, but I do want to find a way to de-dupe them. There was a duplicate contact manager plug in, but it seems to have been abandoned.

Can I do this at the commandline?

share|improve this question
Also, as it turns outthe duplicate contact manager seems to be back and working great.… – Amanda Oct 22 '12 at 16:19
up vote 2 down vote accepted

I had the same problem several years ago, and wrote a very small python script to unify the LDIF export of a Thunderbird addressbook:

  1. export addressbook as LDIF to e.g. abook.ldif
  2. run cat abook.ldif | > abook_new.ldif
  3. import abook_new.ldif again (maybe rename old addressbook before)

The script currently matches duplicate entries on email address and identical name, but this can of course be adapted (in function find_existing_entry). Does this work for you?

The program is here (EDIT: you need the python-ldap package):

#!/usr/bin/env python

import sys
from ldif import LDIFParser, LDIFWriter

def find_existing_entry(ldif_entries, ldif_entry):
    for dn, entry in ldif_entries.items():
        if 'mail' in ldif_entry and 'mail' in entry:
            for mail in ldif_entry['mail']:
                if 'mail' in entry and mail in entry['mail']:
                    return dn
        if 'cn' in ldif_entry and 'cn' in entry and ldif_entry['cn'][0] in entry['cn']:
            return dn
        if 'sn' in ldif_entry and 'sn' in entry and 'givenName' in ldif_entry and 'givenName' in entry and ldif_entry['sn'][0] in entry['sn'] and ldif_entry['givenName'][0] in entry['givenName']:
            return dn
    return ''

class MyLDIF(LDIFParser):
    def __init__(self, input, output):
        LDIFParser.__init__(self, input)
        self.writer = LDIFWriter(output)
        self.entries = {}
    def merge(self, dn, entry):
        if 'mail' in entry.keys():
            if 'mail' in self.entries[dn].keys():
                for mail in entry['mail']:
                    if mail not in self.entries[dn]['mail']:
                self.entries[dn]['mail'] = entry['mail']
        for key in entry.keys():
            if key not in self.entries[dn].keys():
                self.entries[dn][key] = entry[key]

    def handle(self, dn, entry):
        if dn in self.entries.keys():
            self.merge(dn, entry)
            found_dn = find_existing_entry(self.entries, entry)
            if found_dn != '':
                self.merge(found_dn, entry)
                self.entries[dn] = entry
    def output(self):
        for dn, entry in self.entries.items():
            self.writer.unparse(dn, entry)

parser = MyLDIF(sys.stdin, sys.stdout)
share|improve this answer
Curiously, I don't have/can't find the ldif module. Pip says "Could not find any downloads that satisfy the requirement ldif No distributions at all found for ldif" – Amanda Jun 14 '11 at 17:20
I think you have to install the python-ldap package! (I edited my answer) – jmuc Jun 14 '11 at 17:40
I was trying to install it with pip, but it's an Ubuntu package. – Amanda Jun 17 '11 at 19:24
Okay, this starts to work, but it doesn't like entries with commas. It is choking on lines like: 'cn=Last Name, First Name,' I can edit the LDIF to take out the offending comma, but it finds another. – Amanda Jun 17 '11 at 19:25
This worked in vim to clear most of those out: :%s/cn=\([-'a-z]*\),\s\([a-z]*\),mail/cn=\2 \1,mail/g – Amanda Jun 17 '11 at 20:18

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.