Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I am trying to get the name of the user from one file and their corresponding details from my other file. I use awk:

awk -F : '{ print $1 }' user-name

it gives me the list of all the user's. So now how can I match these names with the other file and get a output like:

user-name id contact-details

The format of the two files is like follows:

1.user-name

Tarun:143
Rahul:148
Neeraj:149

2.user-details

Tarun:tarun@gmail.com
Neeraj:neeraj@xyz.com
Rahul:rahul@gmail.com

what I'm trying to get is like:

Neeraj:149:neeraj@xyz.com
Rahul:148:rahul@gmail.com
Tarun:143:tarun@gmail.com
share|improve this question

5 Answers 5

up vote 4 down vote accepted

You can use the join command

join -t ":" username contacts

username file has the format

user1:id1
user2:id2

contacts has the format

user1:contact1
user2:contact2

When the file is not sorted then you can do the following

sort -b username > username.sorted
sort -b contacts > contacts.sorted

and then run the join command on username.sorted and contacts.sorted

or as another post pointed out you can do it directly using

join -t ":" <(sort -b username) (sort -b contacts)
share|improve this answer
    
this solution only work for sorted list of users in both files . but +1 –  KasiyA Aug 21 at 7:09
    
what if the list is not sorted? –  Tarun Aug 21 at 7:16
    
@Tarun If username file like user1:id user2:id and contacts file like user2:details user1:details after running, this command tell you join: contacts:userNUM: is not sorted: user1:details. if you save the output of running and cat it you will see only sorted user written in output. see the output of this (join -t ":" user-name details | tr ":" " " > out ) then cat out –  KasiyA Aug 21 at 7:40
    
updated the answer to handle unsorted files –  Prathik Rajendran M Aug 21 at 7:58
    
Now you removed all : between user-name and contacts! if you want true of that here it is (join -t ':' user-name.sorted details.sorted | tr '' ':') but after sorted. please edit your answer with this correct. –  KasiyA Aug 21 at 8:01

In a python script:

A pragmatic solution

In case it is a "one time job", specific for one situation, the following works:

#!/usr/bin/env python3

with open(file1) as names:
    names = sorted(names.readlines())
with open(file2) as data:
    data = data.readlines()
for i in names:
    item = i.replace("\n", "")+str([d[d.find(":"):].replace("\n", "") for d in data if d.startswith(i.split(":")[0])][0])
    print(item)

Output:

Neeraj:149:neeraj@xyz.com
Rahul:148:rahul@gmail.com
Tarun:143:tarun@gmail.com

Or, if you want to save the output directly into a file:

#!/usr/bin/env python3

with open(file1) as names:
    names = sorted(names.readlines())
with open(file2) as data:
    data = data.readlines()
with open(file3, "wt") as output:
    for i in names:
        output.write(i.replace("\n", "")+str([d[d.find(":"):].replace("\n", "") for d in data if d.startswith(i.split(":")[0])][0])+"\n")

As you probably know, copy the script into an empty file, set the path to the file 1-2 (3) (between quotes), save it as combine.py, run it by the command:

python3 /path/to/combine.py

A more database-worthy solution

Looking at the two files, we are actually dealing with databases, with the first field as a key. The following script is more flexible and covers a more flexible way of making reports of the two files, in situations (for example) where we would have more fields than here is the case.

If we would add an extra ("characterizing") field to the second file:

Neeraj:neeraj@xyz.com:Loves to Cook
Rahul:rahul@gmail.com:Collects empty bottles
Tarun:tarun@gmail.com:Weares his glasses upside down

We might want to add the characterization instead of the email address, or both. That would ask for a script like:

#!/usr/bin/env python3

db1 = "/path/to/file1"; db2 = "/path/to/file2"

with open(db1) as data1:
    rc = [l.replace("\n", "").split(":") for l in data1.readlines()]

with open(db2) as data2:
    records2 = [l.replace("\n", "").split(":") for l in data2.readlines()]

uniques = sorted(set(item[0] for item in rc)) # find keys
report = []

for i in uniques:
    database_1 = [r for r in rc if r[0] == i][0]
    database_2 = [r for r in records2 if r[0] == i][0]
    # -----------------------------------------------------------------------
    # set the required fields for report here:
    new_record = i, database_1[1], database_2[1]
    # -----------------------------------------------------------------------
    report.append((":").join(new_record))

for item in report:
    print(item)

Result

If we set:

new_record = i, database_1[1], database_2[2]

The result is:

Neeraj:149:Loves to Cook
Rahul:148:Collects empty bottles
Tarun:143:Weares his glasses upside down

But if we set:

new_record = i, database_1[1], database_2[1]

The result is:

Neeraj:149:neeraj@xyz.com
Rahul:148:rahul@gmail.com
Tarun:143:tarun@gmail.com

And if we set:

new_record = i, database_1[1], database_2[1], database_2[2]

The result is :

Neeraj:149:neeraj@xyz.com:Loves to Cook
Rahul:148:rahul@gmail.com:Collects empty bottles
Tarun:143:tarun@gmail.com:Weares his glasses upside down
share|improve this answer

With process substitution in bash, we can make a very compact variant of the join solution even for unsorted input files:

join -t: <(sort user-name) <(sort user-details)

The output is just like the example output in the question:

Neeraj:149:neeraj@xyz.com
Rahul:148:rahul@gmail.com
Tarun:143:tarun@gmail.com

We are using the first field/column of both files here. To use use other columns, use the options -1 and -2 (or -j if it's the same field). To be more explicit, we could use join -t: -j 1 ... or join -t: -1 1 -2 1 ... above. (See also man join)

The parts of the form <(command) are replaced by a named pipe from which the output of the command can be read. That means for the join command that it gets two files with sorted input as arguments.

(See man bash | less '+/Process Substitution')

share|improve this answer

Here's an awk solution:

$ awk -F: -v OFS=: 'NR==FNR{a[$1]=$2; next}{print $1,a[$1],$2}' user-name user-details 
Tarun:143:tarun@gmail.com
Neeraj:149:neeraj@xyz.com
Rahul:148:rahul@gmail.com

Explanation

  • -F: : sets the field separator to :.
  • -v OFS=: : sets the output field separator (OFS) to : for pretty printing.
  • NR==FNR : NR is the current line number and FNR is the line number of the current file. When parsing more than 2 files, NR will be equal to FNR only for the 1st file. NR is incremented for each line of input while FNR is reset each time a new file is read.
  • {a[$1]=$2; next} : this creates an associative array called a whose key is the first field and whose value is the second. Once that is done, we skip to the next line with next.
  • {print $1,a[$1],$2} : print the 1st field of this line, the value stored in the a array for that field and then the second field. Because of the next explained above, this will only be executed when NR is not equal to FNR. In other words, it will only be run when the second file is being read.
share|improve this answer

Try with my code:

First sort both user-name and contacts and write the output into one file named user-name_contacts with this:

sort user-name contacts > user-name_contacts

Next, run this command to join two files:

sed -i '/$/N ; s/\n\(.*\):/:/' user-name_contacts


Output:

 Neeraj:149:neeraj@xyz.com
 Rahul:148:rahul@gmail.com
 Tarun:143:tarun@gmail.com
share|improve this answer
    
Won't this break the order? It might not work if the list of contacts and usernames are not in order. –  Prathik Rajendran M Aug 21 at 10:32
    
Nope, not working. Here is the output $cat user-name A:1 B:2 $cat contacts B:Test2 A:Test1 $sed 's/\(.*\)://g' contacts > new-contacts $sort user-name new-contacts > user-name_contacts $sed -i '$!N;s/\n/:/g' user-name_contacts $cat user-name_contacts A:1:B:2 Test1:Test2 Wrong output. –  Prathik Rajendran M Aug 21 at 11:13
    
@PrathikRajendranM I have updated my answer. –  KasiyA Aug 21 at 12:46

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.