0

I am trying to get the name of the user from one file and their corresponding details from my other file. I use awk:

awk -F : '{ print $1 }' user-name

it gives me the list of all the user's. So now how can I match these names with the other file and get a output like:

user-name id contact-details

The format of the two files is like follows:

1.user-name

Tarun:143
Rahul:148
Neeraj:149

2.user-details

Tarun:[email protected]
Neeraj:[email protected]
Rahul:[email protected]

what I'm trying to get is like:

Neeraj:149:[email protected]
Rahul:148:[email protected]
Tarun:143:[email protected]
0

5 Answers 5

4

You can use the join command

join -t ":" username contacts

username file has the format

user1:id1
user2:id2

contacts has the format

user1:contact1
user2:contact2

When the file is not sorted then you can do the following

sort -b username > username.sorted
sort -b contacts > contacts.sorted

and then run the join command on username.sorted and contacts.sorted

or as another post pointed out you can do it directly using

join -t ":" <(sort -b username) <(sort -b contacts)
8
  • this solution only work for sorted list of users in both files . but +1 Aug 21, 2014 at 7:09
  • what if the list is not sorted?
    – Tarun
    Aug 21, 2014 at 7:16
  • @Tarun If username file like user1:id user2:id and contacts file like user2:details user1:details after running, this command tell you join: contacts:userNUM: is not sorted: user1:details. if you save the output of running and cat it you will see only sorted user written in output. see the output of this (join -t ":" user-name details | tr ":" " " > out ) then cat out Aug 21, 2014 at 7:40
  • updated the answer to handle unsorted files Aug 21, 2014 at 7:58
  • Now you removed all : between user-name and contacts! if you want true of that here it is (join -t ':' user-name.sorted details.sorted | tr '' ':') but after sorted. please edit your answer with this correct. Aug 21, 2014 at 8:01
4

In a python script:

A pragmatic solution

In case it is a "one time job", specific for one situation, the following works:

#!/usr/bin/env python3

with open(file1) as names:
    names = sorted(names.readlines())
with open(file2) as data:
    data = data.readlines()
for i in names:
    item = i.replace("\n", "")+str([d[d.find(":"):].replace("\n", "") for d in data if d.startswith(i.split(":")[0])][0])
    print(item)

Output:

Neeraj:149:[email protected]
Rahul:148:[email protected]
Tarun:143:[email protected]

Or, if you want to save the output directly into a file:

#!/usr/bin/env python3

with open(file1) as names:
    names = sorted(names.readlines())
with open(file2) as data:
    data = data.readlines()
with open(file3, "wt") as output:
    for i in names:
        output.write(i.replace("\n", "")+str([d[d.find(":"):].replace("\n", "") for d in data if d.startswith(i.split(":")[0])][0])+"\n")

As you probably know, copy the script into an empty file, set the path to the file 1-2 (3) (between quotes), save it as combine.py, run it by the command:

python3 /path/to/combine.py

A more database-worthy solution

Looking at the two files, we are actually dealing with databases, with the first field as a key. The following script is more flexible and covers a more flexible way of making reports of the two files, in situations (for example) where we would have more fields than here is the case.

If we would add an extra ("characterizing") field to the second file:

Neeraj:[email protected]:Loves to Cook
Rahul:[email protected]:Collects empty bottles
Tarun:[email protected]:Weares his glasses upside down

We might want to add the characterization instead of the email address, or both. That would ask for a script like:

#!/usr/bin/env python3

db1 = "/path/to/file1"; db2 = "/path/to/file2"

with open(db1) as data1:
    rc = [l.replace("\n", "").split(":") for l in data1.readlines()]

with open(db2) as data2:
    records2 = [l.replace("\n", "").split(":") for l in data2.readlines()]

uniques = sorted(set(item[0] for item in rc)) # find keys
report = []

for i in uniques:
    database_1 = [r for r in rc if r[0] == i][0]
    database_2 = [r for r in records2 if r[0] == i][0]
    # -----------------------------------------------------------------------
    # set the required fields for report here:
    new_record = i, database_1[1], database_2[1]
    # -----------------------------------------------------------------------
    report.append((":").join(new_record))

for item in report:
    print(item)

Result

If we set:

new_record = i, database_1[1], database_2[2]

The result is:

Neeraj:149:Loves to Cook
Rahul:148:Collects empty bottles
Tarun:143:Weares his glasses upside down

But if we set:

new_record = i, database_1[1], database_2[1]

The result is:

Neeraj:149:[email protected]
Rahul:148:[email protected]
Tarun:143:[email protected]

And if we set:

new_record = i, database_1[1], database_2[1], database_2[2]

The result is :

Neeraj:149:[email protected]:Loves to Cook
Rahul:148:[email protected]:Collects empty bottles
Tarun:143:[email protected]:Weares his glasses upside down
0
3

With process substitution in bash, we can make a very compact variant of the join solution even for unsorted input files:

join -t: <(sort user-name) <(sort user-details)

The output is just like the example output in the question:

Neeraj:149:[email protected]
Rahul:148:[email protected]
Tarun:143:[email protected]

We are using the first field/column of both files here. To use use other columns, use the options -1 and -2 (or -j if it's the same field). To be more explicit, we could use join -t: -j 1 ... or join -t: -1 1 -2 1 ... above. (See also man join)

The parts of the form <(command) are replaced by a named pipe from which the output of the command can be read. That means for the join command that it gets two files with sorted input as arguments.

(See man bash | less '+/Process Substitution')

2

Here's an awk solution:

$ awk -F: -v OFS=: 'NR==FNR{a[$1]=$2; next}{print $1,a[$1],$2}' user-name user-details 
Tarun:143:[email protected]
Neeraj:149:[email protected]
Rahul:148:[email protected]

Explanation

  • -F: : sets the field separator to :.
  • -v OFS=: : sets the output field separator (OFS) to : for pretty printing.
  • NR==FNR : NR is the current line number and FNR is the line number of the current file. When parsing more than 2 files, NR will be equal to FNR only for the 1st file. NR is incremented for each line of input while FNR is reset each time a new file is read.
  • {a[$1]=$2; next} : this creates an associative array called a whose key is the first field and whose value is the second. Once that is done, we skip to the next line with next.
  • {print $1,a[$1],$2} : print the 1st field of this line, the value stored in the a array for that field and then the second field. Because of the next explained above, this will only be executed when NR is not equal to FNR. In other words, it will only be run when the second file is being read.
1

Try with my code:

First sort both user-name and contacts and write the output into one file named user-name_contacts with this:

sort user-name contacts > user-name_contacts

Next, run this command to join two files:

sed -i '/$/N ; s/\n\(.*\):/:/' user-name_contacts


Output:

 Neeraj:149:[email protected]
 Rahul:148:[email protected]
 Tarun:143:[email protected]
3
  • Won't this break the order? It might not work if the list of contacts and usernames are not in order. Aug 21, 2014 at 10:32
  • Nope, not working. Here is the output $cat user-name A:1 B:2 $cat contacts B:Test2 A:Test1 $sed 's/\(.*\)://g' contacts > new-contacts $sort user-name new-contacts > user-name_contacts $sed -i '$!N;s/\n/:/g' user-name_contacts $cat user-name_contacts A:1:B:2 Test1:Test2 Wrong output. Aug 21, 2014 at 11:13
  • @PrathikRajendranM I have updated my answer. Aug 21, 2014 at 12:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .