-1

I just want data those have unique value, say we write log file for mobile users

<mobile_number1>|20141006 06:15:26||AKQY6LYACZAA4O|12|3|BIHAR|
<mobile_number2>|20141006 06:16:05||AKQY6MAYAEQALE|12|22|UP EAST|
<mobile_number3>|20141006 06:16:39||AKQY6MQICY4BEQ|12|2|ASSAM|
<mobile_number4>|20141006 06:16:49||AKQY6LUIAE4ACI|12|1|ANDHRA PRADESH| 
<mobile_number1>|20141006 06:17:53||AKQY6NFAAEYAJS|12|23|UP WEST| 
<mobile_number6>|20141006 06:18:09||AKQY6M7ACY4ANG|12|18|ORISSA|
<mobile_number7>|20141006 06:18:54||AKQY6MWQCZAAME|12|20|RAJASTHAN|
<mobile_number1>|20141006 06:19:50||AKQY6N2ACZMA2K|12|1|ANDHRA PRADESH|

Now we need to fetch unique mobile number like this.

<mobile_number1>
<mobile_number2>
<mobile_number3>
<mobile_number4>
<mobile_number6>
<mobile_number7>

closed as unclear what you're asking by Sylvain Pineau, αғsнιη, Seth Oct 7 '14 at 2:33

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    Your question isn't clear at all. What's the use case? Which data do you have before the search, which do you expect as a result? How are entries with the same mobile number grouped? – David Foerster Oct 6 '14 at 11:07
  • 1
    I am expecting unique mobile number output instead of repeating number. – Jayesh Dongare Oct 6 '14 at 11:26
  • I still don't understand. Please edit your question with an example of what you expect as output. – David Foerster Oct 6 '14 at 11:35
  • Eventually we found solution to your question, then could you be so kind as to add desired output to your question? – Lety Oct 7 '14 at 10:53
2

Try this:

  sort -t '|' -k 1,1 -u yourFile | awk -F "|" '{print $1}' 

where:

  • -t '|' use | as separator
  • -k 1,1 use first column as key
  • -u get unique line using key

awk print first column and the result is:

<mobile_number1>
<mobile_number2>
<mobile_number3>
<mobile_number4>
<mobile_number6>
<mobile_number7>

This script works if each line in yourFile has mobile_number as first column separated from the rest by |.

  • I would do cut -d '|' -f 1 | LC_COLLATE=C sort -u. It's a bit simpler and puts the projection before the sorting/grouping so to speak. – David Foerster Oct 6 '14 at 11:47
  • Cool! your suggestion is better then mine! I tested on Red Hat Enterprise Linux ES release 4 (Nahant Update 3) and it works like this: cut -d '|' -f 1 | sort -u. Why don't you write your own answer? It is really good! – Lety Oct 6 '14 at 11:54
  • Because I don't know yet, what the question aims at – though this is my most prominent suspicion. – David Foerster Oct 6 '14 at 12:03
  • Its works, also found $ cat <filename> | cut -f 1 -d'|' Thanks Letizia. – Jayesh Dongare Oct 6 '14 at 12:39
  • @DavidFoerster OP said that he like your suggestion, may be is time to write your answer, if you want :) – Lety Oct 6 '14 at 15:58
2

To throw away all columns but the first:

cut -d '|' -f 1

To omit recurring lines:

sort -u

Together (with input from a file):

cut -d '|' -f 1 <FILE> | sort -u

This uses the most simple shell utilities to perform the task. No command interpreters like awk necessary.

1
 awk '!x[$1]++ {print $1}' FS="|" /path/to/file 

example:

[aneesh@mylap /]$  awk '!x[$1]++ {print $1}' FS="|" /tmp/test.txt 

<mobile_number1>
<mobile_number2>
<mobile_number3>
<mobile_number4>
<mobile_number6>
<mobile_number7>

Not the answer you're looking for? Browse other questions tagged or ask your own question.