4
Order:479959,60=20130624-09:45:02.046|35=D|11=884|38=723|21=1|1=30532|10=085|59=0|114=Y|56=MBT|40=1|43=Y|100=MBTX|55=/GCQ3|49=11342|54=1|8=FIX.4.4|34=388|553=2453|9=205|52=20130624-09:45:02.046|

Order:24780,100=MBTX|43=Y|40=1|34=388|553=2453|52=2013062409:45:02.046|9=205|49=11342|54=1|8=FIX.4.4|55=/GCQ3|11=405|35=D|60=20130624-09:45:02.046|56=MBT|59=0|114=Y|10=085|21=1|38=470|1=30532|

Order:799794,55=/GCQ3|49=11342|54=1|8=FIX.4.4|34=388|553=2453|9=205|52=2013062409:45:02.046|40=1|43=Y|100=MBTX|38=350|21=1|1=30532|10=085|59=0|114=Y|56=MBT|60=20130624-09:45:02.046|35=D|11=216|

Order:72896,11=735|35=D|60=2013062409:45:02.046|56=MBT|59=0|114=Y|10=085|1=30532|38=17|21=1|100=MBTX|43=Y|40=1|553=2453|9=205|52=20130624-09:45:02.046|34=388|8=FIX.4.4|54=1|49=11342|55=/GCQ3|

I want to get the number after 38= and the number after 11= which should be renamed Clientid

The output should be:-

Orderid-479959 38= 723 Clientid=884
Orderid-24780 38= 470 Clientid=405
Orderid-799794 38= 350 Clientid=216
Orderid-72896 38= 17 Clientid=735

Any help will be appreciated.

  • 3
    Take a minute to describe how you arrived at this "desired output" which is missing a row – muru Jan 3 '17 at 22:07
  • 2
    OP added missing line and I think I made it clear enough... – Zanna Jan 4 '17 at 7:36
  • 1
    @Zanna of course you did. Please add an explanation to my answer too. – muru Jan 4 '17 at 7:59
  • Cross-posted: unix.stackexchange.com/q/334572/70524 – muru Jan 5 '17 at 9:40
4

You can use

sed -nr 's/Order:([0-9]+),.*[,\|]38=([0-9]+)[,\|].*/Orderid-\1 38= \2/p' file | tee file2

Then

sed -nr 's/.*[,\|]11=([0-9]+)[,\|].*/Clientid=\1/p' file | tee file3

Then

paste -d ' ' file2 file3

You get your output on stdout - redirect as you please.

I can't get it in one line (although someone obviously can) since the 11= and 38= fields could be in either order - I have to read the file twice. You could roll it into a script like this:

#!/bin/bash
sed -nr 's/Order:([0-9]+),.*[,\|]38=([0-9]+)[,\|].*/Orderid-\1 38= \2/p' "$1" > file2
sed -nr 's/.*[,\|]11=([0-9]+)[,\|].*/Clientid=\1/p' "$1" > file3
paste -d ' ' file2 file3 > outfile
rm file2 file3

(this cleans up the files we write in the process and writes the final output to a file outfile)

Usage:

  • paste the script into an empty file and save it
  • give it execute permission: chmod u+x script
  • run it with the name of your input file as argument: ./script file
  • change file2 and file3 in the script if you have existing files with those names in the current directory!

Explanation

  • s/old/new replace old with new
  • -r use ERE
  • -n don't print until we ask (this is just going to take out empty lines)
  • [,\|] match , OR literal |
  • ([0-9]+) some digits to save for later
  • \1 backreference to saved pattern
  • tee write to a file and print to stdout too so you can check it
  • > somefile redirect output to somefile instead of stdout
  • paste -d ' ' file2 file3 paste columns of file3 after columns of file2 using a space as delimiter.
  • rm file2 file3 delete file2 and file3
  • is it possible to do it with one command? – Sonal Jan 4 '17 at 0:07
  • @Sonal I can't figure out a single command, because the 11= field and 38= field could occur in either order. Because of this, we have to read the file twice. You can make it into a script though, I will do that for you... – Zanna Jan 4 '17 at 7:16
4

Using awk

Assuming your data is in a file called data.txt, create a file called script.awk and give it the following contents:

BEGIN { FS="[,|]" }
NF > 0 {
  for(i=1; i <= NF; i++) {
    split($i, f, "[:=]")
    map[f[1]] = f[2]
  }
  printf "Orderid-%s 38= %s Clientid=%s\n", map["Order"], map[38], map[11]
}

Then execute the following command to process the data and get output.

awk -f script.awk < data.txt

See also

In the above code, the map variable is an associative array. I called it map because it's typically called a map in other languages (HashMap in Java, Hash in Ruby, or Dictionary in Python).

  • Sure, I misunderstood the data. I know a better way. – Sam Gleske Jan 4 '17 at 10:17
  • Added a few lines to process the other part of the map which was missing. – Sam Gleske Jan 4 '17 at 10:25
  • Fixed, I had a typo transcribing (programming from my iPad). – Sam Gleske Jan 4 '17 at 10:40
  • 2
    Since this GNU awk, you can use [|,] as the FS so that it splits on both , and | - saves a split, which is not much, I suppose. – muru Jan 4 '17 at 11:07
  • 1
    muru Actually your suggestion helps me to greatly improve the script. – Sam Gleske Jan 6 '17 at 6:59
3

One liners aren't always nice:

$ sed 's/[|,]\(11=[^|]*\).*\(|38=[^|]*|\).*/\2\1|/; s/Order:\([0-9]*\).*|38=\([0-9]*\).*|11=\([0-9]*\)|.*/Orderid-\1 38= \2 Clientid=\3/' foo
Orderid-479959 38= 723 Clientid=884
Orderid-24780 38= 470 Clientid=405
Orderid-799794 38= 350 Clientid=216
Orderid-72896 38= 17 Clientid=735

Explanation

  • s/old/new/ replace old with new
  • [|,] match | or ,
  • \(11=[^|]*\) match any number of any characters except | after 11= and save 11=whatever for later use as \1
  • .* any number of any characters
  • \(|38=[^|]*|\) save |38=whatever| for later use as \2
  • \2\1| backreferences in replacement (this makes the fields consistent so we can deal with them in the next command)
  • ; separates commands, like in the shell
  • Order:\([0-9]*\).*|38=\([0-9]*\).*|11=\([0-9]*\)|.* match this pattern (now we've cleaned it up) saving the parts we want to reuse in \(parentheses\) again
  • Orderid-\1 38= \2 Clientid=\3 replacement with \1 \2 and \3 backreferences to the numbers we saved with \(\)
  • @Zanna tiny correction - \(11=[^|]*\) isn't excluding , - the example input doesn't contain any such instance, so I didn't include that case. – muru Jan 4 '17 at 8:25
  • haha idk where I read that comma in the expression! – Zanna Jan 4 '17 at 8:26
0

Perl solution:

As one-liner:

perl -a -F'[:|,]' -lne  'next if $_ =~ /^$/;printf("%sid-%s ",$F[0],$F[1]);foreach(@F){$t=$_ if $_ =~ "38=";$id=$_ if $_ =~ "11="};$id =~s/11=//;printf("%s Clientid=%s\n",$t,$id)' input.txt

Or as script:

#!/usr/bin/env perl
use strict;
use warnings;

open(my $fh,'<',$ARGV[0]) or die $!;

while(my $line = <$fh>){
    next if $line =~ /^$/;
    my @words  = split /[:|,]/,$line;
    printf("%sid-%s ",$words[0],$words[1]);
    my $t;
    my $id;
    foreach my $word (@words){
        $t = $word if $word =~ "38=";
        $id=$word if     $word =~ "11=";
        $id =~ s/11=// if length($id);
    }
    printf("%s Clientid=%s\n", $t ,$id);
}
close($fh) or die $!;

Test results:

$ ./parse_orders.pl ./input.txt                                                                                          
Orderid-479959 38=723 Clientid=884
Orderid-24780 38=470 Clientid=405
Orderid-799794 38=350 Clientid=216
Orderid-72896 38=17 Clientid=735

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.