4

To print the fields at 29th column in two files I used

paste <(awk -F, '{print $29}' PreRefFile.csv) <(awk -F, '{print $29}' Txlog.csv)

This worked fine. To print all the fields starting from 29 to 189. I wrote a script as follows

 y=29
 while [ $y -le 189 ]
 do
   x="\$$y"
   paste <(awk -F, '{print "'"$x"'"}' PreRefFile.csv) <(awk -F, '{print "'"$x"'"}' Txlog.csv)
   y=`expr $y + 1`
 done

Here the value of x is replaced by "$" followed by number (First round of substitution) and it is printing "$" follwed by that number instead of printing the field at that location. How to get that field in this way. I cannot write the same line for so many times. Suggest a method to proceed.

Also suggest another tool to do this, other than awk

3 Answers 3

4

All what you need is the power of awk and a for Statement:

paste <(awk -F, '{ for (i=29;i<=188; i++) print $i }' PreRefFile.csv) <(awk -F, '{ for (i= 29;i<= 188;i++) print $i }' Txlog.csv)

My test case:

paste <(awk -F, '{ for (i=2;i<=3;i++) print $i }' foo1) <(awk -F, '{ for (i=2;i<=3;i++) print $i }' foo2)

File foo1:

1,2,3,4,5,6
7,8,9,10,11,12

File foo2:

a,b,c,d,e,f,g
A,B,C,D,E,F,G

Output:

2   b
3   c
8   B
9   C
0
3

The variable you set x="\$$y" is not available in the subshells <(...). Thats the problem. Use export to make it available in subsequently executed commands, but it will anyway be expanded by the parent shell. The subshells never see the variable, but instead see the value the parent shell substituted for it. As @EliahKagan noticed in the comments.

Also your awk can be a bit simpler. See the example (I used echo to simulate a file with 2 fields):

x=2
paste <(echo "a1,a2" | awk -F, '{print $'$x'}' ) <(echo "b1,b2" | awk -F, '{print $'$x'}')

The output would then be:

a2      b2
4
  • Good point: subshells, +1
    – A.B.
    Apr 21, 2015 at 12:23
  • @chaos Have you tried your example without export (i.e., with just x=2 instead of export x=2)? It works that way, too. $x is expanded by the parent shell; the subshells never see the text $x but instead see the value the parent shell substituted for it. Apr 21, 2015 at 12:41
  • @EliahKagan Thanks for the clarification, I tested it, it works too and edited your addition to the answer.
    – chaos
    Apr 21, 2015 at 16:45
  • @chaos Thanks for the edit--but this answer is still very confusing and I'm still not sure if your explanation of the problem is at all correct. Is the problem really that a shell variable is unavailable in subshells? I recommend editing further to clarity. In particular, whether or not a variable is available in a subshell is irrelevant to the example you have given. I don't see how it's relevant to the OP's code either--so if it is, I recommend explaining that specifically. Apr 22, 2015 at 13:25
2

Using bash:

#!/bin/bash
paste PreRefFile.csv Txlog.csv | while IFS=$'\t' read a b; do
  i=29
  while [[ $i -le 189 ]]; do
    printf "$(cut -d, -f$i<<<"$a")\t$(cut -d, -f$i<<<"$b")\n"
    i=$((i+1))
  done
done

Using python:

#!/usr/bin/env python2
import csv, itertools
with open('PreRefFile.csv') as a, open('PreRefFile.csv') as b:
    for i, j in itertools.izip_longest(csv.reader(a), csv.reader(b)):
        check = 28
        while check <= 188:
            print i[check] + '\t' + j[check]
            check += 1

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .