To print the fields at 29th column in two files I used

paste <(awk -F, '{print $29}' PreRefFile.csv) <(awk -F, '{print $29}' Txlog.csv)

This worked fine. To print all the fields starting from 29 to 189. I wrote a script as follows

 while [ $y -le 189 ]
   paste <(awk -F, '{print "'"$x"'"}' PreRefFile.csv) <(awk -F, '{print "'"$x"'"}' Txlog.csv)
   y=`expr $y + 1`

Here the value of x is replaced by "$" followed by number (First round of substitution) and it is printing "$" follwed by that number instead of printing the field at that location. How to get that field in this way. I cannot write the same line for so many times. Suggest a method to proceed.

Also suggest another tool to do this, other than awk


All what you need is the power of awk and a for Statement:

paste <(awk -F, '{ for (i=29;i<=188; i++) print $i }' PreRefFile.csv) <(awk -F, '{ for (i= 29;i<= 188;i++) print $i }' Txlog.csv)

My test case:

paste <(awk -F, '{ for (i=2;i<=3;i++) print $i }' foo1) <(awk -F, '{ for (i=2;i<=3;i++) print $i }' foo2)

File foo1:


File foo2:



2   b
3   c
8   B
9   C
| improve this answer | |

The variable you set x="\$$y" is not available in the subshells <(...). Thats the problem. Use export to make it available in subsequently executed commands, but it will anyway be expanded by the parent shell. The subshells never see the variable, but instead see the value the parent shell substituted for it. As @EliahKagan noticed in the comments.

Also your awk can be a bit simpler. See the example (I used echo to simulate a file with 2 fields):

paste <(echo "a1,a2" | awk -F, '{print $'$x'}' ) <(echo "b1,b2" | awk -F, '{print $'$x'}')

The output would then be:

a2      b2
| improve this answer | |
  • Good point: subshells, +1 – A.B. Apr 21 '15 at 12:23
  • @chaos Have you tried your example without export (i.e., with just x=2 instead of export x=2)? It works that way, too. $x is expanded by the parent shell; the subshells never see the text $x but instead see the value the parent shell substituted for it. – Eliah Kagan Apr 21 '15 at 12:41
  • @EliahKagan Thanks for the clarification, I tested it, it works too and edited your addition to the answer. – chaos Apr 21 '15 at 16:45
  • @chaos Thanks for the edit--but this answer is still very confusing and I'm still not sure if your explanation of the problem is at all correct. Is the problem really that a shell variable is unavailable in subshells? I recommend editing further to clarity. In particular, whether or not a variable is available in a subshell is irrelevant to the example you have given. I don't see how it's relevant to the OP's code either--so if it is, I recommend explaining that specifically. – Eliah Kagan Apr 22 '15 at 13:25

Using bash:

paste PreRefFile.csv Txlog.csv | while IFS=$'\t' read a b; do
  while [[ $i -le 189 ]]; do
    printf "$(cut -d, -f$i<<<"$a")\t$(cut -d, -f$i<<<"$b")\n"

Using python:

#!/usr/bin/env python2
import csv, itertools
with open('PreRefFile.csv') as a, open('PreRefFile.csv') as b:
    for i, j in itertools.izip_longest(csv.reader(a), csv.reader(b)):
        check = 28
        while check <= 188:
            print i[check] + '\t' + j[check]
            check += 1
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.