2

My CSV looks like this

A   5   3
B   3   1
...

I need to get the sum of all columns and add those to a new line in the CSV so it becomes

A   5   3
B   3   1
SUM 8   4

I was able to print the sum of a particular column by doing awk -F',' '{sum+=$2} END {print sum}' file.csv but I need to do this for a whole folder of CSV's which eventually have to have a sum added to them. Maybe also an empty row between the data set and the "sum" column but that would just be a bonus thing.

I'm a programmer and I could write something like that in Java but I think AWK would get us there way quicker.

Thank you

2 Answers 2

3

You could use something like this. It works for files with an arbitrary number of columns, assuming that the first column is text and shall have SUM in the result line instead of the sum of all values of that column.

$ awk '{for(i=2;i<=NF;i++)a[i]+=$i;print $0} END{l="SUM";i=2;while(i in a){l=l" "a[i];i++};print l}' data.csv > final.csv
A   5   3
B   3   1
SUM 8 4

The awk code formatted in a more readable way:

{
    for (i=2 ; i<=NF ; i++)
        a[i] += $i
    print $0
}

END {
    l = "SUM"
    i=2
    while(i in a) {
        l = l " " a[i]
        i++
    }
    print l
}
7
  • Remember that the order of array traversal in awk is not guaranteed - this will give all the right sums, but not necessarily in the right order I think? Jul 14, 2019 at 22:28
  • @steeldriver Seems like you're right, thanks. I wasn't aware of that. I changed that loop now, guess it should work reliably that way?
    – Byte Commander
    Jul 14, 2019 at 23:43
  • Yes that should work I think Jul 14, 2019 at 23:51
  • Is use of "while (i in a)" documented? It may be safer to note the number of columns, say with "NR==1 {cols=NF}" and change the while to "for (i=2; i<=cols; i++)" which removes the need for the "i=2" and "i++" statements. Also note that CSV format means comma-separated values, and you're using spaces, so a txt suffix would be more appropriate. Jul 15, 2019 at 4:30
  • 1
    @MadzaYasodaraFariasVirgens Only the question author can mark an answer as accepted, and they are free to do so however they like, or not at all. This may be a conscious decision or simply because they didn't know or didn't return to the post after testing. Nothing to do about that. Leave upvotes on the posts you found helpful (or downvotes if not) to improve the overall scoring quality of them. The accept mark is just one person's subjective note.
    – Byte Commander
    Nov 3, 2020 at 12:16
1

Found something that at first didn't work for me on StackOverflow https://stackoverflow.com/a/27110024/2161301

awk -F',' '{
    print($0);
    len=split($0,a);
    if (maxlen < len) {
        maxlen=len;
    }
    for (i=1;i<=len;i++) {
        b[i]+=a[i];
    }
}

END {
    for (i=1;i<=maxlen;i++) {
        printf("%s,", b[i]);
    }
    print ""
}' data.csv >> final.csv

It for some reason adds another column to my data but that's ok, I can work with that.

1
  • The additional column is likely because you didn't adjust the starting index of the loops to account for the non-numeric first column in your data Jul 14, 2019 at 22:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .