62

I have a file which looks like this:

1
3
4
1
4
3
1
2

How can I find the total of this (i.e. 1+3+4+1+4+3+1+2 = 19)?

8
  • 1
    For context, I'm counting the number of badges I have on Ask Ubuntu from the Stack Exchange API.
    – Tim
    Commented Jun 9, 2016 at 20:17
  • With cat badges.json | grep -o '"award_count":[0-9],"rank":"gold"' | grep -o [0-9]
    – Tim
    Commented Jun 9, 2016 at 20:28
  • Although I now realise the API has a badge_count method.
    – Tim
    Commented Jun 9, 2016 at 22:12
  • If you are counting the badges from the API, the language you are using to query the API can't do this (python, javascript)?
    – Braiam
    Commented Jun 10, 2016 at 2:26
  • 2

7 Answers 7

92

bc with a little help from paste to get the lines in a single one with + as the separator:

paste -sd+ file.txt | bc

To use the output of grep (or any other command) instead a static file, pass the grep's STDOUT to the STDIN of paste:

grep .... | paste -sd+ | bc

Example:

% cat file.txt            
1
3
4
1
4
3
1
2

% paste -sd+ file.txt | bc
19

% grep . file.txt | paste -sd+ | bc
19

If you must use bash, then you can use an array to save the file contents and then iterate over the elements or you can read the file line by line and do the sum for each line, the second approach would be more efficient:

$ time { nums=$(<file.txt); for i in ${nums[@]}; do (( sum+=i )); done; echo $sum ;}
19

real    0m0.002s
user    0m0.000s
sys 0m0.000s

$ time { while read i; do (( sum+=i )); done <file.txt; echo $sum ;}
19

real    0m0.000s
user    0m0.000s
sys 0m0.000s
2
  • 3
    the stdin way didn't work for me until I discovered in another answer that I had to use paste -sd+ -. Please amend that.
    – xeruf
    Commented May 4, 2018 at 0:04
  • 1
    I was suprised to find that the awk solution below runs about 30% faster for me... (On a 14,776,616 row file)
    – dsz
    Commented Jun 25, 2021 at 5:33
28

You could use awk, too. To count the total number of lines in *.txt files that contain the word "hello":

grep -ch 'hello' *.txt | awk '{n += $1}; END{print n}'

To simply sum the numbers in a file:

awk '{n += $1}; END{print n}' file.txt
8
  • But if that line has the value "4" it won't count 4 will it. It will count 1.
    – Tim
    Commented Jun 9, 2016 at 20:23
  • No, it will count 4.
    – CrazyApe84
    Commented Jun 9, 2016 at 20:23
  • I want the total of the numbers in a file. Does this do that?
    – Tim
    Commented Jun 9, 2016 at 20:24
  • I thought you changed your question to the output of grep. I was making some assumptions. I'll add how to simply sum the values in a file.
    – CrazyApe84
    Commented Jun 9, 2016 at 20:26
  • 2
    Yeah. It's really the same answer as heemayl's, but instead of paste and bc it uses awk, which ultimately gives a lot more flexibility. Still, his answer is good and he was first so he deserves your vote!
    – CrazyApe84
    Commented Jun 9, 2016 at 20:28
16

Use numsum from the package num-utils!

(You may need to sudo apt-get install num-utils)

The command numsum does just what you need by default;

$ numsum file.txt 
19

Reading the test numbers line by line from stdin:

$ printf '
1 
3
4
1
4
3
1
2' | numsum
19

Or reading from one line:

$ printf '1 3 4 1 4 3 1 2' | numsum -r
19

More utilities

The package contains some other utilities for number processing that deserve to be more well known:

numaverage   - find the average of the numbers, or the mode or median
numbound     - find minimum of maximum of all lines
numgrep      - to find numbers matching ranges or sets
numinterval  - roughly like the first derivative
numnormalize - normalize numbers to an interval, like 0-1
numrandom    - random numbers from ranges or sets, eg odd.  
numrange     - similar to seq
numround     - round numbers up, down or to nearest

and a more general calculator command numprocess,
that applies an expression from the command line to numbers on input lines.

1
  • Wheil I like options, this package is written in Perl; it's far less performant that awk. numsum 41sec for what awk can do in 4.5s
    – dsz
    Commented Jun 25, 2021 at 5:20
3

You can use awk, a native linux application usefull to scanning and processing files with a pattern per line. For your question, this will produce what you want:

awk 'BEGIN { sum=0 } { sum+=$1 } END {print sum }' file.txt

Pipes are also accept:

cat file.txt | awk 'BEGIN { sum=0 } { sum+=$1 } END {print sum }'
4
  • 1
    There's no need for a BEGIN{} block, see CrazyApe84's answer.
    – terdon
    Commented Jun 10, 2016 at 10:39
  • 2
    It is redundant to this problem but I've preferred to include it due a didactic purpose .
    – gwarah
    Commented Jun 10, 2016 at 11:40
  • But what does it teach? It is redundant to every problem, awk is not C, you don't need to set a variable before using it. Try awk 'BEGIN{print c+=1}'.
    – terdon
    Commented Jun 10, 2016 at 11:46
  • BEGIN {} block wasn't designed just to initialize variables. It takes part on design specification. So, on some problems it could be needed.
    – gwarah
    Commented Jun 10, 2016 at 15:12
3

A simple approach is to use a built-in feature of your shell:

SUM=0; while read N; do SUM=$((SUM+N)); done </path/to/file
echo $SUM

This reads your file linewise, sums up and prints the result.

If you want to use a pipe and only use the 1st row, it works like this:

SUM=0
your_command | while read -r LINE; do for N in $LINE; do break; done; SUM=$((SUM+N)); done
echo $SUM

Getting the first element is done like this:

LIST="foo bar baz"
for OBJ in $LIST; do break; done
echo $OBJ

foo
1
  • A better bash solution, as this streams file file. Howeve, its still very slow in comparison to awk. For nearly 15 million lines, this took over 2 minutes, vs < 8 second in awk.
    – dsz
    Commented Jun 25, 2021 at 6:51
2

Perl solution:

$ perl -lnae '$c+=$_;END{print $c}' input.txt                                                                            
19

The above can sum all numbers across multiple files:

$ perl -lnae '$c+=$_;END{print $c}' input.txt input2.txt                                                                 
34

For multiple files given on command-line where we want to see sum of numbers in individual file we can do this:

$ perl -lnae '$c+=$_;if(eof){printf("%d %s\n",$c,$ARGV);$c=0}' input.txt input2.txt                                      
19 input.txt
15 input2.txt
1

This is a fairly simple use of bash scripting.

SUM=0; for line in `cat file.txt`; do SUM=$((SUM + line)); done
3
  • This is much simpler and toolset minimalistic than the current solution.
    – Det
    Commented Mar 13, 2019 at 12:04
  • Upvote for the simple approach, thanks! However I should mention it's not safe to use SUM as this redefines the built-in SUM in /usr/bin/SUM.
    – s3dev
    Commented Mar 18, 2020 at 11:25
  • Don't do this on a large file: it forces the whole file to be read into a "temp" variable, rather than streaming the numbers line by line. Not a great option.
    – dsz
    Commented Jun 25, 2021 at 6:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .