I have a file which looks like this:
1
3
4
1
4
3
1
2
How can I find the total of this (i.e. 1+3+4+1+4+3+1+2 = 19)?
bc
with a little help from paste
to get the lines in a single one with +
as the separator:
paste -sd+ file.txt | bc
To use the output of grep
(or any other command) instead a static file, pass the grep
's STDOUT to the STDIN of paste
:
grep .... | paste -sd+ | bc
Example:
% cat file.txt
1
3
4
1
4
3
1
2
% paste -sd+ file.txt | bc
19
% grep . file.txt | paste -sd+ | bc
19
If you must use bash
, then you can use an array to save the file contents and then iterate over the elements or you can read the file line by line and do the sum for each line, the second approach would be more efficient:
$ time { nums=$(<file.txt); for i in ${nums[@]}; do (( sum+=i )); done; echo $sum ;}
19
real 0m0.002s
user 0m0.000s
sys 0m0.000s
$ time { while read i; do (( sum+=i )); done <file.txt; echo $sum ;}
19
real 0m0.000s
user 0m0.000s
sys 0m0.000s
paste -sd+ -
. Please amend that.
awk
solution below runs about 30% faster for me... (On a 14,776,616 row file)
You could use awk, too. To count the total number of lines in *.txt files that contain the word "hello":
grep -ch 'hello' *.txt | awk '{n += $1}; END{print n}'
To simply sum the numbers in a file:
awk '{n += $1}; END{print n}' file.txt
numsum
from the package num-utils
!(You may need to sudo apt-get install num-utils
)
The command numsum
does just what you need by default;
$ numsum file.txt
19
Reading the test numbers line by line from stdin
:
$ printf '
1
3
4
1
4
3
1
2' | numsum
19
Or reading from one line:
$ printf '1 3 4 1 4 3 1 2' | numsum -r
19
The package contains some other utilities for number processing that deserve to be more well known:
numaverage - find the average of the numbers, or the mode or median
numbound - find minimum of maximum of all lines
numgrep - to find numbers matching ranges or sets
numinterval - roughly like the first derivative
numnormalize - normalize numbers to an interval, like 0-1
numrandom - random numbers from ranges or sets, eg odd.
numrange - similar to seq
numround - round numbers up, down or to nearest
and a more general calculator command numprocess
,
that applies an expression from the command line to numbers on input lines.
awk
. numsum
41sec for what awk
can do in 4.5s
You can use awk
, a native linux application usefull to scanning and processing files with a pattern per line. For your question, this will produce what you want:
awk 'BEGIN { sum=0 } { sum+=$1 } END {print sum }' file.txt
Pipes are also accept:
cat file.txt | awk 'BEGIN { sum=0 } { sum+=$1 } END {print sum }'
BEGIN{}
block, see CrazyApe84's answer.
awk
is not C, you don't need to set a variable before using it. Try awk 'BEGIN{print c+=1}'
.
BEGIN {}
block wasn't designed just to initialize variables. It takes part on design specification. So, on some problems it could be needed.
A simple approach is to use a built-in feature of your shell:
SUM=0; while read N; do SUM=$((SUM+N)); done </path/to/file
echo $SUM
This reads your file linewise, sums up and prints the result.
If you want to use a pipe and only use the 1st row, it works like this:
SUM=0
your_command | while read -r LINE; do for N in $LINE; do break; done; SUM=$((SUM+N)); done
echo $SUM
Getting the first element is done like this:
LIST="foo bar baz"
for OBJ in $LIST; do break; done
echo $OBJ
foo
bash
solution, as this streams file file. Howeve, its still very slow in comparison to awk
. For nearly 15 million lines, this took over 2 minutes, vs < 8 second in awk.
Perl solution:
$ perl -lnae '$c+=$_;END{print $c}' input.txt
19
The above can sum all numbers across multiple files:
$ perl -lnae '$c+=$_;END{print $c}' input.txt input2.txt
34
For multiple files given on command-line where we want to see sum of numbers in individual file we can do this:
$ perl -lnae '$c+=$_;if(eof){printf("%d %s\n",$c,$ARGV);$c=0}' input.txt input2.txt
19 input.txt
15 input2.txt
This is a fairly simple use of bash
scripting.
SUM=0; for line in `cat file.txt`; do SUM=$((SUM + line)); done
SUM
as this redefines the built-in SUM
in /usr/bin/SUM
.
cat badges.json | grep -o '"award_count":[0-9],"rank":"gold"' | grep -o [0-9]
badge_count
method.