13

For example I have file 1.txt, that contain:

Moscow
Astana
Tokyo
Ottawa

I want to count number of all char as:

a - 4,
b - 0,
c - 1,
...
z - 0
  • 4
    From the accepted answer, it is not completely clear, do you want "A" and "a" distinguished or not? your question suggests you do. – Jacob Vlijm Mar 6 '15 at 16:01

11 Answers 11

20

You could use this:

sed 's/\(.\)/\1\n/g' 1.txt | sort | uniq -ic
  4  
  5 a
  1 c
  1 k
  1 M
  1 n
  5 o
  2 s
  4 t
  2 w
  1 y

The sed part places a newline after every character. Then we sort the ouput alphabetically. And at last uniq counts the number of occurences. The -i flag of uniq can be ommited if you don't want case insensitivity.

  • 3
    This is brilliant. An extra caveat would be to pipe the output again into sort -k 2 to list them alphanumerically. – tetris11 Mar 6 '15 at 13:23
  • 3
    This is the shortest way, most understandable but ufortunately the slowest – c0rp Mar 6 '15 at 17:23
  • On Mac OS X I had to use sed -e $'s/\(.\)/\\1\\\n/g' (see also stackoverflow.com/a/18410122/179014) – asmaier Mar 21 '16 at 14:48
  • To order by the number of occurrences (descending): | sort -rnk 1. And if you're dealing with very large files, like I am, you can just sample a few thousand line to get a proxy for the actual counts: cat 1.txt | shuf -n 10000 | sed 's/\(.\)/\1\n/g' | sort | uniq -ic | sort -rnk 1 – cpury Apr 17 '18 at 8:46
6

A bit late, but to complete the set, another python(3) approach, sorted result:

#!/usr/bin/env python3
import sys

chars = open(sys.argv[1]).read().strip().replace("\n", "")
[print(c+" -", chars.count(c)) for c in sorted(set([c for c in chars]))]

A - 1
M - 1
O - 1
T - 1
a - 4
c - 1
k - 1
n - 1
o - 4
s - 2
t - 3
w - 2
y - 1

Explanation

  1. Read the file, skip spaces and returns as "characters":

    chars = open(sys.argv[1]).read().strip().replace("\n", "")
    
  2. Create a (sorted) set of uniques:

    sorted(set([c for c in chars]))
    
  3. Count and print the occurrence for each of the characters:

    print(c+" -", chars.count(c)) for c in <uniques>
    

How to use

  1. Paste the code into an empty file, save it as chars_count.py
  2. Run it with the file as an argument by either:

    /path/to/chars_count.py </path/to/file>
    

    if the script is executable, or:

    python3 /path/to/chars_count.py </path/to/file>
    

    if it isn't

5

By default in the Field Separator (FS) is space or tab. Since we want to count each character, we will have to redefine the FS to nothing(FS="") to split each character in separate line and save it into an array and at the end insideEND{..} block, print their total occurrences by the following command:

$ awk '{for (i=1;i<=NF;i++) a[$i]++} END{for (c in a) print c,a[c]}' FS="" file
A 1
M 1
O 1
T 1
a 4
c 1
k 1
n 1
o 4
s 2
t 3
w 2
y 1

In {for (i=1;i<=NF;i++) a[$i]++} ... FS="" ... block we just splits the characters. And
in END{for (c in a) print c,a[c]} block we are looping to array a and printing saved character in it print c and its number of occurrences a[c]

3

Do a for loop for all the characters you want to count, and use grep -io to get all occurences of the character and ignoring case, and wc -l to count instances, and print the result.

Like this:

#!/bin/bash

filename="1.txt"

for char in {a..z}
do
    echo "${char} - `grep -io "${char}" ${filename} | wc -l`,"
done

The script outputs this:

a - 5,
b - 0,
c - 1,
d - 0,
e - 0,
f - 0,
g - 0,
h - 0,
i - 0,
j - 0,
k - 1,
l - 0,
m - 1,
n - 1,
o - 5,
p - 0,
q - 0,
r - 0,
s - 2,
t - 4,
u - 0,
v - 0,
w - 2,
x - 0,
y - 1,
z - 0,

EDIT after comment

To create a loop for all printable characters you can do this:

#!/bin/bash

filename="a.txt"

for num in {32..126}
do
   char=`printf "\x$(printf %x ${num})"`
   echo "${char} - `grep -Fo "${char}" ${filename} | wc -l`,"
done

This will count all ANSI characters from 32 to 126 - these are the most commonly readable ones. Note that this does not use ignore case.

output from this will be:

- 0,
! - 0,
" - 0,
# - 0,
$ - 0,
% - 0,
& - 0,
' - 0,
( - 0,
) - 0,
* - 0,
+ - 0,
, - 0,
- - 0,
. - 0,
/ - 0,
0 - 0,
1 - 0,
2 - 0,
3 - 0,
4 - 0,
5 - 0,
6 - 0,
7 - 0,
8 - 0,
9 - 0,
: - 0,
; - 0,
< - 0,
= - 0,
> - 0,
? - 0,
@ - 0,
A - 1,
B - 0,
C - 0,
D - 0,
E - 0,
F - 0,
G - 0,
H - 0,
I - 0,
J - 0,
K - 0,
L - 0,
M - 1,
N - 0,
O - 1,
P - 0,
Q - 0,
R - 0,
S - 0,
T - 1,
U - 0,
V - 0,
W - 0,
X - 0,
Y - 0,
Z - 0,
[ - 0,
\ - 0,
] - 0,
^ - 0,
_ - 0,
` - 0,
a - 4,
b - 0,
c - 1,
d - 0,
e - 0,
f - 0,
g - 0,
h - 0,
i - 0,
j - 0,
k - 1,
l - 0,
m - 0,
n - 1,
o - 4,
p - 0,
q - 0,
r - 0,
s - 2,
t - 3,
u - 0,
v - 0,
w - 2,
x - 0,
y - 1,
z - 0,
{ - 0,
| - 0,
} - 0,
~ - 0,
  • If you dont want to ignore case then remove the i from the grep. (in your question you had only 3 in the expected result) – stalet Mar 6 '15 at 8:33
  • oh, thanks. "{a..z}" - it's all symbols from 'a' to 'z'? what about all printable symbols, how we can designate them without listing all of them – Set-xx Mar 6 '15 at 10:58
  • Ive updated my answer with an example on how to extend the search for all readable characters – stalet Mar 6 '15 at 14:48
  • That's a lot of calls to grep the entire input repeatedly. – 200_success Mar 6 '15 at 22:08
3

Here another solution (in awk)...

awk '
        { for (indx=length($0); indx >= 1; --indx)
                ++chars[tolower(substr($0, indx, 1))]
        }
END     { for (c in chars) print c, chars[c]; }
' 1.txt | sort
  • It creates an associative array with each character as the index value and the count as the array value.
  • The END action prints the array.
  • no need to cat file | awk '...': you can directly say awk '...' file. – fedorqui Mar 6 '15 at 22:22
2

The following perl oneliner will do the count. I put the regex in list context (to get the number of matches) and put that into scalar context:

$ perl -e '$a=join("",<>);for("a".."z"){$d=()=$a=~/$_/gi;print"$_ - $d,\n"}' 1.txt
a - 5,
b - 0,
c - 1,
d - 0,
e - 0,
f - 0,
g - 0,
h - 0,
i - 0,
j - 0,
k - 1,
l - 0,
m - 1,
n - 1,
o - 5,
p - 0,
q - 0,
r - 0,
s - 2,
t - 4,
u - 0,
v - 0,
w - 2,
x - 0,
y - 1,
z - 0,
  • To get rid of the trailing comma seems to require a significant rewrite: perl -Mfeature=say -e '$a=join("",<>);say join(",\n", map { sprintf("%s - %d", $_, ($d=()=$a=~/$_/gi)); } ("a".."z"))' – 200_success Mar 6 '15 at 22:45
2

Here is a solution using Python:

#!/usr/bin/env python2
import collections, string
with open('1.txt') as f:
    input_string = f.read().replace('\n', '').lower()
    count_dict = collections.Counter(input_string)
    for char in string.lowercase:
        print char + ' - ' + str(count_dict[char]) + ','

Here we have used the collections module's Counter class to count the number of occurrences of each character, then for printing purpose we have used the string module to get all the lowercase letters by the variable string.lowercase.

Save the above script in a file giving it any name you want e.g. count.py. Now from the same directory where the file is saved you can simply run python count.py to execute the file, from any other directory use the absolute path to the file to execute it i.e. python /absolute/path/to/count.py.

  • Could you please clarify your solution. I mean: create file file_name, put this code, chmod +x etc. etc. etc. – c0rp Mar 6 '15 at 13:23
  • @c0rp: done.... – heemayl Mar 6 '15 at 16:48
1

A while ago I wrote a C program to do that, because I needed it to look at large files and produce some statics.

#include <stdlib.h>
#include <stdint.h>
#include <stdio.h>
#include <ctype.h>
#include <limits.h>
#include <math.h>
#include <sysexits.h>


inline static double square(double x)
{
    return x * x;
}


int main()
{
    static const unsigned distribution_size = 1 << CHAR_BIT;

    int rv = EX_OK;
    uintmax_t *distribution = calloc(distribution_size, sizeof(*distribution));

    {
        int c;
        while ((c = getchar()) != EOF)
            distribution[c]++;

        if (ferror(stdin)) {
            perror("I/O error on standard input");
            rv = EX_IOERR;
        }
    }

    uintmax_t sum = 0;
    for (unsigned i = 0; i != distribution_size; i++)
        sum += distribution[i];
    double avg = (double) sum / distribution_size;

    double var_accum = 0.0;
    for (unsigned i = 0; i != distribution_size; i++)
    {
        const uintmax_t x = distribution[i];

        printf("'%c' (%02X): %20ju", isprint((int) i) ? i : ' ', i, x);
        if (x != 0) {
            var_accum += square((double) x - avg);
            printf(" (%+.2e %%)\n", ((double) x / avg - 1.0) * 100.0);
        } else {
            var_accum += square(avg);
            putchar('\n');
        }
    }

    double stdev = sqrt(var_accum / distribution_size);
    double varcoeff = stdev / avg;
    printf(
        "total: %ju\n"
        "average: %e\n"
        "standard deviation: %e\n"
        "variation coefficient: %e\n",
        sum, avg, stdev, varcoeff);

    free(distribution);
    return rv;
}

compile with (assuming the source code resides in character-distribution.c):

cc -std=c99 -O2 -g0 -o character-distribution character-distribution.c

run with:

./character-distribution < 1.txt

If you don't have a C compiler ready, install GCC:

sudo apt-get install gcc build-essential
0

Similar solution to @heemayl, with tighter code, which works on Python 2.7 and Python 3.

#!/usr/bin/python

import collections
import fileinput
import itertools
import string

count = collections.Counter(itertools.chain(*fileinput.input()))
print(',\n'.join('{} - {}'.format(c, count[c] + count[c.upper()])
                 for c in string.ascii_lowercase))

The first statement, count = collections.Counter(…) does all the real work.

  • fileinput.input() reads every line of the input, which may be piped via stdin or as command-line arguments.
  • * makes it consider a character at a time rather than a line at a time.
  • count = Counter(…) counts occurrences of each character efficiently, in a single pass, and stores the result in the count variable.

The second line just prints the results.

  • '{} - {}'.format(c, count[c] + count[c.upper()]) for c in string.ascii_lowercase makes a list of each character and its count.
  • print(',\n'.join(…)) puts it in the desired format: one per line, separated by commas, but no comma on the last line.
0

GNU awk 4.1

awk -iwalkarray '{for (;NF;NF--) b[$NF]++} END {walk_array(b)}' FS=
[A] = 1
[O] = 1
[w] = 2
[k] = 1
[y] = 1
[T] = 1
[n] = 1
[a] = 4
[o] = 4
[c] = 1
[s] = 2
[t] = 3
[M] = 1

If you have earlier version of GNU awk you can use for (c in b) print c, b[c].

0

Here is the answer using ruby. It is done by changing the string into a uniq list of the different characters and using the count method on each of them.

#!/usr/bin/env ruby

String content = IO.read("1.txt")
content.split("").uniq.sort.each { |chr| puts( chr + ' - ' + content.count(chr).to_s) }

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.