I have been struggling for a long time to write a script which would have 2 arguments, 1 asking to select a year and 2 asking to select if I want minimum, maximum, average or all to be shown as the last line from the files related to the year selected.

Basically, I have a directory which contains subdirectories of different years (2000, 2001, 2002 etc.) in those directories are subdirectories for months and days which contain (a) file(s) informing about populations (not real info though) of different cities as the last line. That's a part of the tree of the directory:

.
|-- 2000
|   |-- 01
|   |   `-- 18
|   |       `-- ff_1177818640
|   |-- 02
|   |   |-- 02
|   |   |   `-- ff_1669027271
|   |   |-- 03
|   |   |   `-- ff_234075290
|   |   |-- 10
|   |   |   `-- ff_1584524530
|   |   |-- 14
|   |   |   `-- ff_113807345
|   |   `-- 17
|   |       `-- ff_1452228827
|   |-- 03
|   |   |-- 06
|   |   |   `-- ff_58914249
|   |   `-- 11
|   |       `-- ff_2828212321
|   |-- 04
|   |   `-- 17
|   |       `-- ff_302131884
|   |-- 06
|   |   `-- 13
|   |       `-- ff_2175615745
|   |-- 07
|   |   |-- 07
|   |   |   `-- ff_918426998
|   |   `-- 24
|   |       `-- ff_2808316425
|   |-- 08
|   |   `-- 27
|   |       `-- ff_1449825497
|   |-- 09
|   |   `-- 19
|   |       `-- ff_110255856
|   `-- 12
|       `-- 08
|           `-- ff_1621190
|-- 2001
|   |-- 03
|   |   `-- 21
|   |       `-- ff_517010375
|   |-- 05
|   |   `-- 27
|   |       `-- ff_1458621098
|   |-- 06
|   |   |-- 07
|   |   |   `-- ff_155853916
|   |   |-- 25
|   |   |   |-- ff_2382312387
|   |   |   `-- ff_270731174
|   |   `-- 29
|   |       `-- ff_3228522859
|   |-- 07
|   |   `-- 28
|   |       `-- ff_3215021752
|   |-- 09
|   |   `-- 24
|   |       `-- ff_1080314364
|   `-- 11
|       `-- 24
|           `-- ff_2313722442

All files are formatted the same way:

2019-04-03
Wednesday
Newcastle-upon-Tyne
255362

I need to write a script to be asked which year I need (selecting that directory) and then asking if I want average, minimum, maximum or all of the above to be displayed for population (which is the last line of the files).

This is what I have so far:

#!/bin/bash

function min () {
    echo $(sort -n populations | head -1)
}

function max () {
    echo $(sort -n populations | tail -1)
}

function avg () {
    count=0
    sum=0
    while read line ; do
        num='echo ${line#* }'
        sum='expr $sum + $num'
        count='expr $count + 1'
    done < populations
    avg='expr $sum / $count'
    echo $avg
}

echo "Please enter the year: "
read s1
echo "
        Enter an option:
        1. Minimum
        2. Maximum
        3. Average
        4. All"
read s2
#echo $s2
for file in $(find ~/filesToSort/$s1 -type f) ; do
    tail -1 $file >> populations
done
echo $(cat populations)
#min
#max
#avg
rm populations

This lets me pick directories, but does not give me the answers that I need, just spits the last lines of my files.

  • so you want to do math from numbers stored in the 'populations' file ? – cmak.fr May 28 at 16:10

If I was implementing this in bash, I'd do the following. I won't comment on it much: feel free to ask specific questions though -- check the bash man page first if you don't know how a particular command works.

#!/bin/bash

# read the population from all the files
# map the filename to it's population figure
declare -A population
while IFS= read -d '' -r filename; do
    population["$filename"]=$(tail -1 "$filename")
done < <(find . -type f -print0)

# prompt the user for the year
read -rp "What year? " year

# find the relevant files for that year
year_files=()
for filename in "${!population[@]}"; do
    [[ $filename == ./"$year"/* ]] && year_files+=("$filename")
done
if [[ "${#year_files[@]}" -eq 0 ]]; then
    echo "No files for year '$year'"
    exit 1
fi

PS3="Select a function to calculate: "
select func in minimum maximum average quit; do
    case $func in
        minimum)
            min=${population[${year_files[0]}]}
            for file in "${year_files[@]}"; do
                if (( min > ${population[$file]} )); then
                    min=${population[$file]}
                fi
            done
            echo "Minimum for $year is $min"
            ;;
        maximum)
            max=${population[${year_files[0]}]}
            for file in "${year_files[@]}"; do
                if (( max < ${population[$file]} )); then
                    max=${population[$file]}
                fi
            done
            echo "Maximum for $year is $max"
            ;;
        average)
            count=0 sum=0
            for file in "${year_files[@]}"; do
                (( sum += ${population[$file]} ))
                (( count++ ))
            done
            echo "Average for $year is $(( sum / count ))"
            ;;
        quit) exit ;;
    esac
done
  • There should be another select option as "All" – devWeek May 29 at 3:02

I write a simple awk script which does the same as what you are doing:

# read 'year' & 'option' from user
# or you can pass as argument to the command $1<-->$year & $2<-->$option

find /path/to/$year -type f -exec \
    awk -v select=$option '
        FNR==4 { sum+=$0; avg=sum/++count; 
                 max=(max>=$0?max:$0);
                 if (count==1) min=$0;
        }
        count>1 { min=(min<=$0?min:$0);
        }
    END{ stats=min","max","avg","min"\n"max"\n"avg;
         split(stats, to_print,",");
         print to_print[select];
    }' {} +

Explanation inline:

# read 'year' & 'option' from user
# or you can pass as argument to the command $1<-->$year & $2<-->$option

find /path/to/$year -type f -exec \
# find all files under "/path/to/$year". $year will be substitute with the value 
# of 'year' variable read from user-input or replace it with '$1' as first argument to the command

    awk -v select=$option '
    # read the value of shell 'option' variable into an awk 'select' variable 
    # replace with '$2' as argument to the command

        FNR==4 { sum+=$0; avg=sum/++count;
        # if it's 4th line of each input file, sum-up the value into 'sum' variable
        # and calculate the 'avg' too when 'count' will increment once each 4th record in a file is read

                 max=(max>=$0?max:$0);
                 # its a Ternary operator (condition?if-true:if-false) and finding maximum value

                 if (count==1) min=$0;
                 # keep the first file's 4th line's value as minimum. you could use `NR==4` instead
        }
        count>1 { min=(min<=$0?min:$0);
        # same as max, update the 'min' if value in current file is smaller than 'min' in previous file
        }
    END{ stats=min","max","avg","min"\n"max"\n"avg;
    # saving all variables' value into single variable with comma separated. I used <min"\n"max"\n"avg> as 
    # fourth element which we will use it as "All" option that each separated with newlines.

         split(stats, to_print, ",");
         # building an array called 'to_print' from 'stats' variable above with comma separator to distinguish 
         # the elements from each other.

         print to_print[select];
         # this will print the element which user-input as an option.
         # if input 1: will print 'min'
         # if input 2: will print 'max'
         # if input 3: will print 'avg'
         # if input 4: will print 'min' \n 'max' '\n' avg
    }' {} +

As written, the script will do nothing but print the populations because avg, etc. are commented out.

To calculate the avg, those populations must be sent to the avg() function with something like...

echo "$(cat populations | avg)"

Similar lines would be added for min() and max().

You could use a case statement to call the appropriate function(s)...

  :
done
#
case s2
  1|4) echo "$(cat populations | min)" ;;&
  2|4) echo "$(cat populations | max)" ;;&
  3|4) echo "$(cat populations | avg)";;
esac
#
rm populations

The 1|4) echo ... causes the echo to run if either 1 or 4 are entered. Thus, all 3 will execute if 4 was entered.

  • 1
    PerlDuck - You are correct (but it's ';;&'). Corrected. Thanks. – DocSalvager May 31 at 11:17

Thanks for all of the answers, here's what I ended up with:

#!/bin/bash
### Returns the minimum value by sorting the population file's data and displaying the top line.
 function min () {
         echo "Minimum Population: "$(sort -n populations | head -1)
 }
  ### Returns the maximum value by sorting the population file's data and displaying the bottom line.
 function max () {
         echo "Maximum Population: "$(sort -n populations | tail -1)
 }
  ### A function to return the average number of population.
 function avg () {
         count=0
         sum=0
         while read line ; do
                 num=`echo ${line#* }`
                 sum=`expr $sum + $num`
                 count=`expr $count + 1`
         done < populations
         avg=`expr $sum / $count`
         echo "Average Population: "$avg
 }
  ### Advises what the script does and asks for an imput of a year.
 echo "
         ######################
         # Population adviser #
         ######################
          Please enter the year: "
 read s1
  ### If statement checking the year entered is available, if not then the user is informed of invalid selection and program terminates.
 if [[ $s1 -ge 2000 && $s1 -le 2019 && $s1 -ne 2009 ]] ; then
         continue 2>/dev/null
 else
         echo "The year you entered is not valid, program terminating"
         exit
 fi
  ### Prompts user for input
 echo "
         Enter an option:
         1. Minimum
         2. Maximum
         3. Average
         4. All
  -----(minimum) (maximum) (average) (all)-----
 "
 read s2
  ### Loops through all files within the given directory path and appends the population of each file to the population list
 for file in $(find ~/filesToSort/$s1 -type f) ; do
         tail -1 $file >> populations
 done
  ### If statement to validate user input and then use the function(s) required
 if [ "$s2" == "minimum" ] ; then
         min
 elif [ "$s2" == "maximum" ] ; then
         max
 elif [ "$s2" == "average" ] ; then
         avg
 elif [ "$s2" == "all" ] ; then
         min
         max
         avg
 else
         echo "The option you chose is invalid, program terminating"
         rm populations
         exit
 fi
  ### Removes "populations" file upon completion
 rm populations

When choosing the option (1-4) instead of putting numbers a word has to be put in instead, which I hate but was asked to do it this way.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.