I run "ls -lX" over files umbrella31_*.xvg. I run a command that searches the number on the fifth column of the ls command output, that is larger let's say than 20000. It looks like this:

ls -lX umbrella31_*log |  awk '{if($5 >=20000) {print}}' | wc -l

and outputs a number (the number of lines for which the number in column 5 is > 20000).

When I include the above command in a script:

#!/bin/bash -x

ls -lX umbrella31_*log |  awk '{if($5 >=20000) {print}}' | wc -l

and run it, I get in the screen the result of "ls" printed as well (which I don't want). How can I get my script behave like my on-screen command, and print only the desired number of lines?


Your script will print each of the commands in your pipeline to the terminal because you are running it with the -x flag. Fromman bash:

   -x        Print commands and their arguments as they are executed.

However, your approach using ls and wc is not the best way to count files.

To find file that is >= 20000 you can use find:

find -type f -maxdepth 1 -name 'umbrella31_*log' -size +19999c -ls

(because of how find interpreters + sign (greater than rounded up) you get n+1, therefore the odd
-size n)

count output:
(when we count files we just print a newline because we dont really need an output)

wc -l < <(find -maxdepth 1 -type f -name 'umbrella31_*log' -size +19999c -printf \\n)

-maxdepth n
  Descend at most levels (a non-negative integer) levels of directories below the starting-points.
-size n
  File uses n units of space, rounding up.
  List current file in ls -dils format on standard output.

  • 2
    This will return false count output for file names that include newlines. Better use something like this: find -maxdepth 1 -type f -name 'umbrella31_*log' -size +20k -printf '\n' | wc -l – pLumo Oct 11 '19 at 11:24
  • 7
    This answer is missing the critical part of why the results of ls are printed, as that is not at all what -x does. If that's what -x did, one would expect the result of the awk call to be printed as well. However, -x is not printing out the results of ls, it's printing out its arguments, which contain the filenames because of shell expansion of umbrella31_*log. Interestingly, as there is a character limit on argument lists, this may have eventually caused errors — another reason why the find solution is superior. – Félix Saparelli Oct 11 '19 at 23:29

Updated Answer

After posting script used in the question it was discovered:

#!/bin/bash -x

was used where the -x option outputs all commands to the terminal.

Removing the -x solved the original problem.

Original Answer

You're missing the argument flag indicator so this:

ls lX umbrella31_*log |  awk '{if($5 >=20000) {print}}' | wc -l

should be this instead:

ls -lX umbrella31_*log |  awk '{if($5 >=20000) {print}}' | wc -l

On my system looking for bash scripts it works like so:

$ ls -lX *.sh
-rwxrwxr-x 1 rick rick 4183 Jul  1 10:48 aptfileparse.sh
-rwxrwxr-x 1 rick rick  339 Jul 24 17:26 checkrunning.sh
-rwxrwxr-x 1 rick rick  506 Jul 15 17:54 Downloads.sh
-rwxrwxr-x 1 rick rick   78 Jul  6 11:28 runall.sh

$ ls -lX *.sh | awk '{if($5 >=200) {print}}' | wc -l
  • @PanagiotisKrokidas I just put the command into a script and it runs the same as if typed in the terminal. Can you update your question with your script contents? Maybe there is something extra in it that needs to be changed. – WinEunuuchs2Unix Oct 11 '19 at 11:17
  • try doing it without | wc -l – bac0n Oct 11 '19 at 11:29
  • @bac0n Yes removing the wc -l would stop the count from coming out and print all the lines instead. That is the most likely error in OP's script. – WinEunuuchs2Unix Oct 11 '19 at 11:43
  • count it with awk instead – bac0n Oct 11 '19 at 11:45
  • @bac0n Yes there are many ways to skin a cat. What I was trying to do was find out what is broken in the current script, not to replace it with a different script. – WinEunuuchs2Unix Oct 11 '19 at 11:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.