How can I execute a command in a while loop after a background command inside the loop has finished executing, while still looping through the loop? I've included comments in the code to explain what I mean more clearly

while read file; 
      # run command 1 in the background. command 1 takes in "file" as an argument and does some processing on it
      # run command 2 after command 1 is done, but keep going through the loop. Command 2 deletes "file"

EDIT: One solution I can think of is to put the body of the while loop in a script and to run that in the background, but I'm not sure that's the best way of doing it, and if there's a better way

EDIT 2: I tried @steeldriver's suggestion (see comments), but I'm getting a syntax error Here's my code:


set -e 

inotifywait -m -r -e create --format '%w%f' "${MONITORDIR}" | while read NEWFILE; do
    file_dir=$(echo $NEWFILE | rev | cut -d / -f2- | rev)
    if [[ $NEWFILE == *"paired"* ]]; then
        while (( $(ls -1 $file_dir | wc -l) != 2 )); do
            sleep 1

        { kallisto quant -i /home/user/random/Caenorhabditis_elegans.WBcel235.cdna.all.index -o $file_dir -t 12 $(ls $file_dir -1 | head -1) $(ls $file_dir -1 | tail -1) ;
        rm $file_dir/*gz }&
        { kallisto quant -i /home/user/random/Caenorhabditis_elegans.WBcel235.cdna.all.index -o $file_dir --single -t 12 -l 250 -s 30 $(ls $file_dir *gz) ;
        rm $file_dir/*gz }&

Here's the error I get:

./second_queue: line 15: syntax error near unexpected token `else'

  • 1
    Doesn't the wait command fit the bill here - like you asked previously How can I run commands in batches? – steeldriver Jul 14 at 23:05
  • @steeldriver where are you suggesting to put the wait? Because if I wait after command 1, wouldn't that halt the while loop? – An Ignorant Wanderer Jul 14 at 23:09
  • It would halt the while loop until the bacgrounded command exits, then execute the following command - which seems to be exactly what you are asking for? – steeldriver Jul 15 at 0:00
  • 1
    So in that case you would need to place command 2 in the background as well I think ex. { command1 ; command 2; }& – steeldriver Jul 15 at 0:54
  • 2
    It's missing semicolons between rm $file_dir/*gz and the closing } – steeldriver Jul 15 at 1:08

Remember the PID of the long running process, and use that for checking in the loop . bash returns the PID of the background process in $! just after you put the job in the background.

Do something like (untested):

command_1 &
# inside the loop,
if [[ -d /proc/$cpid ]] ; then
   : command_1 still running
   : command_1 not running
| improve this answer | |
  • problem is the long running process depends on what the while loop reads, so I can't put it outside the while loop – An Ignorant Wanderer Jul 15 at 0:51
  • Then put it inside the while. You may have to make cpid an array. – waltinator Jul 15 at 1:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.