How can I execute a command in a while loop after a background command inside the loop has finished executing, while still looping through the loop? I've included comments in the code to explain what I mean more clearly

while read file; 
      # run command 1 in the background. command 1 takes in "file" as an argument and does some processing on it
      # run command 2 after command 1 is done, but keep going through the loop. Command 2 deletes "file"

EDIT: One solution I can think of is to put the body of the while loop in a script and to run that in the background, but I'm not sure that's the best way of doing it, and if there's a better way

EDIT 2: I tried @steeldriver's suggestion (see comments), but I'm getting a syntax error Here's my code:


set -e 

inotifywait -m -r -e create --format '%w%f' "${MONITORDIR}" | while read NEWFILE; do
    file_dir=$(echo $NEWFILE | rev | cut -d / -f2- | rev)
    if [[ $NEWFILE == *"paired"* ]]; then
        while (( $(ls -1 $file_dir | wc -l) != 2 )); do
            sleep 1

        { kallisto quant -i /home/user/random/Caenorhabditis_elegans.WBcel235.cdna.all.index -o $file_dir -t 12 $(ls $file_dir -1 | head -1) $(ls $file_dir -1 | tail -1) ;
        rm $file_dir/*gz }&
        { kallisto quant -i /home/user/random/Caenorhabditis_elegans.WBcel235.cdna.all.index -o $file_dir --single -t 12 -l 250 -s 30 $(ls $file_dir *gz) ;
        rm $file_dir/*gz }&

Here's the error I get:

./second_queue: line 15: syntax error near unexpected token `else'

  • 1
    Doesn't the wait command fit the bill here - like you asked previously How can I run commands in batches? Jul 14 '20 at 23:05
  • @steeldriver where are you suggesting to put the wait? Because if I wait after command 1, wouldn't that halt the while loop? Jul 14 '20 at 23:09
  • It would halt the while loop until the bacgrounded command exits, then execute the following command - which seems to be exactly what you are asking for? Jul 15 '20 at 0:00
  • 1
    So in that case you would need to place command 2 in the background as well I think ex. { command1 ; command 2; }& Jul 15 '20 at 0:54
  • 2
    It's missing semicolons between rm $file_dir/*gz and the closing } Jul 15 '20 at 1:08

Remember the PID of the long running process, and use that for checking in the loop . bash returns the PID of the background process in $! just after you put the job in the background.

Do something like (untested):

command_1 &
# inside the loop,
if [[ -d /proc/$cpid ]] ; then
   : command_1 still running
   : command_1 not running
  • problem is the long running process depends on what the while loop reads, so I can't put it outside the while loop Jul 15 '20 at 0:51
  • Then put it inside the while. You may have to make cpid an array.
    – waltinator
    Jul 15 '20 at 1:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.