I'm operating on large amounts of data, and have a protocol involving several scripts. I launch it separately for each input file, simply swapping filename in text file containing the command. I would like to know if it's possible to automate it for all files with certain extensions, when for x in ... is used in the command.

Example of what I have now

mkdir filename.file && for f in *.different_file; do [[ -n $(awk 'NF{i++} END {if (i<14) print FILENAME}' "$f") ]] && rm "$f" ;done

and the functionality I would like to have:

for x in location/*.file do; mkdir $x && for f in *.different_file; do [[ -n $(awk 'NF{i++} END {if (i<14) print FILENAME}' "$f") ]] && rm "$f" ;done ;done

there would be more loops inside the main loop, but not stacking anymore.

How can I achieve something like that?

EDIT: Some more info on what I want, pardon my unprofessional language:

I would like to have two variables working inside two loops, one inside another. I believe using nested loops as suggested by @Lety might be the best solution although it will take some time to test it.

Some more commands I'm using to visualize the issue:

mkdir filename.file && for i in *.file2; do program  /filename.file/filename.file $i> $i.filename.file.out ; done && for f in *.file.out; do [[ -n $(awk 'NF{i++} END {if (i<14) print FILENAME}' "$f") ]] && rm "$f" ;done && for i in *.file.out; do grep -v '^#' $i >> $i.trim; done

I would like to have filename.file set as a variable that would be the same for all loops:

for x in *.file do; mkdir $x && for i in *.file2; do program  /$x/$x $i> $i.$x.out ; done && for f in *.file.out; do [[ -n $(awk 'NF{i++} END {if (i<14) print FILENAME}' "$f") ]] && rm "$f" ;done && for i in *.file.out; do grep -v '^#' $i >> $i.trim; done; done

I hope this clarifies a bit. I will investigate nested loops in bash script, another difficulty is this it has to work for cygwin as well, but I guess it shouldn't matter as far as the commands are right.

  • 1
    Sorry, but to me it is not clear what you want to do. Nested loops are possible, try to look at this link tldp.org/LDP/abs/html/nestedloops.html. In any case, could you update your question with more details? So maybe someone can help you. – Lety May 14 '15 at 17:33
  • I'd say get rid of && and use ; if [ mkdir $x ] then ...(for statement here)..fi or mkdir $x; if [ $? == 0 ] then ...(for statement) ...fi. Let me know if this works. I'll post as answer – Sergiy Kolodyazhnyy May 14 '15 at 17:47
  • also you can use for within awk, but its c-style for loop – Sergiy Kolodyazhnyy May 14 '15 at 17:48
  • 2
    First of all, do yourself (and others) a favour and put your one liners in a script and use proper indentation. It improves clarity, allows making changes and re-testing much easier. – Marcin Kaminski May 14 '15 at 18:53
  • @Lety I will look into nested loops, but I would like to have this working on cygwin as well, and I noticed some people had problems with this as bash script. – Voii May 14 '15 at 18:56

Many thanks to @Lety for pointing me where to look for the answer. Nested loops worked perfectly.


for x in *.hmm
    mkdir $x.dir
    for i in *.emitcons
        program --different_output $i.$x.1mtblout $x $i> $i.$x.1mout
        rm $i.$x.1mout
        let "inner+=1"
    for f in *.1mtblout
        [[ -n $(awk 'NF{i++} END {if (i<14) print FILENAME}' "$f") ]] && rm "$f"
    for i in *.1mtblout
        grep -v '^#' $i >> $i.trim
    for i in *.trim
        echo $i >> $x\_trim_table
        sed -i -e '$a\' $x\_trim_table
        cat $i >> $x\_trim_table
        sed -i -e '$a\' $x\_trim_table
    let "outer+=1"
exit 0

...and much more, but it all works perfect.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.