2

I'm operating on large amounts of data, and have a protocol involving several scripts. I launch it separately for each input file, simply swapping filename in text file containing the command. I would like to know if it's possible to automate it for all files with certain extensions, when for x in ... is used in the command.

Example of what I have now

mkdir filename.file && for f in *.different_file; do [[ -n $(awk 'NF{i++} END {if (i<14) print FILENAME}' "$f") ]] && rm "$f" ;done

and the functionality I would like to have:

for x in location/*.file do; mkdir $x && for f in *.different_file; do [[ -n $(awk 'NF{i++} END {if (i<14) print FILENAME}' "$f") ]] && rm "$f" ;done ;done

there would be more loops inside the main loop, but not stacking anymore.

How can I achieve something like that?

EDIT: Some more info on what I want, pardon my unprofessional language:

I would like to have two variables working inside two loops, one inside another. I believe using nested loops as suggested by @Lety might be the best solution although it will take some time to test it.

Some more commands I'm using to visualize the issue:

mkdir filename.file && for i in *.file2; do program  /filename.file/filename.file $i> $i.filename.file.out ; done && for f in *.file.out; do [[ -n $(awk 'NF{i++} END {if (i<14) print FILENAME}' "$f") ]] && rm "$f" ;done && for i in *.file.out; do grep -v '^#' $i >> $i.trim; done

I would like to have filename.file set as a variable that would be the same for all loops:

for x in *.file do; mkdir $x && for i in *.file2; do program  /$x/$x $i> $i.$x.out ; done && for f in *.file.out; do [[ -n $(awk 'NF{i++} END {if (i<14) print FILENAME}' "$f") ]] && rm "$f" ;done && for i in *.file.out; do grep -v '^#' $i >> $i.trim; done; done

I hope this clarifies a bit. I will investigate nested loops in bash script, another difficulty is this it has to work for cygwin as well, but I guess it shouldn't matter as far as the commands are right.

7
  • 1
    Sorry, but to me it is not clear what you want to do. Nested loops are possible, try to look at this link tldp.org/LDP/abs/html/nestedloops.html. In any case, could you update your question with more details? So maybe someone can help you.
    – Lety
    May 14, 2015 at 17:33
  • I'd say get rid of && and use ; if [ mkdir $x ] then ...(for statement here)..fi or mkdir $x; if [ $? == 0 ] then ...(for statement) ...fi. Let me know if this works. I'll post as answer May 14, 2015 at 17:47
  • also you can use for within awk, but its c-style for loop May 14, 2015 at 17:48
  • 2
    First of all, do yourself (and others) a favour and put your one liners in a script and use proper indentation. It improves clarity, allows making changes and re-testing much easier. May 14, 2015 at 18:53
  • @Lety I will look into nested loops, but I would like to have this working on cygwin as well, and I noticed some people had problems with this as bash script.
    – Voii
    May 14, 2015 at 18:56

1 Answer 1

1

Many thanks to @Lety for pointing me where to look for the answer. Nested loops worked perfectly.

#!/bin/bash


outer=1
for x in *.hmm
do
    mkdir $x.dir
    inner=1
    for i in *.emitcons
    do
        program --different_output $i.$x.1mtblout $x $i> $i.$x.1mout
        rm $i.$x.1mout
        let "inner+=1"
    done
    for f in *.1mtblout
    do 
        [[ -n $(awk 'NF{i++} END {if (i<14) print FILENAME}' "$f") ]] && rm "$f"
    done
    for i in *.1mtblout
    do
        grep -v '^#' $i >> $i.trim
    done
    for i in *.trim
    do
        echo $i >> $x\_trim_table
        sed -i -e '$a\' $x\_trim_table
        cat $i >> $x\_trim_table
        sed -i -e '$a\' $x\_trim_table
    done
    let "outer+=1"
echo
done
exit 0

...and much more, but it all works perfect.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .