3

For my text-mining project (with an Enron data set), I have this Python script:

from glob import glob
import fileinput
with open('/media/output/inbox.txt', 'w') as out:
for line in fileinput.input(glob('*.')):
    if 'Subject:' in line:
        out.write(line)

Inside more than 1000 folders with one-step depth folder structure like this,

-folder
  --folder 2.

by using this terminal command and it works perfectly.

find allen-p fischer-m kitchen-l phanis-s smith-m arnold-j forney-j kuykendall-t pimenov-v solberg-g ,,,,,,,,,,, zufferli-j -maxdepth 1 -exec cp file.py {} \;

By looking at the Python script, I try to extract a particular line from a file and paste it in another file.

Then I realized that I need to run these scripts one by one.

How can I run each script automatically one after another and with all the folder structure?

I am looking for some terminal code like above or a Python script.

  • 1
    Which scripts? You only show one. – muru Jun 25 '15 at 12:26
  • yes the same script i copy to every folder – Miller Jun 25 '15 at 12:28
3

You don't need to copy the script to each directory. Just modify the find command to run it instead:

find allen-p ... zufferli-j -maxdepth 1 -type d -execdir python "$PWD"/file.py \;

The -execdir option runs the command inside each directory. We use $PWD to give the full path to the script, so that a changing working directory doesn't affect where the command looks for the script.

However, since you have copied it anyway, you can skip the $PWD:

find allen-p ... zufferli-j -maxdepth 1 -type d -execdir python file.py \;

Note

You didn't provide a type check in your find command (like I did with -type d above). Without it, every file in those directories has now been overwritten with file.py. You only want to copy to (and run in) directories, so you should have added a -type d to it.

1

You could also write a loop that moves into each directory and runs the script before moving back:

for dir in allen-p fischer-m ... solberg-g; do
    cd "$d" && /path/to/python/script.py && ../
done

Note that this will fail if the python script fails in any of the directories.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.