Running a directory full of .sh files with one command

Question

I've got a directory with lots of .sh files in it, all of which, when run, open a terminal and begin a youtube-dl download. Since there are so many, I was hoping to find a way to activate them all at once, opening several different terminals immediately, as opposed to executing them all separately. I'm still very new to programming, so I'm not exactly sure how to do this and whether or not I could create a script to run them all, use a command, etc. Any help is appreciated, thanks.

Answer 1

While you can indeed run all .sh files in a directory with a for loop as suggested by Yaron, this is really an over complex approach to take. Don't write many scripts if the only difference between them is the RL they will download! And there's absolutely no reason to spawn a terminal window for each of them either!

Instead, write your YouTube URLs into a file, one per line:

http://youtube.com/foo
http://youtube.com/bar
http://youtube.com/baz

Then, to download them, use (file is the name of the file with the URLs):

while read url; do
    youtube-dl "$url"
done < file

That will download each video in the file, one after the other. If you want to download them all at the same time (not a good idea if you have too many), you can run each download command in the background by adding &:

while read url; do
    youtube-dl "$url" &
done < file

Answer 2

Note, the files should be execute-able - you might want to execute the following command in the directory holding the .sh files before running the above script:

chmod +x *.sh

The following script will take all files ends with .sh and will execute them.

#/bin/bash
for shfile in *.sh
do
    ./$shfile
done

You can copy/paste the above lines (without the first line #/bin/bash) into your terminal, and it will do the trick (thanks @terdon for your comment).

You can save this script in other folder (e.g. /tmp/execute_all.sh, and make it execute-able using:

chmod +x /tmp/execute_all.sh

Then go into the directory with the .sh files and execute the script using:

/tmp/execute_all.sh

The script will try to execute all files which ends with .sh

Answer 3

I have experienced that some video sites are not very stable, and that I have to retry the download several times. GNU Parallel can do that:

cat urls.txt | parallel --retries 50 --timeout 1000% --joblog +my.log youtube-dl

This will run one job per CPU core and try 50 times if the download fails. It will kill jobs that take 10 longer than the median run time (e.g. if the download is stuck). It will also append to the file my.log so you can see the status of each job.

Answer 4

If the program is web based i.e. for linux, Try making a php file to process the shell. and a shell file to handle the php..

For instance: runAllShell.php file can contain a loop.:

<?php
// Developed by A T.
// Count the files
$directory = '/put/directory/path_here/';
$files = glob($directory . '*.*');

if ( $files !== false )
{
    $filecounter = count( $files );

}
else
{
    echo "No Files were found";
}
$fileCount = $filecounter;

// Start the index
$fileNumber = 1;
while($fileNumber <= fileCount){
shell_exec('$fileNumber . ".sh"');
// get some feedback
echo "The ".$fileNumber." file has been excecuted";
// increment file number
$fileNumber++;

}
?>

make sure that all the .sh files in the directory are numerically ordered for this to work i.e: 1.sh 2.sh 3.sh and so on.

Answer 5

You can use the command line utility run-parts. It runs all scripts in a directory.

How to call it:

   run-parts  [--test]  [--verbose]  [--debug]  [--report]  [--lsbsysinit]
   [--regex=RE]   [--umask=umask]    [--arg=argument]    [--exit-on-error]
   [--help] [--version] [--list] [--reverse] [--] DIRECTORY

The square brackets indicate optional parameters. Don’t type them. Only one parameter is mandatory: the DIRECTORY. In most cases, no other parameter is needed.

Description from the man page: run-parts runs all the executable files named within constraints, found in directory DIRECTORY. Other files and directories are silently ignored.