2

I have a media download folder with all sorts of irrelevant files e.g. readme, txt and torrent files in addition to the mp4, avi, mp3 and other valid media files.

I want a script or a set of commands i can run to clean it out via cron or something on a regular basis and go through all directories and sub directories removing unneeded files.

1
  • 2
    So what are needed and what are not-needed files? Be specific about any one type..
    – heemayl
    Commented Apr 22, 2015 at 20:11

3 Answers 3

5

Create a file with the extensions you want to remove, one per line (say, dontwant.files):

.txt
.nfo
.torrent
.csv

Combine find with grep, xargs and rm:

find /path/to/media/directory -type f -print0 |
  grep -zFf /path/to/dontwant.files |
  xargs -0 rm
  • find with -type f and -print0 will print out the names of all the files it finds separated by the ASCII NUL character - the only one which is perfectly safe for delimiting paths and filenames.
  • grepwith -z separates lines by the ASCII NUL character, so that each filename output by find is treated as a separate line. -F disables regex matching, and -f reads in patterns from the file.
  • xargs with -0 reads in lines separated by ASCII NUL characters, and uses them as arguments for the command, in this case rm.

This is easily converted to the the other approach (remove all but some extensions). For example, with a file like want.files, containing:

.mp4
.srt
.avi
.mp3

only the grep command needs changing, using -v to negate matches:

grep -vzFf /path/to/want.files |
8
  • I like the negate method using -v better. Thank you sooo much for this beautiful and thorough answer. I think its the best answer I have ever seen in the Universe. :-) Much thanks. Commented Apr 23, 2015 at 0:10
  • @NormanBird don't be so sure. Do remember to test by using echo instead of rm. :)
    – muru
    Commented Apr 23, 2015 at 0:11
  • when I ran the echo test with find m:// -type f -print0 | grep -zFf dontwant.files | xargs -0 echo and it returned a huge lump of what appears to be all files, those in the dontwant,files and also those I wanted to keep. Looks like all the files.. Please note I am running command from my windows computer from my Git Bash command prompt to search my samba share drive m:// which is ubuntu. Commented Apr 23, 2015 at 11:25
  • @NormanBird Pick one subdirectory of it, run the command, and post the results to paste.ubuntu.com. Did you leave an empty line in dontwant.files?
    – muru
    Commented Apr 23, 2015 at 12:25
  • Here is the paste bin As you can see above it is returning all files in the directory. I do not have .mp4 in the dontwant.files. I also made sure there is no space at the end Commented Apr 23, 2015 at 17:34
2

Based on what we have got so far from your question:

Find and Remove:

If you want to remove all files having .txt or .torrent extension:

find /path/to/dir -type f \( -name '*.txt' -o -name '*.torrent' \) -execdir rm {} +

Alternately, if you want to remove all files that do not have .mp3, .mp4 or .avi extension:

find /path/to/dir -type f -not \( -name '*.mp3' -o -name '*.mp4' -o -name '*.avi' \) -execdir rm {} +

Here we have used find with OR -o option to get our desired files, when we have found the files we removed them by calling a single instance of rm.

Cron:

If you want to run it as a cron job, you can put the command in your user's crontab entry with when you want cron to run this. For example, to run this on Sunday 11:00 PM every week:

00 23 * * 0 find /path/to/dir -type f \( -name '*.txt' -o -name '*.torrent' \) -execdir rm {} +
2
  • You don't need to worry about special characters with -execdir. A -execdir rm {} + is -print0 | xargs -0 rm, without the middleman - find will ensure arguments get passed properly.
    – muru
    Commented Apr 22, 2015 at 22:03
  • @muru: Thanks..did not know that..from manpage any given invocation of command will only list files that exist in the same subdirectory ..that sums it up nicely..
    – heemayl
    Commented Apr 22, 2015 at 22:06
2

The script below cleans up your directory recursively. It can be run from either cron or a shortcut key combination. The use is simple:

  1. Copy it into an empty file, save it as clean_up.py
  2. In the head section of the script, set the extensions to remove, in the line:

    remove = [".txt", ".log"]
    
  3. Run it by the command:

    python3 /path/to/clean_up.py <directory_to_clean_up>
    

The script:

#!/usr/bin/env python3
import os
import sys

#--- set the extensions to remove below
remove = [".txt", ".log"]
#---

for root, dirs, files in os.walk(sys.argv[1]):
    for file in files:
        file = os.path.join(root, file)
        if any([file.endswith(s) for s in remove]):
            os.remove(file)

The negative option

If you'd like to do it the other way around: only keep specific file types (extensions), remove all others, use the (version of the) script below:

#!/usr/bin/env python3
import os
import sys

#--- add all extensions you'd like to keep below
keep = [".mp4", ".mp3"]
#---

for root, dirs, files in os.walk(sys.argv[1]):
    for file in files:
        file = os.path.join(root, file)
        if all([not file.endswith(s) for s in keep]):
            os.remove(file)

To add either one of the scripts above to a shortcut: Choose: System Settings > "Keyboard" > "Shortcuts" > "Custom Shortcuts". Click the "+" and add the command:

python3 /path/to/clean_up.py <directory_to_clean_up>
1
  • This answer is equally juicy , thank you. I like the idea of using the command line shown by heemayl above, which I can create an alias for. thanks again though. Commented Apr 23, 2015 at 0:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .