Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have some files with different extensions such as *.pdf, *.mp3, *.jpg and a few others. All of them are stored in a parent directory.

How can I get a list of all extensions, create some folders based on these extensions and then move all files into their relevant folders?

share|improve this question
3  
KasiyA, I am speechless and impressed by the gesture. – Jacob Vlijm Jan 1 '15 at 8:10
1  
@JacobVlijm seconded! – muru Jan 3 '15 at 4:09
1  
@KasiyA, you're so generous :) – user300458 Jan 6 '15 at 23:14
up vote 12 down vote accepted
+200

The python script below does the job. Hidden files are stored separately in a folder , as well as files without extension.

Since it might be used for a wider range of purposes, I added a few options:

  • You can set extensions you'd like to exclude from the "reorganization". If you simply want to move all, set exclude = ()
  • You can choose what to do with empty folders (remove_emptyfolders = True or False)
  • In case you would like to copy the files instead of moving them, replace the line:
shutil.move(subject, new_dir+"/"+name)

by:

shutil.copy(subject, new_dir+"/"+name) 

The script:

#!/usr/bin/env python3

import os
import subprocess
import shutil

# --------------------------------------------------------
reorg_dir = "/path/to/directory_to_reorganize"
exclude = (".jpg") # for example
remove_emptyfolders = True
# ---------------------------------------------------------

for root, dirs, files in os.walk(reorg_dir):
    for name in files:
        subject = root+"/"+name
        if name.startswith("."):
            extension = ".hidden_files"
        elif not "." in name:
            extension = ".without_extension"
        else:
            extension = name[name.rfind("."):]
        if not extension in exclude:
            new_dir = reorg_dir+"/"+extension[1:]
            if not os.path.exists(new_dir):
                os.mkdir(new_dir)
            shutil.move(subject, new_dir+"/"+name)

def cleanup():
    filelist = []
    for root, dirs, files in os.walk(reorg_dir):
        for name in files:
            filelist.append(root+"/"+name)
    directories = [item[0] for item in os.walk(reorg_dir)]
    for dr in directories:
        matches = [item for item in filelist if dr in item]
        if len(matches) == 0:
            try:
                shutil.rmtree(dr)
            except FileNotFoundError:
                pass

if remove_emptyfolders == True:
    cleanup()

IF there is a risk of unwanted overwriting duplicate files

At the expense of a few extra lines, we can prevent overwriting possible duplicates. With the code below, duplicates will be renamed as:

duplicate_1_filename, duplicate_2_filename 

etc.

The script:

#!/usr/bin/env python3

import os
import subprocess
import shutil

# --------------------------------------------------------
reorg_dir = "/path/to/directory_to_reorganize"
exclude = (".jpg") # for example
remove_emptyfolders = True
# ---------------------------------------------------------

for root, dirs, files in os.walk(reorg_dir):
    for name in files:
        subject = root+"/"+name
        if name.startswith("."):
            extension = ".hidden_files"
        elif not "." in name:
            extension = ".without_extension"
        else:
            extension = name[name.rfind("."):]
        if not extension in exclude:
            new_dir = reorg_dir+"/"+extension[1:]
            if not os.path.exists(new_dir):
                os.mkdir(new_dir)
            n = 1; name_orig = name
            while os.path.exists(new_dir+"/"+name):
                name = "duplicate_"+str(n)+"_"+name_orig
                n = n+1
            newfile = new_dir+"/"+name
            shutil.move(subject, newfile)

def cleanup():
    filelist = []
    for root, dirs, files in os.walk(reorg_dir):
        for name in files:
            filelist.append(root+"/"+name)
    directories = [item[0] for item in os.walk(reorg_dir)]
    for dr in directories:
        matches = [item for item in filelist if dr in item]
        if len(matches) == 0:
            try:
                shutil.rmtree(dr)
            except FileNotFoundError:
                pass

if remove_emptyfolders == True:
    cleanup()

EDIT

With OP in mind, we all forgot to add an instruction on how to use. Since duplicate questions might (and do) appear, it might be useful nevertheless.

How to use

  1. Copy either one of the scripts into an empty file, save it as reorganize.py
  2. In the head section of the script, set the targeted directory (with the files to reorganize):

    reorg_dir = "/path/to/directory_to_reorganize" 
    

    (use quotes if the directory contains spaces)

    possible extensions you'd like to exclude (probably none, like below):

    exclude = ()
    

    and if you'd like to remove empty folders afterwards:

    remove_emptyfolders = True
    
  3. Run the script with the command:

    python3 /path/to/reorganize.py
    

NB if you'd like to copy the files instead of move, replace:

shutil.move(subject, new_dir+"/"+name)

by:

shutil.copy(subject, new_dir+"/"+name)

Please try first on a small sample.

share|improve this answer
up vote 13 down vote
+100

You can use find with a somewhat complex exec command:

find . -iname '*?.?*' -type f -exec bash -c 'EXT="${0##*.}"; mkdir -p "$PWD/${EXT}_dir"; cp --target-directory="$PWD/${EXT}_dir" "$0"' {} \;

# '*?.?*' requires at least one character before and after the '.', 
# so that files like .bashrc and blah. are avoided.
# EXT="${0##*.}" - get the extension
# mkdir -p $PWD/${EXT}_dir - make the folder, ignore if it exists

Replace cp with echo for a dry run.


More efficient and tidier would be to save the bash command in a script (say, at /path/to/the/script.sh):

#! /bin/bash

for i
do
    EXT="${i##*.}" 
    mkdir -p "$PWD/${EXT}_dir"
    mv --target-directory="$PWD/${EXT}_dir" "$i" 
done

And then run find:

find . -iname '*?.?*' -type f -exec /path/to/the/script.sh {} +

This approach is pretty flexible. For example, to use the filename instead of the extension (filename.ext), we'd use this for EXT:

NAME="${i##*/}"
EXT="${NAME%.*}"
share|improve this answer
    
+1; the -iname '*.*' should take care of the corner cases I was worried about... nice idea! – Rmano Aug 27 '14 at 9:53
    
@Rmano not the *.fig.bak or .profile/.bashrc ones, but it should only handle files with extensions, at least. Thanks. – muru Aug 27 '14 at 9:55
ls | gawk -F. 'NF>1 {f= $NF "-DIR"; system("mkdir -p " f ";mv " $0 " " f)}'

Calculating the list of extensions (after moving):

ls -d *-DIR

Calculating the list of extensions (before moving):

ls -X | grep -Po '(?<=\.)(\w+)$'| uniq -c | sort -n

(in this last exemple, we are calculating the number of files for each extension and sorting it)

share|improve this answer
1  
sorry: a typo "mkdir -f" was corrected to "mkdir -p" (to ignore if dir already exist) – JJoao Dec 29 '14 at 16:32
    
Isn't uniq supposed to be applied after sort? And please do not parse the output of ls. – muru Dec 30 '14 at 22:21
    
@muru, (part 1) ls -X guarantees that extensions are sorted. Final sort was just to order the extensions table by number of occurrences -- relevance. (I'm I correct?). – JJoao Dec 30 '14 at 23:32
    
@muru, (part 2) ls -X | grep -Po '(?<=\.)(\w+)$' was my first idea to get the sorted list of extensions. Is it very bad? What do you suggest? – JJoao Dec 30 '14 at 23:35
    
I forgot what ls -X does. As to why I recommend against ls, see unix.stackexchange.com/q/128985/70524 and unix.stackexchange.com/q/112125/70524. To achieve what you do, I'd go about a longer way: find . -type f -name '*?.?*' -print0 | sed -z 's/.*\.//' | sort -zu (with an optional | uniq -cz, if counts are needed). And find ... -print0 | gawk -v RS='\0' (even though that is not very portable) for the first. – muru Dec 31 '14 at 2:06

Try this shell script.

#!/bin/sh
src=`dirname "$1"`/`basename "$1"`;
for file in "$src"/*?.?*; do
  if test -f "$file"; then
    dest="$src${file##*.}"_files;
    mkdir -p "$dest";
    mv "$file" "$dest";
  fi;
done;

# pass the directory to re-organize as first argument
# moves only regular files which have extension
# ignores other type of files including
# files having no extension, hidden files, directories, and links.
share|improve this answer
1  
I am sorry, that is an error. I should have to substituted every occurrence of filepath with file. I'll correct that directly. – Prashant Karmakar Dec 29 '14 at 16:35
    
Please do not parse the output of ls. Instead, do for file in "$src"/*?.?*; do .. – muru Dec 30 '14 at 22:22
    
@muru will that work correctly if the name of some file has spaces? – Prashant Karmakar Dec 31 '14 at 6:10
    
@PrashantKarmakar yes, whereas read may have unexpected behaviour. You should also be quoting the variables in the mkdir and mv commands. – muru Dec 31 '14 at 6:12
    
Test it out, if you will: for i in *; do printf "%s\n" "$i"; done; for i in $(ls -d); do printf "%s\n" "$i"; done – muru Dec 31 '14 at 6:14

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.