Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.


I have some files with different extension such as *.pdf, *.mp3, *.jpg and other extension that I don't know what they are, which stored in parent directory.

How can I get list of all extensions and create some folders based on files extension and then move all related files into its own folder?

share|improve this question
3  
KasiyA, I am speechless and impressed by the gesture. –  Jacob Vlijm Jan 1 at 8:10
1  
@JacobVlijm seconded! –  muru Jan 3 at 4:09
1  
@KasiyA, you're so generous :) –  user300458 Jan 6 at 23:14

4 Answers 4

up vote 11 down vote accepted
+200

The python script below does the job. Hidden files are stored separately in a folder , as well as files without extension.

Since it might be used for a wider range of purposes, I added a few options:

  • You can set extensions you'd like to exclude from the "reorganization". If you simply want to move all, set exclude = ()
  • You can choose what to do with empty folders (remove_emptyfolders = True or False)
  • In case you would like to copy the files instead of moving them, replace the line:
shutil.move(subject, new_dir+"/"+name)

by:

shutil.copy(subject, new_dir+"/"+name) 

The script:

#!/usr/bin/env python3

import os
import subprocess
import shutil

# --------------------------------------------------------
reorg_dir = "/path/to/directory_to_reorganize"
exclude = (".jpg") # for example
remove_emptyfolders = True
# ---------------------------------------------------------

for root, dirs, files in os.walk(reorg_dir):
    for name in files:
        subject = root+"/"+name
        if name.startswith("."):
            extension = ".hidden_files"
        elif not "." in name:
            extension = ".without_extension"
        else:
            extension = name[name.rfind("."):]
        if not extension in exclude:
            new_dir = reorg_dir+"/"+extension[1:]
            if not os.path.exists(new_dir):
                os.mkdir(new_dir)
            shutil.move(subject, new_dir+"/"+name)

def cleanup():
    filelist = []
    for root, dirs, files in os.walk(reorg_dir):
        for name in files:
            filelist.append(root+"/"+name)
    directories = [item[0] for item in os.walk(reorg_dir)]
    for dr in directories:
        matches = [item for item in filelist if dr in item]
        if len(matches) == 0:
            try:
                shutil.rmtree(dr)
            except FileNotFoundError:
                pass

if remove_emptyfolders == True:
    cleanup()

IF there is a risk of unwanted overwriting duplicate files

At the expense of a few extra lines, we can prevent overwriting possible duplicates. With the code below, duplicates will be renamed as:

duplicate_1_filename, duplicate_2_filename 

etc.

The script:

#!/usr/bin/env python3

import os
import subprocess
import shutil

# --------------------------------------------------------
reorg_dir = "/path/to/directory_to_reorganize"
exclude = (".jpg") # for example
remove_emptyfolders = True
# ---------------------------------------------------------

for root, dirs, files in os.walk(reorg_dir):
    for name in files:
        subject = root+"/"+name
        if name.startswith("."):
            extension = ".hidden_files"
        elif not "." in name:
            extension = ".without_extension"
        else:
            extension = name[name.rfind("."):]
        if not extension in exclude:
            new_dir = reorg_dir+"/"+extension[1:]
            if not os.path.exists(new_dir):
                os.mkdir(new_dir)
            n = 1; name_orig = name
            while os.path.exists(new_dir+"/"+name):
                name = "duplicate_"+str(n)+"_"+name_orig
                n = n+1
            newfile = new_dir+"/"+name
            shutil.move(subject, newfile)

def cleanup():
    filelist = []
    for root, dirs, files in os.walk(reorg_dir):
        for name in files:
            filelist.append(root+"/"+name)
    directories = [item[0] for item in os.walk(reorg_dir)]
    for dr in directories:
        matches = [item for item in filelist if dr in item]
        if len(matches) == 0:
            try:
                shutil.rmtree(dr)
            except FileNotFoundError:
                pass

if remove_emptyfolders == True:
    cleanup()
share|improve this answer

You can use find with a somewhat complex exec command:

find . -iname '*?.?*' -type f -exec bash -c 'EXT="${0##*.}"; mkdir -p "$PWD/${EXT}_dir"; cp --target-directory="$PWD/${EXT}_dir" "$0"' {} \;

# '*?.?*' requires at least one character before and after the '.', 
# so that files like .bashrc and blah. are avoided.
# EXT="${0##*.}" - get the extension
# mkdir -p $PWD/${EXT}_dir - make the folder, ignore if it exists

Replace cp with echo for a dry run.


More efficient and tidier would be to save the bash command in a script (say, at /path/to/the/script.sh):

#! /bin/bash

for i
do
    EXT="${i##*.}" 
    mkdir -p "$PWD/${EXT}_dir"
    mv --target-directory="$PWD/${EXT}_dir" "$i" 
done

And then run find:

find . -iname '*?.?*' -type f -exec /path/to/the/script.sh {} +

This approach is pretty flexible. For example, to use the filename instead of the extension (filename.ext), we'd use this for EXT:

NAME="${i##*/}"
EXT="${NAME%.*}"
share|improve this answer
    
+1; the -iname '*.*' should take care of the corner cases I was worried about... nice idea! –  Rmano Aug 27 '14 at 9:53
    
@Rmano not the *.fig.bak or .profile/.bashrc ones, but it should only handle files with extensions, at least. Thanks. –  muru Aug 27 '14 at 9:55
ls | gawk -F. 'NF>1 {f= $NF "-DIR"; system("mkdir -p " f ";mv " $0 " " f)}'

Calculating the list of extensions (after moving):

ls -d *-DIR

Calculating the list of extensions (before moving):

ls -X | grep -Po '(?<=\.)(\w+)$'| uniq -c | sort -n

(in this last exemple, we are calculating the number of files for each extension and sorting it)

share|improve this answer
1  
sorry: a typo "mkdir -f" was corrected to "mkdir -p" (to ignore if dir already exist) –  JJoao Dec 29 '14 at 16:32
    
Isn't uniq supposed to be applied after sort? And please do not parse the output of ls. –  muru Dec 30 '14 at 22:21
    
@muru, (part 1) ls -X guarantees that extensions are sorted. Final sort was just to order the extensions table by number of occurrences -- relevance. (I'm I correct?). –  JJoao Dec 30 '14 at 23:32
    
@muru, (part 2) ls -X | grep -Po '(?<=\.)(\w+)$' was my first idea to get the sorted list of extensions. Is it very bad? What do you suggest? –  JJoao Dec 30 '14 at 23:35
    
I forgot what ls -X does. As to why I recommend against ls, see unix.stackexchange.com/q/128985/70524 and unix.stackexchange.com/q/112125/70524. To achieve what you do, I'd go about a longer way: find . -type f -name '*?.?*' -print0 | sed -z 's/.*\.//' | sort -zu (with an optional | uniq -cz, if counts are needed). And find ... -print0 | gawk -v RS='\0' (even though that is not very portable) for the first. –  muru Dec 31 '14 at 2:06

Try this shell script.

#!/bin/sh
src=`dirname "$1"`/`basename "$1"`;
for file in "$src"/*?.?*; do
  if test -f "$file"; then
    dest="$src${file##*.}"_files;
    mkdir -p "$dest";
    mv "$file" "$dest";
  fi;
done;

# pass the directory to re-organize as first argument
# moves only regular files which have extension
# ignores other type of files including
# files having no extension, hidden files, directories, and links.
share|improve this answer
1  
I am sorry, that is an error. I should have to substituted every occurrence of filepath with file. I'll correct that directly. –  Prashant Karmakar Dec 29 '14 at 16:35
    
Please do not parse the output of ls. Instead, do for file in "$src"/*?.?*; do .. –  muru Dec 30 '14 at 22:22
    
@muru will that work correctly if the name of some file has spaces? –  Prashant Karmakar Dec 31 '14 at 6:10
    
@PrashantKarmakar yes, whereas read may have unexpected behaviour. You should also be quoting the variables in the mkdir and mv commands. –  muru Dec 31 '14 at 6:12
    
Test it out, if you will: for i in *; do printf "%s\n" "$i"; done; for i in $(ls -d); do printf "%s\n" "$i"; done –  muru Dec 31 '14 at 6:14

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.