Move files that are different to another folder

Question

OK, so I have a very large iTunes library and recently decided to sort through it and organize it in a much neater way than just throwing all my music in a folder and hoping for the best. So after many hours of moving files in and out or directories and waiting for iTunes to do whatever it is iTunes does, somehow I've ended up with a few missing songs. If it was 1 or 2, so what, i can find them by myself. But I'm missing a little over 30000 so I'm not too keen on the idea of looking for them manually.

So my question is, "Is there a way for me to, in a linux environment, find all of the files that are in 1 directory, that are not in another, and then move them?"

My folder structure looks a little like this:

• Parent Directory (External Hard Drive)

  • Missing Files (there's 4 of these folders
    • Hundreds of Folders with Artist names (some have .mp3 files here, others have another subfolder, and others have another subfolder again)
  • Parent Itunes Directory (Where iTunes keeps Music, Playlist files, Podcasts, etc)

(I'm really bad at lists in these questions^) So what I need, is to move anything that is in the "Missing Files" directory that isn't in the "iTunes Folder"

Thank you for any help in advance guys, I really appreciate it.

Note: I only want to move the mp3 files from the folders, not the folders themselves.

Answer 1

Assuming you have a directory structure like:

music
  |- source
      |- foobar
      |- spamegg
      |- ...
  |- dest

and you need to move all .mp3 files under any level of subdirectories of music/source/ to music/dest/ only if the file not already present in music/dest/.

If that so, running all commands below from music/ directory:

• Create an array containing all .mp3 files under music/source, recursive operation:

  shopt -s globstar  ## Enables recursive search
  source_files=( source/**/*.mp3 )
  

• Do the same with the music/dest/ directory and strip off the directory portion from file names:

  dest_files=( dest/**/*.mp3 )
  dest_files=( ${dest_files[@]} )
  

• Iterate over the source files and check if the file already exist in the destination, if not mv the file to music/dest/:

  for i in "${source_files[@]}"; do [[ ! ${dest_files[@]} =~ ${i##*/} ]] && \
           echo mv -i "$i" dest/; done 
  

  echo is for doi9ng the dry-run, if satisfied, just remove echo:

  for i in "${source_files[@]}"; do [[ ! ${dest_files[@]} =~ ${i##*/} ]] && \
           mv -i "$i" dest/; done
  

Scripted form:

#!/usr/bin/env bash
shopt -s globstar
source_files=( source/**/*.mp3 )
dest_files=( dest/**/*.mp3 )
dest_files=( ${dest_files[@]} )
for i in "${source_files[@]}"; do [[ ! ${dest_files[@]} =~ ${i##*/} ]] && \
     mv -i "$i" dest/; done

You can obviously use absolute paths, change the relevant paths to meet you need.

Caveats:

• All source and destination .mp3 files are put into arrays, needs bit of extra memory, should not be a problem on modern systems

• The membership test takes time, again should not be noticeable on modern systems

Example:

music$ tree 
.
├── dest
│   ├── 1.mp3
│   ├── 2.mp3
│   ├── 3.mp3
│   ├── 7.mp3
│   ├── 8.mp3
│   └── 9.mp3
└── source
    ├── bar
    │   ├── 10.mp3
    │   ├── 6.mp3
    │   ├── 7.mp3
    │   ├── 8.mp3
    │   └── 9.mp3
    └── foo
        ├── 1.mp3
        ├── 2.mp3
        ├── 3.mp3
        ├── 4.mp3
        └── 5.mp3

music$ shopt -s globstar

music$ source_files=( source/**/*.mp3 )

music$ dest_files=( dest/**/*.mp3 )

music$ dest_files=( ${dest_files[@]} )

music$ for i in "${source_files[@]}"; do [[ ! ${dest_files[@]} =~ ${i##*/} ]] && echo mv "$i" dest/; done
mv source/bar/10.mp3 dest/
mv source/bar/6.mp3 dest/
mv source/foo/4.mp3 dest/
mv source/foo/5.mp3 dest/

music$ for i in "${source_files[@]}"; do [[ ! ${dest_files[@]} =~ ${i##*/} ]] && mv "$i" dest/; done

music$ tree 
.
├── dest
│   ├── 10.mp3
│   ├── 1.mp3
│   ├── 2.mp3
│   ├── 3.mp3
│   ├── 4.mp3
│   ├── 5.mp3
│   ├── 6.mp3
│   ├── 7.mp3
│   ├── 8.mp3
│   └── 9.mp3
└── source
    ├── bar
    │   ├── 7.mp3
    │   ├── 8.mp3
    │   └── 9.mp3
    └── foo
        ├── 1.mp3
        ├── 2.mp3
        └── 3.mp3