1

I made a horrible error in a very long-running computation.

One part of the computation stores result files in a directory structure like so:

path/to/first/[A,B,C,D]/[1,2,3,4,5]/outfiles

And another does it like this:

path/to/second/[1,2,3,4,5]/[A,B,C,D]/outfiles

That is, when part 1 of the script stores files, it creates a directory for, say, A, then stores iterations 1,2,3,4, and 5 as subdirectories. When part 2 does its computations, it creates a directory for iteration 1, and then stores the first iteration of computations A,B,C, and D in subdirectories.

I'd like to "invert" the second directory structure to be like the first, since rerunning the original script with the directory fix will take too long, and my post-processing code that should otherwise work for part 2 already handles part 1's structure.

That is, path/to/second/1/A should become path/to/second/A/1, containing the files that were previously 1/A. Is there a simple way to achieve this?

To complicate matters with making temporary directories, while I used [A,B,C,D] and [1,2,3,4,5] in my example for clarity, the directories for both hierarchies are just numbers and definitely have name collisions (that is, things like 10/10 exist).

  • 2
    Practical question: do you have enough disk space to create a temporary directory? Furthermore: in the last directory, are there more then 26 dirs? – Jacob Vlijm May 19 '16 at 4:46
2

The question is actually to rename all dirs named with an integer, by a capital with the corresponding index(+1) from the alphabet, and vice versa:

1 -> A

while e.g.

C -> 3

Assuming you have no more than 26 folders on a level (number of letters in the alphabet) , the question is not that complicated, but we need to take into account a few things:

  1. We need to rename the directories from bottom to top, since moving directories inside meanwhile renamed directories will fail.
  2. Since you mention possible name clashes, we need to do the renaming in two runs:
    1. rename the dirs, adding a non-sense, unique string to prevent name clashes
    2. remove the string after all renaming took place

That is exactly what the script below does:

The script

#!/usr/bin/env python3
import string
import shutil
import os
import sys

reorg = sys.argv[1]

chars = list(string.ascii_uppercase)
nums = [str(i+1) for i, c in enumerate(chars)]

tempstring = "_temp1234"

# first step: rename from bottom to top
for root, dirs, files in os.walk(reorg, topdown = False):
    for dr in dirs:
        tempname = None
        if dr in chars:
            tempname = str(chars.index(dr)+1)+tempstring
        elif dr in nums:
            tempname = chars[nums.index(dr)]+tempstring
        if tempname:
             print(dr, tempname)
             shutil.move(root+"/"+dr, root+"/"+tempname)

# second step: remove the temporary string
for root, dirs, files in os.walk(reorg, topdown = False):
    for dr in dirs:
        if tempstring in dr:
            shutil.move(root+"/"+dr, root+"/"+dr.replace(tempstring, ""))

To use

  1. Copy the script into an empty file, save it as reorg.py
  2. Run it with the targeted directory as argument:

    python3 /path/to/reorg.py /path/to/second
    

As always, please first try on a sample.

0

This should do what you want in bash:

#!/bin/bash
mkdir -p /path-to-second-new/{A,B,C,D}
for i in [1,2,3,4,5]
do 
    for j in [A,B,C,D]
    do
        cp -v /path-to-second/"$i"/"$j" /path-to-second-new/"$j"/"$i"
    done
done

Your normal structure should then be in /path-to-second-new/ and your initial structure untouched in /path-to-second/.

0

I was facing the same problem recently in a situation where it was not practical to list all folders manually. Wanting to use a bash script, I came up with the following:

for a in */; do
    for b in $a/*/; do
        dir=${b##*//}
        mkdir -p $dir$a
        mv $a$dir* $dir$a
    done
    rm -rf $a
done

This dynamically grabs the folder names, creates the new hierarchy, moves all files and removes the old folders.

Extracting the second-level folder uses a bit of a hack. All directory names $a will end in / (e.g. folder1/). In the second for-loop I append another slash after $a (the first / in $a/*/), so each entry $b will look like folder1//folder2/. This double slash is ignored by Unix, so I can filter out the directory name by cutting away everything up to and including this double slash with dir=${b##*//}.

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.