4

I have a root folder and in that there are many directories plus files. I need to save a list of the contents in each subdirectory with name list.txt.

Suppose I have

A
|
-----B
|    |---Z
|    |---a.txt
|    |---b.jpg
|
|
|----C
|    |--a.txt
|    |--b.txt

Running the command it should give a list.txt in each subdirectory with contents separated by commas.

I have # commented what the contents should be...

A
|
-----B
|    |---Z
|    |---a.txt
|    |---b.jpg
|    |---list.txt  # Z,a.txt,b.jpg
|
|
|----C
|    |--a.txt
|    |--b.txt
|    |--list.txt  # a.txt,b.txt

The closest I could get to list the files is

find . -maxdepth n -type f -printf '%f\n'

but I don't know how to save the contents separately.

Please suggest something.

0
5

The script below will add a list to all sub- dirs from a directory recursively:

#!/usr/bin/env python3
import os
import sys

for root, dirs, files in os.walk(sys.argv[1]):
    for dr in dirs:
        dr = os.path.join(root, dr)
        open(os.path.join(dr, "list.txt"), "wt").write(
            ",".join(f for f in os.listdir(dr) if f != "list.txt")
            )

To use

  1. Copy the script into an empty file, save it as dirlists.py
  2. Run it with the main directory as argument:

    python3 /path/to/dirlists.py /path/to/maindirectory
    

Note

As mentioned, the script adds a list.txt to all subdirs. If you also need or want to have a list in the main (root) -dir of your directory, please mention.

Explanation

  1. List (walk through) all directories recursively inside a directory:

    for root, dirs, files in os.walk(sys.argv[1]):
        for dr in dirs:
            dr = os.path.join(root, dr)
    
  2. Create a list of content for each of them:

    os.listdir(dr)
    
  3. Open (create if necessary) the text file and write the listed content, excluding possible earlier files named list.txt:

    open(os.path.join(dr, "list.txt"), "wt").write(
        ",".join(f for f in os.listdir(dr) if f != "list.txt")
        )
    

EDIT

As requested in a comment:

In case you need the line in list.txt to end with a comma, simply replace:

",".join(f for f in os.listdir(dr) if f != "list.txt")

by:

",".join(f for f in os.listdir(dr) if f != "list.txt")+","

mind the indentation, place the replacement at the exact same position

1
  • Comments are not for extended discussion; this conversation has been moved to chat.
    – terdon
    Apr 3 '17 at 8:53
4

To make it recursive, first turn on globstar

shopt -s globstar

Then, in the parent directory (A in your structure), you can run:

for d in **; do [[ -d "$d" ]] && (find "$d" -mindepth 1 -maxdepth 1 \( -not -name "list.txt" \) -printf '%f,' | sed 's/,$/\n/') |tee "$d"/list.txt ; done

or slightly more readably

for d in **; do 
  [[ -d "$d" ]] && 
  (find "$d" -mindepth 1 -maxdepth 1 \( -not -name "list.txt" \) -printf '%f,' | sed 's/,$/\n/') | tee "$d"/list.txt
done

which, if directory a contains

├── a
│   ├── 1.txt
│   ├── 2.txt
│   ├── 3.txt
│   ├── a badly named file &
│   └── Z

will create a list in directory a that looks like this:

2.txt,1.txt,3.txt,Z,a badly named file &

find does not produce ordered output, so if that's an issue I will have to think of a better way. The \( -not -name "list.txt" \) in the find expression is to prevent the list from including itself, and the sed expression is purely to remove the trailing comma. Shame about all those extra bytes.

You may want to turn off globstar when done

shopt -u globstar
4
  • Thanks for the code.. It worked great for level1 but suppose if I have a file in 'Z' folder the 'list.txt' should be made inside it too with file listing in it.. any workaround for this ? Apr 2 '17 at 7:09
  • @DerekJames sure - easily fixed - see my edit :)
    – Zanna
    Apr 2 '17 at 7:34
  • checked the code... Its adding the name of files which are in the inner directory too.. Apr 2 '17 at 8:41
  • @DerekJames oh yeah, fixed that now... but seems like a stupid amount of depth specifying... must be a better way...
    – Zanna
    Apr 2 '17 at 8:46
3

One-liner version

Use find to get directories first, then shell do the work for you:

$ tree                                                                                                                                                
.
├── a_directory
│   ├── a_file
│   ├── a_subdir
│   └── mv-files.py
└── another_dir
    ├── {file}1
    └── {file}2

3 directories, 4 files

$  find -type d -exec bash -c 'cd $1; find  -maxdepth 1  -not -name "." -not -name "list.txt" -printf "%f," | awk "{print substr(\$0,0,length(\$0)-1)}"  > list.txt' bash "{}" \;

$ cat a_directory/list.txt                                                                                                                            
mv-files.py,a_file,list.txt,a_subdir

The way this works:

  • we use find command with -type d to filter out all the directories
  • -exec statement with \; terminator allows us to run specified command for each argument that find obtains
  • within -exec we run bash with -c flag , to which we pass $0 argument bash and $1 argument being the directory that outter find locaed
  • bash will enter the given directory and use find with - maxdepth 1 argument to limit that command only to that subdirectory. -not -name "." will exclude the . directory link , which is reference to self.
  • Next, we pass the text to awk, which serves only to remove the last , given by find so that we have a valid CSV list. Note the use of double quotes and \$. That's meant to simplify quoting and prevents bash from interpreting $0 as its own positional arguments, but rather as awk command.
  • All the list of items that inner find obtains will be sent to list.txt via > redirection.

Additional improvement to this could be to use -not -name "list.txt" inside the inner find command to exclude list file itself (because due to > always creating the file to write first, list.txt will also appear on the list).

Personally, if I were doing this for myself, I'd write the list of files with \0 separator to avoid dealing with difficult filenames, but that also requires remembering that list.txt is in \0 format and writing a parser function for it.

Full script version

For the sake of readability, here's a full script version instead of one-liner.

Script:

#!/bin/bash
# note : this assumes you run the script from top-most directory
find  -type d  | while IFS= read -r directory;
do
    cd "$directory"
    find  -maxdepth 1  -not -name "." -not -name "list.txt" -printf "%f," |
    awk "{print substr(\$0,0,length(\$0)-1)}"  > list.txt
    cd - > /dev/null
done

Note that this script is run from the top-most directory. It also includes self into the list if it's stored in the same directory. If you place it to ~/bin for example (or any other directory that belongs to $PATH variable), and run it, the script name won't appear on the list.

Test :

$ tree                                                                                                                                                
.
├── a_directory
│   ├── a_file
│   ├── a_subdir
│   └── mv-files.py
├── another_dir
│   ├── {file}1
│   └── {file}2
└── make_lists.sh

3 directories, 5 files

$ ./make_lists.sh                                                                                                                                     

$ tree
.
├── a_directory
│   ├── a_file
│   ├── a_subdir
│   │   └── list.txt
│   ├── list.txt
│   └── mv-files.py
├── another_dir
│   ├── {file}1
│   ├── {file}2
│   └── list.txt
├── list.txt
└── make_lists.sh

3 directories, 9 files

$ cat a_directory/list.txt                                                                                                                            
mv-files.py,a_file,a_subdir
8
  • Thanks for such a great answer. It's working for me but need just a small change in output. If the 'list.txt' can output the files with comma sign just like I mentioned in the example, it would be perfect Apr 2 '17 at 7:30
  • @DerekJames I just made the edit to include that, although it made the command slightly longer to make it a valid CSV file. Quite frankly, I'd write it slightly differently as full shell script. Let me know if you want that included as well. That one probably will be more readable Apr 2 '17 at 7:36
  • @DerekJames output example is still the old one though. Let me edit that in a second Apr 2 '17 at 7:38
  • @DerekJames any other questions ? how it's working for you ? Apr 2 '17 at 7:48
  • getting 'list.txt' in the file created.. and as u used -name this time, how to add -not -name"list.txt" as told by you.. Apr 2 '17 at 8:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.