2

I am in a directory A. This has a lot of sub directories B, C, D. Each sub directories will have a list of files. I need to create a .tar file of all the latest file in each subdirectory.

Example:

Directory A :
  Sub directories: B, C, D

Files in directory B : b/1, b/2, b/3
Files in directory C : c/4, c/5, c/6
Files in directory D : d/7, d/8, d/9

I need a .tar file that contains (b/1, c/4, d/7). 1, 4, 7 are the latest files in the directories. I tried to do that using the find and sort -r. But this prints all the files in the directories.

find . -type f -printf '%TY-%Tm-%Td %TT %p\n' | sort -r 
2

A python solution to do the job completely. It creates a tar.gz file from the latest files of all sub directories of a given directory.

The script

#!/usr/bin/env python3

import os
import time
import tarfile

files_dir = "/path/to/directory/with/subdirectories"
targeted_file = "/path/to/latest_files.tar.gz"

latest_files = []
# get the latest files of all sub directories
for root, dirs, files in os.walk(files_dir):
    for dr in dirs:
        dr = root+"/"+dr
        filelist = []
        for item in os.listdir(dr):
            file = dr+"/"+item
            if os.path.isfile(file):
                filelist.append((file, os.stat(file).st_mtime))
        filelist.sort(key=lambda x: x[1])
        if len(filelist) != 0:
            latest_files.append(filelist[-1][0])
# write to latest_files.tar.gz
tar = tarfile.open(targeted_file, "w:gz")
for file in latest_files:
    tar.add(file, arcname = file.split("/")[-1])
tar.close()

Copy the script into an empty file, set in the head section of the script the directory, containing the sub directories (files_dir =), and the path to the tar.gz file (targeted_file =), and save it as get_latest.py.

Run it by the command:

python3 /path/to/get_latest.py

What it does

The script first lists all (sub-) directories. Within the sub directories, it lists the files, sorts them by modification date and adds the latest ones to the "master" list, to be included in the compressed file.


To set the number of versions to include

To give the answer a more universal character, below a version of the script in which you can set the numbers of (latest) versions to include in the compressed file. If the number of set versions (versions =) exceeds the actual number of files in a folder, all files are included.

The script

#!/usr/bin/env python3

import os
import time
import tarfile

files_dir = "/path/to/directory/with/subdirectories"
targeted_file = "/path/to/latest_files.tar.gz"
versions = 1

latest_files = []
# get the latest files of subdirectories
for root, dirs, files in os.walk(files_dir):
    for dr in dirs:
        dr = root+"/"+dr
        filelist = []
        for item in os.listdir(dr):
            file = dr+"/"+item
            if os.path.isfile(file):
                filelist.append((file, os.stat(file).st_mtime))
        filelist.sort(key=lambda x: x[1])
        if len(filelist) != 0:
            for item in filelist[-versions:]:
                latest_files.append(item[0])
# write to latest_files.tar.gz
tar = tarfile.open(targeted_file, "w:gz")
for file in latest_files:
    tar.add(file, arcname = file.split("/")[-1])
tar.close()
  • Pefect. But I need one small modification. when I untar the files, the entire directory structure is maintained. I need only the files to be tarred. The file names will be unique. – Raj Nov 17 '14 at 22:42
  • @Raj edited my answer, just adding arcname = file.split("/")[-1] did the job; now only the file (name) is in the tar.gz file, not its containing directory structure. – Jacob Vlijm Nov 18 '14 at 6:36
1

"Latest" is hard to define. I think you can use find in two ways here:

  1. List all files modified later than another file:

    find . -type f -newer b/1
    
  2. List all files modified less than x minutes ago (say 10):

    find . -type f -mmin -10
    

Take your pick.

1

You can use :

ls -tl | sed -n 2p

it should returns the name and details of the newest file in a directory .

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.