1

I am writing a script where I need to compare 2 directories (recursively) and write out only files if they have different size or time of modify(YY-MM-DD HH:MM) or if file exists only in one directory.

Output be in format:

<dir1>:<local-path> <size> <last-modify> <dir2>:<local-path> <size> <last-modify>

If file exists only in one directory:

<dir1>:<local-path> <size> <last-modify>

or

<dir2>:<local-path> <size> <last-modify>

So far I managed to get my data in specified format using:

find dir1 -type f -exec stat -c '%n %s %y' {} \; | sed 's,^[^/]*/,,' | sed 's/\:[^:]*$//' | sort # > dir1.txt
find dir2 -type f -exec stat -c '%n %s %y' {} \; | sed 's,^[^/]*/,,' | sed 's/\:[^:]*$//' | sort # > dir2.txt

Which gives me 2 ordered lists of files in given directories and subdirectories and their size and last modified timestamp.

Now I need to somehow compare them and get them to specified format above. I tried using diff -y but it compares line by line but I need same name to same name. I also tried comm but dont know how to transform that output format.

Any ideas?

0
1

I think I'd try to put something together based around using rsync in dry-run mode (--dry-run or -n).

To illustrate, given:

$ tree -Ds Adir/ Bdir/
Adir/
├── [       4096 Nov 19  9:36]  sub1
│   ├── [         35 Nov 19  9:35]  common
│   └── [         23 Nov 19  9:36]  onlyA
├── [       4096 Nov 19  9:41]  sub2
│   ├── [         35 Nov 19  9:35]  common
│   ├── [         44 Nov 19  9:44]  newerA
│   ├── [         44 Nov 19  9:37]  olderA
│   └── [          6 Nov 19 10:36]  size
└── [       4096 Nov 19  9:35]  sub3
    └── [         35 Nov 19  9:35]  common
Bdir/
├── [       4096 Nov 19  9:46]  sub1
│   └── [         35 Nov 19  9:35]  common
├── [       4096 Nov 19 10:36]  sub2
│   ├── [         35 Nov 19  9:35]  common
│   ├── [         44 Nov 19  9:38]  newerA
│   ├── [         44 Nov 19  9:44]  olderA
│   └── [         24 Nov 19 10:36]  size
└── [       4096 Nov 19  9:40]  sub3
    ├── [         35 Nov 19  9:35]  common
    └── [         23 Nov 19  9:40]  onlyB

6 directories, 14 files

then we can list files that have different sizes or modification times as follows:

$ rsync -aOn --delete --itemize-changes Adir/ Bdir/
*deleting   sub3/onlyB
>f+++++++++ sub1/onlyA
>f..t...... sub2/newerA
>f..t...... sub2/olderA
>f.s....... sub2/size

[The change string doesn't really matter for our purposes, but for instance *deleting indicates that sub3/onlyB is not present in the source directory; s indicates a size difference; t indicates a difference in modification time.]

Unfortunately it doesn't seem to be possible to get the actual timestamps directly from the rsync output, but we can simply read the file list and stat the the corresponding files in each directory:

#!/bin/bash

dirA="$1"
dirB="$2"

rsync -aOn --itemize-changes --delete "$dirA"/ "$dirB"/ | while read -r c f ; do
  printf '%s:%s  ' "$dirA" "$(cd "$dirA" && stat -c '%n %s %y' "$f" 2>/dev/null || printf '(none) - - - -')"
  printf '%s:%s\n' "$dirB" "$(cd "$dirB" && stat -c '%n %s %y' "$f" 2>/dev/null || printf '(none) - - - -')"
done

which we can use as follows

$ ./rstat.sh Adir Bdir | column -t
Adir:(none)       -   -           -                   -      Bdir:sub3/onlyB   23  2016-11-19  09:40:12.253318393  -0500
Adir:sub1/onlyA   23  2016-11-19  09:36:52.220421434  -0500  Bdir:(none)       -   -           -                   -
Adir:sub2/newerA  44  2016-11-19  09:44:45.953236221  -0500  Bdir:sub2/newerA  44  2016-11-19  09:38:33.270838033  -0500
Adir:sub2/olderA  44  2016-11-19  09:37:41.675642039  -0500  Bdir:sub2/olderA  44  2016-11-19  09:44:45.953236221  -0500
Adir:sub2/size    6   2016-11-19  10:36:31.460487036  -0500  Bdir:sub2/size    24  2016-11-19  10:36:31.460487036  -0500
3
  • This doesn't produces the output expected by OP! – αғsнιη Nov 19 '16 at 1:37
  • @KasiyA I believe it does now - at least within the limitations of the available time format from stat. Really I shouldn't have to trawl through the comments to someone else's answer to find the OP's desired output though - that should have been made clear in the question </grumble> – steeldriver Nov 19 '16 at 16:28
  • Sorry should have putted example there. – Saintan Nov 20 '16 at 11:23
0

I think you already ended up, here is below for:

If file exists only in directory1 (considering any differences in name, size or modification time:

grep -Fxvf dir2.txt dir1.txt > inDir1Only

Or If file exists only in directory2:

grep -Fxvf dir1.txt dir2.txt > inDir2Only

So at the end for your question "write out only files if they have different size or time of modify(YY-MM-DD HH:MM)", just concatenate two above results, like following : )

assuming inDir1Only and inDir2Only content are as following:

$ cat inDir1Only
    c.txt 26 2016-11-04 14:23
    b.txt 26 2016-11-04 14:23
$ cat inDir2Only
    b.txt 57 2016-11-04 18:20
    a.txt 14 2016-11-04 18:11

since of you desired output would be as like below after executing below awk command,

$ awk 'NR==FNR{seen[$1]=$0;next} {
    print "inDir1Only:"$0, ($1 in seen) ?"inDir2Only:"seen[$1]:"";seen[$1]=""}
END{
    for(x in seen) if (seen[x]!=NULL) print "inDir2Only:"seen[x]
}' inDir2Only inDir1Only

inDir1Only:c.txt 26 2016-11-04 14:23 
inDir1Only:b.txt 26 2016-11-04 14:23 inDir2Only:b.txt 57 2016-11-04 18:20
inDir2Only:a.txt 14 2016-11-04 18:11
10
  • Thanks i tried grep but without the proper options. I have to now match for example: b.txt 26 2016-11-04 14:23 with b.txt 57 2016-11-04 18:20 To look like: dir1: b.txt 26 2016-11-04 14:23 dir2: b.txt 578 2016-11-04 14:23 – Saintan Nov 18 '16 at 17:26
  • So you want same files [in name] (two or more) come in one line? – αғsнιη Nov 18 '16 at 17:40
  • Yes, if 2 files have same name but different size, time they should be in same line. – Saintan Nov 18 '16 at 17:44
  • Ok, give me time to find my Ubuntu system :P – αғsнιη Nov 18 '16 at 18:10
  • Thanks, I appreciate that. The output should be just: <dirgivenas1stparametrofscript>: <local-path(name)> <size> <time> <dirgivenas2ndparametrofscript>: <local-path(name)> <size> <time> on same line – Saintan Nov 18 '16 at 18:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.