1

I have a folder that has a lot of files (various logs generated by a website).

Managed to count them with:

$ find . -maxdepth 1|wc -l
803313

Now I need to check the logs in the past few days. Getting the last 100 / 1000 / ... files is just as good. The problem is the above command took about 5 seconds. So, at almost 1 million files, my regular ls -lrt|tail -n 100 isn't an option.

So, any ideas on how to efficiently get the last modified files ?

PS: Using Ubuntu 12.04 LTS

1
  • Long term, look at something like AIDE.
    – muru
    Sep 25, 2014 at 13:44

2 Answers 2

4

Find itself can filter files within a time range. You said you wanted files newer than a few days ago:

find . -maxdepth 1 -ctime -3

This will find all files changed in the past 3 days.

You can use -ctime (file status changes), -mtime (file's data changes) and -atime (file accessed changes).

If you need more granularity, use -cmin, -mmin and -amin (which use minutes instead of days).

The time range can be a plain number or use +/- signs, as follows:

  Numeric arguments can be specified as

   +n     for greater than n,

   -n     for less than n,

   n      for exactly n.

so -mtime -3 will give you files modified 3 days (and less) ago. -mtime 3 will give you files modified 3 days ago (note that fractional parts are discarded, so this will give you files modified between 3 and 4 days ago). -mtime +3 will give all files modified 3 (and more) days ago.

4

You can use printf to print the date, %T@ gives you the modification time like a unix timestamp:

find . -type f -printf "%T@ %p\n" | sort -rn | head -n 10

On a folder with 250.000 files it takes 10 seconds.

Sidenote: To find the most recent file very quickly, you can use

find . -type f -printf "%T@\0%p\0" | gawk ' { if ($0>max) { max=$0; getline mostrecent } else getline } END{print mostrecent}' RS='\0'

This takes less than 2 seconds on 250.000 files.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .