4

I'm running Ubuntu 14.04 and I want to monitor a folder and drop a mail if new files are not added every 30 mins.

I referred below as well.

How I can monitor new files in a directory and move/rename them into another directory?

I'm not sure how to wait for a period and check for new files in a folder.

  • 1
    It can be done in many ways. One possible way is to use cron to run a script every 30min and then the script itself that will check the directory and send email can be written in pretty any language like bash, python or php... – Michal Przybylowicz Jul 8 at 8:10
  • You could even do this in .net core :)! – JMK Jul 8 at 15:31
  • What if a new file was added 5 minutes ago and deleted? The answers here will still say no new file was added in the last 30 minutes. – WinEunuuchs2Unix Jul 9 at 22:58
  • Is an immediate action useful in your situation, or is blindly checking every 30 minutes sufficient? We can make a signal based smart, adaptive and variable loop, making sure you'll be warned immediately after exceeding 30 minutes. – Jacob Vlijm Jul 10 at 7:35
8

You might be able to do something as simple as:

#!/bin/bash
targetDir="$1"
while true; do
  files=("$targetDir"/*)
  sleep 30m ## wait 30 minutes
  files1=("$targetDir/"*)
  if [[ ${#files[@]} == ${#files1[@]} ]]; then
     echo "No new files added" | mail user@example.com
  fi
done

Save that script as ~/bin/watch.sh or whatever you like, make it executable (chmod a+x ~/bin/watch.sh) and run it giving it the target directory as an argument:

watch.sh /path/to/dir

Now, that is a very naive approach and simply counts the number of files every 30 minutes. So if one file is deleted but another one added, it will think there are no new files. It would probably be a better idea to take the timestamp of the newest file and check if it was last modified more than 30 minutes ago:

#!/bin/bash
targetDir="$1"
newest=0;

while true; do
  ## Wait for half an hour. This is done at the beginning of the loop
  ## to ensure we wait a half hour before the first check
  sleep 30m
  ## Get the current time in seconds since the epoch
  now=$(date '+%s')
  for file in "$targetDir"/*; do
    ## Get the file's modification time in seconds since the epoch
    lastMod=$(stat -c "%Y" "$file")
    ## Is this file newer than the current newest?
    if [[ $lastMod -gt $newest ]]; then
      newest=$lastMod
    fi
  done
  ## If the newest file is older than half an hour
  if [[ $((now - newest)) -gt 1800 ]]; then
    echo "No new files added since $(date -d '- 30 min')" |
      mail user@example.com
  fi
done

If you don't want to run a script for ever like that, you could instead write one that checks the timestamps and call the script every 30 minutes:

#!/bin/bash
targetDir="$1"
newest=0;

for file in "$targetDir"/*; do
  ## Get the file's modification time in seconds since the epoch
  lastMod=$(stat -c "%Y" "$file")
  ## Is this file newer than the current newest?
  if [[ $lastMod -gt $newest ]]; then
    newest=$lastMod
  fi
done
## Get the current time in seconds since the epoch
now=$(date '+%s')
## If the newest file is older than half an hour
if [[ $((now - newest)) -gt 1800 ]]; then
    echo "No new files added since $(date -d '- 30 min')" |
     mail user@example.com
fi

Now, save that as ~/bin/watch.sh and make it executable (chmod a+x ~/bin/watch.sh). Then, run crontab -e and add this line:

*/30 * * * * /home/techie/bin/watch.sh

Save the file (saving adds the crontab) and you're done. The script will be run every 30 minutes.

2

Before checking for new files, I would use "stat" on the directory itself and see if the timestamp changes.

$ stat $PWD
  File: ‘/home/paulm/SCRATCH’
  Size: 6144        Blocks: 8          IO Block: 32768  directory
Device: 26h/38d Inode: 10937103831396301849  Links: 6
Access: (0775/drwxrwxr-x)  Uid: (  520/   paulm)   Gid: ( 1020/   paulm)
Access: 2019-07-09 13:17:15.528000000 +0000
Modify: 2019-07-09 13:17:15.528000000 +0000
Change: 2019-07-09 13:17:15.528000000 +0000
 Birth: -

$ touch bar

$ stat $PWD
  File: ‘/home/paulm/SCRATCH’
  Size: 6144        Blocks: 8          IO Block: 32768  directory
Device: 26h/38d Inode: 10937103831396301849  Links: 6
Access: (0775/drwxrwxr-x)  Uid: (  520/   paulm)   Gid: ( 1020/   paulm)
Access: 2019-07-09 13:23:11.346000000 +0000
Modify: 2019-07-09 13:23:11.346000000 +0000
Change: 2019-07-09 13:23:11.346000000 +0000
 Birth: -

if the date on the directory changes, then it's worth looking for new files, this should be a lot more efficient.

  • True. On the other hand it is still a miracle to me when and why the mtime of a directory changes. Do a touch foo an hour later and the directory's mtime is still the same. But adding/removing files is reflected there. – PerlDuck Jul 9 at 13:34
  • that could be because of mount option "nodiratime" and will defeat this strategy. – Paul M 2 hours ago
2

The answers so far assume that files only go into a directory and never go out. But what if you have a script that empties the directory periodically? Then the script could miss the fact a file went into the directory 5 minutes ago but was processed and taken out.

Another option is using the inotifywait command written in highly efficient C language for the express purposes of monitoring file and directory creation and modification. You can install it with the command:

sudo apt install inotify-tools

The psuedo-code for an appropriate script would be:

  • Wait 30 minutes or until directory changes
  • Did directory change?
  • Yes? We got here because directory changed. Loop back and wait 30 minutes again.
  • No? We got here because 30 minutes expired. Send email and loop back to wait 30 minutes again.

You would call the script from a place like /etc/rc.local so it runs when machine is booted.

The advantage of this methodology is you aren't checking the directory every 30 minutes but rather you are checking it 30 minutes after the last changes to the directory.

0

There is a nifty utility for precisely this task:

https://bitbucket.org/eradman/entr/src/default/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.