I have a folder that connects to a hardware weather logger. The data is pretty huge, and I only wish to keep the last 5 hours worth of data. This folder; however, has also some config files such as myconfig.ini.
How can I set up an automatic service that automatically deletes all files in a folder save for a few?
Would adding a find command to cron work for you?
find /path/ -type f -mtime +5 -name '*.log'
You can layer on extra things to find just the files you want. -mtype is the date-limiting portion which finds files last modified over 5 days ago. You can change this to just look at creation or even access times (if you're not using noatime on your fs)
When you're getting the right files, just stick -exec rm {} \; on the end. You could use -delete but that will crumble if you have too many files being deleted at once. Only do this once you're sure!
When you want to schedule it, su into the right user account (the person who owns the files) and then run crontab -e. You can then write out the command and when you exit, it'll run at the schedule you set. You can read more about cron here:
-name '*.log'. Without the quotes, if there is any file matching *.log in whatever directory is current when you run the command, the shell will substitute them for the pattern, which will lead to either errors or the wrong set of files being acted upon.
Nov 1, 2010 at 19:18
You can use logrotate. It's installed by default; check the manual page for instructions.
This should remove anything modified within the last day. I'd suggest placing it in a daily cronjob. Be sure to TEST before implementing... and bear in mind that since it works on files by modified date it isn't file specific (i.e. if you modify your myconfig.ini, the next time it runs it might kill your myconfig.ini.
find /directory_path -type f -mtime -1 -exec rm{} \;
This obviously has some disadvantages, but it might at least be a good start.
Red Hat/Fedora/Cent has a slick little script called tmpwatch that you can easily pull-over to Ubuntu.
Taking a slightly different tack: why not set up a 5hr frequency job to rename the current collection directory and create a new current collection directory? You could manually or automatically process/archive/delete the individual 5hr directories.
