I want to wget (or other download batch command) the latest file that is added to a large repository. The latest nightly build thru http. I could mirror all files, but the repository are huge so I want to be able to remove old files and only trigger when there is a new file.
A first simple approach would be to use the time stamping capacity of wget, with this, you could only download the new files from the repository. But it leads to make a mirror of the repository (but refreshed smartly with only the new files :) ).
I would advise you use rsync for this task. You have several options on how to do this, there are scripts available here
And there is a full tutorial here: