Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I want to wget (or other download batch command) the latest file that is added to a large repository. The latest nightly build thru http. I could mirror all files, but the repository are huge so I want to be able to remove old files and only trigger when there is a new file.

share|improve this question
can u provide more specifics about your problem? like the repository from where you want to download the file? is it using anykind of versioning? is it a FTP site?? – Rajesh Pantula Jan 4 '12 at 11:39
wget -A.deb -m – Anders Wallenquist Jan 4 '12 at 12:58

A first simple approach would be to use the time stamping capacity of wget, with this, you could only download the new files from the repository. But it leads to make a mirror of the repository (but refreshed smartly with only the new files :) ).

share|improve this answer
Even though it mirrors the repository in local, but according to the wget documentation, But once you mirrow the repository in local after that it will only download the modified files. – Rajesh Pantula Jan 4 '12 at 14:29
I still have to keep the whole repository at my local site, less then a TB but large enought. – Anders Wallenquist Jan 8 '12 at 21:50

I would advise you use rsync for this task. You have several options on how to do this, there are scripts available here

And there is a full tutorial here:

And debmirror:

share|improve this answer
Sorry, I only have http-access to the site. – Anders Wallenquist Jan 8 '12 at 21:48

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.