Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I want to wget (or other download batch command) the latest file that is added to a large repository. The latest nightly build thru http. I could mirror all files, but the repository are huge so I want to be able to remove old files and only trigger when there is a new file.

share|improve this question
1  
can u provide more specifics about your problem? like the repository from where you want to download the file? is it using anykind of versioning? is it a FTP site?? –  rao_555 Jan 4 '12 at 11:39
    
wget -A.deb -m nightly.openerp.com/6.1/deb –  Anders Wallenquist Jan 4 '12 at 12:58

2 Answers 2

A first simple approach would be to use the time stamping capacity of wget, with this, you could only download the new files from the repository. But it leads to make a mirror of the repository (but refreshed smartly with only the new files :) ).

share|improve this answer
    
Even though it mirrors the repository in local, but according to the wget documentation, But once you mirrow the repository in local after that it will only download the modified files. –  rao_555 Jan 4 '12 at 14:29
    
I still have to keep the whole repository at my local site, less then a TB but large enought. –  Anders Wallenquist Jan 8 '12 at 21:50

I would advise you use rsync for this task. You have several options on how to do this, there are scripts available here

https://wiki.ubuntu.com/Mirrors/Scripts

And there is a full tutorial here:

https://help.ubuntu.com/community/Rsyncmirror

And debmirror:

https://help.ubuntu.com/community/Rsyncmirror

share|improve this answer
    
Sorry, I only have http-access to the site. –  Anders Wallenquist Jan 8 '12 at 21:48

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.