Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I have customers who upload large amounts of data to my FTP server (an Ubuntu 9.10 machine). Once the files are uploaded, I am faced with transferring them to a local Droboshare network drive so our technical support staff can retrieve them and analyze the issue.

After I mount the network share at /media/<sharename> I run this command manually to get these files:

sudo mv /home/ftp/<data_dir>/<file_name> /media/<network_share>/<ftp_user_data>

but this takes forever and if there are tens or hundreds of files, I can't realistically do this over and over again. I thought it would be nice to have a script that I could run periodaclly to transfer a group of these files to the network share. I can sort them as the technical support staff uses them.

I am still new to Linux and writing shell scripts. Anyway to do this easily? Just to add to the description here, the files could be ZIP files, FASTA files, TAR.GZ files, and/or TXT files. Also, if the ZIP files are large, certain zipping programs convert these to ZIP.001, ZIP.002, ZIP.003, etc... So the file type in this FTP directory could be quite varied.

I was thinking the way I can identify these files is by simply transferring ALL files in the directory (although this may take a while) or to somewhat designate a subset of these, according to time completed or something like this.

I'm open to ideas. Thanks in advance.

share|improve this question
    
why are you using sudo? –  aperson Aug 26 '10 at 3:04
    
What ftp-server are you using? there are ~6-7 different ftp-server programs in the repository. –  Source Lab Aug 26 '10 at 7:23

3 Answers 3

up vote 5 down vote accepted

Why not just make the FTP folder the actual mount point for the network drive? I run into a similar issue where I'm constantly backing up VirtualMachines on a Linux machine to a Windows network share (since the majority of the company infrastructure is Windows). This is my structure:

/media/windows-share is my mount point

For continuity I've created a symlink in my backup application:

/opt/backup/mnt so if the mount point changes in the future my program doesn't need an update.

Lastly I employ a cool little tool called autofs (sudo apt-get install autofs) That guide is decent (and up to date) - though I've employed this with a slightly different approach on my blog.

In your case I would create a symlink where my FTP dropbox is (/home/ftp/<data_dir>/) to the appropriate network folder in /media/<network_share>/<ftp_user_data> that way there's no wasted time in transferring across disks and the files are available immediately.

share|improve this answer
1  
+1 for removing a step rather than fixing a step :) Simplifying, always good, is. –  Ben Williams Aug 26 '10 at 1:57
    
This is a great answer... Almost done making it work. One thing. In your blog, you used -fstype=cifs and this worked for me too. But when I manually mount these shares, I use smbfs yet when I tried to add this method to the auto.nfs file it didn't work. Any idea why? –  nicorellius Aug 26 '10 at 16:44
    
@nicorellius smbfs and cifs are basically the same thing. smbfs is being depreciated to cifs help.ubuntu.com/community/SettingUpSamba –  Marco Ceppi Aug 26 '10 at 16:56
    
I thought I had it working... But now I can't seem to create that symlink. Getting error: "hard link not allowed for directory." Could you provide some more details exactly how this is setup? –  nicorellius Aug 26 '10 at 17:02
    
@Marco Ceppi: I also was able to mount the network share to the local linux directory /home/ftp/<data_dir> but when I tried to transfer a file to it using FTP, my FTP client threw an error and the operation failed. I may be confused on how to set these permissions. Do I set them on /home/ftp/<data_dir> or on //server/<network_share>/ftp_user_data? –  nicorellius Aug 26 '10 at 17:17

For duplicating files from here to there (or there to here), rsync is a swiss army chainsaw which will likely do what you want.

share|improve this answer

You could try to process files as soon as they're uploaded to the FTP, using a technology sucha s inotify. inoticoming is a tool you can use for that, see http://manpages.ubuntu.com/manpages/lucid/man1/inoticoming.1.html.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.