Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

Does anybody know of any resources that can show me how to make my own "Dropbox, Ubuntu One" server at home?

I really like the idea of these services, but I don't want to put my 'stuff' in the clouds. Ideally, it should have a client that runs on Linux and Windows.

I tried to setup iFolder on my Ubuntu 10.04, but without any success so far.

share|improve this question
    
I'm not sure I understand the problem. What's wrong with dropbox? –  Oli Jan 31 '11 at 19:22
    
It has some storage limits and with many collaborators on many projects, those paid 50 GiB are lost fairly quickly. –  Henrik Jan 31 '11 at 19:37
3  
There may be unforseen consequences in allowing companies like Dropbox to store our personal data. Thankfully, there is a project underway to make an OS that can run your own personal cloud on a plug-style ARM-powered computer. –  daithib8 May 19 '11 at 13:56
2  
Ironic that this isn't as simple as a sudo apt-get install ubuntu-one-server. –  Prateek Jan 28 '12 at 11:08
add comment

12 Answers

up vote 39 down vote accepted

Currently there's not a great open source alternative that's going to work out of the box. The best thing to keep an eye on is the sparkleshare project: http://www.sparkleshare.org/

Hopefully that will grow into a great, do it yourself, alternative.

share|improve this answer
1  
Unfortunately it is using git DVCS as backend not suited for ~1TB binary data, since modifications on binary data will bloat server space usage. But beside that it looks promising. –  math Feb 3 '12 at 21:30
add comment

There are actually lots of them.

  • SparkleShare (deps: git/subversion, mono, python) at github GUI-based sync software.

    a. Versioning: through a source control system, hence it's mutex-based on a central server through a version number.

    b. State: under development

    c. Pros: OSS, mono-based so easily moddable, Cons: user-level process, GC-dependent, ineffective sharing protocol by orders of magnitude as git is primarily for small text files, fairly hard to compile (I tried). Using high-level tools.

  • lipsync (deps: Unison, rsync) Command-line service-based software.

    a. Versioning: through the rsync delta algoritm. I assume programmer must choose conflict resolution.

    b. State: I can't find its source code, so I have no idea. The only things in his git repo are binaries.

    c. Pros: nice setup, using middle-level tools.

  • iFolder - Novell's Dropbox. I haven't studied its source yet. I just want to get this edit over with and if people are interested I'll add more.

    a. Versioning:

    b. State: Problematic getting it to even compile on Ubuntu, let alone packages. Here's a detailed install guide.

    c. Pros: Windows X64 client, mature, AD-integration with ACLs, features no other project has started to implement. I think this might be a good starting point. Cons: Novell might not use its public svn repo as the primary repo and only do code-drops. I don't know exactly about this though. Might be too coupled to openSUSE to easily install on Ubuntu. To check out its algorithms.

  • scp/rcp - deprecated in favor of rsync

  • DRDB - block device mirroring tools for distributed RAID-1, i.e. a server-variant of dropbox. I haven't checked out its source code yet, but it's linux only. The actual algorithm would probably be easy to combine with the source code in my musings below this software-listing.

    a. Versioning: internal message format over LAN/WAN

    b. State: seems mature enough

    c. Pros: stable enough for linux, Cons: no other operating systems are supported


Right now I'm investigating improving compile-times on a Virtualized Windows 7, where the compile-times on a Windows 7 on metal is 40 s, but virtualized approx 3m 20s. I'm thinking of writing an ioctl driver that is a write-through cache that looks like a ram-disk for selected folders on NTFS.

Using the above software, I think a week's worth of 2-3-person full-time development would produce a usable Alpha that doesn't lose you files by combining the above softwares.


On my system then, the general idea would be;

  1. Mount a virtual drive \?{GUID}, that is the ram-disk and RW-cache. The software creating this virtual drive takes two input parameters (that are vital):

    a. The target folder; this is the SMB folder, so I will be letting the operating system's network stack handle the actual IO. In my case this is in turn the VMWare virtual folder, that has in itself a target on an ext4 drive, but it could easily be your file server using SAMBA/SMB.

    b. The path of the folder to be mounted, e.g. C:\ramdisk

    This code for creating virtual volumes be taken from TrueCrypt's code, in /Driver/DriverFilter.c (among other files)

  2. The drive uses SMB/the VMWare/network protocol to fetch data when it starts; it fetches with a low task priority, asynchronously from the network and fills its cache. It could use a simple compacting algorithm and have 1 thread that uses message-box type continuation passing to get great performance. On Windows it could use the normal async IO calls, and on linux it could use the epoll/inotify implementation and take code from nginx.

  3. My service that is the ram-disk mounts the unnamed ramdisk drive as an NTFS folder. All programs can continue writing to C:\ramdisk, or whatever I call it.

  4. Async copy from network still going on. With a read-rate of approx 100 MiB/s and 2 GiB ramdisk, it would be 20.5 s to read all data.

Each call to read would perform an in-CPU calculation of the index into a fixed n:ulong GiB max sized array. It would require conflict resolving though or read-write locks. If we'd implement a conflict-resolve algoritm like those available through Microsoft Sync, we could pass each chunk that conflicts as a message to another conflict resolve-process. Dropbox solves it by creating a new file and naming it "PrevFileName Username's Conflicted Copy (yyyy-MM-dd).ext". Perhaps this could be altered through a small widget, if one is compiling against that single source -- the widget would detect outstanding changes as messages/events and choose the conflict resolution protocol. As such, when programming against a folder in exclusive-mode, the Windows VM could set the widget to 'exclusive'.

This would have these PROs

  • It would be non-blocking / async
  • It would make the assumption but not require that one computer will be writing mostly to the files.
  • It would work for arbitrarily large files
  • It would work on *nix and Windows by tying together the mentioned projects.
  • It would work when high read-performance is needed (i.e. the files are physically located on disk)
  • When the conflicting events are reached, one could provide a user interface app that allows the user to write/download plugins that act sanely for different sorts of events -- i.e. different sorts of files. E.g. a text file could be brought up with Kompare/WinDiff, while a binary would be duplicated and saved as another file.
share|improve this answer
    
lipsync's bin folder contains shell scripts, not binaries. –  haggai_e Oct 2 '11 at 7:19
add comment

OwnCloud! sounds like something you're looking for.

share|improve this answer
add comment

i heard about Syncany on the Ubuntu UK Podcast, currently beta but looks like it meets the requirements

share|improve this answer
    
Nice idea to use free webspace no matter which protocol was originally intended :) –  math Feb 3 '12 at 21:37
add comment

See if this helps:

http://fak3r.com/2009/09/14/howto-build-your-own-open-source-dropbox-clone/

share|improve this answer
    
-1 I don't think this is the best solution and hence not the answer. –  Henrik Jan 31 '11 at 20:41
    
-1 this is a rsync server not a dropbox like solution... –  Arman Aug 7 '11 at 19:53
1  
This solution is limited in helpfulness because it simply links to another article, creating a risk of linkrot. This answer should be edited to be more substantial, while simply citing the given link as a source. –  WarriorIng64 Nov 13 '11 at 21:27
add comment

I use Unison for the client and rsnapshot (rsync with perl script) for backup the server.

share|improve this answer
add comment

I don't think this is quite what you are looking for, but it depends on your intended usage.

CrashPlan is a backup software package and related online backup hosting service, but what's different is that their software has a mode that allows you to have your data backed up over the internet (or LAN) to another PC running the software.

This means the destination doesn't have to be in the cloud. It's not quite like dropbox in that it's more about backing up rather than syncing and accessing files from everywhere, but if it's just backups you want then it works well. If you want to access the backed up files from the other PC I think you can do a "local restore" but it's not something I've tried.

The basic software package is free and supports the "backup to another computer" mode, but only does scheduled backup but there is a "pro" version of the software that also costs and does real time syncing rather than just the scheduled backups. (Cloud storage is also an optional pay per month extra)

share|improve this answer
add comment

If you want to set up two (or more) machines with a replicated folder, have a look at glusterfs.

It is easy to set up if you follow the GlusterFS User Guide.

share|improve this answer
    
Are you saying that within a GlusterFS cluster, files will be synchronized while the clients are connected to others, but the filesystem will still be available locally to a client that is disconnected from the others? I'm thinking of using this to synchronize between my laptop and server. –  Ryan Thompson Sep 18 '10 at 19:46
    
Yes. I haven't finished my testing yet but it appears to work without any problems. –  Richard Holloway Sep 22 '10 at 15:11
    
This doesn't work on Windows, so it's not "dropbox compliant". –  Henrik Jan 31 '11 at 20:42
add comment

I'm keeping my eye on AeroFS. It looks like it could be a Dropbox-like service where storage in the cloud is optional. Don't know if/when they will implement mobile support and I guess that would require that you sync those files to the cloud aswell. I'm primarily interested in a fairly painless syncing solution between Windows, Mac and Linux computers.

They are in early beta but you can signup for an invite if you want.

share|improve this answer
add comment

I use apache with mod_dav (webdav) for a apache webserver, I can mount it as a drive and up/download files over web. This is pretty simple, but might cover your needs.

share|improve this answer
add comment

No one's mentioned bitorrent sync? Runs on anything - Ubuntu, windows, many common smartphone OSes, raspberry pi.... you name it, it probably works, and as a regular user. Encrypted transfers, files arn't stored on the cloud (though I think bittorrent runs the tracker for it), reasonably fast, you can selectively share folders, and almost no complication involved, you just need to copy and paste a key to the other system.

Once it is set up, it just works.

share|improve this answer
add comment

I use SSHFS to mount directories on my server as local directories on my desktop and laptop. All file changes are saved directly onto the server. Unlike dropbox though, the files are not stored locally on your client machines. I think this is great because you don't have to worry about syncing and versioning, but it's not ideal for offline use or very large files.

It's very direct and simple, and I find it to be the best solution. The only thing I don't use it for is large media like pictures and movies because all files get accessed over the network. Those I sync with Rsync.

Link to SSHFS documentation --> http://fuse.sourceforge.net/sshfs.html

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.