Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

We are running an Ubuntu server which is a primary domain controller for several Windows XP clients using samba.

Some of our processes running on Windows generate huge amounts of xml files. Unfortunately we do not have any influence on how this data is stored (e.g. we cannot change the programs to (un)gzip the xml files when they are loaded/saved, they are provided by a 3rd party and we are required to use these programs by contract).

Since our hard disks are overflowing regularly I wonder whether there is such a thing like NTFS compression for Ubuntu/Linux for compressing these files transparently for the clients.

Some more background:

  • Processing these files always means that they are read completely into memory, changed there and written back.
  • Only one client will access a directory of these files at a time so there are no concurrency and locking issues.
  • The server already has 4x 2TB data drives. There is no more physical space to add more (and I would like to avoid adding more disks)
  • The data must be kept active for about two years, after that it will be archived and deleted from the disks.

I think the perfect solution would be some kind of plug in to the samba server that gets called whenever a file which matches a given wildcard / regex is created / written or read which uses gzip / ungzip to compress / uncompress the file on the fly.

Alternatively I could imagine a windows file system driver that does the same on the client side.

I would like to avoid any manual steps in this process because it is too error prone.

Is there something like this? I have already used Google but could not find any solution.

(I would have liked to create a compression tag for this but I am not allowed to do that yet.)

share|improve this question

1 Answer 1

You could mount a compressing filesystem on a directory within the SAMBA share, and save your XML files in that. FUSE provides at least three filesystems that could be used for this purpose: compFUSEd, FuseCompress and LZOlayer_fs.

There is also a kernel patch for plugging transparent compression into the ext2 filesystem but it has a track record of being abandoned and then taken up again, it's for ext2 only (not ext3/4), and it's definitely more invasive than using FUSE.

share|improve this answer
    
I'll wait for btrfs to become stable which will provide this functionality. –  dummzeuch Apr 29 '12 at 16:17

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.