HDD pooling/partitioning suggestions for media server

Question

I have multiple hdds that I would like to pool together. Sizes are as follows: 2x1TB; 2X2TB; and 1x4TB. Running Ubuntu server 14.04 on a separate SSD with the primary purpose of using this machine as a media server. Currently have about 2.5TB-3TB worth of data that would be stored in the pool. In addition, it will be used for storing client OS images, file shares, and running a VM environment for testing/learning.

Data replication for media is crucial, but it does not have to be live replication so long as if a drive fails my data is accessible somewhere (was thinking an rsync cron job). While I could be wrong, I think RAID would be overkill, and difficult to implement with different size drives. However, RAID is a possible upgrade in the future. My intention for right now is just to make use of all the drives I have.

Not all drives need to be apart of the pool, in fact I was considering separating/isolating out one of the 1TB drives for storing OS backups and VM usage. Bulk of storage space would be dedicated to media storage.

Suggestions??? TIA

Answer 1

If you have 8GB mem and run a 64-bit system, have a look at zfs: zfs on linux FAQ, nice presentation.

Answer 2

Thanks, I was checking that out, as well as others. I'm fairly new to linux and the ability to choose my file system. In the end decided to go with btrfs, and so far so good. Just a file system after all. Decided against zfs, as most articles I read indicated it was losing favor over btrfs for various reasons. Haven't done much more than basic setup and creating subvolumes. Its ease of adding more storage (pooling) and snapshots (I believe zfs does these things too) were the key features for me.

Answer 3

When you say HDD pooling, do you mean creating one big (or several bigger) logical volume(s) out of the HDD's? If that's what you mean, you should take a look at LVM. I've used it to combine two 3TB HDD's in my home server (I have a separate one for the OS) into one big volume group and of top of that I created several logical volumes to be used for different purposes (ownCloud data, backup drives for different family members, video/music storage). It's easy to use and never had any big problems with it (yet). (Even if it's not what you meant you could use it to easily change the partition sizes.)

You could use RAID (on top of LVM, I believe) to duplicate the data to ensure the data is still available to the server, but remember that it's not a substitution to regular backups on another machine/HDD!