0

I have an Ubuntu server running as a kvm host. The host has three 1.5TB disks and the host uses lvm and has a 4TB volume group, on which I've created a 3 TB logical volume called nfsserver.
I want to create a guest running ubuntu server, which I'll use as a 3TB fileserver.

Ubuntu vmbuilder uses mbr for the guest filesystem, which cannot handle >2TB harddisks.
I've searched but cannot find any guides or howtos telling, or even mentioning, how to create a guest using GPT for the guest filesystem.

Is there another solution to this that I'm missing?

The command I'm using to create the guest vm:

vmbuilder kvm ubuntu --suite=trusty --flavour=virtual --raw=/dev/server1/nfsserver --rootsize=3000000 --domain=nfsserver --arch=amd64 --mirror=http://dk.archive.ubuntu.com/ubuntu -o --libvirt=qemu:///system --user=admin --name=Admin --pass=PassXXXX --addpkg=unattended-upgrades --addpkg=acpid --addpkg=linux-image-generic --addpkg=openssh-server --mem=1024 --hostname=nfsserver --bridge=br0

With the rootsize set, the vmbuilder fails with "partition length ... exceeds the msdos-partition-table-imposed maximum".
Without it the guest is build and works just fine, but with just the default size disk. When inside the guest system fdisk -l can see that the disk (sda) is 3TB, and the sda1 root partition uses only a few GBs of this. I've tried resizing the partition, but that fails with a similar 'msdos.. max..' error.

I'm aware that I could probably, from inside the guest system, save the small sda1 partition, repartion the filessystem, and change it to use gpt and copy back the saved sda1 partition (to a new 3TB root partition). But that seems like a difficult workaround to me, and I'm thinking that there must be a simpler solution.

lvdisplay on the host:

    --- Logical volume ---
  LV Path                /dev/server1/nfsserver
  LV Name                nfsserver
  VG Name                server1
  LV UUID                F8wWeB-GSUI-rxd3-TDQ1-yjcI-af50-s9jdXZ
  LV Write Access        read/write
  LV Creation host, time server1, 2015-02-02 15:56:20 +0100
  LV Status              available
  # open                 0
  LV Size                3.04 TiB
  Current LE             797964
  Segments               3
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           252:1

EDIT: Thomas' answer below was very helpful in solving my problem, for the benefit of others browsing by this question, here is a detailed description of what I did:

I have a VG called server1 with 4TB of space. To prepare some disks for the guest I created and formatted some LVs:

lvcreate -L 3000G -n nfsserver server1
lvcreate -L 6G -n nfsserver-root server1
lvcreate -L 200M -n nfsserver-boot server1
lvcreate -L 2G -n nfsserver-swap server1
mkfs.ext4 /dev/mapper/server1-nfsserver-boot 
mkfs.ext4 /dev/mapper/server1-nfsserver-root 
mkfs.ext4 /dev/mapper/server1-nfsserver 
mkswap -f /dev/mapper/server1-nfsserver-swap

I installed virtinstall (apt-get install virtinst), because you cannot use vmbuilder for this, as it will create partitions, and as Thomas points out we actually don't need/want partitions for this. With virt-install you can create a guest and run the ubuntu server installation in the guest when it boots - this lets you control what the different disks should be used for. You do need a partition on the disk to be used as /boot though, because the installer is either unable to make the partition itself or will fail at installing Grub. So we use fdisk at the host before creating the guest:

user@server1:~$ sudo fdisk /dev/mapper/server1-nfsserver-boot
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x95a40881.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

The device presents a logical sector size that is smaller than
the physical sector size. Aligning to a physical sector (or optimal
I/O) size boundary is recommended, or performance may be impacted.

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-409599, default 2048): 
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-409599, default 409599): 
Using default value 409599

Command (m for help): p

Disk /dev/mapper/server1-nfsserver-boot: 209 MB, 209715200 bytes
255 heads, 63 sectors/track, 25 cylinders, total 409600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x95a40881

                       Device Boot      Start         End      Blocks   Id  System
/dev/mapper/server1-nfsserver-boot1            2048      409599      203776   83  Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 22: Invalid argument.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
user@server1:~$ sudo fdisk -l /dev/mapper/server1-nfsserver-boot

Disk /dev/mapper/server1-nfsserver-boot: 209 MB, 209715200 bytes
127 heads, 37 sectors/track, 87 cylinders, total 409600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x95a40881

                       Device Boot      Start         End      Blocks   Id  System
/dev/mapper/server1-nfsserver-boot1            2048      409599      203776   83  Linux
user@server1:~$

I ignored the warnings from fdisk, it worked fine. I downloaded the latest Ubuntu LTS

wget http://releases.ubuntu.com/14.04.1/ubuntu-14.04.1-server-amd64.iso

And ran virt-install:

virt-install --name=nfsserver --ram=1024 --os-type=linux --os-variant=ubuntutrusty --cdrom=/home/user/ubuntu-14.04.1-server-amd64.iso --disk path=/dev/mapper/server1-nfsserver-boot --disk path=/dev/mapper/server1-nfsserver-root --disk path=/dev/mapper/server1-nfsserver-swap --network bridge=br0 --vcpus=1 --vnc --noautoconsole

It runs very quickly and boots the guest from the dowloaded iso. Using virt-manager from my desktop computer on the same network as the host, I could connect to the guest and go through the installation. When you come to the point with the partitions, the installer can see all your LV's as disks, and it's just a matter of telling which should be used for what.

I added the 3TB disk afterwords: Shutdown the guest, edit the guest and start it agian:

virsh shutdown nfsserver
virsh edit nfsserver

This opens the guests xml-file defining the guest, I added my 3TB LV as an extra disk below the other disks already present:

<disk type='block' device='disk'>
  <driver name='qemu' type='raw' cache='none' io='native'/>
  <source dev='/dev/mapper/server1-nfsserver'/>
  <target dev='vdd' bus='virtio'/>
</disk>

Then

virsh start nfsserver

And lastly, in your guest you can mount your new drive:

sudo mount /dev/vdd /your/mount/point

And if it mounts, then to make it permanent add this line to your /etc/fstab:

/dev/vdd /your/mount/point ext4 errors=remount-ro 0 0

That's it! You now have a guest with a huge disk. Thanks to Thomas' advice you now also have it amazingly easy, if you want to clone your guest or resize the drives.

1 Answer 1

0

I would just not recommend you to install the guest system on a single 3 TB disk. GPT could solve your problem, but personally I think you get more benefits if you use a more modular approach, and for what reason do we need partitions in a virtual machine anyway? ;) well, MBR is still ok for booting :) Over the last years I found the following setup to be the most flexible when it comes to guests:

  • /dev/sda (100 - 200 MB disk with plain old MBR just for booting)
  • /dev/sda1 mounted on /boot (ext4 or similar)
  • /dev/sdb mounted on / (4 - 6 GB, NO partition, directly formatted with ext4 or similar, used for the base system)
  • /dev/sdc mounted on /srv (in your case ~3 TB, NO partition, directly formatted with ext4 or similar)
  • /dev/sdX: swap or /var or something else and only if we need it, again: NO Partition

First of all I would install a basic ubuntu machine using sda and sdb. as this setup will only consume around 4 GB, you could use it as a template for other guests as well. After you finished the setup, add the 3 TB disk to your guest so it will be recognized as /dev/sdc. Directly format /dev/sdc and mount it under /srv (or any other location you prefer).

When not using partitions you can also resize disks anytime, even while the system is running and even the / partition.

Hope this helps, happy KVMing!

3
  • I'm going to try this. I have some questions though: Is it correctly understood, that I should create LVs for each disk eg. an LV called boot (sda), root (sdb) and nfsserver (the one I already have - sdc) then run mkfs.ext4 on those volumes? After that I'll install the basic machine specifying the LVs as disks. I don't think I can use vmbuilder for this - it will create partitions. So I'm guessing I should use virt-install instead - is that correct? Feb 7, 2015 at 12:46
  • 1. yes, in this case i would use 3 lvs (1 for boot, 1 for / and 1 for the nfsserver). you could use files as well, but this depends on your use case. 2. i also don't think that vmbuilder would support such a setup out of the box, so you can install it manually with virt-install. if you don't care about the partitionless benefits for /, you could just install a base system with vmbuilder using /dev/sda and /dev/sdb and simply add /dev/sdc after the vmbuilder installation has completed. Feb 7, 2015 at 13:31
  • Thanks for your answer Thomas, it was imensely helpfull, and I now have a "base"-guest install, to clone and use when I want new guests, and I feel I now understand a lot more about disks and kvm host/guests. Unfortunately I'm new here, so I don't have enough points to mod you up - hence this comment. Feb 9, 2015 at 13:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .