The Ubuntu minimal virtual install differs from the standard server install in two ways:
- lesser packages are installed;
- a different kernel is installed.
Before Oneiric, the kernel was the one provided by the linux-image-*-virtual packages. Nowadays, the kernel used by the minimal install is linux-image-*-generic. Instead, standard server and desktop installs use both linux-image-*-generic and linux-image-extra-*-generic.
All of this to say that if the minimal install does not work, it's because it lacks some drivers and modules that are shipped with the extra package.
So, the question is: are the drivers sufficient to run Ubuntu on Xen? I don't know and I can't try at the moment. If
vmbuilder does not let you use Ubuntu Minimal with Xen, it's a different matter.
What I can say is that Launchpad PPA builders and Amazon EC2 run on Xen, and I am pretty sure that neither of them are using the extra kernel package.