I am using Ubuntu desktop. I'm under the impression even locally there is a client/server architecture for X.
If I separate it into two VMs, one of which runs the entire desktop environment, and a second VM on the same machine that has the only GPU, that connects to the environment, do I get the same GPU acceleration as a default desktop install?
My reason for this is that my base system is linux and I run a linux desktop VM with single primary gpu passthrough. My base system manages the storage and everything nicely, but it still is almost like a dual boot machine since I have to shut down my desktop VM to be able to give the GPU to the windows VM. If my linux GPU VM is just a shell to my actual 'remote' desktop VM everything in it can stay running while I bounce between linux and windows.