The actual hardware specs are very important to consider. You can test in a virtual machine all day long and find of course that Lubuntu is the lightest in terms of memory usage, installed sized on disk, and very lightweight apps. (meaning also light on memory and small in size, but featured enough to get most daily tasks done easily)
Xubuntu comes in next, though it has been creeping toward the size and heft of regular Ubuntu since 10.04.
Kubuntu and Ubuntu have traded places over time back and forth on which is heftier and slower.
The one thing you can't test in a virtual machine is the processor - you can't mimic anything other than the host machine. This means if you are testing say, a Xubuntu guest on an Ubuntu host and the CPU is a 3.2 GHz beast, when you try to install Xubuntu on a Pentium III, you will be sorely disappointed in performance.
This is no different for Windows guest either.
I have three PIII machines, clocked at 1.2GHz, and 1GB of Ram. Originally, they were 800MHz Celerons with 512MB. They each came with XP Pro and ran respectably, though sluggish over time as XP boxes are want to do. I still have one of these upgraded machines running a clean and stripped down XP and it flies.
Though I'm now running these XP licenses in a VM with only 192MB of RAM allocated to each and they are just as if not more snappy than on bare metal with nearly 5 times the RAM.
However, even Lubuntu on one of these machines is 'syrupy' at best, Xubuntu is visibly sluggish and you have to wait for many basic actions to occur, such as dragging and resizing windows, opening many apps, even the net on basic pages is slow to render at times. (the machines have a 10/100 card which will get upgraded, but the connection I'm on is about 16Mbps - so it shouldn't matter much anyway - I'm seeing no slower downloads than on my other gigabit machines as it is)
Ubuntu is entirely out of the question. I was twiddling my thumbs waiting for dash to appear. (incidentally, it is also sluggish though not as bad on my 2.7GHz AMD with 4GB RAM)
The particular machine I'm testing Xubuntu on though was always noticeably slower even though it was identical to two other machines - same CPU, same brand and type of memory, same motherboard model - all of it. But it was always slower.
My next attempt will be Debian Wheezy LXDE/XFCE, and failing that - I'm going to have to go with a frugal install of Puppy, though perhaps I might venture off into BSD first as I hear it works well on older hardware also.