Unfortunately I am experiencing slow download speeds on Ubuntu 11.10 but not on Windows Vista. My limited knowledge with Ubuntu and my subsequent flawed research to resolve the matter suggest, it might be a problem with Ath9k.
This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, see the FAQ.
I've had a problem on two different machines where the Realtek R8168 wireless network card (use lcpci to verify) was using the R8169 drivers on install. This sort of works, depending on the router, but if you did a "dmesg" you'd see the connection failing all of the time, and the throughput dropping down like what you see. This can also happen with a R8111 card.
Solution: install the right driver for the R8168.
I have no idea if this is your real problem, but it sounds similar to my old problem. Verify the write network drivers are being loaded for your card.
If you're using wireless it might have something to do with power management. Try this:
(where wlan0 is the network interface you use)