Setting up InfiniBand on 12.04 should be smooth (except for one minor thing, see below). Most of the horror stories you read are about people that download OFED packages and mess up their systems running some random installer script that stomps all over their system and installs new kernel packages. Don't do that -- just use what's already packaged in Ubuntu!
For IB hardware, really the only choice is Mellanox. You might as well get the latest ConnectX-3 hardware if you are buying new; otherwise you can get some very cheap hardware on Ebay.
Because of older package versions in 12.04 (launchpad bug: ConnectX-3 not supported until 12.10) you'll want to use my PPA to get up-to-date libibverbs/libmlx4 packages if you are using ConnectX-3 adapters (the latest). Other than that, 12.04 should work pretty smoothly.