There may well be some annoyances along the way. A few of the proprietary components of Ubuntu (like the Oracle Java runtime and Adobe Flash) might be a bit harder to install that you would expect.
There may also be problems with some obscure wireless drivers and the like - but this is the exception rather than the rule.
Other than that, Linux' great 32 bit compatibility layer ensures that your system will be pretty much rock solid and, often time, quite a bit better at computational task that benefit from the larger address size.
Ubuntu server is now recommended by canonical in its 64 bit form per default. There is, all in all, much trust in the 64 bit linux architecture. It is no longer experimental, it is no longer just an add on. And even though some of the applications haven't caught up (flash is the only one of them that matters, really), the Linux kernel is now considered a 64 bit system with a 32 bit compatibility layer, rather than the other way round.
In short: Do it, go 64 bit - it's running millions and millions of Linux servers today, and it's completely ready for your desktop.
As Huygens points out, 32 bit is the safe bet. If you can stand the potential annoyance of re-installing the 32 bit version, it makes sense to try 64 bit first, and test for common issues.
The 32 bit installer will of course automatically enable a PAE kernel if you have more ram then can be addressed in 32 bits, rendering the ram issue a thing of the past. This has been said many times here, but why not include it again :)