All I can say is do your research, and await other replies, as I'm a Ubuntu rookie.
Ubuntu does not perform well (speed-wise) with the very low minimum specifications they say will work (i.e., less than 1 GB RAM). I tried 11.10 and 12.04 with 1.5 GB and 2.0 GB RAM.
I last attempted 12.04 with a Core 2 Duo processor and 2 GB RAM, and was very disappointed by very slow multitasking and file moving. 32-bit was better than 64-bit. The experience was like the initial disappointment with a new Vista system.
I would suggest yes, go for 4GB RAM, as I'm upgrading to 4 GB or maybe 6 GB. Advice I have had from other Linux forums, would support this.
Ubuntu 12.04 is resource hungry, and I am trying out Lubuntu as an option, they say much faster, but less of an overall multimedia sytem than Ubuntu 12.04 LTS.
If you are like me the performance Issues are important. (My PC rage issue.)
I like fast file moving/copying, and I like to multitask ( Vista performs better
at this in my experience in my Ubuntu trials).
So good luck.