This needs to be said over and over: Don't run outdated software.
PHP 5.3 has reached its end-of-life on 14 Aug 2014, that's 1 year and 8 months ago at the time of writing this answer.
Using outdated end-of-life software will bring you nothing but pain and security vulnerabilities, unless you can invest a time to properly research all existing security vulnerabilities against PHP since the last PHP 5.3.x release and carefully cherry-pick security fixes from more recent PHP releases yourself.
I would definitely recommend to fix the software that requires PHP 5.3 instead, by following the PHP migration guides:
And I would even recommend going straight to PHP 5.6 (or even PHP 7.0) as it will have security support until 31 Dec 2018. The migration guides are here:
The code migration to newer PHP code might seem like a big investment, but in the end this will save you time and money as running outdated software is a serious security risk, especially when interfaced with the big-bad-Internet, but even running it firewalled inside the limited network poses a security risk when your internal network is attacked.
The second option would be to use Ubuntu 12.04 LTS that has PHP 5.3.10-1ubuntu3.22 with security patches prepared by Canonical maintainers. You might think that 5.3.10 is less than 5.3.29, but the fact is that the PHP version in Ubuntu 12.04 LTS has less (serious) security vulnerabilities than vanilla upstream PHP 5.3.29, because the Ubuntu maintainers of PHP are watching the security vulnerabilities and patching PHP to mitigate the serious vulnerabilities in the packages.