I installed Zend Server Community edition to get an updated version of PHP.
To me, this is a relatively easy way to keep PHP up to date. Not the latest and greatest, but Zend will eventually catch up IMHO.
The most important steps, for me, was making sure that all the unwanted parts didn't launch when the server rebooted. The Zend Server daemons are controlled via: /usr/local/zend/bin/zendctl.sh which is symlinked to /etc/init.d/zend-server. These can be turned on/off using a variation of
update-rc.d zend-server defaults
PS: If you try this, make sure you type "CE" for "Community edition" during apt-get install, otherwise it's not free (as in beer).