The problem is the age of your Intel Core CPU.
At that time Intel's emphasis was still on processor 'power' (at the expense of excess heat and/or power needs). The only way to really solve this (with your current hardware) is to aggressively limit the CPU and graphics, and slow it further under high load (or heat).
Basically you need add-on software to aggressively manage your thermal envelope (operating range).
That said, I no longer recommend such an approach. Intel have admitted their mistakes, and moved on ..
For Ubuntu:
A modern Notebook with only the current generation (not 'sixth') Intel Core i3 mobile, 4GB RAM, and a hybrid (SSHD) drive will not 'break the bank' and may even out perform your current hardware.
UPDATE: (21-Nov)
I would suggest the HP 250 G4 (N0C12PA) with i3-4005U/4GB RAM, although you may not need the fast processor. Do a price comparison on the lesser model (N0C12PA) with Celeron N3050.
You need to replace the 'standard' (and slow) 5400rpm 500GB HDD (typically Seagate ST500LT012) with a faster Hybrid drive: eg. Seagate 500GB SSHD (with 8GB flash) ST500LM000.
If required replace 4GB SODIMM with 8GB, but first try compiles without this.
PS: This (even with Celeron) also performs well with Windows 10 (64-bit).