11.10 and newer versions of Ubuntu
GRUB2 and its display of all kernels
The latest versions of Grub2 installed in Ubuntu automatically display the latest kernel and hides the older kernels that you may have installed.

If you do not see your grub - then remember to press Shift whilst booting.
As you can see, only the latest kernel is displayed.
If you select the option shown (press Enter) then all the old kernels become visible and available to boot from.
How to permanently delete older kernels
First boot with the latest available kernel.
There are a number of ways to delete old kernels. Personally, I wouldn't touch Computer Janitor since this is acknowledged to break your computer with its suggestions.
synaptic
An alternative is Synaptic (sudo apt-get install synaptic)
search for linux-image, right-click a kernel and choose complete removal and finally click the Apply button to delete the kernel.

Repeat the search but this time for linux-header - you can delete the associated headers for the kernel image chosen previously.
Synaptic though will not attempt to verify what you are trying to remove... you could inadvertently delete your newest kernel - or even delete all of your kernels via this tool leaving you with an unbootable Ubuntu!.
Remember to check which kernel you are using type:
uname -r
The result would be similar to:

Remember the result and the number - make sure you don't delete the corresponding image or header.
Ubuntu-tweak
IMHO, the best GUI tool is Ubuntu-Tweak
It is not available from the standard repositories. To install in 13.10 and older versions you need to use the author's PPA:
sudo add-apt-repository ppa:tualatrix/ppa
sudo apt-get update
sudo apt-get install ubuntu-tweak
In Ubuntu 14.04+ you have to download .deb file from official website and install it using Software Center or manually.
Choose the options shown by the arrows.
Select both the headers and image with the same version number.
It will not allow you to delete the current kernel you are booted with since the current kernel is not displayed.
Recommendation
My recommendation is to keep at least two or preferably three kernels including the latest. The reason for the recommendation is that you will have at least one/two other kernels to boot with, if for what-ever reason the latest kernel you are unable to boot with or introducing a regressed capability such as broken wireless.
/etc/default/grubif you want to get rid of the 'recovery' items in the menu: #GRUB_DISABLE_LINUX_RECOVERY="true" – poolie Nov 29 '10 at 6:04sudo apt-get autoremoveshould do the trick on Ubuntu 14.04+ – hobs Dec 1 '15 at 17:17