First, make a backup copy of /etc/default/grub. In case something goes wrong, you can easily revert to the known-good copy.
sudo cp /etc/default/grub /etc/default/grub.bak
Then edit the file using the text editor of your choice (ie. gedit, etc.).
sudo gedit /etc/default/grub
Find the line that contains GRUB_DEFAULT=0 - this is what you'll want to edit to set the default. To know what to change it to, you must know where it is on the list (1st, 2nd, 3rd, etc.).
For example, on my computer, I have:
- The Latest Kernel
- The Latest Kernel (Recovery mode)
- Previous Linux Versions
- Memory test (memtest86+)
- Memory test (memtest86+, serial console 115200)
If you choose the "Previous Linux Versions", you will see:
- Old Kernel
- Old Kernel (Recovery mode)
- Older Kernel
Essentially, all the older kernels that are still installed. These are sub-choices, of a sort. So in my case, since "Previous Linux Versions" is 3rd on the boot list, the first kernel inside of it would be "2>0", the second "2>1", and so on; counting on the list starts at 2>0 since grub uses 0 as the first option, not 1.
EDIT: As Bealer has stated below, using a numeric value can cause problems following system updates. Therefore, it's better to use a text default, i.e.
GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 3.13.0-53-generic" (including quotes).
In summary, if you want to use a kernel in the "Previous Linux Versions" menu, you'll want to change GRUB_DEFAULT=0 to GRUB_DEFAULT="2>x" (make sure to include the quotations), where x is the placement of the old kernel on the sub-list (assuming the "Previous Linux Versions" is third on the main list).
Save it, then build the updated grub menu.
I hope that wasn't too confusing; I might have babbled a little :/