To allow your computer to boot normally rather than returning to the GRUB menu, follow this: http://www.howtogeek.com/howto/33433/restore-the-windows-boot-loader-after-an-ubuntu-update/
In short: insert Windows installation disk, select language and use the repair feature using a system image that you created earlier.
The link provides more details + images. Any time that I have done this it has automatically fixed the Windows bootloader. However, it does have the manual fix for it, if necessary.
Get something to edit partitions, I just use GParted. Delete all of the Linux partitions. The ext3/ext4, swap, etc. and leave NTFS partitions for Windows. It may show "unknown" partitions instead of ext/swap partitions if you are doing this from Windows.
If you installed Ubuntu from a flash drive and set it to dual boot, you must have selected it from the menu while installing. Fortunately, deleting it is pretty easy. It's mostly just right click -> delete work from GParted.
Note: You will need to put GParted on your USB or burn it to a CD to edit it since you can't delete a partition that you're working on.
Optionally, you can try "Parted Magic" which is for both Windows and Linux, this way you will not need to use your USB or CD. Just install it to Windows. I've never used this program but I have heard good things about it and it should be relatively simple to using GParted.