You installed Ubuntu without having the Windows HDD installed, which means that Ubuntu won't know about Windows until you update GRUB. If you want to boot both and have both on the same menu, then simply connected the Windows HDD, turn the PC on and make sure that the first Boot HDD Device is the HDD that has Ubuntu on it. Boot Ubuntu and on the desktop open a terminal and type the following:
GRUB will find the other hard drive and say "Hey there is a Windows partition here on this other hard drive... and it is bootable!". Then wait a couple of seconds an done.
Next time you boot a GRUB menu will appear asking you which system you want to boot to, Ubuntu or Windows.
Just in case, if the menu does not appear, either press ESC like crazy when the computer starts booting or hold SHIFT (Or a combination of both).
UPDATE: Since you have UEFI I urge you to read the answer provided in the following links:
Installing Ubuntu on a Pre-Installed UEFI Supported Windows 8 system
Configure Dual Boot, Windows 7 and Ubuntu 12.04 with or without EFI
Dual boot UEFI Windows 7 and Ubuntu 12.04 (both 64 bits). W7 entry doesn't appear in GRUB
How to boot Ubuntu from EFI/UEFI?
They will cover the cases related to having Windows on a PC that has UEFI support.
UPDATE 2 - Did an actual test of what I said above. After connecting the Windows HDD again and running
update-grub I got the following:
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-3.8.0-7-generic
Found initrd image: /boot/initrd.img-3.8.0-7-generic
Found linux image: /boot/vmlinuz-3.8.0-6-generic
Found initrd image: /boot/initrd.img-3.8.0-6-generic
Found linux image: /boot/vmlinuz-3.8.0-5-generic
Found initrd image: /boot/initrd.img-3.8.0-5-generic
Found memtest86+ image: /boot/memtest86+.bin
Found Windows 7 (loader) on /dev/sdc1
As you can see, GRUB found the boot loader of Windows 7 on that other HDD I just plugged in. I also removed the HDD again and ran
update-grub which did exactly what I thought it would, remove Windows 7 from the list.