I also encountered this problem and worked around it in a slightly different way from Vijay. I'm running VirtualBox v4.2.18 on a Windows 7 64-bit host, and I'm trying to install Ubuntu Desktop 12.04.3 (32-bit) onto a new virtual machine.
After Ubuntu supposedly finishes installing and it prompts me to restart, I click to restart, and my virtual machine freezes. At this point, I go back to the main window called "Oracle VM VirtualBox Manager," right click on the virtual machine that is frozen, choose Close > Power Off, and click OK when I get to the VirtualBox warning about losing unsaved data. At this point, the virtual machine is powered off.
Next, I start the virtual machine and it boots again from the installation media. When I'm prompted to Try Ubuntu or Install Ubuntu, I choose Try Ubuntu. After that, the virtual machine loads into the trial version of Ubuntu. Once that happens, I click the top-right power icon in the top nav bar in Ubuntu (within the virtual machine window) and choose Shut Down. During the shutdown process, the machine prompts me to remove installation media, close the tray, and hit Enter. I installed from an ISO so I just hit Enter, and the virtual machine fully shuts down.
The next time I start the virtual machine, it boots into the full Ubuntu like I expect, and I'm able to go on and do other things.