Open Terminal: CTRL+ALT+T
Install these Guest Additions Packages:
sudo apt-get install virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11
Make sure you put your sudo user password and packages are installed correctly.
After that installation is Done. Restart your VM
If the problem still stays...
Go to System settings -> Software and Updates -> Additional drivers
Use x86 virtualization solutions...
After that reboot your virtual box
Source: Screen Resolution Problem with Ubuntu 14.04 and VirtualBox
After doing this if you still have the same resolution:
Open Terminal CTRL+ALT+T (Hotkey) type in terminal:
xrandr -s 1024x768
It will set your screen resolution to 1024x768