Seeing this, I had to add to the last answer as there is some possible improvement.
If you removed virtualbox 5.0 and upgraded to 5.1, there may be some packages that were not fully removed.
Here is what works best if you want to keep everything fine without any corrupt packages:
remove Virtualbox 5.1
sudo apt-get remove virtualbox-5.1
Remove all unused packages, which will remove some Virtualbox packages. It may also remove more packages and previous linux headers if you upgraded and did not run this command before.
sudo apt-get autoremove
Make sure there is nothing left from virtualbox
sudo apt-get purge virtualbox virtualbox-qt
Reinstall virtualbox 5.1 (or virtualbox, if you want 5.0)
sudo apt-get install virtualbox-5.1
sudo apt-get install virtualbox
If you want to rollback, you may use the same process, but install virtualbox instead of virtualbox-5.1, but if you add some virtualbox modules installed like guest additions, you may have to reconfigure it.