If a server
If you want to run the computer as a server, you should not keep it busy with a desktop environment. I suggest that you try a simple window manager, for example fluxbox or openbox, that you can start when you need it and exit from it when you don't need it.
It is probably enough to install three packages plus the application program(s) that makes you want the graphic user interface,
sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get install fluxbox xinit xterm
(Installing the packages will bring a lot of 'necessary packages', but much less than a full desktop environment).
You can start the window manager with the following command
and manage it via right-clicking to get menus (where you can also exit from the window manager and return to the text screen).
Else (if not a server)
If you do not want to run the computer as a server, it is probably a good idea to make a fresh installation of standard Ubuntu 16.04.1 LTS or one of the Ubuntu family desktop flavours,
Try Ubuntu (Kubuntu, Lubuntu, Xubuntu, ...) before installing it