Command line is sufficient
I understand the idea of a command line interface may be a bit uncomfortable at first, but really it is a reasonable choice. You use the command line interface as on a Desktop so you should already be a little familiar with bash / zsh / shell of choice.
Advantage - fast and secure (via ssh).
IMO desktop environments do not really help on servers as everything server side is command line.
Start/stop services, edit config files, install/update packages, moving data files, firewall and network tools, and user management such tasks are not enhanced by running a graphical terminal.
Installing a desktop environment adds hundreds of packages the vast majority are unused and thus takes up disk space, complicated upgrades, and adds potential security holes (more applications == more potential security holes).
To manage servers use ssh + screen or an alternate to screen - Is there any user friendly alternative to screen?
This allows you to attach and reattach to your server.
Be sure to secure ssh with a least keys http://bodhizazen.com/Tutorials/SSH_security
nano / vim / emacs are all great tools for editing. You can use vim + ssh edit files remotely https://unix.stackexchange.com/questions/202918/how-do-i-remotely-edit-files-via-ssh
Or you can mount your file system over sshfs https://help.ubuntu.com/community/SSHFS
Graphical tools - WEB solutions are better than a desktop
If you feel you NEED a graphical interface use one of the many web based interfaces. These tools are faster than VNC, more secure than VNC, graphical interfaces, and server task specific.
You can secure them via https or ssh (depending on tool).
You can connect to them from any OS via a browser.
- Webmin http://www.webmin.com/
- Cockpit http://cockpit-project.org/
And if needed there are several specialized web interfaces such as
- phpmyadmin - manages databases https://www.phpmyadmin.net/
Vitrualization also has several options.
There a wide range of alternate tools you can find many web interfaces if you google search. https://www.tecmint.com/web-control-panels-to-manage-linux-servers/