To answer you question about uninstalling, the command is:
apt-get remove <package name>
if you intend to re-install the package later and:
apt-get purge <package name>
if you don't want that package ever again.
As to know what's installed, I suggest you use dpkg directly:
dpkg --list | less
The pipe and less command is so you can easily scroll and search the results of the command. Note that all packages listed are not all installed. The first 2 or 3 letters to the left tell you the current status of the package. For detailed info, check out the man page:
It explains each flag.
Otherwise, as Юнгвирт Тони mentioned, the
apt-cache show command shows the cache, not what you have installed. Not only that, on Ubuntu, if you installed X-Windows, you have a graphical installer. I don't use it and removed the icon, but you should be able to find something named "Software Manager" or alike in your menus or icons...