I see there are discussions addressing installing software using the USC (as well as some comparisons to the Add/Remove Programms feature in Windows), but nothing regarding how one goes about deleting/uninstalling software (such as Google Chrome) that was not installed using the USC (and therefore, is listed in neither the USC nor is it present in the Installed list in Unity).
The Software Center is basically a front-end for the apt package manager. So, if you install a package using apt in the terminal or synaptics (which is also a graphical interface for apt), you'll also find this in the USC and can uninstall it there. It may happen that you have to click on the bottom line "also show technical packages" (or similar) when searching for something for some programs to appear in the USC.
If you install software that does not come as a .deb package but as a script (install.sh for example), hopefully the script also sets up a new script (mostly called uninstall.sh) that will uninstall this software. I think the ATI graphics driver from the ATI website does (or did) that.
Note that the second way requires you to trust that the developer set up the install and uninstall script correctly, otherwise it may happen that not all files are removed.
The message of this is: If you can, use your package manager to install software. This means in the case of Ubuntu, use .deb files, the USC/Synaptics or apt itself. You can even mix them the way you want. They all are built on top of apt, which is the important thing here.