Git is a version control system (a DVCS to be more exact). A VCS (version control system) on a server (or on a computer, or in the cloud), is used to store and track changes to data. This just means that there is a repository of files (usually code in some programming language, along with configuration, instructions, etc.), and in addition to sitting on a server, it has changes made to it tracked across time. The version control system keeps track of any changes to any file in a project, and these projects are hosted, and frequently public or shareable.
A version control system with a front-end (whether git, or subversion/svn, bazaar/bzr, mercurial/hg, etc.) really just allows you to install applications (or packages, code, scripts, patches) that aren't available or up-to-date in the standard channels. Another way to put this is that it allows the authors of the code to share it with you and others.
Git (or another VCS) does this by pulling in all of the data from a repository (the remote IP.IP.IP.IP in your example) and dropping into a directory (that you choose, ultimately) on your system. By itself, this does nothing, unless you are actually running the code that you've gotten.
When you are using a VCS, having gotten the code doesn't necessarily mean you are going to run it as a complete suite. You might test it, update it, merge in some changes, take out some pieces. But in some cases, you can install complete applications (or more) from a repository.
If you can be sure that the code is safe, then this is a great way to play around with goodies that might not be available otherwise.
Having git on your system is unlikely to be a problem. If you're tight on space, you can remove it from the software center or with
sudo apt-get remove git. (There might be additional packages, depending on how you installed it.)
It's hard to know how to undo anything that might have been done with downloaded code without knowing what it is that you've downloaded. You might start by doing something like ls -lart in your home directory (from the command line), or looking in Nautilus file manager for recently modified directories. For both of these methods, you're looking for the last changed directories files, some of which might be the ones you pulled onto your system with git. If you find that, you'll have a better idea if there's anything else you need to remove or get help with undoing.
As far as the adding the host, I'm pretty sure that @Christoph (as well as @psusi) is correct. The easiest way to rollback ssh key changes you have made is to do this:
buntubox#$ rm -i ~/.ssh/known_hosts
Note that this will delete the entire known_hosts file. This isn't usually a problem, because it will be rebuilt as you ssh to additional or repeat hosts.
Alternately, you can edit the file (with gedit, nano, etc.) and remove lines you don't need. (Each line is associated with a remote host.) Again, this should be safe to do -- at worst, you'll have to accept new keys when logging into systems you've logged into before.
