Let me try to explain this with an example. Suppose I trust you and want you to SSH into my computer and do something. I can tell you my password or you can give me your public key.
If you give me your public key, when you try to log into my computer, your SSH client will use your private key to generate the encrypted authentication request at your end and send it to my computer. My computer will use your public key to decrypt the encrypted authentication request and thus authenticate that it is you and not someone else trying to break into my computer.
Note: The private-public key pair is not used to encrypt the transmission between two computers. So once the connection is established the keys are not needed until the connection is terminated and a new connection is attempted. SSH has its own internal data encryption system in place that does not depend on the private-public key pair.
So, by accepting your public key, I have trusted you to enter my computer. If you ever break my trust I will delete your public key from my computer. Then you will not be able to SSH into my computer any more.
If you give your private key to someone else (or if someone steals it) then that person will be able to pretend to be you. If I find out that you have given your private key to someone else, I will delete your public key from my computer. Then neither you nor the other person will be able to SSH into my computer.
The public key you gave me is useless if I try to SSH into your computer, as it only good for decrypting information you generate with your private key. If you want me to SSH into your computer, you will have to put my public key in the right place in your computer.
Ubuntu stores your private keys in password management software called Password and Encryption Keys:
Note, the middle tab My Personal Keys. This to protect your private key from being stolen (and from accidentally deleting your private key).
Here is more information about the Seahorse password manager.
Hope this helps