Unless you have disabled user/password authentication on your server I would suggest changing it. From a security standpoint having one component of the authentication will make it easier.
Setting up SSH key authentication
User@Host:~$ ssh-keygen -t rsa -b 4096
after generating your key you would do the following:
User@Host:~$ ssh-copy-id <username>@<host>
You will then want to verify that the key is working before disabling password authentication.
User@Host:~$ ssh <username>@<host>
You should be prompted for the passphrase you established when you created the key. If you are and you connect to the server you can now disable password authentication.
/etc/ssh/sshd_config by changing the following line:
After this you want to reload the SSH server
User@Host:~$ /etc/init.d/ssh reload