Unless you have disabled user/password authentication on your server I would suggest changing it. From a security standpoint having one component of the authentication will make it easier.
Setting up SSH key authentication
User@Host:~$ ssh-keygen -t rsa -b 4096
after generating your key you would do the following:
User@Host:~$ ssh-copy-id <username>@<host>
You will then want to verify that the key is working before disabling password authentication.
User@Host:~$ ssh <username>@<host>
You should be prompted for the passphrase you established when you created the key. If you are and you connect to the server you can now disable password authentication.
modify your /etc/ssh/sshd_config
by changing the following line:
ChallengeResponseAuthentication no
PasswordAuthentication no
UsePAM no
After this you want to reload the SSH server
User@Host:~$ /etc/init.d/ssh reload