You don't need to worry about passwords when you can use something called Public Key Infrastructure.
This is a method of using public and private keys to authenticate a user. You store a copy of your private key, and the other server has a copy of your public key. When you log in, they have a little conversation with each other which confirms the public and private keys match so you can log in without entering a password.
This is as secure as anything as long as you don't share your private key with anyone!
To set this up is really simple.
On source machine, run ssh-keygen. You can accept all the defaults, that'll be enough for this purpose. It'll generate your private (id_rsa) and public(id_rsa.pub) keys in your ~/.ssh folder.
Now you want to get the public key on the server, which is easy too.
From the source pc, run ssh-copy-id username@servername.
This will place a copy of id_rsa.pub's content in the server's ~/.ssh/authorized_keys.
Now if you ssh from the source to the destination you will get in without requiring a password.
How does this affect rsync I hear you ask? Rsync uses ssh!
Please remember: NEVER GIVE OUT YOUR PRIVATE KEY (id_rsa) or someone can pretend to be you.