I make frequent backups to a local drive which I want to sync daily to a remote server.
The target server is configured for SSH key (no password) access only. Since my primary SSH key for that server is passphrase-protected, I've created a second SSH key (not passphrase protected) + user to use for unattended backups - this way I do not have to be present to enter my passphrase when cron runs.
I'm using cron and rsync, and all of the commands work individually, but fail when combined.
The furthest I've got while troubleshooting is running
env -i sh -c "rsync -lrstRO --delete --exclude 'lost+found' /Backups/auto-daily-backups/./ [email protected]:/backups/desktop/"
which returns the error
Permission denied (publickey).
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.0]
Any tips on how to troubleshoot this further?
Here's what I've tried so far and I'm out of ideas:
- Cron is definitely running
ps aux | grep cron
Nothing unusual in /var/log/syslog
Sep 7 13:22:01 desktop CRON[6735]: (tom) CMD (sh /home/tom/Documents/Scripts/offsite-backup)
SSH in Terminal to remote server as the backup user works
ssh [email protected]
- Running the command in Terminal works perfectly
rsync -lrstRO --delete --exclude 'lost+found' /Backups/auto-daily-backups/./ [email protected]:/backups/desktop/
Manually specifying the path to the backups-user key has no effect
rsync -lrstRO --delete --exclude 'lost+found' -e 'ssh -i /home/tom/.ssh/backups-only' /Backups/auto-daily-backups/./ [email protected]:/backups/desktop/
Replacing the non-functioning command with a simple test command works
echo "Hello world" > ~/Desktop/test.txt
Shouting/swearing at the computer had no effect (but made me feel better temporarily).
Edit 1:
Here's my crontab file and the script it calls.
...
# m h dom mon dow command
MAILTO=""
* * * * * sh /home/tom/Documents/Scripts/offsite-backup
and
#!/bin/bash
rsync -lrstRO --delete --exclude 'lost+found' /Backups/auto-daily-backups/./ [email protected]:/backups/desktop/
Edit 2:
Just to clarify, /var/log/auth.log
on the target server contains the line Sep 11 08:23:01 <hostname> CRON[24421]: pam_unix(cron:session): session closed for user root
This is confusing because I'm no longer running cron every minute locally, but a new entry still appears every minute in the server logs. Crontab files for all users (including root) on the server are empty & do nothing.
Also, user 'backups-only' was created only on the server and with limited rights, with a dedicated SSH key copied to my desktop machine. I'm assuming this is the way to go because everything works when running the commands manually.
The crontab file posted above is for me, user 'tom' on my desktop machine. My intent is to have it call the script which should log in to the server as user 'backups-only'. I just tried running the backup script (rather than the command inside it) and it successfully connected & worked. I ran it on my desktop as user 'tom', same user who created the cron job that won't work. Here's the output from the server log corresponding with that successful login
Sep 11 08:35:31 <hostname> sshd[25071]: error: Could not load host key: /etc/ssh/ssh_host_ed25519_key
Sep 11 08:35:32 <hostname> sshd[25071]: Accepted publickey for backups-only from <desktop IP> port 54242 ssh2: RSA e2:e6:07:27:c1:continues...
Sep 11 08:35:32 <hostname> sshd[25071]: pam_unix(sshd:session): session opened for user backups-only by (uid=0)
Sep 11 08:35:32 <hostname> systemd-logind[638]: New session 12 of user backups-only.
Sep 11 08:36:00 <hostname> sshd[25133]: Received disconnect from <desktop IP>: 11: disconnected by user
Sep 11 08:36:00 <hostname> sshd[25071]: pam_unix(sshd:session): session closed for user backups-only
Sep 7 14:45:01 <hostname> CRON[18716]: pam_unix(cron:session): session closed for user root
Sep 7 16:06:02 <hostname> sshd[6747]...
. Are you 100% positive that this logline is from the server and that it is the correct line? The crontab you posted is the crontab of backups-only? Also, try to add the identity file manually:rsync .... -e 'ssh -i /home/user/.ssh/identity' ...
auth.log
you posted under Edit 2 is for cron running on the server, and should have nothing to do with your login attempts. Can you trytail -f /var/log/auth.log
on the server while you're trying to run the script through cron? Also, I'm not sure if this would work, but can you try your firstenv
command withrsync .... -e 'ssh -vvv -i /home/user/.ssh/identity ...
to see if it spits out more errors?