Logging Issues with SFTP bash script in Cron Job

Question

I have a variation of this question with some additional details. My script is being run by the cron job, but it's not doing the SFTP and not logging like what I hoped for. When I check where it's supposed to be sending the files, they are not there. The files only appear there when I run manually, and not when run in the Cron job.

I have two attempts to log:

First in the cron job itself (I've tried with both my user and "root"):

15 17 * * * nw /bin/sh /home/nw/scripts/SftpScriptLoop.sh >> /home/nw/scripts/SftpScriptLoop_cron.log 

and secondly in the script to log.txt.

When I run the script myself, the log.txt gets populated, but it is never populated when I run with the Cron job.

I know the cron job is running, because SftpScriptLoop_cron.log gets created, but it's empty. I deleted it before testing again just to verify.

I'm using fully qualified disk paths in the script.

My Bash Script (it uses a "here doc" to pass data to the stdin of the SFTP command):

#!/bin/bash
for filename in /home/nealwalters/downloads/*.csv; do 
   currentDate=`date` 
   echo "Loop: Filename=$filename" 
   echo "$currentDate SFTPUser3 Filename=$filename" >> /home/nealwalters/scripts/log.txt

   sftp -i /home/nealwalters/keys/SFTPUser1_Private.pem SFTPUser3@192.168.1.179 << EOF
      cd User3ToMyCompany
      put $filename
      exit
EOF
   echo "--------------------------------------------"
done

Ubuntu 20.04

Results of "grep CRON /var/log/syslog".

---

Part 2 - when in doubt simplify. Updates one day later.

I liked the answer from this question, but I cannot even get the following to work.

I created three files with some dummy text in them: TouchFileScript1.txt and TouchFileScript2.txt and TouchFileCron.txt

I created a bin directory (not sure if that is required, as originally my directory name was "scripts".)

I created a bash script called TouchFileCron1.sh with just the following one line:

touch /home/nealwalters/bin/TouchFileScript1.txt 

I created a bash script called TouchFileCron2.sh with just the following one line (trying to figure out if the /sh/bin is required or not in the crontab):

touch /home/nealwalters/bin/TouchFileScript2.txt 

The /etc/crontab now has two lines as follows:

* *  * * * root touch /home/nealwalters/bin/cronlog.txt 
* *  * * * root /home/nealwalters/bin/TouchFileScript1.sh
* *  * * * root /bin/sh /home/nealwalters/bin/TouchFileScript2.sh 

Basically, the first one works, and the second two do not work.

Result - tail of /var/log/syslog - and listing all my files. You can see which ones are marked as executable. The TestFileCron.txt has the date/time changed every minute. The other two are unaffected.

Does the ownership matter, since I'm running "root" in the crontab? 17:28 was the last time the cron ran, when I took the screen shot. The other files remain at 17:22 when I created them.

Answer 1

Since I was redirecting output to files, I thought my case was different, but it was essentially solved by this post: "(CRON) info (No MTA installed, discarding output)" error in the syslog

I didn't have any redirection for "stderr".

Coming from a Windows background, it was just incredible to me that mail would be used to log error, rather than writing them to a file.

Linux uses mail for sending notifications to the user. Most Linux distributions have a mail service including an MTA (Mail Transfer Agent) installed. Ubuntu doesn't though.

You can install a mail service, postfix for example, to solve this problem.

sudo apt-get install postfix

Then to view the errors:

sudo tail -f /var/mail/nealwalters

(where nealwalters can be substituted with your user).