1

Upgraded from Ubuntu 20.04 to Ubuntu 22.04 using un-requested prompt from Ubuntu to do the upgrade.

I had been running rsync in a Unix shell script from local pc to locally mounted Samba share on a Raspberry Pi for months and it worked fine. Rsync was using -aHv options. Client Samba mount was using CIFS. Shortly after upgrading, I noticed that it was taking longer to rsync. I also noticed that there were more files being transferred than were new on the source local pc.

I investigated and found that any files (these are text files using native Ubuntu text editor) created after the Ubuntu 22 upgrade were getting transferred by rsync during every rsync invocation, even if they had not changed. I also found that all the modified times of the transferred files were current time of transfer and not the local source file's modified time.

Running: rsync -aHv --progress <source directory> <target directory>

This only happens for rsync transfers to a remote Samba share locally-mounted. It does not occur when I test transfer files from one local directory to another local directory.

mount -t cifs -o rw,username=<u>,password=<p>,file_mode=0777,dir_mode=0777 //<IP>/<sharedremote> /media/<sharedlocal>

Ubuntu 22.04 version:

:/var/log/apt$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:    22.04
Codename:   jammy

Probable install date (Aug 16, 2022)

:/var/log/installer$ ls -ltr
total 1160
-rw------- 1 root   root    956 Sep  2  2016 casper.log
-rw------- 1 root   root     19 Sep  2  2016 version
-rw------- 1 root   root 381309 Sep  2  2016 partman
-rw------- 1 root   root   7112 Sep  2  2016 debug
-rw-r--r-- 1 root   root     60 Sep  2  2016 media-info
-rw-rw-r-- 1 root   root 468344 Sep  2  2016 initial-status.gz
-rw------- 1 syslog adm  310006 Aug 16 19:46 syslog
bruce@bruce-Aspire-E1-572:/var/log/installer$

SAMPLE rsync source and target file ls info for 'bad' rsync remote file transfer:

Source:

-rw-rw-r--  1 bruce bruce     26 Aug 30 10:54 test1.txt

Target:

-rwxrwxrwx 1 root root 26 Aug 31 08:23 test1.txt

Hoping this would be enough information to run a test and see if it's happening to others or is something peculiar to my setup. I know enough Ubuntu to be dangerous, but am no expert.

sudodus: thanks for response:

Q: Am I understanding correctly that "during every rsync invocation" means not only the first time after upgrade to 22.04, but every time you run the backup?

A: Yes. Every time the rsync runs, the files get copied and the modified time is updated to current time. Even if the rsync operation is run a few minutes after the first. It seems to be ANY file added after the upgrade and every time after even the first rsync. Files created prior the the upgrade don't get copied (there are hundreds of these).

Q: What operating system (distro and version) are there on the server and the client of the rsync transfer?

Server (Raspberry PI):

$ uname -a
Linux pi4nfs 5.10.103-v7l+ #1529 SMP Tue Mar 8 12:24:00 GMT 2022 armv7l GNU/Linux

cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

$ rsync --version
rsync  version 3.1.3  protocol version 31
Copyright (C) 1996-2018 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes, prealloc

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.

Client:

Ubuntu 22 version:

:/var/log/apt$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:    22.04
Codename:   jammy 

$ rsync --version
rsync  version 3.2.3  protocol version 31
Copyright (C) 1996-2020 by Andrew Tridgell, Wayne Davison, and others.
Web site: https://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, hardlink-specials, symlinks, IPv6, atimes,
    batchfiles, inplace, append, ACLs, xattrs, optional protect-args, iconv,
    symtimes, prealloc, stop-at, no crtimes
Optimizations:
    SIMD, no asm, openssl-crypto
Checksum list:
    xxh128 xxh3 xxh64 (xxhash) md5 md4 none
Compress list:
    zstd lz4 zlibx zlib none

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.

Q; What file system is there on the samba share?

 $ df -Th
Filesystem     Type      Size  Used Avail Use% Mounted on
/dev/root      ext4      458G  171G  264G  40% /
....

Q: Would it be possible to access it some other way, for example a 'normal' mount and directly access it via ssh/rsync?

I can try doing via ssh. I've seen info on that, but have not used it.

Q: Are you prepared to modify the options of the rsync command line until you can avoid copying unmodified files with the current method via samba?

Sure. I have a test script that can easily be changed to do that.

Thanks.

== 09.07.22: sudodus workaround worked. Manual run of rsync using ssh target for rsnch (as in @. (I'll check out new-to-me key generation as he suggested). Since I want to put this in automated script which repeats rsync, I needed to embed the password. So this also worked:

sshpass -f ~/. rsync -aHv --progress <local=source-directory>/ @:/

There are issues with having samba-defined ownership nogroup:nobody ownership to deal with, which I can figure out.

HOWEVER, original issue remains: WHAT behavior changed to create my issue? Samba seems to be the culprit as same physical files work as expected with same rsync but using direct ssh access instead of Samba.

== ANOTHER weirdness. In my testing, I started to notice that editing files in my old (non-test folders) local Samba-connected remote files started to give me pop-up notices that 'the file changed on disk' when I'm the only one on this system (it's personal, not corporate). I'll have to narrow this down, but it seems pretty consistent. It would seem that my old file directories are somehow messed up. So far, it's just an annoyance of having to respond to these popups, especially when saving just having re-opened them.

Improve this question
10
  • Am I understanding correctly that "during every rsync invocation" means not only the first time after upgrade to 22.04, but every time you run the backup? What operating system (distro and version) are there on the server and the client of the rsync transfer? What file system is there on the samba share? Would it be possible to access it some other way, for example a 'normal' mount and directly access it via ssh/rsync? Are you prepared to modify the options of the rsync command line until you can avoid copying unmodified files with the current method via samba?
    – sudodus
    Aug 31, 2022 at 15:15
  • Thanks for response. Since my response is too long to put in comments, I'll post in an answer to my question.
    – ubuntudabbler
    Aug 31, 2022 at 15:44
  • Thanks for a very good explanation (and answer to my questions). - I think that it will work better if you connect via rsync directly (not via samba). I will write an 'answer' with examples of syntax that works for me.
    – sudodus
    Aug 31, 2022 at 16:25
  • 1
    Thanks sudodus for workaround that worked. However, see end of my original question. What changed (seems Samba) and what's with the new text edit weirdness I found? Admittedly I've changed a lot of things testing this, so unless I or someone else can figure out what's happening, it will be a personal problem for me to sort out. Eventually, I might just back everything up, wipe out the files and start from scratch using the rsync target through ssh. Thanks.
    – ubuntudabbler
    Sep 7, 2022 at 13:43
  • 1. Do you mean "it's just an annoyance of having to respond to these popups"? Is this in Ubuntu Desktop 22.04.1 LTS, in the desktop environment? 2. I don't know samba well enough to understand how a change there could cause your 'original' issue.
    – sudodus
    Sep 7, 2022 at 14:21

1 Answer 1

Reset to default
2

I think that it will work better if you connect via rsync directly (not via samba).

This example is copying one file from this computer (my main computer with Ubuntu 18.04.6 LTS) to another one with a test system (Ubuntu 22.04.1 LTS and the package openssh-server installed to provide an ssh server). You may need to install openssh-server.

First a 'dry run' with the option -n

$ rsync -Havn /media/multimed-2/test/mafoelffen/system-info.69 [email protected]:/home/olle/
sending incremental file list
system-info.69

sent 63 bytes  received 19 bytes  164.00 bytes/sec
total size is 76,419  speedup is 931.94 (DRY RUN)

As you see the target syntax is user@IP-address:/path/to/target/directory/. When it looks good, you can remove the n from the option list,

$ rsync -Hav /media/multimed-2/test/mafoelffen/system-info.69 [email protected]:/home/olle/
sending incremental file list
system-info.69

sent 76,538 bytes  received 35 bytes  153,146.00 bytes/sec
total size is 76,419  speedup is 1.00

And when you repeat, nothing is copied, because there is nothing new to copy.

$ rsync -Hav /media/multimed-2/test/mafoelffen/system-info.69 [email protected]:/home/olle/
sending incremental file list

sent 60 bytes  received 12 bytes  144.00 bytes/sec
total size is 76,419  speedup is 1,061.38

When you start with this, you must enter a password. It will be both easier and safer to run rsync between the computers, after providing ssh authorization with a key.

Improve this answer
2
  • Thanks again, sudodus. I'll try that remote server login.
    – ubuntudabbler
    Aug 31, 2022 at 23:26
  • 1
    The rsync behavior with ssh login target is the same as pre-Ubuntu 22 upgrade. Without Samba, there are differences in ownership and a chgrp error when script is first run, but it works. SSHPASS also works: sshpass -f ~/.p127pass.txt rsync -aHv --progress /home/bruce/t1/ [email protected]:/home/pi/Documents/shareit/t1/. I'll have to try your ssh key authorization technique, though my implementation is purely personal, so not as much exposure (or risk) in my non-company setting. Thanks.
    – ubuntudabbler
    Sep 7, 2022 at 2:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .