Using rsync to write to microsd cards

[jonesmz]

I'm attempting to use rsync to write an OS image to a microSD card. My objective is to minimize the number of writes that need to happen so that I can avoid using the limited writes that the card supports during it's lifetime.

I'm using this command:

rsync --write-devices /path/to/image.img /dev/mmcblk0

Re-writing the image with the same command, with stats output turned on, shows that the full contents of the file is being written.
I also observed the write buffer for the kernel increasing, e.g. cat /proc/meminfo

rsync --write-devices --info=progress2,stats2 /path/to/image.img /dev/mmcblk0
  8,589,934,592 100%   17.64MB/s    0:07:44 (xfr#1, to-chk=0/1)
rsync: [receiver] ftruncate failed on "/dev/mmcblk0": Invalid argument (22)
  8,589,934,592 100%   17.64MB/s    0:07:44 (xfr#1, to-chk=0/1)

Number of files: 1 (reg: 1)
Number of created files: 0
Number of deleted files: 0
Number of regular files transferred: 1
Total file size: 8,589,934,592 bytes
Total transferred file size: 8,589,934,592 bytes
Literal data: 8,589,934,592 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 8,592,031,828
Total bytes received: 35

sent 8,592,031,828 bytes  received 35 bytes  18,457,640.95 bytes/sec
total size is 8,589,934,592  speedup is 1.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1330) [sender=3.2.3]

What I expected to see happen is that rsync would read the data off of the card, and determine that no additional data need be written.

Am I using it wrong?

[WayneD]

A local copy does not try to find differences in the file data, as that just tends to slow down the copy. You can override this using the --no-W (--no-whole-file) option. Even with that, though, rsync will be using a fairly large chunk size for such a large file, so while it may not find a truly optimal minimum set of changes to write, it will hopefully reduce the written data for you (assuming that it can find chunks of matching data between the 2 files). It also helps to be using an rsync that understand the xxhash algorithm, since that helps to speed up the checksum search algorithm.

[jonesmz]

The --no-whole-file option does not prevent rsync from writing the entire file to the sdcard.

aegir /home/jonesmz # rsync --no-whole-file --write-devices --info=progress2,stats2 GenPi64Systemd.img /dev/mmcblk0
  5,293,211,648 100%   27.29MB/s    0:03:04 (xfr#1, to-chk=0/1)rsync: [receiver] ftruncate failed on "/dev/mmcblk0": Invalid argument (22)
  5,293,211,648 100%   27.29MB/s    0:03:05 (xfr#1, to-chk=0/1)

Number of files: 1 (reg: 1)
Number of created files: 0
Number of deleted files: 0
Number of regular files transferred: 1
Total file size: 5,293,211,648 bytes
Total transferred file size: 5,293,211,648 bytes
Literal data: 5,293,211,648 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 5,294,504,027
Total bytes received: 35

sent 5,294,504,027 bytes  received 35 bytes  28,541,800.87 bytes/sec
total size is 5,293,211,648  speedup is 1.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1330) [sender=3.2.3]
aegir /home/jonesmz # rsync --no-whole-file --write-devices --info=progress2,stats2 GenPi64Systemd.img /dev/mmcblk0
  5,293,211,648 100%   24.88MB/s    0:03:22 (xfr#1, to-chk=0/1)

rsync: [receiver] ftruncate failed on "/dev/mmcblk0": Invalid argument (22)
  5,293,211,648 100%   14.98MB/s    0:05:36 (xfr#1, to-chk=0/1)

Number of files: 1 (reg: 1)
Number of created files: 0
Number of deleted files: 0
Number of regular files transferred: 1
Total file size: 5,293,211,648 bytes
Total transferred file size: 5,293,211,648 bytes
Literal data: 5,293,211,648 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 5,294,504,027
Total bytes received: 35

sent 5,294,504,027 bytes  received 35 bytes  15,687,419.44 bytes/sec
total size is 5,293,211,648  speedup is 1.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1330) [sender=3.2.3]