Why is the sync option used with the `dd` command?

Question

Following instructions on this page, the page showed a dd command and a sync option at the end. The command is the following:

xzcat ~/ubuntu.img.xz | sudo dd of=/dev/sdX bs=32M sync

I know what is dd and how it works but I've never heard of or used the sync option with it and its manual page entry is like:

sync   pad every input block with NULs to  ibs-size;  when  used  with
       block or unblock, pad with spaces rather than NULs

What is NUL and why is it padded to ibs-size, and why bother padding the data blocks and use the sync option with the dd command? Why not keeping it simple and easy?

When I try to run sync with the command as it is with the appropriate location and values I get the following error:

dd: unrecognized operand ‘sync’
Try 'dd --help' for more information. 

Answer 1

You are misreading the command. It's:

xzcat ~/ubuntu.img.xz | sudo dd of=/dev/sdX bs=32M 
sync

sync is a separate command, here. See man 1 sync:

NAME
       sync - flush file system buffers

SYNOPSIS
       sync [OPTION]

DESCRIPTION
       Force changed blocks to disk, update the super block.

Kinda obvious why it's used once you think of caching.

As for the dd option you read, it's a value for the conv option:

Each CONV symbol may be:
   ascii  from EBCDIC to ASCII
   ...
   sync   pad every input block with NULs  to  ibs-size;  when  used  with
          block or unblock, pad with spaces rather than NULs

If it was used (which it isn't), it would be used thus:

dd ... conv=sync

Answer 2

Dd will read in data one block at a time (the block size is specified by the user). Dd may possibly read in a shorter block than the user specified, either at the end of the file or due to properties of the source device; this is called a partial record. By default, dd will then write out a block that's the same size as the amount that it read.

Dd will repeat this until the specified count is reached, or it sees eof on input, or error on input or output. When it finishes, dd reports the number of full and partial records it read and wrote.

This behavior is fine for copying an ordinary file within a filesystem or over a TCP network connection, since that's considered a stream of bytes. But other filesystem objects, such as raw DVDs and magnetic tape, require write sizes that are within certain ranges and are a round multiple of some amount (such as 512 or 2048 bytes). For example, if you have a disk image that is 255 tracks of 63 512-byte sectors, and want to write to a tape that requires a block size of 1024 bytes, you'd need to do something like

dd if=disk.img of=/dev/rmt0 bs=1k conv=sync

to make sure that dd doesn't try to write out a 512-byte block at the end. The shorter block at the end will be padded with zeroes or blanks. Padding with zeroes is a safe, common choice. The option for dd to pad with blanks is useful in a different situation - converting a file that has variable-length lines to one with fixed-length lines.

Answer 3

You can use sync option for output, which is included in dd comand, by oflag symbol:

xzcat ~/ubuntu.img.xz | sudo dd of=/dev/sdX bs=32M oflag=sync

Answer 4

I have found that the linux system, not dd, caches the buffers. Therefore, when the dd command has completed, I run sync as

sudo dd if=my.downloaded.iso of=/dev/sd(?) bs=1M && sync

I have not used any internal dd option to cause buffer flushing. Typically though, when I am creating a USB version of a distribution, I am the only one on the computer, so I have no qualms about my posted command doing harm. Since I run the sync as &&, sync is done with sudo rights and I am sure that the USB I create is complete.