Merge branch 'docs/readme-refresh' into dev

# Conflicts: # README.md
2026-09-14 18:52:54 +02:00
parent 919a729206 81ad313ee5
commit 09c384d7d0
6 changed files with 558 additions and 148 deletions
@@ -1,4 +1,4 @@
-#FastSync
+# FastSync

 FastSync is a high-performance file synchronization tool designed to become a
 drop-in replacement for common `rsync` workflows. It keeps the familiar
@@ -7,7 +7,7 @@ multithreading, streaming zstd compression, chunking, zero-copy TCP transfers,
 and native TCP/TLS transports.

 The release version is FastSync's client/server protocol version (printed by
-`fastsync --version`); client and server must match. See
+`./build/client --version`); client and server must match. See
 [CHANGELOG.md](CHANGELOG.md) for the history.

 The compatibility target is straightforward:
@@ -51,8 +51,9 @@ replacement for every rsync feature or protocol mode.
 - Rsync-style source and destination arguments.
 - SSH transport using `user@host:destination` paths below the remote authorized root.
 - TCP client/server transfers.
- Dry runs, excludes, includes, size filters, backups, statistics, and
-  bandwidth limiting.
+- Dry runs (server-contacting since protocol 2.21.0 for server-routed targets),
+  excludes, includes, size filters, backups, statistics, and bandwidth
+  limiting.
 - Incremental size/mtime checks and optional xxHash64 content checks.
 - FastSync-native delta transfer for changed files.
 - Optional mode and timestamp preservation.
@@ -64,11 +65,21 @@ replacement for every rsync feature or protocol mode.

 - The FastSync wire protocol is not the rsync wire protocol.
 - SSH mode requires `fastsync-server` on the remote host.
- Archive mode does not yet provide all of rsync's `-rlptgoD` behavior.
- Symlink transfer is incomplete; link targets are not yet recreated in all
-  modes.
- Owner/group, ACL, xattr, and hard-link handling is incomplete or
-  unavailable.
+- Archive mode covers rsync's `-rlptD` behavior — links, permissions, times,
+  devices, and special files — and does not imply compression or multithreading
+  (see [Client](#client)). Owner/group (`-o`/`-g`) are NOT implied; identity is
+  applied only through the opt-in identity flags (`--chown`/`--usermap`/
+  `--groupmap`/`--numeric-ids`/`--copy-as`).
+- Symlink transfer recreates only relative, `..`-free link targets
+  (`-l`/`--links`); an absolute target or any target containing a `..` component
+  is dropped rather than created, even if it would resolve within the receive
+  root. This containment check is skipped under `--trust-sender`.
+- Hard links (`-H`/`--hard-links`), extended attributes (`-X`/`--xattrs`), and
+  POSIX ACLs (`-A`/`--acls`) are preserved; owner/group is applied only through
+  the opt-in identity flags (`--chown`/`--usermap`/`--groupmap`/`--numeric-ids`/
+  `--copy-as`) and only when the receiver has permission. See
+  [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md) for the exact semantics and documented
+  divergences.
 - Device and special-file preservation is implemented with documented
  divergences: recreated device nodes require `CAP_MKNOD` on the receiver (a
  non-root receiver skips the entry), and sockets cannot be recreated (FIFOs
@@ -80,9 +91,8 @@ replacement for every rsync feature or protocol mode.
  the write atomic (temp + rename). With `--partial`, a failed/interrupted write
  now retains the already-written temp at the destination path (best-effort) so
  a later `--append`/`--append-verify` run can resume it.
- `--dirs` is not implemented. Its compatibility aliases `--old-dirs` and
-  `--old-d` are recognized but rejected explicitly rather than silently using
-  FastSync's recursive directory behavior.
+- `-d`/`--dirs` and its aliases `--old-dirs`/`--old-d` transfer the named
+  directory entries without recursing into their contents.
 - Short-option names are now rsync-parity (Phase 7 Wave A): FastSync's former
  collisions were renamed (`-j`/`--threads`, `--preserve`, `--sendfile`,
  `--chunk-serialization`, `--timeout`, `--ssh-port`), so `-m`, `-M`, `-f`,
@@ -96,7 +106,12 @@ partial, alternate, and planned behavior.

 ### Build

-`compile_commands.json` is a symlink to `build/compile_commands.json` and is used by clangd/editor tooling; its target is generated by the build, so it dangles until the first build.
+```bash
+cmake -B build -S .
+cmake --build build -j$(nproc)
+```
+
+This produces `./build/client` and `./build/server`. `compile_commands.json` is a symlink to `build/compile_commands.json` and is used by clangd/editor tooling; its target is generated by the build, so it dangles until the first build.

 ### Client

@@ -105,53 +120,101 @@ partial, alternate, and planned behavior.
 | Positional | `<source> <dest>` — automatic SSH detection if dest contains `:` |
 | `-c, --checksum` | Verify content by checksum instead of size+mtime |
 | `-z, --compress [level]` | Enable streaming zstd compression (level 1–22, default 5) |
-| `-a, --archive` | rsync archive mode (`-rlptgoD`): links, metadata, devices and specials (not compression/multithreading) |
+| `-a, --archive` | rsync archive mode (`-rlptD`): links, metadata, devices and specials; owner/group (`-o`/`-g`) are not implied and stay opt-in via the identity flags (not compression/multithreading) |
 | `-j, --threads[=N]` | Multithreading mode; `N` (1–256) sets the parallel scanner worker count, bare `-j`/`--threads` uses the default |
 | `-m` | rsync `--prune-empty-dirs` (short form now rsync-parity) |
+| `-d, --dirs` | Transfer the named directory entries without recursing into their contents; aliases `--old-dirs`/`--old-d` |
+| `-R, --relative` | With `--files-from`, preserve each listed entry's relative path below the destination root |
 | `--chunk-serialization` | Chunk serialization (batch all files per chunk; long form only) |
 | `-s` | rsync `--secluded-args` compatibility no-op (remote SSH argv is already injection-safe) |
 | `--sendfile` | Sendfile zero-copy. Incompatible with compression / chunk serialization. TCP only. Long form only. |
-| `--preserve` | Preserve supported file metadata (mode and mtime; ownership and atime are unsupported) |
-| `-n, --dry-run` | Scan and print what would be transferred |
+| `--preallocate` | Allocate destination file space up front (fail-fast on a full disk) |
+| `--append` | Resume a shorter destination by appending only its tail (prefix not verified; requires `--incremental`) |
+| `--append-verify` | Like `--append`, but verifies the retained prefix checksum first (falls back to a full transfer on mismatch) |
+| `-W, --whole-file` | Transfer changed files without delta processing |
+| `-I, --ignore-times` | Transfer files even when size and mtime match |
+| `--size-only` | Skip incremental files matching in size, ignoring mtime |
+| `--preserve` | Preserve file metadata (mode and mtime; add `-U`/`--atimes` for atime, or an identity flag for owner/group; `-N`/`--crtimes` captures birth time but cannot apply it) |
+| `-U, --atimes` | Preserve access times. Captured with the metadata payload; does not enable ownership. |
+| `-N, --crtimes` | Capture birth time; cannot be applied (documented divergence) |
 | `-p, --perms` | Preserve permission bits (part of the metadata bundle) |
-| `--ssh-port <port>` | SSH port (default: 22) |
-| `-v, --verbose` | Enable debug logging |
-| `-q, --quiet` | Suppress non-error output |
-| `--progress` | Show real-time transfer speed |
-| `-P` | Enables partial-transfer mode + progress output; interrupted writes retain the already-written temp for resumption |
+| `-E, --executability` | Preserve executable permission bits |
+| `-X, --xattrs` | Preserve user `user.*` extended attributes |
+| `-A, --acls` | Preserve POSIX ACLs |
+| `--chmod <changes>` | Modify transferred permissions (rsync syntax) |
+| `--chown=USER:GROUP` | Override the ownership of transferred files |
+| `--usermap=MAP` | Map usernames when applying ownership |
+| `--groupmap=MAP` | Map group names when applying ownership |
+| `--numeric-ids` | Apply source numeric uid/gid directly instead of mapping by name |
+| `--copy-as=USER[:GROUP]` | Force every written entry to USER[:GROUP] (requires a privileged receiver) |
+| `--fake-super` | Record/replay effective metadata via a reserved `user.fastsync.stat` xattr |
+| `--super` | Permit the receiver to attempt confined super-user activities (device nodes) |
+| `-D` | Preserve device and special files (implies `--devices --specials`) |
+| `--devices` | Recreate device nodes on the destination (privileged; skipped without `CAP_MKNOD`) |
+| `--specials` | Recreate special files (FIFOs); sockets cannot be recreated |
+| `--remove-source-files` | Remove regular source files after a successful transfer |
+| `--exclude <pattern>` | Exclude files matching glob pattern (repeatable) |
+| `--exclude-from <file>` | Read exclude patterns from a file (one per line) |
+| `--include <pattern>` | Only transfer files matching glob pattern (repeatable, whitelist) |
+| `--include-from <file>` | Read include patterns from a file |
+| `--files-from <file>` | Read the source file list from FILE (paths relative to the source root) |
+| `--max-size <n>` | Skip files larger than n bytes |
+| `--min-size <n>` | Skip files smaller than n bytes |
+| `--max-alloc <SIZE>` | Maximum single allocation (binary units: B, K, M, G, T, P, E; default 1G) |
+| `-u, --update` | Skip files newer than the source on the receiver |
+| `--incremental` | Skip files unchanged since last transfer (size + mtime). Auto-enables `--preserve`. Incompatible with `--chunk-serialization`. |
+| `--existing` | Skip files not already present at the destination; update existing files normally. |
+| `--compare-dest <dir>` | Extra comparison basis: unchanged files are not transferred (requires/implies `--incremental`) |
+| `--copy-dest <dir>` | Like `--compare-dest`, but copies the unchanged file from DIR into the destination |
+| `--link-dest <dir>` | Like `--copy-dest`, but hard-links the unchanged file from DIR (repeatable; earlier DIRs win) |
 | `--delete` | Delete files on receiver not present in source (default timing: delete-after, i.e. only after the whole transfer succeeded) |
 | `--delete-before` | Delete extras before the transfer starts (implies `--delete`) |
 | `--delete-during`, `--del` | Delete extras once the keep-set is known, before data is applied (implies `--delete`) |
 | `--delete-delay` | Delete extras only after a successful transfer (implies `--delete`) |
 | `--delete-after` | Explicit delete-after timing (implies `--delete`) |
-| `--exclude <pattern>` | Exclude files matching glob pattern (repeatable) |
-| `--exclude-from <file>` | Read exclude patterns from a file (one per line) |
-| `--include <pattern>` | Only transfer files matching glob pattern (repeatable, whitelist) |
-| `--max-size <n>` | Skip files larger than n bytes |
-| `--min-size <n>` | Skip files smaller than n bytes |
-| `--max-alloc <SIZE>` | Maximum single allocation (binary units: B, K, M, G, T, P, E; default 1G) |
-| `--incremental` | Skip files unchanged since last transfer (size + mtime). Auto-enables `--preserve`. Incompatible with `--chunk-serialization`. |
-| `--existing` | Skip files not already present at the destination; update existing files normally. |
-| `--bwlimit <KB/s>` | Bandwidth limit in kilobytes per second |
-| `--chunk-size <n>` | Chunk size in bytes (default: 10485760) |
-| `--timeout <sec>` | Positive I/O timeout in seconds, applied to both the socket (`SO_RCVTIMEO`/`SO_SNDTIMEO`, built-in default 30 s) and the per-message protocol poll deadline (built-in default 60 s). Omit the option to keep both built-ins; `0` is rejected. The server side keeps the built-in 60 s protocol window (the value is not sent on the wire). |
-| `--contimeout <sec>` | Connection timeout in seconds (default: 10) |
-| `--backup` | Backup existing destination files before overwriting |
-| `--backup-dir <dir>` | Target directory for backups (requires `--backup`) |
+| `--delay-updates` | Put updated files into place only at the end of the transfer |
+| `-n, --dry-run` | Report what would be transferred without mutating the destination. Since protocol 2.21.0 a server-routed target contacts the receiver and reports would-transfer based on receiver state; a plain local destination keeps the client-side scan. Never mutates or deletes. |
+| `-v, --verbose` | Enable debug logging |
+| `-q, --quiet` | Suppress non-error output |
+| `--progress` | Show real-time transfer speed |
+| `-P` | Enables partial-transfer mode + progress output; interrupted writes retain the already-written temp for resumption |
 | `--stats` | Print transfer statistics at end (bytes, files, timing) |
+| `-i, --itemize-changes` | Print an rsync-style per-file change line |
+| `--out-format=FORMAT` | Output format for changed files (`%f %n %l %b %M %%`) |
+| `--list-only` | List source files instead of transferring |
+| `--fsync` | Fsync every written file before publication |
 | `-h, --human-readable` | Format transfer byte sizes with binary units |
 | `--max-depth <n>` | Maximum directory depth to recurse (0 = unlimited, default: 0) |
 | `--log-file <path>` | Write log messages to file instead of stderr |
+| `--write-batch=FILE` | Run the normal live transfer and also emit a self-contained batch file of the source tree |
+| `--only-write-batch=FILE` | Emit the batch file only (no destination, no server) |
+| `--read-batch=FILE` | Apply a batch file to the destination (no source, no server) |
 | `--source-dir <path>` | Source directory (overrides `FASTSYNC_SOURCE_DIR`) |
 | `--dest-dir <path>` | Server destination directory (overrides `FASTSYNC_DEST_DIR`) |
 | `--save-to-disk` | Write received files to disk |
 | `--server-host <ip>` | Server IP address (default: `127.0.0.1`) |
 | `--server-port <n>` | Server port (default: `8080`) |
+| `--ssh-port <port>` | SSH port (default: 22) |
+| `-e, --rsh <command>` | Remote shell to launch for the SSH transport (default: `ssh`; may include arguments, e.g. `-e "ssh -p 2222"`) |
+| `-M, --remote-option=OPT` | Append OPT to the remote server invocation over SSH (repeatable) |
+| `--address <ip>` | Bind the outgoing client socket to this source address |
+| `-4, --ipv4` | Force IPv4 for destination resolution |
+| `-6, --ipv6` | Force IPv6 for destination resolution |
+| `--sockopts=OPTS` | Comma-separated OPT=VAL socket options applied before connect (`TCP_NODELAY`, `SO_KEEPALIVE`, `SO_RCVBUF`, `SO_SNDBUF`, `SO_REUSEADDR`) |
+| `--bwlimit <KB/s>` | Bandwidth limit in kilobytes per second |
+| `--chunk-size <n>` | Chunk size in bytes (default: 10485760) |
+| `--timeout <sec>` | Positive I/O timeout in seconds, applied to both the socket (`SO_RCVTIMEO`/`SO_SNDTIMEO`, built-in default 30 s) and the per-message protocol poll deadline (built-in default 60 s). Omit the option to keep both built-ins; `0` is rejected. The server side keeps the built-in 60 s protocol window (the value is not sent on the wire). |
+| `--contimeout <sec>` | Connection timeout in seconds (default: 10) |
+| `--stop-after=MINS` | Stop the transfer after MINS minutes (a positive integer); whatever was already transferred is kept |
+| `--stop-at=TIME` | Stop at an absolute time (`HH:MM`, `HH:MM:SS`, or `now+N[smhd]`); an early stop skips the late `--delete` keep-set |
+| `--backup` | Backup existing destination files before overwriting |
+| `--backup-dir <dir>` | Target directory for backups (requires `--backup`) |
 | `--tls` | Enable TLS encryption |
 | `--cert <path>` | TLS certificate file (PEM) |
 | `--key <path>` | TLS private key file (PEM) |
 | `--ca <path>` | TLS CA certificate file for verification (PEM) |
-| `--client-cn <name>` | TLS client certificate common name; mandatory with `--tls` (a TLS connection always verifies the client CN) |
+
+The exhaustive rsync flag matrix is in [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md).

 **Per-message vs. connection timeouts.** `--timeout` bounds each individual protocol
 send/receive (the `poll()` deadline), so a peer that stops mid-frame is dropped. It
@@ -190,60 +253,64 @@ transfer is never aborted.
 | `FASTSYNC_SOURCE_DIR` | — | Source directory fallback |
 | `FASTSYNC_DEST_DIR` | — | Destination directory fallback |
 | `FASTSYNC_SAVE_TO_DISK` | `false` | Disk persistence fallback |
-| `FASTSYNC_SSH_PORT` | `22` | Default SSH port |
-| `FASTSYNC_SERVER_HOST` | `127.0.0.1` | Default server host |
-| `FASTSYNC_SERVER_PORT` | `8080` | Default server port |
-| `FASTSYNC_TLS_CERT` | — | Default TLS certificate path |
-| `FASTSYNC_TLS_KEY` | — | Default TLS private key path |
-| `FASTSYNC_TLS_CA` | — | Default TLS CA certificate path |

 ## Implementation Details

 ### Data Structures
-1. **Chunk** — collection of files (~10 MB total by default)
-2. **File** — path, content (`Data`), optional `FileMetadata` pointer
-3. **FileMetadata** — `mode`, `uid`, `gid`, `mtime_sec`, `mtime_nsec`;
-uid / gid are advisory wire fields and are never applied by the receiver;
-atime is unsupported
-4. **Config** — runtime parameters (transported over wire, TLS settings excluded). Includes `timeout`, `contimeout`, `quiet`, `backup`, `backup_dir`, `stats`, `max_depth`, `log_file`.
-5. **Queue** — thread-safe bounded queue with condition variables
-6. **DirectoryScanner** — recursive BFS traversal with exclude and include pattern support, max-depth enforcement
+
+1. **Chunk** — collection of files (~10 MB total by default).
+2. **File** — path, content (`Data`), optional `FileMetadata` pointer.
+3. **FileMetadata** — `mode`, `uid`, `gid`, `mtime_sec`, `mtime_nsec` (plus
+   atime/crtime fields). `uid`/`gid` are applied only through the opt-in
+   identity path; atime is preserved with `-U`/`--atimes`; crtime is captured
+   but cannot be set on the destination.
+4. **Config** — runtime parameters. Most cross the wire (TLS settings
+   excluded); `backup` and `backup_dir` are in the serialized wire table, while
+   `timeout`, `contimeout`, `quiet`, `stats`, `max_depth`, and `log_file` are
+   client-only.
+5. **Queue** — thread-safe bounded queue with condition variables.
+6. **DirectoryScanner** — recursive BFS traversal with exclude and include
+   pattern support, max-depth enforcement.

 ### Key Algorithms
-1. **File scanning** — BFS directory traversal;
-entries matched against exclude and include patterns,
-    max - depth enforced 2. * *Chunking ** — files accumulated until `chunk_size` threshold,
-    then flushed 3. *
-            *Compression ** — streaming zstd
-                via `ZSTD_compressStream2` / `ZSTD_decompressStream` 4. *
-            *Network protocol ** — status -
-        code - driven exchange with metadata packing,
-    keep - alive,
-    and abort support 5. * *Incremental check ** — client sends `STATUS_CHECK` + path + size +
-        mtime and,
-    with `--checksum`, XXH64 content checksum; server compares against destination. Can be batched via `STATUS_CHECK_BATCH` for reduced round-trips.
-6. **Bandwidth limiting** — token-bucket algorithm with `nanosleep` throttling on 64 KB write chunks
-7. **Metadata restoration** — `chmod()`, `chown()`, `utimensat()` on the receiving side
-8. **`--delete`** — sender tracks all sent paths;
-receiver walks destination tree and removes unlisted files / directories 9. *
-        *SSH transport *
-            * — `socketpair()` + `fork()` + `execvp("ssh",
-                                                    ...)` with `ControlMaster` and port support
-                                                10. *
-                                                *TLS transport ** — OpenSSL `SSL_CTX` with TLS
-                                                1.2 minimum,
-    mutual CA verification,
-    transparent `SSL_read`/`SSL_write` via `io_set_ssl()` 11. *
-        *Path traversal protection ** — `has_path_traversal()` rejects any file path
-         containing `..` components,
-    preventing directory escape attacks 12. *
-            *Connection limiting ** — server tracks active connections and rejects
-            new ones beyond `max_connections` (default 100)13. *
-            *Keep
-        - alive ** — idle connections receive periodic `STATUS_KEEPALIVE` to detect half
-        - open TCP connections 14. * *Abort handling ** — `SIGINT` sets an abort flag; the next protocol operation sends `STATUS_ABORT` for clean server cleanup
-15. **Atomic writes** — files are written to a `.tmp` suffix then atomically renamed via `rename()`, preventing partial files
-16. **Backup** — before overwriting, existing files are moved to `--backup-dir` (or same directory with `~` suffix) preserving the original
+
+1. **File scanning** — BFS directory traversal; entries matched against exclude
+   and include patterns, with max-depth enforced.
+2. **Chunking** — files accumulated until the `chunk_size` threshold (default
+   10 MiB) is reached, then flushed.
+3. **Compression** — streaming zstd via `ZSTD_compressStream2()` /
+   `ZSTD_decompressStream()`.
+4. **Network protocol** — status-code-driven exchange with metadata packing,
+   keep-alive, and abort support.
+5. **Incremental check** — the client sends `STATUS_CHECK` + path + size +
+   mtime and, with `--checksum`, a whole-file content checksum (xxHash64 by
+   default, or md5 via `--checksum-choice=md5`/`--cc`, seeded by
+   `--checksum-seed`); the server compares against the destination. Can be
+   batched via `STATUS_CHECK_BATCH` for reduced round-trips.
+6. **Bandwidth limiting** — token-bucket algorithm with sleep throttling on
+   64 KiB write chunks.
+7. **Metadata restoration** — mode via `chmod()`/`fchmod()`, times via
+   `utimensat()`/`futimens()`, and ownership only with an identity flag via
+   fd-relative `fchown()`/`fchownat()`.
+8. **`--delete`** — the sender tracks all sent paths; the receiver walks the
+   destination tree and removes unlisted files and directories.
+9. **SSH transport** — `socketpair()` + `fork()` + `execvp("ssh", ...)` with
+   `ControlMaster` and port support.
+10. **TLS transport** — OpenSSL `SSL_CTX` with TLS 1.2 minimum, mutual CA
+    verification, and transparent `SSL_read()`/`SSL_write()` via
+    `io_set_ssl()`.
+11. **Path traversal protection** — `has_path_traversal()` rejects any file
+    path containing `..` components, preventing directory escape attacks.
+12. **Connection limiting** — the server tracks active connections and rejects
+    new ones beyond `max_connections` (default 100).
+13. **Keep-alive** — idle connections receive periodic `STATUS_KEEPALIVE` to
+    detect half-open TCP connections.
+14. **Abort handling** — `SIGINT` sets an abort flag; the next protocol
+    operation sends `STATUS_ABORT` for clean server cleanup.
+15. **Atomic writes** — files are written to a `.tmp` suffix then atomically
+    renamed via `rename()`, preventing partial files.
+16. **Backup** — before overwriting, existing files are moved to `--backup-dir`
+    (or the same directory with a `~` suffix), preserving the original.

 ## Security Features

@@ -300,7 +367,8 @@ cmake --build build -j$(nproc)

 ### SSH transfer

-The remote host must have `fastsync-server` available in `PATH`, or use
+The remote host must have `fastsync-server` available in `PATH` (install or
+copy the built `./build/server` there as `fastsync-server`), or use
 `--fastsync-server-path`. SSH starts `fastsync-server --stdio` in its remote
 working directory, so use a destination below that directory unless the
 remote server is otherwise configured with a matching authorized root.
@@ -341,8 +409,13 @@ Plain TCP requires the explicit `--allow-unauthenticated` server option. Use TLS
 authenticated network connections.

 ### TLS transfer
+
+Server TLS requires `--cert`, `--key`, `--ca`, and `--client-cn`; the client
+requires `--cert`, `--key`, and `--ca`.
+
 ```bash
-./build/server --destination-root /path/to --tls --cert server.pem --key server-key.pem -p 8443
+./build/server --destination-root /path/to --tls --cert server.pem --key server-key.pem \
+  --ca ca.pem --client-cn client -p 8443
 ./build/client --tls --cert client.pem --key client-key.pem --ca ca.pem \
  --server-host example.com --server-port 8443 \
  --source-dir /path/to/source --dest-dir /path/to/destination \
@@ -355,32 +428,32 @@ These examples show the intended rsync-style workflow. Options marked as
 FastSync-native are optional performance or transport extensions.

 ```bash
-#Basic synchronization
+# Basic synchronization
 ./build/client /source/ /destination/

-#Archive - style synchronization(current FastSync archive behavior)
+# Archive-style synchronization (current FastSync archive behavior)
 ./build/client -a /source/ user@host:destination/

-#Preview a transfer without changing the destination
+# Preview a transfer without changing the destination
 ./build/client -n /source/ /destination/

-#Exclude temporary and object files
+# Exclude temporary and object files
 ./build/client --exclude '*.tmp' --exclude '*.o' \
  /source/ user@host:destination/

-#Remove destination entries not present in the source
+# Remove destination entries not present in the source
 ./build/client --delete /source/ user@host:destination/

-#Skip unchanged files using size and modification time
+# Skip unchanged files using size and modification time
 ./build/client --incremental /source/ user@host:destination/

-#Verify content when size and time are not sufficient
+# Verify content when size and time are not sufficient
 ./build/client --incremental --checksum /source/ user@host:destination/

-#Preserve supported mode and timestamp metadata
+# Preserve supported mode and timestamp metadata
 ./build/client --preserve /source/ user@host:destination/

-#Keep backups of overwritten destination files
+# Keep backups of overwritten destination files
 ./build/client --backup --backup-dir backups \
  /source/ user@host:destination/
 ```
@@ -421,7 +494,8 @@ is `--remote-option`, `-f` is `--filter`, `-s` is `--secluded-args`, `-p` is
 long-form-only or new shorts: multithreading is `-j`/`--threads`, metadata
 is `--preserve`, sendfile is `--sendfile`, chunk serialization is
 `--chunk-serialization`, timeout is `--timeout`, and SSH port is `--ssh-port`.
-`-a`/`--archive` is now real rsync archive (`-rlptgoD`).
+`-a`/`--archive` is now rsync archive `-rlptD`; owner/group (`-o`/`-g`) are not
+implied and remain opt-in via the identity flags.

 `--secluded-args` (and its short form `-s`) is accepted as a compatibility
 no-op. It does not change FastSync's transport or protocol behavior, because
@@ -433,8 +507,25 @@ remote SSH argv is already built injection-safe.

 | Option | Description |
 |---|---|
-| `-a`, `--archive` | rsync archive mode (`-rlptgoD`): links, metadata, devices and specials. |
-| `-n`, `--dry-run` | Scan and report without writing files. |
+| `-a`, `--archive` | rsync archive mode (`-rlptD`): links, metadata, devices and specials; owner/group (`-o`/`-g`) are not implied (opt in via the identity flags). |
+| `-n`, `--dry-run` | Report what would be transferred without mutating the destination. Since protocol 2.21.0 a server-routed target contacts the receiver and reports would-transfer based on receiver state; a plain local destination keeps the client-side scan. Never mutates or deletes. |
+| `--remove-source-files` | Remove regular source files after a successful transfer. |
+| `--incremental` | Skip files matching destination size and mtime. Auto-enables `--preserve`. Incompatible with `--chunk-serialization`. |
+| `--checksum` | Include xxHash64 content checks in incremental comparisons. |
+| `--size-only` | Skip incremental files matching in size, ignoring mtime. |
+| `-I, --ignore-times` | Transfer files even when size and mtime match. |
+| `-u, --update` | Skip files newer than the source on the receiver. |
+| `-W, --whole-file` | Transfer changed files without delta processing. |
+| `-d, --dirs` | Transfer the named directory entries without recursing into their contents (aliases `--old-dirs`/`--old-d`). |
+| `-R, --relative` | With `--files-from`, preserve each listed entry's relative path below the destination root. |
+| `--files-from <file>` | Read the source file list from FILE (paths relative to the source root). |
+| `--delay-updates` | Put updated files into place only at the end of the transfer. |
+| `--compare-dest <dir>` | Extra comparison basis: unchanged files are not transferred (requires/implies `--incremental`). |
+| `--copy-dest <dir>` | Like `--compare-dest`, but copies the unchanged file from DIR into the destination. |
+| `--link-dest <dir>` | Like `--copy-dest`, but hard-links the unchanged file from DIR (repeatable; earlier DIRs win). |
+| `--preallocate` | Allocate destination file space up front (fail-fast on a full disk). |
+| `--append` | Resume a shorter destination by appending only its tail (prefix not verified; requires `--incremental`). |
+| `--append-verify` | Like `--append`, but verifies the retained prefix checksum first (falls back to a full transfer on mismatch). |
 | `--delete` | Request removal of destination entries absent from the source. The server must allow deletion. Default timing is delete-after: extras are removed only after the whole transfer succeeded. |
 | `--delete-before` | Delete extras before the transfer starts (implies `--delete`). |
 | `--delete-during`, `--del` | Delete extras once the keep-set manifest is known, before data is applied (implies `--delete`; early mode, same engine behaviour as `--delete-before`). |
@@ -446,29 +537,49 @@ remote SSH argv is already built injection-safe.
 | `--include-from <file>` | Read include patterns from a file. |
 | `--max-size <bytes>` | Skip files larger than the limit. |
 | `--min-size <bytes>` | Skip files smaller than the limit. |
-| `--max-depth <n>` | Limit recursive scanning depth;
-zero means unlimited.| | `--incremental` | Skip files matching destination size and mtime.|
-    | `--checksum` | Include xxHash64 content checks in incremental comparisons.| | `--backup` |
-    Back up overwritten files.| | `--backup - dir<dir>` | Store backups under a separate directory.|
-| `--suffix<suffix>` | Set the backup filename suffix.| | `--partial` |
-     Select partial - transfer handling. On failed/interrupted writes the
-     already-written temp file is retained (best-effort) for resumption.|
-     With `--partial --partial-dir <dir>`, completed files are written under the
-     partial directory and installed atomically. | | `--partial - dir<dir>` |
-     Set a relative partial - transfer directory below the server destination root.
-     Use with `--partial`. |
+| `--max-alloc <SIZE>` | Maximum single allocation (binary units; default 1G). |
+| `--max-depth <n>` | Limit recursive scanning depth; zero means unlimited. |
+| `--backup` | Back up overwritten files. |
+| `--backup-dir <dir>` | Store backups under a separate directory (requires `--backup`). |
+| `--suffix <suffix>` | Set the backup filename suffix (default: `~`). |
+| `--partial` | Select partial-transfer handling. On failed/interrupted writes the already-written temp file is retained (best-effort) for resumption. With `--partial --partial-dir <dir>`, completed files are written under the partial directory and installed atomically. |
+| `--partial-dir <dir>` | Set a relative partial-transfer directory below the server destination root. Use with `--partial`. |
 | `--inplace` | Write directly to the destination instead of using a temporary file. |
+| `--fsync` | Fsync every written file before publication. |
+| `--write-batch=FILE` | Run the normal live transfer and also emit a self-contained batch file of the source tree. |
+| `--only-write-batch=FILE` | Emit the batch file only (no destination, no server). |
+| `--read-batch=FILE` | Apply a batch file to the destination (no source, no server). |
+| `--stop-after=MINS` | Stop the transfer after MINS minutes; whatever was already transferred is kept. |
+| `--stop-at=TIME` | Stop at an absolute time (`HH:MM`, `HH:MM:SS`, or `now+N[smhd]`). An early stop skips the late `--delete` keep-set. |

 ### Metadata and links

 | Option | Description |
 |---|---|
-| `--preserve` | Preserve supported file metadata, currently mode and modification time (long form only). |
-| `-l`, `--links` | Request symlink preservation;
-link-target transfer remains incomplete. |
+| `--preserve` | Preserve mode and mtime (long form only). Add `-U`/`--atimes` for atime, or an identity flag (`--chown`/`--usermap`/`--groupmap`/`--numeric-ids`/`--copy-as`) for owner/group. |
+| `-U`, `--atimes` | Preserve access times. Captured with the metadata payload; does not enable ownership. |
+| `-N`, `--crtimes` | Capture birth time and transmit it; it cannot be applied because no portable filesystem call can set a birth time (documented divergence). |
+| `-p`, `--perms` | Preserve permission bits (part of the metadata bundle). |
+| `-E`, `--executability` | Preserve executable permission bits. |
+| `-X`, `--xattrs` | Preserve user `user.*` extended attributes. |
+| `-A`, `--acls` | Preserve POSIX ACLs. |
+| `--chmod <changes>` | Modify transferred permissions (rsync syntax). |
+| `--chown=USER:GROUP` | Override the ownership of transferred files (`USER:GROUP`, `USER`, or `:GROUP`). |
+| `--usermap=MAP` | Map usernames when applying ownership (comma-separated `FROM:TO` rules). |
+| `--groupmap=MAP` | Map group names when applying ownership (same syntax as `--usermap`). |
+| `--numeric-ids` | Apply the source numeric uid/gid directly instead of mapping by name. |
+| `--copy-as=USER[:GROUP]` | Force every written entry to USER[:GROUP]; requires a privileged receiver. |
+| `--fake-super` | Record/replay effective metadata via a reserved `user.fastsync.stat` xattr. |
+| `--super` | Permit the receiver to attempt confined super-user activities (device nodes). |
+| `--no-super` | Forbid those super-user activities even when the receiver is root. |
+| `-l`, `--links` | Copy symlinks as symlinks; the target is transmitted and recreated under the receive root. |
 | `--copy-links` | Copy symlink referents. |
 | `--safe-links` | Skip symlinks that point outside the transfer tree. |
 | `--copy-unsafe-links` | Copy unsafe symlink referents. |
+| `-H`, `--hard-links` | Preserve hard-link relationships across the transfer. |
+| `-D` | Preserve device and special files (implies `--devices --specials`). |
+| `--devices` | Recreate device nodes on the destination (privileged; skipped without `CAP_MKNOD`). |
+| `--specials` | Recreate special files (FIFOs); sockets cannot be recreated. |
 | `-S`, `--sparse` | Sparse-file handling: receiver preserves holes (zero runs are written as holes; no wire change). |

 ### Output and logging
@@ -476,8 +587,12 @@ link-target transfer remains incomplete. |
 | Option | Description |
 |---|---|
 | `-v`, `--verbose` | Enable debug logging. |
+| `-q`, `--quiet` | Suppress non-error output. |
 | `--progress` | Show live transfer progress. |
 | `--stats` | Print transfer statistics. |
+| `-i`, `--itemize-changes` | Print an rsync-style per-file change line. |
+| `--out-format=FORMAT` | Output format for changed files (`%f %n %l %b %M %%`). |
+| `--list-only` | List source files instead of transferring. |
 | `--log-file <path>` | Write log output to a file. |
 | `-V`, `--version` | Print the FastSync protocol version. |
 | `--help` | Print command usage. |
@@ -487,31 +602,52 @@ link-target transfer remains incomplete. |
 | Option | Description |
 |---|---|
 | `--ssh-port <port>` | SSH port for the SSH transport (default: 22). Note the short `-p` is now rsync's `--perms`. |
+| `-e`, `--rsh <command>` | Remote shell to launch for the SSH transport (default: `ssh`; may include arguments). |
 | `--fastsync-server-path <path>` | Remote FastSync server path for SSH mode. |
+| `-M`, `--remote-option=OPT` | Append OPT to the remote server invocation over SSH (repeatable). |
 | `--source-dir <path>` | Set the source directory explicitly. |
 | `--dest-dir <path>` | Set the destination directory explicitly. |
 | `--save-to-disk` | Enable server-side disk persistence. |
 | `--server-host <host>` | TCP server address. |
 | `--server-port <port>` | TCP server port. `--port <port>` / `--port=<port>` is an alias. |
-| `--tls` | Enable TLS. Requires `--cert` and `--key`. |
+| `--address <ip>` | Bind the outgoing client socket to this source address. |
+| `-4`, `--ipv4` | Force IPv4 for destination resolution. |
+| `-6`, `--ipv6` | Force IPv6 for destination resolution. |
+| `--sockopts=OPTS` | Comma-separated OPT=VAL socket options applied before connect. |
+| `--tls` | Enable TLS. Requires `--cert`, `--key`, and `--ca`. |
 | `--cert <path>` | TLS certificate file. |
 | `--key <path>` | TLS private key file. |
-| `--ca <path>` | CA file for peer verification. |
+| `--ca <path>` | CA file for peer verification (always required with `--tls`). |

 ## Server Options

 | Option | Description |
 |---|---|
 | `--stdio` | Serve one SSH connection over standard input/output. |
-| `-p <port>` | TCP listen port. |
+| `--daemon` | Run as a persistent daemon listener using a module config file; the daemon default port is 873 (unlike `-p`, which defaults to 8080). |
+| `--config=FILE` | Daemon config file (default: `~/.config/fastsync/fastsyncd.conf`, else `/etc/fastsyncd.conf`). Requires `--daemon`. |
+| `--dparam=KEY=VALUE` | Override one global config key on the command line. Requires `--daemon`. |
+| `--no-detach` | Stay in the foreground (default detaches to the background when running `--daemon`). |
+| `-p <port>` | TCP listen port (default: 8080, range: 1–65535). |
 | `--tls` | Enable TLS. |
-| `--cert <path>` | TLS certificate file. |
-| `--key <path>` | TLS private key file. |
-| `--ca <path>` | CA file for peer verification. |
-| `--destination-root <path>` | Confine received files to this server-side root;
-defaults to the current directory. |
+| `--cert <path>` | TLS certificate file (PEM). |
+| `--key <path>` | TLS private key file (PEM). |
+| `--ca <path>` | CA file for peer verification (PEM). |
+| `--client-cn <name>` | TLS client certificate CN; mandatory with `--tls` (the server verifies the client CN). |
+| `--destination-root <path>` | Confine received files to this server-side root; defaults to the current directory. |
+| `--address <addr>` | Bind the listening socket to this address. |
+| `-4`, `--ipv4` | Bind an IPv4 socket (default). |
+| `-6`, `--ipv6` | Bind an IPv6 socket. |
 | `--allow-delete` | Permit client delete manifests. Deletion is refused by default. This also gates `--force` (which can recursively replace/remove a destination directory tree). |
 | `--allow-super` | Standalone TCP listener only: keep super-user activities enabled for a **root** receiver. Without it a root standalone server forces `SUPER_MODE_OFF`, so client `--devices`/`--write-devices`/`--super` and client-chosen ownership requests are skipped/refused. Rejected with `--stdio` (the SSH remote argv is client-composed; use a forced command if the default must hold). No effect when not root. Daemon modules opt in per module with `client owner = yes`. |
+| `--trust-sender` | Trust the remote sender's file list: skip the receiver's up-front path-traversal and escaping-symlink-target containment re-validation (fewer checks, faster, potentially unsafe; off by default). |
+| `--no-super` | Operator veto: never attempt super-user activities (ownership, device nodes) even as root, and refuse any client `--copy-as`/`--super` request. |
+| `--allow-unauthenticated` | Permit plaintext/anonymous network clients; an auth-required module still accepts only opted-in loopback plaintext. |
+| `--iconv=LOCAL[,REMOTE]` | Declare this server's LOCAL charset for file-name conversion. |
+| `--password-file=FILE` | Credential store for modules that declare `auth users`. Requires `--daemon`. |
+| `--early-input=FILE` | Second credential store layered over `--password-file`. Requires `--daemon`. |
+| `--hash-credentials <file>` | Read `<file>`'s `user:password` lines and print PBKDF2 credential-store lines to stdout, then exit. Cannot be combined with `--daemon` or `--stdio`. |
+| `--iterations N` | PBKDF2 iteration count for `--hash-credentials` (default 600000, range 100000–10000000). Requires `--hash-credentials`. |
 | `-v`, `--verbose` | Enable debug logging. |
 | `--help` | Print server usage. |

@@ -540,8 +676,9 @@ and `address`, the global section accepts:
 - `hosts allow` / `hosts deny` — comma- and/or whitespace-separated host access
  patterns.

-A `[module]` may also set `max connections` (0 = unlimited; enforced per module
-across all connection children) and its own `hosts allow`/`hosts deny`.
+A `[module]` requires `path`, and may also set `read only`, `client owner`,
+`auth users`, `max connections` (0 = unlimited; enforced per module across all
+connection children), and its own `hosts allow`/`hosts deny`.

 The per-host cap and the shared auth lockout identify a source by its numeric
 peer IP. **Loopback peers (127.0.0.0/8, IPv6 `::1`) are exempt**: every local
@@ -647,10 +784,9 @@ mandates `--client-cn`, so a TLS connection to an auth-required module always
 has its client CN verified (`--client-cn` matches the certificate's CN only, not
 a subjectAltName, which is acceptable for a private CA).

-TLS provides encrypted TCP transport. Supplying `--ca` enables certificate
-verification; without it, traffic is encrypted but peer identity is not
-verified. Use certificate verification for deployments where authentication
-matters. The default TCP transport is not encrypted.
+TLS provides encrypted TCP transport. Both the client and the server require
+`--ca` together with `--tls`, so peer certificates are always verified
+(`SSL_VERIFY_PEER`, depth 4). The default TCP transport is not encrypted.

 The receiver protects its destination root with path validation, `openat()`
 directory traversal, `O_NOFOLLOW`, temporary files, and atomic renames. Delete
@@ -664,10 +800,12 @@ The project will reach the drop-in replacement goal in stages:
   and `--option=value` syntax.
 2. Add differential tests that compare FastSync and rsync contents, metadata,
   links, deletes, filters, dry runs, and exit codes.
-3. Make `-a` implement the expected recursive, links, permissions, times,
-   owner/group, and supported special-file behavior.
-4. Complete symlink, sparse-file, metadata, delete-policy, and resumable-write
-   semantics.
+3. `-a` now implements the expected recursive, links, permissions, times, and
+   supported device/special-file behavior; owner/group (`-o`/`-g`) stay opt-in
+   via the identity flags, and remaining work is the documented
+   device/special-file divergences.
+4. Symlink, sparse-file, metadata, delete-policy, and resumable-write semantics
+   are implemented; remaining work is the documented edge cases.
 5. Add rsync remote-shell and daemon protocol interoperability.
 6. Keep FastSync performance options as negotiated, optional extensions.

@@ -709,10 +847,13 @@ rsync protocol or filesystem-semantic compatibility.

 ## Performance Guidance

- Use `-m` for workloads with many files or enough CPU parallelism.
- Use `-c` or `-z` when network bandwidth is more constrained than CPU.
+- Use `-j`/`--threads` for workloads with many files or enough CPU parallelism
+  (`-m` is `--prune-empty-dirs`).
+- Use `-z` when network bandwidth is more constrained than CPU (`-c` is
+  `--checksum`, not a bandwidth option).
 - Tune `--chunk-size` for file sizes, memory limits, and network latency.
- Use `-f` for large uncompressed TCP transfers where zero-copy I/O helps.
+- Use `--sendfile` for large uncompressed TCP transfers where zero-copy I/O
+  helps (`-f` is `--filter`).
 - Use `--incremental` to avoid retransmitting unchanged files.
 - Use `--delta` for changed files when both endpoints are FastSync peers.
 - Use `--bwlimit` when sharing a link with other traffic.