812 lines
103 KiB
Markdown
812 lines
103 KiB
Markdown
# Rsync Feature Compatibility
|
||
|
||
This document maps rsync's full feature set to FastSync's current implementation status.
|
||
|
||
## Summary
|
||
|
||
| Status | Count | Description |
|
||
|--------|-------|-------------|
|
||
| ✅ Implemented | 101 | Feature works end-to-end |
|
||
| 🔀 Alt Arg | 3 | Functionality exists but under different flag/semantics |
|
||
| ⚠️ Partial | 10 | Flag parsed/stored but behavior incomplete |
|
||
| 🔄 Compatibility No-op | 3 | Flag is accepted for CLI compatibility but has no effect |
|
||
| ❌ Not Implemented | 30 | Flag not recognized or no behavior |
|
||
| **Total** | **147** | |
|
||
|
||
---
|
||
|
||
## 1. General Options
|
||
|
||
| Flag | Rsync Description | FastSync Status | Notes |
|
||
|------|-------------------|-----------------|-------|
|
||
| `-a`, `--archive` | Archive mode is -rlptgoD | 🔀 Alt Arg | Maps to -c -m -M (compression + multithread + metadata) |
|
||
| `-v`, `--verbose` | Increase verbosity | ✅ Implemented | Sets `log_level=DEBUG` |
|
||
| `-q`, `--quiet` | Suppress non-error messages | ✅ Implemented | Suppresses client output while preserving errors |
|
||
| `--help` | Show help | ✅ Implemented | Prints usage and exits; `-h` is not accepted |
|
||
| `-V`, `--version` | Print version | ✅ Implemented | |
|
||
| `--info=FLAGS` | Fine-grained info verbosity | ✅ Implemented | Supports `copy`, `misc`, `skip`, `stats`, `all`, and `none`; explicit flags override `--verbose`, and `none` suppresses info output; unsupported names are rejected |
|
||
| `--debug=FLAGS` | Fine-grained debug verbosity | ✅ Implemented | `io`, `proto`, `pack`, and `util` are supported; `--debug=help` lists flags; other rsync categories are rejected |
|
||
| `--stderr=MODE` | Change stderr output mode | ⚠️ Partial | `errors` (default) and `all` are supported; `client` is rejected because FastSync has no rsync message channel |
|
||
| `--no-motd` | Suppress daemon MOTD | ❌ Not Implemented | |
|
||
| `--exclude=PATTERN` | Exclude files matching pattern | ✅ Implemented | Glob matching in scanner |
|
||
| `--include=PATTERN` | Include files matching pattern | ✅ Implemented | Glob matching in scanner |
|
||
| `-C`, `--cvs-exclude` | Auto-ignore CVS files | ✅ Implemented | Applies the well-known rsync default exclude set as exclude rules during scanning (RCS SCCS CVS CVS.adm RCSLOG cvslog.* tags TAGS .make.state .nse_depinfo *~ #* .#* ,* _$* *$ *.old *.bak *.BAK *.orig *.rej .del-* *.a *.olb *.o *.obj *.so *.exe *.Z *.elc *.ln core .svn/ .git/ .hg/ .bzr/); `.git/`-style repo dirs are pruned without descending |
|
||
|
||
## 2. Modifying Output
|
||
|
||
| Flag | Rsync Description | FastSync Status | Notes |
|
||
|------|-------------------|-----------------|-------|
|
||
| `--stats` | Give transfer stats | ✅ Implemented | Prints file/byte counts |
|
||
| `-h`, `--human-readable` | Human-readable numbers | ✅ Implemented | Formats transfer byte sizes using binary units |
|
||
| `-i`, `--itemize-changes` | Per-file change summary | ✅ Implemented | Prints rsync-style `>f+++++++++` lines to stdout only for files actually sent (also under `-m`); unchanged files print nothing, matching single-`-i` behavior |
|
||
| `--progress` | Show progress | ✅ Implemented | Progress callback in sender |
|
||
| `-P` | Same as --partial --progress | ⚠️ Partial | Parses and enables progress, but interrupted files are not retained for resumable transfers |
|
||
| `--out-format=FORMAT` | Custom output format | ✅ Implemented | Per-transfer template on stdout; tokens `%f` `%n` `%l` `%b` `%M` `%%` (`%b` is the source length, always `== %l`; post-compression/delta wire bytes are not counted); unknown escapes preserved |
|
||
| `--log-file=FILE` | Log to file | ✅ Implemented | `log_file` config field |
|
||
| `--log-file-format=FMT` | Log format | ✅ Implemented | Requires `--log-file`; writes one template line per transferred file using the same token set as `--out-format` (including `%b` `==` source length) |
|
||
| `--8-bit-output`, `-8` | Leave high-bit chars unescaped | ✅ Implemented | Applies to displayed paths and protocol debug output |
|
||
| `--list-only` | List files instead of copying | ✅ Implemented | `ls -l`-style listing of files that would be transferred; scans the source only, contacts no server, writes nothing; also works with `-n` |
|
||
|
||
## 3. File Selection
|
||
|
||
| Flag | Rsync Description | FastSync Status | Notes |
|
||
|------|-------------------|-----------------|-------|
|
||
| `--exclude-from=FILE` | Read exclude patterns from file | ✅ Implemented | Reads patterns from file |
|
||
| `--include-from=FILE` | Read include patterns from file | ✅ Implemented | Reads patterns from file |
|
||
| `--filter=RULE` | Add file-filtering rule | ✅ Implemented | Long option only: rsync's short `-f` conflicts with FastSync sendfile (see FastSync-specific list), so `-f` is not reassigned. Supported subset: `+`/`-` include/exclude, implicit-exclude patterns, `include`/`exclude` word forms, a leading `/` anchor (to the transfer root, or to a `.rsync-filter` file's directory), and a trailing `/` for dir-only rules; first match wins with a default of include inside the filter layer. Filters are an independent layer from `--exclude`/`--include` (an entry must pass both). Rejected with a clear error (no silent no-ops): `merge`/`dir-merge`/`hide`/`show`/`protect`/`risk`/`clear` words, rules that begin with `:`/`.`/`!` (merge/dir-merge/list-clear shorthands), and include/exclude modifiers other than `/` (`! C s r p x`) |
|
||
| `--files-from=FILE` | Read source file list from file | ✅ Implemented | Entries are paths relative to the source root (leading `./` stripped, `..`/absolute entries rejected at parse time, blank lines ignored; NUL-delimited with `-0`). A listed regular file is transferred; a listed directory transfers its whole subtree (FastSync recursion is always on, unlike rsync's non-recursive default). Non-listed paths and their subtrees are pruned by the scanner. A listed entry that does not exist under the source (and an empty list) is a hard error reported before any transfer, unless `--ignore-missing-args` / `--delete-missing-args` is given (see the Safety & Security rows): those flags downgrade the listed-but-missing case to a skip and, for `--delete-missing-args`, a destination deletion; an empty list stays a hard error in every mode. Listing `.` (whole tree) and empty listed directories are fine. Scalability note: `file_list_affects` is O(list size) per scanned entry, so a very large `--files-from` list against a huge tree is quadratic; lists are typically small enough that this is acceptable, but it is the documented bound. The delete manifest still derives from what was actually sent, so `--delete` stays consistent with the subset |
|
||
| `-0`, `--from0` | Delimit *-from files with NULs | ✅ Implemented | `--files-from` entries become NUL-delimited; the flag may appear before or after `--files-from` on the command line. NUL mode preserves entry bytes exactly (trailing CR/LF are part of the name; only newline mode trims them) |
|
||
| `--max-size=SIZE` | Skip files larger than SIZE | ✅ Implemented | `max_size` in scanner |
|
||
| `--min-size=SIZE` | Skip files smaller than SIZE | ✅ Implemented | `min_size` in scanner |
|
||
| `-I`, `--ignore-times` | Don't skip files matching size+time | ❌ Not Implemented | |
|
||
| `--size-only` | Skip based on size only | ✅ Implemented | With `--incremental`, ignores mtime |
|
||
| `-@`, `--modify-window=NUM` | Mod-time comparison accuracy | ✅ Implemented | Whole-second tolerance with nanosecond-aware comparisons |
|
||
| `--existing` | Skip creating new files on receiver | ✅ Implemented | Existing destination files continue through normal update handling |
|
||
| `--ignore-existing` | Skip updating existing files | ❌ Not Implemented | |
|
||
| `--remove-source-files` | Sender removes regular files after confirmed transfer | ✅ Implemented | |
|
||
| `-x`, `--one-file-system` | Do not cross filesystem boundaries | ✅ Implemented | Sender scanner captures the root device and skips descending into mount-point crossings (`st_dev` differs); cross-filesystem mount-point subdirectories are dropped entirely, matching rsync |
|
||
| `-F` | Add the default `.rsync-filter` rules | ✅ Implemented | Reads one filter rule per line from each directory's `.rsync-filter` file during traversal and applies it to that directory's subtree; the current directory's rules are evaluated before its ancestors', so deeper files override shallower ones and per-directory files override the command-line `--filter`/`-C` base by default (matching rsync's first-match-wins precedence); `.rsync-filter` files are never transferred. The rsync `-FF` behavior (also `.cvsignore`) is out of scope; unsupported rule types inside the file abort with a clear error |
|
||
|
||
## 4. Directory Options
|
||
|
||
| Flag | Rsync Description | FastSync Status | Notes |
|
||
|------|-------------------|-----------------|-------|
|
||
| `-r`, `--recursive` | Recurse into directories | ✅ Implemented | Default behavior |
|
||
| `-R`, `--relative` | Use relative path names | ✅ Implemented | Meaningful together with `--files-from` (FastSync's default full-tree scan always mirrors the full source argument path below the destination root, so -R does not change it). With `-R` + `--files-from` each listed entry is transmitted under its bare relative destination path: an entry `sub/x.txt` lands at `<dest>/sub/x.txt` (its leading components preserved) instead of under the `<dest>/<full source path>` mirror. Only the path sent on the wire changes; the client still reads the absolute source path, and the delete manifest derives from the sent (relative) paths so `--delete` and `--remove-source-files` stay consistent in both layouts. Works single-threaded and under `-m` (including chunk serialization) |
|
||
| `--no-implied-dirs` | Don't send implied dirs with -R | ✅ Implemented | Client-side, meaningful only with `-R` + `--files-from`. rsync would normally create the ancestor directories implied by a listed file so it can be written; with `--no-implied-dirs` a listed file whose parent directory is not itself (or via an ancestor) explicitly listed cannot be placed, and FastSync fails the whole run up front with a clear error (`--no-implied-dirs: cannot place file '...': parent directory '...' is not explicitly listed`). Listing the directory (or an ancestor of it, or the whole tree `.`) permits the file. In every other mode the option has no effect. FastSync has no per-entry skip channel, so the rsync "omit the file" case is surfaced as a hard pre-transfer error |
|
||
| `-d`, `--dirs`, `--old-dirs`, `--old-d` | Transfer dirs without recursing | ✅ Implemented | `-d <dir>` transmits an explicit directory entry for the source-root directory, so the destination mirror is created empty and nothing is descended into. With `--files-from` exactly the listed items are transferred: a listed directory is created empty (no descent) and a listed file is transferred with its content; the dest layout follows the same -R rules as plain files. A new wire frame (`STATUS_MKDIR`) carries each directory entry (path only); the receiver creates it with the same confined mkdir-parent semantics as regular writes, in single-threaded and `-m` receivers (chunk serialization carries a per-entry type marker). Directory entries appear in the delete manifest so `--delete` prunes correctly. FastSync divergences: directory mtimes/modes are not transmitted, filter/`--exclude` rules are not re-applied to the listed dirs mode (there is no descent during which they would apply), and `-d` never creates the intermediate directories between the destination root and a listed file beyond the usual on-demand parent creation. Under `--delay-updates` only regular files are staged: directory entries are created immediately, so a delayed run that fails part way can leave the already-created empty directories behind (matching rsync, which also creates directories as it processes the file list and only delays regular-file data) |
|
||
| `--mkpath` | Create missing path components | ✅ Implemented | Wire option (client → server). At connection start the server creates the client's destination root directory (and any missing leading components below its own authorized root) when `--mkpath` is set, failing the connection cleanly if it cannot. Without `--mkpath` a destination root that does not exist yet is rejected up front (rsync semantics), so the flag is the only way to transfer into a not-yet-created destination directory. Creation is confined by the same secure mkdir walk as file writes (`O_NOFOLLOW`, no `..`) |
|
||
|
||
## 5. Transfer Modifications
|
||
|
||
| Flag | Rsync Description | FastSync Status | Notes |
|
||
|------|-------------------|-----------------|-------|
|
||
| `-u`, `--update` | Skip files newer on receiver | ❌ Not Implemented | Removed because it had no effect |
|
||
| `--inplace` | Update files in-place | ✅ Implemented | Direct write mode |
|
||
| `--append` | Append data to shorter files | ✅ Implemented | Tail-only resume. When an existing destination file is SHORTER than the source, the receiver negotiates a resume offset with the sender and only the tail is transferred; the receiver rebuilds the full file (retained prefix + tail) and installs it through the normal atomic store path, so the result is byte-identical to the source whenever the retained prefix matches. Plain `--append` does NOT content-verify that prefix (rsync parity): a destination whose prefix differs from the source is resumed anyway, so the result (wrong prefix + correct tail) is NOT byte-identical and the file is effectively left corrupt — the documented rsync-parity risk (use `--append-verify` when the prefix cannot be trusted). Non-content attributes (permissions/ownership/mtime, via `-M`) are still applied. Requires the per-file `STATUS_CHECK` handshake, so it implies `--incremental`; it takes precedence over block delta for a growing file and falls back to delta/full when the destination is not shorter. Incompatible with `-s` (chunk serialization) and `--whole-file` (both rejected up front so the mode never silently degrades to a full transfer). Combines with `--inplace`, `--partial`/`--partial-dir`, and `--delay-updates` (the reconstructed full file flows through those paths unchanged). Divergence: rsync appends in place; FastSync reconstructs and atomically installs, so an interrupted or failed resume never leaves a half-written file at the destination (no corruption window), and `--append` is thus safe to use with the normal atomic path — not only with in-place writes |
|
||
| `--append-verify` | Append with old-data checksum | ✅ Implemented | Like `--append`, but the retained prefix IS verified before resuming: the sender transmits the source prefix checksum and the receiver compares it to the xxHash64 of the retained destination prefix; on a match only the tail is transferred, on a MISMATCH the run falls back to a clean full transfer so the result is always a byte-identical source copy (never a corrupt prefix+tail blend). Wire/protocol: the append handshake adds `STATUS_APPEND` / `STATUS_APPEND_SIG` / `STATUS_APPEND_OK` / `STATUS_APPEND_DATA` frames and `PROTOCOL_VERSION` was bumped **2.9.0 → 2.10.0** (peers must match, and both must be 2.10.0 or the run fails the version check). Same implications/incompatibilities as `--append`; when both spellings are given `--append-verify` wins (the safer semantics). See the Phase-3 append notes below |
|
||
| `-W`, `--whole-file` | Copy whole file (no delta) | ❌ Not Implemented | |
|
||
| `--block-size=SIZE` | Force checksum block-size | ⚠️ Partial | Parsed as `--delta-block`; controls delta transfer block size |
|
||
|
||
## 6. Destination Handling
|
||
|
||
| Flag | Rsync Description | FastSync Status | Notes |
|
||
|------|-------------------|-----------------|-------|
|
||
| `-n`, `--dry-run` | Trial run with no changes | ✅ Implemented | `dry_run` config field |
|
||
| `-b`, `--backup` | Make backups of overwritten files | ✅ Implemented | Backup before overwrite |
|
||
| `--backup-dir=DIR` | Backup directory hierarchy | ✅ Implemented | `backup_dir` config field |
|
||
| `--suffix=SUFFIX` | Backup suffix (default ~) | ✅ Implemented | `suffix` config field |
|
||
| `--delay-updates` | Put updated files in place at end | ✅ Implemented | Successfully received files are staged under a private 0700 `.fastsync-stage` dir inside the receive root and atomically renamed into their final destinations only after the whole transfer (manifest/delete handling included) succeeds, just before the success/outcome frame is sent. The delete walker deliberately skips the staging dir at the receive root, so `--delete` removes genuine extras but never the staged files (deletion runs before publication; rsync's delete-after ordering is not implemented). `--existing`/`--ignore-existing`/`--update` decide against the final destination path at stage time; `--backup` moves the old file aside at publication. Incompatible with `--inplace` and with `--backup-dir=.fastsync-stage` (the internal staging name is reserved; both are rejected). The staging dir name is fixed, so two simultaneous delayed transfers to the same destination root are serialized with an exclusive advisory lock held for the whole transfer: the second session fails cleanly instead of corrupting the first. Aborting or failing before publication installs nothing and removes the staging tree; a crash between stage and publish leaves staged leftovers that the next delayed run wipes at start (process death releases the lock). A stage→publish failure aborts the transfer (best-effort cleanup of the not-yet-published staged files; already-published files are not rolled back). Works in single-threaded and `-m` modes |
|
||
| `-T`, `--temp-dir=DIR` | Create temporary files in DIR | ✅ Implemented | `--temp-dir` only; `-T` stays FastSync's `--timeout` alias. Scratch dir is resolved under the receive root; temp copies use a unique name there and are atomically renamed into place. If the scratch dir and destination are on different filesystems the atomic rename fails with EXDEV and the file save fails, which aborts the whole transfer (FastSync has no per-file skip/resume on a save error; rsync's non-atomic copy fallback is deliberately not used). `--inplace` and `--partial-dir` writes bypass the scratch dir |
|
||
|
||
## 7. Deletion
|
||
|
||
| Flag | Rsync Description | FastSync Status | Notes |
|
||
|------|-------------------|-----------------|-------|
|
||
| `--delete` | Delete extraneous files from dest | ✅ Implemented | `use_delete` config field. Deletion is always derived from the transmitted keep-set manifest of the paths the sender sent/keeps (never from unchecked input), runs through the symlink-safe walker bounded by `MAX_SERVER_DELETE_COUNT`, and skips the `.fastsync-stage` staging dir under `--delay-updates`. FastSync's default timing when no timing flag is given is **delete-after** (extras are removed only once the whole transfer succeeded) — intentionally NOT rsync's `--del`/delete-during default, to preserve FastSync's commit-style safety. By default the destination mirror of a path the source scan pruned (filter/exclude/size rules) is **protected** from deletion — matching rsync, which does not delete excluded files under `--delete`; `--delete-excluded` opts back into deleting them (see below). The bounded deletion is **all-or-nothing**: if the destination holds more extras than the effective bound (a client `--max-delete=NUM` or the 100000-entry server bound) nothing is deleted and the run fails with a distinct error instead of silently truncating |
|
||
| `--delete-before` | Delete before transfer | ✅ Implemented | Implies `--delete`. The sender runs a full source pre-scan (paths only) and transmits the keep-set manifest BEFORE any file data; the receiver validates it, removes every destination entry not listed (all-or-nothing bounded walk, staging-dir skip, protected prefixes honored), then acks `STATUS_OK`. The sender only starts streaming after the deletion committed, or aborts if the receiver reported a deletion error. By definition the deletions already happened when a later transfer phase fails — rsync's delete-before is destructive the same way; a subsequent failure does not restore the removed files. Divergence: the keep-set is the pre-scan snapshot, so a file that appears on the source between the pre-scan and the data pass is still transferred but was not protected from deletion |
|
||
| `--del`, `--delete-during` | Delete during transfer | ✅ Implemented | Both spellings accepted; imply `--delete`. FastSync streams the source in a single directory scan and has no per-directory generator pass, so deletions cannot be interleaved per-directory the way rsync's delete-during does. `--delete-during` therefore selects the same early engine mode as `--delete-before` (manifest transmitted before any data, extras removed and acknowledged before data is applied); observable success/failure behaviour equals `--delete-before`. That is the documented divergence from rsync, where `--del` is the default meaning of `--delete` |
|
||
| `--delete-delay` | Find deletions during, delete after | ✅ Implemented | Implies `--delete`. Commit-mode timing: extras are removed only after the whole transfer succeeded. rsync's delete-delay records the deletion list during its scan and applies it at the end; FastSync never snapshots the destination while data flows (the keep-set is the transmitted manifest and the destination is listed only at deletion time), so `--delete-delay` is implemented as the same end-of-transfer commit as `--delete-after` with identical safety. That is the documented divergence |
|
||
| `--delete-after` | Delete after transfer | ✅ Implemented | Implies `--delete`. The delete-after timing is also what plain `--delete` does: the keep-set manifest closes the data stream and the receiver commits the bounded deletion only after the terminal `STATUS_FINISHED` proves the whole transfer (every data frame received and stored) succeeded. A failed or aborted transfer removes nothing |
|
||
| `--delete-excluded` | Also delete excluded files | ✅ Implemented | `delete_excluded` config field. Under `--delete` FastSync now protects (rsync's default) the destination mirror of paths the sender's source scan pruned by user-selection rules — the `--filter`/`-F`/`-C` layer, the legacy `--exclude`/`--include` layer, and `--max-size`/`--min-size`. The sender transmits those concrete pruned paths as **protected prefixes** in the delete-manifest frame (see the Phase-3 notes below); the walker never descends into or removes them. `--delete-excluded` opts back in: the sender sends an empty protected list, so the excluded destination mirrors become ordinary extras and are removed. Divergences (documented): protection is derived only from what the source scan actually pruned — a stray destination-only file that happens to match an exclude rule is not protected (FastSync never re-applies rules to the destination, keeping deletion sender-derived), and `--files-from` subset pruning stays keep-set-only (an unlisted source path is treated as absent and its mirror is deletable, matching the `--files-from` delete note below). The two are orthogonal: `--delete-excluded` removes filter-excluded mirrors; it does not make `--files-from` prune things |
|
||
| `--max-delete=NUM` | Max files to delete | ✅ Implemented | `max_delete` config field (default -1 = no client limit; 0 = delete nothing). NUM bounds a `--delete` run with rsync's all-or-nothing semantics: the receiver rehearses the deletion first and, if the destination holds more than NUM extras, deletes NOTHING and fails the transfer with a distinct `--max-delete` error. A run at or below NUM deletes exactly the extras. NUM only applies together with `--delete` (it is inert otherwise, matching rsync). The hard server bound `MAX_SERVER_DELETE_COUNT` (100000) still caps the walk; a NUM above it never raises that cap, and exceeding the server bound is its own all-or-nothing error. Directories count toward the limit (each removed empty directory is one deletion), like rsync |
|
||
| `--ignore-errors` | Delete even with I/O errors | ✅ Implemented | Sender-side, client-only config field. rsync suppresses `--delete` when the transfer had I/O errors; FastSync's equivalent is a source-scan I/O error (an unreadable directory, e.g. EACCES): by default the scan aborts the run so no deletion happens. With `--ignore-errors` the scan continues past the unreadable directory, the readable tree is transferred and the deletion still runs (the mirror of the unreadable directory is treated as an extra). The run still exits non-zero (the error is reported, matching rsync's error status). Divergence: without the flag FastSync aborts the whole run on the scan error, whereas rsync transfers the rest of the tree and merely skips the deletion; both leave the deletion undone |
|
||
| `--force` | Force deletion of non-empty dirs | ✅ Implemented | `force_delete` receiver config field (crosses the wire). rsync's `--force` lets an incoming non-directory replace a destination directory; FastSync implements exactly that: when a regular file is written to a path that is currently a (possibly non-empty) destination directory, `--force` removes that directory tree first — confined to the receive root and symlink-safe (O_NOFOLLOW fd walk, symlinks removed by name, never followed) — so the atomic install can place the file. Without `--force` such a write fails and the run aborts. Divergence: `--force` acts on the immediate-install path only; under `--delay-updates` a blocking directory is not cleared (publication renames over regular files) |
|
||
| `--prune-empty-dirs` | Prune empty dir chains | ✅ Implemented | Long-only: FastSync's `-m` is already multithreading (recorded divergence — rsync's `-m` short form is not reassigned). FastSync's recursive transfer never emits directory entries, so empty directories are inherently never transferred (which is rsync's `-m` behavior) and truly-empty destination directory chains are removed by `--delete` regardless of this flag. The flag's additional real effect is on the `--dirs` explicit directory-entry generator: a plain `-d <empty-dir>` run omits the empty source directory's entry, so nothing is created at the destination (no `STATUS_MKDIR`, no `-i`/`--out-format` change line, and an existing empty mirror becomes an extra that `--delete` prunes). Explicitly `--files-from`-listed directories always pass through (documented `--files-from` behavior). A directory that still holds an excluded-but-protected file survives, matching the `--delete-excluded` default |
|
||
|
||
**Deletion-timing implementation notes (Phase 3):** the delete flags above are
|
||
real. Two new config booleans (`delete_during`, `delete_delay`) join the already
|
||
serialized `delete_before`/`delete_after`, so the on-the-wire config layout
|
||
changed and `PROTOCOL_VERSION` was bumped **2.7.0 → 2.8.0** (peers must match).
|
||
The `STATUS_MANIFEST` frame is count-delimited and position-independent: the
|
||
receiver commits the deletion either when the manifest arrives (early modes:
|
||
`--delete-before`/`--delete-during`, which additionally acknowledge with
|
||
`STATUS_OK` before data flows) or after the terminal `STATUS_FINISHED` proves
|
||
the whole transfer succeeded (commit modes: plain `--delete`/`--delete-after`/
|
||
`--delete-delay`). Timing is chosen purely from the config, so server policy
|
||
(`--allow-delete` off) still disables deletion without deadlocking the early
|
||
manifest ack. `--delete-delay` and `--delete-during` are each implemented as
|
||
the closest safe approximation their engine mode allows; the divergences are
|
||
noted in the rows above.
|
||
|
||
**Deletion-policy notes (Phase 3, delete-policy wave):** this wave made the
|
||
deletion family real — `--delete-excluded`, `--max-delete`, `--ignore-errors`,
|
||
`--force`, `--prune-empty-dirs` — and, to support them, the `STATUS_MANIFEST`
|
||
frame now carries **two sections**: the keep-set paths followed by a list of
|
||
**protected prefixes** (destination-relative paths the source scan pruned by
|
||
user-selection rules, which the walker must never delete unless
|
||
`--delete-excluded` opted out). Two config booleans were added for the wave:
|
||
`force_delete` (crosses the wire; the receiver clears a directory that blocks an
|
||
incoming file) and `ignore_errors` (client-only; the sender's scan continues
|
||
past an unreadable directory). `max_delete`'s default became -1 ("no client
|
||
limit"). These wire/layout changes bumped `PROTOCOL_VERSION` **2.8.0 → 2.9.0**
|
||
(peers must match). All four wire additions — `force_delete`,
|
||
`delete_excluded`, `prune_empty_dirs`, `max_delete` — round-trip unchanged and
|
||
are validated on receive.
|
||
|
||
**Missing-args note (Phase 3, missing-args wave):** `--ignore-missing-args` and
|
||
`--delete-missing-args` are implemented as described in the Safety & Security
|
||
rows. Wire impact: the `STATUS_MANIFEST` frame now carries a **third section** —
|
||
a list of destination-relative **exact-delete paths** (the missing entries'
|
||
mirrors) — and the config frame gained a `delete_missing_args` boolean
|
||
(`ignore_missing_args` stays client-only, exactly like `ignore_errors`). These
|
||
wire/layout changes bumped `PROTOCOL_VERSION` **2.9.0 → 2.10.0** (peers must
|
||
match). The receiver validates the third section identically to the keep-set
|
||
(non-empty, relative, traversal-free; `MAX_MANIFEST_ENTRIES` per section, a
|
||
single `MAX_MANIFEST_BYTES` budget shared across all three). On commit the
|
||
receiver runs the exact-path deletions FIRST (`manifest_delete_missing_args`:
|
||
confined per-path unlink/rmdir, deep removal only under `--force`/`--delete`,
|
||
staging/basis protected, never blocked by the protected-prefix list) and then
|
||
the ordinary extras walk when `--delete` is active (`manifest_delete_all`). A
|
||
client may request the exact-path deletions without `--delete`; the server's
|
||
`--allow-delete` policy gates them exactly like `--delete`, so an unauthorized
|
||
server ignores the request while the missing entries are still skipped.
|
||
|
||
The deletion walker is now **all-or-nothing**: before any unlink it rehearses
|
||
the deletion (an fd-relative walk identical to the delete pass, counting every
|
||
regular file it would unlink and every directory it would remove) and refuses to
|
||
start when the extras exceed the effective bound — a client `--max-delete=NUM`
|
||
below the hard bound, or the hard `MAX_SERVER_DELETE_COUNT` (100000) bound
|
||
itself. Previously the walker removed up to `MAX_SERVER_DELETE_COUNT` extras and
|
||
then reported an error (a truncated deletion); it now removes nothing and fails
|
||
with an error naming the bound. Directories count toward the bound. A directory
|
||
that still holds entries the walker leaves in place (a protected excluded file,
|
||
a kept manifest entry, a symlink) is left behind rather than failing the run —
|
||
matching rsync's "cannot delete non-empty directory" behaviour. The
|
||
all-or-nothing guarantee holds only while the destination is not concurrently
|
||
modified: rehearsal and delete are two separate walks, so a concurrent change
|
||
between them (another process adding or removing destination entries) can make
|
||
the actual deletion diverge from the counted set.
|
||
|
||
Manifest size: the sender's keep-set and protected-prefix collections (streaming
|
||
or early pre-scan) are unbounded, but the receiver rejects a manifest beyond
|
||
`MAX_MANIFEST_ENTRIES` (1 048 576 entries, applied to EACH section — a frame can
|
||
therefore total up to 2 097 152 entries) / `MAX_MANIFEST_BYTES` (16 MB of paths,
|
||
counted across BOTH sections) as a hard protocol error. A heavily filtered
|
||
source whose exclusion list grows large thus fails the run cleanly on the
|
||
receiver (STATUS_ERROR) instead of being silently truncated. In the commit
|
||
modes this only means the deletion is refused after the data already arrived; in
|
||
the early modes (`--delete-before`/`--delete-during`) the manifest is the first
|
||
frame, so an oversized keep-set or protected list aborts the whole transfer
|
||
BEFORE any data is sent. Keep the source tree small enough for the receiver's
|
||
manifest caps when using the early timing.
|
||
|
||
Early-delete ACK wait: after committing a large deletion (up to
|
||
`MAX_SERVER_DELETE_COUNT` removals) the receiver's `STATUS_OK`/`STATUS_ERROR`
|
||
reply can legitimately take much longer than a normal round trip, so the sender
|
||
waits for that single ACK with an extended explicit deadline (1 hour) instead
|
||
of the default 60 s per-message receive window. A receiver that is genuinely
|
||
gone still aborts the wait via connection close/error; the extended bound only
|
||
protects against aborting after the deletion already committed on the receiver.
|
||
|
||
Flag-conflict policy: unlike rsync's last-one-wins behaviour, every deletion
|
||
timing flag implies `--delete`, and combining a timing flag with `--no-delete`
|
||
(in either argument order) — or more than one timing flag — is rejected as a
|
||
configuration error rather than silently resolved. Note the check is
|
||
order-independent because it runs over the fully parsed config. The deletion
|
||
POLICY flags (`--delete-excluded`, `--max-delete`, `--ignore-errors`, `--force`)
|
||
do NOT imply `--delete`; without `--delete` they are inert (matching rsync).
|
||
|
||
**Append-resume notes (Phase 3, append wave):** `--append` and `--append-verify`
|
||
are real. Both are negotiated when an existing destination file is found to be
|
||
**shorter** than the source during the per-file `STATUS_CHECK`; the receiver
|
||
replies with a new `STATUS_APPEND` frame carrying the resume offset (the prefix
|
||
length it already holds) instead of `STATUS_NEXT`/`STATUS_DELTA_SIGNATURE`.
|
||
The sender transmits ONLY the tail. For `--append-verify` it first sends the
|
||
source's prefix xxHash64 in a `STATUS_APPEND_SIG` frame; the receiver compares
|
||
it to the retained prefix and answers `STATUS_APPEND_OK` (transfer the tail) or
|
||
`STATUS_NEXT` (prefix mismatch → the sender falls back to a byte-exact full
|
||
transfer). The tail arrives in a `STATUS_APPEND_DATA` frame (compression and
|
||
metadata still apply). The receiver then rebuilds the full file in memory
|
||
(prefix + tail) and routes it through the existing atomic store engine, so all
|
||
of `--inplace`, `--partial`/`--partial-dir`, `--delay-updates`, `--backup`,
|
||
`--existing`/`--ignore-existing`/`--update` and delete-manifest behaviour is
|
||
unchanged and the result is a byte-identical source copy (given a matching
|
||
prefix). These new frames changed the wire, so `PROTOCOL_VERSION` was bumped
|
||
**2.9.0 → 2.10.0** (peers must match; the pre-existing `append`/`append_verify`
|
||
config booleans already crossed the wire). CLI: both flags imply `--incremental`
|
||
(the handshake needs it); they are incompatible with `-s` (chunk serialization)
|
||
and `--whole-file` (both rejected up front, never a silent full transfer); when
|
||
both spellings are given `--append-verify` wins. The FastSync divergence from
|
||
rsync is intentional and safer: rsync appends in place, whereas FastSync
|
||
reconstructs the whole file and atomically installs it, so an interrupted or
|
||
failed resume never leaves a partial/corrupt file at the destination — this is
|
||
why plain `--append` works on the normal atomic path, not only with `--inplace`.
|
||
|
||
## 8. Metadata Preservation
|
||
|
||
| Flag | Rsync Description | FastSync Status | Notes |
|
||
|------|-------------------|-----------------|-------|
|
||
| `-M`, `--preserve` | Preserve file metadata | ✅ Implemented | Mode, uid, gid, mtime |
|
||
| `-p`, `--perms` | Preserve permissions | 🔀 Alt Arg | `-p` means SSH port; permissions preserved via `-M`/`--preserve` |
|
||
| `-o`, `--owner` | Preserve owner | ✅ Implemented | Part of -M |
|
||
| `-g`, `--group` | Preserve group | ✅ Implemented | Part of -M |
|
||
| `-t`, `--times` | Preserve modification times | ✅ Implemented | Part of -M |
|
||
| `-E`, `--executability` | Preserve executability | ✅ Implemented | Preserves executable permission bits (implies metadata preservation) |
|
||
| `--chmod=CHMOD` | Affect file permissions | ✅ Implemented | Supports numeric and symbolic `ugo` `rwx` changes; retains receiver safety masking |
|
||
| `-A`, `--acls` | Preserve ACLs | ✅ Implemented | Implemented on Linux via the POSIX-ACL xattr representation: the sender captures the `system.posix_acl_access` / `system.posix_acl_default` xattrs into the same bounded whitelisted set as `-X`, transmits them per-file, and the receiver re-applies them fd-relative. Setting an ACL the receiver is not permitted to set (non-root on a file it does not own, unsupported filesystem) is logged and skipped, never fatal. libacl is **not** required. Only the `system.posix_acl_*` namespaces plus `user.*` are ever applied; privileged namespaces are never applied (see the Phase-4 xattr/ACL notes below). Implies metadata transmission |
|
||
| `-X`, `--xattrs` | Preserve extended attributes | ✅ Implemented | Preserves unprivileged `user.*` extended attributes (Linux `listxattr`/`getxattr` on capture, `fsetxattr` on the written destination fd). Both capture (sender) and application (receiver) are restricted to the `user.*` namespace and the two POSIX ACL xattrs, so a client can **never** force a `security.*`/`trusted.*`/privileged attribute onto the destination; the receiver independently re-validates every incoming name against this whitelist and rejects anything else. Payloads are bounded (per-name ≤255B, per-value ≤1MiB, per-file count ≤256 total bytes ≤4MiB) on both ends, and an oversized/malformed frame is a clean protocol rejection (no OOM). Applied fd-relative to the exact written file. Implies metadata transmission. Incompatible with `-s` (chunk serialization), rejected up front (see the notes); a `--link-dest`/`-H` hard-link copy fallback re-applies the attributes so they are not dropped when a link is refused |
|
||
| `-H`, `--hard-links` | Preserve hard links | ✅ Implemented | Files on the source that share an inode (`st_dev`+`st_ino`, e.g. a `cp -al` tree) are re-created as hard links to one another on the destination, so duplicate links stay deduplicated and only the first member's data is sent (later members are transmitted as payload-less `STATUS_HARDLINK` frames). The receiver links each sibling to the first member's installed file with an atomic link + rename; on `link()` failure it falls back to a byte-identical local copy of the first member, never a partial/corrupt file. Requires the sequential scan for ordering (the first member is always emitted and installed before any sibling is linked). Works single-threaded and under `-m`, `--inplace`, `--delay-updates` (links staged and published by rename) and `--partial`. Crosses the wire (`preserve_hard_links` bool; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0**, peers must match). Incompatible with `-s` (chunk serialization) and `--append`/`--append-verify`, rejected up front with a distinct error. See the Phase-4 hard-links notes below |
|
||
| `-D` | Same as --devices --specials | ✅ Implemented | Implies `--devices --specials`. `-D` was unassigned in FastSync (verified: no collision), so it is free to imply both device-node and special-file preservation. See the `--devices`/`--specials` rows and the Phase-4 devices notes below |
|
||
| `--devices` | Preserve device files | ⚠️ Partial | Recreates char/block device nodes on the destination via `mknod` instead of transferring content. Type + rdev are validated strictly (S_IFMT from the transmitted mode; major/minor range-checked, non-negative), and creation is **privilege-gated**: `mknod` needs `CAP_MKNOD`, so a non-root receiver (CI runs via setpriv as non-root) logs a warning and **skips the device entry safely** — the whole transfer never aborts just because the node could not be made. The node is created fd-relative below the receive root (`mknodat` on the confined secure parent), so it can never be placed outside the authorized root, never follows a symlink, and never replaces an existing directory. Only a char/block mode is honored. Crosses the wire (a new `STATUS_SPECIAL` frame carries the path + metadata mode + rdev; `PROTOCOL_VERSION` bumped **2.12.0 → 2.13.0**). Divergence: per-entry skip (not a hard error) when the receiver lacks `CAP_MKNOD`, documented in the Phase-4 devices notes |
|
||
| `--specials` | Preserve special files | ⚠️ Partial | Recreates **FIFOs** on the destination via `mkfifo` (unprivileged, so this is a real, assertable behavior under CI). Sockets cannot be recreated by any standard filesystem call and are skipped with an explicit note (best-effort / unsupported, matching the plan). FIFO creation is privileged-gated only in the sense of graceful skip on any permission failure. Node creation is confined below the receive root (`mkfifoat` on the secure fd-relative parent; no `..`, no symlink follow). Crosses the wire like `--devices` (the `STATUS_SPECIAL` frame; `PROTOCOL_VERSION` bumped **2.12.0 → 2.13.0**). See the Phase-4 devices notes |
|
||
| `--copy-devices` | Copy device contents as file | ⚠️ Partial | Copy a device's CONTENT into an ordinary regular file on the destination instead of recreating the node — non-privileged and safe. FastSync scans a device/FIFO as a regular file: its reported size (`st_size`, typically 0 for char devices and FIFOs) is copied, so a FIFO or a non-readable device becomes an empty (or size-bounded) regular file without ever blocking or reading unbounded pseudo-device streams. The run always succeeds and never crashes on such input. **Deliberate, safe divergence from rsync's dd-like unbounded device read.** See the Phase-4 devices notes |
|
||
| `--write-devices` | Write to devices as files | ⚠️ Partial | Write the received data directly into an **existing** device node on the destination instead of creating a regular file. Restricted and best-effort: the destination must already exist and be a char/block device (opened only under the confined receive root, with `O_NOFOLLOW` + `O_NONBLOCK`); a missing, symlinked, FIFO-with-no-reader (`ENXIO`), non-device destination, or any write failure is **skipped with a warning** rather than allowed, so a run can never clobber the system, never blocks on a special-file target, and never aborts on an unusable target. See the Phase-4 devices notes |
|
||
| `-U`, `--atimes` | Preserve access times | ✅ Implemented | Captures the source access time (from the scanner's pre-read stat, so it is not clobbered by reading the file for transfer) and transmits it over the wire; the receiver restores it together with the mtime via `futimens`/`utimensat`. Implies metadata transmission (the times travel inside the `-M` metadata payload), but does not enable ownership application (that stays opt-in via the identity flags). Wire: new `atime` fields on the metadata frame + a `preserve_atimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** |
|
||
| `-N`, `--crtimes` | Preserve create times | ⚠️ Partial | Captures the source birth time via `statx(STATX_BTIME)` on Linux and transmits it (recorded as a wire field), but there is **no portable way to set a birth time** (`utimensat` can only set atime/mtime), so the receiver explicitly does NOT apply it: it logs a debug note and continues — never failing the transfer and never pretending it worked. On platforms without `statx` it parses as a documented no-op (flag accepted; nothing is captured). Implies metadata transmission. Wire: new `crtime` fields + a `preserve_crtimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** (see the Phase-4 metadata-time notes) |
|
||
| `-O`, `--omit-dir-times` | Omit dirs from --times | 🔄 Compatibility No-op | Accepted and parsed for CLI compatibility, and the config boolean crosses the wire, but it has **no effect**: FastSync never preserves directory mtimes in the first place (directories are created via `mkdir` with no metadata, a documented divergence under `-d`/recursive), so there is nothing for an "omit" to suppress. It never breaks a normal run |
|
||
| `-J`, `--omit-link-times` | Omit symlinks from --times | 🔄 Compatibility No-op | Accepted and parsed for CLI compatibility, and the config boolean crosses the wire, but it has **no effect**: FastSync never sets symlink times (`-l`/`--links` copies symlinks as symlinks but the receiver does not apply timestamps/owner to symlink entries), so there is nothing for an "omit" to suppress. It never breaks a normal run |
|
||
| `--super` | Receiver attempts super-user activities | ❌ Not Implemented | |
|
||
| `--fake-super` | Store/recover privileged attrs via xattrs | ⚠️ Partial | Honest, limited subset. The receiver records the source `uid:gid:mode:mtime_sec:mtime_nsec` into a reserved `user.fastsync.stat` xattr on each written file (best-effort, fd-relative), so a later privileged restore could re-apply them — without attempting the (typically failing as non-root) `chown`. Full rsync fake-super **replay** (parsing that xattr to actually re-apply ownership on a later privileged run) is out of scope and is **divergent** from rsync, which uses its own `user.rsync.%stat%` format; no cross-tool conversion is attempted. Implies metadata transmission so the source uid/gid/mode/mtime are available. Both it and `-X`/`-A` are incompatible with `-s` (chunk serialization), rejected up front |
|
||
| `--open-noatime` | Avoid changing access time when opening files | ✅ Implemented | Sender-side policy: the sender opens source files with `O_NOATIME` (Linux) when reading them for transfer, so the open/read does NOT bump the source's on-disk access time. Degrades safely when `O_NOATIME` is unavailable (not defined) or refused (`EPERM`, since it needs `CAP_FOWNER` or file ownership): the code falls back to a normal open, so the data always transfers — only the atime-bump is skipped. It does not itself capture/preserve atime; it only avoids modifying it. **Client-only, never crosses the wire.** Exposed as `file_open_for_read()` and applied to both the buffered data path and the sendfile path |
|
||
| `--numeric-ids` | Do not map uid/gid by name | ✅ Implemented | Ownership is applied through FastSync's opt-in identity path (see the Phase-4 identity notes below). `--numeric-ids` is a mapping-policy modifier: when applying ownership it uses the transmitted numeric uid/gid directly, skipping the name lookup. Without an ownership-affecting option it is inert (FastSync only applies ownership when the user opts in). It does not need `-M` to be parsed, but ownership is only applied when metadata (hence the source uid/gid) is actually transmitted (see the notes) |
|
||
| `--usermap=STRING` | Map usernames | ✅ Implemented | Opt-in ownership application. rsync subset implemented: comma-separated `FROM:TO` rules evaluated in order, first match wins; `FROM`/`TO` are group/user names (resolved on the SOURCE machine at parse time), `*` (FROM matches any id / TO = the receiving process's current euid), and an `@N` or bare `N` numeric id. Rules are carried over the wire as resolved numeric id pairs; the receiver applies a matching rule (else falls back to `--chown`, `--numeric-ids`, then a best-effort name lookup) via an fd-relative `fchown`. Malformed/unresolvable specs are rejected with a clear error, never a silent no-op. Implies metadata preservation so the source uid/gid travel. Only effective when the receiver can actually change ownership (root or membership); otherwise it warns and continues |
|
||
| `--groupmap=STRING` | Map group names | ✅ Implemented | Same rsync subset and semantics as `--usermap` but for the group (gid) side and the group databases. See the Phase-4 identity notes |
|
||
| `--chown=USER:GROUP` | Map owner and group | ✅ Implemented | Opt-in ownership override applied receiver-side. Forms: `USER:GROUP`, `USER` (owner only), `:GROUP` (group only); a `*` for USER/GROUP means the current/root user or group as appropriate; an `@N`/bare `N` numeric id is accepted. A `:` inside a name may be escaped as `\:`. Equivalent to a trailing `*:*` usermap+groupmap rule (so an explicit `--usermap`/`--groupmap` match wins). Malformed or unresolvable specs are clear parse errors. Implies metadata preservation. Only effective when the receiver has permission to chown; otherwise it warns and continues (rsync parity) |
|
||
| `--copy-as=USER[:GROUP]` | Perform the copy as another user/group | ❌ Not Implemented | |
|
||
|
||
**Phase-4 metadata-time notes:** `-U/--atimes`, `-N/--crtimes`,
|
||
`-O/--omit-dir-times`, `-J/--omit-link-times`, and `--open-noatime` are new.
|
||
They change the wire: the per-file metadata frame grows `atime_valid` +
|
||
`atime_sec` + `atime_nsec` and `crtime_valid` + `crtime_sec` + `crtime_nsec`
|
||
(appended after the existing mode/uid/gid/mtime fields, preserving the exact
|
||
positions of every pre-existing field), and the config frame grows four
|
||
booleans — `preserve_atimes`, `preserve_crtimes`, `omit_dir_times`,
|
||
`omit_link_times` — that CROSS the wire so the receiver knows what to apply /
|
||
suppress. `--open-noatime` is **client-only** and is never serialized (it only
|
||
governs the sender's source reads). `PROTOCOL_VERSION` was bumped **2.11.0 →
|
||
2.12.0** (peers must match, exactly as prior phases did).
|
||
|
||
**Client-vs-wire split:** `-U` and `-N` affect both the sender (capture) and the
|
||
receiver (apply), so they and their metadata fields cross the wire;
|
||
`-O`/`-J` are receiver-side preferences and cross as config booleans;
|
||
`--open-noatime` is purely a client/sender open flag and stays off the wire
|
||
(mirroring the existing convention where `ignore_errors` is client-only while
|
||
`force_delete` crosses the wire).
|
||
|
||
**Phase-4 xattr/ACL notes (`-X/--xattrs`, `-A/--acls`, `--fake-super`):** these
|
||
are new in protocol 2.13.0 and add a bounded per-file xattr block to the
|
||
per-file metadata frame (count + each `name`/`value`, sent only when xattr
|
||
transport is enabled, i.e. with zero overhead on unaffected runs). The config
|
||
frame carries `preserve_xattrs`, `preserve_acls` (in the existing file-options
|
||
block) and a trailing `fake_super` boolean — all CROSS the wire so the receiver
|
||
knows the negotiated behavior; the derived `use_xattrs` flag is recomputed on
|
||
the receiver. `PROTOCOL_VERSION` was bumped **2.12.0 → 2.13.0** (peers must
|
||
match, exactly as prior phases did).
|
||
|
||
- **Security model (both `-X` and `-A`):** only `user.*` and the
|
||
`system.posix_acl_access` / `system.posix_acl_default` namespaces are ever
|
||
captured (sender) or applied (receiver). `security.*` (SELinux, capabilities,
|
||
...), `trusted.*`, and all other `system.*` attributes are never transmitted
|
||
or applied, so a client can never compel the receiver to set a privileged
|
||
xattr. The receiver re-validates each incoming name against this whitelist
|
||
even though the sender already filtered, so a malicious/compromised sender's
|
||
`security.capability` payload is rejected outright (a clean protocol error),
|
||
never applied.
|
||
- **Bounds / memory safety:** per-name length ≤ 255 B, per-value ≤ 1 MiB,
|
||
per-file count ≤ 256 names, per-file name+value total ≤ 4 MiB. Both the
|
||
sender (during capture) and the receiver (during receive) enforce these; an
|
||
oversized or malformed frame is rejected, never a large allocation.
|
||
- **Confined application:** xattrs are applied with `fsetxattr` on the exact
|
||
just-written destination file fd (before the atomic rename), never on a
|
||
caller-controlled path; this is the same confinement as mode/time restore.
|
||
The `--link-dest` / `-H` hard-link copy fallback (a byte copy when `link()`
|
||
is refused) also re-applies the incoming (or, for `-H`, the first member's)
|
||
xattrs and the `--fake-super` stat, so attributes are preserved rather than
|
||
silently dropped when the link fails.
|
||
- **Reserved fake-super key is receiver-only:** the `user.fastsync.stat` key is
|
||
excluded from sender capture AND from receiver application, so it can only be
|
||
written by the receiver's own `--fake-super` handling. A source file that
|
||
already carries such a record is never forwarded on a plain `-X` run, so it
|
||
cannot be spoofed to mislead a later privileged restore.
|
||
- **`-A` requires no libacl** — ACLs travel as the `system.posix_acl_*` xattrs.
|
||
Applying an ACL is owner-privileged: `fsetxattr` failure (e.g. non-root,
|
||
unsupported filesystem) is logged (collapsed to one line per file) and never
|
||
fatal.
|
||
- **`--fake-super`**: see the row above; the reserved key is `user.fastsync.stat`
|
||
with the documented `uid:gid:mode:mtime_sec:mtime_nsec` (mode octal) format.
|
||
It is honest but partial — there is no replay, and it does not interoperate
|
||
with rsync's `user.rsync.%stat%`.
|
||
- **Chunk serialization (`-s`) incompatibility:** the per-file xattr block rides
|
||
the streaming per-file frame, which `-s` replaces with a fixed buffer format,
|
||
so `-X` / `-A` combined with `-s` is rejected up front on both ends (mirroring
|
||
the existing `-H` + `-s` rejection) rather than silently dropping attributes.
|
||
|
||
**atime capture does not clobber the source atime:** the sender records the
|
||
access time from the **same pre-read stat the scanner already took** (inside
|
||
`file_metadata_create`), before any file data is read for transfer. So `-U`
|
||
alone captures the correct atime even without `--open-noatime`. `--open-noatime`
|
||
is orthogonal: it keeps the source's on-disk atime from being bumped by the read
|
||
that actually ships the data (only honoured where `O_NOATIME` works; it degrades
|
||
to a normal open otherwise, so the data always transfers).
|
||
|
||
**crtime handling:** `-N` captures the source birth time via `statx`/`STATX_BTIME`
|
||
(guarded `#ifdef STATX_BTIME` on Linux) and transmits it. On the receiver, **no
|
||
portable setter exists** (`utimensat` can only set atime/mtime), so the receiver
|
||
deliberately does **not** apply it: it logs a debug note and continues — it never
|
||
fails the transfer and never pretends the crtime was applied. This is the
|
||
explicit, documented unsupported-attribute handling. On platforms without
|
||
`statx` the flag is accepted but nothing is captured (a documented no-op).
|
||
|
||
**omit-dir-times / omit-link-times:** `-O` and `-J` are **accepted and parsed
|
||
for CLI compatibility** and their config booleans cross the wire, but they are
|
||
genuine **no-ops**: FastSync does not apply directory or symlink times at all
|
||
(directories are made via `mkdir` with no metadata; symlinks are dereferenced
|
||
or skipped, never written with a target), so there is nothing for an "omit" to
|
||
suppress. They never break a normal run. This is documented as a
|
||
divergence — the flags recognize the rsync interface but have no filtering
|
||
effect in FastSync.
|
||
|
||
**-U/-N and -M interaction:** because FastSync carries all metadata (mode, uid,
|
||
gid, mtime, and now atime/crtime) in one bounded payload that is only sent when
|
||
metadata transmission is on, `-U` and `-N` imply metadata transmission (the
|
||
times travel inside that payload). They do **not** enable ownership application,
|
||
which remains opt-in strictly through the identity flags (`--numeric-ids` /
|
||
`--usermap` / `--groupmap` / `--chown`).
|
||
|
||
**Phase-4 identity notes:** `--numeric-ids`, `--usermap`, `--groupmap`, and
|
||
`--chown` are real. They introduce a **controlled, opt-in, privilege-gated**
|
||
ownership-application path on the receiver: plain `-M`/`--preserve` still does
|
||
NOT apply client-supplied ownership (FastSync's deliberate conservative
|
||
default, byte-for-byte backward compatible); ownership is only attempted once a
|
||
client explicitly requests an ownership-affecting option. Application goes
|
||
through an fd-relative `fchown()` in the receiver's metadata-restore path (after
|
||
the file is fully written, before timestamps are set), so it is confined and
|
||
symlink-safe — never a path-based `chown`. When the receiver lacks permission
|
||
(typically non-root, e.g. the CI `nobody` user) `EPERM`/`EACCES` is logged as a
|
||
warning and the transfer CONTINUES with exit status success, matching rsync.
|
||
A no-op default means existing transfers are unaffected.
|
||
|
||
Resolution of the destination uid/gid on the receiver: a matching
|
||
`--usermap`/`--groupmap` rule wins; else the matching `--chown` side; else, with
|
||
`--numeric-ids`, the transmitted numeric id is used raw (no name lookup); else a
|
||
best-effort name lookup on the receiver's own account databases (skipped when
|
||
the transmitted id has no name present there). `--chown` enforces the receiver
|
||
side and is validated at parse time (malformed specs are clear errors, never a
|
||
silent no-op).
|
||
|
||
Wire/version: the config frame gained `numeric_ids`, `chown_uid_set`,
|
||
`chown_uid`, `chown_gid_set`, `chown_gid`, and the `usermap`/`groupmap` tables
|
||
(count-delimited lists of resolved int32 FROM/TO id pairs), so
|
||
`PROTOCOL_VERSION` was bumped **2.10.0 → 2.11.0** (peers must match). All new
|
||
fields cross `config_send`/`config_receive` with full symmetry and are validated
|
||
on receive (bounded map sizes below `MAX_IDENTITY_MAP`, ids `>=` the `-1`
|
||
sentinels).
|
||
|
||
Documented divergences from rsync: because FastSync transmits only numeric
|
||
uid/gid (not names) on the wire, name-based values (`--usermap`/`--groupmap`
|
||
names, `--chown` names) are resolved to numbers at CLI parse time against the
|
||
**client (sender) machine's** account databases; this reproduces rsync's
|
||
semantics on a shared-account source/destination and is documented for a
|
||
genuinely different destination. The interesting named-value subset is
|
||
supported (`*` FROM wildcard, `*` TO = current user, `@N`/bare-`N` numerics); a
|
||
lone-`@` "use the FROM value unchanged" rsync form is not implemented. Also
|
||
unlike rsync, plain `-M` never applies ownership and `--usermap`/`--groupmap`/
|
||
`--chown` each imply metadata preservation so the source uid/gid actually travel
|
||
(the flags only take effect where ownership is being preserved/applied).
|
||
|
||
**Phase-4 hard-links notes:** `-H`/`--hard-links` is real and introduces a
|
||
deduplicating wire path for files whose source entries share a filesystem inode.
|
||
On the sender, the scanner records each distinct `(st_dev, st_ino)` encounter and
|
||
assigns it a stable, run-local link-group id (`HardLinkTable`, mutex-guarded so a
|
||
multi-threaded scan could share one instance). The FIRST member of a group is
|
||
transferred normally and carries the data; each later (sibling) member is
|
||
transmitted as a payload-less `STATUS_HARDLINK` frame carrying its destination
|
||
path, the group id, and the first member's destination-relative wire path.
|
||
Ordering is guaranteed by forcing the sequential scanner whenever `-H` is on
|
||
(even under `-m`), so the first member is always emitted — and, on the receiver's
|
||
single write thread, installed — before any of its siblings; the receiver is
|
||
therefore always able to link to an already-present first member, including the
|
||
"first member already up-to-date/skipped" case (the sibling links to or copies
|
||
the existing file). Asymmetric existence policies are handled gracefully: under
|
||
`--existing`, if the first member's destination is absent (so it is skipped) but
|
||
a sibling's own destination already exists, that existing sibling is left in
|
||
place rather than the transfer aborting on the missing first member. The receiver
|
||
installs each sibling beneath its confined root
|
||
as an atomic hard link (temp link + rename); when `link()` fails (cross-device,
|
||
filesystem refuses links) it falls back to a byte-identical local copy of the
|
||
first member, never a partial/corrupt file. `--delay-updates` stages each sibling
|
||
as a hard link to the first member's STAGED file, so publication's renames
|
||
preserve the shared inode; `--inplace` and `--partial` are unaffected (a sibling
|
||
is a fresh link/copy). Because a hard link shares an inode, metadata is applied
|
||
exactly once on the first member and never re-written through the sibling (whose
|
||
members are byte-identical by construction), so all members agree.
|
||
|
||
Wire/version: `PROTOCOL_VERSION` was bumped **2.11.0 → 2.12.0** (peers must
|
||
match). The config frame already carried the `preserve_hard_links` boolean
|
||
(round-trips through `config_send`/`config_receive`); the only new wire element
|
||
is the `STATUS_HARDLINK` frame described above. Incompatibilities (rejected up
|
||
front with a distinct error on the client, and re-checked on receive): `-H` with
|
||
`-s` chunk serialization (the chunk wire has no per-file hard-link info) and `-H`
|
||
with `--append`/`--append-verify` (a payload-less sibling cannot be tail-resumed).
|
||
|
||
**Phase-4 devices notes:** `--devices`, `--specials`, `-D`, `--copy-devices`,
|
||
and `--write-devices` are new. They change the wire: the config frame grows three
|
||
booleans — `preserve_specials`, `copy_devices`, `write_devices` — that CROSS the
|
||
wire (`preserve_devices` already existed), and a new `STATUS_SPECIAL` frame (used
|
||
by `--devices`/`--specials`/`-D`) carries a special/device entry: the destination
|
||
path, the metadata frame (whose mode's S_IFMT bits carry the node kind, requiring
|
||
the flags to imply metadata transmission), and two int32 `rdev` major/minor
|
||
fields. The chunk-serialized wire (`-s`) grows a matching per-file special
|
||
marker + rdev so `--devices/--specials` also work under `-s`. `PROTOCOL_VERSION`
|
||
was bumped **2.12.0 → 2.13.0** (peers must match, exactly as prior phases did).
|
||
|
||
**Privilege gating (the crux):** making a device node requires `CAP_MKNOD` (root).
|
||
CI runs the integration suite as a NON-ROOT user (via setpriv), so `mknod` fails
|
||
with `EPERM`. The receiver treats this as a graceful, logged *skip of the entry*
|
||
returned as a success/skip outcome — the whole transfer NEVER aborts just because
|
||
the environment cannot create the node. `mkfifo` (FIFOs) is unprivileged, so
|
||
`--specials` FIFO creation is a real, assertable behavior under CI; sockets cannot
|
||
be recreated by any standard filesystem call and are skipped with an explicit
|
||
note. The "device actually created" integration assertions are guarded to run
|
||
only as root. User-facing expectation: point `--devices` at devices and a
|
||
non-root receiver will faithfully skip them while transferring everything else.
|
||
|
||
**Confinement & validation:** a special/device node is created with
|
||
`mknodat`/`mkfifoat` on the parent directory opened fd-relative below the receive
|
||
root (`file_open_secure_parent`: `O_NOFOLLOW`, no `..` components, root-checked),
|
||
so a node can never be created outside the authorized destination root and never
|
||
through a symlinked parent. The transmitted type is derived ONLY from the
|
||
validated S_IFMT bits of the metadata mode (char/block/FIFO honored, socket
|
||
skipped, regular/dir rejected as an invalid special), and the transmitted rdev is
|
||
validated both on the wire (`file_receive_special`, `chunk_deserialize`) and at
|
||
the creation site (`file_special_rdev_valid`): a negative, oversize, or
|
||
non-device-carrying rdev is rejected outright (receiver aborts the frame), and a
|
||
node is never replaced over an existing directory or unrelated entry (a matching
|
||
existing node is left in place). `--write-devices` is the deliberately restricted
|
||
danger path: it only ever opens an existing char/block node under the confined
|
||
root, and every failure mode (missing, non-device, write error, EPERM) is a
|
||
warning + skip, never a system-clobbering write or an abort.
|
||
|
||
**Documented divergences (honest subset):**
|
||
- A device entry the receiver cannot create (missing `CAP_MKNOD`) is *skipped*,
|
||
not a transfer failure — rsync under the same conditions would error.
|
||
- `--copy-devices` copies the device's *reported size* (typically 0 for char
|
||
devices/FIFOs) into a regular file and never reads an unbounded pseudo-device;
|
||
this is the safe, non-hanging alternative to rsync's dd-like read.
|
||
- `--write-devices` requires the device to already exist at the destination and
|
||
never creates it; unsupported/inaccessible targets are skipped, not written.
|
||
- Ownership is not applied to recreated nodes (identity `fchown` needs an fd and
|
||
would require opening the node); permissions and mtime are applied at
|
||
creation / via `utimensat`.
|
||
|
||
## 9. Symlink Handling
|
||
|
||
| Flag | Rsync Description | FastSync Status | Notes |
|
||
|------|-------------------|-----------------|-------|
|
||
| `-l`, `--links` | Copy symlinks as symlinks | ✅ Implemented | A symlink is transmitted as a real symlink: its target string crosses the wire (a new `STATUS_SYMLINK` frame / chunk entry type) and the receiver creates it with `symlinkat` beneath the receive root. This makes the previously-`-l`-included-but-targetless symlink handling complete. See the Phase-4 symlink-trust notes |
|
||
| `-L`, `--copy-links` | Transform symlink to referent | ✅ Implemented | `copy_links` config field |
|
||
| `--copy-unsafe-links` | Transform unsafe symlinks | ✅ Implemented | `copy_unsafe_links` config field |
|
||
| `--safe-links` | Ignore symlinks outside tree | ✅ Implemented | `safe_links` config field |
|
||
| `--munge-links` | Munge symlinks for safety | ✅ Implemented | Sender rewrites each transmitted symlink target with a `#SYMLINK/` marker; a target that could escape the receive root (absolute or containing `..`) is never transmitted (contained/skipped); the receiver strips the marker to restore the real target. See the Phase-4 symlink-trust notes |
|
||
| `-k`, `--copy-dirlinks` | Transform symlink to dir | ✅ Implemented | A symlink whose referent is a directory is dereferenced and recursed as a real directory; a symlink to a regular file stays a symlink. Sender-side only. See the Phase-4 symlink-trust notes |
|
||
| `-K`, `--keep-dirlinks` | Treat symlinked dir as dir | ✅ Implemented | On the receiver, an existing destination symlink-to-a-directory is used as that directory (followed) instead of being replaced; it is followed only when it resolves to a directory that stays beneath the receive root. See the Phase-4 symlink-trust notes |
|
||
|
||
**Phase-4 symlink-trust notes:** `-l/--links`, `-k/--copy-dirlinks`,
|
||
`-K/--keep-dirlinks`, and `--munge-links` form the "symlink trust boundaries"
|
||
row. Making all three new flags have an observable, security-sane effect
|
||
required transmitting symlink targets, so FastSync's `-l/--links` is now real:
|
||
a symlink-type entry carries its target on the wire (a new `STATUS_SYMLINK`
|
||
frame for the per-file path, and a new entry type `2` in the `-s` chunk
|
||
serializer) and the receiver creates it with `symlinkat` under an `O_NOFOLLOW`
|
||
parent walk, never following the target. Wire changes: `STATUS_SYMLINK`,
|
||
the chunk entry type `2`, a per-entry symlink-target string, and two new config
|
||
booleans that CROSS the wire — `munge_links` and `keep_dirlinks`; `PROTOCOL_VERSION`
|
||
was bumped **2.12.0 → 2.13.0** (peers must match, exactly as prior phases did).
|
||
|
||
**Per-flag semantics and divergences.**
|
||
- **`-l/--links`** copies a symlink as a symlink: the scanner `readlink`s the
|
||
target, the sender transmits it, and the receiver `symlinkat`s it. FastSync
|
||
`-l` never preserved symlink targets before (the flag was documented partial
|
||
and, in fact, tried to read the referent as file data); it now does, matching
|
||
rsync. Divergences: because the receiver enforces the symlink containment
|
||
predicate unconditionally, a plain `-l` sync **refuses to round-trip a
|
||
legitimate absolute symlink target** (it is dropped, never created pointing
|
||
outside the root — see the `--munge-links` note for the symmetric trust
|
||
boundary); a relative in-root target is copied as-is. FastSync also does not
|
||
set timestamps/owner on symlinks (no symlink-mode metadata application),
|
||
matching its existing no-op `--omit-link-times`.
|
||
- **`-k/--copy-dirlinks`** (sender): a symlink whose referent is a directory is
|
||
dereferenced and recursed into as a real directory; a symlink to a regular
|
||
file (or any non-directory) is kept as a symlink. This is rsync's `-k`. When
|
||
`-L/--copy-links` or `--safe-links`/`--copy-unsafe-links` are active, their
|
||
(dereference) semantics take precedence, so `-k` is subsumed exactly as in
|
||
rsync.
|
||
- **`-K/--keep-dirlinks`** (receiver, crosses the wire): when a directory is to
|
||
be created (on-demand parent creation for a child write) and the destination
|
||
path is already an existing symlink that resolves to a directory *within* the
|
||
receive root, that symlinked directory is used (followed) instead of being
|
||
replaced by a real directory; new entries are written beneath it. The follow
|
||
is confined: it only happens where `realpath` of the symlink resolves to a
|
||
still-within-root real directory, so a malicious link pointing outside the
|
||
root is never followed. Scope: `-K` acts on the write path (parent/`mkdir`
|
||
creation); the delete walker still never follows symlinks (a documented
|
||
divergence for `--delete` over an existing symlinked dir). Without `-K` the
|
||
destination symlink is not followed (the O_NOFOLLOW walk fails the write),
|
||
which is the safe default.
|
||
- **`--munge-links`** (sender security rewrite; crosses the wire so the receiver
|
||
unmunges): every transmitted symlink target is prefixed with the marker
|
||
`#SYMLINK/`; the receiver strips the marker (only when the negotiated
|
||
`munge_links` policy is on — a plain `-l` run never strips the prefix, so a
|
||
source symlink that genuinely begins with `#SYMLINK/` round-trips verbatim)
|
||
and restores the exact real target. The trust boundary is **symmetric and
|
||
enforced receiver-side**, independent of the sender: `file_symlink_at_secure`
|
||
refuses any target that `file_symlink_target_contained` rejects (absolute
|
||
`/...` or relative with a `..` component), and `file_save_to_disk_full`
|
||
contains such an entry (skipped) rather than materializing it. A deliberate confinement trade-off: because the receiver
|
||
enforces containment unconditionally, a plain `-l` (no `--munge-links`) sync
|
||
*refuses to round-trip a legitimate absolute symlink target* — such target is
|
||
dropped, never created pointing outside the root. This is a stricter subset of
|
||
rsync: rsync stores munged targets on the RECEIVING side and depends on both
|
||
ends running `--munge-links`; FastSync additionally enforces the containment
|
||
predicate at the receiver regardless of what the sender transmitted. When no
|
||
symlink is being transmitted (`-l`/`-k`/`-a` off) `--munge-links` has nothing
|
||
to rewrite and is inert. -*K/`--keep-dirlinks` policy is installed per
|
||
connection at config-accept (stable for the whole transfer, never racy under
|
||
`-m`), and only ever follows an in-root symlink-to-directory.*
|
||
|
||
**Compatibility (byte-identical when all three are absent):** `-k`, `-K` and
|
||
`--munge-links` are opt-in. Without them the scanner's link handling, the wire
|
||
frames, and the receiver's writes are unchanged for every other option set, so a
|
||
run that previously worked continues to behave identically. `-l/--links` itself
|
||
now transmits targets (the prior behavior was broken/partial); its status moved
|
||
`⚠️ Partial → ✅ Implemented`.
|
||
|
||
## 10. Sparse & Device
|
||
|
||
| Flag | Rsync Description | FastSync Status | Notes |
|
||
|------|-------------------|-----------------|-------|
|
||
| `-S`, `--sparse` | Sparse block handling | ⚠️ Partial | Flag is accepted, but full hole preservation is not implemented |
|
||
| `--preallocate` | Allocate dest files before writing | ✅ Implemented | The receiver preallocates the destination file's full expected space before any data is written, so a transfer that would overflow disk fails fast at allocation time (a clean error, not a half-written file) and the file is laid out contiguously, avoiding fragmentation. Crosses the wire (the config frame carries a `preallocate` boolean; `PROTOCOL_VERSION` bumped **2.10.0 → 2.11.0**, peers must match) so the sender knows the receiver will preallocate and the receiver performs it. **Allocation approach:** `posix_fallocate()` is preferred because it reserves *real* disk blocks (true fail-fast on ENOSPC), falling back to plain `ftruncate()` only when the filesystem reports the allocation is unsupported (`EOPNOTSUPP`/`ENOSYS`); `ftruncate` still extends the logical size so the intent degrades gracefully. **Fallback/error semantics:** `EOPNOTSUPP`/`ENOSYS` → clean fallback to `ftruncate` (best-effort, preallocates the logical size and never fails a transfer on filesystems that lack `posix_fallocate`); a genuine allocation failure (`ENOSPC`/`EDQUOT`/`EFBIG`/…) aborts the file/receive with a distinct `preallocate failed ... transfer aborted` error — it does **not** fall back to a normal non-preallocated write, preserving the fail-fast purpose. **Size-known requirement:** preallocation only runs when the final size is already known up front (the normal regular-file case); unknown-length data is skipped (never failed). **Orthogonality:** applies uniformly across the atomic temp+rename store path, `--inplace`, `--partial`/`--partial-dir`, `--delay-updates` (the staged temp file is preallocated before data flows) and the `--link-dest` copy fallback; it neither implies nor conflicts with `-s`, `--append`, or delta. rsync-divergence: rsync signals that `--preallocate` is ignored with `--sparse`; FastSync simply preallocates first and still honours `--sparse`'s `ftruncate` sizing/trim, so the two combine rather than one being silently ignored. See the Phase-4 preallocate notes below |
|
||
|
||
**Preallocate notes (Phase 4, preallocate wave):** `--preallocate` is implemented as a real receiver-side allocation of the destination file's space before data is written. It is a plain boolean config flag that crosses the wire (serialized in the config frame's selection-options block, mirroring `--inplace`/`--append`/`--force`), so the run requires matching ends: `PROTOCOL_VERSION` was bumped **2.10.0 → 2.11.0** (peers must match or the version check fails). The allocation is performed on the exact destination fd, immediately after it is opened, before any bytes are streamed; `posix_fallocate` (and the `ftruncate` fallback) leave the fd's file offset untouched, so the subsequent data write at offset 0 is unaffected and complete. Because FastSync writes each file's byte payload in one in-memory batch, the "full expected size" is exactly the known `data_size`, which is what gets preallocated. Unknown-length/streamed payloads are skipped rather than failed. A failed allocation logs a distinct `preallocate failed` error and aborts the file (the atomic temp is unlinked, the inplace target is left untrimmed) so the run fails cleanly and never silently degrades to a non-preallocated write — preserving rsync's fail-fast intent on a full disk.
|
||
|
||
|
||
## 11. Checksum & Comparison
|
||
|
||
| Flag | Rsync Description | FastSync Status | Notes |
|
||
|------|-------------------|-----------------|-------|
|
||
| `--checksum` | Skip based on checksum | ✅ Implemented | With `--incremental`, compares per-file whole-file content digests to skip unchanged files. The digest algorithm is `xxh64` with seed 0 by default and is selectable via `--checksum-choice`/`--cc` (xxh64/xxhash or md5) and `--checksum-seed=NUM` (see those rows); `-c` remains compression |
|
||
| `--checksum-choice=STR`, `--cc=STR` | Choose checksum algorithm | ✅ Implemented | Real algorithm selection for the per-file whole-file digest used by the `--incremental`/`--checksum` handshake and by the basis-dir content verification. FastSync genuinely supports `xxh64` (the default, exact xxHash64, seeded by `--checksum-seed`) and `md5` (via OpenSSL EVP); `xxhash` is accepted as rsync's spelling of xxHash64. Any other name (md4/sha1/sha256/crc32/none/…) is rejected with a clear error at parse time — never a silent no-op. `--cc` is the alias (`--cc=ALG` and space forms both parse). The algorithm id and seed cross the wire with the config frame, so the receiver hashes its on-disk old file with the SAME algorithm+seed the sender used and both agree on a match; the sender's digest and the receiver's comparison live in the per-file `STATUS_CHECK` handshake, which now carries a length-prefixed, bounded (1..16 byte) digest instead of a fixed 64-bit value, and the receiver pins the received length to the negotiated algorithm's digest length (defense-in-depth: a mismatched/malicious length only forces a safe re-transfer). Note: `md5` is a FIPS-non-approved algorithm, so under an OpenSSL build with FIPS mode enabled `--checksum-choice=md5` fails loudly rather than silently falling back. Protocol/layout: `PROTOCOL_VERSION` bumped **2.9.0 → 2.10.0** (peers must match). Defaults preserve the pre-existing behavior byte-for-byte (xxh64, seed 0). Like rsync, the choice only takes effect where a whole-file digest is actually computed (`--checksum` on, or a basis-dir flag); it does not itself enable `--checksum`. Closely-related divergence: the delta BLOCK strong checksum (§11 delta) stays xxHash32 — `--checksum-choice` selects only the whole-file digest, matching rsync where the per-block checksum is independent of the whole-file checksum choice |
|
||
| `--compare-dest=DIR` | Compare dest files relative to DIR | ✅ Implemented | DIR is a receiver-side basis relative to the destination root (confined below it; absolute/`..`/`.` rejected, `//` collapsed and trailing `/` dropped). On the receiver's per-file check (implies `--incremental`) an exact match = same size + mtime (unless `--size-only`; `-I` disables matching) **and** equal xxHash64 of the sender's file; a match suppresses the data transfer. compare-dest never copies: it only skips a file the destination does **not** already hold (sparse destination, rsync parity), and is consulted before the normal delta/full paths. Repeatable; searched in command-line order, first match wins. Divergences: when the destination already holds a *different* version rsync deletes it but FastSync instead transfers the data (keeps the mirror complete; never deletes without `--delete`); attribute-only differences on a match are not re-applied (data is skipped so the sender never sends metadata); content is verified by xxHash64, stricter than rsync's default quick check. Sizing: FastSync's whole-file payload limit is 256 MiB on **every** transfer path (not basis-specific); rsync applies basis dirs to arbitrary sizes, so FastSync refuses a basis run whose source contains a larger file up front with a clear error before any transfer. Wire: a basis-count field is always present on the config frame (protocol 2.9.0, so clients and servers must both be 2.9.0) |
|
||
| `--copy-dest=DIR` | Include copies of unchanged files | ✅ Implemented | Same basis rules as `--compare-dest`, but an exact match materializes a **local copy** of the DIR file into the destination (via the normal atomic temp+rename store path, so `--existing`/`--ignore-existing`/`--update`/`--backup`/`--delay-updates` all still apply) instead of transferring data. Repeatable; command-line order = priority. Content is xxHash64-verified before the copy. Divergences: a basis-hit destination keeps the basis file's own mode/uid/gid and mtime (the sender sends no metadata on a skip), so with `--size-only` its mtime can differ from the source and attribute-only differences are copied with the basis attributes rather than rsync's "copy + fix attributes". Requires `--incremental` (implied); incompatible with `-s`. Wire: protocol 2.9.0 |
|
||
| `--link-dest=DIR` | Hardlink to files when unchanged | ✅ Implemented | Same basis rules as `--copy-dest`, but an exact match installs an atomic **hard link** to the DIR file (temp hard link + rename) so no data or disk space is used; where the link is impossible (basis on another filesystem, filesystem refuses links) it falls back cleanly to a byte-identical local copy, never a corrupt/partial file. `--delay-updates` stages the link and publishes by rename, so the final entry stays a real hard link. Repeatable (searched in command-line order, first match wins). Content is xxHash64-verified before linking. Divergences and caveats: an already up-to-date destination file is not re-linked to a basis file (only files that would otherwise be written are linked); a link keeps the basis inode's own mode/uid/gid and mtime — metadata is never written through the shared inode (that would mutate the basis file), so a later `--inplace` run that rewrites such a destination path **will mutate the basis snapshot** through the shared inode (use `--copy-dest` when the destination must stay independently writable); with `--size-only` the linked mtime can differ from the source; a `--remove-source-files` source satisfied by a basis dir is treated as skipped and therefore **retained** (never removed); basis dirs are excluded from `--delete`. Requires `--incremental` (implied); incompatible with `-s`. Wire: protocol 2.9.0 |
|
||
| `-y`, `--fuzzy`, `--no-fuzzy` | Find similar file for basis | ✅ Implemented | `-y/--fuzzy` is a pure bandwidth optimization on the existing receiver-driven delta path: when a file must be transferred and the destination holds no usable content at the exact path (file absent, or the destination file is outside the delta engine's size bounds), the receiver searches the SAME destination directory for an existing regular file whose basename is similar to the incoming name and uses it as the delta basis, so the sender transmits only the differences instead of the whole file. The output is always byte-exact regardless of which (or whether any) basis is chosen. Decision location: the receiver performs the candidate search inside `receive_incremental_check` and sends the normal `STATUS_DELTA_SIGNATURE`; the sender never learns the basis was a different file, so no new frame type or sender logic was needed — only the config frame grew a `fuzzy` boolean, so `PROTOCOL_VERSION` was bumped **2.8.0 → 2.9.0** (peers must match). Similarity heuristic (deterministic, simpler than rsync's deliberately-fuzzy matching, and documented precisely): candidates are the target's sibling entries in its destination directory, opened `O_NOFOLLOW`/`AT_SYMLINK_NOFOLLOW` under the confined root (symlinks never followed; nothing outside the destination root is ever read or hashed); dotfiles, directories, the target's own name, and the `.fastsync-stage`/temp scratch names are excluded; like the ordinary delta path, the block signature the receiver transmits is derived from on-disk content it may not otherwise send, so a negotiated `--fuzzy` run exposes the destination's sibling files (at block granularity) to the sender as a known-plaintext oracle — the same information class as the normal delta handshake over the file being replaced; the size gate is the delta engine's own bounds (both files ≥ 16 KiB, ≤ `--delta-max`, ratio ≤ 10×) rather than rsync's ~1.5× size window; the name gate is a Levenshtein edit distance between the basenames accepted only when ≤ half the length of the longer basename; the single best candidate (smallest distance, tie-break size closest to the incoming file then lexicographically smaller basename) is read; the directory scan is capped at 4096 entries so a pathological directory cannot stall a transfer. When fuzzy applies: only to files the receiver would otherwise send whole — the destination's own file is always preferred as the delta basis when it exists and fits the delta size bounds, so fuzzy does NOT replace an existing-but-different destination basis; FastSync's 10× delta size-ratio bound means an existing destination file that is too far away in size still lets the fuzzy search run. When no similar candidate exists the transfer falls back to the normal whole-file transfer. rsync-divergence note: rsync's own matching uses a fuzzy name/size rule set; FastSync implements the closest safe deterministic approximation above. Because FastSync's delta machinery is off by default (rsync's is on), `--fuzzy` implies `--incremental` + `--delta` (unless `--whole-file`/`-W` or an explicit `--no-delta` switched delta off, in which case fuzzy is inert — matching rsync where `--whole-file` makes fuzzy irrelevant). Unlike the basis-dir options, `--fuzzy` honors an explicit `--no-incremental` (it does not force the handshake back on); an explicit `--no-incremental` also suppresses the delta implication so no invalid `--delta requires --incremental` config results. `--no-fuzzy` negates it. All surrounding semantics are untouched: a fuzzy-reconstructed file is stored as a normal file, so `--remove-source-files`, itemize/`-i`, `--stats`, `--backup`, `--delay-updates`, `--existing`/`--ignore-existing`/`--update` behave exactly as for a whole-file transfer (the fuzzy delta does not skip the file) |
|
||
|
||
## 12. Compression
|
||
|
||
| Flag | Rsync Description | FastSync Status | Notes |
|
||
|------|-------------------|-----------------|-------|
|
||
| `-z`, `--compress` | Compress file data | 🔀 Alt Arg | Always uses zstd (rsync supports multiple algorithms) |
|
||
| `--compress-choice=STR`, `--zc=STR` | Choose compression algorithm | ✅ Implemented | FastSync supports `zstd` and `none` |
|
||
| `--compress-level=NUM`, `--zl=NUM` | Set compression level | ✅ Implemented | 1-22, default 5 |
|
||
| `--compress-threads=NUM` | Set compression threads | ❌ Not Implemented | |
|
||
| `--skip-compress=LIST` | Skip compress for suffixes | ✅ Implemented | Comma-separated, case-insensitive suffix list; empty list skips none; incompatible with FastSync chunk serialization (`-s`) |
|
||
|
||
## 13. Connectivity
|
||
|
||
| Flag | Rsync Description | FastSync Status | Notes |
|
||
|------|-------------------|-----------------|-------|
|
||
| `-e`, `--rsh=COMMAND` | Remote shell to use | ❌ Not Implemented | Removed; SSH invokes `ssh` directly |
|
||
| `--rsync-path=PROGRAM` | rsync binary on remote | ❌ Not Implemented | Removed; use `--fastsync-server-path` |
|
||
| `--port=PORT` | Alternate daemon port | ✅ Implemented | `server_port` config field |
|
||
| `--sockopts=OPTIONS` | Custom TCP options | ✅ Implemented | Comma-separated allowlist of `OPT=VAL` applied via `setsockopt` after `socket()` before `connect()`/`bind()`. Only `TCP_NODELAY`, `SO_KEEPALIVE`, `SO_REUSEADDR` (0/1) and `SO_RCVBUF`/`SO_SNDBUF` (byte count) are accepted; an unknown option name or a bad value is rejected up front, never silently ignored. A value is required for every option (`OPT=VAL`; a bare name is an error). Applied to the outgoing TCP and TLS client reset when absent. Local socket concern: never crosses the wire |
|
||
| `--blocking-io` | Use blocking I/O for remote shell | ❌ Not Implemented | |
|
||
| `--outbuf=N\|L\|B` | Set output buffering | ❌ Not Implemented | |
|
||
| `--address=ADDRESS` | Bind address for outgoing socket | ✅ Implemented | Binds the outgoing client socket to a local source address before `connect()` (resolved with the same `-4`/`-6` family hints as the destination). Local socket concern: never crosses the wire |
|
||
| `-4`, `--ipv4` | Prefer IPv4 | ✅ Implemented | Forces `AF_INET` in the `getaddrinfo` hints for client destination/source resolution and the server bind (see the Phase 5, Wave B note). Mutually exclusive with `-6` |
|
||
| `-6`, `--ipv6` | Prefer IPv6 | ✅ Implemented | Forces `AF_INET6` in the `getaddrinfo` hints for client destination/source resolution and the server bind. Mutually exclusive with `-4` |
|
||
| `--remote-option=OPT`, `-M` | Send an option only to the remote side | ❌ Not Implemented | `-M` is FastSync's metadata-preservation flag |
|
||
|
||
## 14. Daemon Mode
|
||
|
||
| Flag | Rsync Description | FastSync Status | Notes |
|
||
|------|-------------------|-----------------|-------|
|
||
| `--daemon` | Run as rsync daemon | ❌ Not Implemented | Removed because it had no effect |
|
||
| `--config=FILE` | Alternate rsyncd.conf file | ❌ Not Implemented | Removed because it had no effect |
|
||
| `--dparam=OVERRIDE` | Override global daemon config | ❌ Not Implemented | |
|
||
| `--no-detach` | Don't detach from parent | ❌ Not Implemented | |
|
||
| `--password-file=FILE` | Read daemon password from file | ❌ Not Implemented | |
|
||
| `--early-input=FILE` | Use FILE for daemon early exec | ❌ Not Implemented | |
|
||
|
||
## 15. Safety & Security
|
||
|
||
| Flag | Rsync Description | FastSync Status | Notes |
|
||
|------|-------------------|-----------------|-------|
|
||
| Path escape detection | Ensure files stay within root | ✅ Implemented | `has_path_traversal()` + realpath |
|
||
| Symlink-safe delete | Skip symlinks in delete walk | ✅ Implemented | `delete_extras_walk()` |
|
||
| Protocol version check | Verify compatible versions | ✅ Implemented | `config_receive()` |
|
||
| Max data/string/chunk sizes | Prevent OOM attacks | ✅ Implemented | Per-message limits |
|
||
| Per-connection memory limit | 1GB per connection | ✅ Implemented | `MAX_CONNECTION_MEMORY` |
|
||
| `--max-alloc=SIZE` | Limit a single memory allocation | ✅ Implemented | Caps the largest single allocation; binary units, default 1G |
|
||
| `--trust-sender` | Trust remote sender's file list | ❌ Not Implemented | |
|
||
| `--old-args` | Disable modern arg protection | ✅ Implemented | SSH-only legacy mode; restores raw remote command construction and permits shell interpretation of the configured server path |
|
||
| `--ignore-missing-args` | Ignore missing source args | ✅ Implemented | FastSync has a single source-root argument (which always exists), so the "explicitly requested source arguments" are the `--files-from` entries and the flags only ever apply there (inert without `--files-from`, like `-R`). Without the flag a listed-but-missing entry stays a hard pre-transfer error (nothing is transferred). With it each missing entry is skipped: nothing is sent for it, it never enters the keep-set, and the run succeeds for the rest — an all-missing non-empty list succeeds transferring nothing, matching rsync. `--dirs` + `--files-from` missing entries are skipped the same way. Every skipped entry is logged and a per-run warning names the count, so the handling is never a silent no-op. Divergences: an EMPTY `--files-from` file stays a hard error in every mode (no argument was requested at all; rsync likewise reports "no source files specified"); missing-arg skipping only applies to the pre-transfer list validation, so an entry that is present at preflight and vanishes mid-transfer still fails (matching rsync, whose flag "does not affect subsequent vanished-file errors"); `--no-ignore-missing-args` is not a supported negation |
|
||
| `--delete-missing-args` | Delete missing source args | ✅ Implemented | Implies `--ignore-missing-args` (order-independent) and additionally removes each missing entry's destination mirror receiver-side. The mirror is computed exactly like a present sibling's wire path: the bare relative entry under `-R`, otherwise the full source-mirror path below the destination root. rsync parity, verified against the man page: it does **not** imply `--delete` generally and is "independent of any other type of delete processing" — unrelated destination extras are untouched unless `--delete` is also present. Composition with `--delete` + timing: the exact-path deletions commit with the manifest, early for `--delete-before`/`--delete-during`, else only after a fully-successful transfer (delete-after/commit). A non-empty directory mirror is removed only when `--force` or `--delete` is in effect (otherwise it is left with a warning and the run continues, like rsync); an absent mirror is a no-op. An explicitly listed missing arg is a user request, not an excluded file: its deletion is never blocked by the filter-exclusion protection of excluded destination mirrors (a mirror sitting inside a filter-excluded directory is still removed). Safety/policy: gated by the server `--allow-delete` policy like `--delete`; the request paths cross the wire only in the delete-manifest frame and are confined by the same receiver validation as the keep-set (non-empty, relative, traversal-free, bounded by the per-section/per-frame manifest caps); the `--delay-updates` staging directory and basis snapshots are protected exactly as in the extras walker. Divergence: the missing-args deletions are not counted toward `--max-delete` (they are explicit per-path requests, not discovered extras). See the Phase-3 wire note below for the `PROTOCOL_VERSION` bump |
|
||
|
||
## 16. Batch Operations
|
||
|
||
| Flag | Rsync Description | FastSync Status | Notes |
|
||
|------|-------------------|-----------------|-------|
|
||
| `--write-batch=FILE` | Write batched update to file | ❌ Not Implemented | |
|
||
| `--only-write-batch=FILE` | Write batch without updating dest | ❌ Not Implemented | |
|
||
| `--read-batch=FILE` | Read batched update from file | ❌ Not Implemented | |
|
||
|
||
## 17. Advanced
|
||
|
||
| Flag | Rsync Description | FastSync Status | Notes |
|
||
|------|-------------------|-----------------|-------|
|
||
| `--stop-after=MINS` | Stop after N minutes | ❌ Not Implemented | |
|
||
| `--stop-at=TIME` | Stop at specified time | ❌ Not Implemented | |
|
||
| `--fsync` | Fsync every written file before publication | ✅ Implemented | |
|
||
| `--protocol=NUM` | Force older protocol version | ❌ Not Implemented | |
|
||
| `--iconv=CONVERT_SPEC` | Charset conversion | ❌ Not Implemented | |
|
||
| `--checksum-seed=NUM` | Set checksum seed | ✅ Implemented | Sets the seed for FastSync's whole-file xxHash64 digest (full 64-bit seed) and for the delta path's per-block xxHash32 strong checksum (low 32 bits of the seed). An explicit seed deterministically changes every computed digest on BOTH endpoints (sender and receiver share the seed via the config frame, protocol 2.10.0), so identical runs with the same seed skip the same files and a changed seed changes the digests — the explicit-seed path that makes xxHash comparisons deterministic. `--checksum-choice=md5` has no seed and ignores it (documented). The value is a strict decimal 0..2⁶⁴-1 (blank, signed, or non-numeric values are rejected). Like rsync, a seed only matters where a digest is actually computed (`--checksum` or a basis-dir run, or a delta transfer); it does not by itself enable `--checksum`/`--delta`. Divergence from rsync: the default is seed 0, and FastSync never randomizes the seed (rsync uses a random per-transfer seed when `--checksum-seed` is unset); FastSync's unset default therefore reproduces its historical byte-for-byte behavior |
|
||
| `--secluded-args` | Use protocol to send args | 🔄 Compatibility No-op | Accepted for CLI compatibility; it does not change FastSync transport or protocol behavior. `-s` remains chunk serialization. |
|
||
| `--no-OPTION` | Turn off implied option | ✅ Supported | Supported boolean FastSync options and archive-implied options; unsafe or value-taking options are rejected. |
|
||
|
||
---
|
||
|
||
## Implementation Difficulty Plan
|
||
|
||
The estimates below cover the currently unimplemented features in this document. They assume one engineer familiar with the codebase, include implementation and focused tests, and exclude production rollout time. A feature should not be marked implemented until its behavior is tested in both local and SSH/TCP paths where applicable.
|
||
|
||
> **Note:** This plan is a superset snapshot written while several of the listed features were still outstanding. The Summary matrix above is the authoritative record of what is already shipped (for example quiet/info/debug output, `--existing`, `--remove-source-files`, `-h`, and `--size-only` are now implemented on `dev`). Treat the phases as sequencing guidance for the work that remains unimplemented.
|
||
|
||
| Effort | Typical duration | Meaning |
|
||
|--------|------------------|---------|
|
||
| XS | 0.5-1 day | CLI alias or a local formatting/validation change |
|
||
| S | 1-3 days | Isolated behavior with little or no protocol change |
|
||
| M | 3-7 days | Cross-cutting client, server, or scanner behavior |
|
||
| L | 1-3 weeks | Protocol, filesystem, privilege, or compatibility work |
|
||
| XL | 3+ weeks | New transfer mode, daemon subsystem, or broad interoperability effort |
|
||
|
||
### Phase 1: Low-Risk CLI and Local Behavior
|
||
|
||
These are the best first changes because they require limited wire-format work and can be tested with existing transfer fixtures.
|
||
|
||
| Features | Effort | Implementation plan |
|
||
|----------|--------|--------------------|
|
||
| `--quiet`, `-q`; `--human-readable`, `-h`; `--8-bit-output`, `-8`; `--stderr=MODE`; `--info=FLAGS`; `--debug=FLAGS` | S | Extend logging and output formatting without changing transferred data. |
|
||
| `--no-OPTION`; `--old-args`; `--secluded-args`, `-s` | M | Add option implication/negation and safely serialize or protect remote arguments. `-s` currently has FastSync-specific semantics and needs a compatibility decision. |
|
||
| `-P`; `--del`; `--old-dirs`, `--old-d`; `--cc`; `--zc`; `--zl` | XS | Add aliases and composed behaviors after the underlying options exist. |
|
||
| `--whole-file`, `-W`; `--ignore-times`, `-I`; `--size-only`; `--modify-window`, `-@`; `--update`, `-u` | S | Extend the existing incremental comparison decision. |
|
||
| `--existing`; `--ignore-existing`; `--remove-source-files` | S | Add scanner/receiver eligibility checks and remove successfully synchronized source files. |
|
||
| `--executability`, `-E`; `--chmod=CHMOD` | M | Apply permission transformations safely while preserving current metadata behavior. |
|
||
| `--skip-compress=LIST`; `--compress-threads=NUM` | S | Make compression selection configurable and validate the thread setting against zstd behavior. |
|
||
| `--max-alloc=SIZE`; `--fsync` | S | Reuse existing allocation limits and add an explicit durability step after file writes. |
|
||
|
||
### Phase 2: Filesystem Selection and Update Semantics
|
||
|
||
These features are moderate because they affect traversal, temporary files, manifests, or the receiver's update policy.
|
||
|
||
| Features | Effort | Implementation plan |
|
||
|----------|--------|--------------------|
|
||
| `--one-file-system`, `-x` | M | Track the source device during scanner traversal and skip mount-point crossings. |
|
||
| `--relative`, `-R`; `--no-implied-dirs`; `--dirs`, `-d`; `--mkpath` | M | Extend path-list construction and destination directory creation while preserving traversal safety. |
|
||
| `--temp-dir`, `-T` | M | Separate temporary-file placement from FastSync's timeout alias and define collision, permissions, and cleanup rules. |
|
||
| `--delay-updates` | L | Stage all successful updates and publish them at completion, including crash and cancellation cleanup. |
|
||
| `--files-from=FILE`; `--from0`, `-0`; `--filter=RULE`, `-f`; `-F`; `--cvs-exclude`, `-C` | L | Build a complete filter/parser layer and integrate it with scanner pruning, manifests, and delete behavior. `-f` conflicts with FastSync sendfile mode. |
|
||
| `--list-only`; `--itemize-changes`, `-i`; `--out-format=FORMAT`; `--log-file-format=FMT` | M | Add a structured change-event model so output modes share one source of truth. |
|
||
|
||
### Phase 3: Deletion, Comparison, and Delta Compatibility
|
||
|
||
These features require careful interaction with manifests, incremental checks, backups, and the existing delta protocol.
|
||
|
||
| Features | Effort | Implementation plan |
|
||
|----------|--------|--------------------|
|
||
| `--delete-during`; `--delete-before`; `--delete-after`; `--delete-delay`; `--del` | L | Add deletion timing to the transfer state machine and ensure failures cannot remove files unexpectedly. |
|
||
| `--delete-excluded`; `--max-delete=NUM`; `--ignore-errors`; `--force`; `--prune-empty-dirs`, `-m` | M | Extend delete walks with policy limits, error handling, empty-directory pruning, and the `-m` short-flag conflict. |
|
||
| `--ignore-missing-args`; `--delete-missing-args` | M | Distinguish missing source arguments from traversal errors and apply explicit deletion policy. |
|
||
| `--compare-dest=DIR`; `--copy-dest=DIR`; `--link-dest=DIR` | L | Add alternate basis roots and hard-link handling, including metadata and cross-filesystem failures. |
|
||
| `--fuzzy`, `-y`; `--no-fuzzy` | L | Index candidate files and select a safe similar basis without making transfer time unbounded. |
|
||
| `--append`; `--append-verify` | M | Negotiate file length and verify the retained prefix before resuming. |
|
||
| `--checksum-choice=STR`, `--cc`; `--checksum-seed=NUM` | M | Negotiate checksum algorithms/seeds and preserve compatibility with existing xxHash checks. |
|
||
|
||
### Phase 4: Metadata, Links, and Devices
|
||
|
||
These features are platform-sensitive and need Linux permission, ACL, xattr, and special-file integration tests.
|
||
|
||
| Features | Effort | Implementation plan |
|
||
|----------|--------|--------------------|
|
||
| `--numeric-ids`; `--usermap=STRING`; `--groupmap=STRING`; `--chown=USER:GROUP` | L | Define identity mapping, privilege failures, and wire representation before applying ownership. |
|
||
| `--open-noatime`; `--atimes`, `-U`; `--crtimes`, `-N`; `--omit-dir-times`, `-O`; `--omit-link-times`, `-J` | L | Extend metadata capture/apply with platform capability checks and explicit unsupported-attribute handling. |
|
||
| `--acls`, `-A`; `--xattrs`, `-X`; `--fake-super` | XL | Add portable serialization, size limits, privilege behavior, and security tests for ACL/xattr data. |
|
||
| `--hard-links`, `-H` | L | Preserve inode relationships across the file list and coordinate hard-link creation order. |
|
||
| `--munge-links`; `--copy-dirlinks`, `-k`; `--keep-dirlinks`, `-K` | L | Define symlink trust boundaries and receiver-side directory/link collision behavior. |
|
||
| `--devices`; `--specials`; `-D`; `--copy-devices`; `--write-devices` | XL | Add privileged special-file handling with strict type, path, and authorization checks. |
|
||
| `--super`; `--copy-as=USER[:GROUP]` | XL | Requires a deliberate privilege model, identity switching, and refusal paths; do not implement by blindly elevating the process. |
|
||
| `--preallocate` | S | Use platform allocation APIs before writes and fall back cleanly when unsupported. |
|
||
|
||
### Phase 5: Connectivity and Daemon Compatibility
|
||
|
||
These options affect process startup, authentication, sockets, and remote execution. They should follow the filesystem and protocol work rather than being added as parser-only flags.
|
||
|
||
| Features | Effort | Implementation plan |
|
||
|----------|--------|--------------------|
|
||
| `--rsh=COMMAND`, `-e`; `--rsync-path=PROGRAM`; `--blocking-io`; `--outbuf=N\|L\|B` | M | Generalize SSH command construction and subprocess I/O while retaining argument escaping and timeout guarantees. |
|
||
| `--address=ADDRESS`; `--ipv4`, `-4`; `--ipv6`, `-6`; `--sockopts=OPTIONS`; `--port=PORT` daemon semantics | M | Add explicit socket-family/bind configuration and validate it independently for TCP client and daemon modes. |
|
||
|
||
**Phase 5, Wave B (socket/bind) shipping note:** `--sockopts` adds a strict allowlisted `OPT=VAL` socket-option layer applied with correct per-option value types; `--address` binds the outgoing client socket to a local source address; `-4`/`-6` pin the address family via `getaddrinfo` hints on both the client connect and the server bind; and the server bind now honors `--address` plus `-4`/`-6` (falling back to the historical IPv4 `INADDR_ANY` when none are given). All of these are local socket concerns and none cross the wire config frame (only `--port` maps to `server_port`).
|
||
| `--remote-option=OPT`, `-M`; `--trust-sender` | L | Add authenticated remote-option/config negotiation and reject unsafe sender-controlled values. `-M` conflicts with FastSync metadata mode. |
|
||
| `--daemon`; `--config=FILE`; `--dparam=OVERRIDE`; `--no-detach`; `--password-file=FILE`; `--early-input=FILE`; `--no-motd` | XL | Implement a real daemon lifecycle, module configuration, authentication, privilege separation, and process management. |
|
||
|
||
### Phase 6: Batch, Encoding, and Protocol Interoperability
|
||
|
||
These are the hardest compatibility items because they require durable formats or behavior that must interoperate with rsync itself.
|
||
|
||
| Features | Effort | Implementation plan |
|
||
|----------|--------|--------------------|
|
||
| `--write-batch=FILE`; `--only-write-batch=FILE`; `--read-batch=FILE` | XL | Specify a versioned batch format, persist all required metadata, and test replay, corruption, and partial application. |
|
||
| `--protocol=NUM` | XL | Add protocol-version negotiation and compatibility branches without weakening current validation. |
|
||
| `--iconv=CONVERT_SPEC` | L | Convert filenames at the protocol boundary with invalid-sequence and normalization tests. |
|
||
| `--stop-after=MINS`; `--stop-at=TIME` | M | Add deadline propagation, interruptible I/O, and safe checkpoint/cleanup behavior. |
|
||
| `--early-input=FILE`; `--password-file=FILE` | M | Securely read startup credentials/input with permission checks and no secret disclosure in logs. |
|
||
|
||
### Recommended Delivery Order
|
||
|
||
1. Resolve short-option conflicts (`-m`, `-M`, `-T`, `-f`, `-s`) and define the compatibility contract.
|
||
2. Implement Phase 1 comparison, update, output, and alias features with unit and integration coverage.
|
||
3. Implement Phase 2 traversal/filtering and Phase 3 deletion semantics.
|
||
4. Implement metadata and link features that are safe on the supported platforms.
|
||
5. Treat daemon mode, special files, batch mode, and protocol-version compatibility as separate projects.
|
||
|
||
The existing priority list below is a feature shortlist, not an implementation schedule; this plan supersedes it for effort and sequencing.
|
||
|
||
---
|
||
|
||
## Recommendations: Top Features to Implement Next
|
||
|
||
Ranked by user demand, implementation complexity, and interoperability impact:
|
||
|
||
| Priority | Feature | Effort | Impact |
|
||
|----------|---------|--------|--------|
|
||
| 1 | `--whole-file` / `-W` | Low | High — users expect opt-out of delta |
|
||
| 2 | `--ignore-times` / `-I` | Low | Medium — useful for forcing re-transfer |
|
||
| 3 | `--size-only` | Low | Medium — common migration scenario |
|
||
| 4 | `--ignore-existing` | Low | Medium — common sync patterns |
|
||
| 5 | `--existing` | Low | Medium — common sync patterns |
|
||
| 6 | `--remove-source-files` | Low | High — common for moves/backup |
|
||
| 7 | `--delete-during` | Medium | High — performance improvement |
|
||
| 8 | `--delay-updates` | Medium | High — atomic updates |
|
||
| 9 | `--chmod` | Low | Medium — permission flexibility |
|
||
| 10 | `--executability` / `-E` | Low | Low — simple flag |
|
||
| 11 | `--skip-compress` | Low | Medium — performance tuning |
|
||
|
||
---
|
||
|
||
## FastSync-Specific Features (Not in rsync)
|
||
|
||
| Feature | Description |
|
||
|---------|-------------|
|
||
| `-m` | Multithreaded pipeline (scanner/loader/sender) |
|
||
| `-s` | Chunk serialization mode |
|
||
| `-f` / `--sendfile` | Zero-copy sendfile() syscall (TCP only) |
|
||
| `-c [level]` | zstd compression level (1-22) |
|
||
| `--chunk-size` | Configurable chunk size |
|
||
| `--tls` | TLS encryption (mutual auth) |
|
||
| `--fastsync-server-path` | Path to fastsync-server binary |
|
||
| `--server-host` / `--server-port` | Direct TCP connection |
|
||
| Incremental sync | Skip unchanged files (size+mtime) |
|
||
| Delta transfer | Block-level delta for changed files |
|