From 8ec8cb7203c573f49dc3961a615b7bfb5e78445d Mon Sep 17 00:00:00 2001 From: TapTap Date: Sat, 19 Sep 2026 13:53:50 +0200 Subject: [PATCH] test(parity): dest-only excluded entry protected under default --delete Adds the exclude_protect_dest_only differential and flips --delete-excluded to parity now that receiver-side rules protect a destination-only excluded entry like rsync; tally 116/10/31. --- RSYNC_COMPAT.md | 6 +++--- tests/integration/test_differential_parity.py | 4 ++++ tmpd4b/fd/workspace/tmpd4b/src/dest_only.log | 1 + tmpd4b/fd/workspace/tmpd4b/src/keep.txt | 1 + tmpd4b/rd/dest_only.log | 1 + tmpd4b/rd/keep.txt | 1 + tmpd4b/src/extra.log | 1 + tmpd4b/src/keep.txt | 1 + 8 files changed, 13 insertions(+), 3 deletions(-) create mode 100644 tmpd4b/fd/workspace/tmpd4b/src/dest_only.log create mode 100644 tmpd4b/fd/workspace/tmpd4b/src/keep.txt create mode 100644 tmpd4b/rd/dest_only.log create mode 100644 tmpd4b/rd/keep.txt create mode 100644 tmpd4b/src/extra.log create mode 100644 tmpd4b/src/keep.txt diff --git a/RSYNC_COMPAT.md b/RSYNC_COMPAT.md index db98926..1afaff1 100644 --- a/RSYNC_COMPAT.md +++ b/RSYNC_COMPAT.md @@ -6,8 +6,8 @@ This document maps rsync's full feature set to FastSync's current implementation | Status | Count | Description | |--------|-------|-------------| -| ✅ Parity | 115 | Reproduces rsync's semantics for this option's scope | -| ⚠️ Caveat | 11 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) | +| ✅ Parity | 116 | Reproduces rsync's semantics for this option's scope | +| ⚠️ Caveat | 10 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) | | ❌ Divergent | 31 | Rejected, an accepted no-op, deliberately non-rsync (native config/auth/batch, privileged namespaces, safe-subset privilege), or impossible on any portable filesystem call | | **Total** | **157** | One row per rsync option/feature group; a row may name several spellings | @@ -168,7 +168,7 @@ Every one of those has an entry below with its remaining caveats. | `--del`, `--delete-during` | Delete during transfer | ⚠️ Caveat | Both spellings accepted; imply `--delete`. **Protocol 2.24.0 implements per-directory delete plans:** as the sender finishes each source directory it streams a `STATUS_DELETE_PLAN` for that directory and the receiver removes that directory's extras before applying the next directory's data, so a mid-transfer failure has already removed the extras of the directories reached (verified with a byte-slicing proxy). **Remaining divergence:** the exact abort boundary and the progressive ordering of removals versus rsync's generator can differ, and `-d`/`--dirs` (no descent) falls back to the end-of-transfer commit. `-R` plans are scoped to the transferred prefix subtree | | `--delete-delay` | Find deletions during, delete after | ⚠️ Caveat | Implies `--delete`. **Protocol 2.24.0 implements rsync's delete-delay timing:** the sender records each directory's delete plan while scanning and the receiver commits those removals only after the whole transfer succeeds (per plan), so an extra created in the destination after its directory's plan survives while `--delete-after` re-scans and removes it, and a failed transfer removes nothing. The **reported** deleted count advances only on an actual removal. **Fixed (no-wire):** the `--max-delete` budget is now charged on ACTUAL removals (an unlink/rmdir that succeeded), not at plan/snapshot time, and a queued directory is re-scanned at commit and removed recursively (content created after the plan included), matching rsync: a snapshotted entry that fails or is skipped consumes no budget, so a later extra rsync would delete is still deleted. The deferred snapshot list keeps an independent hard cap (`DELETE_PLAN_SERVER_LIMIT`) so it cannot grow without bound now that the budget is no longer charged while scanning. A `--max-delete=2` partial delete reports exactly 2 and exits 25 in both tools, and the refilled-directory differential (late content removed, directory removed, budget shared) now matches rsync 3.4.1 on both sides (`test_delete_delay_budget_parity.py`, `test_delete_timing_parity.py`). Unit tests cover recursive removal, actual-removal charging, and the bounded deferred list. **Caveat:** the exact ordering of which extras are removed first under a partial `--max-delete` can still differ from rsync's generator (the survivor set is compared by count, not identity) | | `--delete-after` | Delete after transfer | ✅ Parity | Implies `--delete`. The delete-after timing is also what plain `--delete` does: the keep-set manifest closes the data stream and the receiver commits the bounded deletion only after the terminal `STATUS_FINISHED` proves the whole transfer (every data frame received and stored) succeeded. A failed or aborted transfer removes nothing | -| `--delete-excluded` | Also delete excluded files | ⚠️ Caveat | `delete_excluded` config field. Under `--delete` FastSync protects (rsync's default) the destination mirror of paths the sender's source scan pruned by the user-selection rules — the `--filter`/`-F`/`-C` layer and the legacy `--exclude`/`--include` layer. The sender transmits those concrete pruned paths as **protected prefixes** in the delete-manifest frame (see the Phase-3 notes below); the walker never descends into or removes them. `--delete-excluded` opts back in: the sender sends an empty protected list, so the excluded destination mirrors become ordinary extras and are removed. **`--max-size`/`--min-size` pruned mirrors are a separate, always-on protection** (protocol 2.23.0, rsync parity): size-pruned source mirrors survive `--delete` even with `--delete-excluded`. Track 4a (protocol 2.28.0) additionally re-applies the received `protect`/`risk` rules on the receiver, so a destination-only entry matching an exclude rule is protected (or left at risk) exactly like rsync; the remaining sender-derived `--delete-excluded` behavior (an unqualified rule becomes sender-only, so its source mirror and matching destination-only extras are deleted) is unchanged | +| `--delete-excluded` | Also delete excluded files | ✅ Parity | `delete_excluded` config field. Under `--delete` FastSync protects (rsync's default) the destination mirror of paths the sender's source scan pruned by the user-selection rules — the `--filter`/`-F`/`-C` layer and the legacy `--exclude`/`--include` layer. The sender transmits those concrete pruned paths as **protected prefixes** in the delete-manifest frame (see the Phase-3 notes below); the walker never descends into or removes them. `--delete-excluded` opts back in: the sender sends an empty protected list, so the excluded destination mirrors become ordinary extras and are removed. **`--max-size`/`--min-size` pruned mirrors are a separate, always-on protection** (protocol 2.23.0, rsync parity): size-pruned source mirrors survive `--delete` even with `--delete-excluded`. Track 4a (protocol 2.28.0) additionally re-applies the received `protect`/`risk` rules on the receiver, so a destination-only entry matching an exclude rule is protected (or left at risk) exactly like rsync; the remaining sender-derived `--delete-excluded` behavior (an unqualified rule becomes sender-only, so its source mirror and matching destination-only extras are deleted) is unchanged | | `--max-delete=NUM` | Max files to delete | ✅ Parity | `max_delete` config field (default -1 = no client limit; 0 = delete nothing). **Protocol 2.23.0 matches rsync's partial semantics:** the receiver deletes up to NUM entries (regular files, symlinks and empty directories; each directory removal counts as one) and then **stops deleting, skips the rest, and reports the run as partial**. The client prints a "deletions stopped due to `--max-delete` limit" message and exits **25** (rsync's `RERR_PARTIAL`), not a hard failure — the transfer itself succeeded. NUM only applies together with `--delete` (it is inert otherwise, matching rsync). A client NUM below the server hard bound `MAX_SERVER_DELETE_COUNT` (100000) replaces it; a NUM above it never raises that cap. Deleting an entire destination with no limit is still bounded by the server's 100000-entry ceiling. `--delete-missing-args` exact-path deletions and the ordinary extras walk draw from the same budget, matching rsync | | `--ignore-errors` | Delete even with I/O errors | ✅ Parity | Sender-side, client-only config field. Matches rsync's semantics exactly: an unreadable source subdirectory is always skipped so the readable tree transfers (the transfer root itself stays fatal), and the run reports rsync's partial-transfer exit **23**. Deletion policy follows rsync: by default an I/O error suppresses deletion (`IO error encountered -- skipping file deletion`), while `--ignore-errors` lets the deletion commit. The decision applies to every timing (`--delete`, `--delete-before`, `--delete-during`, `--delete-delay`, `--delete-after`) in both the sequential and `--threads` send paths. Differential-tested against rsync 3.4.1 with both tools run as an unprivileged user (mode-000 source directory); the reference build's root-only gate still excludes the EACCES differential, but the setpriv differential test exercises it. The piece that stays FastSync-specific is documented under the recursive-empty-directory residual: FastSync never emits an unreadable (or empty) directory entry, so that mirror is an extra that a run with `--ignore-errors` removes, where rsync emits the directory and keeps its mirror | | `--force` | Force deletion of non-empty dirs | ✅ Parity | `force_delete` receiver config field (crosses the wire). rsync's `--force` lets an incoming non-directory replace a destination directory; FastSync implements exactly that: when a regular file (or symlink) is written to a path that is currently a (possibly non-empty) destination directory, `--force` removes that directory tree first — confined to the receive root and symlink-safe (O_NOFOLLOW fd walk, symlinks removed by name, never followed) — so the install can place the file. **Protocol 2.23.0 honors `--force` on the `--delay-updates` publication path too**, not only the immediate-install path. Without `--force` such a write fails and the run aborts. Gated by the server `--allow-delete` policy (a client cannot use `--force` to remove a destination tree on a server that forbids deletion) | diff --git a/tests/integration/test_differential_parity.py b/tests/integration/test_differential_parity.py index a36ce04..d3beed5 100644 --- a/tests/integration/test_differential_parity.py +++ b/tests/integration/test_differential_parity.py @@ -221,6 +221,10 @@ _CASES = [ H.Case("delete_excluded", "filters", ["-a", "--delete", "--delete-excluded", "--exclude=*.log"], seed=seed_delete_excluded, server_args=DELETE, ref="--delete-excluded"), + H.Case("exclude_protect_dest_only", "filters", + ["-a", "--delete", "--exclude=*.log"], + seed=seed_delete_excluded, server_args=DELETE, ci=True, + ref="--delete protects a destination-only excluded entry like rsync"), H.Case("max_delete", "basic", ["-a", "--delete", "--max-delete=1"], seed=seed_max_delete, server_args=DELETE, extra_check=max_delete_count_check, compare_tree=False, diff --git a/tmpd4b/fd/workspace/tmpd4b/src/dest_only.log b/tmpd4b/fd/workspace/tmpd4b/src/dest_only.log new file mode 100644 index 0000000..89ed435 --- /dev/null +++ b/tmpd4b/fd/workspace/tmpd4b/src/dest_only.log @@ -0,0 +1 @@ +destone diff --git a/tmpd4b/fd/workspace/tmpd4b/src/keep.txt b/tmpd4b/fd/workspace/tmpd4b/src/keep.txt new file mode 100644 index 0000000..2fa992c --- /dev/null +++ b/tmpd4b/fd/workspace/tmpd4b/src/keep.txt @@ -0,0 +1 @@ +keep diff --git a/tmpd4b/rd/dest_only.log b/tmpd4b/rd/dest_only.log new file mode 100644 index 0000000..89ed435 --- /dev/null +++ b/tmpd4b/rd/dest_only.log @@ -0,0 +1 @@ +destone diff --git a/tmpd4b/rd/keep.txt b/tmpd4b/rd/keep.txt new file mode 100644 index 0000000..2fa992c --- /dev/null +++ b/tmpd4b/rd/keep.txt @@ -0,0 +1 @@ +keep diff --git a/tmpd4b/src/extra.log b/tmpd4b/src/extra.log new file mode 100644 index 0000000..0f22871 --- /dev/null +++ b/tmpd4b/src/extra.log @@ -0,0 +1 @@ +extra diff --git a/tmpd4b/src/keep.txt b/tmpd4b/src/keep.txt new file mode 100644 index 0000000..2fa992c --- /dev/null +++ b/tmpd4b/src/keep.txt @@ -0,0 +1 @@ +keep