feat: --ignore-missing-args / --delete-missing-args #271

Closed
TapTap wants to merge 0 commits from feat/p3-missing-args into dev
Owner

Implements rsync's --ignore-missing-args and --delete-missing-args (Phase-3 row; RSYNC_COMPAT §15 rows 302-303).

Reference semantics (verified against the rsync man page)

  • --ignore-missing-args: a source argument that does not exist is silently ignored; the run succeeds. Without it, a missing explicit source argument fails the run.
  • --delete-missing-args: implies --ignore-missing-args; each missing argument becomes a deletion request of its destination counterpart. Per the man page it is "independent of any other type of delete processing" — it does not imply --delete — and a non-empty directory counterpart is only removed with --force or --delete in effect.

Design summary

FastSync's "explicitly requested source arguments" are the --files-from entries (there is one always-present source root, so the flags are inert without --files-from, like -R).

  • Default unchanged: a listed-but-missing entry is a hard, pre-transfer error.
  • --ignore-missing-args: missing entries are skipped (logged per entry + a per-run warning count — never a silent no-op), not sent, never in the keep-set; the run succeeds for the rest, including the all-missing case (rsync parity). An empty --files-from file stays a hard error in every mode. --dirs+files-from missing entries are skipped too.
  • --delete-missing-args: implies ignore; each missing entry's destination mirror (bare relative path under -R, otherwise the full source-mirror path below the destination root — identical to a present sibling's wire path) is removed receiver-side as an explicit deletion request. Composes with delete timing (--delete-before/--delete-during = early commit, else delete-after/commit) and with --force/--delete for non-empty directory mirrors. It is never blocked by the filter-exclusion protection of excluded destination mirrors (an explicit user request, not an excluded file), and unrelated extras stay unless --delete is also present.

Protocol / wire

  • STATUS_MANIFEST now carries a third section: destination-relative exact-delete paths (validated like the keep-set: non-empty/relative/traversal-free; per-section MAX_MANIFEST_ENTRIES, shared MAX_MANIFEST_BYTES).
  • Config frame gained a delete_missing_args boolean (ignore_missing_args is client-only, like ignore_errors).
  • PROTOCOL_VERSION bumped 2.9.0 → 2.10.0 (peers must match).
  • Server --allow-delete gates it like --delete; --delay-updates staging dir and basis snapshots stay protected; deletions are committed via manifest_delete_all() (exact-path deletions first, extras walk second).

Tests

  • Unit: CLI parse + implies (delete-missing implies ignore, not delete; valid with --delete and timing flags); delete_missing_args wire round-trip (ignore confirmed client-only); three-section manifest round-trip + traversal rejection; manifest_delete_missing_args semantics; commit-time parking. Existing two-section manifest frames updated.
  • Integration (TestMissingArgs, parametrized single-threaded vs -m): default hard error before any transfer; ignore transfers the rest; all-missing transfers nothing; empty list still errors; delete-missing removes exactly the mirror and leaves unrelated extras unless --delete; full-source-mirror (non--R) path; filter-exclusion protection never blocks the explicit deletion; --delete-before early timing.
  • Local: unit 27/27, integration 255 (features+preflight+tcp) + TLS 4, ASan + UBSan unit runs green, strict -Werror build green in the CI image, clang-format + cppcheck clean in the CI image.

Docs

RSYNC_COMPAT.md: both §15 rows marked ✅ with precise Notes; --files-from row and the Phase-3 deletion notes updated (wire note documents the 2.10.0 bump). Summary count line intentionally left for a recount commit.

Implements rsync's `--ignore-missing-args` and `--delete-missing-args` (Phase-3 row; RSYNC_COMPAT §15 rows 302-303). ## Reference semantics (verified against the rsync man page) - `--ignore-missing-args`: a source argument that does not exist is silently ignored; the run succeeds. Without it, a missing explicit source argument fails the run. - `--delete-missing-args`: implies `--ignore-missing-args`; each missing argument becomes a deletion request of its destination counterpart. Per the man page it is "independent of any other type of delete processing" — it does **not** imply `--delete` — and a non-empty directory counterpart is only removed with `--force` or `--delete` in effect. ## Design summary FastSync's "explicitly requested source arguments" are the `--files-from` entries (there is one always-present source root, so the flags are inert without `--files-from`, like `-R`). - Default unchanged: a listed-but-missing entry is a hard, pre-transfer error. - `--ignore-missing-args`: missing entries are skipped (logged per entry + a per-run warning count — never a silent no-op), not sent, never in the keep-set; the run succeeds for the rest, including the all-missing case (rsync parity). An **empty** `--files-from` file stays a hard error in every mode. `--dirs`+files-from missing entries are skipped too. - `--delete-missing-args`: implies ignore; each missing entry's destination mirror (bare relative path under `-R`, otherwise the full source-mirror path below the destination root — identical to a present sibling's wire path) is removed receiver-side as an explicit deletion request. Composes with delete timing (`--delete-before`/`--delete-during` = early commit, else delete-after/commit) and with `--force`/`--delete` for non-empty directory mirrors. It is **never** blocked by the filter-exclusion protection of excluded destination mirrors (an explicit user request, not an excluded file), and unrelated extras stay unless `--delete` is also present. ## Protocol / wire - `STATUS_MANIFEST` now carries a **third section**: destination-relative exact-delete paths (validated like the keep-set: non-empty/relative/traversal-free; per-section `MAX_MANIFEST_ENTRIES`, shared `MAX_MANIFEST_BYTES`). - Config frame gained a `delete_missing_args` boolean (`ignore_missing_args` is client-only, like `ignore_errors`). - `PROTOCOL_VERSION` bumped **2.9.0 → 2.10.0** (peers must match). - Server `--allow-delete` gates it like `--delete`; `--delay-updates` staging dir and basis snapshots stay protected; deletions are committed via `manifest_delete_all()` (exact-path deletions first, extras walk second). ## Tests - Unit: CLI parse + implies (delete-missing implies ignore, not delete; valid with `--delete` and timing flags); `delete_missing_args` wire round-trip (ignore confirmed client-only); three-section manifest round-trip + traversal rejection; `manifest_delete_missing_args` semantics; commit-time parking. Existing two-section manifest frames updated. - Integration (`TestMissingArgs`, parametrized single-threaded vs `-m`): default hard error before any transfer; ignore transfers the rest; all-missing transfers nothing; empty list still errors; delete-missing removes exactly the mirror and leaves unrelated extras unless `--delete`; full-source-mirror (non-`-R`) path; filter-exclusion protection never blocks the explicit deletion; `--delete-before` early timing. - Local: unit 27/27, integration 255 (features+preflight+tcp) + TLS 4, ASan + UBSan unit runs green, strict `-Werror` build green in the CI image, clang-format + cppcheck clean in the CI image. ## Docs RSYNC_COMPAT.md: both §15 rows marked ✅ with precise Notes; `--files-from` row and the Phase-3 deletion notes updated (wire note documents the 2.10.0 bump). Summary count line intentionally left for a recount commit.
TapTap added 4 commits 2026-09-07 14:42:47 +02:00
PROTOCOL_VERSION 2.9.0 -> 2.10.0. The STATUS_MANIFEST frame gains a third
section carrying destination-relative exact-delete paths (the missing
--files-from entries' mirrors); the config frame gains a delete_missing_args
bool (ignore_missing_args stays client-only). The receiver validates the third
section like the keep-set and commits it with manifest_delete_all():
manifest_delete_missing_args runs first (explicit user requests, never blocked
by protected-prefix exclusion protection; staging/basis protected; a non-empty
directory mirror removed only under --force/--delete, rsync parity) and then the
ordinary extras walk. Server --allow-delete gates it like --delete.
New flags parse onto the config; --delete-missing-args implies
--ignore-missing-args (order-independent) and does NOT imply --delete (rsync:
independent of other delete processing). The files-from preflight now classifies
listed-but-missing entries instead of hard-failing: under the flags each is
skipped (logged + counted, never silent) and the run succeeds for the rest,
including the all-missing case; an empty list stays a hard error. Under
--delete-missing-args the missing entries' destination mirrors (bare relative
path with -R, full source mirror otherwise) ride the manifest's third section in
both the single-threaded and -m senders; --dirs listed-but-missing entries are
skipped in the scanner.
Unit: CLI parse + imply relationships (delete-missing implies ignore, not
delete; valid with --delete and delete timing); delete_missing_args config wire
round-trip (ignore_missing_args confirmed client-only); three-section manifest
round-trip and third-section traversal rejection; manifest_delete_missing_args
exact-path semantics; commit-time parking. Existing two-section manifest frames
updated to the three-section format. Integration: default missing entry is a
hard pre-transfer error; --ignore-missing-args transfers the rest and succeeds
(all-missing transfers nothing; empty list still errors); --delete-missing-args
removes exactly the missing mirror and leaves unrelated extras unless --delete is
also present; filter-exclusion protection never blocks the explicit deletion;
--delete-before early timing composes; all parametrized single-threaded vs -m.
docs: mark --ignore-missing-args / --delete-missing-args implemented
CI / lint (pull_request) Successful in 37s
CI / sanitizers (undefined) (pull_request) Successful in 52s
CI / sanitizers (address) (pull_request) Successful in 53s
CI / fuzz-build (pull_request) Successful in 18s
CI / coverage (pull_request) Successful in 42s
CI / valgrind (pull_request) Successful in 37s
CI / build-and-test (pull_request) Successful in 14m17s
0feb545008
Precise Notes on the arg model (files-from entries), implies-relationships
(delete-missing implies ignore, independent of --delete), delete/timing/force/
exclude-protection interplay, the third manifest section and the 2.9.0 -> 2.10.0
wire bump, and the divergences (empty-list hard error, no negation, max-delete
not applied to explicit deletions). The files-from row now references the flags.
The Summary count line is intentionally left for a recount commit.
TapTap added 2 commits 2026-09-07 18:34:57 +02:00
BLOCKER: an absent mirror whose PARENT directory does not exist on the
destination (a deeper --files-from missing entry, -R or full-mirror layout) was
treated as a hard failure because file_open_secure_parent returned -1 when the
parent was missing. That aborted the whole run, skipped the --delete extras
walk, and tore down an early --delete-before/during connection, contradicting
'missing mirror = no-op / all-missing succeeds'. A parent-open failure is now a
no-op when errno is ENOENT/ENOTDIR (matching file_remove_tree_secure); only a
genuine I/O error fails the run.

Also: an already-absent mirror reached via unlinkat-ENOENT no longer prints
'Deleted: <path>' (a no-op dressed as a deletion); the per-path 'Deleted:' line
is printed only when an entry was actually removed.
test: cover absent-parent no-op and --dirs scanner skip
CI / lint (pull_request) Successful in 38s
CI / sanitizers (undefined) (pull_request) Successful in 51s
CI / sanitizers (address) (pull_request) Successful in 51s
CI / fuzz-build (pull_request) Successful in 19s
CI / coverage (pull_request) Successful in 43s
CI / valgrind (pull_request) Successful in 37s
CI / build-and-test (pull_request) Successful in 14m55s
e754f0d4eb
Unit: manifest_delete_missing_args treats a deeper missing entry whose
destination parent directory does not exist as a no-op (run continues, nothing
created). Integration (TestMissingArgs): a deeper missing entry with an absent
parent -R bare and full-mirror layouts, single-threaded and -m, no longer
aborts --delete (the extras walk still removes an unrelated extra); --dirs +
--files-from with --ignore-missing-args skips a listed-but-missing entry and
transfers the rest.
Author
Owner

Merged into dev via local merge (2FA blocks server-side merge). missing-args 681d7a8, append f99e6e1, checksum-choice 4c98744, recount 829e760. Independent c-reviews: missing-args REQUEST CHANGES (absent-parent no-op blocker fixed); append REQUEST CHANGES (blocker verified already-handled + tested, sentinel guard for streamed/-f sources); checksum-choice APPROVE WITH NITS (xxh3 alias removed, digest length pinned). dev CI run #494: all jobs success. Closing without server merge.

Merged into dev via local merge (2FA blocks server-side merge). missing-args 681d7a8, append f99e6e1, checksum-choice 4c98744, recount 829e760. Independent c-reviews: missing-args REQUEST CHANGES (absent-parent no-op blocker fixed); append REQUEST CHANGES (blocker verified already-handled + tested, sentinel guard for streamed/-f sources); checksum-choice APPROVE WITH NITS (xxh3 alias removed, digest length pinned). dev CI run #494: all jobs success. Closing without server merge.
TapTap closed this pull request 2026-09-07 20:25:58 +02:00

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: TapTap/FastSync#271