feat(parity): rsync parity tracks 1-6 (protocol 2.28.0) #303
+14
@@ -81,6 +81,20 @@
|
|||||||
integrated stats+options+fs branch stands at **111 ✅ / 13 ⚠️ / 33 ❌ = 157**;
|
integrated stats+options+fs branch stands at **111 ✅ / 13 ⚠️ / 33 ❌ = 157**;
|
||||||
full suite + ASan + clang-format + cppcheck clean.
|
full suite + ASan + clang-format + cppcheck clean.
|
||||||
|
|
||||||
|
11. **No-wire parity track 1** on `feat/parity-2.28` (no protocol change):
|
||||||
|
`-n --delete` now sends the same filter-excluded + size-pruned protected
|
||||||
|
prefixes and synchronized-directory scope as a real run (dry-run would-delete
|
||||||
|
matches rsync for source-derived protections; the destination-only exclude
|
||||||
|
residual and readdir ordering remain); `--delete-delay` now charges
|
||||||
|
`--max-delete` on actual removals and re-scans a queued directory at commit
|
||||||
|
to remove content created after the plan, with an independent deferred-list
|
||||||
|
cap (only partial-delete ordering remains); and `--info=name2` emits `NAME is
|
||||||
|
uptodate` plus the leading `./` root name line for `--info=name` (only the
|
||||||
|
root-line trigger condition and receiver-side `skip` wording remain). Matrix
|
||||||
|
now **111 ✅ / 14 ⚠️ / 32 ❌ = 157**; differential + unit tests added in
|
||||||
|
`test_features.py`, `test_option_parity.py`, `test_delete_plan.c`,
|
||||||
|
`test_delete_delay_budget_parity.py`, `test_delete_timing_parity.py`.
|
||||||
|
|
||||||
## Next steps
|
## Next steps
|
||||||
1. **Merge PR #284** (`dev` -> `main`) once reviewed (protected branch).
|
1. **Merge PR #284** (`dev` -> `main`) once reviewed (protected branch).
|
||||||
2. **Deferred security items** (documented, not implemented):
|
2. **Deferred security items** (documented, not implemented):
|
||||||
|
|||||||
+8
-6
@@ -7,8 +7,8 @@ This document maps rsync's full feature set to FastSync's current implementation
|
|||||||
| Status | Count | Description |
|
| Status | Count | Description |
|
||||||
|--------|-------|-------------|
|
|--------|-------|-------------|
|
||||||
| ✅ Parity | 111 | Reproduces rsync's semantics for this option's scope |
|
| ✅ Parity | 111 | Reproduces rsync's semantics for this option's scope |
|
||||||
| ⚠️ Caveat | 13 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
|
| ⚠️ Caveat | 14 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
|
||||||
| ❌ Divergent | 33 | Rejected, an accepted no-op, deliberately non-rsync (native config/auth/batch, privileged namespaces, safe-subset privilege), or impossible on any portable filesystem call |
|
| ❌ Divergent | 32 | Rejected, an accepted no-op, deliberately non-rsync (native config/auth/batch, privileged namespaces, safe-subset privilege), or impossible on any portable filesystem call |
|
||||||
| **Total** | **157** | One row per rsync option/feature group; a row may name several spellings |
|
| **Total** | **157** | One row per rsync option/feature group; a row may name several spellings |
|
||||||
|
|
||||||
This matrix reports honest rsync parity, not "implemented" as a synonym for
|
This matrix reports honest rsync parity, not "implemented" as a synonym for
|
||||||
@@ -37,6 +37,8 @@ combined `fix/parity-stats` + `fix/parity-options` + `fix/parity-fs` passes (and
|
|||||||
the later `fix/parity-review` correction that moved `--delete-delay` to ⚠️) the
|
the later `fix/parity-review` correction that moved `--delete-delay` to ⚠️) the
|
||||||
matrix is **111 ✅ / 13 ⚠️ / 33 ❌ = 157**.
|
matrix is **111 ✅ / 13 ⚠️ / 33 ❌ = 157**.
|
||||||
|
|
||||||
|
**Parity track 1 (no-wire, on `feat/parity-2.28`).** Three residual burn-downs that required no protocol change: (1) `-n --delete` now propagates the filter-excluded and size-pruned protected prefixes (and the synchronized-directory scope) into the dry-run keep-set manifest, so its would-delete report matches rsync for source-derived protections and no longer lists the file merely being updated (the destination-only exclude/filter residual and readdir ordering remain, leaving the row ⚠️); (2) `--delete-delay` now charges `--max-delete` on actual removals and re-scans a queued directory at commit to remove content created after the plan, with an independent cap bounding the deferred list and the row's prior caveat closed (ordering of partial removals remains, leaving the row ⚠️); and (3) `--info=name2` now emits `NAME is uptodate` and `--info=name` emits the leading `./` root name line (only the root-line trigger condition and the receiver-side `skip` wording remain, leaving the row ⚠️). The matrix is now **111 ✅ / 14 ⚠️ / 32 ❌ = 157**.
|
||||||
|
|
||||||
**Parity completion wave (protocol 2.23.0 → 2.26.0).** This wave closed the
|
**Parity completion wave (protocol 2.23.0 → 2.26.0).** This wave closed the
|
||||||
remaining gaps the rsync-parity wave left open (delete timing, wire counters and
|
remaining gaps the rsync-parity wave left open (delete timing, wire counters and
|
||||||
output, codec breadth, general `-R`/`-d`, the full filter grammar, receiver-side
|
output, codec breadth, general `-R`/`-d`, the full filter grammar, receiver-side
|
||||||
@@ -67,7 +69,7 @@ Every one of those has an entry below with its remaining caveats.
|
|||||||
| `-q`, `--quiet` | Suppress non-error messages | ✅ Parity | Suppresses client output while preserving errors |
|
| `-q`, `--quiet` | Suppress non-error messages | ✅ Parity | Suppresses client output while preserving errors |
|
||||||
| `--help` | Show help | ✅ Parity | Prints usage and exits. A lone `-h` with no other transfer arguments also prints help (protocol 2.26.0), matching the rsync idiom; `-h` alongside a transfer keeps its rsync meaning of `--human-readable` (see that row) |
|
| `--help` | Show help | ✅ Parity | Prints usage and exits. A lone `-h` with no other transfer arguments also prints help (protocol 2.26.0), matching the rsync idiom; `-h` alongside a transfer keeps its rsync meaning of `--human-readable` (see that row) |
|
||||||
| `-V`, `--version` | Print version | ✅ Parity | |
|
| `-V`, `--version` | Print version | ✅ Parity | |
|
||||||
| `--info=FLAGS` | Fine-grained info verbosity | ⚠️ Caveat | Accepts rsync 3.4.1's full `--info` vocabulary — `backup`, `copy`, `del`, `flist`, `misc`, `mount`, `name`, `nonreg`, `progress`, `remove`, `skip`, `stats`, `symsafe`, `all`, `none` — with optional level suffixes (`--info=stats2`), so a valid rsync invocation is never rejected up front. Protocol 2.27.0 wires the categories that map to a real FastSync event, matching rsync's line format: `name` prints the updated entry names (with the ` -> target` link suffix), `flist` prints `sending incremental file list`, `del` prints `deleting PATH` (or `*deleting PATH` under `-i`/`--out-format`) for both dry-run would-delete and real deletions (real runs carry the removed paths over the new `report_deletes` wire bool), `remove` prints `sender removed PATH`, `nonreg` prints `skipping non-regular file "NAME"`, `progress` drives the per-file progress output, and `copy`/`misc`/`skip`/`stats` keep their existing channels. `none` suppresses info output, explicit flags override `--verbose`, and a genuinely unknown name is still rejected by name (matching rsync). **Caveat:** the categories with no client-observable event stay accepted-but-silent — `symsafe`, `mount`, and `backup` (the backup happens on the receiver, which FastSync's protocol does not echo back); `name` level 2 (`is uptodate` lines) and the leading `./` root name line are not emitted; and `skip` maps to FastSync's sender-side skip logging rather than rsync's receiver-side "not creating new file" lines |
|
| `--info=FLAGS` | Fine-grained info verbosity | ⚠️ Caveat | Accepts rsync 3.4.1's full `--info` vocabulary — `backup`, `copy`, `del`, `flist`, `misc`, `mount`, `name`, `nonreg`, `progress`, `remove`, `skip`, `stats`, `symsafe`, `all`, `none` — with optional level suffixes (`--info=stats2`), so a valid rsync invocation is never rejected up front. Protocol 2.27.0 wires the categories that map to a real FastSync event, matching rsync's line format: `name` prints the updated entry names (with the ` -> target` link suffix), `flist` prints `sending incremental file list`, `del` prints `deleting PATH` (or `*deleting PATH` under `-i`/`--out-format`) for both dry-run would-delete and real deletions (real runs carry the removed paths over the new `report_deletes` wire bool), `remove` prints `sender removed PATH`, `nonreg` prints `skipping non-regular file "NAME"`, `progress` drives the per-file progress output, and `copy`/`misc`/`skip`/`stats` keep their existing channels. `none` suppresses info output, explicit flags override `--verbose`, and a genuinely unknown name is still rejected by name (matching rsync). **Fixed (no-wire):** `--info=name2` (and higher) also prints rsync's `NAME is uptodate` lines for entries the receiver already has, and `--info=name` emits the leading transfer-root `./` name line before the first transferred entry (the marker rides in the existing `info_level` bitset; differential tests vs rsync 3.4.1). **Caveat:** the root `./` line is emitted before the first transferred name rather than keyed off rsync's root-attribute-change decision, so a pre-existing root that rsync leaves untouched can differ; the categories with no client-observable event stay accepted-but-silent — `symsafe`, `mount`, and `backup` (the backup happens on the receiver, which FastSync's protocol does not echo back); and `skip` maps to FastSync's sender-side skip logging rather than rsync's receiver-side "not creating new file" lines |
|
||||||
| `--debug=FLAGS` | Fine-grained debug verbosity | ⚠️ Caveat | Protocol 2.26.0 accepts rsync 3.4.1's full `--debug` vocabulary with optional level suffixes. FastSync emits for its own channels (`io`, `proto`, `pack`, `util`, plus the aliases `hl`/`owner`); the rsync-only categories (`acl`, `filter`, `send`, ...) are accepted silently. `--debug=help` lists the flags; a genuinely unknown name is rejected by name. **Caveat:** most accepted rsync categories produce no output (e.g. `acl`, `filter`, `send`, `flist`, `del`, `deltasum`, `hash`, `recv`, `time`), so they are accepted for CLI compatibility only |
|
| `--debug=FLAGS` | Fine-grained debug verbosity | ⚠️ Caveat | Protocol 2.26.0 accepts rsync 3.4.1's full `--debug` vocabulary with optional level suffixes. FastSync emits for its own channels (`io`, `proto`, `pack`, `util`, plus the aliases `hl`/`owner`); the rsync-only categories (`acl`, `filter`, `send`, ...) are accepted silently. `--debug=help` lists the flags; a genuinely unknown name is rejected by name. **Caveat:** most accepted rsync categories produce no output (e.g. `acl`, `filter`, `send`, `flist`, `del`, `deltasum`, `hash`, `recv`, `time`), so they are accepted for CLI compatibility only |
|
||||||
| `--stderr=MODE` | Change stderr output mode | ❌ Divergent | `errors` (default) and `all` are supported; `client` is rejected with a clear error (`--stderr=client is not supported`) because FastSync has no rsync client-message channel — the rejection itself is the documented behavior (Phase 7 Wave B decision). The modes that exist work; the missing rsync channel cannot be emulated without a wire change |
|
| `--stderr=MODE` | Change stderr output mode | ❌ Divergent | `errors` (default) and `all` are supported; `client` is rejected with a clear error (`--stderr=client is not supported`) because FastSync has no rsync client-message channel — the rejection itself is the documented behavior (Phase 7 Wave B decision). The modes that exist work; the missing rsync channel cannot be emulated without a wire change |
|
||||||
| `--msgs2stderr`, `--no-msgs2stderr` | Deprecated `--stderr` aliases | ⚠️ Caveat | `--msgs2stderr` maps to `--stderr=all` (supported, matching rsync). `--no-msgs2stderr` is rsync's spelling of `--stderr=client`, which FastSync has no client-message channel for, so it maps to the errors-only default instead of reproducing rsync's client mode. See `--stderr=MODE` |
|
| `--msgs2stderr`, `--no-msgs2stderr` | Deprecated `--stderr` aliases | ⚠️ Caveat | `--msgs2stderr` maps to `--stderr=all` (supported, matching rsync). `--no-msgs2stderr` is rsync's spelling of `--stderr=client`, which FastSync has no client-message channel for, so it maps to the errors-only default instead of reproducing rsync's client mode. See `--stderr=MODE` |
|
||||||
@@ -138,7 +140,7 @@ Every one of those has an entry below with its remaining caveats.
|
|||||||
|
|
||||||
| Flag | Rsync Description | FastSync Status | Notes |
|
| Flag | Rsync Description | FastSync Status | Notes |
|
||||||
|------|-------------------|-----------------|-------|
|
|------|-------------------|-----------------|-------|
|
||||||
| `-n`, `--dry-run` | Trial run with no changes | ❌ Divergent | Server-contacting since protocol 2.21.0. The routing predicate `dry_run_targets_server()` selects the server-contacting path for any target a real run would reach over the wire (SSH, daemon `host::module`, explicit `--server-host`/`--server-port`, TLS, source-bind `--address`); the client handshakes with the receiver, which runs the normal read-only per-file check and answers `STATUS_DRY_RUN_TRANSFER`/`STATUS_OK` without mutating anything. Protocol 2.25.0 also reports would-delete lines: with `--delete` the receiver's `STATUS_STATS` carries the extras it would have removed and the client prints rsync-style `*deleting` lines (sequential and `--threads`; control bytes escaped). Dry-run never deletes. **Reclassified Divergent (differential evidence):** the would-delete report over-reports — it includes the file that is merely being updated (the receiver's extras walk does not see the would-be-transferred file in its keep-set) and, unlike rsync, also lists an excluded-but-protected extra under `--exclude`, and its line ordering differs from rsync's delete-during walk (`test_dry_run_delete_lines_over_report_residual`). A real run remains correct; only the dry-run report diverges |
|
| `-n`, `--dry-run` | Trial run with no changes | ⚠️ Caveat | Server-contacting since protocol 2.21.0. The routing predicate `dry_run_targets_server()` selects the server-contacting path for any target a real run would reach over the wire (SSH, daemon `host::module`, explicit `--server-host`/`--server-port`, TLS, source-bind `--address`); the client handshakes with the receiver, which runs the normal read-only per-file check and answers `STATUS_DRY_RUN_TRANSFER`/`STATUS_OK` without mutating anything. Protocol 2.25.0 also reports would-delete lines: with `--delete` the receiver's `STATUS_STATS` carries the extras it would have removed and the client prints rsync-style `*deleting` lines (sequential and `--threads`; control bytes escaped). Dry-run never deletes. **Fixed (no-wire):** the dry-run keep-set manifest now carries the same filter-excluded and size-pruned protected prefixes and synchronized-directory scope a real run sends, and the would-be-transferred entries are in the keep-set, so `-n --delete` lists exactly rsync's extras for source-derived protections — the file merely being updated is kept and an excluded/pruned source entry is protected (`test_dry_run_delete_lines_match_rsync`, differential vs rsync 3.4.1). **Remaining caveat:** a destination-only entry that matches an exclude/filter rule is still reported (and really removed in a real run) because FastSync derives delete protection from the source scan rather than re-applying rules on the receiver — the same divergence as the `--filter` protect row, pinned by `TestOptionParity`; and the would-delete line ordering follows the destination readdir order rather than rsync's reverse-sorted walk (the differential compares the sorted set) |
|
||||||
| `-b`, `--backup` | Make backups of overwritten files | ✅ Parity | Backup before overwrite |
|
| `-b`, `--backup` | Make backups of overwritten files | ✅ Parity | Backup before overwrite |
|
||||||
| `--backup-dir=DIR` | Backup directory hierarchy | ✅ Parity | `backup_dir` config field |
|
| `--backup-dir=DIR` | Backup directory hierarchy | ✅ Parity | `backup_dir` config field |
|
||||||
| `--suffix=SUFFIX` | Backup suffix (default ~) | ✅ Parity | `suffix` config field |
|
| `--suffix=SUFFIX` | Backup suffix (default ~) | ✅ Parity | `suffix` config field |
|
||||||
@@ -154,7 +156,7 @@ Every one of those has an entry below with its remaining caveats.
|
|||||||
| `--delete` | Delete extraneous files from dest | ⚠️ Caveat | `use_delete` config field. Deletion is always derived from the transmitted keep-set manifest of the paths the sender sent/keeps (never from unchecked input), runs through the symlink-safe walker bounded by `MAX_SERVER_DELETE_COUNT`, and skips the `.fastsync-stage` staging dir under `--delay-updates`. FastSync's default timing when no timing flag is given is **delete-after** (extras are removed only once the whole transfer succeeded) — intentionally NOT rsync's `--del`/delete-during default, to preserve FastSync's commit-style safety. By default the destination mirror of a path the source scan pruned (filter/exclude/size rules) is **protected** from deletion — matching rsync, which does not delete excluded files under `--delete`; `--delete-excluded` opts back into deleting them (see below). Deletion is scoped to the **synchronized directories** sent in the manifest (protocol 2.23.0), so a `--files-from` subset no longer deletes untransmitted paths outside the listed directory subtrees. The walk is bounded: a client `--max-delete=NUM` (or the 100000-entry server bound) makes it **partial** — entries up to the bound are removed, the rest are skipped, and the client exits **25** (`RERR_PARTIAL`), matching rsync, rather than failing the transfer. Extraneous destination symlinks are unlinked by name (never followed); a directory still holding a kept/protected entry is left behind rather than failing |
|
| `--delete` | Delete extraneous files from dest | ⚠️ Caveat | `use_delete` config field. Deletion is always derived from the transmitted keep-set manifest of the paths the sender sent/keeps (never from unchecked input), runs through the symlink-safe walker bounded by `MAX_SERVER_DELETE_COUNT`, and skips the `.fastsync-stage` staging dir under `--delay-updates`. FastSync's default timing when no timing flag is given is **delete-after** (extras are removed only once the whole transfer succeeded) — intentionally NOT rsync's `--del`/delete-during default, to preserve FastSync's commit-style safety. By default the destination mirror of a path the source scan pruned (filter/exclude/size rules) is **protected** from deletion — matching rsync, which does not delete excluded files under `--delete`; `--delete-excluded` opts back into deleting them (see below). Deletion is scoped to the **synchronized directories** sent in the manifest (protocol 2.23.0), so a `--files-from` subset no longer deletes untransmitted paths outside the listed directory subtrees. The walk is bounded: a client `--max-delete=NUM` (or the 100000-entry server bound) makes it **partial** — entries up to the bound are removed, the rest are skipped, and the client exits **25** (`RERR_PARTIAL`), matching rsync, rather than failing the transfer. Extraneous destination symlinks are unlinked by name (never followed); a directory still holding a kept/protected entry is left behind rather than failing |
|
||||||
| `--delete-before` | Delete before transfer | ⚠️ Caveat | Implies `--delete`. The sender runs a full source pre-scan (paths only) and transmits the keep-set manifest BEFORE any file data; the receiver validates it, removes every destination entry not listed (bounded walk, staging-dir skip, protected prefixes honored), then acks `STATUS_OK`. The sender only starts streaming after the deletion committed, or aborts if the receiver reported a deletion error. By definition the deletions already happened when a later transfer phase fails — rsync's delete-before is destructive the same way; a subsequent failure does not restore the removed files. Divergence: the keep-set is the pre-scan snapshot, so a file that appears on the source between the pre-scan and the data pass is still transferred but was not protected from deletion |
|
| `--delete-before` | Delete before transfer | ⚠️ Caveat | Implies `--delete`. The sender runs a full source pre-scan (paths only) and transmits the keep-set manifest BEFORE any file data; the receiver validates it, removes every destination entry not listed (bounded walk, staging-dir skip, protected prefixes honored), then acks `STATUS_OK`. The sender only starts streaming after the deletion committed, or aborts if the receiver reported a deletion error. By definition the deletions already happened when a later transfer phase fails — rsync's delete-before is destructive the same way; a subsequent failure does not restore the removed files. Divergence: the keep-set is the pre-scan snapshot, so a file that appears on the source between the pre-scan and the data pass is still transferred but was not protected from deletion |
|
||||||
| `--del`, `--delete-during` | Delete during transfer | ⚠️ Caveat | Both spellings accepted; imply `--delete`. **Protocol 2.24.0 implements per-directory delete plans:** as the sender finishes each source directory it streams a `STATUS_DELETE_PLAN` for that directory and the receiver removes that directory's extras before applying the next directory's data, so a mid-transfer failure has already removed the extras of the directories reached (verified with a byte-slicing proxy). **Remaining divergence:** the exact abort boundary and the progressive ordering of removals versus rsync's generator can differ, and `-d`/`--dirs` (no descent) falls back to the end-of-transfer commit. `-R` plans are scoped to the transferred prefix subtree |
|
| `--del`, `--delete-during` | Delete during transfer | ⚠️ Caveat | Both spellings accepted; imply `--delete`. **Protocol 2.24.0 implements per-directory delete plans:** as the sender finishes each source directory it streams a `STATUS_DELETE_PLAN` for that directory and the receiver removes that directory's extras before applying the next directory's data, so a mid-transfer failure has already removed the extras of the directories reached (verified with a byte-slicing proxy). **Remaining divergence:** the exact abort boundary and the progressive ordering of removals versus rsync's generator can differ, and `-d`/`--dirs` (no descent) falls back to the end-of-transfer commit. `-R` plans are scoped to the transferred prefix subtree |
|
||||||
| `--delete-delay` | Find deletions during, delete after | ⚠️ Caveat | Implies `--delete`. **Protocol 2.24.0 implements rsync's delete-delay timing:** the sender records each directory's delete plan while scanning and the receiver commits those removals only after the whole transfer succeeds (per plan), so an extra created in the destination after its directory's plan survives while `--delete-after` re-scans and removes it, and a failed transfer removes nothing. The **reported** deleted count advances only on an actual removal; the `--max-delete` budget is charged at plan/snapshot time (the `planned` counter, incremented by `defer_add`) to bound the deferred list, so a snapshotted extra that is later skipped or fails removal still consumes budget. A directory snapshotted into the plan that is refilled before the commit and survives `ENOTEMPTY` is **not** reported, and a `--max-delete=2` partial delete reports exactly 2 (differential test vs rsync 3.4.1, exit 25 both). **Caveat:** rsync charges `--max-delete` on actual removals and its deferred removal recurses into a queued directory, so when a snapshotted entry fails removal FastSync skips a later extra that rsync would still delete, and content created in a refilled extra directory is removed by rsync but left in place by FastSync (the differential test pins both sides). Unit tests cover the refilled-directory, removed-file, and plan-time budget accounting. Exact ordering of which extras are removed first can still differ from rsync's generator |
|
| `--delete-delay` | Find deletions during, delete after | ⚠️ Caveat | Implies `--delete`. **Protocol 2.24.0 implements rsync's delete-delay timing:** the sender records each directory's delete plan while scanning and the receiver commits those removals only after the whole transfer succeeds (per plan), so an extra created in the destination after its directory's plan survives while `--delete-after` re-scans and removes it, and a failed transfer removes nothing. The **reported** deleted count advances only on an actual removal. **Fixed (no-wire):** the `--max-delete` budget is now charged on ACTUAL removals (an unlink/rmdir that succeeded), not at plan/snapshot time, and a queued directory is re-scanned at commit and removed recursively (content created after the plan included), matching rsync: a snapshotted entry that fails or is skipped consumes no budget, so a later extra rsync would delete is still deleted. The deferred snapshot list keeps an independent hard cap (`DELETE_PLAN_SERVER_LIMIT`) so it cannot grow without bound now that the budget is no longer charged while scanning. A `--max-delete=2` partial delete reports exactly 2 and exits 25 in both tools, and the refilled-directory differential (late content removed, directory removed, budget shared) now matches rsync 3.4.1 on both sides (`test_delete_delay_budget_parity.py`, `test_delete_timing_parity.py`). Unit tests cover recursive removal, actual-removal charging, and the bounded deferred list. **Caveat:** the exact ordering of which extras are removed first under a partial `--max-delete` can still differ from rsync's generator (the survivor set is compared by count, not identity) |
|
||||||
| `--delete-after` | Delete after transfer | ✅ Parity | Implies `--delete`. The delete-after timing is also what plain `--delete` does: the keep-set manifest closes the data stream and the receiver commits the bounded deletion only after the terminal `STATUS_FINISHED` proves the whole transfer (every data frame received and stored) succeeded. A failed or aborted transfer removes nothing |
|
| `--delete-after` | Delete after transfer | ✅ Parity | Implies `--delete`. The delete-after timing is also what plain `--delete` does: the keep-set manifest closes the data stream and the receiver commits the bounded deletion only after the terminal `STATUS_FINISHED` proves the whole transfer (every data frame received and stored) succeeded. A failed or aborted transfer removes nothing |
|
||||||
| `--delete-excluded` | Also delete excluded files | ⚠️ Caveat | `delete_excluded` config field. Under `--delete` FastSync protects (rsync's default) the destination mirror of paths the sender's source scan pruned by the user-selection rules — the `--filter`/`-F`/`-C` layer and the legacy `--exclude`/`--include` layer. The sender transmits those concrete pruned paths as **protected prefixes** in the delete-manifest frame (see the Phase-3 notes below); the walker never descends into or removes them. `--delete-excluded` opts back in: the sender sends an empty protected list, so the excluded destination mirrors become ordinary extras and are removed. **`--max-size`/`--min-size` pruned mirrors are a separate, always-on protection** (protocol 2.23.0, rsync parity): size-pruned source mirrors survive `--delete` even with `--delete-excluded`. Divergences (documented): protection is derived only from what the source scan actually pruned — a stray destination-only file that happens to match an exclude rule is not protected (FastSync never re-applies rules to the destination, keeping deletion sender-derived) |
|
| `--delete-excluded` | Also delete excluded files | ⚠️ Caveat | `delete_excluded` config field. Under `--delete` FastSync protects (rsync's default) the destination mirror of paths the sender's source scan pruned by the user-selection rules — the `--filter`/`-F`/`-C` layer and the legacy `--exclude`/`--include` layer. The sender transmits those concrete pruned paths as **protected prefixes** in the delete-manifest frame (see the Phase-3 notes below); the walker never descends into or removes them. `--delete-excluded` opts back in: the sender sends an empty protected list, so the excluded destination mirrors become ordinary extras and are removed. **`--max-size`/`--min-size` pruned mirrors are a separate, always-on protection** (protocol 2.23.0, rsync parity): size-pruned source mirrors survive `--delete` even with `--delete-excluded`. Divergences (documented): protection is derived only from what the source scan actually pruned — a stray destination-only file that happens to match an exclude rule is not protected (FastSync never re-applies rules to the destination, keeping deletion sender-derived) |
|
||||||
| `--max-delete=NUM` | Max files to delete | ✅ Parity | `max_delete` config field (default -1 = no client limit; 0 = delete nothing). **Protocol 2.23.0 matches rsync's partial semantics:** the receiver deletes up to NUM entries (regular files, symlinks and empty directories; each directory removal counts as one) and then **stops deleting, skips the rest, and reports the run as partial**. The client prints a "deletions stopped due to `--max-delete` limit" message and exits **25** (rsync's `RERR_PARTIAL`), not a hard failure — the transfer itself succeeded. NUM only applies together with `--delete` (it is inert otherwise, matching rsync). A client NUM below the server hard bound `MAX_SERVER_DELETE_COUNT` (100000) replaces it; a NUM above it never raises that cap. Deleting an entire destination with no limit is still bounded by the server's 100000-entry ceiling. `--delete-missing-args` exact-path deletions and the ordinary extras walk draw from the same budget, matching rsync |
|
| `--max-delete=NUM` | Max files to delete | ✅ Parity | `max_delete` config field (default -1 = no client limit; 0 = delete nothing). **Protocol 2.23.0 matches rsync's partial semantics:** the receiver deletes up to NUM entries (regular files, symlinks and empty directories; each directory removal counts as one) and then **stops deleting, skips the rest, and reports the run as partial**. The client prints a "deletions stopped due to `--max-delete` limit" message and exits **25** (rsync's `RERR_PARTIAL`), not a hard failure — the transfer itself succeeded. NUM only applies together with `--delete` (it is inert otherwise, matching rsync). A client NUM below the server hard bound `MAX_SERVER_DELETE_COUNT` (100000) replaces it; a NUM above it never raises that cap. Deleting an entire destination with no limit is still bounded by the server's 100000-entry ceiling. `--delete-missing-args` exact-path deletions and the ordinary extras walk draw from the same budget, matching rsync |
|
||||||
@@ -909,7 +911,7 @@ These are the last compatibility items and the closing phase toward rsync flag p
|
|||||||
|
|
||||||
**Wire:** two trailing config-frame blocks after the `--iconv` spec, in fixed order — `send_privilege_options`/`receive_privilege_options` (one `super_mode` int, validated `0..2`), then `send_copy_as_options`/`receive_copy_as_options` (presence int + two int32 ids, validated `>= 0`, with `copy_as_set ⇒ use_metadata`). `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Divergences from rsync:** rsync's `--super` elevates the receiver and `--copy-as` actually switches its credentials; FastSync never elevates and only permits/forwards confined attempts, and `--copy-as` forces ownership rather than switching identity.
|
**Wire:** two trailing config-frame blocks after the `--iconv` spec, in fixed order — `send_privilege_options`/`receive_privilege_options` (one `super_mode` int, validated `0..2`), then `send_copy_as_options`/`receive_copy_as_options` (presence int + two int32 ids, validated `>= 0`, with `copy_as_set ⇒ use_metadata`). `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Divergences from rsync:** rsync's `--super` elevates the receiver and `--copy-as` actually switches its credentials; FastSync never elevates and only permits/forwards confined attempts, and `--copy-as` forces ownership rather than switching identity.
|
||||||
|
|
||||||
**Honest status after the parity-completion wave (protocol 2.27.0), updated by the rsync-parity-stats, rsync-parity-options, rsync-parity-fs, and parity-review passes.** ✅ Parity 111 / ⚠️ Caveat 13 / ❌ Divergent 33 = 157 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. This reclassification makes every difference explicit. The completion wave closed 23 previously-caveated rows (9 that triage showed were already parity, plus 14 genuine fixes) and turned the 17 inherently non-rsync rows — native daemon config/auth, the FastSync batch container, the safe-subset device/privilege flags, `-X`'s privileged namespaces, `--fake-super`'s native xattr format, and the `--old-args` no-op — into explicit ❌ divergences. The stats pass flipped `--delete-delay` to ✅ (actual-removal accounting), but the parity-review pass moved it back to ⚠️: FastSync charges the `--max-delete` budget at plan/snapshot time and leaves a refilled snapshotted directory in place, whereas rsync charges on actual removals and recursively removes a queued directory (including content created after its plan), so the two diverge when a snapshotted entry fails removal (see the `--delete-delay` row and the differential test). The stats pass also reclassified `--out-format` to ❌ (protocol-specific `%b`/delta-`%c`), and sharpened the `--stats`/`--progress`/`--checksum-choice` residuals. The options pass flipped `--bwlimit` and `--ignore-errors` to ✅ (rsync-exact size parsing and ~100 ms leaky-bucket throttling, and rsync's skip-unreadable-subdir plus IO-error-suppressed deletion with exit 23) and emits rsync-format `--info=name/flist/del/remove/nonreg/progress` lines (real-run `deleting`/`*deleting` carried over a new trailing `report_deletes` wire bool, `PROTOCOL_VERSION` 2.26.0 → 2.27.0), while reclassifying `-M` over daemon/TCP and receiver-side `protect`/`risk` re-derivation to ❌ (no argv channel / receiver filter engine). The fs pass flipped `-d/--dirs` and `--iconv` to ✅ — recursive transfers now recreate empty source directories (and replace a blocking destination non-directory with an incoming directory); `-R --no-implied-dirs --files-from` places a listed file under a missing implied parent with default attributes instead of refusing; and `--iconv` now reproduces rsync's push direction (destination charset = the spec's REMOTE half) — and reclassified six rows to ❌ after reproducing their exact residual with differential tests: `--temp-dir` (the receiver confines the scratch dir to the receive root, so an absolute temp dir is deliberately rejected although standalone rsync follows it), the three basis-dir options (FastSync xxHash-verifies a basis hit while rsync's `--size-only` quick check installs the wrong basis content), `--delay-updates` (fixed staging name wipes an unrelated destination entry of that name), and `--dry-run` (would-delete report over-reports). `--fuzzy` was also reclassified to ❌ (deterministic heuristic with a 10× size window, not rsync's matcher), but its residual is the candidate-selection heuristic itself: the final tree is byte-exact by design, so no destination differential can expose it and the row is pinned by the `TestFuzzy` threshold suite rather than a byte-level rsync differential. The remaining ⚠️ rows are the ones with a documented residual (see the row notes and the **Parity Completion Wave (protocol 2.26.0)** section below).
|
**Honest status after the parity-completion wave (protocol 2.27.0), updated by the rsync-parity-stats, rsync-parity-options, rsync-parity-fs, parity-review, and no-wire parity-track-1 passes.** ✅ Parity 111 / ⚠️ Caveat 14 / ❌ Divergent 32 = 157 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. This reclassification makes every difference explicit. The completion wave closed 23 previously-caveated rows (9 that triage showed were already parity, plus 14 genuine fixes) and turned the 17 inherently non-rsync rows — native daemon config/auth, the FastSync batch container, the safe-subset device/privilege flags, `-X`'s privileged namespaces, `--fake-super`'s native xattr format, and the `--old-args` no-op — into explicit ❌ divergences. The stats pass flipped `--delete-delay` to ✅ (actual-removal accounting), but the parity-review pass moved it back to ⚠️ because FastSync charged the `--max-delete` budget at plan/snapshot time and left a refilled snapshotted directory in place, whereas rsync charges on actual removals and recursively removes a queued directory (including content created after its plan). The no-wire parity-track-1 pass fixed both (actual-removal charging plus recursive deferred removal with an independent deferred-list cap), narrowing the caveat to the partial-delete ordering. The stats pass also reclassified `--out-format` to ❌ (protocol-specific `%b`/delta-`%c`), and sharpened the `--stats`/`--progress`/`--checksum-choice` residuals. The options pass flipped `--bwlimit` and `--ignore-errors` to ✅ (rsync-exact size parsing and ~100 ms leaky-bucket throttling, and rsync's skip-unreadable-subdir plus IO-error-suppressed deletion with exit 23) and emits rsync-format `--info=name/flist/del/remove/nonreg/progress` lines (real-run `deleting`/`*deleting` carried over a new trailing `report_deletes` wire bool, `PROTOCOL_VERSION` 2.26.0 → 2.27.0), while reclassifying `-M` over daemon/TCP and receiver-side `protect`/`risk` re-derivation to ❌ (no argv channel / receiver filter engine). The fs pass flipped `-d/--dirs` and `--iconv` to ✅ — recursive transfers now recreate empty source directories (and replace a blocking destination non-directory with an incoming directory); `-R --no-implied-dirs --files-from` places a listed file under a missing implied parent with default attributes instead of refusing; and `--iconv` now reproduces rsync's push direction (destination charset = the spec's REMOTE half) — and reclassified six rows to ❌ after reproducing their exact residual with differential tests: `--temp-dir` (the receiver confines the scratch dir to the receive root, so an absolute temp dir is deliberately rejected although standalone rsync follows it), the three basis-dir options (FastSync xxHash-verifies a basis hit while rsync's `--size-only` quick check installs the wrong basis content), `--delay-updates` (fixed staging name wipes an unrelated destination entry of that name), and `--dry-run` (would-delete report over-reports). `--fuzzy` was also reclassified to ❌ (deterministic heuristic with a 10× size window, not rsync's matcher), but its residual is the candidate-selection heuristic itself: the final tree is byte-exact by design, so no destination differential can expose it and the row is pinned by the `TestFuzzy` threshold suite rather than a byte-level rsync differential. The remaining ⚠️ rows are the ones with a documented residual (see the row notes and the **Parity Completion Wave (protocol 2.26.0)** section below).
|
||||||
|
|
||||||
**Preserve-attribute split (protocol 2.21.0 → 2.22.0) — ✅ implemented.** FastSync splits the former single metadata bundle into four independent, rsync-compatible per-attribute flags — `-p/--perms`, `-t/--times`, `-o/--owner`, `-g/--group` — each with a negation (`--no-perms`/`--no-times`/`--no-owner`/`--no-group`, short `--no-p`/`--no-t`/`--no-o`/`--no-g`), plus `--no-preserve` clearing all four. `-a/--archive` is now full rsync `-rlptgoD` (owner and group included, though their application stays privilege-gated), `-A/--acls` implies `-p`, `-X/--xattrs` does not, `-E/--executability` sets only executability, and `-U`/`-N` do not imply `-t`. `--incremental`/`--delta` still auto-preserve perms+times unless the user explicitly negated them. Wire: the binary config frame gains four appended booleans (`preserve_perms`/`preserve_times`/`preserve_owner`/`preserve_group`) after `omit_link_times`, so `PROTOCOL_VERSION` is bumped **2.21.0 → 2.22.0**; the fixed-width `FileMetadata` layout is unchanged and the receiver gates the metadata frame on a derived `use_metadata`. Receiver behavior: each attribute is applied independently, directory modes are applied under `-p` (at the end of the transfer, alongside dir times), symlink mode under `-p`, and `-O/--omit-dir-times` suppresses directory times only. Documented divergences as of 2.22.0, **all but (d)/(e) removed by the rsync-parity wave (protocol 2.23.0)**: (a) the mode-masking divergence is **gone** — under `-p` the source mode is now copied exactly, including `S_IWGRP`/`S_IWOTH` and setuid/setgid/sticky; (b) a brand-new file without `-p` still gets `source_mode & ~umask` when metadata is present (else the historical fixed `0644`), and a new *directory* without `-p` still uses FastSync's `0755` default; (c) the `--chmod`-implies-`-p` divergence is **gone** — `--chmod` no longer implies `-p` (rsync parity); (d) `-o`/`-g` map by name on the receiver with a raw-numeric fallback (only numeric ids cross the wire); (e) a daemon module without `client owner = yes` does not refuse a plain `-a`/`-o`/`-g` — it forces super off, applies no ownership, and logs a warning, while explicit `--chown`/`--usermap`/`--groupmap`/`--numeric-ids`/`--copy-as`/`--super` are still refused.
|
**Preserve-attribute split (protocol 2.21.0 → 2.22.0) — ✅ implemented.** FastSync splits the former single metadata bundle into four independent, rsync-compatible per-attribute flags — `-p/--perms`, `-t/--times`, `-o/--owner`, `-g/--group` — each with a negation (`--no-perms`/`--no-times`/`--no-owner`/`--no-group`, short `--no-p`/`--no-t`/`--no-o`/`--no-g`), plus `--no-preserve` clearing all four. `-a/--archive` is now full rsync `-rlptgoD` (owner and group included, though their application stays privilege-gated), `-A/--acls` implies `-p`, `-X/--xattrs` does not, `-E/--executability` sets only executability, and `-U`/`-N` do not imply `-t`. `--incremental`/`--delta` still auto-preserve perms+times unless the user explicitly negated them. Wire: the binary config frame gains four appended booleans (`preserve_perms`/`preserve_times`/`preserve_owner`/`preserve_group`) after `omit_link_times`, so `PROTOCOL_VERSION` is bumped **2.21.0 → 2.22.0**; the fixed-width `FileMetadata` layout is unchanged and the receiver gates the metadata frame on a derived `use_metadata`. Receiver behavior: each attribute is applied independently, directory modes are applied under `-p` (at the end of the transfer, alongside dir times), symlink mode under `-p`, and `-O/--omit-dir-times` suppresses directory times only. Documented divergences as of 2.22.0, **all but (d)/(e) removed by the rsync-parity wave (protocol 2.23.0)**: (a) the mode-masking divergence is **gone** — under `-p` the source mode is now copied exactly, including `S_IWGRP`/`S_IWOTH` and setuid/setgid/sticky; (b) a brand-new file without `-p` still gets `source_mode & ~umask` when metadata is present (else the historical fixed `0644`), and a new *directory* without `-p` still uses FastSync's `0755` default; (c) the `--chmod`-implies-`-p` divergence is **gone** — `--chmod` no longer implies `-p` (rsync parity); (d) `-o`/`-g` map by name on the receiver with a raw-numeric fallback (only numeric ids cross the wire); (e) a daemon module without `client owner = yes` does not refuse a plain `-a`/`-o`/`-g` — it forces super off, applies no ownership, and logs a warning, while explicit `--chown`/`--usermap`/`--groupmap`/`--numeric-ids`/`--copy-as`/`--super` are still refused.
|
||||||
|
|
||||||
|
|||||||
@@ -75,6 +75,14 @@ bool change_list_enabled(const Config* config) {
|
|||||||
(config->info_level & LOG_INFO_NAME) != 0);
|
(config->info_level & LOG_INFO_NAME) != 0);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* Emitted once, lazily, ahead of the first --info=name entry: rsync prints the
|
||||||
|
* transfer-root `./` name line when the root directory is (re)created. */
|
||||||
|
static bool name_root_printed = false;
|
||||||
|
|
||||||
|
void change_reset_name_root(void) {
|
||||||
|
name_root_printed = false;
|
||||||
|
}
|
||||||
|
|
||||||
/* ---- Itemize code ---- */
|
/* ---- Itemize code ---- */
|
||||||
|
|
||||||
/* Format the permission bits as an `ls -l` string, e.g. `-rw-r--r--`. */
|
/* Format the permission bits as an `ls -l` string, e.g. `-rw-r--r--`. */
|
||||||
@@ -212,6 +220,23 @@ static char* change_render_name(const ChangeEvent* event) {
|
|||||||
return line.data;
|
return line.data;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* rsync's `--info=name2` line for an unchanged entry: `NAME is uptodate`. */
|
||||||
|
static char* change_render_name_uptodate(const ChangeEvent* event) {
|
||||||
|
char* name = change_render_name(event);
|
||||||
|
if (name == NULL)
|
||||||
|
return NULL;
|
||||||
|
size_t length = strlen(name);
|
||||||
|
char* line = malloc(length + sizeof(" is uptodate"));
|
||||||
|
if (line == NULL) {
|
||||||
|
free(name);
|
||||||
|
return NULL;
|
||||||
|
}
|
||||||
|
memcpy(line, name, length);
|
||||||
|
memcpy(line + length, " is uptodate", sizeof(" is uptodate"));
|
||||||
|
free(name);
|
||||||
|
return line;
|
||||||
|
}
|
||||||
|
|
||||||
/* ---- --out-format / --log-file-format ---- */
|
/* ---- --out-format / --log-file-format ---- */
|
||||||
|
|
||||||
/* rsync 3.4.1's `%C` uses the negotiated TRANSFER checksum (the first name of a
|
/* rsync 3.4.1's `%C` uses the negotiated TRANSFER checksum (the first name of a
|
||||||
@@ -461,10 +486,24 @@ static void print_escaped_line(FILE* stream, const char* line, bool eight_bit_ou
|
|||||||
void change_emit(const Config* config, const ChangeEvent* event) {
|
void change_emit(const Config* config, const ChangeEvent* event) {
|
||||||
if (event == NULL || !change_list_enabled(config))
|
if (event == NULL || !change_list_enabled(config))
|
||||||
return;
|
return;
|
||||||
if (event->decision == CHANGE_UP_TO_DATE)
|
|
||||||
return;
|
|
||||||
bool to_stdout = config->itemize_changes || config->out_format != NULL;
|
bool to_stdout = config->itemize_changes || config->out_format != NULL;
|
||||||
bool to_log = config->log_file != NULL && config->log_file_format != NULL;
|
bool to_log = config->log_file != NULL && config->log_file_format != NULL;
|
||||||
|
bool progress_active = config->show_progress || (config->info_level & LOG_INFO_PROGRESS);
|
||||||
|
if (event->decision == CHANGE_UP_TO_DATE) {
|
||||||
|
/* --info=name2 prints `NAME is uptodate` for entries the receiver already
|
||||||
|
had. An itemize/out-format run reports them through its own format (or
|
||||||
|
not at all), the progress stream has no frame for them, and neither the
|
||||||
|
itemize nor the log-file stream previously reported an up-to-date entry,
|
||||||
|
so nothing else here changes. */
|
||||||
|
if (!to_stdout && (config->info_level & LOG_INFO_NAME_UPTODATE) != 0 && !progress_active) {
|
||||||
|
char* line = change_render_name_uptodate(event);
|
||||||
|
if (line != NULL) {
|
||||||
|
print_escaped_line(stdout, line, config->eight_bit_output);
|
||||||
|
free(line);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
if (to_stdout) {
|
if (to_stdout) {
|
||||||
char* line = config->out_format != NULL
|
char* line = config->out_format != NULL
|
||||||
? change_render_format(config->out_format, config, event)
|
? change_render_format(config->out_format, config, event)
|
||||||
@@ -473,11 +512,15 @@ void change_emit(const Config* config, const ChangeEvent* event) {
|
|||||||
print_escaped_line(stdout, line, config->eight_bit_output);
|
print_escaped_line(stdout, line, config->eight_bit_output);
|
||||||
free(line);
|
free(line);
|
||||||
}
|
}
|
||||||
} else if ((config->info_level & LOG_INFO_NAME) != 0 &&
|
} else if ((config->info_level & LOG_INFO_NAME) != 0 && !progress_active) {
|
||||||
!(config->show_progress || (config->info_level & LOG_INFO_PROGRESS))) {
|
|
||||||
/* --info=name without -i/--out-format: print the updated entry's name. The
|
/* --info=name without -i/--out-format: print the updated entry's name. The
|
||||||
--progress path owns the name line when progress output is active (it
|
--progress path owns the name line when progress output is active (it
|
||||||
emits the same names before the progress frames), so do not duplicate. */
|
emits the same names before the progress frames), so do not duplicate.
|
||||||
|
The transfer-root `./` line precedes the first such name. */
|
||||||
|
if (!name_root_printed) {
|
||||||
|
name_root_printed = true;
|
||||||
|
fputs("./\n", stdout);
|
||||||
|
}
|
||||||
char* line = change_render_name(event);
|
char* line = change_render_name(event);
|
||||||
if (line != NULL) {
|
if (line != NULL) {
|
||||||
print_escaped_line(stdout, line, config->eight_bit_output);
|
print_escaped_line(stdout, line, config->eight_bit_output);
|
||||||
@@ -643,6 +686,29 @@ void change_emit_file_sent(const Config* config, const File* file) {
|
|||||||
change_emit_file_sent_bytes(config, file, payload, 0);
|
change_emit_file_sent_bytes(config, file, payload, 0);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
void change_emit_file_uptodate(const Config* config, const File* file) {
|
||||||
|
if (file == NULL || !change_list_enabled(config))
|
||||||
|
return;
|
||||||
|
ChangeEvent event;
|
||||||
|
memset(&event, 0, sizeof(event));
|
||||||
|
event.decision = CHANGE_UP_TO_DATE;
|
||||||
|
event.is_directory = false;
|
||||||
|
event.is_symlink = file->is_symlink;
|
||||||
|
event.is_special = file->is_special;
|
||||||
|
event.is_hardlink = file->link_group != 0 && !file->link_first;
|
||||||
|
event.symlink_target = file->symlink_target;
|
||||||
|
event.hardlink_target = file->hardlink_target;
|
||||||
|
event.size = file->data != NULL ? file->data->size : 0;
|
||||||
|
event.dest = file->dest_state;
|
||||||
|
char* name = NULL;
|
||||||
|
char* path = NULL;
|
||||||
|
fill_event_from_file(config, file, &event, &name, &path);
|
||||||
|
if (name != NULL && path != NULL)
|
||||||
|
change_emit(config, &event);
|
||||||
|
free(name);
|
||||||
|
free(path);
|
||||||
|
}
|
||||||
|
|
||||||
void change_emit_dir_sent(const Config* config, const File* file) {
|
void change_emit_dir_sent(const Config* config, const File* file) {
|
||||||
if (file == NULL || !change_list_enabled(config))
|
if (file == NULL || !change_list_enabled(config))
|
||||||
return;
|
return;
|
||||||
|
|||||||
@@ -102,4 +102,13 @@ void change_emit_file_sent(const Config* config, const File* file);
|
|||||||
/* Build and emit a CHANGE_SENT event for an explicit directory entry (-d). */
|
/* Build and emit a CHANGE_SENT event for an explicit directory entry (-d). */
|
||||||
void change_emit_dir_sent(const Config* config, const File* file);
|
void change_emit_dir_sent(const Config* config, const File* file);
|
||||||
|
|
||||||
|
/* Build and emit a CHANGE_UP_TO_DATE event for a file the receiver already had.
|
||||||
|
* With --info=name2 it renders rsync's "NAME is uptodate" line (no output
|
||||||
|
* otherwise). */
|
||||||
|
void change_emit_file_uptodate(const Config* config, const File* file);
|
||||||
|
|
||||||
|
/* Reset the lazy transfer-root `./` line emitted ahead of the first
|
||||||
|
* --info=name entry. Call once at the start of a transfer. */
|
||||||
|
void change_reset_name_root(void);
|
||||||
|
|
||||||
#endif
|
#endif
|
||||||
|
|||||||
+13
-3
@@ -614,9 +614,19 @@ static int parse_info_flags(const char* value, Config* config) {
|
|||||||
}
|
}
|
||||||
if (strcmp(name, "copy") == 0)
|
if (strcmp(name, "copy") == 0)
|
||||||
flag = LOG_INFO_COPY;
|
flag = LOG_INFO_COPY;
|
||||||
else if (strcmp(name, "name") == 0)
|
else if (strcmp(name, "name") == 0) {
|
||||||
flag = LOG_INFO_NAME;
|
/* name level 2 adds rsync's "is uptodate" lines. */
|
||||||
else if (strcmp(name, "misc") == 0)
|
if (level == 0)
|
||||||
|
parsed &= ~(uint32_t)(LOG_INFO_NAME | LOG_INFO_NAME_UPTODATE);
|
||||||
|
else {
|
||||||
|
parsed |= LOG_INFO_NAME;
|
||||||
|
if (level >= 2)
|
||||||
|
parsed |= LOG_INFO_NAME_UPTODATE;
|
||||||
|
else
|
||||||
|
parsed &= ~(uint32_t)LOG_INFO_NAME_UPTODATE;
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
} else if (strcmp(name, "misc") == 0)
|
||||||
flag = LOG_INFO_MISC;
|
flag = LOG_INFO_MISC;
|
||||||
else if (strcmp(name, "skip") == 0)
|
else if (strcmp(name, "skip") == 0)
|
||||||
flag = LOG_INFO_SKIP;
|
flag = LOG_INFO_SKIP;
|
||||||
|
|||||||
+38
-22
@@ -350,6 +350,7 @@ static void print_delete_reports(const Config* config, const ArrayList* paths) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
static void client_progress_begin(const Config* config) {
|
static void client_progress_begin(const Config* config) {
|
||||||
|
change_reset_name_root();
|
||||||
g_progress_active =
|
g_progress_active =
|
||||||
(config->show_progress || info_flag_enabled(config, LOG_INFO_PROGRESS)) && !config->quiet;
|
(config->show_progress || info_flag_enabled(config, LOG_INFO_PROGRESS)) && !config->quiet;
|
||||||
g_progress_xferred = 0;
|
g_progress_xferred = 0;
|
||||||
@@ -1928,11 +1929,41 @@ static int send_dry_run_remote(Config* config) {
|
|||||||
DirectoryScanner* scanner = NULL;
|
DirectoryScanner* scanner = NULL;
|
||||||
ArrayList* dry_manifest = NULL;
|
ArrayList* dry_manifest = NULL;
|
||||||
ArrayList* dry_dirs = NULL;
|
ArrayList* dry_dirs = NULL;
|
||||||
|
ArrayList* dry_excluded = NULL;
|
||||||
|
ArrayList* dry_size_skipped = NULL;
|
||||||
if (!config_send(client->file_descriptor, config))
|
if (!config_send(client->file_descriptor, config))
|
||||||
goto dry_fail;
|
goto dry_fail;
|
||||||
receive_daemon_motd(client, config);
|
receive_daemon_motd(client, config);
|
||||||
if (!prepare_scanner(config, 0, &prepared))
|
if (!prepare_scanner(config, 0, &prepared))
|
||||||
goto dry_fail;
|
goto dry_fail;
|
||||||
|
/* -n --delete: build the same keep-set manifest, protected prefixes, and
|
||||||
|
synchronized-directory scope a real run would send, so the receiver's
|
||||||
|
read-only extras walk enumerates exactly the deletions a real run makes. */
|
||||||
|
if (config->use_delete) {
|
||||||
|
dry_manifest = array_list_create(free);
|
||||||
|
dry_dirs = array_list_create(free);
|
||||||
|
dry_size_skipped = array_list_create(free);
|
||||||
|
if (!dry_manifest || !dry_dirs || !dry_size_skipped)
|
||||||
|
goto dry_fail;
|
||||||
|
if (!config->delete_excluded) {
|
||||||
|
dry_excluded = array_list_create(free);
|
||||||
|
if (!dry_excluded)
|
||||||
|
goto dry_fail;
|
||||||
|
prepared.options.excluded_paths = dry_excluded;
|
||||||
|
}
|
||||||
|
prepared.options.size_skipped_paths = dry_size_skipped;
|
||||||
|
/* A --files-from subset confines the extras walk to the directories the
|
||||||
|
scan synchronized; a full recursive transfer marks the root itself. */
|
||||||
|
if (config->files_from_set == NULL) {
|
||||||
|
char* root_marker = delete_scope_root_marker(config);
|
||||||
|
if (!root_marker || !array_list_add(dry_dirs, root_marker)) {
|
||||||
|
free(root_marker);
|
||||||
|
goto dry_fail;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
prepared.options.synced_dirs = dry_dirs;
|
||||||
|
}
|
||||||
|
}
|
||||||
scanner = directory_scanner_create_with_options(config->send_directory, &prepared.options);
|
scanner = directory_scanner_create_with_options(config->send_directory, &prepared.options);
|
||||||
if (!scanner)
|
if (!scanner)
|
||||||
goto dry_fail;
|
goto dry_fail;
|
||||||
@@ -1940,26 +1971,6 @@ static int send_dry_run_remote(Config* config) {
|
|||||||
int file_count = 0;
|
int file_count = 0;
|
||||||
unsigned long long total_bytes = 0;
|
unsigned long long total_bytes = 0;
|
||||||
char size_buffer[32];
|
char size_buffer[32];
|
||||||
/* -n --delete: build the same keep-set manifest a real run would send so the
|
|
||||||
receiver can enumerate (read-only) the destination extras. Filter-excluded
|
|
||||||
and size-pruned protections are not propagated here, so a filtered dry-run
|
|
||||||
may over-report; the no-filter case is exact. */
|
|
||||||
dry_manifest = config->use_delete ? array_list_create(free) : NULL;
|
|
||||||
if (config->use_delete && !dry_manifest)
|
|
||||||
goto dry_fail;
|
|
||||||
/* Scope the receiver-side extras walk to the receive root (the "." sentinel),
|
|
||||||
exactly as the recursive transfer path does. */
|
|
||||||
if (config->use_delete) {
|
|
||||||
dry_dirs = array_list_create(free);
|
|
||||||
char* root_marker = dry_dirs ? str_dup(".") : NULL;
|
|
||||||
if (!dry_dirs || !root_marker || !array_list_add(dry_dirs, root_marker)) {
|
|
||||||
free(root_marker);
|
|
||||||
if (dry_dirs)
|
|
||||||
array_list_delete(dry_dirs);
|
|
||||||
dry_dirs = NULL;
|
|
||||||
goto dry_fail;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (!config->quiet)
|
if (!config->quiet)
|
||||||
printf("Dry run: files to be transferred\n");
|
printf("Dry run: files to be transferred\n");
|
||||||
Chunk* chunk;
|
Chunk* chunk;
|
||||||
@@ -2034,8 +2045,8 @@ static int send_dry_run_remote(Config* config) {
|
|||||||
the terminal FINISHED. */
|
the terminal FINISHED. */
|
||||||
bool early_delete = config->use_delete && config_delete_timing_early(config);
|
bool early_delete = config->use_delete && config_delete_timing_early(config);
|
||||||
if (dry_manifest) {
|
if (dry_manifest) {
|
||||||
if (send_delete_manifest(client->file_descriptor, dry_manifest, NULL, NULL, NULL, dry_dirs) !=
|
if (send_delete_manifest(client->file_descriptor, dry_manifest, dry_excluded, dry_size_skipped,
|
||||||
0)
|
NULL, dry_dirs) != 0)
|
||||||
goto dry_fail;
|
goto dry_fail;
|
||||||
if (early_delete) {
|
if (early_delete) {
|
||||||
Status ack;
|
Status ack;
|
||||||
@@ -2091,6 +2102,10 @@ dry_fail:
|
|||||||
array_list_delete(dry_manifest);
|
array_list_delete(dry_manifest);
|
||||||
if (dry_dirs)
|
if (dry_dirs)
|
||||||
array_list_delete(dry_dirs);
|
array_list_delete(dry_dirs);
|
||||||
|
if (dry_excluded)
|
||||||
|
array_list_delete(dry_excluded);
|
||||||
|
if (dry_size_skipped)
|
||||||
|
array_list_delete(dry_size_skipped);
|
||||||
if (scanner)
|
if (scanner)
|
||||||
directory_scanner_destroy(scanner);
|
directory_scanner_destroy(scanner);
|
||||||
prepared_scanner_destroy(&prepared);
|
prepared_scanner_destroy(&prepared);
|
||||||
@@ -2423,6 +2438,7 @@ static int send_chunk_with_removal(Client* client, Chunk* chunk, Config* config,
|
|||||||
int rc = send_single_file(client, f, config, config->use_incremental, use_sendfile);
|
int rc = send_single_file(client, f, config, config->use_incremental, use_sendfile);
|
||||||
if (rc == 1) {
|
if (rc == 1) {
|
||||||
source_file_destroy(source);
|
source_file_destroy(source);
|
||||||
|
change_emit_file_uptodate(config, f);
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
if (rc < 0) {
|
if (rc < 0) {
|
||||||
|
|||||||
+91
-25
@@ -412,11 +412,17 @@ struct DeletePlanSession {
|
|||||||
bool dry_run;
|
bool dry_run;
|
||||||
size_t max_delete;
|
size_t max_delete;
|
||||||
size_t deleted;
|
size_t deleted;
|
||||||
/* Removals charged against --max-delete. In --delete-delay mode a path is
|
/* Removals charged against --max-delete. The budget is charged on ACTUAL
|
||||||
planned (and the budget consumed) while scanning, but `deleted` advances
|
removals (an unlink/rmdir that succeeded), matching rsync: a snapshotted
|
||||||
only when the commit actually unlinks it, so an entry that survives the
|
entry that fails removal consumes nothing, so a later extra is still
|
||||||
commit (a directory refilled mid-transfer -> ENOTEMPTY) is not reported. */
|
deleted. `planned` and `deleted` advance together for the inline paths and
|
||||||
|
`apply_missing`; `deleted` is the reported count. */
|
||||||
size_t planned;
|
size_t planned;
|
||||||
|
/* Hard bound on the deferred snapshot list. Because the budget is no longer
|
||||||
|
charged at snapshot time, this independent cap keeps a huge destination
|
||||||
|
from growing the list without limit (it matches the receiver's overall
|
||||||
|
deletion bound). */
|
||||||
|
size_t defer_cap;
|
||||||
size_t skipped;
|
size_t skipped;
|
||||||
bool limit_hit;
|
bool limit_hit;
|
||||||
bool limit_logged;
|
bool limit_logged;
|
||||||
@@ -448,6 +454,7 @@ DeletePlanSession* delete_plan_session_create(const Config* config) {
|
|||||||
config->max_delete >= 0 && (size_t)config->max_delete < DELETE_PLAN_SERVER_LIMIT;
|
config->max_delete >= 0 && (size_t)config->max_delete < DELETE_PLAN_SERVER_LIMIT;
|
||||||
session->max_delete =
|
session->max_delete =
|
||||||
user_limited ? (size_t)config->max_delete : (size_t)DELETE_PLAN_SERVER_LIMIT;
|
user_limited ? (size_t)config->max_delete : (size_t)DELETE_PLAN_SERVER_LIMIT;
|
||||||
|
session->defer_cap = DELETE_PLAN_SERVER_LIMIT;
|
||||||
session->protected_prefixes = array_list_create(free);
|
session->protected_prefixes = array_list_create(free);
|
||||||
session->size_skipped = array_list_create(free);
|
session->size_skipped = array_list_create(free);
|
||||||
session->missing = array_list_create(free);
|
session->missing = array_list_create(free);
|
||||||
@@ -592,10 +599,15 @@ static void log_deleted(const char* rel) {
|
|||||||
free(escaped);
|
free(escaped);
|
||||||
}
|
}
|
||||||
|
|
||||||
/* Append a snapshot path for --delete-delay. The budget is charged here, but
|
/* Append a snapshot path for --delete-delay. The budget is NOT charged here:
|
||||||
* `deleted` is not: the path counts only once apply_deferred_path truly
|
* the remover charges --max-delete only when a path is actually unlinked (see
|
||||||
* unlinks it. */
|
* apply_deferred_path), so a snapshotted entry that survives ENOTEMPTY cannot
|
||||||
|
* deny budget to a later extra. The independent `defer_cap` bounds the list. */
|
||||||
static bool defer_add(DeletePlanSession* session, const char* rel) {
|
static bool defer_add(DeletePlanSession* session, const char* rel) {
|
||||||
|
if ((size_t)session->deferred->size >= session->defer_cap) {
|
||||||
|
note_skipped(session);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
char* copy = str_dup(rel);
|
char* copy = str_dup(rel);
|
||||||
if (!copy)
|
if (!copy)
|
||||||
return false;
|
return false;
|
||||||
@@ -603,7 +615,6 @@ static bool defer_add(DeletePlanSession* session, const char* rel) {
|
|||||||
free(copy);
|
free(copy);
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
session->planned++;
|
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -636,16 +647,16 @@ static bool process_extra_dir(int dirfd, const char* name, const char* child_rel
|
|||||||
return false;
|
return false;
|
||||||
if (survives)
|
if (survives)
|
||||||
return true;
|
return true;
|
||||||
if (!budget_available(session)) {
|
|
||||||
note_skipped(session);
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
if (session->defer && !force_now) {
|
if (session->defer && !force_now) {
|
||||||
if (!defer_add(session, child_rel))
|
if (!defer_add(session, child_rel))
|
||||||
return false;
|
return false;
|
||||||
*removed = true;
|
*removed = true;
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
if (!budget_available(session)) {
|
||||||
|
note_skipped(session);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
if (unlinkat(dirfd, name, AT_REMOVEDIR) == 0) {
|
if (unlinkat(dirfd, name, AT_REMOVEDIR) == 0) {
|
||||||
session->deleted++;
|
session->deleted++;
|
||||||
session->planned++;
|
session->planned++;
|
||||||
@@ -665,13 +676,13 @@ static bool process_extra_dir(int dirfd, const char* name, const char* child_rel
|
|||||||
|
|
||||||
static bool process_extra_file(int dirfd, const char* name, const char* child_rel, bool force_now,
|
static bool process_extra_file(int dirfd, const char* name, const char* child_rel, bool force_now,
|
||||||
DeletePlanSession* session) {
|
DeletePlanSession* session) {
|
||||||
|
if (session->defer && !force_now) {
|
||||||
|
return defer_add(session, child_rel);
|
||||||
|
}
|
||||||
if (!budget_available(session)) {
|
if (!budget_available(session)) {
|
||||||
note_skipped(session);
|
note_skipped(session);
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
if (session->defer && !force_now) {
|
|
||||||
return defer_add(session, child_rel);
|
|
||||||
}
|
|
||||||
if (unlinkat(dirfd, name, 0) == 0) {
|
if (unlinkat(dirfd, name, 0) == 0) {
|
||||||
session->deleted++;
|
session->deleted++;
|
||||||
session->planned++;
|
session->planned++;
|
||||||
@@ -858,7 +869,9 @@ int delete_plan_session_receive(DeletePlanSession* session, const Config* config
|
|||||||
}
|
}
|
||||||
|
|
||||||
/* Apply one snapshotted --delete-delay path (post-order: children precede their
|
/* Apply one snapshotted --delete-delay path (post-order: children precede their
|
||||||
* parent directory). */
|
* parent directory). A directory that is still present is re-scanned so content
|
||||||
|
* created after the plan is removed too; every actual removal charges
|
||||||
|
* --max-delete. */
|
||||||
static bool apply_deferred_path(DeletePlanSession* session, const Config* config, const char* rel) {
|
static bool apply_deferred_path(DeletePlanSession* session, const Config* config, const char* rel) {
|
||||||
char* full = path_cat(config->receive_root_directory, rel);
|
char* full = path_cat(config->receive_root_directory, rel);
|
||||||
if (!full)
|
if (!full)
|
||||||
@@ -872,25 +885,78 @@ static bool apply_deferred_path(DeletePlanSession* session, const Config* config
|
|||||||
}
|
}
|
||||||
struct stat st;
|
struct stat st;
|
||||||
if (fstatat(parent_fd, leaf, &st, AT_SYMLINK_NOFOLLOW) != 0) {
|
if (fstatat(parent_fd, leaf, &st, AT_SYMLINK_NOFOLLOW) != 0) {
|
||||||
bool absent = errno == ENOENT;
|
bool absent = errno == ENOENT || errno == ENOTDIR;
|
||||||
close(parent_fd);
|
close(parent_fd);
|
||||||
free(leaf);
|
free(leaf);
|
||||||
return absent;
|
return absent;
|
||||||
}
|
}
|
||||||
int rc;
|
if (S_ISDIR(st.st_mode)) {
|
||||||
if (S_ISDIR(st.st_mode))
|
if (!budget_available(session)) {
|
||||||
rc = unlinkat(parent_fd, leaf, AT_REMOVEDIR);
|
note_skipped(session);
|
||||||
else
|
close(parent_fd);
|
||||||
rc = unlinkat(parent_fd, leaf, 0);
|
free(leaf);
|
||||||
bool ok = rc == 0 || errno == ENOENT || errno == ENOTEMPTY || errno == EEXIST;
|
return true;
|
||||||
if (rc == 0) {
|
}
|
||||||
|
int dirfd = openat(parent_fd, leaf, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
|
||||||
|
if (dirfd < 0) {
|
||||||
|
bool absent = errno == ENOENT || errno == ENOTDIR;
|
||||||
|
close(parent_fd);
|
||||||
|
free(leaf);
|
||||||
|
return absent;
|
||||||
|
}
|
||||||
|
PlanSkips skips;
|
||||||
|
if (!build_plan_skips(config, session, &skips)) {
|
||||||
|
close(dirfd);
|
||||||
|
close(parent_fd);
|
||||||
|
free(leaf);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
bool survives = false;
|
||||||
|
bool ok = process_children(dirfd, rel, NULL, NULL, false, true, &skips, session, &survives);
|
||||||
|
free(skips.entries);
|
||||||
|
close(dirfd);
|
||||||
|
if (!ok) {
|
||||||
|
close(parent_fd);
|
||||||
|
free(leaf);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
if (!survives) {
|
||||||
|
if (!budget_available(session)) {
|
||||||
|
note_skipped(session);
|
||||||
|
} else if (unlinkat(parent_fd, leaf, AT_REMOVEDIR) == 0) {
|
||||||
|
session->deleted++;
|
||||||
|
session->planned++;
|
||||||
|
log_deleted(rel);
|
||||||
|
notify_deleted(session, rel);
|
||||||
|
} else if (errno != ENOENT && errno != ENOTEMPTY && errno != EEXIST) {
|
||||||
|
close(parent_fd);
|
||||||
|
free(leaf);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
close(parent_fd);
|
||||||
|
free(leaf);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (!budget_available(session)) {
|
||||||
|
note_skipped(session);
|
||||||
|
close(parent_fd);
|
||||||
|
free(leaf);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (unlinkat(parent_fd, leaf, 0) == 0) {
|
||||||
session->deleted++;
|
session->deleted++;
|
||||||
|
session->planned++;
|
||||||
log_deleted(rel);
|
log_deleted(rel);
|
||||||
notify_deleted(session, rel);
|
notify_deleted(session, rel);
|
||||||
|
} else if (errno != ENOENT) {
|
||||||
|
close(parent_fd);
|
||||||
|
free(leaf);
|
||||||
|
return false;
|
||||||
}
|
}
|
||||||
close(parent_fd);
|
close(parent_fd);
|
||||||
free(leaf);
|
free(leaf);
|
||||||
return ok;
|
return true;
|
||||||
}
|
}
|
||||||
|
|
||||||
void delete_plan_session_set_delete_observer(DeletePlanSession* session,
|
void delete_plan_session_set_delete_observer(DeletePlanSession* session,
|
||||||
|
|||||||
@@ -33,6 +33,11 @@ typedef enum {
|
|||||||
LOG_INFO_FLIST = 1u << 7,
|
LOG_INFO_FLIST = 1u << 7,
|
||||||
LOG_INFO_NONREG = 1u << 8,
|
LOG_INFO_NONREG = 1u << 8,
|
||||||
LOG_INFO_PROGRESS = 1u << 9,
|
LOG_INFO_PROGRESS = 1u << 9,
|
||||||
|
/* Marker for `--info=name2` and higher: also print rsync's
|
||||||
|
"NAME is uptodate" line for entries the receiver already has. It rides in
|
||||||
|
the info_level bitset (there is no separate Config field) and is never set
|
||||||
|
by --info=all (which selects level 1). */
|
||||||
|
LOG_INFO_NAME_UPTODATE = 1u << 10,
|
||||||
LOG_INFO_ALL = LOG_INFO_COPY | LOG_INFO_MISC | LOG_INFO_SKIP | LOG_INFO_STATS | LOG_INFO_DEL |
|
LOG_INFO_ALL = LOG_INFO_COPY | LOG_INFO_MISC | LOG_INFO_SKIP | LOG_INFO_STATS | LOG_INFO_DEL |
|
||||||
LOG_INFO_REMOVE | LOG_INFO_NAME | LOG_INFO_FLIST | LOG_INFO_NONREG |
|
LOG_INFO_REMOVE | LOG_INFO_NAME | LOG_INFO_FLIST | LOG_INFO_NONREG |
|
||||||
LOG_INFO_PROGRESS,
|
LOG_INFO_PROGRESS,
|
||||||
|
|||||||
@@ -0,0 +1 @@
|
|||||||
|
OLDDEST
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
NEWCONTENT
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
NEWCONTENT
|
||||||
@@ -1,19 +1,16 @@
|
|||||||
"""Differential coverage for ``--delete-delay`` + ``--max-delete`` with a
|
"""Differential coverage for ``--delete-delay`` + ``--max-delete`` with a
|
||||||
refilled deferred directory.
|
refilled deferred directory.
|
||||||
|
|
||||||
FastSync snapshots a directory's extras at plan time (``defer_add``) and charges
|
FastSync snapshots a directory's extras at plan time (``defer_add``) but charges
|
||||||
``--max-delete`` then, and its deferred commit only removes the snapshot path, so
|
``--max-delete`` only when a path is actually removed, and its deferred commit
|
||||||
a directory refilled before the commit survives ``ENOTEMPTY``. rsync computes
|
re-scans a queued directory and removes content created after the plan -- the
|
||||||
the deferred deletions during the transfer, charges ``--max-delete`` on actual
|
same rules as rsync. These tests run both tools on the same fixture and assert
|
||||||
removals, and recursively removes a queued directory -- so content created after
|
both sides remove the late content (recursively) and bound the deletion with
|
||||||
the plan inside an extra directory is removed too.
|
``--max-delete`` identically.
|
||||||
|
|
||||||
These tests run both tools on the same fixture and pin the shared budget bound
|
They are not part of the fast PR gate because the rsync side needs a wide
|
||||||
(the later extra survives, both exit 25) plus the documented residual (the
|
real-time injection window (a throttled transfer), while the FastSync side uses
|
||||||
refilled directory's late content survives under FastSync, not rsync). They are
|
the existing byte-deterministic slicing proxy.
|
||||||
not part of the fast PR gate because the rsync side needs a wide real-time
|
|
||||||
injection window (a throttled transfer), while the FastSync side uses the
|
|
||||||
existing byte-deterministic slicing proxy.
|
|
||||||
|
|
||||||
The refilled directory sits at the transfer ROOT, whose delete plan is always
|
The refilled directory sits at the transfer ROOT, whose delete plan is always
|
||||||
processed before any subdirectory's, so the budget is deterministically charged
|
processed before any subdirectory's, so the budget is deterministically charged
|
||||||
@@ -101,7 +98,7 @@ def _rsync(args, timeout=120):
|
|||||||
|
|
||||||
|
|
||||||
class TestDeleteDelayRefilledDirVsRsync:
|
class TestDeleteDelayRefilledDirVsRsync:
|
||||||
"""The shared budget bound and the documented recursive-removal residual."""
|
"""Both tools charge --max-delete on actual removals and recurse."""
|
||||||
|
|
||||||
def _fastsync_refilled(self, tag, max_delete=None):
|
def _fastsync_refilled(self, tag, max_delete=None):
|
||||||
"""Run FastSync with the refill injected deterministically by the proxy
|
"""Run FastSync with the refill injected deterministically by the proxy
|
||||||
@@ -125,18 +122,18 @@ class TestDeleteDelayRefilledDirVsRsync:
|
|||||||
return result, received, late
|
return result, received, late
|
||||||
|
|
||||||
@requires_rsync
|
@requires_rsync
|
||||||
def test_max_delete_budget_bound_matches_and_residual_pinned(self):
|
def test_max_delete_budget_bound_matches(self):
|
||||||
# --- FastSync: budget charged at snapshot; late content preserved ---
|
# --- FastSync: the one actual removal is the late file; dirs survive ---
|
||||||
result, received, late = self._fastsync_refilled("budget_fs", max_delete=1)
|
result, received, late = self._fastsync_refilled("budget_fs", max_delete=1)
|
||||||
assert result.returncode == 25, (result.stderr or result.stdout)[:300]
|
assert result.returncode == 25, (result.stderr or result.stdout)[:300]
|
||||||
assert _deleted_count(result.stdout) == 0, result.stdout
|
assert _deleted_count(result.stdout) == 1, result.stdout
|
||||||
assert os.path.exists(late), "FastSync removed the late content of a snapshotted dir"
|
assert not os.path.exists(late), "FastSync kept the late content of a queued dir"
|
||||||
assert os.path.isdir(os.path.join(received, "xdir"))
|
assert os.path.isdir(os.path.join(received, "xdir"))
|
||||||
assert os.path.isdir(os.path.join(received, "b", "ydir")), (
|
assert os.path.isdir(os.path.join(received, "b", "ydir")), (
|
||||||
"FastSync did not charge the plan-time budget: b/ydir was removed"
|
"FastSync did not bound the deletion with --max-delete=1"
|
||||||
)
|
)
|
||||||
|
|
||||||
# --- rsync: budget charged on actual removals; dirs removed recursively ---
|
# --- rsync: same budget rule and recursive removal ---
|
||||||
source, rsync_dst = _seed_rsync("budget_rsync")
|
source, rsync_dst = _seed_rsync("budget_rsync")
|
||||||
|
|
||||||
def inject():
|
def inject():
|
||||||
@@ -151,26 +148,23 @@ class TestDeleteDelayRefilledDirVsRsync:
|
|||||||
)
|
)
|
||||||
t.join()
|
t.join()
|
||||||
assert rsync_result.returncode == 25, rsync_result.stderr
|
assert rsync_result.returncode == 25, rsync_result.stderr
|
||||||
# rsync removes the late content (recursive deferred removal); FastSync
|
assert _deleted_count(rsync_result.stdout) == _deleted_count(result.stdout)
|
||||||
# keeps it and charges the snapshot directive instead.
|
|
||||||
assert not os.path.exists(os.path.join(rsync_dst, "xdir", "new.txt")), (
|
assert not os.path.exists(os.path.join(rsync_dst, "xdir", "new.txt")), (
|
||||||
"rsync kept late content inside a queued directory"
|
"rsync kept late content inside a queued directory"
|
||||||
)
|
)
|
||||||
# The shared observable: the later extra survives in both tools under
|
|
||||||
# --max-delete=1, and both report the capped run with exit 25.
|
|
||||||
assert os.path.isdir(os.path.join(rsync_dst, "b", "ydir")), (
|
assert os.path.isdir(os.path.join(rsync_dst, "b", "ydir")), (
|
||||||
"rsync did not bound the deletion with --max-delete=1"
|
"rsync did not bound the deletion with --max-delete=1"
|
||||||
)
|
)
|
||||||
assert os.path.isdir(os.path.join(received, "b", "ydir"))
|
assert os.path.isdir(os.path.join(received, "b", "ydir"))
|
||||||
|
|
||||||
@requires_rsync
|
@requires_rsync
|
||||||
def test_refilled_extra_dir_recursive_removal_residual(self):
|
def test_refilled_extra_dir_recursive_removal_matches(self):
|
||||||
"""Without --max-delete the residual is a plain tree difference: rsync
|
"""Without --max-delete both tools remove the refilled extra directory
|
||||||
removes the refilled extra directory (and its late content), FastSync
|
(and its late content)."""
|
||||||
leaves the snapshot path in place on ENOTEMPTY."""
|
|
||||||
result, received, late = self._fastsync_refilled("recur_fs")
|
result, received, late = self._fastsync_refilled("recur_fs")
|
||||||
assert result.returncode == 0, (result.stderr or result.stdout)[:300]
|
assert result.returncode == 0, (result.stderr or result.stdout)[:300]
|
||||||
assert os.path.exists(late), "FastSync removed the refilled directory's late content"
|
assert not os.path.exists(late), "FastSync kept the refilled directory's late content"
|
||||||
|
assert not os.path.isdir(os.path.join(received, "xdir"))
|
||||||
|
|
||||||
source, rsync_dst = _seed_rsync("recur_rsync")
|
source, rsync_dst = _seed_rsync("recur_rsync")
|
||||||
|
|
||||||
|
|||||||
@@ -320,10 +320,10 @@ class TestDeleteTimingFailure:
|
|||||||
class TestDeleteDelayDeletedCount:
|
class TestDeleteDelayDeletedCount:
|
||||||
"""The reported deleted count must reflect entries actually removed."""
|
"""The reported deleted count must reflect entries actually removed."""
|
||||||
|
|
||||||
def test_refilled_deferred_dir_is_not_counted(self):
|
def test_refilled_deferred_dir_is_recursively_removed_and_counted(self):
|
||||||
"""A directory snapshotted into a --delete-delay plan that is refilled
|
"""A directory snapshotted into a --delete-delay plan that is refilled
|
||||||
before the commit survives ENOTEMPTY and must NOT inflate "Number of
|
before the commit is re-scanned and removed recursively (rsync parity):
|
||||||
deleted files" (regression for delete_plan.c counting at snapshot)."""
|
the late file and the directory are both counted as deleted."""
|
||||||
source = os.path.join(TEST_DATA_DIR, "ddc_src")
|
source = os.path.join(TEST_DATA_DIR, "ddc_src")
|
||||||
dest = os.path.join(TEST_DATA_DIR, "ddc_dst")
|
dest = os.path.join(TEST_DATA_DIR, "ddc_dst")
|
||||||
clean_dir(source)
|
clean_dir(source)
|
||||||
@@ -347,12 +347,13 @@ class TestDeleteDelayDeletedCount:
|
|||||||
proxy.finish()
|
proxy.finish()
|
||||||
assert result.returncode == 0, (result.stderr or result.stdout)[:400]
|
assert result.returncode == 0, (result.stderr or result.stdout)[:400]
|
||||||
assert proxy.hook_called.is_set(), "hook never fired"
|
assert proxy.hook_called.is_set(), "hook never fired"
|
||||||
assert os.path.exists(os.path.join(extra_dir, "new.txt")), "late file vanished"
|
assert not os.path.exists(os.path.join(extra_dir, "new.txt")), "late file survived"
|
||||||
|
assert not os.path.isdir(extra_dir), "refilled extra dir survived"
|
||||||
deleted = None
|
deleted = None
|
||||||
for line in result.stdout.splitlines():
|
for line in result.stdout.splitlines():
|
||||||
if line.startswith("Number of deleted files:"):
|
if line.startswith("Number of deleted files:"):
|
||||||
deleted = int(line.split(":", 1)[1].split()[0])
|
deleted = int(line.split(":", 1)[1].split()[0])
|
||||||
assert deleted == 0, (deleted, result.stdout)
|
assert deleted == 2, (deleted, result.stdout)
|
||||||
|
|
||||||
|
|
||||||
class TestDeleteDelayMaxDeleteParity:
|
class TestDeleteDelayMaxDeleteParity:
|
||||||
@@ -468,10 +469,9 @@ class TestDeleteAfterThreadsKeepSet:
|
|||||||
)
|
)
|
||||||
|
|
||||||
class TestDeleteDelayMaxDeleteRefilledDir:
|
class TestDeleteDelayMaxDeleteRefilledDir:
|
||||||
"""--delete-delay charges the --max-delete budget at plan/snapshot time, so a
|
"""--delete-delay charges the --max-delete budget on ACTUAL removals: the
|
||||||
refilled snapshotted directory that survives ENOTEMPTY still spends its slot
|
refilled directory's late content is removed first (consuming the one slot),
|
||||||
and a later extra is skipped, while the reported count stays at actual
|
so the directory itself and a later extra are skipped, matching rsync.
|
||||||
removals.
|
|
||||||
|
|
||||||
The refilled directory is at the destination ROOT (its plan is always sent
|
The refilled directory is at the destination ROOT (its plan is always sent
|
||||||
first) and the skipped extra is under a separate source directory, so the
|
first) and the skipped extra is under a separate source directory, so the
|
||||||
@@ -479,7 +479,7 @@ class TestDeleteDelayMaxDeleteRefilledDir:
|
|||||||
order. The refill is injected through the byte-barrier proxy so it is
|
order. The refill is injected through the byte-barrier proxy so it is
|
||||||
causally after the plan frame."""
|
causally after the plan frame."""
|
||||||
|
|
||||||
def test_budget_charged_at_plan_time(self):
|
def test_budget_charged_on_actual_removal(self):
|
||||||
source = os.path.join(TEST_DATA_DIR, "ddmb_src")
|
source = os.path.join(TEST_DATA_DIR, "ddmb_src")
|
||||||
dest = os.path.join(TEST_DATA_DIR, "ddmb_dst")
|
dest = os.path.join(TEST_DATA_DIR, "ddmb_dst")
|
||||||
clean_dir(source)
|
clean_dir(source)
|
||||||
@@ -504,9 +504,9 @@ class TestDeleteDelayMaxDeleteRefilledDir:
|
|||||||
proxy.finish()
|
proxy.finish()
|
||||||
assert result.returncode == 25, (result.stderr or result.stdout)[:400]
|
assert result.returncode == 25, (result.stderr or result.stdout)[:400]
|
||||||
assert proxy.hook_called.is_set(), "hook never fired"
|
assert proxy.hook_called.is_set(), "hook never fired"
|
||||||
# The refilled directory still consumes the plan-time budget, so the
|
# The late content consumes the single budget slot; the refilled
|
||||||
# later extra is skipped...
|
# directory itself and the later extra are skipped.
|
||||||
assert os.path.exists(os.path.join(refilled_dir, "new.txt")), "late file vanished"
|
assert not os.path.exists(os.path.join(refilled_dir, "new.txt")), "late file survived"
|
||||||
assert os.path.isdir(later_dir), "later extra was not skipped by the plan-time budget"
|
assert os.path.isdir(later_dir), "later extra was not skipped by the budget"
|
||||||
# ...while the reported count reflects only actual removals (none here).
|
# The one actual removal is reported.
|
||||||
assert _deleted_count(result.stdout) == 0, result.stdout
|
assert _deleted_count(result.stdout) == 1, result.stdout
|
||||||
|
|||||||
@@ -584,53 +584,53 @@ class TestRemoteDryRun:
|
|||||||
assert _snapshot_tree(received) == before, f"{flags} mutated the destination"
|
assert _snapshot_tree(received) == before, f"{flags} mutated the destination"
|
||||||
|
|
||||||
@pytest.mark.skipif(shutil.which("rsync") is None, reason="rsync not installed")
|
@pytest.mark.skipif(shutil.which("rsync") is None, reason="rsync not installed")
|
||||||
def test_dry_run_delete_lines_over_report_residual(self):
|
def test_dry_run_delete_lines_match_rsync(self):
|
||||||
"""Documented residual (RSYNC_COMPAT.md `-n/--dry-run` row): FastSync's
|
"""`-n --delete` lists exactly the destination extras rsync would remove.
|
||||||
dry-run would-delete report includes the file that is merely being
|
|
||||||
updated (derived from the receiver's STATUS_STATS extras) and, unlike
|
Covers the three cases that a real run protects: the file being updated
|
||||||
rsync, also reports an excluded-but-protected extra. rsync `-n -i
|
(in the keep set), a filter-excluded source entry (protected prefix), and
|
||||||
--delete` lists only genuine extras. Pins the residual that keeps the
|
a --max-size-pruned source entry (always-protected prefix). Only the
|
||||||
row Divergent."""
|
genuine destination-only extras may appear. Residual: a destination-only
|
||||||
|
entry matching an exclude pattern is still removed (FastSync derives
|
||||||
|
delete protection from the source scan, not a receiver filter engine);
|
||||||
|
that divergence is pinned by TestOptionParity.
|
||||||
|
"""
|
||||||
source = os.path.join(TEST_DATA_DIR, "dryrep_src")
|
source = os.path.join(TEST_DATA_DIR, "dryrep_src")
|
||||||
rdst = os.path.join(TEST_DATA_DIR, "dryrep_rdst")
|
rdst = os.path.join(TEST_DATA_DIR, "dryrep_rdst")
|
||||||
fdst = os.path.join(TEST_DATA_DIR, "dryrep_fdst")
|
fdst = os.path.join(TEST_DATA_DIR, "dryrep_fdst")
|
||||||
clean_dir(source)
|
clean_dir(source)
|
||||||
clean_dir(rdst)
|
clean_dir(rdst)
|
||||||
clean_dir(fdst)
|
clean_dir(fdst)
|
||||||
with open(os.path.join(source, "a.txt"), "wb") as fh:
|
for name, data in (("a.txt", b"new content\n"), ("keep.log", b"log\n"),
|
||||||
fh.write(b"new content\n")
|
("big.bin", b"B" * 2000)):
|
||||||
|
with open(os.path.join(source, name), "wb") as fh:
|
||||||
|
fh.write(data)
|
||||||
os.utime(os.path.join(source, "a.txt"), (1_700_000_000, 1_700_000_000))
|
os.utime(os.path.join(source, "a.txt"), (1_700_000_000, 1_700_000_000))
|
||||||
for root in (rdst, fdst):
|
received = get_dest_received_dir(fdst, source)
|
||||||
with open(os.path.join(root, "a.txt"), "wb") as fh:
|
os.makedirs(received, exist_ok=True)
|
||||||
fh.write(b"old\n")
|
for root in (rdst, received):
|
||||||
for name, data in (("extra.log", b"log\n"), ("extra.txt", b"extra\n")):
|
for name, data in (("a.txt", b"old\n"), ("keep.log", b"log\n"),
|
||||||
|
("big.bin", b"B" * 2000), ("extra.txt", b"extra\n")):
|
||||||
with open(os.path.join(root, name), "wb") as fh:
|
with open(os.path.join(root, name), "wb") as fh:
|
||||||
fh.write(data)
|
fh.write(data)
|
||||||
for p in (os.path.join(root, "a.txt"), os.path.join(root, "extra.log"),
|
os.utime(os.path.join(root, name), (1_500_000_000, 1_500_000_000))
|
||||||
os.path.join(root, "extra.txt")):
|
|
||||||
os.utime(p, (1_500_000_000, 1_500_000_000))
|
|
||||||
|
|
||||||
|
flags = ["-a", "-n", "-i", "--delete", "--exclude=*.log", "--max-size=1000"]
|
||||||
r = subprocess.run(["rsync", "-an", "-i", "--delete", "--exclude=*.log",
|
r = subprocess.run(["rsync", "-an", "-i", "--delete", "--exclude=*.log",
|
||||||
source + "/", rdst + "/"],
|
"--max-size=1000", source + "/", rdst + "/"],
|
||||||
capture_output=True, text=True,
|
capture_output=True, text=True,
|
||||||
env=dict(os.environ, LC_ALL="C"))
|
env=dict(os.environ, LC_ALL="C"))
|
||||||
assert r.returncode == 0, r.stderr
|
assert r.returncode == 0, r.stderr
|
||||||
rsync_del = {l.split(None, 1)[1] for l in r.stdout.splitlines()
|
rsync_del = sorted(l for l in r.stdout.splitlines() if l.startswith("*deleting"))
|
||||||
if l.startswith("*deleting")}
|
assert rsync_del == ["*deleting extra.txt"], f"unexpected rsync set: {rsync_del}"
|
||||||
assert rsync_del == {"extra.txt"}, f"unexpected rsync deleting set: {rsync_del}"
|
|
||||||
|
|
||||||
with ServerManager() as server:
|
with ServerManager() as server:
|
||||||
server.start(extra_args=["--allow-delete"])
|
server.start(extra_args=["--allow-delete"])
|
||||||
result, _ = run_client(source, fdst,
|
result, _ = run_client(source, fdst, flags=flags, port=server.port)
|
||||||
flags=["-a", "-n", "-i", "--delete", "--exclude=*.log"],
|
|
||||||
port=server.port)
|
|
||||||
assert result.returncode == 0, (result.stderr or result.stdout)[:300]
|
assert result.returncode == 0, (result.stderr or result.stdout)[:300]
|
||||||
fs_del = {l.split(None, 1)[1] for l in (result.stdout or "").splitlines()
|
fs_del = sorted(l for l in (result.stdout or "").splitlines()
|
||||||
if l.startswith("*deleting")}
|
if l.startswith("*deleting"))
|
||||||
# Documented over-report: the transferred/updated file and the excluded
|
assert fs_del == rsync_del, f"rsync={rsync_del}\nfastsync={fs_del}"
|
||||||
# extra appear in FastSync's would-delete set.
|
|
||||||
assert "a.txt" in fs_del, "residual changed: FastSync no longer over-reports the update"
|
|
||||||
assert "extra.log" in fs_del, "residual changed: FastSync no longer reports excluded extra"
|
|
||||||
|
|
||||||
@pytest.mark.ci
|
@pytest.mark.ci
|
||||||
def test_remote_dry_run_quiet_is_silent(self, shared_server):
|
def test_remote_dry_run_quiet_is_silent(self, shared_server):
|
||||||
|
|||||||
@@ -164,6 +164,64 @@ class TestInfoParity:
|
|||||||
f"rsync={entries(rsync_result.stdout)} fastsync={entries(result.stdout)}"
|
f"rsync={entries(rsync_result.stdout)} fastsync={entries(result.stdout)}"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
@requires_rsync
|
||||||
|
@pytest.mark.ci
|
||||||
|
def test_info_name_root_line_matches_rsync(self, shared_server):
|
||||||
|
"""A fresh destination: rsync prints `created directory`, then the
|
||||||
|
transfer-root `./` name line before the entries; FastSync must emit the
|
||||||
|
same `./` line."""
|
||||||
|
source = os.path.join(TEST_DATA_DIR, "inf_root_src")
|
||||||
|
dest = os.path.join(TEST_DATA_DIR, "inf_root_dst")
|
||||||
|
rdst = os.path.join(TEST_DATA_DIR, "inf_root_rdst")
|
||||||
|
clean_dir(source)
|
||||||
|
_write(os.path.join(source, "f.bin"), b"payload\n")
|
||||||
|
clean_dir(dest)
|
||||||
|
shutil.rmtree(rdst, ignore_errors=True)
|
||||||
|
rsync_result = _rsync(["-a", "--info=name", source + "/", rdst + "/"])
|
||||||
|
assert rsync_result.returncode == 0, rsync_result.stderr
|
||||||
|
result, _ = run_client(source, dest, flags=["-a", "--info=name"],
|
||||||
|
port=shared_server.port)
|
||||||
|
assert result.returncode == 0, (result.stderr or result.stdout)[:200]
|
||||||
|
|
||||||
|
def names(text):
|
||||||
|
return [l for l in text.splitlines()
|
||||||
|
if l and not l.startswith("created directory")
|
||||||
|
and not (l.endswith("/") and l != "./")]
|
||||||
|
|
||||||
|
assert names(rsync_result.stdout) == ["./", "f.bin"], names(rsync_result.stdout)
|
||||||
|
assert names(result.stdout) == ["./", "f.bin"], names(result.stdout)
|
||||||
|
|
||||||
|
@requires_rsync
|
||||||
|
@pytest.mark.ci
|
||||||
|
def test_info_name2_uptodate_matches_rsync(self, shared_server):
|
||||||
|
"""--info=name2 prints `NAME is uptodate` for entries the receiver
|
||||||
|
already has, matching rsync byte-for-byte."""
|
||||||
|
source = os.path.join(TEST_DATA_DIR, "inf_up_src")
|
||||||
|
dest = os.path.join(TEST_DATA_DIR, "inf_up_dst")
|
||||||
|
rdst = os.path.join(TEST_DATA_DIR, "inf_up_rdst")
|
||||||
|
clean_dir(source)
|
||||||
|
os.makedirs(os.path.join(source, "sub"))
|
||||||
|
_write(os.path.join(source, "a.txt"), b"a\n")
|
||||||
|
_write(os.path.join(source, "sub", "b.txt"), b"b\n")
|
||||||
|
clean_dir(rdst)
|
||||||
|
assert _rsync(["-a", source + "/", rdst + "/"]).returncode == 0
|
||||||
|
rsync_result = _rsync(["-a", "--info=name2", source + "/", rdst + "/"])
|
||||||
|
assert rsync_result.returncode == 0, rsync_result.stderr
|
||||||
|
|
||||||
|
clean_dir(dest)
|
||||||
|
seed, _ = run_client(source, dest, flags=["-a", "--incremental"],
|
||||||
|
port=shared_server.port)
|
||||||
|
assert seed.returncode == 0, (seed.stderr or seed.stdout)[:200]
|
||||||
|
result, _ = run_client(source, dest, flags=["-a", "--incremental", "--info=name2"],
|
||||||
|
port=shared_server.port)
|
||||||
|
assert result.returncode == 0, (result.stderr or result.stdout)[:200]
|
||||||
|
rsync_lines = sorted(l for l in rsync_result.stdout.splitlines()
|
||||||
|
if l.endswith("is uptodate"))
|
||||||
|
fast_lines = sorted(l for l in result.stdout.splitlines()
|
||||||
|
if l.endswith("is uptodate"))
|
||||||
|
assert fast_lines == rsync_lines, (rsync_lines, fast_lines)
|
||||||
|
assert fast_lines == ["a.txt is uptodate", "sub/b.txt is uptodate"], fast_lines
|
||||||
|
|
||||||
@requires_rsync
|
@requires_rsync
|
||||||
@pytest.mark.ci
|
@pytest.mark.ci
|
||||||
def test_info_nonreg_matches_rsync(self, shared_server):
|
def test_info_nonreg_matches_rsync(self, shared_server):
|
||||||
|
|||||||
@@ -1393,6 +1393,29 @@ static void test_parse_args_debug_info_levels() {
|
|||||||
EXPECT_EQ_INT(cfg->info_level, LOG_INFO_STATS);
|
EXPECT_EQ_INT(cfg->info_level, LOG_INFO_STATS);
|
||||||
config_delete(cfg);
|
config_delete(cfg);
|
||||||
|
|
||||||
|
/* --info=name level 2 enables the "is uptodate" marker; a later level-1 or
|
||||||
|
level-0 token clears it again. */
|
||||||
|
cfg = config_create();
|
||||||
|
char* name2_argv[] = {"fastsync", "--info=name2", "/src", "/dst"};
|
||||||
|
positional_count = 0;
|
||||||
|
EXPECT_EQ_INT(parse_args(cfg, 4, name2_argv, positional_args, &positional_count), 0);
|
||||||
|
EXPECT_EQ_INT(cfg->info_level, LOG_INFO_NAME | LOG_INFO_NAME_UPTODATE);
|
||||||
|
config_delete(cfg);
|
||||||
|
|
||||||
|
cfg = config_create();
|
||||||
|
char* name1_argv[] = {"fastsync", "--info=name2,name1", "/src", "/dst"};
|
||||||
|
positional_count = 0;
|
||||||
|
EXPECT_EQ_INT(parse_args(cfg, 4, name1_argv, positional_args, &positional_count), 0);
|
||||||
|
EXPECT_EQ_INT(cfg->info_level, LOG_INFO_NAME);
|
||||||
|
config_delete(cfg);
|
||||||
|
|
||||||
|
cfg = config_create();
|
||||||
|
char* name0_argv[] = {"fastsync", "--info=name2,name0", "/src", "/dst"};
|
||||||
|
positional_count = 0;
|
||||||
|
EXPECT_EQ_INT(parse_args(cfg, 4, name0_argv, positional_args, &positional_count), 0);
|
||||||
|
EXPECT_EQ_INT(cfg->info_level, 0);
|
||||||
|
config_delete(cfg);
|
||||||
|
|
||||||
cfg = config_create();
|
cfg = config_create();
|
||||||
char* bad_argv[] = {"fastsync", "--debug=123", "/src", "/dst"};
|
char* bad_argv[] = {"fastsync", "--debug=123", "/src", "/dst"};
|
||||||
positional_count = 0;
|
positional_count = 0;
|
||||||
|
|||||||
+28
-29
@@ -23,10 +23,9 @@ static void send_plan_frame(int fd, const char* dir) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/* --delete-delay: a directory snapshotted into the plan that is refilled before
|
/* --delete-delay: a directory snapshotted into the plan that is refilled before
|
||||||
* the commit must NOT be counted as deleted once its unlink fails ENOTEMPTY.
|
* the commit is re-scanned and removed recursively (rsync parity). Regression
|
||||||
* Regression for delete_plan.c counting at snapshot (defer_add) instead of at
|
* for the old single-unlink ENOTEMPTY path that left the directory behind. */
|
||||||
* the actual removal. */
|
static void test_delete_delay_refilled_dir_removed_recursively(void) {
|
||||||
static void test_delete_delay_refilled_dir_not_counted(void) {
|
|
||||||
char root[] = "/tmp/fastsync_dp_refill_XXXXXX";
|
char root[] = "/tmp/fastsync_dp_refill_XXXXXX";
|
||||||
EXPECT_TRUE(mkdtemp(root) != NULL);
|
EXPECT_TRUE(mkdtemp(root) != NULL);
|
||||||
char extra[1024];
|
char extra[1024];
|
||||||
@@ -57,16 +56,14 @@ static void test_delete_delay_refilled_dir_not_counted(void) {
|
|||||||
close(fd);
|
close(fd);
|
||||||
|
|
||||||
EXPECT_EQ_INT(delete_plan_session_commit(session, config), DELETE_COMMIT_OK);
|
EXPECT_EQ_INT(delete_plan_session_commit(session, config), DELETE_COMMIT_OK);
|
||||||
/* ENOTEMPTY: the directory survives, so it must not be reported as deleted. */
|
/* The late content and the directory itself are both removed. */
|
||||||
EXPECT_EQ_INT((int)delete_plan_session_deleted(session), 0);
|
EXPECT_EQ_INT((int)delete_plan_session_deleted(session), 2);
|
||||||
struct stat st;
|
struct stat st;
|
||||||
EXPECT_EQ_INT(lstat(extra, &st), 0);
|
EXPECT_TRUE(lstat(extra, &st) != 0);
|
||||||
|
|
||||||
delete_plan_session_destroy(session);
|
delete_plan_session_destroy(session);
|
||||||
close(p[0]);
|
close(p[0]);
|
||||||
close(p[1]);
|
close(p[1]);
|
||||||
unlink(refill);
|
|
||||||
rmdir(extra);
|
|
||||||
rmdir(root);
|
rmdir(root);
|
||||||
config_delete(config);
|
config_delete(config);
|
||||||
}
|
}
|
||||||
@@ -106,8 +103,9 @@ static void test_delete_delay_removed_file_counted(void) {
|
|||||||
config_delete(config);
|
config_delete(config);
|
||||||
}
|
}
|
||||||
|
|
||||||
/* --max-delete still bounds the deferred plan; the actual (removed) count must
|
/* --max-delete is charged on actual removals, not at plan/snapshot time: after
|
||||||
* not exceed the limit even though more extras existed. */
|
* receiving the plans the budget is untouched, and only the commit removes up to
|
||||||
|
* the limit. */
|
||||||
static void test_delete_delay_max_delete_bounds_actual(void) {
|
static void test_delete_delay_max_delete_bounds_actual(void) {
|
||||||
char root[] = "/tmp/fastsync_dp_max_XXXXXX";
|
char root[] = "/tmp/fastsync_dp_max_XXXXXX";
|
||||||
EXPECT_TRUE(mkdtemp(root) != NULL);
|
EXPECT_TRUE(mkdtemp(root) != NULL);
|
||||||
@@ -133,8 +131,11 @@ static void test_delete_delay_max_delete_bounds_actual(void) {
|
|||||||
EXPECT_NOT_NULL(session);
|
EXPECT_NOT_NULL(session);
|
||||||
send_plan_frame(p[1], ".");
|
send_plan_frame(p[1], ".");
|
||||||
EXPECT_EQ_INT(delete_plan_session_receive(session, config, p[0]), 0);
|
EXPECT_EQ_INT(delete_plan_session_receive(session, config, p[0]), 0);
|
||||||
EXPECT_TRUE(delete_plan_session_limit_reached(session));
|
/* Nothing removed yet, so the budget is not consumed at snapshot time. */
|
||||||
|
EXPECT_FALSE(delete_plan_session_limit_reached(session));
|
||||||
|
EXPECT_EQ_INT((int)delete_plan_session_deleted(session), 0);
|
||||||
EXPECT_EQ_INT(delete_plan_session_commit(session, config), DELETE_COMMIT_LIMIT_REACHED);
|
EXPECT_EQ_INT(delete_plan_session_commit(session, config), DELETE_COMMIT_LIMIT_REACHED);
|
||||||
|
EXPECT_TRUE(delete_plan_session_limit_reached(session));
|
||||||
EXPECT_EQ_INT((int)delete_plan_session_deleted(session), 1);
|
EXPECT_EQ_INT((int)delete_plan_session_deleted(session), 1);
|
||||||
|
|
||||||
delete_plan_session_destroy(session);
|
delete_plan_session_destroy(session);
|
||||||
@@ -149,13 +150,12 @@ static void test_delete_delay_max_delete_bounds_actual(void) {
|
|||||||
config_delete(config);
|
config_delete(config);
|
||||||
}
|
}
|
||||||
|
|
||||||
/* --max-delete is charged at plan/snapshot time, not at actual removal: a
|
/* --max-delete is charged on ACTUAL removals: the refilled directory's late
|
||||||
* deferred entry that survives ENOTEMPTY still consumes its budget slot, so a
|
* content is removed first (consuming the single budget slot), so the directory
|
||||||
* later directory's extra is skipped even though nothing was actually removed.
|
* itself and the later extra are skipped, matching rsync. The two plans are
|
||||||
* The reported count stays 0 (actual removals) while the run is partial. The
|
* sent as separate frames for "a" then "b", so the ordering that decides which
|
||||||
* two plans are sent as separate frames for "a" then "b", so the ordering that
|
* entry gets the budget is deterministic (unlike readdir order). */
|
||||||
* decides which entry gets the budget is deterministic (unlike readdir order). */
|
static void test_delete_delay_actual_removal_charges_budget(void) {
|
||||||
static void test_delete_delay_refilled_dir_charges_budget_at_plan(void) {
|
|
||||||
char root[] = "/tmp/fastsync_dp_planbudget_XXXXXX";
|
char root[] = "/tmp/fastsync_dp_planbudget_XXXXXX";
|
||||||
EXPECT_TRUE(mkdtemp(root) != NULL);
|
EXPECT_TRUE(mkdtemp(root) != NULL);
|
||||||
char adir[1024], bdir[1024], xdir[1024], ydir[1024];
|
char adir[1024], bdir[1024], xdir[1024], ydir[1024];
|
||||||
@@ -180,18 +180,16 @@ static void test_delete_delay_refilled_dir_charges_budget_at_plan(void) {
|
|||||||
|
|
||||||
DeletePlanSession* session = delete_plan_session_create(config);
|
DeletePlanSession* session = delete_plan_session_create(config);
|
||||||
EXPECT_NOT_NULL(session);
|
EXPECT_NOT_NULL(session);
|
||||||
/* Plan "a" first: its empty extra dir snapshots and charges the budget. */
|
/* Both plan snapshots are taken; neither consumes budget yet. */
|
||||||
send_plan_frame(p[1], "a");
|
send_plan_frame(p[1], "a");
|
||||||
EXPECT_EQ_INT(delete_plan_session_receive(session, config, p[0]), 0);
|
EXPECT_EQ_INT(delete_plan_session_receive(session, config, p[0]), 0);
|
||||||
EXPECT_FALSE(delete_plan_session_limit_reached(session));
|
EXPECT_FALSE(delete_plan_session_limit_reached(session));
|
||||||
/* Plan "b": the budget is already spent at snapshot time, so b/y is skipped
|
|
||||||
even though a/x has not (and will not) be removed. */
|
|
||||||
send_plan_frame(p[1], "b");
|
send_plan_frame(p[1], "b");
|
||||||
EXPECT_EQ_INT(delete_plan_session_receive(session, config, p[0]), 0);
|
EXPECT_EQ_INT(delete_plan_session_receive(session, config, p[0]), 0);
|
||||||
EXPECT_TRUE(delete_plan_session_limit_reached(session));
|
EXPECT_FALSE(delete_plan_session_limit_reached(session));
|
||||||
EXPECT_EQ_INT((int)delete_plan_session_deleted(session), 0);
|
EXPECT_EQ_INT((int)delete_plan_session_deleted(session), 0);
|
||||||
|
|
||||||
/* Refill a/x so its deferred rmdir fails ENOTEMPTY. */
|
/* Refill a/x after its plan: the recursive commit must remove this content. */
|
||||||
char refill[1200];
|
char refill[1200];
|
||||||
snprintf(refill, sizeof(refill), "%s/new.txt", xdir);
|
snprintf(refill, sizeof(refill), "%s/new.txt", xdir);
|
||||||
int fd = open(refill, O_WRONLY | O_CREAT | O_TRUNC, 0600);
|
int fd = open(refill, O_WRONLY | O_CREAT | O_TRUNC, 0600);
|
||||||
@@ -199,16 +197,17 @@ static void test_delete_delay_refilled_dir_charges_budget_at_plan(void) {
|
|||||||
close(fd);
|
close(fd);
|
||||||
|
|
||||||
EXPECT_EQ_INT(delete_plan_session_commit(session, config), DELETE_COMMIT_LIMIT_REACHED);
|
EXPECT_EQ_INT(delete_plan_session_commit(session, config), DELETE_COMMIT_LIMIT_REACHED);
|
||||||
/* Nothing was actually removed, and the plan-time budget still stopped b/y. */
|
/* The one budget slot removed the late content; the two directories survive. */
|
||||||
EXPECT_EQ_INT((int)delete_plan_session_deleted(session), 0);
|
EXPECT_EQ_INT((int)delete_plan_session_deleted(session), 1);
|
||||||
|
EXPECT_TRUE(delete_plan_session_limit_reached(session));
|
||||||
struct stat st;
|
struct stat st;
|
||||||
|
EXPECT_TRUE(lstat(refill, &st) != 0);
|
||||||
EXPECT_EQ_INT(lstat(xdir, &st), 0);
|
EXPECT_EQ_INT(lstat(xdir, &st), 0);
|
||||||
EXPECT_EQ_INT(lstat(ydir, &st), 0);
|
EXPECT_EQ_INT(lstat(ydir, &st), 0);
|
||||||
|
|
||||||
delete_plan_session_destroy(session);
|
delete_plan_session_destroy(session);
|
||||||
close(p[0]);
|
close(p[0]);
|
||||||
close(p[1]);
|
close(p[1]);
|
||||||
unlink(refill);
|
|
||||||
rmdir(xdir);
|
rmdir(xdir);
|
||||||
rmdir(ydir);
|
rmdir(ydir);
|
||||||
rmdir(adir);
|
rmdir(adir);
|
||||||
@@ -218,8 +217,8 @@ static void test_delete_delay_refilled_dir_charges_budget_at_plan(void) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
void test_delete_plan(void) {
|
void test_delete_plan(void) {
|
||||||
test_delete_delay_refilled_dir_not_counted();
|
test_delete_delay_refilled_dir_removed_recursively();
|
||||||
test_delete_delay_removed_file_counted();
|
test_delete_delay_removed_file_counted();
|
||||||
test_delete_delay_max_delete_bounds_actual();
|
test_delete_delay_max_delete_bounds_actual();
|
||||||
test_delete_delay_refilled_dir_charges_budget_at_plan();
|
test_delete_delay_actual_removal_charges_budget();
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user