feat(basis): rsync quick-check default + FastSync-only --verify-basis
- default basis match is rsync's metadata quick-check (size + mtime; size-only drops mtime; -I disables), no mandatory content digest - new long-only --verify-basis (wire bool, protocol stays 2.28.0) restores the strict whole-file content equality - --copy-dest re-applies source attributes; basis-hit 256 MiB cap removed by streaming the copy/hash; basis miss keeps the normal payload bound - compare/copy/link-dest rows -> caveat; tally 116/13/28
This commit is contained in:
+25
@@ -132,6 +132,31 @@
|
||||
**115 ✅ / 11 ⚠️ / 31 ❌ = 157**; unit tests, the three named integration
|
||||
files, clang-format and cppcheck clean.
|
||||
|
||||
14. **Wire parity track 5a** on `feat/parity-2.28` (`PROTOCOL_VERSION` stays
|
||||
`2.28.0` by project decision): the three basis-dir options now default to
|
||||
rsync's metadata quick-check (equal size + equal mtime, or size alone under
|
||||
`--size-only`; `-I` disables matching) instead of FastSync's historical
|
||||
xxHash64 content equality, so a same-size/different-content basis is trusted
|
||||
exactly as rsync trusts it. A new FastSync-only, long-only `--verify-basis`
|
||||
flag restores the strict whole-file content equality; its bool is appended to
|
||||
the basis block of the config frame (golden wire frame 882 → 886 bytes).
|
||||
`--verify-basis` streams the confined basis descriptor to hash it, and a
|
||||
basis hit is no longer capped at the 256 MiB whole-file payload bound:
|
||||
`--copy-dest` streams the basis through a bounded buffer and `--link-dest`'s
|
||||
copy fallback streams from the basis, so an over-limit hit materializes (a
|
||||
basis MISS still falls back to the normal transfer and keeps its own bound).
|
||||
A `--copy-dest` hit re-applies the SOURCE attributes (the sender transmits
|
||||
the source metadata with the basis check frame), matching rsync's
|
||||
"copy then fix attributes"; a `--link-dest` success keeps the shared inode's
|
||||
attributes (writing through it would mutate the basis). Differential cases
|
||||
`copy_dest` and `verify_basis` added; `test_basis_dir_size_only_content_residual`
|
||||
converted to a passing parity assertion; `TestBasisDestDirs` updated for the
|
||||
new default + `--verify-basis`; unit tests cover the quick-check/verify
|
||||
decision and the same-size/different-content handshake. The
|
||||
`--compare-dest`/`--copy-dest`/`--link-dest` rows move ❌ → ⚠️ (relative-DIR
|
||||
resolution base and over-limit MISS refusal): matrix now
|
||||
**116 ✅ / 13 ⚠️ / 28 ❌ = 157**.
|
||||
|
||||
## Next steps
|
||||
1. **Merge PR #284** (`dev` -> `main`) once reviewed (protected branch).
|
||||
2. **Deferred security items** (documented, not implemented):
|
||||
|
||||
@@ -205,6 +205,7 @@ This produces `./build/client` and `./build/server`. `compile_commands.json` is
|
||||
| `--compare-dest <dir>` | Extra comparison basis: unchanged files are not transferred (requires/implies `--incremental`) |
|
||||
| `--copy-dest <dir>` | Like `--compare-dest`, but copies the unchanged file from DIR into the destination |
|
||||
| `--link-dest <dir>` | Like `--copy-dest`, but hard-links the unchanged file from DIR (repeatable; earlier DIRs win) |
|
||||
| `--verify-basis` | FastSync-only: require a basis hit (`--compare-dest`/`--copy-dest`/`--link-dest`) to match the source by whole-file digest instead of trusting the size+mtime quick-check (default matches rsync) |
|
||||
| `--delete` | Delete files on receiver not present in source (default timing: delete-after, i.e. only after the whole transfer succeeded). Scoped to the synchronized directories, so `--files-from` subsets are safe |
|
||||
| `--delete-before` | Delete extras before the transfer starts (implies `--delete`) |
|
||||
| `--delete-during`, `--del` | Delete extras once the keep-set is known, before data is applied (implies `--delete`) |
|
||||
@@ -567,6 +568,7 @@ remote SSH argv is already built injection-safe.
|
||||
| `--compare-dest <dir>` | Extra comparison basis: unchanged files are not transferred (requires/implies `--incremental`). |
|
||||
| `--copy-dest <dir>` | Like `--compare-dest`, but copies the unchanged file from DIR into the destination. |
|
||||
| `--link-dest <dir>` | Like `--copy-dest`, but hard-links the unchanged file from DIR (repeatable; earlier DIRs win). |
|
||||
| `--verify-basis` | FastSync-only: require a basis hit to match the source by whole-file digest instead of trusting the size+mtime quick-check (default matches rsync). |
|
||||
| `--preallocate` | Allocate destination file space up front (fail-fast on a full disk). |
|
||||
| `--append` | Resume a shorter destination by appending only its tail (prefix not verified; requires `--incremental`). |
|
||||
| `--append-verify` | Like `--append`, but verifies the retained prefix checksum first (falls back to a full transfer on mismatch). |
|
||||
|
||||
+15
-10
@@ -7,8 +7,8 @@ This document maps rsync's full feature set to FastSync's current implementation
|
||||
| Status | Count | Description |
|
||||
|--------|-------|-------------|
|
||||
| ✅ Parity | 116 | Reproduces rsync's semantics for this option's scope |
|
||||
| ⚠️ Caveat | 10 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
|
||||
| ❌ Divergent | 31 | Rejected, an accepted no-op, deliberately non-rsync (native config/auth/batch, privileged namespaces, safe-subset privilege), or impossible on any portable filesystem call |
|
||||
| ⚠️ Caveat | 13 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
|
||||
| ❌ Divergent | 28 | Rejected, an accepted no-op, deliberately non-rsync (native config/auth/batch, privileged namespaces, safe-subset privilege), or impossible on any portable filesystem call |
|
||||
| **Total** | **157** | One row per rsync option/feature group; a row may name several spellings |
|
||||
|
||||
This matrix reports honest rsync parity, not "implemented" as a synonym for
|
||||
@@ -49,6 +49,8 @@ matrix is **111 ✅ / 13 ⚠️ / 33 ❌ = 157**.
|
||||
|
||||
**Parity track 4a (wire, on `feat/parity-2.28`; `PROTOCOL_VERSION` stays 2.28.0).** The receiver now has a filter engine for deletion: the sender compiles its root-level selection rules exactly as the scanner does (`filter_base_build`, covering `--filter`/`-f`, `--exclude`/`--include`, `-C` and the `protect`/`risk`/`hide`/`show` words) and streams them as one bounded, self-describing block appended to the config frame (per rule: action, sides, anchored, dir-only, negate, owner, pattern; strictly validated with a bounded rule count and total pattern+owner bytes, and an unknown action/sides is a protocol error). The receiver reconstructs `protect_rules` and evaluates them first-match-wins against each extraneous destination path in every delete timing — the whole-tree commit walker (plain `--delete`/`--delete-before`/`--delete-after`), the `--delete-during`/`--delete-delay` per-directory plans, and the `-n` would-delete enumeration — so a `protect`/`P` rule now shields a DESTINATION-ONLY entry that never appeared on the sender, with `risk`/`R` cancelling. The existing sender-derived protected-prefix behavior is preserved when no rules are sent (and for source-derived protections), and `--delete-excluded` semantics are unchanged. Per-directory merge (`:`/`.`) receiver-side re-derivation is the remaining residual: those rules are still enforced only through the sender-derived protected prefixes, so a destination-only entry matching ONLY a per-directory merge rule is not yet shielded (the `-F` row keeps this documented). Differential-tested against rsync 3.4.1: `filter_protect`, `filter_protect_during`, `filter_protect_delay` (`-a --delete[-during|-delay] --filter='P *.log'` over seeded destination-only `.log` extras) plus the dry-run would-delete enumeration (`TestFilterProtect`). The `--filter=RULE` row moves ❌ → ✅. The matrix is now **115 ✅ / 11 ⚠️ / 31 ❌ = 157**.
|
||||
|
||||
**Parity track 5a (wire, on `feat/parity-2.28`; `PROTOCOL_VERSION` stays 2.28.0 by project decision).** The three basis-dir options (`--compare-dest`/`--copy-dest`/`--link-dest`) now default to rsync's metadata quick-check instead of FastSync's historical xxHash64 content equality: a basis hit is accepted on equal size plus equal mtime (or size alone under `--size-only`; `-I` disables matching), so a same-size/different-content basis is trusted exactly as rsync trusts it. A new FastSync-only, long-only `--verify-basis` flag restores the stricter whole-file content equality, and its bool is appended to the basis block of the config frame (the golden wire frame grew by one int to 886 bytes; still 2.28.0). `--verify-basis` hashes the basis by streaming its confined descriptor, so an arbitrarily large basis is verified without buffering. Basis materialization is also no longer capped at the 256 MiB whole-file payload bound: a `--copy-dest` hit streams the basis through a bounded buffer, a `--link-dest` copy fallback streams from the basis, and a hit of any size is materialized (a basis MISS still falls back to the normal transfer, which keeps its own bound). A `--copy-dest` hit re-applies the SOURCE attributes (the sender now transmits the source metadata with the basis check frame), matching rsync's "copy then fix attributes"; a `--link-dest` success keeps the shared inode's own attributes exactly as before (writing through the shared inode would mutate the basis). The `--compare-dest`/`--copy-dest`/`--link-dest` rows move ❌ → ⚠️ (residuals: the relative-DIR resolution base and the over-limit MISS refusal). The matrix is now **116 ✅ / 13 ⚠️ / 28 ❌ = 157**.
|
||||
|
||||
**Parity completion wave (protocol 2.23.0 → 2.26.0).** This wave closed the
|
||||
remaining gaps the rsync-parity wave left open (delete timing, wire counters and
|
||||
output, codec breadth, general `-R`/`-d`, the full filter grammar, receiver-side
|
||||
@@ -115,7 +117,7 @@ Every one of those has an entry below with its remaining caveats.
|
||||
| `--max-size=SIZE` | Skip files larger than SIZE | ✅ Parity | `max_size` in scanner |
|
||||
| `--min-size=SIZE` | Skip files smaller than SIZE | ✅ Parity | `min_size` in scanner |
|
||||
| `-I`, `--ignore-times` | Don't skip files matching size+time | ✅ Parity | `ignore_times` config field (crosses the wire). Disables the size+mtime quick-check in the `--incremental` per-file handshake and the basis-dir quick-match, forcing the file to be transferred rather than skipped as unchanged. Receiver-side policy: `match_by_metadata` (file_receive.c) is bypassed, so the receiver never replies `STATUS_OK` for a matching size+mtime. Requires `--incremental` to have the handshake to act on (rsync does its quick check by default; FastSync's `-I`/`--size-only`/`--modify-window` only take effect under `--incremental`, exactly like they take effect through the basis check) |
|
||||
| `--size-only` | Skip based on size only | ✅ Parity | With `--incremental`, ignores mtime |
|
||||
| `--size-only` | Skip based on size only | ✅ Parity | With `--incremental`, ignores mtime. Applies identically to the basis-dir quick-check (track 5a): a same-size basis is a hit on size alone, and under the FastSync-only `--verify-basis` the content digest is still required (size-only drops the mtime leg, never the explicit verify) |
|
||||
| `-@`, `--modify-window=NUM` | Mod-time comparison accuracy | ✅ Parity | Whole-second tolerance with nanosecond-aware comparisons |
|
||||
| `--existing` | Skip creating new files on receiver | ✅ Parity | Existing destination files continue through normal update handling. The rsync man-page alias `--ignore-non-existing` sets the same flag |
|
||||
| `--ignore-existing` | Skip updating existing files | ✅ Parity | `ignore_existing` config field (crosses the wire; receiver-side policy). Protocol 2.26.0 short-circuits in the per-file check **before any payload**: when the destination entry already exists, the receiver answers the skip during the incremental handshake instead of letting the sender stream data that would be discarded, so an existing 4 MiB destination costs only the config/check frames (verified with a counting proxy, matching rsync). The write-time paths (regular, delay-updates-staged, hardlink-sibling, special/device) still return `FILE_SAVE_SKIPPED` without overwriting, and `--backup` is disabled for skipped files. Like rsync, it does not apply to directories/symlinks. Combines with `-j`/`--threads` and `--delay-updates` |
|
||||
@@ -658,9 +660,9 @@ targets verbatim, matching rsync.
|
||||
|------|-------------------|-----------------|-------|
|
||||
| `--checksum` | Skip based on checksum | ✅ Parity | `-c`/`--checksum` compares per-file whole-file content digests to skip unchanged files. **As of protocol 2.23.0 the short `-c` implies the checksum quick-check**, so a plain `-c` run verifies content rather than only affecting the `--incremental` handshake. The digest algorithm is `xxh128` by default (protocol 2.26.0's negotiated default) and is selectable via `--checksum-choice`/`--cc` (`xxh128`/`xxh3`/`xxh64`/`xxhash`/`md5`/`md4`/`sha1`/`none`/`auto`, plus rsync's two-name form) and `--checksum-seed=NUM` (see those rows) |
|
||||
| `--checksum-choice=STR`, `--cc=STR` | Choose checksum algorithm | ✅ Parity | Real algorithm selection for the per-file whole-file digest used by the `--incremental`/`--checksum` handshake and basis-dir verification. **Protocol 2.26.0 accepts rsync 3.4.1's full set** — `xxh128` (the negotiated default), `xxh3`, `xxh64`, `xxhash`, `md5`, `md4`, `sha1`, `none`, `auto`, and the two-name `transfer,pre-transfer` form — with rsync's exit-4 rejection of an unknown name and of `none` on the transfer side when `--checksum` is on. `--cc=ALG` and space forms both parse. The algorithm id and seed cross the wire; the receiver hashes its old file with the same algorithm+seed and the per-file `STATUS_CHECK` handshake carries a bounded digest pinned to the negotiated length. `checksum_digest_file` now streams **every** supported algorithm (md4 via the self-contained RFC 1320 code, sha1/md5 via EVP, none as an empty digest), so the streaming path matches its contract, and `--out-format %C` uses the selected **transfer** half of a two-name choice and renders each algorithm byte-for-byte like rsync (xxh128 high-then-low, xxh64/xxh3 big-endian, md5/md4/sha1 standard hex, none a blank 2-char column) — differential-tested across all algorithms. **Track-3b finding — the block-checksum residual is not observable.** rsync applies the choice to the block checksum on its wire too, while FastSync selects only the whole-file comparison digest and keeps the delta BLOCK strong checksum fixed at xxHash32 (`DeltaBlockSig`, `delta_signature_create_seeded`). Because FastSync does not interoperate with rsync on the wire, only the compared surface matters, and a pre-seeded delta differential against rsync 3.4.1 (`--no-whole-file -B8192 --stats --out-format=%c|%C %n` vs `--incremental --delta`) shows the choice does not move it: across `xxh64`, `xxh128`, `xxh3`, `md5`, `md4`, `sha1` and both two-name orders the destination tree is byte-identical, `Matched data`/`Literal data`/`Total transferred file size` are unchanged (and equal to rsync's with the block size pinned), the `%c` block-checksum token is invariant (rsync `16 + 6·ceil(size/block)`, already documented under `--out-format`; FastSync its own basis-read counter), and the exit code is 0. The negotiated algorithm is visible only in `%C`, which applies it to the whole-file transfer digest and matches rsync byte-for-byte. A false block match would require the 4-byte adler32 AND the 4-byte xxHash32 to collide; at the 256 MiB maximum with the 1 KiB minimum block size the expected false matches are ≤2⁻¹⁸, and the choice cannot change this because FastSync's block strong sum is fixed. `auto` now consults `RSYNC_CHECKSUM_LIST` (rsync's whitespace-separated preference list; unknown names skipped, first supported wins, all-unknown is exit 4) before the compiled-in order; because both peers run the identical build this deterministic resolution needs no rsync peer probe, and an explicit `--cc` still wins. The list is differential-tested through `--out-format %C` (byte-identical digests to rsync for md5/sha1/xxh3) |
|
||||
| `--compare-dest=DIR` | Compare dest files relative to DIR | ❌ Divergent | DIR is a receiver-side basis; protocol 2.26.0 uses an absolute path verbatim (rsync semantics) and resolves a relative path below the destination root (`..` components are rejected, `//` collapsed and trailing `/` dropped). On the receiver's per-file check (implies `--incremental`) an exact match = same size + mtime (unless `--size-only`; `-I` disables matching) **and** equal xxHash64 of the sender's file; a match suppresses the data transfer. compare-dest never copies: it only skips a file the destination does **not** already hold (sparse destination, rsync parity), and is consulted before the normal delta/full paths. Repeatable; searched in command-line order, first match wins. **Reclassified Divergent (differential evidence):** FastSync verifies a basis hit's content with xxHash64 while rsync's `--size-only` quick check trusts size (and mtime) alone, so with a same-size/different-content basis rsync skips/links the *wrong* basis content while FastSync transfers the source — a deliberate safety-stricter behavior that cannot match rsync (see `test_basis_dir_size_only_content_residual` in `tests/integration/test_parity_quickwins.py`). Attribute-only differences on a match are also not re-applied (data is skipped so the sender never sends metadata). Sizing: FastSync's whole-file payload limit is 256 MiB on **every** transfer path (not basis-specific); rsync applies basis dirs to arbitrary sizes, so FastSync refuses a basis run whose source contains a larger file up front with a clear error before any transfer. Wire: a basis-count field is always present on the config frame (protocol 2.9.0, so clients and servers must both be 2.9.0) |
|
||||
| `--copy-dest=DIR` | Include copies of unchanged files | ❌ Divergent | Same basis rules as `--compare-dest`, but an exact match materializes a **local copy** of the DIR file into the destination (via the normal atomic temp+rename store path, so `--existing`/`--ignore-existing`/`--update`/`--backup`/`--delay-updates` all still apply) instead of transferring data. Repeatable; command-line order = priority. Content is xxHash64-verified before the copy. **Reclassified Divergent** for the same basis-hit verification divergence as `--compare-dest`: rsync's `--size-only` size/quick-check match rings a same-size/different-content file as unchanged and copies the wrong basis bytes, while FastSync's xxHash verification transfers the source (differential test `test_basis_dir_size_only_content_residual`); a basis-hit also keeps whatever metadata the copy derived from the basis rather than rsync's "copy + fix attributes" in attribute-only cases. Requires `--incremental` (implied); incompatible with `-s`. Wire: protocol 2.9.0 |
|
||||
| `--link-dest=DIR` | Hardlink to files when unchanged | ❌ Divergent | Same basis rules as `--copy-dest`, but an exact match installs an atomic **hard link** to the DIR file (temp hard link + rename) so no data or disk space is used; where the link is impossible (basis on another filesystem, filesystem refuses links) it falls back cleanly to a byte-identical local copy, never a corrupt/partial file. `--delay-updates` stages the link and publishes by rename, so the final entry stays a real hard link. Repeatable (searched in command-line order, first match wins). Content is xxHash64-verified before linking. **Reclassified Divergent** for the shared basis-hit divergence: with `--size-only` a same-size/different-content basis is linked by rsync (installing wrong content) but FastSync detects the xxHash mismatch and transfers the source (differential test `test_basis_dir_size_only_content_residual`). Other inherent caveats: protocol 2.26.0 re-links an already up-to-date destination file to the basis; a link keeps the basis inode's own mode/uid/gid and mtime — metadata is never written through the shared inode (that would mutate the basis file), so a later `--inplace` run that rewrites such a destination path **will mutate the basis snapshot** through the shared inode (use `--copy-dest` when the destination must stay independently writable); a `--remove-source-files` source satisfied by a basis dir is treated as skipped and therefore **retained** (never removed); basis dirs are excluded from `--delete`. Requires `--incremental` (implied); incompatible with `-s`. Wire: protocol 2.9.0 |
|
||||
| `--compare-dest=DIR` | Compare dest files relative to DIR | ⚠️ Caveat | DIR is a receiver-side basis; protocol 2.26.0 uses an absolute path verbatim (rsync semantics) and resolves a relative path below the destination root (`..` components are rejected, `//` collapsed and trailing `/` dropped) — note rsync resolves a relative DIR against the destination directory while FastSync resolves it below the receive root and appends the mirrored source path, so the same relative spelling addresses a different tree (use an absolute DIR for exact parity). On the receiver's per-file check (implies `--incremental`) an exact match is rsync's metadata quick-check: same size and mtime (unless `--size-only`; `-I` disables matching), with NO content digest required by default (track 5a). A match suppresses the data transfer. The FastSync-only `--verify-basis` restores the stricter whole-file content equality. compare-dest never copies: it only skips a file the destination does **not** already hold (sparse destination, rsync parity), and is consulted before the normal delta/full paths. Repeatable; searched in command-line order, first match wins. Differential-tested against rsync 3.4.1 (`compare_dest`, and `test_verify_basis_restores_strict_content`). Residual: a basis MISS above the 256 MiB whole-file payload bound is refused up front (FastSync's general whole-file limit, not basis-specific); rsync applies basis dirs to arbitrary sizes. Wire: a basis-count field plus the `verify_basis` bool are present on the config frame (protocol 2.9.0/2.28.0) |
|
||||
| `--copy-dest=DIR` | Include copies of unchanged files | ⚠️ Caveat | Same basis rules as `--compare-dest`, but an exact match materializes a **local copy** of the DIR file into the destination (via the atomic temp+rename store path, so `--existing`/`--ignore-existing`/`--update`/`--backup`/`--delay-updates` all still apply) instead of transferring data. Track 5a re-applies the SOURCE attributes on the copy (rsync's "copy then fix attributes"): the sender transmits the source metadata with the basis check frame, so the copy's mode/uid/gid/mtime match the source rather than the basis inode (differential `copy_dest` compares modes). The copy streams the basis file through a bounded buffer, so a basis larger than the whole-file payload bound still materializes. Repeatable; command-line order = priority. Requires `--incremental` (implied); incompatible with `-s`. Wire: protocol 2.9.0 |
|
||||
| `--link-dest=DIR` | Hardlink to files when unchanged | ⚠️ Caveat | Same basis rules as `--copy-dest`, but an exact match installs an atomic **hard link** to the DIR file (temp hard link + rename) so no data or disk space is used; where the link is impossible (basis on another filesystem, filesystem refuses links) it falls back cleanly to a byte-identical local copy (streamed from the basis, so an over-limit basis still works), never a corrupt/partial file. `--delay-updates` stages the link and publishes by rename, so the final entry stays a real hard link. Repeatable (searched in command-line order, first match wins). Differential-tested against rsync 3.4.1 (`link_dest`). Inherent shared-inode semantics (identical to rsync): a link keeps the basis inode's own mode/uid/gid and mtime — metadata is never written through the shared inode (that would mutate the basis file), so a later `--inplace` run that rewrites such a destination path **will mutate the basis snapshot** through the shared inode (use `--copy-dest` when the destination must stay independently writable); protocol 2.26.0 re-links an already up-to-date destination file to the basis; a `--remove-source-files` source satisfied by a basis dir is treated as skipped and therefore **retained** (never removed); basis dirs are excluded from `--delete`. Residual: a basis MISS above the 256 MiB whole-file payload bound is refused (FastSync's general whole-file limit). Requires `--incremental` (implied); incompatible with `-s`. Wire: protocol 2.9.0 |
|
||||
| `-y`, `--fuzzy`, `--no-fuzzy` | Find similar file for basis | ❌ Divergent | `-y/--fuzzy` is a pure bandwidth optimization on the existing receiver-driven delta path: when a file must be transferred and the destination holds no usable content at the exact path (file absent, or the destination file is outside the delta engine's size bounds), the receiver searches the SAME destination directory for an existing regular file whose basename is similar to the incoming name and uses it as the delta basis, so the sender transmits only the differences instead of the whole file. The output is always byte-exact regardless of which (or whether any) basis is chosen. Decision location: the receiver performs the candidate search inside `receive_incremental_check` and sends the normal `STATUS_DELTA_SIGNATURE`; the sender never learns the basis was a different file, so no new frame type or sender logic was needed — only the config frame grew a `fuzzy` boolean, so `PROTOCOL_VERSION` was bumped **2.8.0 → 2.9.0** (peers must match). Similarity heuristic (deterministic, simpler than rsync's deliberately-fuzzy matching, and documented precisely): candidates are the target's sibling entries in its destination directory, opened `O_NOFOLLOW`/`AT_SYMLINK_NOFOLLOW` under the confined root (symlinks never followed; nothing outside the destination root is ever read or hashed); dotfiles, directories, the target's own name, and the `.fastsync-stage`/temp scratch names are excluded; like the ordinary delta path, the block signature the receiver transmits is derived from on-disk content it may not otherwise send, so a negotiated `--fuzzy` run exposes the destination's sibling files (at block granularity) to the sender as a known-plaintext oracle — the same information class as the normal delta handshake over the file being replaced; the size gate is the delta engine's own bounds (both files ≥ 16 KiB, ≤ `--delta-max`, ratio ≤ 10×) rather than rsync's ~1.5× size window; protocol 2.26.0 uses a name-distance/suffix heuristic modelled on rsync's plus an exact size+mtime pass, and reads a single best candidate; the exact tie-break order can still differ from rsync's; the directory scan is capped at 4096 entries so a pathological directory cannot stall a transfer. When fuzzy applies: only to files the receiver would otherwise send whole — the destination's own file is always preferred as the delta basis when it exists and fits the delta size bounds, so fuzzy does NOT replace an existing-but-different destination basis; FastSync's 10× delta size-ratio bound means an existing destination file that is too far away in size still lets the fuzzy search run. When no similar candidate exists the transfer falls back to the normal whole-file transfer. rsync-divergence note: rsync's own matching uses a fuzzy name/size rule set; FastSync implements the closest safe deterministic approximation above. Because FastSync's delta machinery is off by default (rsync's is on), `--fuzzy` implies `--incremental` + `--delta` (unless `--whole-file`/`-W` or an explicit `--no-delta` switched delta off, in which case fuzzy is inert — matching rsync where `--whole-file` makes fuzzy irrelevant). Unlike the basis-dir options, `--fuzzy` honors an explicit `--no-incremental` (it does not force the handshake back on); an explicit `--no-incremental` also suppresses the delta implication so no invalid `--delta requires --incremental` config results. `--no-fuzzy` negates it. All surrounding semantics are untouched: a fuzzy-reconstructed file is stored as a normal file, so `--remove-source-files`, itemize/`-i`, `--stats`, `--backup`, `--delay-updates`, `--existing`/`--ignore-existing`/`--update` behave exactly as for a whole-file transfer (the fuzzy delta does not skip the file). **Reclassified Divergent:** because the output is always byte-exact, the residual is the candidate-selection heuristic itself — a deterministic name-distance/suffix rule with a 10× size-ratio window (vs rsync's ~1.5× window), not rsync's deliberately fuzzy matcher, so the chosen basis (and thus the wire bytes) can differ from rsync even though the final tree cannot. The Integration fuzzy suite (`TestFuzzy`) pins FastSync's thresholds (exact-size+mtime pass, name-distance rejection, 10× unsuitable-destination fallback); a bit-identical basis choice is not achievable without porting rsync's matcher |
|
||||
|
||||
## 12. Compression
|
||||
@@ -761,7 +763,7 @@ modes or links.
|
||||
| `--stop-after=MINS` | Stop after N minutes | ✅ Parity | Client-only sender stop deadline (Phase 6): computing `--stop-after=MINS` (a positive minute count; 0/negative/garbage rejected) and `--stop-at=TIME` (`HH:MM`, `HH:MM:SS`, or `now+N[smhd]`; a past time stops immediately). The transfer stops ELEGANTLY at the next chunk boundary: everything already fully sent is kept and applied, the run returns 0, and --delete (late/delete-after timing) does NOT wipe the destination — when the scan is cut short the partial keep-set manifest is suppressed with a warning (the delete walk is skipped rather than acting on an incomplete keep-set, so unscanned source mirrors survive). `--delete-before`/`--delete-during` still run their complete pre-scan (which ignores the deadline). Local client-only fields: never serialized into the wire config frame, so no PROTOCOL_VERSION bump. `--stop-after` uses CLOCK_MONOTONIC; `--stop-at` uses the wall clock. Works single-threaded and under `-j`/`--threads` (multithreaded). Divergence: rsync computes `--stop-after` from the run start; FastSync likewise. When both are given, the earlier of the two deadlines wins (checked per iteration). See the Phase-6 stop notes below |
|
||||
| `--stop-at=TIME` | Stop at specified time | ✅ Parity | Deadline transfer stop (client-only, never serialized). Protocol 2.26.0 accepts rsync's full date/time grammar (`2030-12-31T23:59`, `2030/12/31T23:59`, `2030-12-31`, `12-31`, `14:00`, `:59`, `1`) in addition to FastSync's `HH:MM[:SS]` and `now+N[smhd]`; a past time stops immediately. Everything already transferred is kept and an early stop suppresses the late `--delete` keep-set so unscanned source mirrors survive. Works single-threaded and under `-j`/`--threads` |
|
||||
| `--fsync` | Fsync every written file before publication | ✅ Parity | |
|
||||
| `--protocol=NUM` | Force older protocol version | ❌ Divergent | Forces the wire protocol version for this transfer. FastSync has exactly ONE wire format (`PROTOCOL_VERSION`, currently 2.27.0) with no downgrade/backward-compat code paths, so `--protocol=2.27.0` is accepted (it sets the version claim the client sends, which the server already requires to match exactly) and **every other value is rejected up front** with a clear error before any connection — it does not and cannot speak an older or virtual wire format. Divergence from rsync (which negotiates a range and downgrades to an integer 0..31): FastSync's honest contract is force-to-the-one-supported-value; a genuine downgrade would require a per-version compatibility layer that does not exist. Client-only; the server-side exact-match check is unchanged. `--protocol=2.26.0`/`2.25.0`/`2.24.0`/`2.23.0`/`2.22.0`/`2.21.0`/`2.20.0`/`2.19.0`/`2.18.0`/`2.17.0`/`2.16.0`/`2.15.0`/`216`/`31`/garbage are all rejected. See the Phase-6 protocol note below |
|
||||
| `--protocol=NUM` | Force older protocol version | ❌ Divergent | Forces the wire protocol version for this transfer. FastSync has exactly ONE wire format (`PROTOCOL_VERSION`, currently 2.28.0) with no downgrade/backward-compat code paths, so `--protocol=2.28.0` is accepted (it sets the version claim the client sends, which the server already requires to match exactly) and **every other value is rejected up front** with a clear error before any connection — it does not and cannot speak an older or virtual wire format. Divergence from rsync (which negotiates a range and downgrades to an integer 0..31): FastSync's honest contract is force-to-the-one-supported-value; a genuine downgrade would require a per-version compatibility layer that does not exist. Client-only; the server-side exact-match check is unchanged. `--protocol=2.27.0`/`2.26.0`/`2.25.0`/`2.24.0`/`2.23.0`/`2.22.0`/`2.21.0`/`2.20.0`/`2.19.0`/`2.18.0`/`2.17.0`/`2.16.0`/`2.15.0`/`216`/`31`/garbage are all rejected. See the Phase-6 protocol note below |
|
||||
| `--iconv=CONVERT_SPEC` | Charset conversion | ✅ Parity | Charset conversion of FILE NAMES (not content) at the protocol boundary via iconv(3): `--iconv=LOCAL[,REMOTE]` — the sender converts each local filename LOCAL→REMOTE before transmitting, matching rsync's rule that the spec "stays the same whether you're pushing or pulling": on a PUSH the destination end's charset is the spec's REMOTE half, so the default receiver writes the wire bytes verbatim, and only a server started with its own `--iconv` (the daemon `charset` analog) declares a different destination charset and converts REMOTE→that LOCAL (rsync push parity, differential-tested with and without a server `--iconv`). The full CONVERT_SPEC is serialized into the config frame as a new trailing string field so the peer knows the wire charset; **PROTOCOL_VERSION bumped 2.15.0 → 2.16.0**. `LOCAL[,REMOTE]` parse: single charset ⇒ LOCAL==REMOTE (identity both ways); garbage rejected up front; protocol 2.26.0 additionally accepts `--iconv=.` (the locale's default charset for both directions), `--iconv=-` and `--no-iconv` (disable conversion). Validation probes BOTH directions (a spec that only opens one way is refused, as is a NUL-emitting target charset like utf-16/utf-32/ucs-2, since filenames cannot contain NUL). An unrepresentable name (EILSEQ/EINVAL) fails that path cleanly with a logged `--iconv: cannot convert file name ...` and is never written mangled/truncated. Conversion is applied at EVERY wire-path site (regular/MKDIR/hardlink path+target/symlink path+target/SPECIAL, the delete manifest, the incremental-check path, and the `-s`/`chunk_serialize` embedded blob path), on both client and server (`--iconv` is also a server/daemon option). Zero overhead when unset. See the Phase-6 iconv notes below |
|
||||
| `--checksum-seed=NUM` | Set checksum seed | ✅ Parity | Sets the seed for FastSync's whole-file xxHash digest (full 64-bit seed) and for the delta path's per-block xxHash32 strong checksum (low 32 bits of the seed). **As of protocol 2.23.0 a seed of `0` — the default when the flag is unset — is randomized per transfer and the chosen seed is sent to the receiver**, exactly like rsync, so two runs against different content do not share a predictable seed; an explicit non-zero seed is used verbatim, so an explicit seed deterministically reproduces every computed digest on BOTH endpoints (the seed crosses in the config frame). `--checksum-choice=md5` has no seed and ignores it (documented). The value is a strict decimal 0..2⁶⁴-1 (blank, signed, or non-numeric values are rejected). Like rsync, a seed only matters where a digest is actually computed (`--checksum` or a basis-dir run, or a delta transfer); it does not by itself enable `--checksum`/`--delta` |
|
||||
| `--secluded-args`, `-s` | Use protocol to send args | ❌ Divergent | Accepted for CLI compatibility (including the rsync short `-s`, Phase 7 Wave A) but a documented **no-op / divergence**. rsync's `-s` protects arguments from shell expansion by shipping them over the protocol; FastSync never passes remote arguments through a shell expansion boundary in the first place — its SSH transport builds the remote argv as **single-quote-escaped shell words** (`ssh_build_remote_command`), so the injection/leak that `-s` guards against does not exist and there is nothing to "seclude". Implementing a true arg-send protocol would mean replacing the argv-based SSH launch with an in-band argument channel, a large redesign of the transport that buys no security here. Chunk serialization remains the long-only `--chunk-serialization`. |
|
||||
@@ -921,7 +923,7 @@ These are the last compatibility items and the closing phase toward rsync flag p
|
||||
|
||||
**Wire:** two trailing config-frame blocks after the `--iconv` spec, in fixed order — `send_privilege_options`/`receive_privilege_options` (one `super_mode` int, validated `0..2`), then `send_copy_as_options`/`receive_copy_as_options` (presence int + two int32 ids, validated `>= 0`, with `copy_as_set ⇒ use_metadata`). `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Divergences from rsync:** rsync's `--super` elevates the receiver and `--copy-as` actually switches its credentials; FastSync never elevates and only permits/forwards confined attempts, and `--copy-as` forces ownership rather than switching identity.
|
||||
|
||||
**Honest status after the parity-completion wave (protocol 2.27.0), updated by the rsync-parity-stats, rsync-parity-options, rsync-parity-fs, parity-review, no-wire parity-track-1/2b and wire parity-track-4a passes.** ✅ Parity 115 / ⚠️ Caveat 11 / ❌ Divergent 31 = 157 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. This reclassification makes every difference explicit. The completion wave closed 23 previously-caveated rows (9 that triage showed were already parity, plus 14 genuine fixes) and turned the 17 inherently non-rsync rows — native daemon config/auth, the FastSync batch container, the safe-subset device/privilege flags, `-X`'s privileged namespaces, `--fake-super`'s native xattr format, and the `--old-args` no-op — into explicit ❌ divergences. The stats pass flipped `--delete-delay` to ✅ (actual-removal accounting), but the parity-review pass moved it back to ⚠️ because FastSync charged the `--max-delete` budget at plan/snapshot time and left a refilled snapshotted directory in place, whereas rsync charges on actual removals and recursively removes a queued directory (including content created after its plan). The no-wire parity-track-1 pass fixed both (actual-removal charging plus recursive deferred removal with an independent deferred-list cap), narrowing the caveat to the partial-delete ordering. The stats pass also reclassified `--out-format` to ❌ (protocol-specific `%b`/delta-`%c`), and sharpened the `--stats`/`--progress`/`--checksum-choice` residuals. The options pass flipped `--bwlimit` and `--ignore-errors` to ✅ (rsync-exact size parsing and ~100 ms leaky-bucket throttling, and rsync's skip-unreadable-subdir plus IO-error-suppressed deletion with exit 23) and emits rsync-format `--info=name/flist/del/remove/nonreg/progress` lines (real-run `deleting`/`*deleting` carried over a new trailing `report_deletes` wire bool, `PROTOCOL_VERSION` 2.26.0 → 2.27.0), while reclassifying `-M` over daemon/TCP
|
||||
**Honest status after the parity-completion wave (protocol 2.27.0), updated by the rsync-parity-stats, rsync-parity-options, rsync-parity-fs, parity-review, no-wire parity-track-1/2b and wire parity-track-4a/5a passes.** ✅ Parity 116 / ⚠️ Caveat 13 / ❌ Divergent 28 = 157 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. This reclassification makes every difference explicit. The completion wave closed 23 previously-caveated rows (9 that triage showed were already parity, plus 14 genuine fixes) and turned the 17 inherently non-rsync rows — native daemon config/auth, the FastSync batch container, the safe-subset device/privilege flags, `-X`'s privileged namespaces, `--fake-super`'s native xattr format, and the `--old-args` no-op — into explicit ❌ divergences. The stats pass flipped `--delete-delay` to ✅ (actual-removal accounting), but the parity-review pass moved it back to ⚠️ because FastSync charged the `--max-delete` budget at plan/snapshot time and left a refilled snapshotted directory in place, whereas rsync charges on actual removals and recursively removes a queued directory (including content created after its plan). The no-wire parity-track-1 pass fixed both (actual-removal charging plus recursive deferred removal with an independent deferred-list cap), narrowing the caveat to the partial-delete ordering. The stats pass also reclassified `--out-format` to ❌ (protocol-specific `%b`/delta-`%c`), and sharpened the `--stats`/`--progress`/`--checksum-choice` residuals. The options pass flipped `--bwlimit` and `--ignore-errors` to ✅ (rsync-exact size parsing and ~100 ms leaky-bucket throttling, and rsync's skip-unreadable-subdir plus IO-error-suppressed deletion with exit 23) and emits rsync-format `--info=name/flist/del/remove/nonreg/progress` lines (real-run `deleting`/`*deleting` carried over a new trailing `report_deletes` wire bool, `PROTOCOL_VERSION` 2.26.0 → 2.27.0), while reclassifying `-M` over daemon/TCP
|
||||
and receiver-side `protect`/`risk` re-derivation to ❌ (no argv channel /
|
||||
receiver filter engine); the wire parity-track-4a pass later added that
|
||||
receiver filter engine, flipping `--filter=RULE` back to ✅ (see above). The fs pass flips `-d/--dirs` and `--iconv` to ✅ — recursive transfers now recreate empty source directories (and replace a blocking destination non-directory with an incoming directory); `-R --no-implied-dirs --files-from` places a listed file under a missing implied parent with default attributes instead of refusing; and `--iconv` now reproduces rsync's push direction (destination charset = the spec's REMOTE half) — and reclassified six rows to ❌ after reproducing their exact residual with differential tests: `--temp-dir` (the receiver confines the scratch dir to the receive root, so an absolute temp dir is deliberately rejected although standalone rsync follows it), the three basis-dir options (FastSync xxHash-verifies a basis hit while rsync's `--size-only` quick check installs the wrong basis content), `--delay-updates` (fixed staging name wipes an unrelated destination entry of that name), and `--dry-run` (would-delete report over-reports). `--fuzzy` was also reclassified to ❌ (deterministic heuristic with a 10× size window, not rsync's matcher), but its residual is the candidate-selection heuristic itself: the final tree is byte-exact by design, so no destination differential can expose it and the row is pinned by the `TestFuzzy` threshold suite rather than a byte-level rsync differential. The remaining ⚠️ rows are the ones with a documented residual (see the row notes and the **Parity Completion Wave (protocol 2.26.0)** section below).
|
||||
@@ -1178,8 +1180,10 @@ These remain after the wave; the individual rows carry the precise wording.
|
||||
(deliberately confined, see the row); **`--remote-option`** is SSH-only.
|
||||
**`--iconv`** now matches rsync's push direction (destination charset = the
|
||||
spec's REMOTE half; a server `--iconv` overrides it).
|
||||
- **Basis dirs** do not re-apply attributes on a match, keep the
|
||||
`--size-only` mtime caveat, and share the 256 MiB whole-file cap; **`--fuzzy`**
|
||||
- **Basis dirs** now use rsync's metadata quick-check by default (track 5a) and
|
||||
stream a hit of any size; the FastSync-only `--verify-basis` restores the
|
||||
stricter content equality. Remaining residuals: the relative-DIR resolution
|
||||
base and the over-limit basis-MISS refusal. **`--fuzzy`**
|
||||
has a different tie-break order; and **`--bwlimit`** rejects rsync's
|
||||
`0`/decimal/suffixed rates.
|
||||
- **`--inc-recursive`/`--no-inc-recursive`** are not implemented (rejected).
|
||||
@@ -1255,5 +1259,6 @@ Ranked by user demand, implementation complexity, and interoperability impact (_
|
||||
| `--tls` | TLS encryption (mutual auth) |
|
||||
| `--fastsync-server-path` | Path to fastsync-server binary |
|
||||
| `--server-host` / `--server-port` | Direct TCP connection |
|
||||
| `--verify-basis` | FastSync-only (long form, not in rsync): require a `--compare-dest`/`--copy-dest`/`--link-dest` basis hit to match the source by whole-file digest instead of trusting rsync's size+mtime (or `--size-only`) quick-check. Off by default (the default matches rsync). Crosses the wire (protocol 2.28.0) |
|
||||
| Incremental sync | Skip unchanged files (size+mtime) |
|
||||
| Delta transfer | Block-level delta for changed files |
|
||||
|
||||
@@ -1054,6 +1054,11 @@ static const OptionEntry OPTION_TABLE[] = {
|
||||
* --remote-option is parsed. --trust-sender is a local receiver policy and
|
||||
* never travels to the remote peer. */
|
||||
{"--trust-sender", NULL, OPT_FLAG, offsetof(Config, trust_sender)},
|
||||
/* FastSync-only (not an rsync option): require a basis-hit's content to
|
||||
* match the source by whole-file digest instead of trusting rsync's
|
||||
* size+mtime quick-check. Long-only; crosses the wire so the receiver
|
||||
* performs the extra read/hash. */
|
||||
{"--verify-basis", NULL, OPT_FLAG, offsetof(Config, verify_basis)},
|
||||
};
|
||||
|
||||
/* Only boolean options with no required argument are safe to negate. */
|
||||
@@ -1096,6 +1101,7 @@ static const NegatableOption NEGATABLE_OPTIONS[] = {
|
||||
{"xattrs", "X", offsetof(Config, preserve_xattrs)},
|
||||
{"acls", "A", offsetof(Config, preserve_acls)},
|
||||
{"fake-super", NULL, offsetof(Config, fake_super)},
|
||||
{"verify-basis", NULL, offsetof(Config, verify_basis)},
|
||||
};
|
||||
|
||||
static bool opt_is(const char* arg, const char* name, const char* alias) {
|
||||
|
||||
+16
-68
@@ -1047,53 +1047,6 @@ static bool files_from_list_check(const Config* config, ArrayList* missing_dest,
|
||||
return true;
|
||||
}
|
||||
|
||||
/* Basis directories are honored by the receiver's per-file incremental check,
|
||||
which (like every whole-file payload path in FastSync) is bounded by
|
||||
MAX_RECEIVE_WHOLE_FILE_SIZE. rsync would apply basis dirs to files of any
|
||||
size; FastSync cannot, so when basis dirs are requested this preflight scan
|
||||
refuses the run up front with a clear diagnostic instead of letting the
|
||||
receiver abort the whole transfer mid-stream with no client explanation.
|
||||
Returns true when the tree can be transferred. */
|
||||
static bool basis_oversize_preflight(const Config* config) {
|
||||
PreparedScanner prepared;
|
||||
if (!prepare_scanner(config, 0, &prepared))
|
||||
return false;
|
||||
DirectoryScanner* scanner =
|
||||
directory_scanner_create_with_options(config->send_directory, &prepared.options);
|
||||
if (!scanner) {
|
||||
prepared_scanner_destroy(&prepared);
|
||||
return false;
|
||||
}
|
||||
bool ok = true;
|
||||
Chunk* chunk;
|
||||
while ((chunk = directory_scanner_next(scanner)) != NULL) {
|
||||
for (int i = 0; i < chunk->element_count; i++) {
|
||||
File* f = chunk->items[i];
|
||||
if (f == NULL || f->is_dir || f->data == NULL || f->data->size <= MAX_RECEIVE_WHOLE_FILE_SIZE)
|
||||
continue;
|
||||
char* escaped = output_escape(file_wire_path(f), config->eight_bit_output);
|
||||
log_message(LOG_LEVEL_ERROR,
|
||||
"%s is %llu bytes, larger than the %llu-byte whole-file transfer limit; "
|
||||
"--compare-dest/--copy-dest/--link-dest cannot sync files above this limit",
|
||||
escaped ? escaped : "<allocation failed>", (unsigned long long)f->data->size,
|
||||
(unsigned long long)MAX_RECEIVE_WHOLE_FILE_SIZE);
|
||||
free(escaped);
|
||||
ok = false;
|
||||
break;
|
||||
}
|
||||
chunk_destroy(chunk);
|
||||
if (!ok)
|
||||
break;
|
||||
}
|
||||
if (directory_scanner_failed(scanner) || directory_scanner_had_io_error(scanner))
|
||||
ok = false;
|
||||
/* The scanner borrows prepared.options' base_filters/hardlinks pointers, so
|
||||
prepared must outlive the scanner. */
|
||||
directory_scanner_destroy(scanner);
|
||||
prepared_scanner_destroy(&prepared);
|
||||
return ok;
|
||||
}
|
||||
|
||||
/* Read the daemon's MOTD frame and, unless --no-motd, display it on stdout.
|
||||
*
|
||||
* The daemon sends the MOTD as the first thing after the config-frame STATUS_OK
|
||||
@@ -1939,11 +1892,13 @@ static int incremental_check(Client* client, File* file, const Config* config,
|
||||
return -1;
|
||||
if (!send_n_data(client->file_descriptor, &mtime_nsec, sizeof(mtime_nsec)))
|
||||
return -1;
|
||||
/* With alternate basis directories the receiver must be able to verify the
|
||||
* content of every candidate basis file, so the sender supplies its whole-file
|
||||
* digest (computed with the negotiated --checksum-choice algorithm and
|
||||
* --checksum-seed) for every file even when --checksum was not requested. */
|
||||
if (config->checksum || config_has_basis(config)) {
|
||||
/* The whole-file digest (negotiated --checksum-choice algorithm and
|
||||
* --checksum-seed) is only needed when it drives a decision: --checksum's
|
||||
* per-file quick check, or a --verify-basis content equality. Under the
|
||||
* default metadata quick-check the receiver never reads it, so the sender
|
||||
* skips the full-file read/hash exactly as rsync does for a plain
|
||||
* --link-dest run. */
|
||||
if (config->checksum || config->verify_basis) {
|
||||
uint8_t digest[CHECKSUM_MAX_DIGEST_LEN];
|
||||
size_t digest_len = 0;
|
||||
if (!file_checksum(file, (ChecksumAlgo)config->checksum_algo, config->checksum_seed, digest,
|
||||
@@ -1954,6 +1909,15 @@ static int incremental_check(Client* client, File* file, const Config* config,
|
||||
!send_n_data(client->file_descriptor, digest, wire_len))
|
||||
return -1;
|
||||
}
|
||||
/* Basis directories: the receiver materializes a hit from the basis without a
|
||||
* data frame, so it would otherwise only have the basis inode's metadata.
|
||||
* Transmit the SOURCE metadata with the check (rsync's copy-then-fix) so a
|
||||
* --copy-dest hit / --link-dest copy fallback applies the source's
|
||||
* attributes. Symmetric with incremental_check_receive_request. */
|
||||
if (config_has_basis(config) && config->use_metadata) {
|
||||
if (!metadata_send(client->file_descriptor, file->metadata))
|
||||
return -1;
|
||||
}
|
||||
Status s;
|
||||
if (!receive_status(client->file_descriptor, &s))
|
||||
return -1;
|
||||
@@ -2193,12 +2157,6 @@ static int send_dry_run_remote(Config* config) {
|
||||
}
|
||||
if (missing_args)
|
||||
array_list_delete(missing_args);
|
||||
/* Alternate basis dirs force the whole-file per-file check on the real
|
||||
receiver; refuse an oversize source up front exactly as send_files does so
|
||||
dry-run reports the same clear diagnostic instead of aborting mid-stream. */
|
||||
if (config_has_basis(config) && !basis_oversize_preflight(config))
|
||||
return 1;
|
||||
|
||||
/* A live session may follow, so arm graceful abort handling. */
|
||||
client_set_abort_armed(true);
|
||||
Client* client = connect_transfer_client(config);
|
||||
@@ -3240,11 +3198,6 @@ int send_files(Config* config) {
|
||||
array_list_delete(missing_args);
|
||||
return 1;
|
||||
}
|
||||
if (config_has_basis(config) && !basis_oversize_preflight(config)) {
|
||||
if (missing_args)
|
||||
array_list_delete(missing_args);
|
||||
return 1;
|
||||
}
|
||||
|
||||
/* From here on a server session may be live, so Ctrl-C/SIGTERM should set the
|
||||
abort flag (and be forwarded as STATUS_ABORT) instead of terminating. */
|
||||
@@ -3682,11 +3635,6 @@ int send_files_multithreaded(Config** config_ptr) {
|
||||
array_list_delete(missing_args);
|
||||
return 1;
|
||||
}
|
||||
if (config_has_basis(config) && !basis_oversize_preflight(config)) {
|
||||
if (missing_args)
|
||||
array_list_delete(missing_args);
|
||||
return 1;
|
||||
}
|
||||
|
||||
/* Armed only once a session may go live (see send_files). */
|
||||
client_set_abort_armed(true);
|
||||
|
||||
@@ -138,6 +138,9 @@ void print_usage(void) {
|
||||
printf(" into the destination instead of transferring its data\n");
|
||||
printf(" --link-dest <dir> Like --copy-dest, but hard-links the unchanged file from DIR\n");
|
||||
printf(" into the destination (repeatable; earlier DIRs win)\n");
|
||||
printf(" --verify-basis FastSync-only: require a basis hit's content to match the\n");
|
||||
printf(" source by whole-file digest instead of trusting rsync's\n");
|
||||
printf(" size+mtime (or --size-only) quick-check\n");
|
||||
printf(" --checksum-choice, --cc <alg> Whole-file checksum algorithm for --incremental/\n");
|
||||
printf(" --checksum compares. Accepted: xxh128 (default), xxh3, xxh64\n");
|
||||
printf(" (aka xxhash), md5, md4, sha1, or none. A two-name\n");
|
||||
|
||||
+13
-11
@@ -224,23 +224,31 @@ bool checksum_digest_file(ChecksumAlgo algo, uint64_t seed, const char* path, ui
|
||||
if (fd < 0)
|
||||
return false;
|
||||
|
||||
bool ok = checksum_digest_fd(algo, seed, fd, out, out_capacity, out_len);
|
||||
close(fd);
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool checksum_digest_fd(ChecksumAlgo algo, uint64_t seed, int fd, uint8_t* out, size_t out_capacity,
|
||||
size_t* out_len) {
|
||||
if (fd < 0 || !out || !out_len || out_capacity < CHECKSUM_MAX_DIGEST_LEN)
|
||||
return false;
|
||||
|
||||
if (algo == CHECKSUM_ALGO_NONE) {
|
||||
/* No checksum requested: nothing to read; an empty digest succeeds. */
|
||||
close(fd);
|
||||
*out_len = 0;
|
||||
return true;
|
||||
}
|
||||
|
||||
uint8_t buffer[64 * 1024];
|
||||
bool ok = false;
|
||||
lseek(fd, 0, SEEK_SET);
|
||||
|
||||
if (algo == CHECKSUM_ALGO_MD5 || algo == CHECKSUM_ALGO_SHA1) {
|
||||
const EVP_MD* md = algo == CHECKSUM_ALGO_MD5 ? EVP_md5() : EVP_sha1();
|
||||
EVP_MD_CTX* ctx = EVP_MD_CTX_new();
|
||||
if (!ctx) {
|
||||
close(fd);
|
||||
if (!ctx)
|
||||
return false;
|
||||
}
|
||||
unsigned int digest_len = 0;
|
||||
if (EVP_DigestInit_ex(ctx, md, NULL) == 1) {
|
||||
ok = true;
|
||||
@@ -259,7 +267,6 @@ bool checksum_digest_file(ChecksumAlgo algo, uint64_t seed, const char* path, ui
|
||||
ok = false;
|
||||
}
|
||||
EVP_MD_CTX_free(ctx);
|
||||
close(fd);
|
||||
return ok;
|
||||
}
|
||||
|
||||
@@ -276,7 +283,6 @@ bool checksum_digest_file(ChecksumAlgo algo, uint64_t seed, const char* path, ui
|
||||
md4_final(&ctx, out);
|
||||
*out_len = 16;
|
||||
}
|
||||
close(fd);
|
||||
return ok;
|
||||
}
|
||||
|
||||
@@ -286,16 +292,13 @@ bool checksum_digest_file(ChecksumAlgo algo, uint64_t seed, const char* path, ui
|
||||
XXH64_reset(&xxh64, seed);
|
||||
} else if (algo == CHECKSUM_ALGO_XXH3 || algo == CHECKSUM_ALGO_XXH128) {
|
||||
xxh3 = XXH3_createState();
|
||||
if (!xxh3) {
|
||||
close(fd);
|
||||
if (!xxh3)
|
||||
return false;
|
||||
}
|
||||
if (algo == CHECKSUM_ALGO_XXH3)
|
||||
XXH3_64bits_reset_withSeed(xxh3, seed);
|
||||
else
|
||||
XXH3_128bits_reset_withSeed(xxh3, seed);
|
||||
} else {
|
||||
close(fd);
|
||||
return false;
|
||||
}
|
||||
|
||||
@@ -329,7 +332,6 @@ bool checksum_digest_file(ChecksumAlgo algo, uint64_t seed, const char* path, ui
|
||||
}
|
||||
if (xxh3)
|
||||
XXH3_freeState(xxh3);
|
||||
close(fd);
|
||||
return ok;
|
||||
}
|
||||
|
||||
|
||||
@@ -51,6 +51,13 @@ bool checksum_digest(ChecksumAlgo algo, uint64_t seed, const void* data, size_t
|
||||
bool checksum_digest_file(ChecksumAlgo algo, uint64_t seed, const char* path, uint8_t* out,
|
||||
size_t out_capacity, size_t* out_len);
|
||||
|
||||
/* Descriptor form of the streaming digest: rewinds `fd` to the start and hashes
|
||||
* to EOF without closing it. Used by the --verify-basis path to hash an
|
||||
* already-open, root-confined basis descriptor. Same contract as
|
||||
* checksum_digest_file. */
|
||||
bool checksum_digest_fd(ChecksumAlgo algo, uint64_t seed, int fd, uint8_t* out, size_t out_capacity,
|
||||
size_t* out_len);
|
||||
|
||||
/* Resolve a --checksum-choice string (case-insensitive) to an algorithm id.
|
||||
* Accepts "xxh64"/"xxhash", "xxh3", "xxh128", "md5", "md4", "sha1", "none".
|
||||
* "auto" is not an algorithm here; the caller resolves it to the negotiated
|
||||
|
||||
+10
-1
@@ -198,9 +198,18 @@ typedef enum SuperMode { SUPER_MODE_AUTO = 0, SUPER_MODE_ON = 1, SUPER_MODE_OFF
|
||||
X(skip_compress_count, int, 0, INT_SKIPCOUNT) \
|
||||
X(skip_compress_suffixes, char**, NULL, BLOCK_SKIP_SUFFIXES)
|
||||
|
||||
/* FastSync-only --verify-basis (protocol 2.28.0, no version bump by project
|
||||
* decision): restores the stricter content equality on a basis hit. By
|
||||
* default a basis hit is accepted on rsync's metadata quick-check alone (equal
|
||||
* size plus equal mtime, or size alone under --size-only); with this flag the
|
||||
* receiver ALSO requires the basis bytes' whole-file digest (the negotiated
|
||||
* --checksum-choice algorithm) to equal the sender's, exactly FastSync's
|
||||
* historical behavior. It is a receiver policy and crosses the wire so the
|
||||
* receiver knows whether to read and hash the basis content. */
|
||||
#define CONFIG_WIRE_BASIS_FIELDS(X) \
|
||||
X(basis_count, int, 0, INT_BASISCOUNT) \
|
||||
X(basis_dirs, BasisDest*, NULL, BLOCK_BASIS)
|
||||
X(basis_dirs, BasisDest*, NULL, BLOCK_BASIS) \
|
||||
X(verify_basis, bool, false, BOOL)
|
||||
|
||||
#define CONFIG_WIRE_FUZZY_FIELDS(X) X(fuzzy, bool, false, BOOL)
|
||||
|
||||
|
||||
+209
-1
@@ -15,6 +15,7 @@
|
||||
#include <unistd.h>
|
||||
|
||||
#include "data.h"
|
||||
#include "checksum.h"
|
||||
#include "delta.h"
|
||||
#include "file.h"
|
||||
#include "file_store.h"
|
||||
@@ -24,6 +25,13 @@
|
||||
#include "utils.h"
|
||||
#include "protocol.h"
|
||||
#include "xattr.h"
|
||||
#include <fcntl.h>
|
||||
#include <unistd.h>
|
||||
|
||||
/* Files larger than this are not loaded whole for transfer (the sender streams
|
||||
* them); a whole-file digest is computed from the path instead. Kept in sync
|
||||
* with the sender's streaming threshold. */
|
||||
#define STREAM_THRESHOLD (64ULL * 1024 * 1024)
|
||||
|
||||
static bool write_all(int fd, const void* data, unsigned long long size) {
|
||||
const unsigned char* p = data;
|
||||
@@ -39,6 +47,31 @@ static bool write_all(int fd, const void* data, unsigned long long size) {
|
||||
return true;
|
||||
}
|
||||
|
||||
/* Streaming copy of an open source descriptor into the just-created destination
|
||||
`fd` (already at offset 0). Used by the --copy-dest basis install so a basis
|
||||
larger than any in-memory whole-file bound still materializes without
|
||||
buffering the entire file. `expected_size` is the caller-verified basis
|
||||
size; the copy must produce exactly that many bytes (a short source is a hard
|
||||
error, never a silently truncated destination). The final ftruncate drops
|
||||
any residual tail a raced-in longer source might have left. */
|
||||
static bool copy_fd_all(int dst_fd, int src_fd, unsigned long long expected_size) {
|
||||
unsigned char buf[1 << 20];
|
||||
unsigned long long done = 0;
|
||||
while (done < expected_size) {
|
||||
unsigned long long remaining = expected_size - done;
|
||||
size_t want = remaining < sizeof(buf) ? (size_t)remaining : sizeof(buf);
|
||||
ssize_t n = read(src_fd, buf, want);
|
||||
if (n < 0 && errno == EINTR)
|
||||
continue;
|
||||
if (n <= 0)
|
||||
return false;
|
||||
if (!write_all(dst_fd, buf, (unsigned long long)n))
|
||||
return false;
|
||||
done += (unsigned long long)n;
|
||||
}
|
||||
return ftruncate(dst_fd, (off_t)expected_size) == 0;
|
||||
}
|
||||
|
||||
/* Preallocate `size` bytes on `fd` before any data is written (--preallocate).
|
||||
* fallocate(2) reserves real disk blocks, so an out-of-space condition
|
||||
* (ENOSPC/EDQUOT) surfaces up front instead of partway through a transfer;
|
||||
@@ -140,6 +173,12 @@ bool file_checksum(File* file, ChecksumAlgo algo, uint64_t seed, uint8_t* out, s
|
||||
if (file->data->size == 0) {
|
||||
return checksum_digest(algo, seed, "", 0, out, out_capacity, out_len);
|
||||
}
|
||||
/* A streamed source (data not loaded) may exceed any in-memory whole-file
|
||||
bound; hash it from the file path in bounded buffers instead of forcing a
|
||||
full load. This is the same digest the receiver recomputes on the basis. */
|
||||
if (!file->data->data && file->path && file->data->size > STREAM_THRESHOLD &&
|
||||
checksum_digest_file(algo, seed, file->path, out, out_capacity, out_len))
|
||||
return true;
|
||||
if (!file->data->data && !file_load_data(file))
|
||||
return false;
|
||||
return checksum_digest(algo, seed, file->data->data, file->data->size, out, out_capacity,
|
||||
@@ -176,6 +215,7 @@ File* file_create(const char* path) {
|
||||
file->is_dir = false;
|
||||
file->dir_time_only = false;
|
||||
file->basis_link = NULL;
|
||||
file->basis_copy = NULL;
|
||||
file->link_group = 0;
|
||||
file->link_first = false;
|
||||
file->hardlink_target = NULL;
|
||||
@@ -205,6 +245,8 @@ void file_destroy(void* item) {
|
||||
file->send_path = NULL;
|
||||
free(file->basis_link);
|
||||
file->basis_link = NULL;
|
||||
free(file->basis_copy);
|
||||
file->basis_copy = NULL;
|
||||
free(file->hardlink_target);
|
||||
file->hardlink_target = NULL;
|
||||
free(file->symlink_target);
|
||||
@@ -1512,6 +1554,161 @@ bool file_to_disk_secure_attrs_counted(const char* path, const void* data,
|
||||
* basis). Likewise `xattrs`/`fake_super` are applied only on the copy
|
||||
* fallback, so a fallback copy preserves the per-file attributes instead of
|
||||
* silently dropping them. */
|
||||
/* Streaming --copy-dest basis install: atomically materialize `path` from the
|
||||
* bytes of `basis_path` without holding the file in memory, so a basis larger
|
||||
* than any whole-file bound still works. Mirrors the ordinary secure store
|
||||
* path (confined parent walk, temp + rename, --update/--ignore-existing/
|
||||
* --preallocate/--temp-dir) but sources the data from the basis descriptor
|
||||
* rather than a caller buffer, and applies the SOURCE metadata (rsync copies
|
||||
* then fixes attributes). A hard-link install that falls back to a byte copy
|
||||
* also routes through here when the caller supplies the basis path. */
|
||||
static bool file_copy_basis_stream_impl(const char* path, const char* basis_path,
|
||||
unsigned long long expected_size, bool preallocate,
|
||||
const FileMetadata* metadata, FileAttrPolicy policy,
|
||||
bool update, bool no_replace, bool use_fsync,
|
||||
const FileXattrList* xattrs, bool fake_super,
|
||||
const char* temp_dir, unsigned* dirs_created,
|
||||
const char* count_floor) {
|
||||
if (!path || !basis_path)
|
||||
return false;
|
||||
char* leaf = NULL;
|
||||
int dirfd = file_open_secure_parent_counted(path, &leaf, true, dirs_created, count_floor);
|
||||
if (dirfd < 0)
|
||||
return false;
|
||||
|
||||
char* basis_leaf = NULL;
|
||||
int basis_dirfd = file_open_secure_parent(basis_path, &basis_leaf, false);
|
||||
int src_fd = -1;
|
||||
if (basis_dirfd >= 0 && basis_leaf != NULL) {
|
||||
/* O_NONBLOCK rejects a raced-in FIFO without blocking; the S_ISREG gate
|
||||
below is the real type check. */
|
||||
src_fd = openat(basis_dirfd, basis_leaf, O_RDONLY | O_CLOEXEC | O_NOFOLLOW | O_NONBLOCK);
|
||||
struct stat src_st;
|
||||
if (src_fd >= 0 && (fstat(src_fd, &src_st) != 0 || !S_ISREG(src_st.st_mode))) {
|
||||
close(src_fd);
|
||||
src_fd = -1;
|
||||
}
|
||||
}
|
||||
if (basis_dirfd >= 0)
|
||||
close(basis_dirfd);
|
||||
free(basis_leaf);
|
||||
if (src_fd < 0) {
|
||||
close(dirfd);
|
||||
free(leaf);
|
||||
return false;
|
||||
}
|
||||
|
||||
struct stat destination_stat;
|
||||
bool destination_is_regular = fstatat(dirfd, leaf, &destination_stat, AT_SYMLINK_NOFOLLOW) == 0 &&
|
||||
S_ISREG(destination_stat.st_mode);
|
||||
if (update && metadata && destination_is_regular && stat_is_newer(&destination_stat, metadata)) {
|
||||
close(src_fd);
|
||||
close(dirfd);
|
||||
free(leaf);
|
||||
return true;
|
||||
}
|
||||
if (no_replace && file_path_exists_secure(path)) {
|
||||
close(src_fd);
|
||||
close(dirfd);
|
||||
free(leaf);
|
||||
return true;
|
||||
}
|
||||
|
||||
int scratch_dirfd = -1;
|
||||
if (temp_dir) {
|
||||
scratch_dirfd = file_open_temp_dir(temp_dir);
|
||||
if (scratch_dirfd < 0) {
|
||||
int saved_errno = errno;
|
||||
log_message(LOG_LEVEL_ERROR,
|
||||
"--temp-dir '%s' could not be opened (rsync requires it to already exist): %s",
|
||||
temp_dir, strerror(saved_errno));
|
||||
close(src_fd);
|
||||
close(dirfd);
|
||||
free(leaf);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
int target_dirfd = scratch_dirfd >= 0 ? scratch_dirfd : dirfd;
|
||||
int tmp_size = snprintf(NULL, 0, ".%s.tmp.%ld.%llu", leaf, (long)getpid(), ~0ULL);
|
||||
char* tmp = NULL;
|
||||
bool ok = false;
|
||||
if (tmp_size >= 0)
|
||||
tmp = malloc((size_t)tmp_size + 1);
|
||||
if (tmp) {
|
||||
for (unsigned int i = 0; i < 100 && !ok; ++i) {
|
||||
if (scratch_dirfd >= 0)
|
||||
snprintf(tmp, (size_t)tmp_size + 1, ".%s.tmp.%ld.%llu", leaf, (long)getpid(),
|
||||
next_temp_sequence());
|
||||
else
|
||||
snprintf(tmp, (size_t)tmp_size + 1, ".%s.tmp.%ld.%u", leaf, (long)getpid(), i);
|
||||
int fd =
|
||||
openat(target_dirfd, tmp, O_WRONLY | O_CREAT | O_EXCL | O_CLOEXEC | O_NOFOLLOW, 0600);
|
||||
if (fd < 0) {
|
||||
if (errno != EEXIST)
|
||||
break;
|
||||
continue;
|
||||
}
|
||||
bool wrote = true;
|
||||
if (preallocate && expected_size > 0 && preallocate_fd(fd, expected_size) != 0)
|
||||
wrote = false;
|
||||
if (wrote)
|
||||
wrote = copy_fd_all(fd, src_fd, expected_size);
|
||||
if (wrote && metadata) {
|
||||
if (!policy.perms &&
|
||||
fchmod(fd, file_mode_base(metadata, destination_is_regular,
|
||||
destination_is_regular ? destination_stat.st_mode & 0777
|
||||
: 0)) != 0)
|
||||
wrote = false;
|
||||
if (wrote)
|
||||
wrote = file_restore_metadata_fd(fd, metadata, policy);
|
||||
} else if (wrote && fchmod(fd, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH) != 0) {
|
||||
wrote = false;
|
||||
}
|
||||
if (wrote)
|
||||
restore_extra_fd(fd, metadata, xattrs, fake_super, policy);
|
||||
if (wrote && use_fsync)
|
||||
wrote = fsync(fd) == 0;
|
||||
if (close(fd) != 0)
|
||||
wrote = false;
|
||||
if (wrote && renameat(target_dirfd, tmp, dirfd, leaf) != 0)
|
||||
wrote = false;
|
||||
if (!wrote)
|
||||
unlinkat(target_dirfd, tmp, 0);
|
||||
ok = wrote;
|
||||
}
|
||||
free(tmp);
|
||||
}
|
||||
if (!ok && scratch_dirfd >= 0) {
|
||||
/* Retry once with no scratch dir (rsync's EXDEV fallback). */
|
||||
close(scratch_dirfd);
|
||||
close(src_fd);
|
||||
close(dirfd);
|
||||
free(leaf);
|
||||
return file_copy_basis_stream_impl(path, basis_path, expected_size, preallocate, metadata,
|
||||
policy, update, no_replace, use_fsync, xattrs, fake_super,
|
||||
NULL, dirs_created, count_floor);
|
||||
}
|
||||
if (scratch_dirfd >= 0)
|
||||
close(scratch_dirfd);
|
||||
close(src_fd);
|
||||
close(dirfd);
|
||||
free(leaf);
|
||||
return ok;
|
||||
}
|
||||
|
||||
/* --copy-dest basis install (streaming). Applies the source metadata and the
|
||||
per-file xattrs / --fake-super record. */
|
||||
bool file_copy_basis_stream_attrs(const char* path, const char* basis_path,
|
||||
unsigned long long expected_size, bool preallocate,
|
||||
const FileMetadata* metadata, FileAttrPolicy policy, bool update,
|
||||
bool use_fsync, const FileXattrList* xattrs, bool fake_super,
|
||||
const char* temp_dir) {
|
||||
return file_copy_basis_stream_impl(path, basis_path, expected_size, preallocate, metadata, policy,
|
||||
update, false, use_fsync, xattrs, fake_super, temp_dir, NULL,
|
||||
NULL);
|
||||
}
|
||||
|
||||
static bool file_to_disk_secure_link_impl(const char* path, const char* basis_path,
|
||||
const void* data, unsigned long long data_size,
|
||||
bool preallocate, const FileMetadata* metadata,
|
||||
@@ -1521,6 +1718,10 @@ static bool file_to_disk_secure_link_impl(const char* path, const char* basis_pa
|
||||
const char* count_floor) {
|
||||
if (!path || !basis_path)
|
||||
return false;
|
||||
/* The caller-supplied buffer is no longer used: the copy fallback streams
|
||||
from the basis path (which may hold an over-limit file). Kept in the
|
||||
signature for the existing API. */
|
||||
(void)data;
|
||||
char* leaf = NULL;
|
||||
int dirfd = file_open_secure_parent_counted(path, &leaf, true, dirs_created, count_floor);
|
||||
if (dirfd < 0)
|
||||
@@ -1604,7 +1805,14 @@ static bool file_to_disk_secure_link_impl(const char* path, const char* basis_pa
|
||||
close(dirfd);
|
||||
free(leaf);
|
||||
/* The basis file could not be linked in (missing, cross-device, refused
|
||||
by the filesystem). Write a byte-identical local copy instead. */
|
||||
by the filesystem). Stream a byte-identical local copy from the basis
|
||||
itself (never the possibly-absent caller buffer) so an over-limit basis
|
||||
still materializes. When the basis path is not a readable regular file
|
||||
(e.g. a directory raced in), fall back to the caller-supplied bytes. */
|
||||
if (file_copy_basis_stream_impl(path, basis_path, data_size, preallocate, metadata, policy,
|
||||
false, false, use_fsync, xattrs, fake_super, temp_dir,
|
||||
dirs_created, count_floor))
|
||||
return true;
|
||||
return file_to_disk_secure_attrs_counted(
|
||||
path, data, data_size, false, false, preallocate, metadata, policy, false, false, use_fsync,
|
||||
xattrs, fake_super, false, temp_dir, dirs_created, count_floor);
|
||||
|
||||
@@ -163,6 +163,16 @@ bool file_to_disk_secure_link_attrs(const char* path, const char* basis_path, co
|
||||
const FileMetadata* metadata, FileAttrPolicy policy,
|
||||
bool use_fsync, const FileXattrList* xattrs, bool fake_super,
|
||||
const char* temp_dir);
|
||||
/* Streaming --copy-dest install: atomically materialize `path` by copying the
|
||||
* bytes of `basis_path` through a bounded buffer (no whole-file buffering, so
|
||||
* an arbitrarily large basis works), applying the SOURCE metadata and the
|
||||
* per-file xattrs / --fake-super record. `update` honors a newer destination;
|
||||
* a --temp-dir scratch location falls back to a direct write on EXDEV. */
|
||||
bool file_copy_basis_stream_attrs(const char* path, const char* basis_path,
|
||||
unsigned long long expected_size, bool preallocate,
|
||||
const FileMetadata* metadata, FileAttrPolicy policy, bool update,
|
||||
bool use_fsync, const FileXattrList* xattrs, bool fake_super,
|
||||
const char* temp_dir);
|
||||
/* Protocol 2.28.0 receiver-stat variants: like the two above but additionally
|
||||
* report through `dirs_created` (when non-NULL) how many parent directories the
|
||||
* confined secure walk had to create that lie strictly below `count_floor` (a
|
||||
|
||||
+146
-98
@@ -99,6 +99,12 @@ static FileSaveResult file_stage_delayed_update(const char* root_directory,
|
||||
if (file->basis_link) {
|
||||
ok = file_to_disk_secure_link(staged_path, file->basis_link, file->data->data, file->data->size,
|
||||
config->preallocate, metadata, policy, config->use_fsync, NULL);
|
||||
} else if (file->basis_copy) {
|
||||
/* --copy-dest basis hit: stream the basis into the staging tree (bounded
|
||||
buffers, so an over-limit basis still stages). */
|
||||
ok = file_copy_basis_stream_attrs(staged_path, file->basis_copy, file->data->size,
|
||||
config->preallocate, metadata, policy, config->update,
|
||||
config->use_fsync, file->xattrs, config->fake_super, NULL);
|
||||
} else {
|
||||
ok =
|
||||
file_to_disk_secure_attrs(staged_path, file->data->data, file->data->size, false, sparse,
|
||||
@@ -652,7 +658,8 @@ FileSaveResult file_save_to_disk_full_ex(const char* root_directory, const File*
|
||||
char* destination_path = NULL;
|
||||
char *backup_path = NULL, *parent_copy = NULL;
|
||||
|
||||
if (!file || !file->path || !file->data || (file->data->size != 0 && !file->data->data) ||
|
||||
if (!file || !file->path || !file->data ||
|
||||
(file->data->size != 0 && !file->data->data && !file->basis_link && !file->basis_copy) ||
|
||||
(!file_get_trust_sender() && has_path_traversal(file->path)) ||
|
||||
(backup_enabled &&
|
||||
(!backup_suffix || backup_suffix[0] == '\0' || strchr(backup_suffix, '/') != NULL ||
|
||||
@@ -975,6 +982,13 @@ FileSaveResult file_save_to_disk_full_ex(const char* root_directory, const File*
|
||||
disk_path, file->basis_link, file->data->data, file->data->size, config->preallocate,
|
||||
metadata, policy, config->use_fsync, file->xattrs, config->fake_super, confined_temp,
|
||||
created_dirs, count_floor);
|
||||
} else if (config && file->basis_copy) {
|
||||
/* --copy-dest: stream the basis bytes through a bounded buffer so a basis
|
||||
larger than any whole-file bound still materializes. The source
|
||||
metadata was transmitted with the check frame. */
|
||||
ok = file_copy_basis_stream_attrs(
|
||||
disk_path, file->basis_copy, file->data->size, config->preallocate, metadata, policy,
|
||||
config->update, config->use_fsync, file->xattrs, config->fake_super, confined_temp);
|
||||
} else {
|
||||
/* The plain no-replace / update / with-fsync engines, plus per-file xattr
|
||||
(-X/-A) and --fake-super application on the written fd. */
|
||||
@@ -1298,16 +1312,17 @@ static File* receive_delta_file(int fd, const Config* config, const char* check_
|
||||
|
||||
/* ---- Alternate basis directories (--compare-dest / --copy-dest / --link-dest) ----
|
||||
* The receiver consults the ordered basis-dir list only when the destination
|
||||
* entry is NOT already up to date. An "exact match" requires an equal size,
|
||||
* an equal mtime (unless --size-only), and an equal content xxHash64, so a
|
||||
* hard link / local copy is only ever made from byte-identical content. */
|
||||
* entry is NOT already up to date. By default an "exact match" is rsync's
|
||||
* metadata quick-check: an equal size and an equal mtime (unless --size-only).
|
||||
* The FastSync-only --verify-basis additionally requires an equal whole-file
|
||||
* content digest, so a hard link / local copy is only then made from
|
||||
* byte-verified content. */
|
||||
|
||||
typedef struct BasisMatch {
|
||||
bool hit;
|
||||
BasisDestType type;
|
||||
char* basis_path; /* owned absolute path of the matched basis file */
|
||||
struct stat st; /* fstat() of the matched basis file */
|
||||
Data* content; /* owned basis bytes (or empty Data), NULL when not loaded */
|
||||
} BasisMatch;
|
||||
|
||||
static void basis_match_free(BasisMatch* match) {
|
||||
@@ -1315,8 +1330,6 @@ static void basis_match_free(BasisMatch* match) {
|
||||
return;
|
||||
free(match->basis_path);
|
||||
match->basis_path = NULL;
|
||||
data_destroy(match->content);
|
||||
match->content = NULL;
|
||||
match->hit = false;
|
||||
match->type = BASIS_DEST_NONE;
|
||||
}
|
||||
@@ -1348,32 +1361,10 @@ static bool basis_open_regular(const char* path, unsigned long long expected_siz
|
||||
return true;
|
||||
}
|
||||
|
||||
/* Read the whole remaining content of an open descriptor. A zero-length file
|
||||
yields an empty Data (data pointer NULL). */
|
||||
static Data* basis_read_content(int fd, unsigned long long size) {
|
||||
if (size == 0)
|
||||
return data_create_reserve(0);
|
||||
if (size > MAX_RECEIVE_WHOLE_FILE_SIZE || size > SIZE_MAX)
|
||||
return NULL;
|
||||
void* buf = protocol_alloc((size_t)size);
|
||||
if (!buf)
|
||||
return NULL;
|
||||
size_t got = 0;
|
||||
while (got < (size_t)size) {
|
||||
ssize_t n = read(fd, (char*)buf + got, (size_t)size - got);
|
||||
if (n <= 0) {
|
||||
free(buf);
|
||||
return NULL;
|
||||
}
|
||||
got += (size_t)n;
|
||||
}
|
||||
return data_create(buf, (size_t)size);
|
||||
}
|
||||
|
||||
/* --ignore-times forces every file to be updated, so no basis hit is ever
|
||||
declared (matching rsync, where -I prevents link-dest from linking). */
|
||||
static bool basis_quick_matches(const Config* config, const struct stat* st, time_t check_mtime,
|
||||
long check_mtime_nsec) {
|
||||
bool file_basis_quick_match(const Config* config, const struct stat* st, time_t check_mtime,
|
||||
long check_mtime_nsec) {
|
||||
if (config->size_only)
|
||||
return true;
|
||||
long mtime_nsec = 0;
|
||||
@@ -1384,28 +1375,39 @@ static bool basis_quick_matches(const Config* config, const struct stat* st, tim
|
||||
config->modify_window);
|
||||
}
|
||||
|
||||
/* Search the basis-dir list in command-line order and return the first exact
|
||||
match. When load_content is true the matched bytes are kept in out->content
|
||||
so the caller can materialize the file without re-reading it.
|
||||
/* True when a basis hit must be confirmed by a whole-file content digest
|
||||
(--verify-basis). False is the rsync-parity default: the metadata
|
||||
quick-check alone decides a hit. */
|
||||
bool file_basis_content_required(const Config* config) {
|
||||
return config != NULL && config->verify_basis;
|
||||
}
|
||||
|
||||
An exact match ALSO requires the basis bytes' digest to equal the source's,
|
||||
so `hash_content` gates the content read/hash itself. A server-contacting
|
||||
--dry-run passes hash_content=false: no basis file may be read or hashed
|
||||
(that would be a 1-bit content oracle against a client-supplied digest), so a
|
||||
metadata-only pass can never confirm a hit and declines it. The real path
|
||||
always passes hash_content=true, keeping its behavior byte-for-byte. */
|
||||
/* Search the basis-dir list in command-line order and return the first match.
|
||||
By default (no --verify-basis) rsync's metadata quick-check is sufficient:
|
||||
basis_open_regular has already required an equal size, and
|
||||
file_basis_quick_match applies rsync's mtime (or --size-only) rule.
|
||||
--verify-basis additionally requires the basis bytes' whole-file digest to
|
||||
equal the sender's, restoring FastSync's historical content equality; that
|
||||
digest is computed by streaming the open basis descriptor, so an arbitrarily
|
||||
large basis is verified without buffering it. A copy/link install re-reads
|
||||
the basis from its path in bounded buffers, so no content buffer is kept.
|
||||
|
||||
`hash_content` gates content READS under --verify-basis: a server-contacting
|
||||
--dry-run passes false because hashing a basis against a client-supplied
|
||||
digest would be a 1-bit content oracle. Without --verify-basis a dry-run can
|
||||
still confirm the metadata-only hit without reading any basis bytes, matching
|
||||
rsync's read-only quick-check. */
|
||||
static bool basis_match_find(const Config* config, const char* check_path,
|
||||
unsigned long long check_size, time_t check_mtime,
|
||||
long check_mtime_nsec, const uint8_t* check_digest,
|
||||
size_t check_digest_len, bool load_content, bool hash_content,
|
||||
BasisMatch* out) {
|
||||
size_t check_digest_len, bool hash_content, BasisMatch* out) {
|
||||
memset(out, 0, sizeof(*out));
|
||||
if (!config || !config_has_basis(config) || config->ignore_times)
|
||||
return false;
|
||||
/* Dry-run: never read/hash basis content. A hit cannot be decided from
|
||||
metadata alone, so report no match (the caller treats it as would-transfer)
|
||||
without touching the file's contents. */
|
||||
if (!hash_content)
|
||||
/* --verify-basis needs the basis content; a content-blind (dry-run) pass can
|
||||
never confirm it and must not read the file, so decline without touching
|
||||
the basis bytes. */
|
||||
if (file_basis_content_required(config) && !hash_content)
|
||||
return false;
|
||||
for (int i = 0; i < config->basis_count; i++) {
|
||||
const BasisDest* entry = &config->basis_dirs[i];
|
||||
@@ -1424,29 +1426,26 @@ static bool basis_match_find(const Config* config, const char* check_path,
|
||||
int fd;
|
||||
struct stat st;
|
||||
if (basis_open_regular(candidate, check_size, &fd, &st)) {
|
||||
if (basis_quick_matches(config, &st, check_mtime, check_mtime_nsec)) {
|
||||
Data* content = basis_read_content(fd, check_size);
|
||||
if (content) {
|
||||
if (file_basis_quick_match(config, &st, check_mtime, check_mtime_nsec)) {
|
||||
bool hit = true;
|
||||
if (file_basis_content_required(config)) {
|
||||
uint8_t basis_digest[CHECKSUM_MAX_DIGEST_LEN];
|
||||
size_t basis_len = 0;
|
||||
bool hashed = checksum_digest((ChecksumAlgo)config->checksum_algo, config->checksum_seed,
|
||||
content->data, content->size, basis_digest,
|
||||
sizeof(basis_digest), &basis_len);
|
||||
if (hashed && basis_len == check_digest_len && check_digest_len > 0 &&
|
||||
memcmp(basis_digest, check_digest, check_digest_len) == 0) {
|
||||
out->hit = true;
|
||||
out->type = entry->type;
|
||||
out->basis_path = candidate;
|
||||
candidate = NULL; /* ownership transferred to out */
|
||||
out->st = st;
|
||||
out->content = load_content ? content : NULL;
|
||||
if (!load_content)
|
||||
data_destroy(content);
|
||||
close(fd);
|
||||
return true;
|
||||
}
|
||||
bool hashed =
|
||||
checksum_digest_fd((ChecksumAlgo)config->checksum_algo, config->checksum_seed, fd,
|
||||
basis_digest, sizeof(basis_digest), &basis_len);
|
||||
hit = hashed && basis_len == check_digest_len && check_digest_len > 0 &&
|
||||
memcmp(basis_digest, check_digest, check_digest_len) == 0;
|
||||
}
|
||||
if (hit) {
|
||||
out->hit = true;
|
||||
out->type = entry->type;
|
||||
out->basis_path = candidate;
|
||||
candidate = NULL; /* ownership transferred to out */
|
||||
out->st = st;
|
||||
close(fd);
|
||||
return true;
|
||||
}
|
||||
data_destroy(content);
|
||||
}
|
||||
close(fd);
|
||||
}
|
||||
@@ -1871,6 +1870,12 @@ typedef struct {
|
||||
long long check_mtime_nsec;
|
||||
uint8_t check_digest[CHECKSUM_MAX_DIGEST_LEN];
|
||||
size_t check_digest_len;
|
||||
/* Source metadata carried alongside the check frame whenever a basis dir is
|
||||
configured (rsync keeps the whole file list; FastSync's sender-driven
|
||||
incremental path otherwise never transmits metadata for a SKIPPED file).
|
||||
A basis materialization applies these SOURCE attributes instead of the
|
||||
basis inode's, matching rsync's "copy then fix attributes". */
|
||||
FileMetadata* source_metadata;
|
||||
bool dest_exists; /* any destination entry exists (lstat succeeded) */
|
||||
bool has_old_file;
|
||||
int old_fd;
|
||||
@@ -1903,6 +1908,8 @@ static void incremental_check_state_cleanup(IncrementalCheckState* state) {
|
||||
if (state->old_fd >= 0)
|
||||
close(state->old_fd);
|
||||
state->old_fd = -1;
|
||||
file_metadata_destroy(state->source_metadata);
|
||||
state->source_metadata = NULL;
|
||||
free(state->full_path);
|
||||
state->full_path = NULL;
|
||||
free(state->check_path);
|
||||
@@ -1927,7 +1934,7 @@ static IncrementalCheckOutcome incremental_check_receive_request(IncrementalChec
|
||||
send_error_detail(fd, "invalid check mtime nanoseconds");
|
||||
return INCREMENTAL_ERROR;
|
||||
}
|
||||
if ((config->checksum || config_has_basis(config))) {
|
||||
if ((config->checksum || config->verify_basis)) {
|
||||
uint8_t wire_len;
|
||||
if (!receive_n_data(fd, &wire_len, sizeof(wire_len)) || wire_len == 0 ||
|
||||
wire_len > CHECKSUM_MAX_DIGEST_LEN ||
|
||||
@@ -1939,8 +1946,24 @@ static IncrementalCheckOutcome incremental_check_receive_request(IncrementalChec
|
||||
if (!receive_n_data(fd, state->check_digest, state->check_digest_len))
|
||||
return INCREMENTAL_ERROR;
|
||||
}
|
||||
/* The sender transmits the source metadata with every basis-configured check
|
||||
so a basis hit can be materialized with the SOURCE's attributes (rsync
|
||||
copies/copies-then-fixes; the receiver would otherwise only have the basis
|
||||
inode's stat). The block is symmetric and consumed unconditionally here,
|
||||
whether or not this file ends up as a basis hit. */
|
||||
if (config_has_basis(config) && config->use_metadata) {
|
||||
int meta_ok = 1;
|
||||
state->source_metadata = metadata_receive(fd, &meta_ok);
|
||||
if (!meta_ok)
|
||||
return INCREMENTAL_ERROR;
|
||||
}
|
||||
|
||||
if (state->check_size > MAX_RECEIVE_WHOLE_FILE_SIZE) {
|
||||
/* A basis-configured run may materialize a file larger than the whole-file
|
||||
payload bound: a basis hit is streamed from the basis path (bounded
|
||||
buffers), so the check size is not itself an allocation. Every other
|
||||
path (delta/append/full) still applies MAX_RECEIVE_WHOLE_FILE_SIZE, and a
|
||||
miss simply falls through to the normal transfer with its own bound. */
|
||||
if (!config_has_basis(config) && state->check_size > MAX_RECEIVE_WHOLE_FILE_SIZE) {
|
||||
send_error_detail(fd, "check size exceeds receiver limit");
|
||||
return INCREMENTAL_ERROR;
|
||||
}
|
||||
@@ -2030,6 +2053,20 @@ incremental_check_ignore_existing(const IncrementalCheckState* state) {
|
||||
return INCREMENTAL_SKIP;
|
||||
}
|
||||
|
||||
/* Metadata for a materialized basis hit: prefer the SOURCE metadata the sender
|
||||
transmitted with the check frame (rsync copies then fixes the destination to
|
||||
the source's attributes); fall back to the basis inode's own stat when
|
||||
metadata was not negotiated. Consumes state->source_metadata on success. */
|
||||
static FileMetadata* basis_take_metadata(IncrementalCheckState* state,
|
||||
const struct stat* basis_st) {
|
||||
if (state->source_metadata) {
|
||||
FileMetadata* meta = state->source_metadata;
|
||||
state->source_metadata = NULL;
|
||||
return meta;
|
||||
}
|
||||
return file_metadata_create(NULL, basis_st, false, false);
|
||||
}
|
||||
|
||||
/* --link-dest relink of an already up-to-date destination. rsync hard-links a
|
||||
destination entry to a matching basis even when the entry is already correct,
|
||||
so a run over an existing tree still maximizes sharing with the basis. Only a
|
||||
@@ -2047,7 +2084,7 @@ static IncrementalCheckOutcome incremental_check_link_dest_relink(IncrementalChe
|
||||
BasisMatch basis;
|
||||
basis_match_find(config, state->check_path, state->check_size, (time_t)state->check_mtime,
|
||||
(long)state->check_mtime_nsec, state->check_digest, state->check_digest_len,
|
||||
true, true, &basis);
|
||||
true, &basis);
|
||||
/* Only a link-dest hit relinks; a copy-dest/compare-dest hit (or a miss) lets
|
||||
the up-to-date check below keep the existing destination. */
|
||||
if (!basis.hit || basis.type != BASIS_DEST_LINK) {
|
||||
@@ -2060,11 +2097,16 @@ static IncrementalCheckOutcome incremental_check_link_dest_relink(IncrementalChe
|
||||
return INCREMENTAL_CONTINUE;
|
||||
}
|
||||
File* materialized = file_create(state->check_path);
|
||||
if (materialized && basis.content) {
|
||||
if (materialized) {
|
||||
data_destroy(materialized->data);
|
||||
materialized->data = basis.content;
|
||||
basis.content = NULL;
|
||||
materialized->metadata = file_metadata_create(NULL, &basis.st, false, false);
|
||||
materialized->data = data_create_reserve((size_t)state->check_size);
|
||||
if (!materialized->data) {
|
||||
file_destroy(materialized);
|
||||
materialized = NULL;
|
||||
}
|
||||
}
|
||||
if (materialized) {
|
||||
materialized->metadata = basis_take_metadata(state, &basis.st);
|
||||
materialized->skip = true;
|
||||
materialized->basis_link = basis.basis_path;
|
||||
basis.basis_path = NULL;
|
||||
@@ -2072,9 +2114,6 @@ static IncrementalCheckOutcome incremental_check_link_dest_relink(IncrementalChe
|
||||
file_destroy(materialized);
|
||||
materialized = NULL;
|
||||
}
|
||||
} else {
|
||||
file_destroy(materialized);
|
||||
materialized = NULL;
|
||||
}
|
||||
if (materialized) {
|
||||
if (!send_status(state->fd, STATUS_OK)) {
|
||||
@@ -2175,12 +2214,13 @@ static IncrementalCheckOutcome incremental_check_quick_skip(IncrementalCheckStat
|
||||
materialize nothing (no basis link/copy, no append/delta/full transfer) and
|
||||
the sender must send no data, so answer STATUS_DRY_RUN_TRANSFER and stop.
|
||||
|
||||
The basis lookup is deliberately content-blind: a real run would only accept
|
||||
a --compare-dest exact hit after hashing the basis file and comparing it with
|
||||
the client-supplied digest, which in a dry-run is a 1-bit content oracle.
|
||||
Under dry_run no basis bytes may be read, so an otherwise-matching entry is
|
||||
treated as would-transfer instead of a skip. Everything read here (the
|
||||
destination file's metadata, basis candidates' metadata) is read-only. */
|
||||
The basis lookup is content-blind: under the default metadata quick-check a
|
||||
hit needs no basis bytes and is honored here just as in a real run; under
|
||||
--verify-basis a real run hashes the basis against the client-supplied digest,
|
||||
which in a dry-run is a 1-bit content oracle, so no basis bytes may be read
|
||||
and an otherwise-matching entry is reported as would-transfer. Everything
|
||||
read here (the destination file's metadata, basis candidates' metadata) is
|
||||
read-only. */
|
||||
static IncrementalCheckOutcome incremental_check_dry_run_shortcut(IncrementalCheckState* state,
|
||||
bool* skipped,
|
||||
bool* would_transfer) {
|
||||
@@ -2191,12 +2231,14 @@ static IncrementalCheckOutcome incremental_check_dry_run_shortcut(IncrementalChe
|
||||
bool skip_via_compare = false;
|
||||
if (config_has_basis(config) && !config->ignore_times) {
|
||||
BasisMatch basis;
|
||||
/* hash_content=false: a dry-run must not read or hash the basis file. No
|
||||
content comparison is possible, so no compare-dest hit can be confirmed
|
||||
and an otherwise-matching file is reported as would-transfer. */
|
||||
/* hash_content=false: a dry-run must not read or hash the basis file, so
|
||||
under --verify-basis no compare-dest hit can be confirmed and an
|
||||
otherwise-matching file is reported as would-transfer. Without
|
||||
--verify-basis the metadata quick-check confirms it without touching any
|
||||
basis bytes. */
|
||||
basis_match_find(config, state->check_path, state->check_size, (time_t)state->check_mtime,
|
||||
(long)state->check_mtime_nsec, state->check_digest, state->check_digest_len,
|
||||
false, false, &basis);
|
||||
false, &basis);
|
||||
if (basis.hit && basis.type == BASIS_DEST_COMPARE && !state->has_old_file)
|
||||
skip_via_compare = true;
|
||||
basis_match_free(&basis);
|
||||
@@ -2224,7 +2266,7 @@ static IncrementalCheckOutcome incremental_check_try_basis(IncrementalCheckState
|
||||
BasisMatch basis;
|
||||
basis_match_find(config, state->check_path, state->check_size, (time_t)state->check_mtime,
|
||||
(long)state->check_mtime_nsec, state->check_digest, state->check_digest_len,
|
||||
true, true, &basis);
|
||||
true, &basis);
|
||||
if (basis.hit) {
|
||||
if (basis.type == BASIS_DEST_COMPARE) {
|
||||
basis_match_free(&basis);
|
||||
@@ -2234,24 +2276,30 @@ static IncrementalCheckOutcome incremental_check_try_basis(IncrementalCheckState
|
||||
return INCREMENTAL_SKIP;
|
||||
}
|
||||
} else {
|
||||
/* Copy/link installs source their bytes from the basis PATH at install
|
||||
time (bounded buffers), so no whole-file content buffer is needed here
|
||||
even for an over-limit basis. */
|
||||
File* materialized = file_create(state->check_path);
|
||||
if (materialized && basis.content) {
|
||||
if (materialized) {
|
||||
data_destroy(materialized->data);
|
||||
materialized->data = basis.content;
|
||||
basis.content = NULL;
|
||||
materialized->metadata = file_metadata_create(NULL, &basis.st, false, false);
|
||||
materialized->skip = true; /* receiver must not ack this as a data file */
|
||||
if (basis.type == BASIS_DEST_LINK) {
|
||||
materialized->basis_link = basis.basis_path;
|
||||
basis.basis_path = NULL;
|
||||
materialized->data = data_create_reserve((size_t)state->check_size);
|
||||
if (!materialized->data) {
|
||||
file_destroy(materialized);
|
||||
materialized = NULL;
|
||||
}
|
||||
}
|
||||
if (materialized) {
|
||||
materialized->metadata = basis_take_metadata(state, &basis.st);
|
||||
materialized->skip = true; /* receiver must not ack this as a data file */
|
||||
if (basis.type == BASIS_DEST_LINK)
|
||||
materialized->basis_link = basis.basis_path;
|
||||
else
|
||||
materialized->basis_copy = basis.basis_path;
|
||||
basis.basis_path = NULL;
|
||||
if (!materialized->metadata) {
|
||||
file_destroy(materialized);
|
||||
materialized = NULL;
|
||||
}
|
||||
} else {
|
||||
file_destroy(materialized);
|
||||
materialized = NULL;
|
||||
}
|
||||
if (materialized) {
|
||||
if (!send_status(fd, STATUS_OK)) {
|
||||
|
||||
@@ -24,6 +24,16 @@ File* file_receive_hardlink(int file_descriptor);
|
||||
File* file_receive_symlink(int file_descriptor, const Config* config);
|
||||
File* file_receive_special(int file_descriptor);
|
||||
bool file_special_rdev_valid(int32_t major, int32_t minor, mode_t mode);
|
||||
/* Testable basis quick-check / verification policy. file_basis_quick_match is
|
||||
* rsync's metadata quick-check for a basis candidate (equal size is required
|
||||
* separately by the caller; this adds the --size-only / mtime / --modify-window
|
||||
* leg). file_basis_content_required reports whether a hit must ALSO be
|
||||
* confirmed by a whole-file content digest (--verify-basis; false is the
|
||||
* default rsync-parity behavior). */
|
||||
bool file_basis_quick_match(const Config* config, const struct stat* st, time_t check_mtime,
|
||||
long check_mtime_nsec);
|
||||
bool file_basis_content_required(const Config* config);
|
||||
|
||||
File* receive_incremental_check(int fd, const Config* config, bool* skipped);
|
||||
/* Extended variant used by the receiver. `would_transfer` (may be NULL) is set
|
||||
* true only on the server-contacting --dry-run path when the file is not up to
|
||||
|
||||
@@ -57,6 +57,11 @@ typedef struct {
|
||||
* equals the incoming file, and `data` is kept as the cross-filesystem
|
||||
* fallback (a local copy) if the hard link cannot be created. */
|
||||
char* basis_link;
|
||||
/* Receiver-only, --copy-dest: when set (and basis_link is NULL), stream the
|
||||
* basis file's bytes into the destination instead of `data`/`data->size`.
|
||||
* This lets a basis larger than any whole-file bound materialize without
|
||||
* buffering it; the source metadata on `metadata` is applied afterwards. */
|
||||
char* basis_copy;
|
||||
/* --hard-links (-H), sender + receiver wire state. link_group is a run-local
|
||||
* id shared by every member of one source inode (0 = not part of a group).
|
||||
* The FIRST member (link_first == true) carries its data on the wire and is
|
||||
|
||||
@@ -28,6 +28,7 @@ from common import ( # noqa: E402
|
||||
ServerManager,
|
||||
TEST_DATA_DIR,
|
||||
clean_dir,
|
||||
get_dest_received_dir,
|
||||
)
|
||||
from parity_caveats import ASPECTS, caveat_for # noqa: E402
|
||||
import parity_harness as H # noqa: E402
|
||||
@@ -350,7 +351,9 @@ def _result_aspects(result):
|
||||
_STANDALONE_REFS = {
|
||||
"incremental_modified": "-i/--itemize-changes + incremental second run",
|
||||
"compare_dest": "--compare-dest",
|
||||
"copy_dest": "--copy-dest",
|
||||
"link_dest": "--link-dest",
|
||||
"verify_basis": "--verify-basis (FastSync-only)",
|
||||
"added_and_deleted": "--delete across two runs",
|
||||
"added_and_deleted_seed": "--delete across two runs",
|
||||
"one_file_system": "-x/--one-file-system",
|
||||
@@ -411,14 +414,17 @@ def test_compare_dest_skips_basis(parity_server_factory):
|
||||
fdst = os.path.join(TEST_DATA_DIR, "parity_cmpd_fdst")
|
||||
clean_dir(src)
|
||||
_mk(os.path.join(src, "f.txt"), b"basis-content\n")
|
||||
_pin(os.path.join(src, "f.txt"), _OLD_MTIME)
|
||||
server = parity_server_factory(SUPER)
|
||||
rel = os.path.abspath(src).lstrip(os.sep)
|
||||
|
||||
# rsync resolves --compare-dest relative to the destination dir; FastSync
|
||||
# resolves it under the receive root and appends the mirrored source path.
|
||||
# Both rely on rsync's size+mtime quick-check, so the basis mtime is pinned
|
||||
# to the source's to keep the match deterministic across a second boundary.
|
||||
def seed(_src, rroot, froot):
|
||||
_mk(os.path.join(rroot, "basis", "f.txt"), b"basis-content\n")
|
||||
_mk(os.path.join(fdst, "basis", rel, "f.txt"), b"basis-content\n")
|
||||
_mk(os.path.join(rroot, "basis", "f.txt"), b"basis-content\n", _OLD_MTIME)
|
||||
_mk(os.path.join(fdst, "basis", rel, "f.txt"), b"basis-content\n", _OLD_MTIME)
|
||||
|
||||
def extra(_src, rroot, froot, _rs, _fs):
|
||||
out = []
|
||||
@@ -446,12 +452,13 @@ def test_link_dest_hardlinks_basis(parity_server_factory):
|
||||
fdst = os.path.join(TEST_DATA_DIR, "parity_linkd_fdst")
|
||||
clean_dir(src)
|
||||
_mk(os.path.join(src, "f.txt"), b"link-basis-content\n")
|
||||
_pin(os.path.join(src, "f.txt"), _OLD_MTIME)
|
||||
server = parity_server_factory(SUPER)
|
||||
rel = os.path.abspath(src).lstrip(os.sep)
|
||||
|
||||
def seed(_src, rroot, froot):
|
||||
_mk(os.path.join(rroot, "basis", "f.txt"), b"link-basis-content\n")
|
||||
_mk(os.path.join(fdst, "basis", rel, "f.txt"), b"link-basis-content\n")
|
||||
_mk(os.path.join(rroot, "basis", "f.txt"), b"link-basis-content\n", _OLD_MTIME)
|
||||
_mk(os.path.join(fdst, "basis", rel, "f.txt"), b"link-basis-content\n", _OLD_MTIME)
|
||||
|
||||
def extra(_src, rroot, froot, _rs, _fs):
|
||||
r_basis = os.stat(os.path.join(rroot, "basis", "f.txt")).st_ino
|
||||
@@ -474,6 +481,106 @@ def test_link_dest_hardlinks_basis(parity_server_factory):
|
||||
_run_and_check(case_id, result)
|
||||
|
||||
|
||||
@requires_rsync
|
||||
@parity
|
||||
def test_copy_dest_copies_basis(parity_server_factory):
|
||||
"""--copy-dest: a basis match is materialized as an independent copy with the
|
||||
source's attributes, matching rsync (copy then fix attributes)."""
|
||||
case_id = "copy_dest"
|
||||
src = os.path.join(TEST_DATA_DIR, "parity_copyd_src")
|
||||
rdst = os.path.join(TEST_DATA_DIR, "parity_copyd_rdst")
|
||||
fdst = os.path.join(TEST_DATA_DIR, "parity_copyd_fdst")
|
||||
clean_dir(src)
|
||||
_mk(os.path.join(src, "f.txt"), b"copy-basis-content\n")
|
||||
_pin(os.path.join(src, "f.txt"), 1_600_000_000)
|
||||
os.chmod(os.path.join(src, "f.txt"), 0o755)
|
||||
server = parity_server_factory(SUPER)
|
||||
rel = os.path.abspath(src).lstrip(os.sep)
|
||||
|
||||
def seed(_src, rroot, froot):
|
||||
# Basis content matches the source; give the basis a different mode so a
|
||||
# wrong "keep basis attributes" implementation is visible.
|
||||
_mk(os.path.join(rroot, "basis", "f.txt"), b"copy-basis-content\n",
|
||||
1_600_000_000)
|
||||
os.chmod(os.path.join(rroot, "basis", "f.txt"), 0o644)
|
||||
_mk(os.path.join(fdst, "basis", rel, "f.txt"), b"copy-basis-content\n",
|
||||
1_600_000_000)
|
||||
os.chmod(os.path.join(fdst, "basis", rel, "f.txt"), 0o644)
|
||||
|
||||
def extra(_src, rroot, froot, _rs, _fs):
|
||||
out = []
|
||||
bases = {"rsync": os.path.join(rroot, "basis", "f.txt"),
|
||||
"fastsync": os.path.join(fdst, "basis", rel, "f.txt")}
|
||||
for label, root in (("rsync", rroot), ("fastsync", froot)):
|
||||
target = os.path.join(root, "f.txt")
|
||||
if not os.path.exists(target):
|
||||
out.append(f"{label}: f.txt missing")
|
||||
continue
|
||||
if os.stat(target).st_ino == os.stat(bases[label]).st_ino:
|
||||
out.append(f"{label}: f.txt is hard-linked, not copied")
|
||||
if (os.stat(target).st_mode & 0o777) != 0o755:
|
||||
out.append(f"{label}: f.txt mode "
|
||||
f"{oct(os.stat(target).st_mode & 0o777)} != 0o755")
|
||||
return out
|
||||
|
||||
result = H.run_differential(
|
||||
src, rdst, fdst,
|
||||
["-a", "--copy-dest=basis"],
|
||||
["-a", f"--copy-dest={os.path.join(fdst, 'basis')}", "--incremental"],
|
||||
server, seed=seed, ignore_paths=("basis",), extra_check=extra,
|
||||
compare_modes=True)
|
||||
_run_and_check(case_id, result)
|
||||
|
||||
|
||||
@requires_rsync
|
||||
@parity
|
||||
def test_verify_basis_restores_strict_content(parity_server_factory):
|
||||
"""Default matches rsync's metadata quick-check; FastSync-only
|
||||
`--verify-basis` restores strict content equality and transfers the source
|
||||
when a same-size/different-content basis would otherwise be trusted."""
|
||||
case_id = "verify_basis"
|
||||
src = os.path.join(TEST_DATA_DIR, "parity_vbasis_src")
|
||||
rdst = os.path.join(TEST_DATA_DIR, "parity_vbasis_rdst")
|
||||
fdst = os.path.join(TEST_DATA_DIR, "parity_vbasis_fdst")
|
||||
clean_dir(src)
|
||||
_mk(os.path.join(src, "f.txt"), b"AAAA\n")
|
||||
_pin(os.path.join(src, "f.txt"), _OLD_MTIME)
|
||||
server = parity_server_factory(SUPER)
|
||||
rel = os.path.abspath(src).lstrip(os.sep)
|
||||
|
||||
def seed(_src, rroot, froot):
|
||||
# Same size and mtime as the source, different bytes: a metadata
|
||||
# quick-check trusts it; --verify-basis must not.
|
||||
for root, basis_rel in ((rroot, os.path.join("basis", "f.txt")),
|
||||
(fdst, os.path.join("basis", rel, "f.txt"))):
|
||||
_mk(os.path.join(root, basis_rel), b"BBBB\n", _OLD_MTIME)
|
||||
|
||||
# Default: both tools trust the basis (rsync's quick check), so the
|
||||
# destination carries the basis bytes and the trees match.
|
||||
result = H.run_differential(
|
||||
src, rdst, fdst,
|
||||
["-a", "--link-dest=basis"],
|
||||
["-a", f"--link-dest={os.path.join(fdst, 'basis')}", "--incremental"],
|
||||
server, seed=seed, ignore_paths=("basis",))
|
||||
_run_and_check(case_id + "_default", result)
|
||||
|
||||
# --verify-basis (FastSync only): the digest mismatch rejects the basis and
|
||||
# the source is transferred, so the destination is the source bytes. rsync
|
||||
# has no such flag; assert the FastSync outcome directly against the source.
|
||||
fdst2 = os.path.join(TEST_DATA_DIR, "parity_vbasis_fdst2")
|
||||
clean_dir(fdst2)
|
||||
for root, basis_rel in ((fdst2, os.path.join("basis", rel, "f.txt")),):
|
||||
_mk(os.path.join(root, basis_rel), b"BBBB\n", _OLD_MTIME)
|
||||
result, _ = H.run_fastsync(src, fdst2,
|
||||
["-a", f"--link-dest={os.path.join(fdst2, 'basis')}",
|
||||
"--incremental", "--verify-basis"], server.port)
|
||||
assert result.returncode == 0, (result.stderr or result.stdout)[:300]
|
||||
target = os.path.join(get_dest_received_dir(fdst2, src), "f.txt")
|
||||
with open(target, "rb") as fh:
|
||||
assert fh.read() == b"AAAA\n", \
|
||||
"--verify-basis must reject the same-size/different-content basis"
|
||||
|
||||
|
||||
@requires_rsync
|
||||
@parity
|
||||
def test_added_and_deleted_between_runs(parity_server_factory):
|
||||
|
||||
@@ -4740,11 +4740,12 @@ class TestBasisDestDirs:
|
||||
STAGING = ".fastsync-stage"
|
||||
TS = 1577836800 # 2020-01-01 00:00:00 UTC, used to pin matching mtimes
|
||||
|
||||
# fixture files: source and basis share the mtime pin, so a basis "match"
|
||||
# is decided purely by content (xxHash). unchanged.txt is byte-identical;
|
||||
# changed.txt is byte-DIFFERENT but has the SAME SIZE as the source (and
|
||||
# the same pinned mtime), which is what forces the content-hash gate;
|
||||
# added.txt does not exist in the basis at all.
|
||||
# fixture files: source and basis share the mtime pin, so the DEFAULT
|
||||
# (rsync-parity) quick-check is a size+mtime match and trusts the basis even
|
||||
# when the body differs. unchanged.txt is byte-identical; changed.txt is
|
||||
# byte-DIFFERENT but has the SAME SIZE as the source (and the same pinned
|
||||
# mtime), which is what the FastSync-only --verify-basis content gate
|
||||
# rejects; added.txt does not exist in the basis at all.
|
||||
UNCHANGED = "unchanged.txt"
|
||||
CHANGED = "changed.txt"
|
||||
ADDED = "added.txt"
|
||||
@@ -4784,18 +4785,19 @@ class TestBasisDestDirs:
|
||||
}
|
||||
|
||||
def _basis_tree(self, prefix):
|
||||
# unchanged.txt is identical to the source; changed.txt has the SAME
|
||||
# byte size and pinned mtime but a different body (equal size forces
|
||||
# the xxHash gate); added.txt is missing from the basis.
|
||||
# unchanged.txt is identical to the source; changed.txt has a DIFFERENT
|
||||
# size (and body) so the size leg of the quick-check fails and it is
|
||||
# transferred normally; added.txt is missing from the basis.
|
||||
return {
|
||||
self.UNCHANGED: b"stable content v1\n",
|
||||
self.CHANGED: b"CHANGED CONTENT NOW\n",
|
||||
self.CHANGED: b"CHANGED CONTENT NOW AND LONGER\n",
|
||||
}
|
||||
|
||||
def test_same_size_different_content_is_not_a_basis_match(self, shared_server):
|
||||
# Core safety property: equal size + pinned mtime but different content
|
||||
# must NEVER be hard-linked or copied from the basis -- the xxHash gate
|
||||
# rejects it and the sender's data is transferred instead.
|
||||
def test_same_size_different_content_default_trusts_quick_check(self, shared_server):
|
||||
# Default rsync-parity behavior: equal size + pinned mtime is a basis
|
||||
# match, so the basis body is materialized/linked without reading it.
|
||||
# This mirrors rsync 3.4.1's quick check (differential-tested in
|
||||
# test_differential_parity.py::test_verify_basis_restores_strict_content).
|
||||
for flag, basis_dir in (("--link-dest", "szlb"), ("--copy-dest", "szcp"),
|
||||
("--compare-dest", "szcmp")):
|
||||
source = self._make_source("basis_same_size_src",
|
||||
@@ -4807,7 +4809,36 @@ class TestBasisDestDirs:
|
||||
result, _ = run_client(source, dest, flags=[f"{flag}={basis_dir}"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode == 0, \
|
||||
f"{flag} same-size mismatch failed: {result.stderr[:300]}"
|
||||
f"{flag} same-size quick-check failed: {result.stderr[:300]}"
|
||||
received = get_dest_received_dir(dest, source)
|
||||
dest_file = os.path.join(received, self.UNCHANGED)
|
||||
if flag == "--compare-dest":
|
||||
assert not os.path.exists(dest_file), \
|
||||
f"{flag}: compare-dest must leave a matching file sparse"
|
||||
else:
|
||||
assert _read_file(dest_file) == b"SAME LENGTH BODY!", \
|
||||
f"{flag}: default quick-check did not trust the basis body"
|
||||
if flag == "--link-dest":
|
||||
assert os.stat(dest_file).st_ino == os.stat(basis_file).st_ino, \
|
||||
f"{flag}: basis was not hard-linked"
|
||||
|
||||
def test_verify_basis_rejects_same_size_different_content(self, shared_server):
|
||||
# FastSync-only --verify-basis: the whole-file digest gate rejects the
|
||||
# same-size/different-content basis, so the source data is transferred
|
||||
# instead of the wrong basis bytes.
|
||||
for flag, basis_dir in (("--link-dest", "vszlb"), ("--copy-dest", "vszcp"),
|
||||
("--compare-dest", "vszcmp")):
|
||||
source = self._make_source("basis_verify_src",
|
||||
{self.UNCHANGED: b"same length body\n"})
|
||||
dest = os.path.join(TEST_DATA_DIR, f"basis_verify_dst_{basis_dir}")
|
||||
clean_dir(dest)
|
||||
basis_file = self._seed_basis_file(dest, source, basis_dir, self.UNCHANGED,
|
||||
b"SAME LENGTH BODY!")
|
||||
result, _ = run_client(source, dest,
|
||||
flags=[f"{flag}={basis_dir}", "--verify-basis"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode == 0, \
|
||||
f"{flag} --verify-basis failed: {result.stderr[:300]}"
|
||||
received = get_dest_received_dir(dest, source)
|
||||
dest_file = os.path.join(received, self.UNCHANGED)
|
||||
assert _read_file(dest_file) == b"same length body\n", \
|
||||
@@ -4839,11 +4870,13 @@ class TestBasisDestDirs:
|
||||
self._source_tree("c")[self.ADDED], "added file not transferred"
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_dry_run_compare_dest_does_not_read_basis(self, shared_server):
|
||||
# A dry-run --compare-dest must never read/hash the basis file: doing so
|
||||
# is a 1-bit content oracle against the client-supplied digest. Even a
|
||||
# byte-identical basis with a matching size+mtime is therefore reported
|
||||
# as would-transfer, and nothing is created.
|
||||
def test_dry_run_compare_dest_quick_check_does_not_read_basis(self, shared_server):
|
||||
# A dry-run --compare-dest must never read/hash the basis file. Under
|
||||
# the default metadata quick-check a matching basis is reported as a
|
||||
# skip (matching rsync) without reading it; nothing is created. Under
|
||||
# --verify-basis, which would require hashing, the dry-run cannot
|
||||
# confirm the hit (that would be a 1-bit content oracle) and reports
|
||||
# would-transfer instead.
|
||||
source = self._make_source("basis_dry_src", {self.UNCHANGED: b"stable content v1\n"})
|
||||
dest = os.path.join(TEST_DATA_DIR, "basis_dry_dst")
|
||||
clean_dir(dest)
|
||||
@@ -4854,11 +4887,30 @@ class TestBasisDestDirs:
|
||||
port=shared_server.port)
|
||||
assert result.returncode == 0, \
|
||||
f"dry-run compare-dest failed: {result.stderr[:300]}"
|
||||
assert self.UNCHANGED in result.stdout, (
|
||||
"dry-run compare-dest silently skipped: receiver read the basis content"
|
||||
assert self.UNCHANGED not in result.stdout, (
|
||||
"dry-run compare-dest did not honor the metadata quick-check "
|
||||
"(reported would-transfer for a matching basis)"
|
||||
)
|
||||
assert _snapshot_tree(dest) == before, "dry-run compare-dest mutated the destination"
|
||||
|
||||
# --verify-basis: the hit needs the basis content, which a dry-run must
|
||||
# not read, so the file is reported as would-transfer.
|
||||
dest2 = os.path.join(TEST_DATA_DIR, "basis_dry_verify_dst")
|
||||
clean_dir(dest2)
|
||||
self._seed_basis(dest2, source, "drybasis", {self.UNCHANGED: b"stable content v1\n"})
|
||||
before2 = _snapshot_tree(dest2)
|
||||
result, _ = run_client(source, dest2,
|
||||
flags=["--compare-dest=drybasis", "--dry-run",
|
||||
"--verify-basis"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode == 0, \
|
||||
f"dry-run --verify-basis compare-dest failed: {result.stderr[:300]}"
|
||||
assert self.UNCHANGED in result.stdout, (
|
||||
"dry-run --verify-basis must not read the basis to confirm a hit"
|
||||
)
|
||||
assert _snapshot_tree(dest2) == before2, \
|
||||
"dry-run --verify-basis compare-dest mutated the destination"
|
||||
|
||||
def test_compare_dest_content_mismatch_forces_transfer(self, shared_server):
|
||||
# The basis holds a file with a DIFFERENT body: even though it shares
|
||||
# the mtime pin, the xxHash check fails and the data must be sent.
|
||||
@@ -5097,27 +5149,36 @@ class TestBasisDestDirs:
|
||||
assert os.stat(dest_file).st_ino != os.stat(basis_file).st_ino, \
|
||||
"--ignore-times must not hard-link to a basis file"
|
||||
|
||||
def test_basis_refuses_file_above_whole_file_limit(self, shared_server):
|
||||
# Every whole-file payload path in FastSync (basis dirs included) is
|
||||
# bounded by MAX_RECEIVE_WHOLE_FILE_SIZE. rsync supports basis dirs for
|
||||
# arbitrary sizes; FastSync refuses such a run up front with a clear
|
||||
# diagnostic instead of letting the receiver abort the whole transfer
|
||||
# mid-stream with no client-side explanation.
|
||||
def test_basis_handles_file_above_whole_file_limit(self, shared_server):
|
||||
# Track 5a: a basis hit streams the copy (and the --verify-basis digest
|
||||
# streams the basis), so a source larger than the whole-file payload
|
||||
# bound is supported for basis dirs exactly like rsync. A basis MISS
|
||||
# still falls back to the normal transfer, which keeps its own bound.
|
||||
source = self._make_source("basis_oversize_src", {"small.txt": b"ok\n"})
|
||||
big = os.path.join(source, "huge.bin")
|
||||
with open(big, "wb") as fh:
|
||||
os.ftruncate(fh.fileno(), 256 * 1024 * 1024 + 4096)
|
||||
dest = os.path.join(TEST_DATA_DIR, "basis_oversize_dst")
|
||||
clean_dir(dest)
|
||||
result, _ = run_client(source, dest, flags=["--link-dest=nope"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode != 0, \
|
||||
"basis run with an over-limit file unexpectedly succeeded"
|
||||
assert "larger than" in result.stderr, \
|
||||
f"no clear over-limit diagnostic: {result.stderr[:300]}"
|
||||
received = get_dest_received_dir(dest, source)
|
||||
assert not os.path.exists(received), \
|
||||
"over-limit basis run transferred files before failing"
|
||||
rel = os.path.relpath(received, dest)
|
||||
basis_big = os.path.join(dest, "ob", rel, "huge.bin")
|
||||
os.makedirs(os.path.dirname(basis_big), exist_ok=True)
|
||||
shutil.copyfile(big, basis_big)
|
||||
os.utime(basis_big, (self.TS, self.TS))
|
||||
os.utime(big, (self.TS, self.TS))
|
||||
|
||||
result, _ = run_client(source, dest,
|
||||
flags=["--link-dest=ob", "--incremental"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode == 0, \
|
||||
f"over-limit basis run failed: {result.stderr[:300]}"
|
||||
dest_big = os.path.join(received, "huge.bin")
|
||||
assert os.path.exists(dest_big), "over-limit basis hit was not materialized"
|
||||
assert os.path.getsize(dest_big) == 256 * 1024 * 1024 + 4096
|
||||
assert os.stat(dest_big).st_ino == os.stat(basis_big).st_ino, \
|
||||
"over-limit --link-dest did not hard-link to the basis"
|
||||
assert _read_file(os.path.join(received, "small.txt")) == b"ok\n"
|
||||
|
||||
|
||||
def _random_payloads(size=2 * 1024 * 1024, changed=64 * 1024, seed=1234):
|
||||
|
||||
@@ -719,13 +719,18 @@ class TestVerifyAndFlip:
|
||||
source = self._src("cmpd")
|
||||
dest = self._dst("cmpd")
|
||||
rdst = self._dst("cmpd_r")
|
||||
# Pin the mtime so rsync's size+mtime quick-check (and FastSync's
|
||||
# default) matches deterministically across a second boundary.
|
||||
OLD = 1_500_000_000
|
||||
with open(os.path.join(source, "f.txt"), "wb") as fh:
|
||||
fh.write(b"basis-content\n")
|
||||
os.utime(os.path.join(source, "f.txt"), (OLD, OLD))
|
||||
# rsync resolves --compare-dest relative to the destination dir; its
|
||||
# basis file sits at the transfer-relative path.
|
||||
os.makedirs(os.path.join(rdst, "basis"), exist_ok=True)
|
||||
with open(os.path.join(rdst, "basis", "f.txt"), "wb") as fh:
|
||||
fh.write(b"basis-content\n")
|
||||
os.utime(os.path.join(rdst, "basis", "f.txt"), (OLD, OLD))
|
||||
rs = _rsync(["-a", "--compare-dest=basis", source + "/", rdst + "/"])
|
||||
assert rs.returncode == 0, rs.stderr
|
||||
assert not os.path.exists(os.path.join(rdst, "f.txt")), \
|
||||
@@ -738,6 +743,7 @@ class TestVerifyAndFlip:
|
||||
os.makedirs(basis, exist_ok=True)
|
||||
with open(os.path.join(basis, "f.txt"), "wb") as fh:
|
||||
fh.write(b"basis-content\n")
|
||||
os.utime(os.path.join(basis, "f.txt"), (OLD, OLD))
|
||||
received = get_dest_received_dir(dest, source)
|
||||
result, _ = run_client(source, dest,
|
||||
flags=["--compare-dest=basis", "--incremental"],
|
||||
@@ -751,14 +757,17 @@ class TestVerifyAndFlip:
|
||||
def test_link_dest_hardlinks_matches_rsync(self, shared_server):
|
||||
source = self._src("linkd")
|
||||
dest = self._dst("linkd")
|
||||
OLD = 1_500_000_000
|
||||
with open(os.path.join(source, "f.txt"), "wb") as fh:
|
||||
fh.write(b"link-basis-content\n")
|
||||
os.utime(os.path.join(source, "f.txt"), (OLD, OLD))
|
||||
rel = os.path.abspath(source).lstrip(os.sep)
|
||||
basis = os.path.join(dest, "basis", rel)
|
||||
os.makedirs(basis, exist_ok=True)
|
||||
basis_file = os.path.join(basis, "f.txt")
|
||||
with open(basis_file, "wb") as fh:
|
||||
fh.write(b"link-basis-content\n")
|
||||
os.utime(basis_file, (OLD, OLD))
|
||||
received = get_dest_received_dir(dest, source)
|
||||
result, _ = run_client(source, dest,
|
||||
flags=["--link-dest=basis", "--incremental"],
|
||||
@@ -771,12 +780,11 @@ class TestVerifyAndFlip:
|
||||
|
||||
@requires_rsync
|
||||
def test_basis_dir_size_only_content_residual(self, shared_server):
|
||||
"""Documented residual (RSYNC_COMPAT.md basis-dir rows): FastSync
|
||||
xxHash-verifies a basis hit, while rsync's `--size-only` quick check
|
||||
trusts the size alone. With a same-size, different-content basis,
|
||||
rsync links/copies the wrong basis content while FastSync transfers the
|
||||
source. This test pins both observed behaviors (FastSync is stricter,
|
||||
so the rows are reclassified Divergent)."""
|
||||
"""rsync parity (default): a basis hit is decided by the metadata
|
||||
quick-check alone. With `--size-only`, a same-size, different-content
|
||||
basis is trusted, so rsync links the basis content and FastSync must now
|
||||
do the same instead of xxHash-verifying it. `--verify-basis` restores
|
||||
the stricter content equality (covered by the differential test)."""
|
||||
source = self._src("basissz")
|
||||
rdest = self._dst("basissz_r")
|
||||
fdest = self._dst("basissz_f")
|
||||
@@ -806,9 +814,37 @@ class TestVerifyAndFlip:
|
||||
flags=["-a", "--size-only", "--link-dest=basis", "--incremental"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:300]
|
||||
with open(os.path.join(received, "f.txt"), "rb") as fh:
|
||||
assert fh.read() == b"BBBB\n", \
|
||||
"FastSync must trust the metadata quick-check exactly like rsync"
|
||||
|
||||
@requires_rsync
|
||||
def test_verify_basis_restores_content_check(self, shared_server):
|
||||
"""FastSync-only `--verify-basis`: a same-size, same-mtime basis with
|
||||
different content is rejected by the whole-file digest, so the source is
|
||||
transferred instead of installing the wrong basis bytes. The default
|
||||
(no flag) installs the basis content, matching rsync."""
|
||||
source = self._src("vbasis")
|
||||
fdest = self._dst("vbasis_f")
|
||||
with open(os.path.join(source, "f.txt"), "wb") as fh:
|
||||
fh.write(b"AAAA\n")
|
||||
OLD = 1_400_000_000
|
||||
os.utime(os.path.join(source, "f.txt"), (OLD, OLD))
|
||||
rel = os.path.abspath(source).lstrip(os.sep)
|
||||
basis = os.path.join(fdest, "basis", rel)
|
||||
os.makedirs(basis, exist_ok=True)
|
||||
with open(os.path.join(basis, "f.txt"), "wb") as fh:
|
||||
fh.write(b"BBBB\n")
|
||||
os.utime(os.path.join(basis, "f.txt"), (OLD, OLD))
|
||||
received = get_dest_received_dir(fdest, source)
|
||||
result, _ = run_client(source, fdest,
|
||||
flags=["-a", "--link-dest=basis", "--incremental",
|
||||
"--verify-basis"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:300]
|
||||
with open(os.path.join(received, "f.txt"), "rb") as fh:
|
||||
assert fh.read() == b"AAAA\n", \
|
||||
"FastSync must verify the basis content and transfer the source"
|
||||
"--verify-basis must reject the same-size/different-content basis"
|
||||
|
||||
|
||||
class TestIgnoreExistingShortCircuit:
|
||||
|
||||
@@ -118,8 +118,8 @@ class TestProtocol:
|
||||
shutil.rmtree(dest, ignore_errors=True)
|
||||
os.makedirs(dest)
|
||||
_seed_protocol_source(source)
|
||||
for bad in ("2.22.0", "2.21.0", "2.20.0", "2.19.0", "2.18.0", "2.17.0", "2.15.0", "2.16.0",
|
||||
"216", "31"):
|
||||
for bad in ("2.27.0", "2.26.0", "2.25.0", "2.24.0", "2.23.0", "2.22.0", "2.21.0", "2.20.0",
|
||||
"2.19.0", "2.18.0", "2.17.0", "2.15.0", "2.16.0", "216", "31"):
|
||||
result, _ = run_client(source, dest, flags=[f"--protocol={bad}"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode != 0, f"--protocol={bad} should be rejected"
|
||||
|
||||
@@ -340,6 +340,7 @@ static void test_parse_args_protocol_accept_current() {
|
||||
static void test_parse_args_protocol_rejects_other_versions() {
|
||||
static const char* const bad_versions[] = {"2.17", "2.16", "2.15.0", "2.16.0", "2.17.0",
|
||||
"2.18.0", "2.19.0", "2.20.0", "2.21.0", "2.22.0",
|
||||
"2.23.0", "2.24.0", "2.25.0", "2.26.0", "2.27.0",
|
||||
"216", "31", "abc", ""};
|
||||
for (size_t i = 0; i < sizeof(bad_versions) / sizeof(bad_versions[0]); i++) {
|
||||
Config* cfg = valid_client_config();
|
||||
@@ -1045,6 +1046,23 @@ static void test_parse_args_basis_dirs() {
|
||||
config_delete(cfg);
|
||||
}
|
||||
|
||||
/* --verify-basis (FastSync-only, long-only): default off; parses on as a plain
|
||||
boolean and leaves the basis implications intact. */
|
||||
static void test_parse_args_verify_basis() {
|
||||
Config* cfg = config_create();
|
||||
EXPECT_FALSE(cfg->verify_basis);
|
||||
config_delete(cfg);
|
||||
|
||||
cfg = config_create();
|
||||
int positional_args[2];
|
||||
int positional_count = 0;
|
||||
char* argv[] = {"fastsync", "--link-dest=prior", "--verify-basis", "/src", "/dst"};
|
||||
EXPECT_EQ_INT(parse_args(cfg, 5, argv, positional_args, &positional_count), 0);
|
||||
EXPECT_TRUE(cfg->verify_basis);
|
||||
EXPECT_TRUE(config_has_basis(cfg));
|
||||
config_delete(cfg);
|
||||
}
|
||||
|
||||
/* Escaping or degenerate basis-dir values must be rejected up front (they would
|
||||
resolve outside the destination root on the receiver); an absolute path is
|
||||
accepted (rsync parity) and canonicalized with its leading '/' preserved. */
|
||||
@@ -4831,6 +4849,7 @@ void test_client_cli() {
|
||||
test_parse_args_filter_rules();
|
||||
test_parse_args_from0_cvs_filter_file_flags();
|
||||
test_parse_args_basis_dirs();
|
||||
test_parse_args_verify_basis();
|
||||
test_parse_args_basis_invalid_paths();
|
||||
test_validate_config_basis_rejects_chunk_serialization();
|
||||
test_parse_args_delete_policy_flags();
|
||||
|
||||
+8
-4
@@ -2826,6 +2826,7 @@ static void golden_config_populate(Config* c) {
|
||||
c->skip_compress_suffixes[1] = str_dup(".xz");
|
||||
EXPECT_EQ_INT(config_basis_append(c, BASIS_DEST_COMPARE, "compare"), 0);
|
||||
EXPECT_EQ_INT(config_basis_append(c, BASIS_DEST_LINK, "link"), 0);
|
||||
c->verify_basis = true;
|
||||
c->fuzzy = true;
|
||||
c->checksum_algo = CHECKSUM_ALGO_MD5;
|
||||
c->checksum_seed = 0x1122334455667788ULL;
|
||||
@@ -2888,10 +2889,12 @@ static void golden_config_populate(Config* c) {
|
||||
* report_stats bool, 2.26.0 appended the compression_algo int, 2.27.0 appended
|
||||
* the report_deletes bool, and 2.28.0 changed only the version string and
|
||||
* appended the receiver-side delete-protection rule block (the STATUS_STATS
|
||||
* body also grew, but that is not part of this frame). The byte-exact values
|
||||
* are recomputed for the merged layout. */
|
||||
#define GOLDEN_WIRE_LEN 882
|
||||
#define GOLDEN_WIRE_HASH 10588362715396735070ULL
|
||||
* body also grew, but that is not part of this frame). Track 5a appends the
|
||||
* FastSync-only verify_basis bool to the basis block WITHOUT a version bump
|
||||
* (project decision), so the frame grew by one int to 886 bytes. The
|
||||
* byte-exact values are recomputed for the merged layout. */
|
||||
#define GOLDEN_WIRE_LEN 886
|
||||
#define GOLDEN_WIRE_HASH 5809509022716816757ULL
|
||||
|
||||
static unsigned long long fnv1a_64(const unsigned char* buf, size_t len) {
|
||||
unsigned long long h = 1469598103934665603ULL;
|
||||
@@ -3040,6 +3043,7 @@ static void test_config_wire_golden_receive() {
|
||||
recv->groupmap[0].to_name != NULL && strcmp(recv->groupmap[0].to_name, "root") == 0;
|
||||
ok = ok && recv->basis_count == 2 && recv->basis_dirs[0].type == BASIS_DEST_COMPARE &&
|
||||
recv->basis_dirs[1].type == BASIS_DEST_LINK;
|
||||
ok = ok && recv->verify_basis;
|
||||
ok = ok && recv->module != NULL && strcmp(recv->module, "goldenmod") == 0;
|
||||
ok = ok && recv->copy_as_set && recv->copy_as_uid == 111 && recv->copy_as_gid == 222;
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
#include "file_receive.h"
|
||||
#include "data.h"
|
||||
#include "config.h"
|
||||
#include "charset.h"
|
||||
#include "utils.h"
|
||||
#include "protocol.h"
|
||||
#include "test_utils.h"
|
||||
@@ -1807,6 +1808,159 @@ static void test_receive_incremental_check_empty_path() {
|
||||
config_delete(cfg);
|
||||
}
|
||||
|
||||
/* Pure policy helpers behind the basis quick-check / --verify-basis decision. */
|
||||
static void test_file_basis_quick_match_decision() {
|
||||
Config* cfg = config_create();
|
||||
EXPECT_NOT_NULL(cfg);
|
||||
EXPECT_FALSE(file_basis_content_required(cfg));
|
||||
cfg->verify_basis = true;
|
||||
EXPECT_TRUE(file_basis_content_required(cfg));
|
||||
cfg->verify_basis = false;
|
||||
|
||||
struct stat st;
|
||||
memset(&st, 0, sizeof(st));
|
||||
st.st_mtime = 1500000000;
|
||||
#ifdef __linux__
|
||||
st.st_mtim.tv_nsec = 500;
|
||||
#endif
|
||||
/* Equal size is required by the caller; this leg is the mtime / --size-only
|
||||
rule. Equal mtime matches, a different mtime misses by default. */
|
||||
EXPECT_TRUE(file_basis_quick_match(cfg, &st, 1500000000, 500));
|
||||
EXPECT_FALSE(file_basis_quick_match(cfg, &st, 1500000001, 500));
|
||||
cfg->size_only = true;
|
||||
EXPECT_TRUE(file_basis_quick_match(cfg, &st, 1500000001, 500));
|
||||
cfg->size_only = false;
|
||||
cfg->modify_window = 2;
|
||||
EXPECT_TRUE(file_basis_quick_match(cfg, &st, 1500000002, 500));
|
||||
config_delete(cfg);
|
||||
}
|
||||
|
||||
/* End-to-end handshake decision for a same-size, same-mtime, DIFFERENT-content
|
||||
basis. Default (rsync parity): the metadata quick-check is trusted, the
|
||||
receiver answers STATUS_OK and materializes the basis bytes. --verify-basis:
|
||||
the whole-file digest is required, the basis is rejected and the receiver
|
||||
asks for the source (STATUS_NEXT + full transfer). */
|
||||
static void test_receive_incremental_check_basis_quick_check_and_verify() {
|
||||
const char* root = "test_basis_quick_root";
|
||||
const char* basis_dir = "test_basis_quick_root/basis";
|
||||
const char* basis_file = "test_basis_quick_root/basis/f.txt";
|
||||
unlink(basis_file);
|
||||
unlink("test_basis_quick_root/f.txt");
|
||||
rmdir(basis_dir);
|
||||
rmdir(root);
|
||||
EXPECT_EQ_INT(mkdir(root, 0755), 0);
|
||||
EXPECT_EQ_INT(mkdir(basis_dir, 0755), 0);
|
||||
|
||||
const char* src_bytes = "AAAA";
|
||||
const unsigned long long size = 4;
|
||||
const time_t mtime = 1500000000;
|
||||
{
|
||||
FILE* fh = fopen(basis_file, "wb");
|
||||
EXPECT_NOT_NULL(fh);
|
||||
// cppcheck-suppress knownConditionTrueFalse
|
||||
if (fh) {
|
||||
EXPECT_EQ_INT((int)fwrite("BBBB", 1, (size_t)size, fh), (int)size);
|
||||
fclose(fh);
|
||||
}
|
||||
}
|
||||
struct timespec ts[2] = {{mtime, 0}, {mtime, 0}};
|
||||
EXPECT_EQ_INT(utimensat(AT_FDCWD, basis_file, ts, 0), 0);
|
||||
|
||||
char root_abs[PATH_MAX];
|
||||
EXPECT_NOT_NULL(realpath(root, root_abs));
|
||||
int root_fd = open(root_abs, O_RDONLY | O_DIRECTORY | O_CLOEXEC);
|
||||
EXPECT_TRUE(root_fd >= 0);
|
||||
// cppcheck-suppress knownConditionTrueFalse
|
||||
if (root_fd < 0) {
|
||||
unlink(basis_file);
|
||||
rmdir(basis_dir);
|
||||
rmdir(root);
|
||||
return;
|
||||
}
|
||||
EXPECT_TRUE(utils_set_authorized_root(root_fd, root_abs));
|
||||
|
||||
uint8_t digest[CHECKSUM_MAX_DIGEST_LEN];
|
||||
size_t digest_len = 0;
|
||||
EXPECT_TRUE(checksum_digest(CHECKSUM_ALGO_XXH64, 0, src_bytes, size, digest, sizeof(digest),
|
||||
&digest_len));
|
||||
|
||||
/* Route protocol I/O through the explicit descriptors (a previous test group
|
||||
may have left io_set_fds() bound to its own pipe). */
|
||||
io_set_fds(-1, -1);
|
||||
io_set_bwlimit(0);
|
||||
|
||||
for (int verify = 0; verify <= 1; verify++) {
|
||||
Config* cfg = config_create();
|
||||
EXPECT_NOT_NULL(cfg);
|
||||
cfg->receive_root_directory = str_dup(root_abs);
|
||||
cfg->checksum = false;
|
||||
cfg->checksum_algo = CHECKSUM_ALGO_XXH64;
|
||||
cfg->checksum_seed = 0;
|
||||
cfg->use_incremental = true;
|
||||
cfg->use_delta = false;
|
||||
cfg->use_metadata = false;
|
||||
cfg->verify_basis = (verify != 0);
|
||||
EXPECT_EQ_INT(config_basis_append(cfg, BASIS_DEST_LINK, "basis"), 0);
|
||||
|
||||
int p[2];
|
||||
EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0);
|
||||
EXPECT_TRUE(send_wire_str(p[1], "f.txt"));
|
||||
unsigned long long check_size = size;
|
||||
long long check_mtime = (long long)mtime;
|
||||
long long check_mtime_nsec = 0;
|
||||
EXPECT_TRUE(send_n_data(p[1], &check_size, sizeof(check_size)));
|
||||
EXPECT_TRUE(send_n_data(p[1], &check_mtime, sizeof(check_mtime)));
|
||||
EXPECT_TRUE(send_n_data(p[1], &check_mtime_nsec, sizeof(check_mtime_nsec)));
|
||||
/* Only --verify-basis needs the digest (cfg->checksum is false) and the
|
||||
pre-staged fallback full transfer the receiver will request. */
|
||||
if (verify) {
|
||||
uint8_t wire_len = (uint8_t)digest_len;
|
||||
EXPECT_TRUE(send_n_data(p[1], &wire_len, sizeof(wire_len)));
|
||||
EXPECT_TRUE(send_n_data(p[1], digest, digest_len));
|
||||
char* payload_bytes = str_dup(src_bytes);
|
||||
EXPECT_NOT_NULL(payload_bytes);
|
||||
Data* payload = data_create(payload_bytes, size);
|
||||
EXPECT_NOT_NULL(payload);
|
||||
// cppcheck-suppress knownConditionTrueFalse
|
||||
if (payload)
|
||||
EXPECT_TRUE(send_data(p[1], payload));
|
||||
data_destroy(payload);
|
||||
}
|
||||
|
||||
bool skipped = false;
|
||||
File* file = receive_incremental_check(p[0], cfg, &skipped);
|
||||
EXPECT_NOT_NULL(file);
|
||||
// cppcheck-suppress knownConditionTrueFalse
|
||||
if (file) {
|
||||
EXPECT_FALSE(skipped);
|
||||
Status reply = STATUS_ERROR;
|
||||
EXPECT_TRUE(receive_status(p[1], &reply));
|
||||
if (verify) {
|
||||
EXPECT_EQ_INT((int)reply, (int)STATUS_NEXT);
|
||||
EXPECT_NOT_NULL(file->data->data);
|
||||
EXPECT_TRUE(file->data->data != NULL && memcmp(file->data->data, src_bytes, size) == 0);
|
||||
} else {
|
||||
EXPECT_EQ_INT((int)reply, (int)STATUS_OK);
|
||||
EXPECT_TRUE(file->skip);
|
||||
/* The default quick-check hit materializes from the basis PATH at
|
||||
install time (streaming), so no content is buffered on the File. */
|
||||
EXPECT_NOT_NULL(file->basis_link);
|
||||
EXPECT_NULL(file->data->data);
|
||||
}
|
||||
file_destroy(file);
|
||||
}
|
||||
close(p[0]);
|
||||
close(p[1]);
|
||||
config_delete(cfg);
|
||||
}
|
||||
|
||||
utils_set_authorized_root(-1, NULL);
|
||||
close(root_fd);
|
||||
unlink(basis_file);
|
||||
rmdir(basis_dir);
|
||||
rmdir(root);
|
||||
}
|
||||
|
||||
/* -K/--keep-dirlinks secure open: with an authorized root, a destination path
|
||||
* component that is a symlink to an IN-ROOT directory is used as that directory
|
||||
* (its referent is opened through a relative O_NOFOLLOW walk from the root fd,
|
||||
@@ -2140,6 +2294,8 @@ void test_file() {
|
||||
test_dir_time_list();
|
||||
test_dir_time_list_cap();
|
||||
test_receive_incremental_check_empty_path();
|
||||
test_file_basis_quick_match_decision();
|
||||
test_receive_incremental_check_basis_quick_check_and_verify();
|
||||
test_keep_dirlinks_secure_open();
|
||||
test_inplace_overwrite_clears_special_mode_bits();
|
||||
test_inplace_overwrite_metadata_strips_special_bits();
|
||||
|
||||
+26
-13
@@ -1063,14 +1063,18 @@ static void test_incremental_check_fifo_destination_does_not_hang() {
|
||||
}
|
||||
|
||||
/* A server-contacting --dry-run with an alternate basis dir must never read or
|
||||
hash the basis file. An exact (size+mtime+content) basis match would
|
||||
otherwise let a client probe the basis bytes against its own supplied digest
|
||||
(a 1-bit content oracle). The dry-run decision is metadata-only, so even a
|
||||
byte-identical basis is reported as would-transfer, not a compare-dest skip. */
|
||||
static void test_incremental_check_dry_run_basis_does_not_read_content() {
|
||||
hash the basis file. Under the default metadata quick-check a hit needs no
|
||||
basis bytes, so a compare-dest match is reported as a skip (STATUS_OK) just
|
||||
like a real run -- and still no content is read. Under --verify-basis a hit
|
||||
would require hashing the basis against the client-supplied digest (a 1-bit
|
||||
content oracle), which a dry-run must never do, so even a byte-identical
|
||||
basis is reported as would-transfer. The destination is never materialized
|
||||
in either arm. */
|
||||
static void run_dry_run_basis_check(bool verify, Status expected) {
|
||||
Config* cfg = config_create();
|
||||
EXPECT_NOT_NULL(cfg);
|
||||
cfg->dry_run = true;
|
||||
cfg->verify_basis = verify;
|
||||
char* root = make_check_root("dryb");
|
||||
EXPECT_NOT_NULL(root);
|
||||
cfg->receive_root_directory = str_dup(root);
|
||||
@@ -1086,8 +1090,8 @@ static void test_incremental_check_dry_run_basis_does_not_read_content() {
|
||||
EXPECT_EQ_INT(stat(basis_path, &bst), 0);
|
||||
EXPECT_EQ_INT(config_basis_append(cfg, BASIS_DEST_COMPARE, "basis"), 0);
|
||||
|
||||
/* The (correct) source digest for the basis bytes: an unfixed dry-run would
|
||||
read+hash the basis and treat this as an exact compare-dest hit. */
|
||||
/* The (correct) source digest for the basis bytes: a buggy dry-run that read
|
||||
and hashed the basis would treat this as an exact compare-dest hit. */
|
||||
uint8_t digest[CHECKSUM_MAX_DIGEST_LEN];
|
||||
size_t digest_len = 0;
|
||||
EXPECT_TRUE(checksum_digest((ChecksumAlgo)cfg->checksum_algo, cfg->checksum_seed, content,
|
||||
@@ -1106,7 +1110,8 @@ static void test_incremental_check_dry_run_basis_does_not_read_content() {
|
||||
bool skipped = false;
|
||||
bool would_transfer = false;
|
||||
File* file = receive_incremental_check_ex(p[0], cfg, &skipped, &would_transfer);
|
||||
bool ok = file == NULL && !skipped && would_transfer;
|
||||
bool ok = file == NULL && skipped == (expected == STATUS_OK) &&
|
||||
would_transfer == (expected != STATUS_OK);
|
||||
file_destroy(file);
|
||||
config_delete(cfg);
|
||||
close(p[0]);
|
||||
@@ -1124,13 +1129,16 @@ static void test_incremental_check_dry_run_basis_does_not_read_content() {
|
||||
EXPECT_TRUE(send_n_data(p[1], &size, sizeof(size)));
|
||||
EXPECT_TRUE(send_n_data(p[1], &mtime, sizeof(mtime)));
|
||||
EXPECT_TRUE(send_n_data(p[1], &mtime_nsec, sizeof(mtime_nsec)));
|
||||
uint8_t wire_len = (uint8_t)digest_len;
|
||||
EXPECT_TRUE(send_n_data(p[1], &wire_len, sizeof(wire_len)));
|
||||
EXPECT_TRUE(send_n_data(p[1], digest, digest_len));
|
||||
/* The digest is only on the wire when --checksum or --verify-basis needs it
|
||||
(cfg->checksum is false here); the default quick-check arm sends none. */
|
||||
if (verify) {
|
||||
uint8_t wire_len = (uint8_t)digest_len;
|
||||
EXPECT_TRUE(send_n_data(p[1], &wire_len, sizeof(wire_len)));
|
||||
EXPECT_TRUE(send_n_data(p[1], digest, digest_len));
|
||||
}
|
||||
Status s;
|
||||
EXPECT_TRUE(receive_status(p[1], &s));
|
||||
/* A skip here would mean the receiver read+hashed the basis file. */
|
||||
EXPECT_EQ_INT(s, STATUS_DRY_RUN_TRANSFER);
|
||||
EXPECT_EQ_INT(s, expected);
|
||||
|
||||
int status;
|
||||
waitpid(pid, &status, 0);
|
||||
@@ -1148,6 +1156,11 @@ static void test_incremental_check_dry_run_basis_does_not_read_content() {
|
||||
}
|
||||
}
|
||||
|
||||
static void test_incremental_check_dry_run_basis_does_not_read_content() {
|
||||
run_dry_run_basis_check(false, STATUS_OK);
|
||||
run_dry_run_basis_check(true, STATUS_DRY_RUN_TRANSFER);
|
||||
}
|
||||
|
||||
/* B1: a FIFO planted in a --link-dest basis directory must not block
|
||||
* basis_open_regular() either; the basis match is simply declined. */
|
||||
static void test_incremental_check_basis_fifo_does_not_hang() {
|
||||
|
||||
+2
-1
@@ -210,7 +210,8 @@ static void test_xattr_receive_drops_acl_without_preserve_acls() {
|
||||
|
||||
/* MINOR-2: a --link-dest / -H copy fallback (linkat refused) must still apply
|
||||
* the per-file xattrs and --fake-super stat. A DIRECTORY basis forces linkat
|
||||
* to fail with EPERM, exercising the byte-copy fallback deterministically.
|
||||
* to fail with EPERM, exercising the byte-copy fallback deterministically (the
|
||||
* basis is not a regular file, so the fallback uses the caller's bytes).
|
||||
* Guarded on filesystem xattr support. */
|
||||
static void test_link_copy_fallback_preserves_xattrs() {
|
||||
const char* dest = "test_link_xattr_dest.txt";
|
||||
|
||||
Reference in New Issue
Block a user