diff --git a/CHANGELOG.md b/CHANGELOG.md index d2292e3..5afd9cd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,36 @@ run the same version because the handshake is strict. ## [Unreleased] +Wire backlog cycle (protocol 2.29.0 → 2.30.0; config-frame layout unchanged). + +- **`--stderr=client` client-message channel (#313):** the client now accepts + `--stderr=client` (and maps the deprecated `--no-msgs2stderr` to it), routing + its own diagnostics over the new bounded `STATUS_CLIENT_MSG` client->server + frame instead of writing them locally; the server writes each received + message to its stderr (respecting the server log destination). `errors`/`all` + behavior is unchanged. +- **Receiver partial failures exit 23 (#320):** a per-entry receiver failure + that does not abort the stream (e.g. an unprivileged `--devices` mknod) now + sends the terminal `STATUS_PARTIAL`; the client exits 23 like rsync and, under + `--remove-source-files`, still removes the sources it successfully + transferred. A clean run stays 0 and a fatal/connection error stays non-23. +- **Directory/symlink destination-state itemize (#314):** when `report_dest_info` + is negotiated (now also for `--progress`), the receiver answers `STATUS_MKDIR` + and `STATUS_SYMLINK` with the entry's pre-transfer destination snapshot + (existence, type, perms/owner/group/time, and whether an existing symlink's + target already matches), and the sender probes every ancestor directory before + the receiver creates it implicitly. A re-run over an unchanged tree no longer + emits per-directory `cd+++++++++` or unchanged-symlink lines, a changed + directory renders rsync's `.d..t......`, and a changed symlink renders + `cLc........` / `.L..t......`. Directory/symlink time comparison uses whole + seconds (rsync's `cmp_time`). The `STATUS_MKDIR` body gains a probe flag and + the `STATUS_DEST_INFO` record gains a symlink-target-match field; the + config-frame layout is unchanged. Differential-tested against rsync 3.4.1. +- **`--stats` deleted per-type breakdown (#316):** `STATUS_STATS` gains + `deleted_reg/dir/link/special`, tallied by the delete observers and rendered + as rsync's `Number of deleted files: X (reg: A, dir: B, link: C, special: D)`. + Differential-tested against rsync 3.4.1 for a mixed-type `--delete` tree. + ## [2.29.0] - 2026-09-23 The rsync-parity cycle 2.29 (no wire change; `PROTOCOL_VERSION` stays 2.28.0). diff --git a/CMakeLists.txt b/CMakeLists.txt index 308fa82..a75efe6 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -1,6 +1,6 @@ cmake_minimum_required(VERSION 3.22) -project(FastFileTransfer VERSION 2.29.0) +project(FastFileTransfer VERSION 2.30.0) set(CMAKE_EXPORT_COMPILE_COMMANDS ON) set(CMAKE_C_STANDARD 11) diff --git a/README.md b/README.md index 67d548c..af2240f 100644 --- a/README.md +++ b/README.md @@ -674,9 +674,9 @@ remote SSH argv is already built injection-safe. | `--outbuf=MODE` | stdout/stderr buffering: `N` (none/unbuffered), `L` (line-buffered), or `B` (block-buffered, default). | | `--log-file ` | Write log output to a file. | | `--log-file-format=FORMAT` | Per-file log-line format (requires `--log-file`). | -| `--stderr=MODE` | Route logging to stderr: `errors` or `all`. | +| `--stderr=MODE` | Route logging: `errors` (default), `all`, or `client` (forward the client's diagnostics to the server's stderr over the client-message channel). | | `--msgs2stderr` | Route all messages to stderr (deprecated spelling of `--stderr=all`). | -| `--no-msgs2stderr` | Select errors-only stderr (deprecated spelling; the default). | +| `--no-msgs2stderr` | Forward the client's diagnostics to the server (deprecated spelling of `--stderr=client`). | | `-V`, `--version` | Print the FastSync protocol version. | | `--help` | Print command usage. | @@ -832,7 +832,7 @@ before the module list, before authentication, and the connecting peer address ## Protocol and Security -FastSync protocol version `2.29.0` is shared by the client and server. The +FastSync protocol version `2.30.0` is shared by the client and server. The current protocol is sender-driven and includes configuration negotiation, including the maximum allocation limit, incremental checks, checksums, manifests, keep-alives, abort handling, per-file remove-source results, and diff --git a/RSYNC_COMPAT.md b/RSYNC_COMPAT.md index 031c4f6..135dc4d 100644 --- a/RSYNC_COMPAT.md +++ b/RSYNC_COMPAT.md @@ -110,8 +110,8 @@ Every one of those has an entry below with its remaining caveats. | `-V`, `--version` | Print version | ✅ Parity | | | `--info=FLAGS` | Fine-grained info verbosity | ⚠️ Caveat | Accepts rsync 3.4.1's full `--info` vocabulary — `backup`, `copy`, `del`, `flist`, `misc`, `mount`, `name`, `nonreg`, `progress`, `remove`, `skip`, `stats`, `symsafe`, `all`, `none` — with optional level suffixes (`--info=stats2`), so a valid rsync invocation is never rejected up front. Protocol 2.27.0 wires the categories that map to a real FastSync event, matching rsync's line format: `name` prints the updated entry names (with the ` -> target` link suffix), `flist` prints `sending incremental file list`, `del` prints `deleting PATH` (or `*deleting PATH` under `-i`/`--out-format`) for both dry-run would-delete and real deletions (real runs carry the removed paths over the new `report_deletes` wire bool), `remove` prints `sender removed PATH`, `nonreg` prints `skipping non-regular file "NAME"`, `progress` drives the per-file progress output, `copy`/`misc`/`skip` keep their existing channels, `stats` enables the same transfer-statistics block as `--stats`, and `mount` prints rsync's `[sender] skipping mount-point dir NAME` when `-xx` drops a mount-point directory (plain `-x` keeps the empty directory entry and stays silent, matching rsync; both differential-tested). `none` suppresses info output, explicit flags override `--verbose`, and a genuinely unknown name is still rejected by name (matching rsync). **Fixed (no-wire):** `--info=name2` (and higher) also prints rsync's `NAME is uptodate` lines for entries the receiver already has, and `--info=name` emits the leading transfer-root `./` name line before the first transferred entry (the marker rides in the existing `info_level` bitset; differential tests vs rsync 3.4.1). **Caveat:** the root `./` line is emitted before the first transferred name rather than keyed off rsync's root-attribute-change decision, so a pre-existing root that rsync leaves untouched can differ; the categories with no client-observable event stay accepted-but-silent — `symsafe` and `backup` (the backup happens on the receiver, which FastSync's protocol does not echo back); and `skip` maps to FastSync's sender-side skip logging rather than rsync's receiver-side "not creating new file" lines | | `--debug=FLAGS` | Fine-grained debug verbosity | ⚠️ Caveat | Protocol 2.26.0 accepts rsync 3.4.1's full `--debug` vocabulary with optional level suffixes. FastSync emits for its own channels (`io`, `proto`, `pack`, `util`, plus the aliases `hl`/`owner`) and maps the remaining categories that have a natural FastSync event onto real debug output: `flist` (per-directory scan progress), `del` (receiver-removed paths, riding the existing `report_deletes` wire bool), `hash`/`deltasum` (whole-file hashing and delta-sum generation), `recv` (receiver verdicts/signatures), `filter` (selection/exclusion decisions) and `send` (files handed to the sender). A normal run prints none of it; `--debug=help` lists the flags and a genuinely unknown name is rejected by name. **Caveat:** the output is FastSync's own timestamped debug format (it does not reproduce rsync's exact per-category lines), and the synthetic/rsync-internal categories (`acl`, `backup`, `bind`, `time`, ...) stay accepted-but-silent, so the row remains ⚠️ | -| `--stderr=MODE` | Change stderr output mode | ❌ Divergent | `errors` (default) and `all` are supported; `client` is rejected with a clear error (`--stderr=client is not supported`) because FastSync has no rsync client-message channel — the rejection itself is the documented behavior (Phase 7 Wave B decision). The modes that exist work; the missing rsync channel cannot be emulated without a wire change | -| `--msgs2stderr`, `--no-msgs2stderr` | Deprecated `--stderr` aliases | ⚠️ Caveat | `--msgs2stderr` maps to `--stderr=all` (supported, matching rsync). `--no-msgs2stderr` is rsync's spelling of `--stderr=client`, which FastSync has no client-message channel for, so it maps to the errors-only default instead of reproducing rsync's client mode. See `--stderr=MODE` | +| `--stderr=MODE` | Change stderr output mode | ⚠️ Caveat | `errors` (default) and `all` are supported and match rsync. As of protocol 2.30.0 `client` is accepted, and the client's own diagnostics are forwarded to the peer's stderr over the new bounded `STATUS_CLIENT_MSG` client-message channel (the server writes each received message to its stderr respecting the server log destination) instead of writing locally. Caveat: rsync's `client` mode is its historical single-client-process multiplexing of every process's messages (the client's errors on its own stderr, info on stdout), whereas FastSync's push-only protocol has no server->client message stream, so only the client->server direction is reproduced | +| `--msgs2stderr`, `--no-msgs2stderr` | Deprecated `--stderr` aliases | ⚠️ Caveat | `--msgs2stderr` maps to `--stderr=all` (supported, matching rsync). `--no-msgs2stderr` is rsync's spelling of `--stderr=client`; as of protocol 2.30.0 it maps to the `client` mode and forwards the client's diagnostics to the server's stderr over the `STATUS_CLIENT_MSG` channel instead of the old errors-only approximation. See `--stderr=MODE` for the one-direction caveat | | `--no-motd` | Suppress daemon MOTD | ✅ Parity | Client-only display switch (Wave C): the daemon still sends the configured `motd file` on a `host::module/path` connection; the client reads and discards the frame without showing it. Without the flag the MOTD is printed to stdout after the config/auth handshake and escaped so control bytes cannot inject terminal sequences | | `--exclude=PATTERN` | Exclude files matching pattern | ✅ Parity | Glob matching in scanner | | `--include=PATTERN` | Include files matching pattern | ✅ Parity | Glob matching in scanner | @@ -121,12 +121,12 @@ Every one of those has an entry below with its remaining caveats. | Flag | Rsync Description | FastSync Status | Notes | |------|-------------------|-----------------|-------| -| `--stats` | Give transfer stats | ⚠️ Caveat | Prints transfer statistics. Protocol 2.25.0 populates the receiver-only counters the sender cannot observe (`Matched data`, `Number of deleted files`) from the receiver's `STATUS_STATS` report; the sender tracks the scanned file list per type so `Number of files` carries rsync's `(reg: X, dir: Y, link: Z, special: W)` breakdown (directories come from the scanner's captured directory list for `-a`/`-t`/`-p`, or from a lightweight traversed-directory counter on a plain `-r` run so the `dir:` category is present there too), `Number of regular files transferred` excludes symlinks/specials and up-to-date files, `Total file size` includes symlink target lengths, and `Total transferred file size` counts only transferred files. **Protocol 2.28.0 extends `STATUS_STATS`** with receiver-observed `literal_bytes` and the four `created_*` counters: `Number of created files` now carries rsync's `(reg/dir/link/special)` breakdown (the receiver reports which destination entries it newly created, including implicitly-created parent directories below the transfer root) and `Literal data` is exact for a delta transfer (the receiver counts the literal fragments it stored, not the whole source size) — all differential-tested in the sequential and `--threads` paths against rsync 3.4.1 for fresh-create, update and delta shapes. **Remaining divergences:** rsync's per-type breakdown on `Number of deleted files` is not reproduced; and `Total bytes sent`/`received` are FastSync wire bytes framed differently from rsync's, so they are not numerically comparable | +| `--stats` | Give transfer stats | ⚠️ Caveat | Prints transfer statistics. Protocol 2.25.0 populates the receiver-only counters the sender cannot observe (`Matched data`, `Number of deleted files`) from the receiver's `STATUS_STATS` report; the sender tracks the scanned file list per type so `Number of files` carries rsync's `(reg: X, dir: Y, link: Z, special: W)` breakdown (directories come from the scanner's captured directory list for `-a`/`-t`/`-p`, or from a lightweight traversed-directory counter on a plain `-r` run so the `dir:` category is present there too), `Number of regular files transferred` excludes symlinks/specials and up-to-date files, `Total file size` includes symlink target lengths, and `Total transferred file size` counts only transferred files. **Protocol 2.28.0 extends `STATUS_STATS`** with receiver-observed `literal_bytes` and the four `created_*` counters: `Number of created files` now carries rsync's `(reg/dir/link/special)` breakdown (the receiver reports which destination entries it newly created, including implicitly-created parent directories below the transfer root) and `Literal data` is exact for a delta transfer (the receiver counts the literal fragments it stored, not the whole source size) — all differential-tested in the sequential and `--threads` paths against rsync 3.4.1 for fresh-create, update and delta shapes. **Protocol 2.30.0 appends the four `deleted_*` counters**: the delete observers classify every entry the receiver ACTUALLY removed (regular/dir/symlink/special, a strict partition of the existing scalar), so `Number of deleted files: X (reg: A, dir: B, link: C, special: D)` matches rsync exactly — differential-tested for a mixed-type `--delete` tree (`test_differential_parity.py::test_stats_deleted_breakdown_matches_rsync`). **Remaining divergence:** `Total bytes sent`/`received` are FastSync wire bytes framed differently from rsync's, so they are not numerically comparable | | `-h`, `--human-readable` | Human-readable numbers | ✅ Parity | Formats transfer byte and rate counts using rsync's **decimal** (base-1000) units, matching rsync `-h` (e.g. `1.23M`), not binary units. **A lone `-h` with no transfer arguments prints help instead** (protocol 2.26.0), matching the rsync idiom; `-h` alongside a transfer remains human-readable | -| `-i`, `--itemize-changes` | Per-file change summary | ⚠️ Caveat | Prints rsync-style itemize lines to stdout for files actually sent (also under `-j`/`--threads`). Directory and transfer-root lines are now emitted too: a run produces rsync's `./` root line and per-directory `cd+++++++++`/`.d..t......` lines, rendered by the shared itemize code. **Residual:** the root `./` line is emitted unconditionally rather than keyed off rsync's root-attribute-change decision; **every** non-root directory is rendered as created (`cd+++++++++`) because the sender never probes a directory's destination state, so a pre-existing destination directory that rsync reports as unchanged (`.d..t......`) is still itemized as created — this is not limited to re-runs; an incremental re-run additionally itemizes directories/symlinks that lack a quick-check where rsync stays silent (unchanged regular files still print nothing, matching single-`-i`); and directory attribute columns (`%M`/`%U`/`%G`) come from the source | -| `--progress` | Show progress | ⚠️ Caveat | Protocol 2.25.0 prints rsync-style per-file progress blocks (percent, transferred/total bytes, rate, elapsed, `(xfr#N, to-chk=M/T)`) fed by the receiver's `STATUS_STATS`, in both the sequential and `--threads` send paths. FastSync also prints rsync's leading `./` transfer-root line and, when progress is requested (`--progress`/`-P`/`--info=progress`) and not `--quiet`, runs a **paths-only metadata pre-scan** (no file reads, no hashing) that supplies rsync's file-list total `T` for the `to-chk` denominator and the directory names; `--delete-during`/`--delete-delay` reuse their existing keep-set pre-scan instead of walking twice, and non-progress runs are untouched. Per-directory name lines are emitted (trailing `/`), and symlink (` -> target`) and special entries are named too, so a **fresh multi-directory tree's name set and `to-chk` denominator match rsync 3.4.1** (differential test, sequential and `--threads`) and a **single-file transfer's name lines and deterministic frames remain byte-identical** to rsync. **Order parity (parity-2.29):** the sequential scanner now emits entries in rsync's sorted depth-first flist order (non-directories ascending, then directories ascending), so the interleaving and the `to-chk` numerator match rsync for the default single-threaded transfer (differential `test_parity_order.py`; `--threads` has no rsync analogue and stays unordered). **Remaining divergences:** the leading `./` root line is emitted unconditionally rather than keyed off rsync's root-attribute-change decision, and an ancestor directory line is emitted whenever a child transfers (rsync suppresses it when the directory itself is unchanged); on a re-run, entries without a quick-check (symlinks, empty directories) are still named where rsync stays silent; and the rate/ETA are wall-clock dependent | +| `-i`, `--itemize-changes` | Per-file change summary | ⚠️ Caveat | Prints rsync-style itemize lines to stdout for files actually sent (also under `-j`/`--threads`). Directory and transfer-root lines are emitted too: a run produces rsync's `./` root line and per-directory `cd+++++++++`/`.d..t......` lines, rendered by the shared itemize code. **Protocol 2.30.0 closes the directory/symlink destination-state residual (#314):** when `report_dest_info` is negotiated (now also for `--progress`) the receiver answers `STATUS_MKDIR`/`STATUS_SYMLINK` with the entry's pre-transfer snapshot — including whether an existing symlink's target already matches — and the sender probes every ancestor directory before the receiver creates it implicitly. A re-run over an unchanged tree therefore emits no per-directory `cd+++++++++` and no unchanged-symlink line (matching rsync), a changed directory renders `.d..t......` (not `cd`), and a changed symlink renders `cLc........` / `.L..t......`. The time column compares whole seconds for directories/symlinks (rsync's `cmp_time`), so a sub-second-only difference is not a spurious `t`. Differential: `test_differential_parity.py::test_itemize_rerun_dirs_symlinks_matches_rsync`. **Residual:** the root `./` line is still emitted unconditionally rather than keyed off rsync's root-attribute-change decision; a directory whose ONLY change is an attribute and which has no transferred child is itemized at the end of the run (the pending-directory flush) rather than in rsync's depth-first position; and directory attribute columns (`%M`/`%U`/`%G`) come from the source | +| `--progress` | Show progress | ⚠️ Caveat | Protocol 2.25.0 prints rsync-style per-file progress blocks (percent, transferred/total bytes, rate, elapsed, `(xfr#N, to-chk=M/T)`) fed by the receiver's `STATUS_STATS`, in both the sequential and `--threads` send paths. FastSync also prints rsync's leading `./` transfer-root line and, when progress is requested (`--progress`/`-P`/`--info=progress`) and not `--quiet`, runs a **paths-only metadata pre-scan** (no file reads, no hashing) that supplies rsync's file-list total `T` for the `to-chk` denominator and the directory names; `--delete-during`/`--delete-delay` reuse their existing keep-set pre-scan instead of walking twice, and non-progress runs are untouched. Per-directory name lines are emitted (trailing `/`), and symlink (` -> target`) and special entries are named too, so a **fresh multi-directory tree's name set and `to-chk` denominator match rsync 3.4.1** (differential test, sequential and `--threads`) and a **single-file transfer's name lines and deterministic frames remain byte-identical** to rsync. **Order parity (parity-2.29):** the sequential scanner now emits entries in rsync's sorted depth-first flist order (non-directories ascending, then directories ascending), so the interleaving and the `to-chk` numerator match rsync for the default single-threaded transfer (differential `test_parity_order.py`; `--threads` has no rsync analogue and stays unordered). **Destination-state suppression (#314, protocol 2.30.0):** a pre-existing directory is no longer named (rsync names a directory only when it creates it) and an unchanged symlink is no longer named, so a re-run over an unchanged tree prints no directory/symlink name lines where rsync stays silent. **Remaining divergences:** the leading `./` root line is emitted unconditionally rather than keyed off rsync's root-attribute-change decision; and the rate/ETA are wall-clock dependent | | `-P` | Same as --partial --progress | ✅ Parity | Parses to `--partial` + `--progress`. The independent `--partial` retention semantics are rsync parity: an interrupted write retains the already-written temp at the destination (best-effort) so a later `--append`/`--append-verify` can resume. Progress presentation is owned by the `--progress` row; there is no separate `-P` divergence | -| `--out-format=FORMAT` | Custom output format | ❌ Divergent | Per-transfer template on stdout; tokens `%f` `%n` `%l` `%b` `%c` `%C` `%i` `%M` `%o` `%U` `%G` `%t` `%%`. `%C` now uses the negotiated transfer algorithm (`--checksum-choice`, default `xxh128`, seed 0) and renders every algorithm exactly like rsync — xxh128 high-then-low, xxh64/xxh3 big-endian, md5/md4/sha1 standard hex, `none` a blank 2-char column — differential-tested across all algorithms. `%f`/`%n`/`%l`/`%i`/`%M`/`%U`/`%G`/`%B` also match. **Reclassified because `%b`/`%c` are protocol-specific and cannot match:** a differential against rsync 3.4.1 shows whole-file `%c = 16` for both, but rsync whole-file `%b = filesize + 27 + transfer-digest-bytes` (39 for a 0-byte file; 43/35/47 for xxh128/xxh64/sha1 on a 12-byte file) while FastSync `%b` counts its own framing; in delta mode rsync `%c = 16 + 6·ceil(filesize/block_size)` (verified at block sizes 512/700/1024/2048) while FastSync counts its own signature handshake, and rsync `%b` is its token stream. FastSync's wire bytes are a different quantity, so exact `%b`/delta-`%c` equality is impossible. Directory and transfer-root lines are now emitted (rsync's `./` root line and per-directory `cd...`/`.d..t...` lines), with the same residual as `-i`: the root line is emitted unconditionally, every non-root directory renders as created because the sender does not probe directory destination state (so a pre-existing unchanged directory still shows `cd+++++++++`), and directory attribute columns (`%M`/`%U`/`%G`) come from the source | +| `--out-format=FORMAT` | Custom output format | ❌ Divergent | Per-transfer template on stdout; tokens `%f` `%n` `%l` `%b` `%c` `%C` `%i` `%M` `%o` `%U` `%G` `%t` `%%`. `%C` now uses the negotiated transfer algorithm (`--checksum-choice`, default `xxh128`, seed 0) and renders every algorithm exactly like rsync — xxh128 high-then-low, xxh64/xxh3 big-endian, md5/md4/sha1 standard hex, `none` a blank 2-char column — differential-tested across all algorithms. `%f`/`%n`/`%l`/`%i`/`%M`/`%U`/`%G`/`%B` also match. **Reclassified because `%b`/`%c` are protocol-specific and cannot match:** a differential against rsync 3.4.1 shows whole-file `%c = 16` for both, but rsync whole-file `%b = filesize + 27 + transfer-digest-bytes` (39 for a 0-byte file; 43/35/47 for xxh128/xxh64/sha1 on a 12-byte file) while FastSync `%b` counts its own framing; in delta mode rsync `%c = 16 + 6·ceil(filesize/block_size)` (verified at block sizes 512/700/1024/2048) while FastSync counts its own signature handshake, and rsync `%b` is its token stream. FastSync's wire bytes are a different quantity, so exact `%b`/delta-`%c` equality is impossible. Directory and transfer-root lines are now emitted (rsync's `./` root line and per-directory `cd...`/`.d..t...` lines); protocol 2.30.0 also gives them destination state (#314), so a pre-existing unchanged directory renders `.d..t......` (or nothing) instead of `cd+++++++++` and an unchanged symlink is suppressed, with the same `-i` residuals: the root line is emitted unconditionally and a directory whose only change is an attribute (no transferred child) is itemized at the end of the run; directory attribute columns (`%M`/`%U`/`%G`) come from the source | | `--log-file=FILE` | Log to file | ✅ Parity | `log_file` config field | | `--log-file-format=FMT` | Log format | ✅ Parity | Requires `--log-file`; writes one template line per transferred file using the same token set as `--out-format` (including `%b` as the wire byte count) | | `--8-bit-output`, `-8` | Leave high-bit chars unescaped | ✅ Parity | Applies to displayed paths and protocol debug output | @@ -138,7 +138,7 @@ Every one of those has an entry below with its remaining caveats. |------|-------------------|-----------------|-------| | `--exclude-from=FILE` | Read exclude patterns from file | ✅ Parity | Reads patterns from file | | `--include-from=FILE` | Read include patterns from file | ✅ Parity | Reads patterns from file | -| `--filter=RULE` | Add file-filtering rule | ⚠️ Caveat | The short `-f` **is** bound to `--filter` (the old FastSync sendfile conflict is gone; sendfile is long-only `--sendfile`), and `-f RULE`, `-f=RULE`, `--filter=RULE` and the two-argument form all parse. Protocol 2.26.0 implements rsync's filter grammar: `+`/`-`, `include`/`exclude`, a leading `/` anchor (to the transfer root or a `.rsync-filter` file's directory), a trailing `/` dir-only rule, and the `merge`/`.`, `dir-merge`/`:`, `hide`/`H`, `show`/`S`, `protect`/`P`, `risk`/`R` and `clear`/`!` words, including the `:`/`.` modifiers. The xattr-name `x` modifier is **explicitly rejected everywhere with a clear error**. The merge-only `e`/`n`/`w` and `-` modifiers are **accepted and consumed on `merge`/`dir-merge` rules** (so they no longer leak into the merge filename) while still being **rejected on non-merge rules**, matching rsync; their semantics remain unimplemented, so they are accepted-but-ignored (the reason this row is a caveat rather than parity). A token made up solely of modifier characters that names an unsupported modifier is rejected on non-merge rules, while glued patterns (`-newfile`, `-e2e`) and mixed tokens (`H,!secret`) keep their historical parsing. First match wins; the filter layer is independent of `--exclude`/`--include`. **Track 4a (protocol 2.28.0) adds the receiver filter engine:** the sender compiles its root-level rules exactly as the scanner does (`filter_base_build`) and streams them as one bounded, self-describing config-frame block; the receiver reconstructs them and re-applies first-match-wins to every extraneous destination path during deletion, so a `P *.log` rule protects a destination-only `extra.log` (differential `filter_protect`/`filter_protect_during`/`filter_protect_delay` vs rsync 3.4.1, plus the `-n` would-delete enumeration) — matching rsync's dual-sided engine for the command-line rule set. **Remaining residual:** per-directory merge (`:`/`.`, and therefore `-F`) is not yet re-derived on the receiver; a destination-only entry that matches ONLY a per-directory merge rule is still protected only through the sender-derived source-mirror prefixes, not by the received base rule list | +| `--filter=RULE` | Add file-filtering rule | ⚠️ Caveat | The short `-f` **is** bound to `--filter` (the old FastSync sendfile conflict is gone; sendfile is long-only `--sendfile`), and `-f RULE`, `-f=RULE`, `--filter=RULE` and the two-argument form all parse. Protocol 2.26.0 implements rsync's filter grammar: `+`/`-`, `include`/`exclude`, a leading `/` anchor (to the transfer root or a `.rsync-filter` file's directory), a trailing `/` dir-only rule, and the `merge`/`.`, `dir-merge`/`:`, `hide`/`H`, `show`/`S`, `protect`/`P`, `risk`/`R` and `clear`/`!` words, including the `:`/`.` modifiers. The xattr-name `x` modifier is **explicitly rejected everywhere with a clear error**. The merge-only `e` (exclude the merge file itself), `n` (do not inherit into subdirectories), `w` (word-split the file on whitespace) and `-` (read the file as bare exclude/include patterns) modifiers are **implemented** on `merge`/`dir-merge` rules (they are still **rejected on non-merge rules**, matching rsync 3.4.1), verified by the `dir_merge_e`/`dir_merge_n`/`dir_merge_dash`/`dir_merge_w` differential cases. A token made up solely of modifier characters that names an unsupported modifier is rejected on non-merge rules, while glued patterns (`-newfile`, `-e2e`) and mixed tokens (`H,!secret`) keep their historical parsing. First match wins; the filter layer is independent of `--exclude`/`--include`. **Track 4a (protocol 2.28.0) adds the receiver filter engine:** the sender compiles its root-level rules exactly as the scanner does (`filter_base_build`) and streams them as one bounded, self-describing config-frame block; the receiver reconstructs them and re-applies first-match-wins to every extraneous destination path during deletion, so a `P *.log` rule protects a destination-only `extra.log` (differential `filter_protect`/`filter_protect_during`/`filter_protect_delay` vs rsync 3.4.1, plus the `-n` would-delete enumeration) — matching rsync's dual-sided engine for the command-line rule set. **Per-directory rules (protocol 2.30.0, issue #315) now reach the receiver:** the sender streams each traversed directory's compiled rules (owner directory + no-inherit flag, self-describing and bounded/validated) on the delete carrier, and the receiver evaluates the containing directory's rules before each ancestor's and then the command-line base (rsync's per-directory-before-ancestors order), so a destination-only entry that matches ONLY a per-directory `.rsync-filter`/dir-merge rule is shielded under every delete timing — the whole-tree commit, the `--delete-during`/`--delete-delay` per-directory plans, and the `-n` would-delete enumeration (differential `filter_perdir_protect{,_during,_delay,_after}`, `filter_perdir_exclude_{protect,deleted}`, plus the `TestFilterProtect::test_perdir_protect_*` regressions). **Narrowed residual:** rsync's receiver reads the merge file from its OWN side (the destination), whereas FastSync carries the sender's compiled source-side rules, so the two differ when the merge files differ — most visibly a destination-only `.rsync-filter` with no source counterpart is honored by rsync but not by FastSync; rsync's merge `C` (CVS-compatible) modifier is also not implemented | | `--files-from=FILE` | Read source file list from file | ✅ Parity | Entries are paths relative to the source root (leading `./` stripped, `..`/absolute rejected at parse time, blank lines ignored; NUL-delimited with `-0`). A listed regular file is transferred; a listed directory transfers its whole subtree (FastSync recursion is always on). Non-listed paths are pruned by the scanner; the delete manifest is scoped to the listed directory subtrees. A listed entry that does not exist is a hard error unless `--ignore-missing-args`/`--delete-missing-args` is given. **An empty list is a zero-transfer success (exit 0), matching rsync 3.4.1** — the earlier claim that rsync reports "no source files specified" was wrong. Scalability note: `file_list_affects` is O(list size) per scanned entry, so a very large list against a huge tree is quadratic (the documented bound) | | `-0`, `--from0` | Delimit *-from files with NULs | ✅ Parity | `--files-from` entries become NUL-delimited; the flag may appear before or after `--files-from` on the command line. NUL mode preserves entry bytes exactly (trailing CR/LF are part of the name; only newline mode trims them) | | `--max-size=SIZE` | Skip files larger than SIZE | ✅ Parity | `max_size` in scanner | @@ -150,7 +150,7 @@ Every one of those has an entry below with its remaining caveats. | `--ignore-existing` | Skip updating existing files | ✅ Parity | `ignore_existing` config field (crosses the wire; receiver-side policy). Protocol 2.26.0 short-circuits in the per-file check **before any payload**: when the destination entry already exists, the receiver answers the skip during the incremental handshake instead of letting the sender stream data that would be discarded, so an existing 4 MiB destination costs only the config/check frames (verified with a counting proxy, matching rsync). The write-time paths (regular, delay-updates-staged, hardlink-sibling, special/device) still return `FILE_SAVE_SKIPPED` without overwriting, and `--backup` is disabled for skipped files. Like rsync, it does not apply to directories/symlinks. Combines with `-j`/`--threads` and `--delay-updates` | | `--remove-source-files` | Sender removes regular files after confirmed transfer | ✅ Parity | | | `-x`, `--one-file-system` | Do not cross filesystem boundaries | ✅ Parity | Sender scanner captures the root device and does not descend into mount-point crossings (`st_dev` differs). **Protocol 2.23.0 matches rsync's entry emission:** the mount-point directory itself is emitted as a payload-less directory entry (so the destination gets an empty directory) while its contents are skipped; previously the crossing subdirectory was dropped entirely | -| `-F` | Add the default `.rsync-filter` rules | ⚠️ Caveat | Reads one filter rule per line from each directory's `.rsync-filter` file during traversal and applies it to that directory's subtree; the current directory's rules are evaluated before its ancestors', so deeper files override shallower ones and per-directory files override the command-line `--filter`/`-C` base by default (first match wins). **A single `-F` transfers the `.rsync-filter` files themselves, matching rsync; a repeated `-FF` additionally excludes them** (rsync 3.4.1's `-F`/`-FF` are exactly these two rules, with no `.cvsignore` branch). Unsupported/unparseable rules inside a per-directory file fail the scan with a clear error. **Residual (track 4a):** per-directory rules are still enforced receiver-side only through the sender-derived source-mirror protected prefixes; the base-rule receiver filter engine does not carry per-directory rules, so a destination-only entry matching ONLY a `.rsync-filter` rule is not yet shielded from `--delete` | +| `-F` | Add the default `.rsync-filter` rules | ⚠️ Caveat | Reads one filter rule per line from each directory's `.rsync-filter` file during traversal and applies it to that directory's subtree; the current directory's rules are evaluated before its ancestors', so deeper files override shallower ones and per-directory files override the command-line `--filter`/`-C` base by default (first match wins). **A single `-F` transfers the `.rsync-filter` files themselves, matching rsync; a repeated `-FF` additionally excludes them** (rsync 3.4.1's `-F`/`-FF` are exactly these two rules, with no `.cvsignore` branch). Unsupported/unparseable rules inside a per-directory file fail the scan with a clear error. The merge-only `e`/`n`/`w`/`-` modifiers on a `dir-merge` registration are implemented (`e` excludes the merge file, `n` stops inheritance into subdirectories, `w` word-splits, `-` reads bare patterns). **Per-directory rules (protocol 2.30.0, issue #315) are now carried to the receiver:** the sender streams each traversed directory's compiled rules (owner + no-inherit flag, bounded and validated) on the delete carrier and the receiver applies them deepest-directory-first before the command-line base, so a destination-only entry matching ONLY a `.rsync-filter` rule is shielded from `--delete` under every timing, including the `-n` would-delete enumeration (`filter_perdir_protect*` differential and `TestFilterProtect::test_perdir_protect_*`). **Residual:** rsync's receiver scans the DESTINATION's `.rsync-filter` files while FastSync carries the source's compiled rules, so a rule present only in a destination-side `.rsync-filter` is honored by rsync but not by FastSync | ## 4. Directory Options @@ -342,7 +342,7 @@ why plain `--append` works on the normal atomic path, not only with `--inplace`. | `-X`, `--xattrs` | Preserve extended attributes | ❌ Divergent | Deliberately restricted to unprivileged `user.*` extended attributes plus the two POSIX ACL xattrs; `security.*` (SELinux, capabilities, ...) and `trusted.*` are **never** captured or applied — a client can never force a privileged attribute onto the destination, and the receiver independently re-validates every incoming name against the whitelist. This is a security-policy divergence from rsync, which can preserve the privileged namespaces with the needed privilege; implementing them would defeat FastSync's privilege-escalation guard. `user.*` capture/apply matches rsync in a differential test. Payloads are bounded on both ends. Incompatible with `-s`. **Symlink xattrs are now carried (protocol 2.29.0):** a symlink entry appends the same bounded trailing xattr block to its `STATUS_SYMLINK` frame as every other entry kind, captured with `llistxattr`/`lgetxattr` so the link's OWN attributes are read and never the referent's, and re-applied no-follow with `lsetxattr` through the already-confined parent directory (`fsetxattr` cannot target a symlink: there is no `*at` xattr syscall and an `O_PATH` fd is rejected). On Linux the VFS refuses to associate xattrs with a symlink at all — every `lsetxattr` on a link fails with `EPERM` for `user.*`, `trusted.*` and `security.*`, even as root, verified in the CI container — so on FastSync's supported platforms the captured block is always empty and the apply is a no-op; the wire block is present for correctness and for a filesystem/platform that does support symlink xattrs. rsync 3.4.1's `--fake-super` is not a counterexample: it stores a symlink as a regular file whose `user.rsync.%stat` records the `S_IFLNK` mode bits, not an xattr on a real symlink. The row stays divergent only for the never-preserved privileged namespaces above | | `-H`, `--hard-links` | Preserve hard links | ✅ Parity | Files on the source that share an inode (`st_dev`+`st_ino`, e.g. a `cp -al` tree) are re-created as hard links to one another on the destination, so duplicate links stay deduplicated and only the first member's data is sent (later members are transmitted as payload-less `STATUS_HARDLINK` frames). The receiver links each sibling to the first member's installed file with an atomic link + rename; on `link()` failure it falls back to a byte-identical local copy of the first member, never a partial/corrupt file. Requires the sequential scan for ordering (the first member is always emitted and installed before any sibling is linked). Works single-threaded and under `-j`/`--threads`, `--inplace`, `--delay-updates` (links staged and published by rename) and `--partial`. Crosses the wire (`preserve_hard_links` bool; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0**, peers must match). Incompatible with `-s` (chunk serialization) and `--append`/`--append-verify`, rejected up front with a distinct error. See the Phase-4 hard-links notes below | | `-D` | Same as --devices --specials | ✅ Parity | Implies `--devices --specials`. `-D` was unassigned in FastSync (verified: no collision), so it is free to imply both device-node and special-file preservation. As of protocol 2.23.0 `--specials` genuinely covers **both FIFOs and unix sockets**, so `-D` covers the full rsync set. See the `--devices`/`--specials` rows and the Phase-4 devices notes below | -| `--devices` | Preserve device files | ⚠️ Caveat | Recreates char/block device nodes with `mknodat` (type + rdev strictly validated, confined fd-relative below the receive root). A device whose `mknodat` fails with `EPERM`/`EACCES` (no `CAP_MKNOD`, or super-user activity forbidden) is a **per-entry failure**: FastSync logs `cannot create device ...` (rsync logs `mknod ... failed`), counts it, **continues with the remaining files**, and ends the run with a non-OK terminal status. rsync parity: rsync likewise continues and exits partial (23). Residuals: (1) FastSync's default AUTO still *attempts* the node on a non-root receiver and therefore reports the per-entry failure, whereas rsync without `--super` silently ignores `--devices` and skips the non-regular entry with exit 0 — use `--no-super` for rsync's silent-skip behavior; (2) FastSync's process exit code for a receiver-side per-entry failure is the general error code 1, not rsync's partial 23 (a client exit-code-mapping residual that applies to every receiver file error, not just this branch); (3) with `--remove-source-files`, the non-OK terminal status means successfully transferred sources are not removed on a partial run. `--specials` (FIFOs and unix sockets) keeps the unprivileged skip path and remains parity | +| `--devices` | Preserve device files | ⚠️ Caveat | Recreates char/block device nodes with `mknodat` (type + rdev strictly validated, confined fd-relative below the receive root). A device whose `mknodat` fails with `EPERM`/`EACCES` (no `CAP_MKNOD`, or super-user activity forbidden) is a **per-entry failure**: FastSync logs `cannot create device ...` (rsync logs `mknod ... failed`), counts it, **continues with the remaining files**, and ends the run with a partial terminal status (`STATUS_PARTIAL`, protocol 2.30.0) so the client exits 23 like rsync, and under `--remove-source-files` the successfully transferred sources are still removed. rsync parity: rsync likewise continues and exits partial (23). Residual: FastSync's default AUTO still *attempts* the node on a non-root receiver and therefore reports the per-entry failure, whereas rsync without `--super` silently ignores `--devices` and skips the non-regular entry with exit 0 — use `--no-super` for rsync's silent-skip behavior. `--specials` (FIFOs and unix sockets) keeps the unprivileged skip path and remains parity | | `--specials` | Preserve special files | ✅ Parity | **FIFO and unix-socket recreation work** (protocol 2.23.0): FIFOs are recreated with `mkfifoat`, and sockets with `mknodat(..., S_IFSOCK)` — the latter is unprivileged on Linux because it materializes the socket *node*, not a live bound socket, so it is a real, assertable behavior under CI (it matches rsync, which also recreates a socket by `mknod`). Node creation is confined below the receive root (fd-relative parent; no `..`, no symlink follow) and type/rdev are validated strictly; a matching existing node is left in place and an unrelated entry is never replaced. Crosses the wire like `--devices` (the `STATUS_SPECIAL` frame). See the Phase-4 devices notes | | `--copy-devices` | Copy device contents as file | ❌ Divergent | Copies a device/FIFO's reported `st_size` into an ordinary regular file and never reads an unbounded pseudo-device, so `--sendfile` cannot hang and the run always succeeds. Deliberate safe divergence from rsync's dd-like unbounded device read, which can block; the dangerous behavior will not be implemented | | `--write-devices` | Write to devices as files | ❌ Divergent | Writes only into an existing char/block node under the confined receive root (`O_NOFOLLOW` + `O_NONBLOCK`); a missing, symlinked, FIFO-with-no-reader, non-device, or otherwise unusable destination is skipped with a warning rather than allowed or aborted. Deliberate confinement divergence from rsync's more permissive behavior | @@ -804,7 +804,7 @@ modes or links. | `--stop-after=MINS` | Stop after N minutes | ✅ Parity | Client-only sender stop deadline (Phase 6): computing `--stop-after=MINS` (a positive minute count; 0/negative/garbage rejected) and `--stop-at=TIME` (`HH:MM`, `HH:MM:SS`, or `now+N[smhd]`; a past time stops immediately). The transfer stops ELEGANTLY at the next chunk boundary: everything already fully sent is kept and applied, the run returns 0, and --delete (late/delete-after timing) does NOT wipe the destination — when the scan is cut short the partial keep-set manifest is suppressed with a warning (the delete walk is skipped rather than acting on an incomplete keep-set, so unscanned source mirrors survive). `--delete-before`/`--delete-during` still run their complete pre-scan (which ignores the deadline). Local client-only fields: never serialized into the wire config frame, so no PROTOCOL_VERSION bump. `--stop-after` uses CLOCK_MONOTONIC; `--stop-at` uses the wall clock. Works single-threaded and under `-j`/`--threads` (multithreaded). Divergence: rsync computes `--stop-after` from the run start; FastSync likewise. When both are given, the earlier of the two deadlines wins (checked per iteration). See the Phase-6 stop notes below | | `--stop-at=TIME` | Stop at specified time | ✅ Parity | Deadline transfer stop (client-only, never serialized). Protocol 2.26.0 accepts rsync's full date/time grammar (`2030-12-31T23:59`, `2030/12/31T23:59`, `2030-12-31`, `12-31`, `14:00`, `:59`, `1`) in addition to FastSync's `HH:MM[:SS]` and `now+N[smhd]`; a past time stops immediately. Everything already transferred is kept and an early stop suppresses the late `--delete` keep-set so unscanned source mirrors survive. Works single-threaded and under `-j`/`--threads` | | `--fsync` | Fsync every written file before publication | ✅ Parity | | -| `--protocol=NUM` | Force older protocol version | ❌ Divergent | Forces the wire protocol version for this transfer. FastSync has exactly ONE wire format (`PROTOCOL_VERSION`, currently 2.29.0) with no downgrade/backward-compat code paths, so `--protocol=2.29.0` is accepted (it sets the version claim the client sends, which the server already requires to match exactly) and **every other value is rejected up front** with a clear error before any connection — it does not and cannot speak an older or virtual wire format. Divergence from rsync (which negotiates a range and downgrades to an integer 0..31): FastSync's honest contract is force-to-the-one-supported-value; a genuine downgrade would require a per-version compatibility layer that does not exist. Client-only; the server-side exact-match check is unchanged. `--protocol=2.28.0`/`2.27.0`/`2.26.0`/`2.25.0`/`2.24.0`/`2.23.0`/`2.22.0`/`2.21.0`/`2.20.0`/`2.19.0`/`2.18.0`/`2.17.0`/`2.16.0`/`2.15.0`/`216`/`31`/garbage are all rejected. See the Phase-6 protocol note below | +| `--protocol=NUM` | Force older protocol version | ❌ Divergent | Forces the wire protocol version for this transfer. FastSync has exactly ONE wire format (`PROTOCOL_VERSION`, currently 2.30.0) with no downgrade/backward-compat code paths, so `--protocol=2.30.0` is accepted (it sets the version claim the client sends, which the server already requires to match exactly) and **every other value is rejected up front** with a clear error before any connection — it does not and cannot speak an older or virtual wire format. Divergence from rsync (which negotiates a range and downgrades to an integer 0..31): FastSync's honest contract is force-to-the-one-supported-value; a genuine downgrade would require a per-version compatibility layer that does not exist. Client-only; the server-side exact-match check is unchanged. `--protocol=2.29.0`/`2.28.0`/`2.27.0`/`2.26.0`/`2.25.0`/`2.24.0`/`2.23.0`/`2.22.0`/`2.21.0`/`2.20.0`/`2.19.0`/`2.18.0`/`2.17.0`/`2.16.0`/`2.15.0`/`216`/`31`/garbage are all rejected. See the Phase-6 protocol note below | | `--iconv=CONVERT_SPEC` | Charset conversion | ✅ Parity | Charset conversion of FILE NAMES (not content) at the protocol boundary via iconv(3): `--iconv=LOCAL[,REMOTE]` — the sender converts each local filename LOCAL→REMOTE before transmitting, matching rsync's rule that the spec "stays the same whether you're pushing or pulling": on a PUSH the destination end's charset is the spec's REMOTE half, so the default receiver writes the wire bytes verbatim, and only a server started with its own `--iconv` (the daemon `charset` analog) declares a different destination charset and converts REMOTE→that LOCAL (rsync push parity, differential-tested with and without a server `--iconv`). The full CONVERT_SPEC is serialized into the config frame as a new trailing string field so the peer knows the wire charset; **PROTOCOL_VERSION bumped 2.15.0 → 2.16.0**. `LOCAL[,REMOTE]` parse: single charset ⇒ LOCAL==REMOTE (identity both ways); garbage rejected up front; protocol 2.26.0 additionally accepts `--iconv=.` (the locale's default charset for both directions), `--iconv=-` and `--no-iconv` (disable conversion). Validation probes BOTH directions (a spec that only opens one way is refused, as is a NUL-emitting target charset like utf-16/utf-32/ucs-2, since filenames cannot contain NUL). An unrepresentable name (EILSEQ/EINVAL) fails that path cleanly with a logged `--iconv: cannot convert file name ...` and is never written mangled/truncated. Conversion is applied at EVERY wire-path site (regular/MKDIR/hardlink path+target/symlink path+target/SPECIAL, the delete manifest, the incremental-check path, and the `-s`/`chunk_serialize` embedded blob path), on both client and server (`--iconv` is also a server/daemon option). Zero overhead when unset. See the Phase-6 iconv notes below | | `--checksum-seed=NUM` | Set checksum seed | ✅ Parity | Sets the seed for FastSync's whole-file xxHash digest (full 64-bit seed) and for the delta path's per-block xxHash32 strong checksum (low 32 bits of the seed). **As of protocol 2.23.0 a seed of `0` — the default when the flag is unset — is randomized per transfer and the chosen seed is sent to the receiver**, exactly like rsync, so two runs against different content do not share a predictable seed; an explicit non-zero seed is used verbatim, so an explicit seed deterministically reproduces every computed digest on BOTH endpoints (the seed crosses in the config frame). `--checksum-choice=md5` has no seed and ignores it (documented). The value is a strict decimal 0..2⁶⁴-1 (blank, signed, or non-numeric values are rejected). Like rsync, a seed only matters where a digest is actually computed (`--checksum` or a basis-dir run, or a delta transfer); it does not by itself enable `--checksum`/`--delta` | | `--secluded-args`, `-s` | Use protocol to send args | ❌ Divergent | Accepted for CLI compatibility (including the rsync short `-s`, Phase 7 Wave A) but a documented **no-op / divergence**. rsync's `-s` protects arguments from shell expansion by shipping them over the protocol; FastSync never passes remote arguments through a shell expansion boundary in the first place — its SSH transport builds the remote argv as **single-quote-escaped shell words** (`ssh_build_remote_command`), so the injection/leak that `-s` guards against does not exist and there is nothing to "seclude". Implementing a true arg-send protocol would mean replacing the argv-based SSH launch with an in-band argument channel, a large redesign of the transport that buys no security here. Chunk serialization remains the long-only `--chunk-serialization`. | @@ -944,7 +944,7 @@ These are the last compatibility items and the closing phase toward rsync flag p | `-T` / `--timeout` | `-T` = `--temp-dir` | → `--timeout` (long-only) | | `-a` / `--archive` (= `-c -m -M`) | `-a` = `-rlptD` | → becomes **real rsync `-a`** after the renames | -**Wave B — Output & filesystem completion (✅ implemented).** `-S`/`--sparse` (`⚠️→✅`): real hole preservation — a sparse-aware writer (`write_all_sparse`) skips all-zero runs ≥ 4096 bytes with `lseek(SEEK_CUR)` and `ftruncate`s the final size, wired into both the atomic temp+rename store and `--inplace` receiver-side with **no wire change** (the full file image is already in memory; the ftruncate presize is kept). `-P` (`⚠️→✅`): interrupted-write retention — on a save failure after data reached the temp fd, `--partial` now renames the already-written temp to the destination path (best-effort; falls through to the normal unlink on failure, never retains when `--partial` is off) so a later `--append`/`--append-verify` run can resume. `--block-size=SIZE` (`⚠️→✅`): promoted after verification — `--block-size` is now an alias for `--delta-block`, both set `config->delta_block_size`, which the delta engine already honored end-to-end (`delta_signature_create_seeded` + `delta_apply`); out-of-range values keep the default. `--fake-super` (`⚠️→✅`): added `fake_super_restore_fd` to parse and re-apply the recorded `user.fastsync.stat` record fd-relative (mode/time only — protocol 2.23.0: **never a real chown**; the resolved owner is recorded for a later privileged restore); a save under `--fake-super` now re-applies the recorded attrs instead of only recording them. (The later fake-super xattr-interop pass replaced that native `user.fastsync.stat` format with rsync's `user.rsync.%stat` grammar — see the row and Phase-4 notes.) `--stderr=client` (`⚠️→❌ Divergent`): FastSync has no rsync client-message channel, and `client` is rejected at CLI parse — the rejection is the documented behavior (unit-tested). `-N`/`--crtimes` (`⚠️→❌ Divergent`): birth-times cannot be set by any portable fs call (`utimensat` sets only atime/mtime); capture/transmit stays, setting is impossible, the flag is accepted and safely inert. Review-hardening (post-eval): fake-super replay applies the mode through the shared `metadata_mode_for_policy` helper (protocol 2.23.0: exactly the source mode under `-p`, with no masking); `--sparse` takes precedence over `--preallocate` (posix_fallocate skipped so holes survive) — **reversed by the parity-completion wave: `--preallocate` now wins, matching rsync**; `--partial` retention is disabled for `--no_replace` (ignore/existing) and only marks a write-attempt after the actual write begins; `--block-size=SIZE`/`--delta-block=SIZE` inline forms are accepted. +**Wave B — Output & filesystem completion (✅ implemented).** `-S`/`--sparse` (`⚠️→✅`): real hole preservation — a sparse-aware writer (`write_all_sparse`) skips all-zero runs ≥ 4096 bytes with `lseek(SEEK_CUR)` and `ftruncate`s the final size, wired into both the atomic temp+rename store and `--inplace` receiver-side with **no wire change** (the full file image is already in memory; the ftruncate presize is kept). `-P` (`⚠️→✅`): interrupted-write retention — on a save failure after data reached the temp fd, `--partial` now renames the already-written temp to the destination path (best-effort; falls through to the normal unlink on failure, never retains when `--partial` is off) so a later `--append`/`--append-verify` run can resume. `--block-size=SIZE` (`⚠️→✅`): promoted after verification — `--block-size` is now an alias for `--delta-block`, both set `config->delta_block_size`, which the delta engine already honored end-to-end (`delta_signature_create_seeded` + `delta_apply`); out-of-range values keep the default. `--fake-super` (`⚠️→✅`): added `fake_super_restore_fd` to parse and re-apply the recorded `user.fastsync.stat` record fd-relative (mode/time only — protocol 2.23.0: **never a real chown**; the resolved owner is recorded for a later privileged restore); a save under `--fake-super` now re-applies the recorded attrs instead of only recording them. (The later fake-super xattr-interop pass replaced that native `user.fastsync.stat` format with rsync's `user.rsync.%stat` grammar — see the row and Phase-4 notes.) `--stderr=client` (`⚠️→❌ Divergent` then, in the wire backlog cycle, `❌→⚠️ Caveat`): the Phase-7 Wave B decision rejected `client` at CLI parse because no client-message channel existed; protocol 2.30.0 adds one (`STATUS_CLIENT_MSG`), so `client` is now accepted and forwards the client's diagnostics to the server's stderr (see the row for the remaining one-direction caveat). `-N`/`--crtimes` (`⚠️→❌ Divergent`): birth-times cannot be set by any portable fs call (`utimensat` sets only atime/mtime); capture/transmit stays, setting is impossible, the flag is accepted and safely inert. Review-hardening (post-eval): fake-super replay applies the mode through the shared `metadata_mode_for_policy` helper (protocol 2.23.0: exactly the source mode under `-p`, with no masking); `--sparse` takes precedence over `--preallocate` (posix_fallocate skipped so holes survive) — **reversed by the parity-completion wave: `--preallocate` now wins, matching rsync**; `--partial` retention is disabled for `--no_replace` (ignore/existing) and only marks a write-attempt after the actual write begins; `--block-size=SIZE`/`--delta-block=SIZE` inline forms are accepted. **Wave C — Devices & special files (finalize statuses + tests) (✅ implemented).** The four special-file rows are finalized with coverage tests. `--devices`, `--copy-devices`, and `--write-devices` are **✅ Implemented**, each with a documented, safety-driven divergence: device-node creation is privilege-gated, so a receiver without `CAP_MKNOD` skips that entry with a warning (a per-entry skip, never a transfer failure); `--copy-devices` copies a device/FIFO's reported size into an ordinary regular file (a size-bounded safe divergence from rsync's unbounded dd-like read); `--write-devices` writes only into an existing char/block node under the confined receive root and skips every unusable target rather than clobbering or aborting. `--specials` reclassified from **⛔ Impossible/Divergence** to **✅ Parity** in protocol 2.23.0: **FIFO recreation works** (unprivileged `mkfifo`) **and unix sockets are recreated** with `mknod(S_IFSOCK)`, which Linux permits unprivileged (the flag previously assumed sockets were impossible — see the `--specials` row). Tests assert FIFO recreation, socket recreation, the regular-file result of `--copy-devices`, the skipped/missing and non-device `--write-devices` targets, and (root-gated) real device-node creation; a root runner additionally drops the receiver to an unprivileged user to assert the `CAP_MKNOD` skip is graceful. (The parity-completion wave later reclassified `--devices`, `--copy-devices`, and `--write-devices` as explicit **❌ Divergent** rows, because their safe subsets are deliberately not rsync's behavior; the implementation itself is unchanged.) @@ -964,7 +964,7 @@ These are the last compatibility items and the closing phase toward rsync flag p **Wire:** two trailing config-frame blocks after the `--iconv` spec, in fixed order — `send_privilege_options`/`receive_privilege_options` (one `super_mode` int, validated `0..2`), then `send_copy_as_options`/`receive_copy_as_options` (presence int + two int32 ids, validated `>= 0`, with `copy_as_set ⇒ use_metadata`). `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Divergences from rsync:** rsync's `--super` elevates the receiver and `--copy-as` actually switches its credentials; FastSync never elevates and only permits/forwards confined attempts, and `--copy-as` forces ownership rather than switching identity. -**Honest status after the parity 2.29 cycle (protocol 2.29.0 since the symlink-xattr wire wave, which adds no config-frame field and leaves this matrix unchanged), updated by the parity cycle 2.29 pass, the audit-cycle follow-ups, the triage cycle, and a later no-wire parity pass.** ✅ Parity 119 / ⚠️ Caveat 14 / ❌ Divergent 24 = 157 rows. The no-wire parity pass accepted `--inc-recursive`/`--no-inc-recursive` as inert no-ops (❌ → ✅, since FastSync's full scan is rsync's `--no-inc-recursive` and the destination is identical), narrowed the `--temp-dir` divergence by accepting an absolute path that canonicalizes inside the receive root (the row stays ❌ for out-of-root absolute paths), closed the `--delete-before` phase-0 divergence (⚠️ → ✅: both the single-threaded and the `--threads` data passes now replay the pre-scan file list, so a source file created after the scan is neither transferred nor kept, matching rsync), and moved `--fake-super` and `--devices` ❌ → ⚠️ (`--fake-super` now writes/reads rsync's exact `user.rsync.%stat` key and ` , :` grammar, interoperating with real rsync 3.4.1 for regular files and faking char/block devices as regular files carrying the real rdev; `--devices` now logs a failed device `mknod` as a per-entry failure that continues the transfer instead of a silent non-root skip — see those rows for the remaining directory-faking and exit-code residuals). A review pass then hardened the fake-super stat parser (strict range-checked parsing), made rsync-style daemon modules read-only by default with a startup warning for accepted-but-unenforced access-control keys, and extended the `--delete-before` replay to the `--threads` path. The 2.29 cycle closed the scanner-order, delete-timing, relative-basis, and fuzzy-eligibility residuals (moving `-n`/`--delete`/`--del`/`--delete-delay` to ✅) and improved the `--info`/`--stats`/`--debug` partial rows; the triage cycle moved `-F` and `-i`/`--itemize-changes` ✅ → ⚠️ for their documented residuals. The remaining ⚠️ rows are `--info`, `--debug`, `--msgs2stderr`, `--stats`, `--progress`, `-i`, `--filter`, `-F`, the three basis-dir options, `-y/--fuzzy`, `--fake-super`, and `--devices`. Earlier: **Honest status after the parity 2.28.0 cycle (protocol 2.28.0), updated by the rsync-parity-stats, rsync-parity-options, rsync-parity-fs, parity-review, no-wire parity-track-1/2b and wire parity-track-4a/5a passes.** ✅ Parity 116 / ⚠️ Caveat 14 / ❌ Divergent 27 = 157 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. This reclassification makes every difference explicit. The completion wave closed 23 previously-caveated rows (9 that triage showed were already parity, plus 14 genuine fixes) and turned the 17 inherently non-rsync rows — native daemon config/auth, the FastSync batch container, the safe-subset device/privilege flags, `-X`'s privileged namespaces, `--fake-super`'s native xattr format, and the `--old-args` no-op — into explicit ❌ divergences. The stats pass flipped `--delete-delay` to ✅ (actual-removal accounting), but the parity-review pass moved it back to ⚠️ because FastSync charged the `--max-delete` budget at plan/snapshot time and left a refilled snapshotted directory in place, whereas rsync charges on actual removals and recursively removes a queued directory (including content created after its plan). The no-wire parity-track-1 pass fixed both (actual-removal charging plus recursive deferred removal with an independent deferred-list cap), narrowing the caveat to the partial-delete ordering. The stats pass also reclassified `--out-format` to ❌ (protocol-specific `%b`/delta-`%c`), and sharpened the `--stats`/`--progress`/`--checksum-choice` residuals. The options pass flipped `--bwlimit` and `--ignore-errors` to ✅ (rsync-exact size parsing and ~100 ms leaky-bucket throttling, and rsync's skip-unreadable-subdir plus IO-error-suppressed deletion with exit 23) and emits rsync-format `--info=name/flist/del/remove/nonreg/progress` lines (real-run `deleting`/`*deleting` carried over a new trailing `report_deletes` wire bool, `PROTOCOL_VERSION` 2.26.0 → 2.27.0), while reclassifying `-M` over daemon/TCP +**Honest status after the parity 2.29 cycle (protocol 2.29.0 since the symlink-xattr wire wave, which adds no config-frame field and leaves this matrix unchanged), updated by the parity cycle 2.29 pass, the audit-cycle follow-ups, the triage cycle, and a later no-wire parity pass.** The wire backlog cycle (protocol 2.30.0) then moved `--stderr=MODE` ❌ → ⚠️ (the `client` mode is now accepted over the new `STATUS_CLIENT_MSG` channel; only the client->server direction is reproduced) and closed the `--devices` exit-code and `--remove-source-files` residuals via `STATUS_PARTIAL` (exit 23 with successful sources removed), leaving ✅ Parity 119 / ⚠️ Caveat 15 / ❌ Divergent 23 = 157 rows. The same cycle then extended the destination-state report to directories and symlinks (#314: the receiver answers `STATUS_MKDIR`/`STATUS_SYMLINK` and an ancestor probe, so `-i`/`--progress`/`--out-format` render `.d..t......`/`cLc........` instead of `cd`/`cL` and suppress unchanged entries) and appended the per-type `deleted_*` counters to `STATUS_STATS` (#316: `--stats` now reproduces rsync's `Number of deleted files (reg/dir/link/special)` breakdown) — both differential-tested against rsync 3.4.1; the affected rows' caveats narrow but their classifications are unchanged, so the matrix stays **119 ✅ / 15 ⚠️ / 23 ❌ = 157**. The no-wire parity pass accepted `--inc-recursive`/`--no-inc-recursive` as inert no-ops (❌ → ✅, since FastSync's full scan is rsync's `--no-inc-recursive` and the destination is identical), narrowed the `--temp-dir` divergence by accepting an absolute path that canonicalizes inside the receive root (the row stays ❌ for out-of-root absolute paths), closed the `--delete-before` phase-0 divergence (⚠️ → ✅: both the single-threaded and the `--threads` data passes now replay the pre-scan file list, so a source file created after the scan is neither transferred nor kept, matching rsync), and moved `--fake-super` and `--devices` ❌ → ⚠️ (`--fake-super` now writes/reads rsync's exact `user.rsync.%stat` key and ` , :` grammar, interoperating with real rsync 3.4.1 for regular files and faking char/block devices as regular files carrying the real rdev; `--devices` now logs a failed device `mknod` as a per-entry failure that continues the transfer instead of a silent non-root skip — see those rows for the remaining directory-faking and exit-code residuals). A review pass then hardened the fake-super stat parser (strict range-checked parsing), made rsync-style daemon modules read-only by default with a startup warning for accepted-but-unenforced access-control keys, and extended the `--delete-before` replay to the `--threads` path. The 2.29 cycle closed the scanner-order, delete-timing, relative-basis, and fuzzy-eligibility residuals (moving `-n`/`--delete`/`--del`/`--delete-delay` to ✅) and improved the `--info`/`--stats`/`--debug` partial rows; the triage cycle moved `-F` and `-i`/`--itemize-changes` ✅ → ⚠️ for their documented residuals. The remaining ⚠️ rows are `--info`, `--debug`, `--stderr=MODE`, `--msgs2stderr`, `--stats`, `--progress`, `-i`, `--filter`, `-F`, the three basis-dir options, `-y/--fuzzy`, `--fake-super`, and `--devices` (15). The wire cycle for issue #315 then closed the `--filter`/`-F` merge-modifier and per-directory-receiver residuals (protocol 2.30.0 carries each directory's compiled rules and implements `e`/`n`/`w`/`-`), narrowing both rows to the merge-file-side residual (rsync reads the destination's merge file, FastSync carries the source's) and leaving the tally at **119 ✅ / 15 ⚠️ / 23 ❌ = 157**. Earlier: **Honest status after the parity 2.28.0 cycle (protocol 2.28.0), updated by the rsync-parity-stats, rsync-parity-options, rsync-parity-fs, parity-review, no-wire parity-track-1/2b and wire parity-track-4a/5a passes.** ✅ Parity 116 / ⚠️ Caveat 14 / ❌ Divergent 27 = 157 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. This reclassification makes every difference explicit. The completion wave closed 23 previously-caveated rows (9 that triage showed were already parity, plus 14 genuine fixes) and turned the 17 inherently non-rsync rows — native daemon config/auth, the FastSync batch container, the safe-subset device/privilege flags, `-X`'s privileged namespaces, `--fake-super`'s native xattr format, and the `--old-args` no-op — into explicit ❌ divergences. The stats pass flipped `--delete-delay` to ✅ (actual-removal accounting), but the parity-review pass moved it back to ⚠️ because FastSync charged the `--max-delete` budget at plan/snapshot time and left a refilled snapshotted directory in place, whereas rsync charges on actual removals and recursively removes a queued directory (including content created after its plan). The no-wire parity-track-1 pass fixed both (actual-removal charging plus recursive deferred removal with an independent deferred-list cap), narrowing the caveat to the partial-delete ordering. The stats pass also reclassified `--out-format` to ❌ (protocol-specific `%b`/delta-`%c`), and sharpened the `--stats`/`--progress`/`--checksum-choice` residuals. The options pass flipped `--bwlimit` and `--ignore-errors` to ✅ (rsync-exact size parsing and ~100 ms leaky-bucket throttling, and rsync's skip-unreadable-subdir plus IO-error-suppressed deletion with exit 23) and emits rsync-format `--info=name/flist/del/remove/nonreg/progress` lines (real-run `deleting`/`*deleting` carried over a new trailing `report_deletes` wire bool, `PROTOCOL_VERSION` 2.26.0 → 2.27.0), while reclassifying `-M` over daemon/TCP and receiver-side `protect`/`risk` re-derivation to ❌ (no argv channel / receiver filter engine); the wire parity-track-4a pass later added that receiver filter engine, flipping `--filter=RULE` back to ✅ (see above; the @@ -1207,9 +1207,12 @@ wire protocol three times (full rationale in `src/shared/config.h`): modifiers; `-f` is bound to `--filter`; a single `-F` transfers `.rsync-filter` and `-FF` excludes it. The xattr-name `x` modifier is **not implemented** and is rejected with a clear error everywhere. The merge-only - `e`/`n`/`w` and `-` modifiers are accepted and consumed on `merge`/`dir-merge` - rules (rejected elsewhere, matching rsync), but their semantics are **not - implemented** (accepted-but-ignored). + `e`/`n`/`w` and `-` modifiers are implemented on `merge`/`dir-merge` rules + (rejected elsewhere, matching rsync). Per-directory rules are carried to the + receiver (protocol 2.30.0) and re-applied deepest-first before the + command-line base, so a destination-only entry matching only a per-directory + rule is shielded; the residual is that rsync reads the destination's merge + file while FastSync carries the source's. - **Absolute basis directories** are used verbatim (rsync semantics) and **`--link-dest`** relinks an already up-to-date destination. @@ -1242,9 +1245,11 @@ These remain after the wave; the individual rows carry the precise wording. rsync's generator removes all extras ahead of its throttled sender while FastSync removes only the reached directories (completed runs agree). `--delete-before` keeps its pre-scan snapshot race; and while - base-rule `protect`/`risk` rules are now re-applied on the receiver (track 4a), - per-directory merge (`.rsync-filter`) protection is still sender-derived, so a - destination-only entry matching only a per-directory rule is not re-derived. + base-rule `protect`/`risk` rules and the per-directory merge rules are now + re-applied on the receiver (protocol 2.30.0), so a destination-only entry + matching only a per-directory rule is re-derived; the residual is the side + the merge file is read from (rsync reads the destination's, FastSync carries + the source's). `--ignore-errors` exits 23 but its EACCES differential is not exercised in CI. - **`--delay-updates`** uses a fixed staging name with an advisory lock and diff --git a/src/client/change_list.c b/src/client/change_list.c index 892cef7..ad4340b 100644 --- a/src/client/change_list.c +++ b/src/client/change_list.c @@ -123,8 +123,15 @@ static char itemize_type_char(const ChangeEvent* event) { static bool times_match(const Config* config, const ChangeEvent* event) { if (!event->dest.known || !event->dest.existed) return false; - if (event->mtime_sec == event->dest.mtime_sec) + if (event->mtime_sec == event->dest.mtime_sec) { + /* A regular file's sub-second mtime IS preserved by the receiver, so an nsec + difference is a real change. A directory or symlink has no preserved + sub-second mtime (rsync's quick-check compares whole seconds there), so a + nanosecond-only difference must not render a spurious `.d..t` / `.L..t`. */ + if (event->is_directory || event->is_symlink || event->is_special) + return true; return event->mtime_nsec == event->dest.mtime_nsec; + } long long delta = (long long)event->mtime_sec - (long long)event->dest.mtime_sec; if (delta < 0) delta = -delta; @@ -145,6 +152,10 @@ static void itemize_code(const Config* config, const ChangeEvent* event, char co /* rsync: an existing directory that only has attribute changes carries no transfer, so the update column is `.` rather than `>`. */ update = '.'; + else if (event->is_symlink) + /* rsync: an existing symlink whose target is unchanged is a `.` update + (attributes only); a changed target is `c` (the link value changed). */ + update = event->dest.target_matches ? '.' : 'c'; else update = '>'; code[0] = update; @@ -155,12 +166,20 @@ static void itemize_code(const Config* config, const ChangeEvent* event, char co code[11] = '\0'; return; } - bool size_diff = event->size != event->dest.size; - bool time_diff = !times_match(config, event); + /* rsync's value/checksum column: `c` for a symlink whose target changed (the + link value is the compared content); no destination digest is available for + a regular file. */ + bool value_diff = event->is_symlink && !event->dest.target_matches; + /* rsync itemizes size only for regular files: a directory's st_size and a + symlink's target length are not compared. */ + bool size_diff = !event->is_directory && !event->is_symlink && !event->is_special && + event->size != event->dest.size; + /* rsync itemizes the time column only when -t/--times is in effect. */ + bool time_diff = config->preserve_times && !times_match(config, event); bool perms_diff = (event->mode & 07777) != (event->dest.mode & 07777); bool owner_diff = event->uid != (uid_t)event->dest.uid; bool group_diff = event->gid != (gid_t)event->dest.gid; - code[2] = '.'; /* checksum: no destination digest available */ + code[2] = value_diff ? 'c' : '.'; code[3] = size_diff ? 's' : '.'; code[4] = time_diff ? 't' : '.'; code[5] = (config->preserve_perms && perms_diff) ? 'p' : '.'; @@ -172,6 +191,24 @@ static void itemize_code(const Config* config, const ChangeEvent* event, char co code[11] = '\0'; } +/* True when the itemized destination entry is unchanged, i.e. rsync would print + * no line at all. Reuses itemize_code so suppression is exactly consistent + * with what would have been rendered: the update column must be `.` and every + * attribute column must be `.`. */ +static bool itemize_is_unchanged(const Config* config, const ChangeEvent* event) { + if (!event->dest.known || !event->dest.existed) + return false; + char code[12]; + itemize_code(config, event, code); + if (code[0] != '.') + return false; + for (int i = 2; i < 11; i++) { + if (code[i] != '.') + return false; + } + return true; +} + /* rsync %n: the transfer-relative name, with a trailing slash for directories. * The transfer root is `.` (so `%n` renders `./`), matching rsync's root entry. */ static bool append_name(StrBuf* buf, const ChangeEvent* event) { @@ -678,6 +715,19 @@ void change_emit_file_sent_bytes(const Config* config, const File* file, char* name = NULL; char* path = NULL; fill_event_from_file(config, file, &event, &name, &path); + if (file->is_symlink) { + /* Output parity (protocol 2.30.0): an unchanged symlink is silent, like + rsync's quick check. The itemize/log stream suppresses it only when every + attribute matches; the name stream suppresses it whenever the link target + is unchanged (rsync names a symlink only when it relinks or creates it). */ + bool itemize_output = config->itemize_changes || config->out_format != NULL || + (config->log_file != NULL && config->log_file_format != NULL); + bool suppress = itemize_output + ? itemize_is_unchanged(config, &event) + : (event.dest.known && event.dest.existed && event.dest.target_matches); + if (suppress) + event.decision = CHANGE_UP_TO_DATE; + } if (name != NULL && path != NULL) { fill_event_checksum(config, file, &event); change_emit(config, &event); @@ -729,6 +779,19 @@ void change_emit_dir_sent(const Config* config, const File* file) { char* name = NULL; char* path = NULL; fill_event_from_file(config, file, &event, &name, &path); + /* Output parity (protocol 2.30.0): suppress a directory rsync would leave + silent. The itemize/log stream suppresses it only when every attribute + matches (`.d.........`); the name stream suppresses any pre-existing + directory (rsync names a directory only when it is created). */ + bool itemize_output = config->itemize_changes || config->out_format != NULL || + (config->log_file != NULL && config->log_file_format != NULL); + bool suppress = itemize_output ? itemize_is_unchanged(config, &event) + : (event.dest.known && event.dest.existed); + /* The transfer root's line is an unconditional FastSync residual (rsync keys + it off the root's own attribute change); keep emitting it. */ + bool is_root = event.name != NULL && event.name[0] == '\0'; + if (suppress && !is_root) + event.decision = CHANGE_UP_TO_DATE; if (name != NULL && path != NULL) change_emit(config, &event); free(name); diff --git a/src/client/client_cli.c b/src/client/client_cli.c index d89177a..9063b56 100644 --- a/src/client/client_cli.c +++ b/src/client/client_cli.c @@ -433,12 +433,10 @@ static int set_stderr_mode(const char* value) { log_set_stderr_mode(LOG_STDERR_ERRORS); else if (strcmp(value, "all") == 0 || strcmp(value, "a") == 0) log_set_stderr_mode(LOG_STDERR_ALL); - else if (strcmp(value, "client") == 0 || strcmp(value, "c") == 0) { - log_message(LOG_LEVEL_ERROR, - "--stderr=client is not supported: FastSync has no client message channel"); - return -1; - } else { - log_message(LOG_LEVEL_ERROR, "--stderr must be errors or all"); + else if (strcmp(value, "client") == 0 || strcmp(value, "c") == 0) + log_set_stderr_mode(LOG_STDERR_CLIENT); + else { + log_message(LOG_LEVEL_ERROR, "--stderr must be errors, all, or client"); return -1; } return 0; @@ -1353,10 +1351,9 @@ static bool cli_handle_pre_negation(CliParseCtx* ctx) { return true; } /* "--no-msgs2stderr" is the deprecated spelling of --stderr=client (rsync - * 3.4.1). FastSync has no separate client message channel, so the closest - * supported mode is the errors-only default. */ + * 3.4.1); the client-message channel now exists, so it maps to `client`. */ if (strcmp(arg, "--no-msgs2stderr") == 0) - return set_stderr_mode("errors") == 0; + return set_stderr_mode("client") == 0; /* "--no-motd" is a real rsync option name (client-side daemon MOTD display * suppression), not a negation of a "--motd" flag, so it is handled before * the generic --no-* negation branch. */ @@ -2810,8 +2807,11 @@ static int cli_finalize_config(Config* config, bool verbose, bool no_delta, bool * need the pre-transfer destination snapshot (new vs modified and which * attributes differ), so ask the receiver to report it on every per-file * check. This is a wire field. */ + bool progress_active = + !config->quiet && (config->show_progress || (config->info_level & LOG_INFO_PROGRESS) != 0); config->report_dest_info = config->itemize_changes || config->out_format != NULL || - (config->log_file != NULL && config->log_file_format != NULL); + (config->log_file != NULL && config->log_file_format != NULL) || + progress_active; /* Wire-stats parity: --stats, --progress/-P, an --out-format token that needs * a wire counter (%b/%c), or a dry-run --delete need the receiver's * end-of-transfer STATUS_STATS report. This is a wire field (protocol diff --git a/src/client/client_manifest.c b/src/client/client_manifest.c index 55a3f67..04fae0b 100644 --- a/src/client/client_manifest.c +++ b/src/client/client_manifest.c @@ -330,9 +330,11 @@ int send_list_only(const Config* config) { } /* Send the delete manifest to the server. Returns 0 on success, -1 on - failure. It carries FOUR sections: the keep-set paths, the protected - excluded prefixes, the --delete-missing-args exact-delete paths, and the - destination-relative directories the sender synchronized this run. + failure. It carries FOUR path sections (keep-set paths, protected excluded + prefixes, --delete-missing-args exact-delete paths, and the destination- + relative directories the sender synchronized this run) followed by the + protocol-2.30.0 per-directory filter-rule block (`per_dir_rules`, the rules + the scan compiled from each directory's merge files). When --delete-excluded is given `protected` is empty: excluded destination mirrors are then ordinary extras and are removed. When --delete-missing-args is active `missing_args` holds the destination mirrors @@ -346,7 +348,8 @@ int send_list_only(const Config* config) { frame. A heavily filtered source whose exclusion list is large therefore fails the run cleanly on the receiver rather than being truncated. */ int send_delete_manifest(int fd, ArrayList* manifest, ArrayList* protected_prefixes, - ArrayList* size_skipped, ArrayList* missing_args, ArrayList* synced_dirs) { + ArrayList* size_skipped, ArrayList* missing_args, ArrayList* synced_dirs, + const FilterRuleList* per_dir_rules) { if (!send_status(fd, STATUS_MANIFEST)) return -1; int keep_count = manifest ? manifest->size : 0; @@ -389,6 +392,11 @@ int send_delete_manifest(int fd, ArrayList* manifest, ArrayList* protected_prefi if (!send_wire_str(fd, (char*)synced_dirs->items[i])) return -1; } + /* Protocol 2.30.0: the receiver-side per-directory filter rules discovered by + the sender's scan, so the whole-tree commit walker can shield a + destination-only entry that matches only a per-directory merge rule. */ + if (!delete_filter_dir_rules_send(fd, per_dir_rules)) + return -1; return 0; } @@ -409,11 +417,11 @@ int send_delete_manifest(int fd, ArrayList* manifest, ArrayList* protected_prefi bool send_delete_manifest_early(Client* client, ArrayList* manifest, ArrayList* protected_prefixes, ArrayList* size_skipped, ArrayList* missing_args, - ArrayList* synced_dirs) { + ArrayList* synced_dirs, const FilterRuleList* per_dir_rules) { if (!client || !manifest) return false; if (send_delete_manifest(client->file_descriptor, manifest, protected_prefixes, size_skipped, - missing_args, synced_dirs) != 0) + missing_args, synced_dirs, per_dir_rules) != 0) return false; Status ack; /* The wait is long (up to an hour) and runs inline on this thread: a helper @@ -487,6 +495,7 @@ int send_dry_run_remote(Config* config) { protocol_session_bind(&session); int ret = 1; + bool partial = false; time_t dry_start = time(NULL); ReceiverStats dry_stats; memset(&dry_stats, 0, sizeof(dry_stats)); @@ -497,6 +506,7 @@ int send_dry_run_remote(Config* config) { ArrayList* dry_dirs = NULL; ArrayList* dry_excluded = NULL; ArrayList* dry_size_skipped = NULL; + FilterRuleList* dry_per_dir = NULL; if (!config_send(client->file_descriptor, config)) goto dry_fail; receive_daemon_motd(client, config); @@ -509,8 +519,10 @@ int send_dry_run_remote(Config* config) { dry_manifest = array_list_create(free); dry_dirs = array_list_create(free); dry_size_skipped = array_list_create(free); - if (!dry_manifest || !dry_dirs || !dry_size_skipped) + dry_per_dir = filter_rule_list_create(); + if (!dry_manifest || !dry_dirs || !dry_size_skipped || !dry_per_dir) goto dry_fail; + prepared.options.per_dir_rules = dry_per_dir; if (!config->delete_excluded) { dry_excluded = array_list_create(free); if (!dry_excluded) @@ -612,7 +624,7 @@ int send_dry_run_remote(Config* config) { bool early_delete = config->use_delete && config_delete_timing_early(config); if (dry_manifest) { if (send_delete_manifest(client->file_descriptor, dry_manifest, dry_excluded, dry_size_skipped, - NULL, dry_dirs) != 0) + NULL, dry_dirs, dry_per_dir) != 0) goto dry_fail; if (early_delete) { Status ack; @@ -642,8 +654,14 @@ int send_dry_run_remote(Config* config) { if (!receive_status(client->file_descriptor, &status)) goto dry_fail; } - if (status != STATUS_OK) + /* A per-entry receiver failure is rsync's PARTIAL transfer (exit 23), not a + hard failure: a dry run transfers nothing, but keep the verdict consistent + with the normal path instead of treating it as a protocol error. */ + if (status == STATUS_PARTIAL) { + partial = true; + } else if (status != STATUS_OK) { goto dry_fail; + } if (!config->quiet) { if (config->human_readable) printf("Total: %d files, %s\n", file_count, @@ -661,7 +679,7 @@ int send_dry_run_remote(Config* config) { dry_transfer.literal_data = total_bytes; report_transfer_stats(config, &dry_transfer, dry_start, &dry_stats); } - ret = io_error ? 1 : 0; + ret = io_error ? 1 : (partial ? 23 : 0); dry_fail: if (dry_manifest) @@ -672,6 +690,8 @@ dry_fail: array_list_delete(dry_excluded); if (dry_size_skipped) array_list_delete(dry_size_skipped); + if (dry_per_dir) + filter_rule_list_free(dry_per_dir); if (scanner) directory_scanner_destroy(scanner); prepared_scanner_destroy(&prepared); diff --git a/src/client/client_report.c b/src/client/client_report.c index ff45260..837f9b9 100644 --- a/src/client/client_report.c +++ b/src/client/client_report.c @@ -12,6 +12,7 @@ #include #include #include +#include #include /* Surface a server rejection to the user. When the last status exchange @@ -161,6 +162,9 @@ void report_transfer_stats(const Config* config, const TransferStats* stats, tim created_breakdown, sizeof(created_breakdown)); unsigned long long created_total = recv->created_reg + recv->created_dir + recv->created_link + recv->created_special; + char deleted_breakdown[128]; + type_breakdown(recv->deleted_reg, recv->deleted_dir, recv->deleted_link, recv->deleted_special, + deleted_breakdown, sizeof(deleted_breakdown)); printf("\n"); if (breakdown[0] != '\0') printf("Number of files: %llu %s\n", flist_total, breakdown); @@ -172,7 +176,13 @@ void report_transfer_stats(const Config* config, const TransferStats* stats, tim printf("Number of created files: %llu %s\n", created_total, created_breakdown); else printf("Number of created files: %llu\n", created_total); - printf("Number of deleted files: %llu\n", recv->deleted_files); + /* Protocol 2.30.0: the receiver reports the removed entries split by type, so + this line matches rsync's `Number of deleted files: X (reg: A, dir: B, + link: C, special: D)` (only the non-zero categories are listed). */ + if (deleted_breakdown[0] != '\0') + printf("Number of deleted files: %llu %s\n", recv->deleted_files, deleted_breakdown); + else + printf("Number of deleted files: %llu\n", recv->deleted_files); printf("Number of regular files transferred: %llu\n", stats->transferred_regular); printf("Total file size: %s bytes\n", total); printf("Total transferred file size: %s bytes\n", transferred); @@ -525,9 +535,16 @@ static void client_progress_emit_ancestors(const Config* config, const char* rel if (dir != NULL) change_emit_dir_sent(config, dir); } else { - char* escaped = output_escape(prefix, config->eight_bit_output); - printf("%s/\n", escaped ? escaped : prefix); - free(escaped); + /* --progress/-P names a directory only when it is newly created; + rsync stays silent for a pre-existing directory even when one of + its children changed (protocol 2.30.0 dest-state report). */ + const File* dir = progress_dir_lookup(prefix); + bool existed = dir != NULL && dir->dest_state.known && dir->dest_state.existed; + if (!existed) { + char* escaped = output_escape(prefix, config->eight_bit_output); + printf("%s/\n", escaped ? escaped : prefix); + free(escaped); + } } g_progress_index++; } else { @@ -552,6 +569,122 @@ void client_change_emit_ancestors(const Config* config, const File* file) { client_progress_emit_ancestors(config, rel); } +/* Send one STATUS_MKDIR probe (probe=1) for a pre-count directory and cache the + * receiver's pre-transfer destination snapshot in `dir->dest_state`. Returns + * false on a protocol/transport error. */ +static bool client_probe_dir_state(int fd, File* dir) { + if (dir == NULL || file_wire_path(dir) == NULL) + return true; + if (!send_status(fd, STATUS_MKDIR) || !send_int(fd, 1) || !send_wire_str(fd, file_wire_path(dir))) + return false; + Status status = STATUS_ERROR; + if (!receive_status(fd, &status) || status != STATUS_DEST_INFO || + !format_dest_state_receive(fd, &dir->dest_state)) { + log_message(LOG_LEVEL_ERROR, "Directory destination-state probe failed"); + return false; + } + return true; +} + +/* Output parity (protocol 2.30.0): ask the receiver for each not-yet-probed + * ancestor directory's pre-transfer state BEFORE the entry that first triggers + * it is sent, so the ancestor's -i/--out-format line renders rsync's + * `.d..t......` (existing, attributes changed) versus `cd+++++++++` (created) + * and an unchanged directory is suppressed. The probe is a STATUS_MKDIR frame + * with probe=1 (the receiver reports and creates nothing), so it must run before + * the receiver implicitly creates the parent for the child. Each directory is + * probed at most once; the result is cached in the pre-count File's dest_state. + * Returns false on a protocol/transport error (the caller aborts the transfer). */ +bool client_change_probe_ancestors(const Config* config, const File* file, int fd) { + if (config == NULL || file == NULL || fd < 0 || !config->report_dest_info) + return true; + if (!g_progress_dir_index_valid || !g_progress_precount.dir_refs) + return true; + const char* rel = delete_display_path(config, file_wire_path(file)); + if (rel == NULL) + return true; + size_t rel_len = strlen(rel); + for (size_t i = 1; i < rel_len; i++) { + if (rel[i] != '/') + continue; + char* prefix = malloc(i + 1); + if (prefix == NULL) + return false; + memcpy(prefix, rel, i); + prefix[i] = '\0'; + if (path_index_contains(&g_progress_dir_index, prefix)) { + File* dir = progress_dir_lookup(prefix); + /* The probe path is the same wire path the real STATUS_MKDIR would carry + (the pre-count File's send_path, or its absolute source path for a + plain recursive scan), not the display-relative prefix. */ + if (dir != NULL && !dir->dest_state.known && !client_probe_dir_state(fd, dir)) { + free(prefix); + return false; + } + } + free(prefix); + } + return true; +} + +/* Mark a directory the data pass already itemized/named so the end-of-transfer + * pending-directory flush does not report it a second time. `file` is a + * transferred directory entry (an empty-directory STATUS_MKDIR). */ +void client_change_mark_dir(const Config* config, const File* file) { + if (config == NULL || file == NULL || !g_progress_emitted_valid || + g_progress_emitted_keys == NULL) + return; + const char* rel = delete_display_path(config, file_wire_path(file)); + if (rel == NULL || rel[0] == '\0' || str_hash_set_lookup(&g_progress_emitted, rel)) + return; + char* key = str_dup(rel); + if (key == NULL) + return; + if (!array_list_add(g_progress_emitted_keys, key)) { + free(key); + return; + } + str_hash_set_insert_ref(&g_progress_emitted, key); +} + +/* Emit the itemize lines for source directories that CHANGED but had no + * transferred child, so no ancestor emission reached them (rsync reports a + * directory whose attributes changed even when its contents did not). Runs at + * the end of the data pass, BEFORE the deferred STATUS_DIR_TIMES apply, so the + * probe still observes each untouched directory's pre-transfer state. Only the + * itemize/out-format/log streams report attribute-only directory changes; + * --progress/-P stays silent for them, matching rsync. Best-effort: a probe + * failure simply stops the flush (the transfer's verdict is unaffected). */ +void client_change_emit_pending_dirs(const Config* config, int fd) { + if (config == NULL || fd < 0 || !config->report_dest_info) + return; + bool itemize_output = config->itemize_changes || config->out_format != NULL || + (config->log_file != NULL && config->log_file_format != NULL); + if (!itemize_output) + return; + if (!g_progress_dir_index_valid || g_progress_precount.dir_refs == NULL || + !g_progress_emitted_valid || g_progress_emitted_keys == NULL) + return; + for (int i = 0; i < g_progress_precount.dir_refs->size; i++) { + DirRef* ref = (DirRef*)g_progress_precount.dir_refs->items[i]; + if (ref == NULL || ref->name == NULL || ref->name[0] == '\0') + continue; + if (str_hash_set_lookup(&g_progress_emitted, ref->name)) + continue; + File* dir = ref->file; + if (dir == NULL) + continue; + if (!dir->dest_state.known && !client_probe_dir_state(fd, dir)) + return; + change_emit_dir_sent(config, dir); + char* key = str_dup(ref->name); + if (key != NULL && array_list_add(g_progress_emitted_keys, key)) + str_hash_set_insert_ref(&g_progress_emitted, key); + else + free(key); + } +} + /* rsync's --info=name/progress line for one entry: transfer-relative name (a * trailing slash for directories) plus the ` -> target` symlink suffix. */ static char* progress_entry_line(const File* file, const char* rel) { @@ -644,6 +777,15 @@ void client_progress_file(const Config* config, const File* file) { void client_progress_name(const Config* config, const File* file) { if (!g_progress_active || file == NULL) return; + /* rsync's --progress/-P name stream reports an entry only when it is created + or (for a symlink) actually relinked: a pre-existing directory or an + unchanged symlink is silent (protocol 2.30.0 dest-state report). */ + if (file->dest_state.known && file->dest_state.existed) { + if (file->is_dir || (file->is_symlink && file->dest_state.target_matches)) { + g_progress_index++; + return; + } + } const char* rel = delete_display_path(config, file_wire_path(file)); if (!config->itemize_changes && config->out_format == NULL) { char* line = progress_entry_line(file, rel ? rel : ""); @@ -1051,3 +1193,129 @@ const char* delete_display_path(const Config* config, const char* path) { return path; return utils_strip_transfer_root(path, config->send_directory); } + +/* ---- --stderr=client diagnostic channel (protocol 2.30.0) ---- + * + * When the client's --stderr mode is `client`, log_message() hands each of the + * client's own diagnostics to the sink installed here instead of writing them + * locally. The sink QUEUES the text (it may be called from scanner worker + * threads while the sender is streaming) and the sender thread -- the sole + * writer of the protocol stream -- drains the queue over the wire at frame + * boundaries via client_flush_client_messages(). A bounded queue caps the + * memory a chatty run can pin; overflow falls back to local output so a + * diagnostic is never silently dropped. */ +#define CLIENT_MSG_MAX_QUEUED 256 +#define CLIENT_MSG_MAX_BYTES (256 * 1024) + +static mtx_t client_msg_mutex; +static once_flag client_msg_mutex_once = ONCE_FLAG_INIT; +static ArrayList* client_msg_queue = NULL; /* owns char* */ +static size_t client_msg_bytes = 0; +/* True only while a live transfer session exists: before the connection is up + (or after it drops) the sink declines so log_message falls back to local + output, matching rsync's documented fallback. Written by the sender thread + (client_messages_activate) and read by scanner worker threads in + client_msg_enqueue, so it must be atomic: the queue itself stays guarded by + client_msg_mutex, but the flag is polled before taking that lock. */ +static _Atomic bool client_msg_active = false; + +static void client_msg_mutex_init(void) { + mtx_init(&client_msg_mutex, mtx_plain); +} + +static bool client_msg_enqueue(const char* message); + +/* Install the global log sink for the duration of one transfer. Safe to call + * more than once; the queue is created lazily. */ +void client_messages_install(void) { + call_once(&client_msg_mutex_once, client_msg_mutex_init); + mtx_lock(&client_msg_mutex); + if (!client_msg_queue) + client_msg_queue = array_list_create(free); + bool ready = client_msg_queue != NULL; + mtx_unlock(&client_msg_mutex); + /* Only arm the sink once the queue exists; on allocation failure leave the + sink uninstalled so log_message keeps writing locally instead of handing + messages to a sink that would silently drop them. */ + if (ready) + log_set_client_msg_sink(client_msg_enqueue); +} + +void client_messages_activate(bool active) { + atomic_store(&client_msg_active, active); +} + +/* log_message sink: takes ownership (queues) the message when a session is + * live; returns false otherwise so the caller writes it locally. */ +static bool client_msg_enqueue(const char* message) { + bool active = atomic_load(&client_msg_active); + if (!message || message[0] == '\0') + return active; + if (!active) + return false; + size_t len = strlen(message); + call_once(&client_msg_mutex_once, client_msg_mutex_init); + mtx_lock(&client_msg_mutex); + bool queued = false; + if (client_msg_queue && (size_t)client_msg_queue->size < CLIENT_MSG_MAX_QUEUED && + client_msg_bytes + len <= CLIENT_MSG_MAX_BYTES) { + char* copy = str_dup(message); + if (copy) { + if (array_list_add(client_msg_queue, copy)) { + client_msg_bytes += len; + queued = true; + } else { + free(copy); + } + } + } + mtx_unlock(&client_msg_mutex); + return queued; +} + +/* Drain the queued diagnostics as STATUS_CLIENT_MSG frames on the sender + * thread. Swaps the queue out under the mutex so a concurrent worker logging + * never blocks on the wire. Must be called at a protocol frame boundary. */ +void client_flush_client_messages(int fd) { + if (fd < 0) + return; + call_once(&client_msg_mutex_once, client_msg_mutex_init); + mtx_lock(&client_msg_mutex); + ArrayList* pending = client_msg_queue; + if (pending) { + ArrayList* fresh = array_list_create(free); + if (fresh) { + client_msg_queue = fresh; + } else { + /* No memory for a replacement queue: stop queuing new diagnostics (they + fall back to local output) and drain this batch below so nothing is + silently dropped. */ + client_msg_queue = NULL; + log_set_client_msg_sink(NULL); + } + client_msg_bytes = 0; + } + mtx_unlock(&client_msg_mutex); + if (!pending) + return; + for (int i = 0; i < pending->size; i++) { + const char* message = pending->items[i]; + if (message && message[0] != '\0' && !send_client_message(fd, message)) + break; /* peer is gone; the rest would fail too */ + } + array_list_delete(pending); +} + +/* Tear down the sink after a transfer and free anything still queued. */ +void client_messages_end(void) { + log_set_client_msg_sink(NULL); + atomic_store(&client_msg_active, false); + call_once(&client_msg_mutex_once, client_msg_mutex_init); + mtx_lock(&client_msg_mutex); + ArrayList* pending = client_msg_queue; + client_msg_queue = NULL; + client_msg_bytes = 0; + mtx_unlock(&client_msg_mutex); + if (pending) + array_list_delete(pending); +} diff --git a/src/client/client_send.c b/src/client/client_send.c index 16e386c..354b86f 100644 --- a/src/client/client_send.c +++ b/src/client/client_send.c @@ -127,6 +127,11 @@ Client* connect_transfer_client(const Config* config) { void disconnect_transfer_client(Client* client) { if (!client) return; + /* --stderr=client: push any diagnostics logged during the transfer to the + peer before the socket closes; once deactivated, later messages fall back + to local output instead of being lost. */ + client_flush_client_messages(client->file_descriptor); + client_messages_activate(false); client_disconnect(client); client_delete(client); } @@ -240,13 +245,28 @@ static void mark_sender_done(PipelineContextSender* context) { When --remove-source-files is active the receiver acknowledges each data file it processed, in send order: STATUS_NEXT means the file was written, STATUS_OK means the file was skipped/unchanged. Skipped sources are marked - so the later removal pass keeps them. */ + so the later removal pass keeps them. `partial_out` is set when the receiver + reported STATUS_PARTIAL (a per-entry receiver failure): the transfer is + otherwise complete, so successfully stored sources are still removed and the + caller exits 23 (rsync's partial transfer) instead of a fatal non-zero. */ static bool finalize_transfer(Client* client, const Config* config, ArrayList* remove_sources, - bool* delete_limit_out, ReceiverStats* stats_out) { + bool* delete_limit_out, bool* partial_out, ReceiverStats* stats_out) { if (delete_limit_out) *delete_limit_out = false; + if (partial_out) + *partial_out = false; + /* --stderr=client: the receiver consumes frames until it reads + STATUS_FINISHED, after which it no longer reads. Flush every diagnostic + queued during the transfer here -- the last frame boundary at which the + peer is still reading -- so nothing is stranded in the queue. */ + client_flush_client_messages(client->file_descriptor); if (!send_status(client->file_descriptor, STATUS_FINISHED)) return false; + /* Past STATUS_FINISHED the receiver has stopped reading, so any diagnostic + logged from here on (notably the STATUS_PARTIAL warning below) can no + longer be forwarded. Deactivate the channel so those messages fall back + to local output instead of being queued for a closed peer and lost. */ + client_messages_activate(false); /* The receiver emits its optional wire-stats frame (protocol 2.25.0) FIRST, then any per-file --remove-source-files acks, then the terminal status. */ Status status; @@ -298,6 +318,16 @@ static bool finalize_transfer(Client* client, const Config* config, ArrayList* r *delete_limit_out = true; return true; } + /* A per-entry receiver failure the receiver chose to continue past is a + rsync PARTIAL transfer: everything else succeeded and the stored sources + may be removed, but the client must exit 23. */ + if (status == STATUS_PARTIAL) { + log_message(LOG_LEVEL_WARNING, + "some files could not be transferred (see the server log for details)"); + if (partial_out) + *partial_out = true; + return true; + } if (status != STATUS_OK) { log_server_rejection("Receiver reported transfer failure"); return false; @@ -595,14 +625,34 @@ static bool send_file_direct(File* file, int fd, bool use_metadata, int compress static bool send_directory_entry(const Client* client, File* file, const Config* config) { if (!file || !file_wire_path(file)) return false; - if (!send_status(client->file_descriptor, STATUS_MKDIR) || - !send_wire_str(client->file_descriptor, file_wire_path(file))) + int fd = client->file_descriptor; + if (!send_status(fd, STATUS_MKDIR)) return false; - if (config->use_metadata && !metadata_send(client->file_descriptor, file->metadata)) + /* Output parity (protocol 2.30.0): when report_dest_info is negotiated every + STATUS_MKDIR body is prefixed with a probe flag (1 = probe only, 0 = a real + create), so the receiver knows whether to expect the metadata/xattr block. */ + if (config->report_dest_info && !send_int(fd, 0)) + return false; + if (!send_wire_str(fd, file_wire_path(file))) + return false; + if (config->use_metadata && !metadata_send(fd, file->metadata)) return false; /* Directory xattrs/ACLs (-X/-A) ride the same trailing block as regular files when the xattr transport was negotiated. */ - return !config->use_xattrs || xattr_send(client->file_descriptor, file->xattrs); + if (config->use_xattrs && !xattr_send(fd, file->xattrs)) + return false; + /* The receiver answers with the directory's pre-transfer destination state + BEFORE creating it, so the sender can render rsync's `.d..t......` versus + `cd+++++++++` and suppress an unchanged directory. */ + if (config->report_dest_info) { + Status status; + if (!receive_status(fd, &status) || status != STATUS_DEST_INFO || + !format_dest_state_receive(fd, &file->dest_state)) { + log_message(LOG_LEVEL_ERROR, "Unexpected reply to the directory destination-state report"); + return false; + } + } + return true; } /* P7 Wave D: transmit every captured source directory's metadata in terminal @@ -658,7 +708,21 @@ static bool send_symlink_entry(const Client* client, File* file, const Config* c return false; /* Symlink xattrs/ACLs (-X/-A) ride the same trailing block as regular files and directories when the xattr transport was negotiated. */ - return !config->use_xattrs || xattr_send(fd, file->xattrs); + if (config->use_xattrs && !xattr_send(fd, file->xattrs)) + return false; + /* The receiver answers with the symlink's pre-transfer destination state + (including whether the on-disk link target already matches) BEFORE creating + it, so the sender can render rsync's `cLc........` / `.L..t......` and + suppress an unchanged symlink. */ + if (config->report_dest_info) { + Status status; + if (!receive_status(fd, &status) || status != STATUS_DEST_INFO || + !format_dest_state_receive(fd, &file->dest_state)) { + log_message(LOG_LEVEL_ERROR, "Unexpected reply to the symlink destination-state report"); + return false; + } + } + return true; } // Send a single file directly via sendfile (non-incremental path). @@ -815,6 +879,9 @@ static bool source_is_regular_file(const File* file) { static int send_chunk_with_removal(Client* client, Chunk* chunk, Config* config, ArrayList* remove_sources, TransferStats* stats) { + /* --stderr=client: this is a frame boundary, so forward any diagnostics the + scanner/log emitted since the previous chunk before the next frame. */ + client_flush_client_messages(client->file_descriptor); if (config->use_chunk_serialization) { if (remove_sources) { for (int i = 0; i < chunk->element_count; i++) { @@ -842,14 +909,23 @@ static int send_chunk_with_removal(Client* client, Chunk* chunk, Config* config, if (chunk->items[i] == NULL) continue; transfer_stats_note_entry(stats, chunk->items[i]); + /* Output parity: probe each entry's ancestor directories' destination + state before emitting its itemize line, exactly as the non-serialized + loop does. Without this, dest_state.known stays false and -i/-P + renders an existing dir/symlink as created instead of `.d..t...` (or + suppressing it). */ + if (!client_change_probe_ancestors(config, chunk->items[i], client->file_descriptor)) + return -1; /* The chunk-serialization path emits no --progress name lines, so only feed -i/--out-format its ancestor directory lines here. */ if (config->itemize_changes || config->out_format != NULL) client_change_emit_ancestors(config, chunk->items[i]); - if (chunk->items[i]->is_dir) + if (chunk->items[i]->is_dir) { change_emit_dir_sent(config, chunk->items[i]); - else + client_change_mark_dir(config, chunk->items[i]); + } else { change_emit_file_sent(config, chunk->items[i]); + } if (!chunk->items[i]->is_dir) transfer_stats_note_transferred(stats, chunk->items[i]); } @@ -861,6 +937,11 @@ static int send_chunk_with_removal(Client* client, Chunk* chunk, Config* config, if (f == NULL) continue; transfer_stats_note_entry(stats, f); + /* Output parity: probe this entry's ancestor directories' destination state + before the entry (or the first child below them) is sent, while the + receiver has not yet created them implicitly. */ + if (!client_change_probe_ancestors(config, f, client->file_descriptor)) + return -1; if (f->is_dir) { /* Explicit directory entry (--dirs): a MKDIR frame carrying the destination path (and metadata when negotiated). Directories have no @@ -869,6 +950,7 @@ static int send_chunk_with_removal(Client* client, Chunk* chunk, Config* config, return -1; client_change_emit_ancestors(config, f); change_emit_dir_sent(config, f); + client_change_mark_dir(config, f); client_progress_name(config, f); continue; } @@ -954,6 +1036,7 @@ static int send_chunks_multithreaded(void* pipeline_context) { protocol_session_set_io_timeout(&session, context->config->timeout); protocol_session_set_ssl(&session, (SSL*)client->ssl); protocol_session_bind(&session); + client_messages_activate(true); if (!config_send(client->file_descriptor, context->config)) { pipeline_cancel(context); disconnect_transfer_client(client); @@ -967,7 +1050,7 @@ static int send_chunks_multithreaded(void* pipeline_context) { and wait for the receiver to delete extras before streaming any data. */ if (!send_delete_manifest_early(client, context->manifest, context->excluded_paths, context->size_skipped_paths, context->missing_args, - context->synced_dirs)) { + context->synced_dirs, context->per_dir_rules)) { pipeline_cancel(context); disconnect_transfer_client(client); mark_sender_done(context); @@ -1097,7 +1180,8 @@ static int send_chunks_multithreaded(void* pipeline_context) { log_message(LOG_LEVEL_WARNING, "IO error encountered -- skipping file deletion"); } else if (send_delete_manifest(client->file_descriptor, context->manifest, context->excluded_paths, context->size_skipped_paths, - context->missing_args, context->synced_dirs) != 0) { + context->missing_args, context->synced_dirs, + context->per_dir_rules) != 0) { goto send_fail; } } else if (context->config->delete_missing_args && !context->early_delete && @@ -1105,7 +1189,7 @@ static int send_chunks_multithreaded(void* pipeline_context) { /* --delete-missing-args without --delete: no keep-set is built, but the exact-delete paths still ride the same manifest frame (commit once the transfer succeeded). */ - if (send_delete_manifest(client->file_descriptor, NULL, NULL, NULL, context->missing_args, + if (send_delete_manifest(client->file_descriptor, NULL, NULL, NULL, context->missing_args, NULL, NULL) != 0) goto send_fail; } @@ -1113,15 +1197,19 @@ static int send_chunks_multithreaded(void* pipeline_context) { (and all parallel workers) has been joined before scanner_done was set, so the list is complete and race-free; on an early stop the list may be incomplete and is deliberately not sent. */ + client_change_emit_pending_dirs(context->config, client->file_descriptor); if (!context->scan_stopped_early && !send_dir_times(client, context->config, context->dir_entries)) goto send_fail; bool delete_limit = false; + bool partial = false; ReceiverStats recv_stats; memset(&recv_stats, 0, sizeof(recv_stats)); + client_flush_client_messages(client->file_descriptor); bool ok = finalize_transfer(client, context->config, context->remove_source_files, &delete_limit, - &recv_stats); + &partial, &recv_stats); context->delete_limit = delete_limit; + context->partial = partial; if (!ok && context->config->use_delete) log_message(LOG_LEVEL_ERROR, "server reported a deletion failure (--delete); see the server log for the reason"); @@ -1215,6 +1303,7 @@ static int scan_directory_multithreaded(void* pipeline_context) { its protected lists, so the data pass must not append to them again. */ if (!context->early_delete && !context->delete_plans) { prepared.options.excluded_paths = context->excluded_paths; + prepared.options.per_dir_rules = context->per_dir_rules; /* The root marker for a full recursive transfer is already in the list; do not let the scanner append every directory to it. */ if (context->config->files_from_set != NULL) @@ -1448,6 +1537,8 @@ typedef struct { ArrayList* synced_dirs; ArrayList* plan_dirs; ArrayList* missing_args; + /* Per-directory filter rules compiled by the scan (protocol 2.30.0). */ + FilterRuleList* per_dir_rules; PreparedScanner prepared; StopCondition stop; TransferStats transfer_stats; @@ -1492,9 +1583,11 @@ static bool send_files_prepare(Config* config, SendFilesState* state) { } state->size_skipped = array_list_create(free); state->synced_dirs = array_list_create(free); - if (!state->size_skipped || !state->synced_dirs) + state->per_dir_rules = filter_rule_list_create(); + if (!state->size_skipped || !state->synced_dirs || !state->per_dir_rules) return false; state->prepared.options.size_skipped_paths = state->size_skipped; + state->prepared.options.per_dir_rules = state->per_dir_rules; /* Only a --files-from subset confines the extras walk to the directories the scan synchronized; a full recursive transfer deletes throughout the receive root, so mark the root itself (the "." sentinel) and let the @@ -1559,9 +1652,9 @@ static bool send_files_prepare_delete(Config* config, SendFilesState* state) { log_message(LOG_LEVEL_WARNING, "IO error encountered -- skipping file deletion"); skip_delete = true; } else { - early_ok = - send_delete_manifest_early(client, early_manifest, state->excluded, state->size_skipped, - state->missing_args, state->synced_dirs); + early_ok = send_delete_manifest_early(client, early_manifest, state->excluded, + state->size_skipped, state->missing_args, + state->synced_dirs, state->per_dir_rules); } } array_list_delete(early_manifest); @@ -1570,6 +1663,7 @@ static bool send_files_prepare_delete(Config* config, SendFilesState* state) { state->prepared.options.excluded_paths = NULL; state->prepared.options.size_skipped_paths = NULL; state->prepared.options.synced_dirs = NULL; + state->prepared.options.per_dir_rules = NULL; if (!prescan_ok || (!early_ok && !skip_delete)) { array_list_delete(prescan_chunks); return false; @@ -1599,7 +1693,7 @@ static bool send_files_prepare_delete(Config* config, SendFilesState* state) { config->files_from_set ? state->synced_dirs : (walk_root ? state->synced_dirs : NULL); delete_plan_sender_finalize(state->plan_sender, scope, walk_root); delete_plan_sender_set_config(state->plan_sender, state->excluded, state->size_skipped, - state->missing_args); + state->missing_args, state->per_dir_rules); if (state->had_scan_io && delete_plan_sender_empty(state->plan_sender)) { log_message(LOG_LEVEL_ERROR, "source scan hit an I/O error before finding any file; refusing to delete " @@ -1623,6 +1717,7 @@ static bool send_files_prepare_delete(Config* config, SendFilesState* state) { state->prepared.options.size_skipped_paths = NULL; state->prepared.options.synced_dirs = NULL; state->prepared.options.plan_dirs = NULL; + state->prepared.options.per_dir_rules = NULL; if (!prescan_ok || (!plans_ok && !skip_delete)) return false; } else if (config->use_delete) { @@ -1804,7 +1899,8 @@ static int send_files_finalize(const Config* config, SendFilesState* state) { modes the deletion already went out with the data, so nothing is re-sent here. */ if (send_delete_manifest(client->file_descriptor, state->manifest, state->excluded, - state->size_skipped, state->missing_args, state->synced_dirs) != 0) { + state->size_skipped, state->missing_args, state->synced_dirs, + state->per_dir_rules) != 0) { if (state->manifest) { array_list_delete(state->manifest); state->manifest = NULL; @@ -1817,15 +1913,22 @@ static int send_files_finalize(const Config* config, SendFilesState* state) { } } } + /* Output parity: report changed directories that had no transferred child + before the deferred directory times are applied (so the probe still sees + their pre-transfer state). */ + client_change_emit_pending_dirs(config, client->file_descriptor); /* P7 Wave D: every directory has now been traversed (or the scan stopped early), so transmit the captured directory times last. The receiver defers applying them until after its own deletion/publication phase. */ if (!send_dir_times(client, config, state->dir_entries)) return 1; bool delete_limit = false; + bool partial = false; ReceiverStats recv_stats; memset(&recv_stats, 0, sizeof(recv_stats)); - bool ok = finalize_transfer(client, config, state->remove_sources, &delete_limit, &recv_stats); + client_flush_client_messages(client->file_descriptor); + bool ok = finalize_transfer(client, config, state->remove_sources, &delete_limit, &partial, + &recv_stats); if (!ok && config->use_delete) log_message(LOG_LEVEL_ERROR, "server reported a deletion failure (--delete); see the server log for the reason"); @@ -1843,12 +1946,12 @@ static int send_files_finalize(const Config* config, SendFilesState* state) { (double)state->transfer_stats.transferred_file_size / (double)BYTES_PER_MIB); /* A skipped source entry (--ignore-errors past an unreadable directory, or a dereferenced symlink with no referent) makes rsync report a partial - transfer (exit 23) even though the rest of the run succeeded. A - --max-delete-capped commit is a successful transfer that rsync reports + transfer (exit 23), as does a receiver per-entry failure (STATUS_PARTIAL). + A --max-delete-capped commit is a successful transfer that rsync reports with exit code 25. */ if (!ok) return 1; - if (state->had_scan_io) + if (state->had_scan_io || partial) return 23; return delete_limit ? 25 : 0; } @@ -1872,6 +1975,8 @@ static void send_files_cleanup(SendFilesState* state) { array_list_delete(state->plan_dirs); if (state->missing_args) array_list_delete(state->missing_args); + if (state->per_dir_rules) + filter_rule_list_free(state->per_dir_rules); if (state->remove_sources) array_list_delete(state->remove_sources); if (state->dir_entries) @@ -1885,7 +1990,18 @@ static void send_files_cleanup(SendFilesState* state) { client_set_abort_armed(false); } +static int send_files_impl(Config* config); + int send_files(Config* config) { + /* Install the --stderr=client sink for the whole run (it only queues while a + session is live) and release its queue on every return path. */ + client_messages_install(); + int rc = send_files_impl(config); + client_messages_end(); + return rc; +} + +static int send_files_impl(Config* config) { if (config->list_only) return send_list_only(config); if (config->dry_run) @@ -1931,6 +2047,7 @@ int send_files(Config* config) { protocol_session_set_io_timeout(&session, config->timeout); protocol_session_set_ssl(&session, (SSL*)client->ssl); protocol_session_bind(&session); + client_messages_activate(true); int ret = 1; if (!send_files_prepare(config, &state)) @@ -1946,7 +2063,16 @@ send_fail: return ret; } +static int send_files_multithreaded_impl(Config* config); + int send_files_multithreaded(Config* config) { + client_messages_install(); + int rc = send_files_multithreaded_impl(config); + client_messages_end(); + return rc; +} + +static int send_files_multithreaded_impl(Config* config) { if (!config) return 1; if (config->list_only) @@ -2032,7 +2158,8 @@ int send_files_multithreaded(Config* config) { confined; only a --files-from subset records concrete directories. */ context->size_skipped_paths = array_list_create(free); context->synced_dirs = array_list_create(free); - if (!context->size_skipped_paths || !context->synced_dirs) { + context->per_dir_rules = filter_rule_list_create(); + if (!context->size_skipped_paths || !context->synced_dirs || !context->per_dir_rules) { pipeline_context_sender_destroy(context); return 1; } @@ -2061,6 +2188,7 @@ int send_files_multithreaded(Config* config) { if (context->excluded_paths) prepared.options.excluded_paths = context->excluded_paths; prepared.options.size_skipped_paths = context->size_skipped_paths; + prepared.options.per_dir_rules = context->per_dir_rules; /* The root marker for a full recursive transfer is already in the list; only a --files-from subset needs the scanner to record directories. */ if (config->files_from_set != NULL) @@ -2101,7 +2229,8 @@ int send_files_multithreaded(Config* config) { : (walk_root ? context->synced_dirs : NULL); delete_plan_sender_finalize(context->delete_plans, scope, walk_root); delete_plan_sender_set_config(context->delete_plans, context->excluded_paths, - context->size_skipped_paths, context->missing_args); + context->size_skipped_paths, context->missing_args, + context->per_dir_rules); } bool empty = per_dir ? (context->delete_plans && delete_plan_sender_empty(context->delete_plans)) @@ -2202,16 +2331,18 @@ int send_files_multithreaded(Config* config) { mtx_unlock(&context->mutex_scanner); bool sender_ok = sender_result == thrd_success; bool delete_limit = context->delete_limit; + bool partial = context->partial; /* A skipped source entry (--ignore-errors past an unreadable directory, or a dereferenced symlink with no referent) makes rsync report a partial - transfer (exit 23). A --max-delete-capped commit is a successful transfer - that rsync reports with exit code 25. */ + transfer (exit 23), as does a receiver per-entry failure (STATUS_PARTIAL). + A --max-delete-capped commit is a successful transfer that rsync reports + with exit code 25. */ pipeline_context_sender_destroy(context); client_progress_cleanup(); client_set_abort_armed(false); if (!sender_ok) return 1; - if (scan_io) + if (scan_io || partial) return 23; return delete_limit ? 25 : 0; } diff --git a/src/client/client_send_internal.h b/src/client/client_send_internal.h index 7ce71bc..4197ce2 100644 --- a/src/client/client_send_internal.h +++ b/src/client/client_send_internal.h @@ -68,10 +68,25 @@ void client_progress_name(const Config* config, const File* file); /* Emit a transferred entry's ancestor directories (as -i/--out-format change * lines or --progress name lines) before the entry's own line. */ void client_change_emit_ancestors(const Config* config, const File* file); +/* Output parity (protocol 2.30.0): probe each not-yet-known ancestor directory's + * pre-transfer destination state before the entry that first triggers it is + * sent. Returns false on a protocol/transport error. */ +bool client_change_probe_ancestors(const Config* config, const File* file, int fd); +/* Mark a transferred directory entry as already reported, and flush the + * itemize lines for changed directories that had no transferred child. */ +void client_change_mark_dir(const Config* config, const File* file); +void client_change_emit_pending_dirs(const Config* config, int fd); void client_progress_uptodate(const Config* config, const File* file); void client_progress_prepare(const Config* config, const ArrayList* plan_dirs, unsigned long long plan_non_dir_count); bool receive_stats_record(int fd, ReceiverStats* stats, ArrayList* would_delete); +/* --stderr=client diagnostic channel (client_report.c): install the queueing + * log sink for a transfer, mark the session live, flush queued diagnostics over + * the wire at a frame boundary, and tear the sink down. */ +void client_messages_install(void); +void client_messages_activate(bool active); +void client_flush_client_messages(int fd); +void client_messages_end(void); /* client_send.c */ void receive_daemon_motd(Client* client, const Config* config); @@ -87,9 +102,10 @@ int send_dry_run_manifest(const Config* config); int send_list_only(const Config* config); int send_dry_run_remote(Config* config); int send_delete_manifest(int fd, ArrayList* manifest, ArrayList* protected_prefixes, - ArrayList* size_skipped, ArrayList* missing_args, ArrayList* synced_dirs); + ArrayList* size_skipped, ArrayList* missing_args, ArrayList* synced_dirs, + const FilterRuleList* per_dir_rules); bool send_delete_manifest_early(Client* client, ArrayList* manifest, ArrayList* protected_prefixes, ArrayList* size_skipped, ArrayList* missing_args, - ArrayList* synced_dirs); + ArrayList* synced_dirs, const FilterRuleList* per_dir_rules); #endif diff --git a/src/client/scanner.h b/src/client/scanner.h index a9fa3d0..88ff79d 100644 --- a/src/client/scanner.h +++ b/src/client/scanner.h @@ -115,6 +115,13 @@ typedef struct { * directories, exactly like rsync; the receive root is the "." sentinel. * Guarded by `excluded_mutex`. */ ArrayList* synced_dirs; + /* Per-directory filter-rule sink (optional): when non-NULL the scanner appends + * a deep copy of every rule it reads from a per-directory merge file, each + * carrying its owner directory and no-inherit flag (see filter.h). The delete + * carriers transmit them so the receiver re-derives the per-directory + * protect/risk set for destination-only entries. Guarded by `excluded_mutex` + * like the other sinks. */ + FilterRuleList* per_dir_rules; /* Delete-plan directory sink (optional): when non-NULL the scanner appends * the destination-relative path of every directory it traverses (except the * receive root). The per-directory --delete-during/--delete-delay plan diff --git a/src/client/scanner_filter.c b/src/client/scanner_filter.c index 374aa45..d4b0007 100644 --- a/src/client/scanner_filter.c +++ b/src/client/scanner_filter.c @@ -443,17 +443,16 @@ void scanner_record_size_skipped(DirectoryScanner* scanner, const char* fs_path) scanner_record_protected(scanner, fs_path, scanner->options.size_skipped_paths); } -/* Record a directory the scan synchronized. `fs_path` is its absolute path and - `rel` its path relative to the transfer root ("" for the root); the stored - form matches the wire layout (the bare relative path in -R+--files-from, else - the source path with a leading '/' removed, with "." for the receive root). - Returns false on allocation failure. */ -bool scanner_record_synced_dir(const ScannerOptions* options, const char* fs_path, const char* rel, - bool relative_mode) { - if (!options->synced_dirs && !options->plan_dirs) - return true; - if (!file_list_dir_in_scope(options->file_list, rel)) - return true; +/* The destination-relative coordinate the receiver's delete walkers match + against for an entry at `fs_path` (with `rel` its path relative to the + transfer root, "" for the root): `relative_prefix + rel` under -R+--relative, + the bare relative path under -R+--files-from, else the source path with a + leading '/' removed, with "." for the receive root. Shared by the + synchronized-directory sink and the mirrored per-directory rule owners so + both live in the same coordinate system. Returns an owned string, or NULL on + allocation failure. */ +char* scanner_dest_rel_path(const ScannerOptions* options, const char* fs_path, const char* rel, + bool relative_mode) { char* prefixed = NULL; const char* dest; if (relative_mode) { @@ -461,7 +460,7 @@ bool scanner_record_synced_dir(const ScannerOptions* options, const char* fs_pat } else if (options->relative_prefix) { prefixed = scanner_prefix_send_path(options->relative_prefix, rel); if (!prefixed) - return false; + return NULL; dest = prefixed; } else { dest = fs_path; @@ -470,6 +469,24 @@ bool scanner_record_synced_dir(const ScannerOptions* options, const char* fs_pat dest++; if (dest[0] == '\0') dest = "."; + char* out = str_dup(dest); + free(prefixed); + return out; +} + +/* Record a directory the scan synchronized. `fs_path` is its absolute path and + `rel` its path relative to the transfer root ("" for the root); the stored + form matches the wire layout (see scanner_dest_rel_path). Returns false on + allocation failure. */ +bool scanner_record_synced_dir(const ScannerOptions* options, const char* fs_path, const char* rel, + bool relative_mode) { + if (!options->synced_dirs && !options->plan_dirs) + return true; + if (!file_list_dir_in_scope(options->file_list, rel)) + return true; + char* dest = scanner_dest_rel_path(options, fs_path, rel, relative_mode); + if (!dest) + return false; bool ok = true; if (options->synced_dirs) ok = excluded_sink_append(options->synced_dirs, options->excluded_mutex, dest); @@ -478,7 +495,7 @@ bool scanner_record_synced_dir(const ScannerOptions* options, const char* fs_pat than deleted as an extra; the receive root (".") is implicit. */ if (ok && options->plan_dirs && strcmp(dest, ".") != 0) ok = excluded_sink_append(options->plan_dirs, options->excluded_mutex, dest); - free(prefixed); + free(dest); return ok; } @@ -487,7 +504,8 @@ bool scanner_record_synced_dir(const ScannerOptions* options, const char* fs_pat * fresh list. Returns NULL on allocation/parse failure (message in `err`); * returns an empty list (and *any_exists=false) when no file exists. */ FilterRuleList* read_dir_filters(const ScannerOptions* options, const char* dir_path, - const char* rel, bool* any_exists, char* err, size_t err_size) { + const char* rel, bool relative_mode, bool* any_exists, char* err, + size_t err_size) { if (err && err_size > 0) err[0] = '\0'; const FilterRuleList* base = options->base_filters; @@ -511,13 +529,40 @@ FilterRuleList* read_dir_filters(const ScannerOptions* options, const char* dir_ } if (base) { for (int i = 0; i < base->dir_merge_count; i++) { - if (!filter_file_append(own, dir_path, base->dir_merge_names[i], rel, &opts, &exists, err, - err_size)) + if (!filter_dir_merge_append(own, dir_path, &base->dir_merges[i], rel, &opts, &exists, err, + err_size)) goto fail; if (exists && any_exists) *any_exists = true; } } + /* Mirror the directory's rules into the delete-carrier sink so the receiver + * can reconstruct its per-directory protect/risk set. The mirrored rules + * carry the destination-relative owner coordinate (not the transfer-root- + * relative one the sender's own evaluation uses) so the receiver's delete + * walkers, which match against receive-root-relative paths, find them. */ + if (options->per_dir_rules && own->count > 0) { + char* owner = scanner_dest_rel_path(options, dir_path, rel, relative_mode); + if (!owner) + goto fail; + mtx_t* mtx = options->excluded_mutex; + if (mtx) + mtx_lock(mtx); + for (int i = 0; i < own->count; i++) { + FilterRule* copy = filter_rule_clone(own->items[i]); + if (!copy || !filter_rule_set_owner(copy, owner) || + !filter_rule_list_add(options->per_dir_rules, copy)) { + filter_rule_free(copy); + if (mtx) + mtx_unlock(mtx); + free(owner); + goto fail; + } + } + if (mtx) + mtx_unlock(mtx); + free(owner); + } return own; fail: filter_rule_list_free(own); @@ -534,7 +579,7 @@ int open_directory_filter_context(DirectoryScanner* scanner, const FilterNode* i bool any_exists = false; FilterRuleList* own = read_dir_filters(&scanner->options, scanner->current_path, scanner->current_rel ? scanner->current_rel : "", - &any_exists, err, sizeof(err)); + scanner->relative_mode, &any_exists, err, sizeof(err)); if (!own) { /* read_dir_filters() leaves `err` set on a parse/allocation failure even when an earlier merge file in the same directory existed (any_exists true); diff --git a/src/client/scanner_internal.h b/src/client/scanner_internal.h index c772d9d..d32bd40 100644 --- a/src/client/scanner_internal.h +++ b/src/client/scanner_internal.h @@ -73,6 +73,8 @@ File* scanner_build_dir_file(const char* path, const struct stat* stats, const ScannerOptions* options); char* child_rel_path(const char* parent_rel, const char* name); char* scanner_prefix_send_path(const char* prefix, const char* rel); +char* scanner_dest_rel_path(const ScannerOptions* options, const char* fs_path, const char* rel, + bool relative_mode); bool entry_passes_selection(const FileListSet* file_list, const FilterRuleList* base, const FilterNode* node, const char* rel, const char* leaf, bool is_dir, bool per_dir_filters, bool exclude_filter_files, bool* protect_out); @@ -92,7 +94,8 @@ void scanner_record_size_skipped(DirectoryScanner* scanner, const char* fs_path) bool scanner_record_synced_dir(const ScannerOptions* options, const char* fs_path, const char* rel, bool relative_mode); FilterRuleList* read_dir_filters(const ScannerOptions* options, const char* dir_path, - const char* rel, bool* any_exists, char* err, size_t err_size); + const char* rel, bool relative_mode, bool* any_exists, char* err, + size_t err_size); int open_directory_filter_context(DirectoryScanner* scanner, const FilterNode* inherited); int scanner_inspect_entry(const ScannerOptions* options, const char* containing_dir, const char* link_rel, const char* name, ScannerEntry* entry); diff --git a/src/client/scanner_parallel.c b/src/client/scanner_parallel.c index f82eccb..40e4330 100644 --- a/src/client/scanner_parallel.c +++ b/src/client/scanner_parallel.c @@ -589,8 +589,9 @@ ParallelScanner* parallel_scanner_create_with_options(const char* root_directory { char err[256]; bool any_exists = false; - FilterRuleList* own = - read_dir_filters(options, root_directory, "", &any_exists, err, sizeof(err)); + FilterRuleList* own = read_dir_filters(options, root_directory, "", + options->relative && options->file_list != NULL, + &any_exists, err, sizeof(err)); if (!own) { /* A parse/allocation failure must fail the scan even when an earlier merge file in the same directory existed (see the sequential scanner). */ diff --git a/src/client/usage.c b/src/client/usage.c index 328df37..71af5a9 100644 --- a/src/client/usage.c +++ b/src/client/usage.c @@ -297,11 +297,13 @@ void print_usage(void) { printf(" --max-depth Maximum directory depth (0=unlimited)\n"); printf(" -x, --one-file-system Do not cross filesystem boundaries\n"); printf(" --log-file , --log-file= Write log messages to file\n"); - printf(" --stderr=MODE Route logging to stderr: errors or all\n"); + printf(" --stderr=MODE Route logging: errors (default), all, or client\n"); + printf(" (forward the client's diagnostics to the server's\n"); + printf(" stderr)\n"); printf(" --msgs2stderr Route all messages to stderr (deprecated spelling of\n"); printf(" --stderr=all)\n"); - printf(" --no-msgs2stderr Select errors-only stderr (deprecated spelling; the\n"); - printf(" default)\n"); + printf(" --no-msgs2stderr Forward the client's diagnostics to the server\n"); + printf(" (deprecated spelling of --stderr=client)\n"); printf(" --partial Keep partial files on interrupted transfer\n"); printf(" --partial-dir Directory for partial files (implies --partial)\n"); printf(" -T, --temp-dir Scratch dir for temp files before atomic install.\n"); diff --git a/src/server/receiver.c b/src/server/receiver.c index 765501d..25a183b 100644 --- a/src/server/receiver.c +++ b/src/server/receiver.c @@ -11,10 +11,13 @@ #include "metadata.h" #include "protocol.h" #include "utils.h" +#include +#include #include #include #include #include +#include bool receiver_outcomes_append(ReceiverOutcomes* outcomes, unsigned char code) { if (!outcomes) @@ -101,14 +104,35 @@ static void receiver_tally_deleted(const ReceiverSink* sink, size_t deleted) { sink->stats->deleted_files += deleted; } -/* Observer for --info=del: record each truly-removed destination-relative path - in the ArrayList passed as the observer context, so the terminal STATUS_STATS - frame can list it. A failed append is best-effort (the deletion already - happened; output is cosmetic). Shared by the single-threaded receiver and - the -m pipeline's deferred commit. */ -void receiver_record_deleted_path(void* context, const char* rel_path) { - ArrayList* paths = context; - if (!paths || !rel_path) +/* Observer for --info=del/--stats: record each truly-removed destination- + relative path (when the context carries a path list) and tally it by type + (when it carries a stats record), so the terminal STATUS_STATS frame can list + the paths and render rsync's per-type `Number of deleted files` breakdown. A + failed append is best-effort (the deletion already happened; output is + cosmetic). Shared by the single-threaded receiver and the -m pipeline's + deferred commit. */ +void receiver_record_deleted_path(void* context, const char* rel_path, DeleteEntryType type) { + ReceiverDeleteContext* del = context; + if (!del || !rel_path) + return; + if (del->stats) { + switch (type) { + case DELETE_ENTRY_DIR: + del->stats->deleted_dir++; + break; + case DELETE_ENTRY_LINK: + del->stats->deleted_link++; + break; + case DELETE_ENTRY_SPECIAL: + del->stats->deleted_special++; + break; + default: + del->stats->deleted_reg++; + break; + } + } + ArrayList* paths = del->deleted_paths; + if (!paths) return; /* Bound the retained list like the keep-set manifest: only MAX_MANIFEST_ENTRIES paths are ever transmitted in the terminal STATUS_STATS frame, so recording @@ -120,6 +144,16 @@ void receiver_record_deleted_path(void* context, const char* rel_path) { free(copy); } +/* Install the delete observer (and its context) for one commit when the sink + carries a stats record or a path list. Returns NULL when neither is needed, + so the delete engines skip the observer entirely. */ +static DeletePathObserver receiver_delete_observer(const ReceiverSink* sink, + ReceiverDeleteContext* del) { + del->stats = sink ? sink->stats : NULL; + del->deleted_paths = sink ? sink->deleted_paths : NULL; + return (del->stats || del->deleted_paths) ? receiver_record_deleted_path : NULL; +} + static bool receiver_process_chunk(Chunk* chunk, const ReceiverSink* sink) { if (!chunk || !sink || !sink->store_file) return false; @@ -277,6 +311,7 @@ static bool status_counts_as_progress(Status status) { case STATUS_ABORT: case STATUS_CHECK_BATCH: case STATUS_DIR_TIMES: + case STATUS_CLIENT_MSG: return false; default: return true; @@ -306,7 +341,13 @@ static bool receiver_note_status(const struct timespec* session_start, } int receiver_process(Config* config, int file_descriptor, const ReceiverSink* sink) { - return receiver_process_pending(config, file_descriptor, sink, NULL, NULL); + return receiver_process_pending_ctx(config, file_descriptor, sink, NULL, NULL, NULL); +} + +int receiver_process_pending(Config* config, int file_descriptor, const ReceiverSink* sink, + DeleteManifest** pending_manifest, DeletePlanSession** pending_plans) { + return receiver_process_pending_ctx(config, file_descriptor, sink, pending_manifest, + pending_plans, NULL); } /* Per-connection state threaded through the status handlers below. The parked @@ -327,6 +368,11 @@ typedef struct { a successful FINISHED it is either committed here or handed to *pending_plans so the -m caller commits after its disk writer drained. */ DeletePlanSession* plan_session; + /* Observer context for the per-directory delete session, which outlives the + frame handler; must stay alive until the session commits. For a session + handed to the caller (pending_plans) this points at a caller-owned + long-lived context; otherwise it points at an internal stack context. */ + ReceiverDeleteContext* delete_ctx; bool early_delete; bool per_dir_delete; bool delete_limit_noted; @@ -347,6 +393,23 @@ static ReceiverStep receiver_handle_keepalive(ReceiverPendingState* state) { return RECEIVER_STEP_NEXT; } +/* rsync --stderr=client: a client diagnostic forwarded over the wire. Read the + * bounded string and log it through the normal destination/level gate. The + * body is peer-controlled text: log_client_message() escapes every + * non-printable byte (newlines, CR, ANSI ESC, ...) before writing, so a hostile + * client cannot forge log lines or inject terminal control sequences. A + * malformed string (over-long or embedded NUL) is a framing error and tears the + * connection down. */ +static ReceiverStep receiver_handle_client_msg(ReceiverPendingState* state) { + char* message = receive_str(state->fd); + if (!message) + return RECEIVER_STEP_FAIL; + if (message[0] != '\0') + log_client_message(message); + free(message); + return RECEIVER_STEP_NEXT; +} + static ReceiverStep receiver_handle_abort(ReceiverPendingState* state) { (void)state; log_message(LOG_LEVEL_INFO, "Received abort from client, cleaning up"); @@ -383,9 +446,95 @@ static ReceiverStep receiver_handle_check_batch(ReceiverPendingState* state) { return RECEIVER_STEP_NEXT; } +/* Probe a destination entry's pre-transfer state for the output-parity + dest-info report (protocol 2.30.0). `wire_path` is the destination-relative + path; `incoming_target` is non-NULL only for a symlink probe, in which case + the on-disk link target is compared with the target the receiver is about to + store (after --munge-links). The final component is never followed and the + parent walk is confined below the receive root. Returns false only on an + allocation/secure-walk failure; a missing entry is reported as existed=false. */ +static bool receiver_probe_dest_state(const Config* config, const char* wire_path, + const char* incoming_target, OutputDestState* out) { + memset(out, 0, sizeof(*out)); + out->known = true; + if (!config || !wire_path || wire_path[0] == '\0') + return false; + char* full = path_cat(config->receive_root_directory, wire_path); + if (!full) + return false; + char* leaf = NULL; + int parent_fd = file_open_secure_parent(full, &leaf, false); + free(full); + if (parent_fd < 0) { + /* A missing/unreachable parent means the entry cannot exist yet. */ + free(leaf); + return true; + } + struct stat st; + if (fstatat(parent_fd, leaf, &st, AT_SYMLINK_NOFOLLOW) == 0) { + out->existed = true; + out->size = (unsigned long long)st.st_size; + out->mtime_sec = (long long)st.st_mtime; +#ifdef __linux__ + out->mtime_nsec = st.st_mtim.tv_nsec; +#endif + out->mode = (uint32_t)st.st_mode; + out->uid = (int32_t)st.st_uid; + out->gid = (int32_t)st.st_gid; + if (incoming_target && S_ISLNK(st.st_mode)) { + char target_buf[PATH_MAX]; + ssize_t n = readlinkat(parent_fd, leaf, target_buf, sizeof(target_buf) - 1); + if (n >= 0) { + target_buf[n] = '\0'; + char* expected = + config->munge_links ? file_symlink_munge(incoming_target) : str_dup(incoming_target); + if (expected) { + out->target_matches = strcmp(target_buf, expected) == 0; + free(expected); + } + } + } + } + close(parent_fd); + free(leaf); + return true; +} + static ReceiverStep receiver_handle_mkdir(ReceiverPendingState* state) { - File* dir = file_receive_directory(state->fd, state->config); - if (!dir || !state->sink->store_file(dir, state->sink->context)) + const Config* config = state->config; + int fd = state->fd; + if (config->report_dest_info) { + int probe = 0; + if (!receive_int(fd, &probe) || (probe != 0 && probe != 1)) + return RECEIVER_STEP_FAIL; + if (probe) { + /* Probe-only frame: report the destination state and create nothing. */ + char* path = receive_wire_str(fd); + if (!path || path[0] == '\0' || (!file_get_trust_sender() && has_path_traversal(path))) { + free(path); + send_status(fd, STATUS_ERROR); + return RECEIVER_STEP_FAIL; + } + OutputDestState info; + bool ok = receiver_probe_dest_state(config, path, NULL, &info); + free(path); + if (!ok || !send_status(fd, STATUS_DEST_INFO) || !format_dest_state_send(fd, &info)) + return RECEIVER_STEP_FAIL; + return RECEIVER_STEP_NEXT; + } + } + File* dir = file_receive_directory(fd, config); + if (!dir) + return RECEIVER_STEP_ERROR; + if (config->report_dest_info) { + OutputDestState info; + bool ok = receiver_probe_dest_state(config, file_wire_path(dir), NULL, &info); + if (!ok || !send_status(fd, STATUS_DEST_INFO) || !format_dest_state_send(fd, &info)) { + file_destroy(dir); + return RECEIVER_STEP_FAIL; + } + } + if (!state->sink->store_file(dir, state->sink->context)) return RECEIVER_STEP_ERROR; return RECEIVER_STEP_NEXT; } @@ -404,8 +553,20 @@ static ReceiverStep receiver_handle_hardlink(ReceiverPendingState* state) { } static ReceiverStep receiver_handle_symlink(ReceiverPendingState* state) { - File* sym = file_receive_symlink(state->fd, state->config); - if (!sym || !state->sink->store_file(sym, state->sink->context)) + const Config* config = state->config; + File* sym = file_receive_symlink(state->fd, config); + if (!sym) + return RECEIVER_STEP_ERROR; + if (config->report_dest_info) { + OutputDestState info; + bool ok = receiver_probe_dest_state(config, file_wire_path(sym), sym->symlink_target, &info); + if (!ok || !send_status(state->fd, STATUS_DEST_INFO) || + !format_dest_state_send(state->fd, &info)) { + file_destroy(sym); + return RECEIVER_STEP_FAIL; + } + } + if (!state->sink->store_file(sym, state->sink->context)) return RECEIVER_STEP_ERROR; return RECEIVER_STEP_NEXT; } @@ -448,12 +609,11 @@ static ReceiverStep receiver_handle_manifest(ReceiverPendingState* state) { --max-delete-capped commit still succeeds and the transfer proceeds; the terminal success frame reports the cap. */ size_t deleted = 0; - DeletePathObserver observer = - (config->report_deletes && sink->deleted_paths) ? receiver_record_deleted_path : NULL; + ReceiverDeleteContext delctx; + DeletePathObserver observer = receiver_delete_observer(sink, &delctx); DeleteCommitResult deletion = (config->use_delete || config->delete_missing_args) - ? manifest_delete_all_observed(config, manifest, &deleted, observer, - (void*)sink->deleted_paths) + ? manifest_delete_all_observed(config, manifest, &deleted, observer, &delctx) : DELETE_COMMIT_OK; receiver_tally_deleted(sink, deleted); delete_manifest_free(manifest); @@ -496,9 +656,9 @@ static ReceiverStep receiver_handle_delete_plan(ReceiverPendingState* state) { } if (!state->plan_session) { state->plan_session = delete_plan_session_create(config); - if (state->plan_session && config->report_deletes && sink->deleted_paths) + if (state->plan_session && (sink->stats || sink->deleted_paths)) delete_plan_session_set_delete_observer(state->plan_session, receiver_record_deleted_path, - (void*)sink->deleted_paths); + state->delete_ctx); } if (!state->plan_session || delete_plan_session_receive(state->plan_session, config, fd) != 0) return RECEIVER_STEP_FAIL; @@ -527,6 +687,8 @@ static ReceiverStep receiver_dispatch_status(ReceiverPendingState* state, Status switch (status) { case STATUS_KEEPALIVE: return receiver_handle_keepalive(state); + case STATUS_CLIENT_MSG: + return receiver_handle_client_msg(state); case STATUS_ABORT: return receiver_handle_abort(state); case STATUS_CHECK: @@ -579,8 +741,10 @@ static void receiver_drop_pending(ReceiverPendingState* state) { delete-during plan mode (no manifest at all). See the per-frame handlers above for how the -m receiver defers that commit until its disk writer has drained. */ -int receiver_process_pending(Config* config, int file_descriptor, const ReceiverSink* sink, - DeleteManifest** pending_manifest, DeletePlanSession** pending_plans) { +int receiver_process_pending_ctx(Config* config, int file_descriptor, const ReceiverSink* sink, + DeleteManifest** pending_manifest, + DeletePlanSession** pending_plans, + ReceiverDeleteContext* observer_ctx) { Status status; if (!receive_status(file_descriptor, &status)) return -1; @@ -593,6 +757,13 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver last_progress = session_start; if (!receiver_note_status(&session_start, &last_progress, status, file_descriptor, sink)) return -1; + /* A per-directory delete session handed to the caller outlives this stack + frame, so its observer context must be caller-owned (observer_ctx); only + the default inline-commit case may use the stack context. */ + ReceiverDeleteContext local_ctx; + ReceiverDeleteContext* delete_ctx = observer_ctx ? observer_ctx : &local_ctx; + delete_ctx->stats = sink ? sink->stats : NULL; + delete_ctx->deleted_paths = sink ? sink->deleted_paths : NULL; ReceiverPendingState state = { .config = config, .fd = file_descriptor, @@ -601,6 +772,7 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver .pending_plans = pending_plans, .deferred_manifest = NULL, .plan_session = NULL, + .delete_ctx = delete_ctx, .early_delete = config_delete_timing_early(config), .per_dir_delete = config_delete_timing_per_dir(config), .delete_limit_noted = false, @@ -610,7 +782,7 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver status == STATUS_KEEPALIVE || status == STATUS_ABORT || status == STATUS_CHECK_BATCH || status == STATUS_MKDIR || status == STATUS_MANIFEST || status == STATUS_HARDLINK || status == STATUS_SYMLINK || status == STATUS_SPECIAL || status == STATUS_DIR_TIMES || - status == STATUS_DELETE_PLAN) { + status == STATUS_DELETE_PLAN || status == STATUS_CLIENT_MSG) { ReceiverStep step = receiver_dispatch_status(&state, status); if (step == RECEIVER_STEP_FAIL) goto fail; @@ -640,10 +812,9 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver state.deferred_manifest = NULL; } else { size_t deleted = 0; - DeletePathObserver observer = - (config->report_deletes && sink->deleted_paths) ? receiver_record_deleted_path : NULL; + DeletePathObserver observer = receiver_delete_observer(sink, state.delete_ctx); DeleteCommitResult deletion = manifest_delete_all_observed( - config, state.deferred_manifest, &deleted, observer, (void*)sink->deleted_paths); + config, state.deferred_manifest, &deleted, observer, state.delete_ctx); receiver_tally_deleted(sink, deleted); delete_manifest_free(state.deferred_manifest); state.deferred_manifest = NULL; @@ -661,9 +832,9 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver hands the session to its caller instead, which commits after the disk writer drained. */ if (state.plan_session) { - if (config->report_deletes && sink->deleted_paths) + if (sink->stats || sink->deleted_paths) delete_plan_session_set_delete_observer(state.plan_session, receiver_record_deleted_path, - (void*)sink->deleted_paths); + state.delete_ctx); if (state.pending_plans) { *state.pending_plans = state.plan_session; state.plan_session = NULL; @@ -793,13 +964,14 @@ static void receiver_note_delete_limit(void* context_pointer) { /* Terminal status for a run. A capped --delete limit wins (rsync exit 25); otherwise any per-entry failure (for example an unprivileged --devices - mknod) makes the terminal frame non-OK so the client exits non-zero. rsync - reports 23 here; mapping the client's exact exit code to 23 is a separate, - pre-existing concern. A clean run keeps STATUS_OK. */ + mknod) makes the terminal frame STATUS_PARTIAL so the client exits 23 + (rsync's "partial transfer due to error") while still removing the sources + it successfully transferred under --remove-source-files. A fatal stream + error keeps STATUS_ERROR (a non-23 exit). A clean run keeps STATUS_OK. */ static Status receiver_final_status(bool delete_limit_reached, size_t failed_entries) { if (delete_limit_reached) return STATUS_DELETE_LIMIT; - return failed_entries > 0 ? STATUS_ERROR : STATUS_OK; + return failed_entries > 0 ? STATUS_PARTIAL : STATUS_OK; } static bool receiver_send_success_frame(int fd, void* context_pointer) { diff --git a/src/server/receiver.h b/src/server/receiver.h index 3b2dcff..1c6b341 100644 --- a/src/server/receiver.h +++ b/src/server/receiver.h @@ -55,10 +55,20 @@ typedef struct { bool receiver_outcomes_append(ReceiverOutcomes* outcomes, unsigned char code); void receiver_outcomes_destroy(ReceiverOutcomes* outcomes); -/* DeletePathObserver implementation for --info=del: `context` is an ArrayList* - that receives owned copies of every truly-removed destination-relative path. - Shared by the single-threaded receiver and the -m pipeline's deferred commit. */ -void receiver_record_deleted_path(void* context, const char* rel_path); +/* Delete observer context: `deleted_paths` (optional) receives owned copies of + every truly-removed destination-relative path for --info=del; `stats` + (optional) receives the per-type `Number of deleted files` tallies for + --stats. Both may be NULL, in which case the observer is a no-op. */ +typedef struct { + ReceiverStats* stats; + struct ArrayList* deleted_paths; +} ReceiverDeleteContext; + +/* DeletePathObserver implementation: records each truly-removed path (when the + context carries a path list) and tallies it by type (when it carries a stats + record). Shared by the single-threaded receiver and the -m pipeline's + deferred commit. */ +void receiver_record_deleted_path(void* context, const char* rel_path, DeleteEntryType type); /* Send the terminal success frame. `final_status` is usually STATUS_OK, or STATUS_DELETE_LIMIT when a --max-delete commit was capped. */ @@ -83,6 +93,17 @@ int receiver_process(Config* config, int file_descriptor, const ReceiverSink* si for either to keep the default behaviour (delete before the success frame). */ int receiver_process_pending(Config* config, int file_descriptor, const ReceiverSink* sink, DeleteManifest** pending_manifest, DeletePlanSession** pending_plans); +/* receiver_process_pending() with an explicit observer context for a + per-directory delete session that is handed to the caller via + `pending_plans`. The session outlives this call (the -m pipeline commits it + after joining its disk writer), so its observer context must too: pass a + long-lived object such as PipelineContextReceiver.delete_ctx. When + `delete_ctx` is NULL an internal stack context is used, which is only safe + when the session is committed before returning (the default behaviour). */ +int receiver_process_pending_ctx(Config* config, int file_descriptor, const ReceiverSink* sink, + DeleteManifest** pending_manifest, + DeletePlanSession** pending_plans, + ReceiverDeleteContext* delete_ctx); int receiver_receive_files(Config* config, int file_descriptor); /* ---- Connection time bounds (anti-slowloris) ---- diff --git a/src/server/receiver_pipeline.c b/src/server/receiver_pipeline.c index 6feab4e..b28ab1b 100644 --- a/src/server/receiver_pipeline.c +++ b/src/server/receiver_pipeline.c @@ -28,6 +28,8 @@ PipelineContextReceiver* pipeline_context_receiver_create(Config* config, Queue* context->max_queue_bytes = 0; context->deferred_manifest = NULL; context->deferred_plans = NULL; + context->delete_ctx.stats = NULL; + context->delete_ctx.deleted_paths = NULL; context->delete_limit_reached = false; context->failed_entries = 0; memset(&context->stats, 0, sizeof(context->stats)); @@ -205,8 +207,9 @@ int receive_thread(void* pipeline_context) { &context->stats, context->would_delete, context->deleted_paths}; - if (receiver_process_pending((Config*)config, file_descriptor, &sink, &context->deferred_manifest, - &context->deferred_plans) != 0) { + if (receiver_process_pending_ctx((Config*)config, file_descriptor, &sink, + &context->deferred_manifest, &context->deferred_plans, + &context->delete_ctx) != 0) { receiver_thread_fail(context); protocol_session_unbind(); return thrd_error; diff --git a/src/server/receiver_pipeline.h b/src/server/receiver_pipeline.h index 33e9f73..84d4142 100644 --- a/src/server/receiver_pipeline.h +++ b/src/server/receiver_pipeline.h @@ -46,6 +46,11 @@ typedef struct PipelineContextReceiver { committing while the disk writer may still be draining; server.c commits it after both threads joined. NULL for every other timing. */ DeletePlanSession* deferred_plans; + /* Observer context for `deferred_plans`. It must outlive the receive thread + (the session is committed by server.c after both threads join), so it lives + here rather than on receiver_process_pending()'s stack; receive_thread + installs it on the session. */ + ReceiverDeleteContext delete_ctx; /* Set by server.c when the deferred delete commit hit the --max-delete budget; the terminal success frame then carries STATUS_DELETE_LIMIT (rsync exit 25) while the transfer itself still succeeds. */ diff --git a/src/server/server.c b/src/server/server.c index f84a616..f374ecc 100644 --- a/src/server/server.c +++ b/src/server/server.c @@ -990,9 +990,11 @@ static void server_run_mt_receiver(ServerSession* state) { server-contacting --dry-run deletes nothing (no manifest is sent). */ if (context->deferred_manifest) { size_t deleted = 0; - DeletePathObserver observer = config->report_deletes ? receiver_record_deleted_path : NULL; - DeleteCommitResult deletion = manifest_delete_all_observed( - config, context->deferred_manifest, &deleted, observer, (void*)context->deleted_paths); + ReceiverDeleteContext delctx = {&context->stats, context->deleted_paths}; + DeletePathObserver observer = + (delctx.stats || delctx.deleted_paths) ? receiver_record_deleted_path : NULL; + DeleteCommitResult deletion = manifest_delete_all_observed(config, context->deferred_manifest, + &deleted, observer, &delctx); context->stats.deleted_files += deleted; if (deletion == DELETE_COMMIT_ERROR) { transfer_ok = false; @@ -1009,10 +1011,9 @@ static void server_run_mt_receiver(ServerSession* state) { --delete-during already applied its plans on the receive thread. */ if (context->deferred_plans) { /* Defence in depth (the enclosing block already excludes dry-run): a - -n run never commits a deletion. */ - if (config->report_deletes) - delete_plan_session_set_delete_observer( - context->deferred_plans, receiver_record_deleted_path, (void*)context->deleted_paths); + -n run never commits a deletion. The session's observer context was + installed by receive_thread from context->delete_ctx, which outlives + both threads, so no stack context is needed here. */ DeleteCommitResult deletion = config->dry_run ? DELETE_COMMIT_OK : delete_plan_session_commit(context->deferred_plans, config); @@ -1050,7 +1051,7 @@ static void server_run_mt_receiver(ServerSession* state) { context->failed_entries, context->failed_entries == 1 ? "y" : "ies"); Status final_status = context->delete_limit_reached ? STATUS_DELETE_LIMIT - : (context->failed_entries > 0 ? STATUS_ERROR : STATUS_OK); + : (context->failed_entries > 0 ? STATUS_PARTIAL : STATUS_OK); /* Emit the optional wire-stats record first (protocol 2.25.0), then the success/outcome frame, exactly like the single-threaded receiver. */ if (!receiver_send_stats_frame(state->fd, config, &context->stats, context->would_delete, diff --git a/src/shared/config.h b/src/shared/config.h index a7e5bf7..924a090 100644 --- a/src/shared/config.h +++ b/src/shared/config.h @@ -83,7 +83,7 @@ typedef struct { typedef enum SuperMode { SUPER_MODE_AUTO = 0, SUPER_MODE_ON = 1, SUPER_MODE_OFF = 2 } SuperMode; /* =========================================================================== - * Config wire-field table (single source of truth for protocol 2.29.0). + * Config wire-field table (single source of truth for protocol 2.30.0). * * Every field below crosses the wire. The table is the ONLY place a * serialized field is named: config.h expands CONFIG_WIRE_FIELDS() to declare @@ -1084,7 +1084,20 @@ typedef struct Config { * version must bump; the strict same-version handshake (config_receive rejects a * mismatched version before parsing anything else) keeps a 2.29 client and a * 2.28 server from ever reaching that state. */ -#define PROTOCOL_VERSION "2.29.0" +/* (11) Client-message channel + partial exit (protocol 2.30.0): the + * config-frame LAYOUT is unchanged (no new config field), but the frame stream + * gains two statuses. STATUS_CLIENT_MSG (client->server) carries a bounded, + * length-prefixed diagnostic string so a client running with --stderr=client + * (rsync's --no-msgs2stderr spelling) can forward its own diagnostics to the + * server's stderr. STATUS_PARTIAL (receiver->client) is the terminal status + * sent instead of STATUS_OK when a per-entry receiver failure (e.g. an + * unprivileged --devices mknod) did not abort the stream; the sender exits 23 + * (rsync's partial transfer) and still removes successfully transferred + * --remove-source-files sources. A 2.29 peer that does not know these status + * values would reject them as an unknown status and tear the connection down, + * so the protocol version must bump; the strict same-version handshake keeps a + * 2.30 client and a 2.29 server from ever reaching that state. */ +#define PROTOCOL_VERSION "2.30.0" #define DEFAULT_CHUNK_SIZE (10 * 1024 * 1024) /* Upper bound on total basis-dir entries (rsync caps --link-dest at 20). */ #define MAX_BASIS_DIRS 64 diff --git a/src/shared/delete.c b/src/shared/delete.c index a47cc3b..0388cd5 100644 --- a/src/shared/delete.c +++ b/src/shared/delete.c @@ -32,6 +32,30 @@ static bool keep_is_file(const PathIndex* index, const char* rel_path) { return path_index_contains(index, rel_path); } +/* rsync's receiver-side verdict for one candidate extra: the per-directory + * chain first (deepest directory before ancestors), then the command-line base + * rules. Either rule set may be absent. */ +FilterAction delete_protect_verdict(const DeleteProtectRules* protect, const char* rel_path, + const char* leaf, bool is_dir) { + if (!protect) + return FILTER_ACTION_NONE; + FilterAction action = filter_dir_rules_apply_side(protect->dir_rules, rel_path, leaf, is_dir); + if (action != FILTER_ACTION_NONE) + return action; + return filter_rules_apply_side(protect->base_rules, rel_path, leaf, is_dir, FILTER_SIDE_RECEIVER); +} + +/* Classify a removed entry from its st_mode for the per-type delete counters. */ +DeleteEntryType delete_entry_type_of_mode(mode_t mode) { + if (S_ISDIR(mode)) + return DELETE_ENTRY_DIR; + if (S_ISLNK(mode)) + return DELETE_ENTRY_LINK; + if (S_ISREG(mode)) + return DELETE_ENTRY_REG; + return DELETE_ENTRY_SPECIAL; +} + /* True when child_rel is, or lies below, a protected entry. A prefix "a" therefore protects "a" and "a/b/c" but not "ab". Entries with top_level_only set only protect DIRECT children of the receive root (at_root); nested @@ -126,6 +150,7 @@ bool delete_dir_entries_collect(int dirfd, DeleteDirEntry** out, size_t* count, break; } entries[used].is_dir = S_ISDIR(st.st_mode); + entries[used].mode = st.st_mode; used++; } closedir(dir); @@ -197,7 +222,7 @@ typedef struct { static bool delete_walk_fd(int dirfd, const char* rel_path, const PathIndex* keep, const PathIndex* dirs, DeleteWalkState* state, const DeleteSkipEntry* skips, int skip_count, - const FilterRuleList* protect_rules, bool parent_deletable, + const DeleteProtectRules* protect, bool parent_deletable, bool* all_removed) { DeleteDirEntry* entries = NULL; size_t count = 0; @@ -248,9 +273,8 @@ static bool delete_walk_fd(int dirfd, const char* rel_path, const PathIndex* kee if (path_under_skip_prefix(child_rel, at_root, skips, skip_count)) { shielded[i] = true; local_survives = true; - } else if (protect_rules && - filter_rules_apply_side(protect_rules, child_rel, entries[i].name, entries[i].is_dir, - FILTER_SIDE_RECEIVER) == FILTER_ACTION_PROTECT) { + } else if (delete_protect_verdict(protect, child_rel, entries[i].name, entries[i].is_dir) == + FILTER_ACTION_PROTECT) { /* A first-match protect rule shields the extra; for a directory the whole subtree is shielded (rsync prunes an excluded directory), so do not descend. */ @@ -281,7 +305,7 @@ static bool delete_walk_fd(int dirfd, const char* rel_path, const PathIndex* kee int childfd = openat(dirfd, entries[i].name, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC); bool child_all_removed = false; if (childfd >= 0) { - if (!delete_walk_fd(childfd, child_rel, keep, dirs, state, skips, skip_count, protect_rules, + if (!delete_walk_fd(childfd, child_rel, keep, dirs, state, skips, skip_count, protect, deletable, &child_all_removed)) operation_ok = false; close(childfd); @@ -329,10 +353,10 @@ static bool delete_walk_fd(int dirfd, const char* rel_path, const PathIndex* kee memcpy(with_slash, child_rel, len); with_slash[len] = '/'; with_slash[len + 1] = '\0'; - state->observer(state->observer_context, with_slash); + state->observer(state->observer_context, with_slash, DELETE_ENTRY_DIR); free(with_slash); } else { - state->observer(state->observer_context, child_rel); + state->observer(state->observer_context, child_rel, DELETE_ENTRY_DIR); } } } @@ -373,7 +397,8 @@ static bool delete_walk_fd(int dirfd, const char* rel_path, const PathIndex* kee char* child_rel = path_cat((char*)rel_path, entries[i].name); if (child_rel) { if (state->observer) - state->observer(state->observer_context, child_rel); + state->observer(state->observer_context, child_rel, + delete_entry_type_of_mode(entries[i].mode)); char* escaped_path = output_escape(child_rel, log_get_8_bit_output()); fprintf(stderr, " Deleted: %s\n", escaped_path ? escaped_path : ""); free(escaped_path); @@ -395,7 +420,7 @@ static bool delete_walk_fd(int dirfd, const char* rel_path, const PathIndex* kee int childfd = openat(dirfd, entries[i].name, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC); bool child_all_removed = false; if (childfd >= 0) { - if (!delete_walk_fd(childfd, child_rel, keep, dirs, state, skips, skip_count, protect_rules, + if (!delete_walk_fd(childfd, child_rel, keep, dirs, state, skips, skip_count, protect, deletable, &child_all_removed)) operation_ok = false; close(childfd); @@ -430,7 +455,7 @@ static int open_destination_root(const char* dest_root) { bool delete_extras_list(const char* dest_root, const ArrayList* manifest, const ArrayList* synced_dirs, const DeleteSkipEntry* skips, int skip_count, - const FilterRuleList* protect_rules, ArrayList* out, size_t* count_out) { + const DeleteProtectRules* protect, ArrayList* out, size_t* count_out) { if (count_out) *count_out = 0; if (!manifest || !out) @@ -461,7 +486,7 @@ bool delete_extras_list(const char* dest_root, const ArrayList* manifest, .observer = NULL, .observer_context = NULL}; bool ok = delete_walk_fd(rootfd, "", &keep, have_dirs ? &dirs : NULL, &state, skips, skip_count, - protect_rules, false, &all_removed); + protect, false, &all_removed); if (close(rootfd) != 0) ok = false; path_index_free(&keep); @@ -475,7 +500,7 @@ bool delete_extras_list(const char* dest_root, const ArrayList* manifest, DeleteWalkResult delete_extras_limited_observed(const char* dest_root, const ArrayList* manifest, const ArrayList* synced_dirs, size_t max_delete, const DeleteSkipEntry* skips, int skip_count, - const FilterRuleList* protect_rules, + const DeleteProtectRules* protect, size_t* deleted_out, size_t* skipped_out, DeletePathObserver observer, void* observer_context) { @@ -514,7 +539,7 @@ DeleteWalkResult delete_extras_limited_observed(const char* dest_root, const Arr .observer = observer, .observer_context = observer_context}; bool ok = delete_walk_fd(rootfd, "", &keep, have_dirs ? &dirs : NULL, &state, skips, skip_count, - protect_rules, false, &all_removed); + protect, false, &all_removed); if (close(rootfd) != 0) ok = false; path_index_free(&keep); @@ -532,11 +557,10 @@ DeleteWalkResult delete_extras_limited_observed(const char* dest_root, const Arr DeleteWalkResult delete_extras_limited(const char* dest_root, const ArrayList* manifest, const ArrayList* synced_dirs, size_t max_delete, const DeleteSkipEntry* skips, int skip_count, - const FilterRuleList* protect_rules, size_t* deleted_out, + const DeleteProtectRules* protect, size_t* deleted_out, size_t* skipped_out) { return delete_extras_limited_observed(dest_root, manifest, synced_dirs, max_delete, skips, - skip_count, protect_rules, deleted_out, skipped_out, NULL, - NULL); + skip_count, protect, deleted_out, skipped_out, NULL, NULL); } bool delete_extras(const char* dest_root, const ArrayList* manifest) { diff --git a/src/shared/delete.h b/src/shared/delete.h index f0921a1..47c8eea 100644 --- a/src/shared/delete.h +++ b/src/shared/delete.h @@ -3,8 +3,10 @@ #include "array_list.h" #include "config.h" +#include "filter.h" #include #include +#include /* Delete engine. * @@ -28,6 +30,24 @@ typedef enum { DELETE_WALK_ERROR } DeleteWalkResult; +/* Receiver-side delete-protection rules for one walk. `base_rules` is the + * command-line rule set the config frame carried (owner "" rules); `dir_rules` + * is the received per-directory rule set (rules carrying their owner directory + * and no-inherit flag). Either may be NULL. */ +typedef struct { + const FilterRuleList* base_rules; + const FilterRuleList* dir_rules; +} DeleteProtectRules; + +/* rsync's first-match-wins receiver verdict for one candidate extra: the + * per-directory chain is evaluated first (the containing directory's rules, + * then each ancestor's, then the receive root's), then the base rules. Returns + * FILTER_ACTION_PROTECT when the entry is shielded by a receiver-side exclude, + * FILTER_ACTION_RISK when an include explicitly leaves it at risk, or + * FILTER_ACTION_NONE when no rule matched. */ +FilterAction delete_protect_verdict(const DeleteProtectRules* protect, const char* rel_path, + const char* leaf, bool is_dir); + /* One protected entry for the delete walker. When top_level_only is true the prefix is skipped only as a DIRECT child of dest_root (the --delay-updates staging directory, which must not hide genuine extras inside a nested @@ -66,6 +86,9 @@ bool path_under_skip_prefix(const char* child_rel, bool at_root, const DeleteSki typedef struct { char* name; bool is_dir; + /* The entry's full st_mode from the AT_SYMLINK_NOFOLLOW stat, so a delete + observer can classify a removed non-directory as reg/link/special. */ + mode_t mode; } DeleteDirEntry; /* Collect the entries of the directory open on `dirfd` (excluding "." and ".."), stat'ing each with AT_SYMLINK_NOFOLLOW. On success *out is a malloc'd array of @@ -96,24 +119,38 @@ int delete_dir_entry_cmp_asc(const void* a, const void* b); DeleteWalkResult delete_extras_limited(const char* dest_root, const ArrayList* manifest, const ArrayList* synced_dirs, size_t max_delete, const DeleteSkipEntry* skips, int skip_count, - const FilterRuleList* protect_rules, size_t* deleted_out, + const DeleteProtectRules* protect, size_t* deleted_out, size_t* skipped_out); +/* Entry kind of a removed path, reported to the delete observer so the receiver + can build rsync's `--stats` `Number of deleted files` per-type breakdown. The + four categories are a strict partition of every removed entry. */ +typedef enum { + DELETE_ENTRY_REG = 0, + DELETE_ENTRY_DIR, + DELETE_ENTRY_LINK, + DELETE_ENTRY_SPECIAL +} DeleteEntryType; + /* Optional per-deletion observer: called for each destination-relative path - actually removed (a file, symlink, or directory), in removal order, so the - receiver can stream rsync's `--info=del`/`--info=remove` lines. */ -typedef void (*DeletePathObserver)(void* context, const char* rel_path); + actually removed (a file, symlink, or directory) with its entry kind, in + removal order, so the receiver can stream rsync's `--info=del`/`--info=remove` + lines and tally the per-type `--stats` counters. */ +typedef void (*DeletePathObserver)(void* context, const char* rel_path, DeleteEntryType type); + +/* Classify a removed entry from its st_mode for the per-type delete counters. */ +DeleteEntryType delete_entry_type_of_mode(mode_t mode); /* `delete_extras_limited_observed` is delete_extras_limited with an optional * observer; the observer is invoked only for entries truly removed. When - * `protect_rules` is non-NULL its receiver-side verdict is evaluated for every + * `protect` is non-NULL its receiver-side verdict is evaluated for every * candidate extra: a first-match PROTECT leaves the entry (and, for a * directory, its whole subtree) in place, while RISK/NONE fall through to the * ordinary skip-prefix/keep-set logic. */ DeleteWalkResult delete_extras_limited_observed(const char* dest_root, const ArrayList* manifest, const ArrayList* synced_dirs, size_t max_delete, const DeleteSkipEntry* skips, int skip_count, - const FilterRuleList* protect_rules, + const DeleteProtectRules* protect, size_t* deleted_out, size_t* skipped_out, DeletePathObserver observer, void* observer_context); @@ -124,7 +161,7 @@ DeleteWalkResult delete_extras_limited_observed(const char* dest_root, const Arr strings appended to `out` and receives their count in *count_out. */ bool delete_extras_list(const char* dest_root, const ArrayList* manifest, const ArrayList* synced_dirs, const DeleteSkipEntry* skips, int skip_count, - const FilterRuleList* protect_rules, ArrayList* out, size_t* count_out); + const DeleteProtectRules* protect, ArrayList* out, size_t* count_out); bool delete_extras(const char* dest_root, const ArrayList* manifest); /* Build the delete walk's skip-prefix set from the config's --delay-updates diff --git a/src/shared/delete_commit.c b/src/shared/delete_commit.c index 0a2b66f..d175e9c 100644 --- a/src/shared/delete_commit.c +++ b/src/shared/delete_commit.c @@ -19,6 +19,7 @@ #include "data.h" #include "delay_updates.h" #include "delete_commit.h" +#include "delete_plan.h" #include "delta.h" #include "file.h" #include "format.h" @@ -102,6 +103,13 @@ DeleteManifest* receive_manifest_entries(int fd) { delete_manifest_free(manifest); return NULL; } + /* Per-directory filter rules (protocol 2.30.0) follow the manifest sections + * with their own bounded self-describing format. */ + if (!delete_filter_dir_rules_receive(fd, &manifest->per_dir_rules)) { + delete_manifest_free(manifest); + send_status(fd, STATUS_ERROR); + return NULL; + } return manifest; } @@ -112,6 +120,7 @@ void delete_manifest_free(DeleteManifest* manifest) { array_list_delete(manifest->protected); array_list_delete(manifest->missing); array_list_delete(manifest->dirs); + filter_rule_list_free(manifest->per_dir_rules); free(manifest); } @@ -160,9 +169,11 @@ static bool delete_extras_budgeted_observed(const Config* config, const DeleteMa remaining = budget->max_delete - budget->deleted; size_t deleted = 0; size_t skipped = 0; + DeleteProtectRules protect = {.base_rules = config->protect_rules, + .dir_rules = manifest->per_dir_rules}; DeleteWalkResult result = delete_extras_limited_observed( config->receive_root_directory, manifest->keeps, manifest->dirs, remaining, skips.entries, - skips.count, config->protect_rules, &deleted, &skipped, observer, observer_context); + skips.count, &protect, &deleted, &skipped, observer, observer_context); delete_skips_free(&skips); budget->deleted += deleted; budget->skipped += skipped; @@ -191,13 +202,13 @@ typedef struct { const char* prefix; } PrefixedDeleteObserver; -static void prefixed_delete_observer(void* context, const char* rel) { +static void prefixed_delete_observer(void* context, const char* rel, DeleteEntryType type) { PrefixedDeleteObserver* prefixed = context; if (!prefixed->inner || !rel) return; char* joined = path_cat((char*)prefixed->prefix, rel); if (joined) { - prefixed->inner(prefixed->inner_context, joined); + prefixed->inner(prefixed->inner_context, joined, type); free(joined); } } @@ -359,7 +370,7 @@ static bool delete_missing_args_budgeted_observed(const Config* config, if (removed) { budget->deleted++; if (observer) - observer(observer_context, rel); + observer(observer_context, rel, delete_entry_type_of_mode(st.st_mode)); char* escaped = output_escape(rel, log_get_8_bit_output()); fprintf(stderr, " Deleted: %s\n", escaped ? escaped : ""); free(escaped); @@ -386,8 +397,10 @@ bool manifest_would_delete_list(const Config* config, const DeleteManifest* mani DeleteSkipSet skips; if (!delete_skips_build(config, manifest->protected, NULL, true, &skips)) return false; + DeleteProtectRules protect = {.base_rules = config->protect_rules, + .dir_rules = manifest->per_dir_rules}; bool ok = delete_extras_list(config->receive_root_directory, manifest->keeps, manifest->dirs, - skips.entries, skips.count, config->protect_rules, out, count_out); + skips.entries, skips.count, &protect, out, count_out); delete_skips_free(&skips); return ok; } diff --git a/src/shared/delete_commit.h b/src/shared/delete_commit.h index ba760f1..1602e4b 100644 --- a/src/shared/delete_commit.h +++ b/src/shared/delete_commit.h @@ -4,6 +4,7 @@ #include "array_list.h" #include "config.h" #include "delete.h" +#include "filter.h" #include /* Delete-commit module: delete-manifest receive plus the budgeted extras and @@ -30,6 +31,13 @@ typedef struct DeleteManifest { leave untransmitted directories and the unlisted parts of listed ones alone, matching rsync's "delete only in synchronized directories". */ ArrayList* dirs; + /* Per-directory filter rules the sender compiled while scanning (protocol + 2.30.0), each carrying its owner directory and no-inherit flag. The + receiver evaluates them (deepest before ancestors, then the command-line + base rules) against every candidate extra so a destination-only entry that + matches ONLY a per-directory `.rsync-filter`/dir-merge rule is shielded. + NULL when the sender transmitted none. */ + FilterRuleList* per_dir_rules; } DeleteManifest; void delete_manifest_free(DeleteManifest* manifest); diff --git a/src/shared/delete_plan.c b/src/shared/delete_plan.c index f69213b..91e3bb4 100644 --- a/src/shared/delete_plan.c +++ b/src/shared/delete_plan.c @@ -48,6 +48,7 @@ struct DeletePlanSender { const ArrayList* protected_prefixes; const ArrayList* size_skipped; const ArrayList* missing_args; + const FilterRuleList* per_dir_rules; size_t entries; /* Transmitted FILE entries only. The caller's "empty scan" safety guard keys off this (an I/O error that hid every file must refuse to delete even when @@ -284,12 +285,14 @@ bool delete_plan_sender_empty(const DeletePlanSender* sender) { } void delete_plan_sender_set_config(DeletePlanSender* sender, const ArrayList* protected_prefixes, - const ArrayList* size_skipped, const ArrayList* missing_args) { + const ArrayList* size_skipped, const ArrayList* missing_args, + const FilterRuleList* per_dir_rules) { if (!sender) return; sender->protected_prefixes = protected_prefixes; sender->size_skipped = size_skipped; sender->missing_args = missing_args; + sender->per_dir_rules = per_dir_rules; } /* True when `dir` is `root` itself or a descendant of it (path-component @@ -320,6 +323,181 @@ static int send_str_section(int fd, const ArrayList* list) { return 0; } +/* The directory a rule belongs to (its owner, or the transfer root for ""). */ +static const char* filter_dir_rule_owner(const FilterRule* rule) { + return (rule && rule->owner) ? rule->owner : ""; +} + +/* Transmit the received-side per-directory filter rules (protocol 2.30.0) as a + * self-describing list of directory groups: a group count, then for each group + * the relative owner directory followed by that directory's rule records (run + * order = the sender's traversal/rule order). Rules of one directory are + * appended to the sink contiguously, so runs reproduce the compilation order. + * Bounded by MAX_FILTER_RULES / MAX_FILTER_BYTES and MAX_PROTECT_PATTERN_LEN so + * the peer never sees a frame it would reject. The sender enforces exactly the + * receiver's limits (including the cumulative owner+pattern byte budget) and + * fails with a clear local error instead of emitting a frame that would abort + * the transfer with STATUS_ERROR. */ +bool delete_filter_dir_rules_send(int fd, const FilterRuleList* rules) { + int count = rules ? rules->count : 0; + if (count < 0 || count > MAX_FILTER_RULES) { + log_message(LOG_LEVEL_ERROR, "too many per-directory filter rules: %d (maximum %d)", count, + MAX_FILTER_RULES); + return false; + } + size_t bytes = 0; + for (int i = 0; i < count; i++) { + const FilterRule* rule = rules->items[i]; + const char* owner = filter_dir_rule_owner(rule); + size_t owner_len = strlen(owner); + size_t pattern_len = rule && rule->pattern ? strlen(rule->pattern) : 0; + if (!rule || !rule->pattern || pattern_len == 0) { + log_message(LOG_LEVEL_ERROR, "invalid per-directory filter pattern"); + return false; + } + if (pattern_len > MAX_PROTECT_PATTERN_LEN) { + log_message(LOG_LEVEL_ERROR, + "per-directory filter pattern exceeds %d bytes (use a shorter pattern)", + MAX_PROTECT_PATTERN_LEN); + return false; + } + if (!(owner_len == 0 || (owner[0] != '/' && !has_path_traversal(owner)))) { + log_message(LOG_LEVEL_ERROR, "invalid per-directory filter owner directory"); + return false; + } + if (owner_len + pattern_len > MAX_FILTER_BYTES - bytes) { + log_message(LOG_LEVEL_ERROR, "per-directory filter rules exceed %d bytes", MAX_FILTER_BYTES); + return false; + } + bytes += owner_len + pattern_len; + } + int groups = 0; + for (int i = 0; i < count;) { + const char* owner = filter_dir_rule_owner(rules->items[i]); + groups++; + i++; + while (i < count && strcmp(filter_dir_rule_owner(rules->items[i]), owner) == 0) + i++; + } + if (!send_int(fd, groups)) + return false; + for (int i = 0; i < count;) { + const char* owner = filter_dir_rule_owner(rules->items[i]); + int start = i; + i++; + while (i < count && strcmp(filter_dir_rule_owner(rules->items[i]), owner) == 0) + i++; + if (!send_wire_str(fd, owner) || !send_int(fd, i - start)) + return false; + for (int j = start; j < i; j++) { + const FilterRule* rule = rules->items[j]; + if (!send_int(fd, (int)rule->action) || !send_int(fd, (int)rule->sides) || + !send_int(fd, rule->anchored ? 1 : 0) || !send_int(fd, rule->dir_only ? 1 : 0) || + !send_int(fd, rule->negate ? 1 : 0) || !send_int(fd, rule->no_inherit ? 1 : 0) || + !send_wire_str(fd, rule->pattern)) + return false; + } + } + return true; +} + +/* Read one wire flag (an int restricted to 0/1). */ +static bool receive_flag(int fd, bool* value) { + int raw; + if (!receive_int(fd, &raw) || (raw != 0 && raw != 1)) + return false; + *value = raw != 0; + return true; +} + +/* Read the per-directory filter block emitted by delete_filter_dir_rules_send. + * Reconstructs a flat FilterRuleList whose rules carry their owner directory; + * `*out` is NULL when the sender transmitted no rules. Every bound is enforced + * (group/rule counts, owner/pattern bytes, pattern length, action/sides domain) + * so a malicious peer can neither overread nor allocate unboundedly. Returns + * false on a malformed frame (the caller signals STATUS_ERROR). */ +bool delete_filter_dir_rules_receive(int fd, FilterRuleList** out) { + if (!out) + return false; + *out = NULL; + int groups; + if (!receive_int(fd, &groups) || groups < 0 || groups > MAX_FILTER_RULES) + return false; + if (groups == 0) + return true; + FilterRuleList* list = filter_rule_list_create(); + if (!list) + return false; + int total_rules = 0; + size_t bytes = 0; + for (int g = 0; g < groups; g++) { + char* dir = receive_wire_str(fd); + if (!dir) + goto fail; + size_t dir_bytes = strlen(dir); + if (!(dir[0] == '\0' || (dir[0] != '/' && !has_path_traversal(dir))) || + dir_bytes > MAX_FILTER_BYTES - bytes) { + free(dir); + goto fail; + } + bytes += dir_bytes; + int rule_count; + if (!receive_int(fd, &rule_count) || rule_count < 0 || rule_count > MAX_FILTER_RULES || + rule_count > MAX_FILTER_RULES - total_rules) { + free(dir); + goto fail; + } + for (int r = 0; r < rule_count; r++) { + int action, sides; + bool anchored, dir_only, negate, no_inherit; + if (!receive_int(fd, &action) || + (action != FILTER_ACTION_EXCLUDE && action != FILTER_ACTION_INCLUDE) || + !receive_int(fd, &sides) || sides < (int)FILTER_SIDE_SENDER || + sides > (int)(FILTER_SIDE_SENDER | FILTER_SIDE_RECEIVER) || + !receive_flag(fd, &anchored) || !receive_flag(fd, &dir_only) || + !receive_flag(fd, &negate) || !receive_flag(fd, &no_inherit)) { + free(dir); + goto fail; + } + char* pattern = receive_wire_str(fd); + size_t pattern_bytes = pattern ? strlen(pattern) : 0; + if (!pattern || pattern_bytes == 0 || pattern_bytes > MAX_PROTECT_PATTERN_LEN || + pattern_bytes > MAX_FILTER_BYTES - bytes) { + free(pattern); + free(dir); + goto fail; + } + bytes += pattern_bytes; + FilterRule* rule = calloc(1, sizeof(FilterRule)); + if (!rule) { + free(pattern); + free(dir); + goto fail; + } + rule->action = (FilterAction)action; + rule->sides = (unsigned)sides; + rule->anchored = anchored; + rule->dir_only = dir_only; + rule->negate = negate; + rule->no_inherit = no_inherit; + rule->owner = str_dup(dir); + rule->pattern = pattern; + if (!rule->owner || !filter_rule_list_add(list, rule)) { + filter_rule_free(rule); + free(dir); + goto fail; + } + total_rules++; + } + free(dir); + } + *out = list; + return true; +fail: + filter_rule_list_free(list); + return false; +} + static int send_plan_node(int fd, DeletePlanSender* sender, PlanNode* node) { if (!send_status(fd, STATUS_DELETE_PLAN)) return -1; @@ -328,7 +506,8 @@ static int send_plan_node(int fd, DeletePlanSender* sender, PlanNode* node) { if (!sender->config_sent) { if (send_str_section(fd, sender->protected_prefixes) != 0 || send_str_section(fd, sender->size_skipped) != 0 || - send_str_section(fd, sender->missing_args) != 0) + send_str_section(fd, sender->missing_args) != 0 || + !delete_filter_dir_rules_send(fd, sender->per_dir_rules)) return -1; sender->config_sent = true; } @@ -355,7 +534,8 @@ static int send_config_only(int fd, DeletePlanSender* sender) { return -1; if (send_str_section(fd, sender->protected_prefixes) != 0 || send_str_section(fd, sender->size_skipped) != 0 || - send_str_section(fd, sender->missing_args) != 0) + send_str_section(fd, sender->missing_args) != 0 || + !delete_filter_dir_rules_send(fd, sender->per_dir_rules)) return -1; sender->config_sent = true; if (!send_int(fd, 0)) /* apply = false */ @@ -472,15 +652,19 @@ struct DeletePlanSession { ArrayList* protected_prefixes; ArrayList* size_skipped; ArrayList* missing; + /* Received per-directory filter rules (protocol 2.30.0), or NULL. Evaluated + deepest-directory-first for every candidate extra so a destination-only + entry matching only a per-directory rule is protected. */ + FilterRuleList* per_dir_rules; ArrayList* deferred; DeletePathObserver observer; void* observer_context; }; /* Report one path the session truly removed (no-op without an observer). */ -static void notify_deleted(DeletePlanSession* session, const char* rel) { +static void notify_deleted(DeletePlanSession* session, const char* rel, DeleteEntryType type) { if (session && session->observer && rel) - session->observer(session->observer_context, rel); + session->observer(session->observer_context, rel, type); } /* A removed directory is reported with rsync's trailing slash (`deleting dir/`) @@ -491,13 +675,13 @@ static void notify_deleted_dir(DeletePlanSession* session, const char* rel) { size_t len = strlen(rel); char* with_slash = malloc(len + 2); if (!with_slash) { - session->observer(session->observer_context, rel); + session->observer(session->observer_context, rel, DELETE_ENTRY_DIR); return; } memcpy(with_slash, rel, len); with_slash[len] = '/'; with_slash[len + 1] = '\0'; - session->observer(session->observer_context, with_slash); + session->observer(session->observer_context, with_slash, DELETE_ENTRY_DIR); free(with_slash); } @@ -532,6 +716,7 @@ void delete_plan_session_destroy(DeletePlanSession* session) { array_list_delete(session->protected_prefixes); array_list_delete(session->size_skipped); array_list_delete(session->missing); + filter_rule_list_free(session->per_dir_rules); array_list_delete(session->deferred); free(session); } @@ -604,16 +789,18 @@ static int open_plan_dir(const Config* config, const char* dir) { typedef struct { DeleteSkipSet set; - /* Receiver-side delete-protection rules received on the config frame (NULL - when the sender sent none). Evaluated per extra so a protect/risk rule is + /* Receiver-side delete-protection rules. `base` is the config-frame + command-line set and `dir` the received per-directory set (both NULL when + the sender sent none). Evaluated per extra so a protect/risk rule is honored under --delete-during/--delete-delay exactly like the whole-tree commit walker. */ - const FilterRuleList* protect_rules; + DeleteProtectRules protect; } PlanSkips; static bool build_plan_skips(const Config* config, const DeletePlanSession* session, PlanSkips* out) { - out->protect_rules = config->protect_rules; + out->protect.base_rules = config->protect_rules; + out->protect.dir_rules = session->per_dir_rules; /* The per-directory plan walk keeps each basis path verbatim (it does not convert an absolute under-root path to its root-relative form, unlike the whole-tree commit walk). */ @@ -711,8 +898,8 @@ static bool process_extra_dir(int dirfd, const char* name, const char* child_rel return errno == ENOTEMPTY || errno == EEXIST; } -static bool process_extra_file(int dirfd, const char* name, const char* child_rel, bool force_now, - DeletePlanSession* session) { +static bool process_extra_file(int dirfd, const char* name, const char* child_rel, mode_t mode, + bool force_now, DeletePlanSession* session) { if (session->defer && !force_now) { return defer_add(session, child_rel); } @@ -724,7 +911,7 @@ static bool process_extra_file(int dirfd, const char* name, const char* child_re session->deleted++; session->planned++; log_deleted(child_rel); - notify_deleted(session, child_rel); + notify_deleted(session, child_rel, delete_entry_type_of_mode(mode)); } else if (errno != ENOENT) { return false; } @@ -778,10 +965,8 @@ static bool process_children(int dirfd, const char* dir_rel, const ArrayList* ke bool is_dir = entries[i].is_dir; bool in_keep_dirs = is_dir && list_contains_str(keep_dirs, entries[i].name); bool in_keep_files = !is_dir && list_contains_str(keep_files, entries[i].name); - bool rule_protected = - skips->protect_rules && - filter_rules_apply_side(skips->protect_rules, child_rel, entries[i].name, is_dir, - FILTER_SIDE_RECEIVER) == FILTER_ACTION_PROTECT; + bool rule_protected = delete_protect_verdict(&skips->protect, child_rel, entries[i].name, + is_dir) == FILTER_ACTION_PROTECT; if (in_keep_dirs || in_keep_files || rule_protected) { shielded[i] = true; local_survives = true; @@ -828,7 +1013,8 @@ static bool process_children(int dirfd, const char* dir_rel, const ArrayList* ke operation_ok = false; continue; } - if (!process_extra_file(dirfd, entries[i].name, child_rel, force[i] || force_now, session)) + if (!process_extra_file(dirfd, entries[i].name, child_rel, entries[i].mode, + force[i] || force_now, session)) operation_ok = false; free(child_rel); } @@ -902,7 +1088,8 @@ int delete_plan_session_receive(DeletePlanSession* session, const Config* config if (has_config) { if (session->config_seen || !read_section(fd, session->protected_prefixes, true, &bytes) || !read_section(fd, session->size_skipped, true, &bytes) || - !read_section(fd, session->missing, true, &bytes)) { + !read_section(fd, session->missing, true, &bytes) || + !delete_filter_dir_rules_receive(fd, &session->per_dir_rules)) { send_status(fd, STATUS_ERROR); return -1; } @@ -1032,7 +1219,7 @@ static bool apply_deferred_path(DeletePlanSession* session, const Config* config session->deleted++; session->planned++; log_deleted(rel); - notify_deleted(session, rel); + notify_deleted(session, rel, delete_entry_type_of_mode(st.st_mode)); } else if (errno != ENOENT) { close(parent_fd); free(leaf); diff --git a/src/shared/delete_plan.h b/src/shared/delete_plan.h index 9472d65..52d0720 100644 --- a/src/shared/delete_plan.h +++ b/src/shared/delete_plan.h @@ -25,6 +25,19 @@ * --delete-missing-args exact deletions, the shared --max-delete budget and, * for --delete-delay, the snapshotted extras. */ +/* Per-directory filter-rule block (protocol 2.30.0). The sender compiles the + * source's per-directory merge rules as it scans and streams them so the + * receiver can re-derive the receiver-side protect/risk verdicts for + * destination-only entries. The wire format is a group count, then for each + * directory group its relative owner path followed by that directory's rule + * records (action, sides, anchored, dir-only, negate, no-inherit, pattern). + * All bounds (MAX_FILTER_RULES, MAX_FILTER_BYTES, MAX_PROTECT_PATTERN_LEN) are + * enforced on both sides; a malformed receive frame signals STATUS_ERROR and + * returns false. Receive yields a flat FilterRuleList whose rules carry their + * owner, or NULL when no rules were sent. */ +bool delete_filter_dir_rules_send(int fd, const FilterRuleList* rules); +bool delete_filter_dir_rules_receive(int fd, FilterRuleList** out); + /* ---- Sender: plan builder ---- */ typedef struct DeletePlanSender DeletePlanSender; @@ -53,7 +66,8 @@ bool delete_plan_sender_empty(const DeletePlanSender* sender); * block is always transmitted by delete_plan_send_root(), on a config-only * carrier frame when the scope allows no directory plan. */ void delete_plan_sender_set_config(DeletePlanSender* sender, const ArrayList* protected_prefixes, - const ArrayList* size_skipped, const ArrayList* missing_args); + const ArrayList* size_skipped, const ArrayList* missing_args, + const FilterRuleList* per_dir_rules); /* Send the root plan (even before any data, so root extras are handled like * rsync's first generator directory), after transmitting the per-run config * block on its own carrier frame. Returns -1 on I/O error. */ diff --git a/src/shared/filter.c b/src/shared/filter.c index 56cc6ed..f8365bd 100644 --- a/src/shared/filter.c +++ b/src/shared/filter.c @@ -21,6 +21,28 @@ void filter_rule_free(FilterRule* rule) { free(rule); } +FilterRule* filter_rule_clone(const FilterRule* rule) { + if (!rule) + return NULL; + FilterRule* copy = calloc(1, sizeof(FilterRule)); + if (!copy) + return NULL; + copy->action = rule->action; + copy->sides = rule->sides; + copy->anchored = rule->anchored; + copy->dir_only = rule->dir_only; + copy->negate = rule->negate; + copy->perishable = rule->perishable; + copy->no_inherit = rule->no_inherit; + copy->owner = str_dup(rule->owner ? rule->owner : ""); + copy->pattern = str_dup(rule->pattern ? rule->pattern : ""); + if (!copy->owner || !copy->pattern) { + filter_rule_free(copy); + return NULL; + } + return copy; +} + FilterRuleList* filter_rule_list_create(void) { return calloc(1, sizeof(FilterRuleList)); } @@ -48,37 +70,94 @@ void filter_rule_list_free(FilterRuleList* list) { for (int i = 0; i < list->count; i++) filter_rule_free(list->items[i]); for (int i = 0; i < list->dir_merge_count; i++) - free(list->dir_merge_names[i]); - free(list->dir_merge_names); + free(list->dir_merges[i].name); + free(list->dir_merges); free(list->items); free(list); } +/* Append an implicit exclude rule for the merge file itself (rsync's 'e' + * modifier). The rule is owned by the transfer root and matches the basename + * anywhere, exactly like rsync's EXCLUDE_SELF (a dual-sided exclude: it hides + * the file and protects its destination mirror from --delete). */ +static bool filter_list_add_exclude_self(FilterRuleList* list, const char* name) { + const char* base = strrchr(name, '/'); + base = base ? base + 1 : name; + if (base[0] == '\0') + return true; + FilterRule* rule = calloc(1, sizeof(FilterRule)); + if (!rule) + return false; + rule->action = FILTER_ACTION_EXCLUDE; + rule->sides = FILTER_SIDE_SENDER | FILTER_SIDE_RECEIVER; + rule->pattern = str_dup(base); + if (!rule->pattern || !filter_rule_set_owner(rule, "")) { + filter_rule_free(rule); + return false; + } + if (!filter_rule_list_add(list, rule)) { + filter_rule_free(rule); + return false; + } + return true; +} + /* Register a per-directory merge-file basename (for "dir-merge NAME"/": NAME" * and -F's .rsync-filter). Duplicate names are ignored. */ bool filter_rule_list_add_dir_merge(FilterRuleList* list, const char* name) { + return filter_rule_list_add_dir_merge_ex(list, name, false, false, false, false, false); +} + +bool filter_rule_list_add_dir_merge_ex(FilterRuleList* list, const char* name, bool no_prefixes, + bool include, bool word_split, bool no_inherit, + bool exclude_self) { if (!list || !name || name[0] == '\0') return false; for (int i = 0; i < list->dir_merge_count; i++) { - if (strcmp(list->dir_merge_names[i], name) == 0) + if (strcmp(list->dir_merges[i].name, name) == 0) { + /* rsync keeps the first registration (first-wins), but the 'e' modifier + is a list side effect, not a registration field: honor it on the + duplicate path too, adding the implicit exclude-self rule at most + once. */ + if (exclude_self && !list->dir_merges[i].exclude_self) { + if (!filter_list_add_exclude_self(list, name)) + return false; + list->dir_merges[i].exclude_self = true; + } return true; + } } if (list->dir_merge_count == list->dir_merge_capacity) { + if (list->dir_merge_capacity > INT_MAX / 2) + return false; int new_cap = list->dir_merge_capacity > 0 ? list->dir_merge_capacity * 2 : 4; - char** grown = realloc(list->dir_merge_names, (size_t)new_cap * sizeof(char*)); + FilterDirMerge* grown = realloc(list->dir_merges, (size_t)new_cap * sizeof(*grown)); if (!grown) return false; - list->dir_merge_names = grown; + list->dir_merges = grown; list->dir_merge_capacity = new_cap; } char* dup = str_dup(name); if (!dup) return false; - list->dir_merge_names[list->dir_merge_count++] = dup; + if (exclude_self && !filter_list_add_exclude_self(list, name)) { + free(dup); + return false; + } + FilterDirMerge* entry = &list->dir_merges[list->dir_merge_count]; + entry->name = dup; + entry->no_prefixes = no_prefixes; + entry->include = include; + entry->word_split = word_split; + entry->no_inherit = no_inherit; + entry->exclude_self = exclude_self; + list->dir_merge_count++; return true; } -static bool set_rule_owner(FilterRule* rule, const char* owner) { +bool filter_rule_set_owner(FilterRule* rule, const char* owner) { + if (!rule) + return false; char* dup = str_dup(owner ? owner : ""); if (!dup) return false; @@ -177,10 +256,11 @@ static bool is_unsupported_modifier_char(char c) { return c == 'e' || c == 'n' || c == 'w'; } -/* Merge-file modifiers rsync accepts on merge/dir-merge rules: 'e', 'n', 'w' - * and '-' (do not transfer the merge file). */ +/* Merge-file modifiers rsync accepts on merge/dir-merge rules: 'e' (exclude + * self), 'n' (no inherit), 'w' (word split), '-' (bare excludes) and '+' + * (bare includes). */ static bool is_merge_modifier_char(char c) { - return c == 'e' || c == 'n' || c == 'w' || c == '-'; + return c == 'e' || c == 'n' || c == 'w' || c == '-' || c == '+'; } /* Characters that count as part of a modifier run for `kind` when deciding @@ -228,8 +308,10 @@ static char unsupported_modifier_in_token(const char* tok, RuleKind kind) { * generic syntax error so callers can emit a precise diagnostic. */ static bool parse_rule_syntax(const char* text, RuleKind* kind, unsigned* sides, bool* sides_explicit, bool* negate, bool* anchored_mod, - bool* perishable, bool* xattr, bool* cvs_inject, - const char** pat_start, size_t* pat_len, char* bad_mod) { + bool* perishable, bool* xattr, bool* cvs_inject, bool* no_prefixes, + bool* include_defaults, bool* word_split, bool* no_inherit, + bool* exclude_self, const char** pat_start, size_t* pat_len, + char* bad_mod) { const char* p = text; *sides = FILTER_SIDE_SENDER | FILTER_SIDE_RECEIVER; *sides_explicit = false; @@ -238,6 +320,11 @@ static bool parse_rule_syntax(const char* text, RuleKind* kind, unsigned* sides, *perishable = false; *xattr = false; *cvs_inject = false; + *no_prefixes = false; + *include_defaults = false; + *word_split = false; + *no_inherit = false; + *exclude_self = false; *pat_start = NULL; *pat_len = 0; *bad_mod = '\0'; @@ -312,6 +399,21 @@ static bool parse_rule_syntax(const char* text, RuleKind* kind, unsigned* sides, case 'C': *cvs_inject = true; break; + case '-': + *no_prefixes = true; + break; + case '+': + *include_defaults = true; + break; + case 'e': + *exclude_self = true; + break; + case 'n': + *no_inherit = true; + break; + case 'w': + *word_split = true; + break; default: break; } @@ -347,11 +449,13 @@ FilterRule* filter_rule_parse(const char* line, const FilterParseOptions* opts, RuleKind kind = RULE_KIND_UNKNOWN; unsigned sides; bool sides_explicit, negate, anchored_mod, perishable, xattr, cvs_inject; + bool no_prefixes, include_defaults, word_split, no_inherit, exclude_self; const char* pat; size_t pat_len; char bad_mod; if (!parse_rule_syntax(p, &kind, &sides, &sides_explicit, &negate, &anchored_mod, &perishable, - &xattr, &cvs_inject, &pat, &pat_len, &bad_mod)) { + &xattr, &cvs_inject, &no_prefixes, &include_defaults, &word_split, + &no_inherit, &exclude_self, &pat, &pat_len, &bad_mod)) { if (bad_mod != '\0') filter_set_error(err, err_size, "unsupported filter modifier '%c'", bad_mod); else @@ -504,7 +608,7 @@ static bool filter_list_append_cvs(FilterRuleList* list, unsigned sides) { } memcpy(rule->pattern, CVS_DEFAULTS[i].pattern, plen); rule->pattern[plen] = '\0'; - if (!set_rule_owner(rule, "")) { + if (!filter_rule_set_owner(rule, "")) { filter_rule_free(rule); return false; } @@ -522,16 +626,138 @@ static bool filter_list_parse_append_depth(FilterRuleList* list, const char* lin const FilterParseOptions* opts, const char* base_dir, int depth, char* err, size_t err_size); -/* Read a merge file and splice its rules into `list`. A relative path is - * resolved below `base_dir` when given, else used as-is (rsync resolves a - * command-line merge file relative to the current directory). */ +/* Append one merge-file token/line to `list`, honoring the merge rule's + * no-prefix/include mode. In no-prefix mode the token is a bare pattern whose + * include/exclude default comes from the merge rule (rsync's "-"/"+" merge + * modifiers); otherwise the token is parsed as a full filter rule. */ +static bool filter_merge_append_token(FilterRuleList* list, const char* token, + const FilterDirMerge* spec, const FilterParseOptions* opts, + const char* base_dir, int depth, char* err, size_t err_size) { + if (spec->no_prefixes || spec->include) { + size_t tlen = strlen(token); + char* text = malloc(tlen + 3); + if (!text) { + filter_set_error(err, err_size, "memory allocation failed"); + return false; + } + text[0] = spec->include ? '+' : '-'; + text[1] = ' '; + memcpy(text + 2, token, tlen + 1); + bool ok = filter_list_parse_append_depth(list, text, opts, base_dir, depth + 1, err, err_size); + free(text); + return ok; + } + return filter_list_parse_append_depth(list, token, opts, base_dir, depth + 1, err, err_size); +} + +/* Read a merge file's tokens/lines into `list` for `spec`. A `w` merge rule + * word-splits on whitespace (turning comments off); otherwise lines are parsed + * and whole-line `#` comments skipped. When `owner_rel` is non-NULL the newly + * added rules are owned by that directory; a no-inherit spec marks them so they + * apply only there. Returns false on parse/allocation failure. */ +static bool filter_merge_read(FilterRuleList* list, FILE* fp, const char* display_path, + const FilterDirMerge* spec, const FilterParseOptions* opts, + const char* base_dir, const char* owner_rel, int depth, char* err, + size_t err_size) { + /* Lowest list index this read is responsible for. A "clear"/"!" inside the + * file resets list->count to 0 (freeing the caller's earlier rules too), so + * the base must follow it down: otherwise post-clear rules sit below the + * original count and never receive an owner (nor no-inherit) and are missed + * by the rollback. */ + int floor = list->count; + char* line = NULL; + size_t cap = 0; + bool ok = true; + while (ok) { + ssize_t n = utils_getdelim_bounded(fp, &line, &cap, '\n', UTILS_MAX_LINE_LEN); + if (n < 0) { + if (errno == EFBIG) + filter_set_error(err, err_size, "line in %s exceeds %d bytes", display_path, + (int)UTILS_MAX_LINE_LEN); + else + filter_set_error(err, err_size, "error reading %s: %s", display_path, strerror(errno)); + ok = false; + break; + } + if (n == 0) + break; + if (spec->word_split) { + /* Whitespace-separated tokens; newlines are ordinary separators and + * comments are disabled. */ + const char* s = line; + while (*s) { + while (*s == ' ' || *s == '\t' || *s == '\n' || *s == '\r') + s++; + if (*s == '\0') + break; + const char* start = s; + while (*s != '\0' && *s != ' ' && *s != '\t' && *s != '\n' && *s != '\r') + s++; + size_t tlen = (size_t)(s - start); + char* token = malloc(tlen + 1); + if (!token) { + filter_set_error(err, err_size, "memory allocation failed"); + ok = false; + break; + } + memcpy(token, start, tlen); + token[tlen] = '\0'; + if (!filter_merge_append_token(list, token, spec, opts, base_dir, depth, err, err_size)) + ok = false; + if (list->count < floor) + floor = list->count; /* a "clear" reset the list below this read's base */ + free(token); + } + } else { + const char* lp = line; + while (*lp == ' ' || *lp == '\t') + lp++; + if (*lp == '\0' || *lp == '\n' || *lp == '\r' || *lp == '#') + continue; + if (!filter_merge_append_token(list, lp, spec, opts, base_dir, depth, err, err_size)) + ok = false; + if (list->count < floor) + floor = list->count; /* a "clear" reset the list below this read's base */ + } + } + free(line); + if (!ok) { + /* Drop every live rule this read is responsible for. After a "clear" that + * base is 0, so the post-clear rules are freed too instead of leaking. */ + for (int i = floor; i < list->count; i++) + filter_rule_free(list->items[i]); + list->count = floor; + return false; + } + for (int i = floor; i < list->count; i++) { + FilterRule* rule = list->items[i]; + if (spec->no_inherit) + rule->no_inherit = true; + if (owner_rel && !filter_rule_set_owner(rule, owner_rel)) { + filter_set_error(err, err_size, "memory allocation failed"); + for (int j = floor; j < list->count; j++) + filter_rule_free(list->items[j]); + list->count = floor; + return false; + } + } + return true; +} + +/* Read a single-instance merge file and splice its rules into `list`. A + * relative path is resolved below `base_dir` when given, else used as-is (rsync + * resolves a command-line merge file relative to the current directory). */ static bool filter_list_merge_file(FilterRuleList* list, const char* name, - const FilterParseOptions* opts, const char* base_dir, int depth, - char* err, size_t err_size) { + const FilterDirMerge* spec, const FilterParseOptions* opts, + const char* base_dir, int depth, char* err, size_t err_size) { if (name[0] == '\0') { filter_set_error(err, err_size, "merge requires a filename"); return false; } + if (spec->exclude_self && !filter_list_add_exclude_self(list, name)) { + filter_set_error(err, err_size, "memory allocation failed"); + return false; + } char* path = (base_dir && base_dir[0] && name[0] != '/') ? path_cat(base_dir, name) : str_dup(name); if (!path) { @@ -544,29 +770,7 @@ static bool filter_list_merge_file(FilterRuleList* list, const char* name, free(path); return false; } - char* line = NULL; - size_t cap = 0; - bool ok = true; - while (true) { - ssize_t n = utils_getdelim_bounded(fp, &line, &cap, '\n', UTILS_MAX_LINE_LEN); - if (n < 0) { - filter_set_error(err, err_size, "error reading merge file '%s'", path); - ok = false; - break; - } - if (n == 0) - break; - const char* lp = line; - while (*lp == ' ' || *lp == '\t') - lp++; - if (*lp == '\0' || *lp == '\n' || *lp == '\r' || *lp == '#') - continue; - if (!filter_list_parse_append_depth(list, lp, opts, base_dir, depth + 1, err, err_size)) { - ok = false; - break; - } - } - free(line); + bool ok = filter_merge_read(list, fp, path, spec, opts, base_dir, NULL, depth, err, err_size); fclose(fp); free(path); return ok; @@ -590,11 +794,13 @@ static bool filter_list_parse_append_depth(FilterRuleList* list, const char* lin RuleKind kind = RULE_KIND_UNKNOWN; unsigned sides; bool sides_explicit, negate, anchored_mod, perishable, xattr, cvs_inject; + bool no_prefixes, include_defaults, word_split, no_inherit, exclude_self; const char* pat; size_t pat_len; char bad_mod; if (!parse_rule_syntax(p, &kind, &sides, &sides_explicit, &negate, &anchored_mod, &perishable, - &xattr, &cvs_inject, &pat, &pat_len, &bad_mod)) { + &xattr, &cvs_inject, &no_prefixes, &include_defaults, &word_split, + &no_inherit, &exclude_self, &pat, &pat_len, &bad_mod)) { if (bad_mod != '\0') filter_set_error(err, err_size, "unsupported filter modifier '%c': %s", bad_mod, p); else @@ -640,7 +846,15 @@ static bool filter_list_parse_append_depth(FilterRuleList* list, const char* lin } memcpy(name, pat, pat_len); name[pat_len] = '\0'; - bool ok = filter_list_merge_file(list, name, opts, base_dir, depth, err, err_size); + /* A single-instance merge has no inheritance, so 'n' is meaningless; the + * other merge modifiers still shape how the file is read. */ + FilterDirMerge spec = {.name = name, + .no_prefixes = no_prefixes, + .include = include_defaults, + .word_split = word_split, + .no_inherit = false, + .exclude_self = exclude_self}; + bool ok = filter_list_merge_file(list, name, &spec, opts, base_dir, depth, err, err_size); free(name); return ok; } @@ -656,7 +870,8 @@ static bool filter_list_parse_append_depth(FilterRuleList* list, const char* lin } memcpy(name, pat, pat_len); name[pat_len] = '\0'; - bool ok = filter_rule_list_add_dir_merge(list, name); + bool ok = filter_rule_list_add_dir_merge_ex(list, name, no_prefixes, include_defaults, + word_split, no_inherit, exclude_self); free(name); if (!ok) { filter_set_error(err, err_size, "memory allocation failed"); @@ -717,27 +932,30 @@ FilterRuleList* filter_base_build(const char* const* rule_texts, int rule_count, /* Undo the rules and dir-merge registrations that one merge file appended, * leaving the caller's earlier content intact. A "clear" rule inside the file * frees every rule, including the caller's; clamp to the surviving count so - * those already-freed rules are never resurrected and freed a second time. */ + * those already-freed rules are never resurrected and freed a second time. + * (filter_merge_read() has already rolled its own range back by the time this + * runs, so on a post-clear failure `list->count` is below `rules_before` and + * this is a no-op for the rules.) */ static void filter_file_rollback(FilterRuleList* list, int rules_before, int dir_merges_before) { int first = rules_before < list->count ? rules_before : list->count; for (int i = first; i < list->count; i++) filter_rule_free(list->items[i]); list->count = first; for (int i = dir_merges_before; i < list->dir_merge_count; i++) - free(list->dir_merge_names[i]); + free(list->dir_merges[i].name); list->dir_merge_count = dir_merges_before; } -bool filter_file_append(FilterRuleList* list, const char* dir_path, const char* name, - const char* owner_rel, const FilterParseOptions* opts, bool* exists, - char* err, size_t err_size) { +bool filter_dir_merge_append(FilterRuleList* list, const char* dir_path, const FilterDirMerge* spec, + const char* owner_rel, const FilterParseOptions* opts, bool* exists, + char* err, size_t err_size) { if (err && err_size > 0) err[0] = '\0'; if (exists) *exists = false; - if (!list) + if (!list || !spec || !spec->name) return false; - char* filter_path = path_cat(dir_path, name); + char* filter_path = path_cat(dir_path, spec->name); if (!filter_path) { filter_set_error(err, err_size, "memory allocation failed"); return false; @@ -748,7 +966,7 @@ bool filter_file_append(FilterRuleList* list, const char* dir_path, const char* if (errno == ENOENT || errno == ENOTDIR) return true; char* escaped_dir = output_escape(dir_path, log_get_8_bit_output()); - log_message(LOG_LEVEL_WARNING, "Could not read %s in %s: %s", name, + log_message(LOG_LEVEL_WARNING, "Could not read %s in %s: %s", spec->name, escaped_dir ? escaped_dir : "", strerror(errno)); free(escaped_dir); return true; @@ -757,51 +975,25 @@ bool filter_file_append(FilterRuleList* list, const char* dir_path, const char* *exists = true; int rules_before = list->count; int dir_merges_before = list->dir_merge_count; - char* line = NULL; - size_t line_cap = 0; - bool ok = true; - while (true) { - ssize_t n = utils_getdelim_bounded(fp, &line, &line_cap, '\n', UTILS_MAX_LINE_LEN); - if (n < 0) { - if (errno == EFBIG) { - filter_set_error(err, err_size, "line in %s exceeds %d bytes", name, - (int)UTILS_MAX_LINE_LEN); - } else { - filter_set_error(err, err_size, "error reading %s: %s", name, strerror(errno)); - } - ok = false; - break; - } - if (n == 0) - break; - const char* p = line; - while (*p == ' ' || *p == '\t') - p++; - if (*p == '\0' || *p == '\n' || *p == '\r' || *p == '#') - continue; - /* Merge files inside a per-directory file resolve relative to that - directory. */ - if (!filter_list_parse_append_depth(list, p, opts, dir_path, 0, err, err_size)) { - ok = false; - break; - } - } - free(line); + /* Merge files inside a per-directory file resolve relative to that + directory. */ + bool ok = + filter_merge_read(list, fp, spec->name, spec, opts, dir_path, owner_rel, 0, err, err_size); fclose(fp); if (!ok) { filter_file_rollback(list, rules_before, dir_merges_before); return false; } - for (int i = rules_before; i < list->count; i++) { - if (!set_rule_owner(list->items[i], owner_rel)) { - filter_set_error(err, err_size, "memory allocation failed"); - filter_file_rollback(list, rules_before, dir_merges_before); - return false; - } - } return true; } +bool filter_file_append(FilterRuleList* list, const char* dir_path, const char* name, + const char* owner_rel, const FilterParseOptions* opts, bool* exists, + char* err, size_t err_size) { + FilterDirMerge spec = {.name = (char*)name}; + return filter_dir_merge_append(list, dir_path, &spec, owner_rel, opts, exists, err, err_size); +} + FilterRuleList* filter_file_read_named(const char* dir_path, const char* name, const char* owner_rel, const FilterParseOptions* opts, bool* exists, char* err, size_t err_size) { @@ -843,11 +1035,16 @@ static FilterAction rule_matches(const FilterRule* rule, const char* rel_path, c return FILTER_ACTION_NONE; if (!(rule->sides & side)) return FILTER_ACTION_NONE; - /* A rule applies only to entries below its owner directory. */ + /* A rule applies only to entries below its owner directory. The receive + * root's destination-relative coordinate may be written as "." (the + * synced-directory sentinel), which is the same scope as the empty owner. */ + const char* owner = rule->owner; + if (owner && strcmp(owner, ".") == 0) + owner = ""; const char* rel2 = rel_path; - if (rule->owner && rule->owner[0] != '\0') { - size_t owner_len = strlen(rule->owner); - if (strncmp(rule->owner, rel_path, owner_len) != 0) + if (owner && owner[0] != '\0') { + size_t owner_len = strlen(owner); + if (strncmp(owner, rel_path, owner_len) != 0) return FILTER_ACTION_NONE; if (rel_path[owner_len] != '/') return FILTER_ACTION_NONE; @@ -855,6 +1052,10 @@ static FilterAction rule_matches(const FilterRule* rule, const char* rel_path, c } if (rel2[0] == '\0') return FILTER_ACTION_NONE; + /* A no-inherit rule ('n' on its dir-merge) applies only to direct children of + * its owner directory, never to deeper entries. */ + if (rule->no_inherit && strchr(rel2, '/') != NULL) + return FILTER_ACTION_NONE; bool matched; if (rule->dir_only && !is_dir) matched = false; @@ -884,3 +1085,45 @@ FilterAction filter_rules_apply_side(const FilterRuleList* list, const char* rel } return FILTER_ACTION_NONE; } + +FilterAction filter_dir_rules_apply_side(const FilterRuleList* dir_rules, const char* rel_path, + const char* leaf, bool is_dir) { + if (!dir_rules || !rel_path) + return FILTER_ACTION_NONE; + size_t len = strlen(rel_path); + if (len == 0) + return FILTER_ACTION_NONE; + /* The containing directory of rel_path is the prefix before its final '/'. */ + size_t owner_len = 0; + for (size_t i = 0; i < len; i++) { + if (rel_path[i] == '/') + owner_len = i; + } + for (;;) { + for (int i = 0; i < dir_rules->count; i++) { + const FilterRule* rule = dir_rules->items[i]; + const char* rule_owner = rule && rule->owner ? rule->owner : ""; + /* "." is the receive root's coordinate (see rule_matches). */ + if (strcmp(rule_owner, ".") == 0) + rule_owner = ""; + size_t rule_owner_len = strlen(rule_owner); + if (rule_owner_len != owner_len) + continue; + if (owner_len != 0 && memcmp(rule_owner, rel_path, owner_len) != 0) + continue; + FilterAction action = rule_matches(rule, rel_path, leaf, is_dir, FILTER_SIDE_RECEIVER); + if (action != FILTER_ACTION_NONE) + return action; + } + if (owner_len == 0) + break; + /* Move to the parent directory: the last '/' before owner_len. */ + size_t parent = 0; + for (size_t j = 0; j < owner_len; j++) { + if (rel_path[j] == '/') + parent = j; + } + owner_len = parent; + } + return FILTER_ACTION_NONE; +} diff --git a/src/shared/filter.h b/src/shared/filter.h index c651afc..9c4ae2f 100644 --- a/src/shared/filter.h +++ b/src/shared/filter.h @@ -25,11 +25,12 @@ * Modifiers: '/' absolute anchor, '!' negate match, 'C' inject CVS defaults, * 's' sender side, 'r' receiver side, 'p' perishable. The rsync 'x' * (xattr-name) modifier is not implemented and is rejected explicitly - * everywhere. The merge-file modifiers 'e' (exclude the merge file itself), - * 'n' (do not inherit the merge file), 'w' (word-split the merge file) and '-' - * (do not transfer the merge file) are accepted and consumed only on merge/ - * dir-merge rules (rejected on every other rule, matching rsync); their - * semantics are not implemented and they are otherwise ignored. + * everywhere. The merge-only modifiers are accepted only on merge/dir-merge + * rules (rejected on every other rule, matching rsync): 'e' excludes the merge + * file itself, 'n' makes the merged rules non-inheriting (they apply only to + * the directory that holds the merge file), 'w' word-splits the merge file on + * whitespace instead of lines, and '-' reads the merge file as a list of bare + * exclude patterns with no rule prefixes. * A trailing '/' makes a pattern match directories only. A leading '/' anchors * the pattern to its owner directory. */ @@ -55,18 +56,32 @@ typedef struct { bool dir_only; /* pattern had a trailing '/': matches directories only */ bool negate; /* '!' modifier: match succeeds when the pattern does not */ bool perishable; /* 'p' modifier (ignored in deleted directories) */ + bool no_inherit; /* 'n' on the owning dir-merge: applies only in `owner` */ char* owner; /* owning directory rel path ("" == transfer root) */ char* pattern; /* cleaned glob pattern (no leading '/', no trailing '/') */ } FilterRule; +/* One per-directory merge-file registration ("dir-merge NAME"/": NAME", "merge + * NAME"/". NAME" and -F's .rsync-filter) together with the merge-only modifiers + * parsed from the rule. The scanner reads `name` in every directory it + * traverses and applies `no_prefixes`/`include`/`word_split`/`no_inherit`/ + * `exclude_self` while merging the file's rules. */ +typedef struct { + char* name; + bool no_prefixes; /* '-' : file holds only bare exclude patterns */ + bool include; /* '+' : file holds only bare include patterns */ + bool word_split; /* 'w' : split the file on whitespace, not lines */ + bool no_inherit; /* 'n' : the merged rules do not inherit below their dir */ + bool exclude_self; /* 'e' : exclude the merge file itself from the transfer */ +} FilterDirMerge; + typedef struct FilterRuleList { FilterRule** items; /* owned array of rule pointers */ int count; int capacity; - /* Per-directory merge-file basenames registered by "dir-merge NAME"/": NAME" - * or by -F (.rsync-filter). Owned strings; the scanner reads each name in - * every directory it traverses. */ - char** dir_merge_names; + /* Per-directory merge-file registrations. Owned; the scanner reads each + * name in every directory it traverses. */ + FilterDirMerge* dir_merges; int dir_merge_count; int dir_merge_capacity; } FilterRuleList; @@ -83,13 +98,28 @@ typedef struct { FilterRule* filter_rule_parse(const char* line, const FilterParseOptions* opts, char* err, size_t err_size); void filter_rule_free(FilterRule* rule); +/* Deep-copy a rule (owned pattern/owner). Returns NULL on allocation failure. */ +FilterRule* filter_rule_clone(const FilterRule* rule); +/* Replace a rule's owner directory (owned copy of `owner`, "" for the transfer + * root). Returns false on allocation failure, leaving the rule unchanged. + * Used to re-express a mirrored per-directory rule in the receiver's + * destination-relative coordinate system. */ +bool filter_rule_set_owner(FilterRule* rule, const char* owner); FilterRuleList* filter_rule_list_create(void); /* Append a fully-parsed rule (takes ownership). Returns false on OOM. */ bool filter_rule_list_add(FilterRuleList* list, FilterRule* rule); -/* Register a per-directory merge-file basename (idempotent). Returns false on - * OOM. Used by the scanner to read custom "dir-merge" files. */ +/* Register a per-directory merge-file basename (idempotent, no modifiers). + * Returns false on OOM. Used by the scanner to read custom "dir-merge" files. */ bool filter_rule_list_add_dir_merge(FilterRuleList* list, const char* name); +/* Register a per-directory merge file with its merge-only modifiers. On a + * duplicate name the existing registration is kept (rsync's first wins) and true + * is returned. When `exclude_self` is set an implicit exclude rule for the + * merge file's basename is appended to the list at this position, matching + * rsync's `e` modifier. Returns false on OOM. */ +bool filter_rule_list_add_dir_merge_ex(FilterRuleList* list, const char* name, bool no_prefixes, + bool include, bool word_split, bool no_inherit, + bool exclude_self); /* Parse `line` and append it. Handles "clear"/"!" (resets the list), "merge * FILE"/". FILE" (splices the file's rules) and "dir-merge NAME"/": NAME" * (registers a per-directory filename). Returns false and fills `err` on bad @@ -122,6 +152,15 @@ bool filter_file_append(FilterRuleList* list, const char* dir_path, const char* const char* owner_rel, const FilterParseOptions* opts, bool* exists, char* err, size_t err_size); +/* Append a per-directory merge file honoring its merge-only modifiers: `-` + * reads every (word-split, when `w`) token as a bare exclude, `+` as a bare + * include, and `n` marks each read rule non-inheriting. A plain name behaves + * like filter_file_append. A missing file yields *exists=false with no error; + * returns false only on a parse/allocation failure (message in `err`). */ +bool filter_dir_merge_append(FilterRuleList* list, const char* dir_path, const FilterDirMerge* spec, + const char* owner_rel, const FilterParseOptions* opts, bool* exists, + char* err, size_t err_size); + /* filter_file_read_named with the default ".rsync-filter" name. */ FilterRuleList* filter_file_read(const char* dir_path, const char* owner_rel, bool* exists, char* err, size_t err_size); @@ -135,4 +174,15 @@ FilterRuleList* filter_file_read(const char* dir_path, const char* owner_rel, bo FilterAction filter_rules_apply_side(const FilterRuleList* list, const char* rel_path, const char* leaf, bool is_dir, unsigned side); +/* Evaluate a received per-directory rule set for the receiver side, using + * rsync's per-directory-before-ancestors order: the entry's containing + * directory's rules are tried first, then each ancestor's, then the receive + * root's. `dir_rules` is a flat list whose rules carry `owner`; a rule applies + * only when `owner` is exactly the directory being examined (a no-inherit rule + * therefore applies only to that directory's direct children). Returns the + * first matching rule's receiver verdict (PROTECT/RISK) or FILTER_ACTION_NONE. + * The caller evaluates the command-line base rules after this chain. */ +FilterAction filter_dir_rules_apply_side(const FilterRuleList* dir_rules, const char* rel_path, + const char* leaf, bool is_dir); + #endif diff --git a/src/shared/format.c b/src/shared/format.c index df2c463..91b4cc0 100644 --- a/src/shared/format.c +++ b/src/shared/format.c @@ -62,14 +62,16 @@ bool format_dest_state_send(int fd, const OutputDestState* state) { if (!state) return false; int32_t has_old = state->existed ? 1 : 0; + int32_t target_matches = state->target_matches ? 1 : 0; uint64_t size = (uint64_t)state->size; int64_t mtime = (int64_t)state->mtime_sec; int64_t mtime_nsec = state->mtime_nsec; uint32_t mode = state->mode; int32_t uid = state->uid; int32_t gid = state->gid; - return send_n_data(fd, &has_old, sizeof(has_old)) && send_n_data(fd, &size, sizeof(size)) && - send_n_data(fd, &mtime, sizeof(mtime)) && + return send_n_data(fd, &has_old, sizeof(has_old)) && + send_n_data(fd, &target_matches, sizeof(target_matches)) && + send_n_data(fd, &size, sizeof(size)) && send_n_data(fd, &mtime, sizeof(mtime)) && send_n_data(fd, &mtime_nsec, sizeof(mtime_nsec)) && send_n_data(fd, &mode, sizeof(mode)) && send_n_data(fd, &uid, sizeof(uid)) && send_n_data(fd, &gid, sizeof(gid)); } @@ -78,14 +80,16 @@ bool format_dest_state_receive(int fd, OutputDestState* state) { if (!state) return false; int32_t has_old = 0; + int32_t target_matches = 0; uint64_t size = 0; int64_t mtime = 0; int64_t mtime_nsec = 0; uint32_t mode = 0; int32_t uid = 0; int32_t gid = 0; - if (!receive_n_data(fd, &has_old, sizeof(has_old)) || !receive_n_data(fd, &size, sizeof(size)) || - !receive_n_data(fd, &mtime, sizeof(mtime)) || + if (!receive_n_data(fd, &has_old, sizeof(has_old)) || + !receive_n_data(fd, &target_matches, sizeof(target_matches)) || + !receive_n_data(fd, &size, sizeof(size)) || !receive_n_data(fd, &mtime, sizeof(mtime)) || !receive_n_data(fd, &mtime_nsec, sizeof(mtime_nsec)) || !receive_n_data(fd, &mode, sizeof(mode)) || !receive_n_data(fd, &uid, sizeof(uid)) || !receive_n_data(fd, &gid, sizeof(gid))) @@ -93,6 +97,7 @@ bool format_dest_state_receive(int fd, OutputDestState* state) { memset(state, 0, sizeof(*state)); state->known = true; state->existed = has_old != 0; + state->target_matches = target_matches != 0; state->size = size; state->mtime_sec = mtime; state->mtime_nsec = mtime_nsec; @@ -105,9 +110,10 @@ bool format_dest_state_receive(int fd, OutputDestState* state) { bool format_stats_send(int fd, const ReceiverStats* stats) { if (!stats) return false; - unsigned long long fields[8] = { + unsigned long long fields[12] = { stats->matched_data, stats->deleted_files, stats->would_delete_count, stats->literal_bytes, stats->created_reg, stats->created_dir, stats->created_link, stats->created_special, + stats->deleted_reg, stats->deleted_dir, stats->deleted_link, stats->deleted_special, }; return send_n_data(fd, fields, sizeof(fields)); } @@ -115,7 +121,7 @@ bool format_stats_send(int fd, const ReceiverStats* stats) { bool format_stats_receive(int fd, ReceiverStats* stats) { if (!stats) return false; - unsigned long long fields[8] = {0}; + unsigned long long fields[12] = {0}; if (!receive_n_data(fd, fields, sizeof(fields))) return false; memset(stats, 0, sizeof(*stats)); @@ -127,5 +133,9 @@ bool format_stats_receive(int fd, ReceiverStats* stats) { stats->created_dir = fields[5]; stats->created_link = fields[6]; stats->created_special = fields[7]; + stats->deleted_reg = fields[8]; + stats->deleted_dir = fields[9]; + stats->deleted_link = fields[10]; + stats->deleted_special = fields[11]; return true; } diff --git a/src/shared/format.h b/src/shared/format.h index 9da9a93..99d5dd5 100644 --- a/src/shared/format.h +++ b/src/shared/format.h @@ -17,10 +17,18 @@ /* Pre-transfer destination snapshot, reported by the receiver when the wire * config carries report_dest_info. `known` distinguishes "no report was * requested/received" from "the destination did not exist" (`existed == false` - * with `known == true`). */ + * with `known == true`). + * + * `target_matches` is meaningful only for a symlink destination (protocol + * 2.30.0): the receiver compares its on-disk link target with the incoming + * target and reports whether they are equal, so the sender can render rsync's + * `cLc........` (target changed) versus `.L..t......` (attributes only) and + * suppress an unchanged symlink's line entirely. It is always false for every + * other entry kind. */ typedef struct { bool known; bool existed; + bool target_matches; unsigned long long size; long long mtime_sec; long long mtime_nsec; @@ -57,17 +65,24 @@ bool format_dest_state_send(int fd, const OutputDestState* state); bool format_dest_state_receive(int fd, OutputDestState* state); /* End-of-transfer receiver counters reported through STATUS_STATS (protocol - * 2.25.0, extended in 2.28.0) when the wire config carries report_stats. - * `would_delete_count` is the number of destination-relative paths the receiver - * would have deleted in a -n/--dry-run --delete run; that many wire strings - * immediately follow the fixed record (sent/read by the caller). + * 2.25.0, extended in 2.28.0 and 2.30.0) when the wire config carries + * report_stats. `would_delete_count` is the number of destination-relative + * paths the receiver would have deleted in a -n/--dry-run --delete run; that + * many wire strings immediately follow the fixed record (sent/read by the + * caller). * * Protocol 2.28.0 adds the receiver-observed counters the sender cannot see: * `literal_bytes` is the file data the receiver actually stored literally * (whole files plus the literal fragments of a delta) and the four `created_*` * counters split the destination entries the receiver newly created by type, * reproducing rsync's `Number of created files` breakdown and an exact - * `Literal data` for a delta run. */ + * `Literal data` for a delta run. + * + * Protocol 2.30.0 appends the four `deleted_*` counters: the same reg/dir/link/ + * special split for the entries the receiver ACTUALLY removed, so `--stats` can + * render rsync's `Number of deleted files: X (reg: A, dir: B, link: C, + * special: D)` parenthetical. The scalar `deleted_files` stays the authoritative + * total (the breakdown is a strict partition of it). */ typedef struct { unsigned long long matched_data; unsigned long long deleted_files; @@ -77,6 +92,10 @@ typedef struct { unsigned long long created_dir; unsigned long long created_link; unsigned long long created_special; + unsigned long long deleted_reg; + unsigned long long deleted_dir; + unsigned long long deleted_link; + unsigned long long deleted_special; } ReceiverStats; /* Fixed-width STATUS_STATS counter record. The status frame and the optional diff --git a/src/shared/log.c b/src/shared/log.c index 4e69a05..f3db46b 100644 --- a/src/shared/log.c +++ b/src/shared/log.c @@ -1,4 +1,5 @@ #include "log.h" +#include "utils.h" #include #include #include @@ -16,6 +17,7 @@ static bool info_flags_explicit = false; static FILE* log_fp = NULL; static _Thread_local bool eight_bit_output; static LogStderrMode stderr_mode = LOG_STDERR_ERRORS; +static LogClientMsgSink client_msg_sink = NULL; /* Serializes access to log_fp and makes each emitted line atomic: the * timestamp prefix, formatted body, and trailing newline are written as one @@ -77,6 +79,51 @@ LogStderrMode log_get_stderr_mode(void) { return stderr_mode; } +void log_set_client_msg_sink(LogClientMsgSink sink) { + client_msg_sink = sink; +} + +LogClientMsgSink log_get_client_msg_sink(void) { + return client_msg_sink; +} + +/* Format just the message body (no prefix/newline) into a freshly allocated + * buffer. Shared by log_message (which may hand the body to a client-message + * sink) and log_client_message. Returns NULL on allocation/format failure. */ +static char* format_log_body(const char* format, va_list args) { + va_list copy; + va_copy(copy, args); + int body_len = vsnprintf(NULL, 0, format, copy); + va_end(copy); + if (body_len < 0) + return NULL; + char* body = malloc((size_t)body_len + 1); + if (!body) + return NULL; + vsnprintf(body, (size_t)body_len + 1, format, args); + return body; +} + +/* Assemble a complete log line (prefix + body + newline) from an already + * formatted body. Returns NULL on allocation failure. */ +static char* format_log_line_from_body(LogLevel log_level, const struct tm* t, const char* body) { + char prefix[64]; + int prefix_len = snprintf( + prefix, sizeof(prefix), "%04d-%02d-%02d %02d:%02d:%02d [%s]: ", t->tm_year + 1900, + t->tm_mon + 1, t->tm_mday, t->tm_hour, t->tm_min, t->tm_sec, log_level_strings[log_level]); + if (prefix_len < 0 || prefix_len >= (int)sizeof(prefix)) + return NULL; + size_t body_len = strlen(body); + char* line = malloc((size_t)prefix_len + body_len + 2); /* body + '\n' + NUL */ + if (!line) + return NULL; + memcpy(line, prefix, (size_t)prefix_len); + memcpy(line + prefix_len, body, body_len); + line[(size_t)prefix_len + body_len] = '\n'; + line[(size_t)prefix_len + body_len + 1] = '\0'; + return line; +} + /* Format one complete log line (timestamp prefix + body + newline) into a * freshly allocated buffer. This is pure CPU/malloc work and must happen * OUTSIDE the log mutex: the mutex only guards the log_fp pointer, so a @@ -84,26 +131,11 @@ LogStderrMode log_get_stderr_mode(void) { * on allocation/formatting failure. */ static char* format_log_line(LogLevel log_level, const struct tm* t, const char* format, va_list args) { - char prefix[64]; - int prefix_len = snprintf( - prefix, sizeof(prefix), "%04d-%02d-%02d %02d:%02d:%02d [%s]: ", t->tm_year + 1900, - t->tm_mon + 1, t->tm_mday, t->tm_hour, t->tm_min, t->tm_sec, log_level_strings[log_level]); - if (prefix_len < 0 || prefix_len >= (int)sizeof(prefix)) + char* body = format_log_body(format, args); + if (!body) return NULL; - va_list copy; - va_copy(copy, args); - int body_len = vsnprintf(NULL, 0, format, copy); - va_end(copy); - if (body_len < 0) - return NULL; - size_t total = (size_t)prefix_len + (size_t)body_len; - char* line = malloc(total + 2); /* body bytes + '\n' + NUL */ - if (!line) - return NULL; - memcpy(line, prefix, (size_t)prefix_len); - vsnprintf(line + prefix_len, (size_t)body_len + 1, format, args); - line[total] = '\n'; - line[total + 1] = '\0'; + char* line = format_log_line_from_body(log_level, t, body); + free(body); return line; } @@ -121,6 +153,26 @@ static void emit_log_line(FILE* console, const char* line) { mtx_unlock(&log_mutex); } +void log_client_message(const char* message) { + if (!message) + return; + /* The body is peer-controlled: escape every non-printable byte (newlines, + CR, ANSI ESC, ...) so a hostile client cannot forge log lines or inject + terminal control sequences. output_escape() is the codebase's canonical + escaper and leaves printable text untouched. */ + char* escaped = output_escape(message, log_get_8_bit_output()); + if (!escaped) + return; + /* Route through the ordinary log level / destination gate (log_message): + this respects --log-file, the configured stderr mode and the level + threshold instead of always writing to stderr. The wire body carries no + severity, so the forwarded diagnostic is emitted as a warning -- the + lowest level the default gate admits, which keeps the peer's messages + visible without bypassing --quiet. */ + log_message(LOG_LEVEL_WARNING, "%s", escaped); + free(escaped); +} + void log_message(LogLevel log_level, const char* format, ...) { if (log_level < current_log_level) return; @@ -138,8 +190,23 @@ void log_message(LogLevel log_level, const char* format, ...) { va_list args; va_start(args, format); - char* line = format_log_line(log_level, &t, format, args); + char* body = format_log_body(format, args); va_end(args); + if (!body) + return; + /* LOG_STDERR_CLIENT: hand the diagnostic to the client-message channel. A + sink that takes ownership suppresses the local write; otherwise (no sink + yet, or the peer connection is not up) fall through to local output so the + diagnostic is never lost. */ + if (stderr_mode == LOG_STDERR_CLIENT) { + LogClientMsgSink sink = client_msg_sink; + if (sink && sink(body)) { + free(body); + return; + } + } + char* line = format_log_line_from_body(log_level, &t, body); + free(body); if (!line) return; emit_log_line(dest_io, line); diff --git a/src/shared/log.h b/src/shared/log.h index d64c467..f7ea308 100644 --- a/src/shared/log.h +++ b/src/shared/log.h @@ -15,7 +15,11 @@ #endif typedef enum { LOG_LEVEL_DEBUG, LOG_LEVEL_INFO, LOG_LEVEL_WARNING, LOG_LEVEL_ERROR } LogLevel; -typedef enum { LOG_STDERR_ERRORS, LOG_STDERR_ALL } LogStderrMode; +/* --stderr=MODE destinations. ERRORS keeps errors on stderr and everything + * else on stdout; ALL sends every message to stderr; CLIENT routes the client's + * own diagnostics over the protocol stream to the peer's stderr (rsync's + * --stderr=client / --no-msgs2stderr). */ +typedef enum { LOG_STDERR_ERRORS, LOG_STDERR_ALL, LOG_STDERR_CLIENT } LogStderrMode; typedef enum { LOG_DEBUG_IO = 1u << 0, @@ -86,5 +90,20 @@ void log_set_8_bit_output(bool enabled); bool log_get_8_bit_output(void); void log_set_stderr_mode(LogStderrMode mode); LogStderrMode log_get_stderr_mode(void); +/* Write a message a peer forwarded over the client-message channel to this + * process's stderr (and log file), with the standard log prefix. Used by the + * server side of rsync's --stderr=client. */ +void log_client_message(const char* message); + +/* Sink for LOG_STDERR_CLIENT. log_message() passes the un-prefixed message + * body to the installed sink; a `true` return means the sink took ownership + * (e.g. queued it for protocol transmission) and the message must NOT also be + * written locally. A `false` return (or a NULL sink) makes log_message fall + * back to the normal local destination, so a diagnostic emitted before the peer + * connection exists is never lost (rsync's documented fallback). The sink may + * be called from any thread and must be tolerant of that. */ +typedef bool (*LogClientMsgSink)(const char* message); +void log_set_client_msg_sink(LogClientMsgSink sink); +LogClientMsgSink log_get_client_msg_sink(void); #endif diff --git a/src/shared/multiprocessing.c b/src/shared/multiprocessing.c index 3e54e87..26c23f5 100644 --- a/src/shared/multiprocessing.c +++ b/src/shared/multiprocessing.c @@ -34,6 +34,7 @@ PipelineContextSender* pipeline_context_sender_create(Config* config, Queue* que context->synced_dirs = NULL; context->plan_dirs = NULL; context->missing_args = NULL; + context->per_dir_rules = NULL; context->scan_had_io_error = false; context->remove_source_files = NULL; context->early_delete = false; @@ -53,6 +54,7 @@ PipelineContextSender* pipeline_context_sender_create(Config* config, Queue* que context->dir_entries_mutex_init = false; atomic_init(&context->dir_count, 0); context->delete_limit = false; + context->partial = false; int init = 0; if (config->use_metadata) { context->dir_entries = array_list_create(file_destroy); @@ -209,6 +211,8 @@ void pipeline_context_sender_destroy(PipelineContextSender* context) { array_list_delete(context->plan_dirs); if (context->missing_args) array_list_delete(context->missing_args); + if (context->per_dir_rules) + filter_rule_list_free(context->per_dir_rules); if (context->remove_source_files) array_list_delete(context->remove_source_files); if (context->dir_entries) diff --git a/src/shared/multiprocessing.h b/src/shared/multiprocessing.h index 92ae4cc..2150ace 100644 --- a/src/shared/multiprocessing.h +++ b/src/shared/multiprocessing.h @@ -67,6 +67,12 @@ typedef struct { them in the manifest frame's third section and the receiver deletes each as an explicit request. */ ArrayList* missing_args; + /* Per-directory filter rules the source scan compiled (protocol 2.30.0), + sent with the delete manifest/plan config so the receiver can re-derive the + per-directory protect/risk set. NULL when --delete is off. Populated by + the scanner (parallel workers append under mutex_scanner) or the early + pre-scan. */ + FilterRuleList* per_dir_rules; /* A source I/O error (unreadable directory) was recorded during the scan. Set by the pre-scan (before the threads start) or by the scanner thread under mutex_scanner; the caller turns it into a non-zero exit when @@ -134,6 +140,11 @@ typedef struct { deletion (STATUS_DELETE_LIMIT): the transfer succeeded and the process must exit 25 like rsync. Read by the caller after the sender thread is joined. */ bool delete_limit; + /* Set by the sender thread when the receiver reported STATUS_PARTIAL (a + per-entry receiver failure that did not abort the stream): the transfer + otherwise succeeded, successfully stored --remove-source-files sources were + removed, and the process must exit 23 like rsync. Read after join. */ + bool partial; } PipelineContextSender; /* `config` is borrowed and must outlive the context: destroy does NOT free it, diff --git a/src/shared/protocol.c b/src/shared/protocol.c index 6bce72d..035c5fc 100644 --- a/src/shared/protocol.c +++ b/src/shared/protocol.c @@ -664,6 +664,10 @@ static const char* status_to_string(Status status) { return "DELETE_LIMIT"; case STATUS_DEST_INFO: return "DEST_INFO"; + case STATUS_CLIENT_MSG: + return "CLIENT_MSG"; + case STATUS_PARTIAL: + return "PARTIAL"; default: return "UNKNOWN"; } @@ -672,10 +676,10 @@ static const char* status_to_string(Status status) { /* Reject a raw wire status outside the known enum range before it is handed to * callers, so an unknown/corrupt frame fails as a protocol error instead of * being silently interpreted as an unexpected-but-valid verdict. STATUS_OK is - * the first enumerator and STATUS_STATS the last, so the range check accepts + * the first enumerator and STATUS_PARTIAL the last, so the range check accepts * every status the protocol defines. */ static bool status_is_valid(Status status) { - return status >= STATUS_OK && status <= STATUS_STATS; + return status >= STATUS_OK && status <= STATUS_PARTIAL; } /* Shared string send/receive implementation. `redact` selects whether the @@ -1144,6 +1148,19 @@ bool send_error_detail(int fd, const char* message) { return send_status(fd, STATUS_ERROR_DETAIL) && send_str(fd, message); } +bool send_client_message(int fd, const char* message) { + if (!message) + message = ""; + char bounded[MAX_CLIENT_MSG_BYTES + 1]; + size_t len = strlen(message); + if (len > MAX_CLIENT_MSG_BYTES) { + memcpy(bounded, message, MAX_CLIENT_MSG_BYTES); + bounded[MAX_CLIENT_MSG_BYTES] = '\0'; + message = bounded; + } + return send_status(fd, STATUS_CLIENT_MSG) && send_str(fd, message); +} + const char* protocol_last_error(void) { return io_error_detail; } diff --git a/src/shared/protocol.h b/src/shared/protocol.h index d852fb1..4a04def 100644 --- a/src/shared/protocol.h +++ b/src/shared/protocol.h @@ -15,6 +15,12 @@ * this for a rejection and the detail frame stays a small, fixed bound. */ #define MAX_ERROR_DETAIL_BYTES 4096 +/* Hard cap on a client diagnostic forwarded over the STATUS_CLIENT_MSG channel + * (protocol 2.30.0, rsync's --stderr=client). The body is reused from the + * bounded-string wire helper and sliced to this many bytes before it is sent, + * so a peer can never be made to retain more than this per message. */ +#define MAX_CLIENT_MSG_BYTES 4096 + /* Maximum uncompressed file payload accepted by the receiver's whole-file * paths. A single whole file is charged against the per-connection memory * reservation (MAX_CONNECTION_MEMORY) and against the server allocation @@ -123,8 +129,13 @@ enum NET_STATUS { STATUS_KEEPALIVE, STATUS_ABORT, STATUS_CHECK_BATCH, - /* An explicit directory entry (--dirs): the sender transmits only the path; - * the receiver creates the directory below the receive root. */ + /* An explicit directory entry (--dirs / an empty source directory): the sender + * transmits the path and, when metadata/xattrs are negotiated, their blocks; + * the receiver creates the directory below the receive root. Protocol 2.30.0 + * inserts an int32 probe flag right after the status when report_dest_info is + * negotiated: probe=1 is a report-only frame (path only; the receiver answers + * STATUS_DEST_INFO and creates nothing), probe=0 is a real create that is + * answered with the directory's pre-transfer state before it is created. */ STATUS_MKDIR, /* --append / --append-verify tail resume. STATUS_APPEND is sent by the * receiver after a per-file STATUS_CHECK when the existing destination file @@ -207,16 +218,22 @@ enum NET_STATUS { * limit stopped deletions"). Appended after STATUS_DRY_RUN_TRANSFER so no * existing status is renumbered. */ STATUS_DELETE_LIMIT, - /* Destination-state report for output parity (protocol 2.23.0). When the - * wire config carries report_dest_info=true, the receiver answers every - * per-file STATUS_CHECK request with STATUS_DEST_INFO FIRST, followed by a - * fixed record describing the pre-transfer destination entry - * (int32 has_old; uint64 size; int64 mtime; int64 mtime_nsec; uint32 mode; - * int32 uid; int32 gid). The ordinary STATUS_OK/STATUS_NEXT/... verdict - * follows, so the sender can render rsync-accurate -i/--out-format columns - * (new vs modified, and which of size/time/perms/owner/group differ) without - * changing the transfer decision itself. Appended after - * STATUS_DELETE_LIMIT so no existing status is renumbered. */ + /* Destination-state report for output parity (protocol 2.23.0; extended to + * directories/symlinks in 2.30.0). When the wire config carries + * report_dest_info=true, the receiver answers every per-file STATUS_CHECK + * request with STATUS_DEST_INFO FIRST, followed by a fixed record describing + * the pre-transfer destination entry (int32 has_old; int32 target_matches; + * uint64 size; int64 mtime; int64 mtime_nsec; uint32 mode; int32 uid; + * int32 gid). The ordinary STATUS_OK/STATUS_NEXT/... verdict follows, so the + * sender can render rsync-accurate -i/--out-format columns (new vs modified, + * and which of size/time/perms/owner/group differ) without changing the + * transfer decision itself. Protocol 2.30.0 also uses this record for + * STATUS_MKDIR and STATUS_SYMLINK: the sender consumes it into the entry's + * dest_state before emitting its change line, and target_matches reports + * whether an existing symlink's on-disk target already equals the incoming + * one (so the sender can render `cLc........` vs `.L..t......` and suppress + * an unchanged symlink). Appended after STATUS_DELETE_LIMIT so no existing + * status is renumbered. */ STATUS_DEST_INFO, /* Per-directory delete plan (protocol 2.24.0). The sender of a * --delete-during/--delete-delay transfer streams one frame per source @@ -238,9 +255,29 @@ enum NET_STATUS { * config carries report_stats=true, the receiver sends this status once, * immediately before its terminal success status, followed by a fixed stats * record (see format_stats_send/receive in format.h) and, when the run is a - * --dry-run with --delete, the would-delete path list. Appended after - * STATUS_DELETE_PLAN so no existing status is renumbered. */ - STATUS_STATS + * --dry-run with --delete, the would-delete path list. Protocol 2.30.0 + * appends the four deleted_reg/dir/link/special counters to that record, so + * --stats can render rsync's `Number of deleted files` per-type breakdown. + * Appended after STATUS_DELETE_PLAN so no existing status is renumbered. */ + STATUS_STATS, + /* Client diagnostic channel (protocol 2.30.0, rsync's --stderr=client / + * --no-msgs2stderr). When the client's --stderr mode is `client`, the + * client forwards its own diagnostics over this client->server frame + * (STATUS_CLIENT_MSG followed by a bounded length-prefixed string, capped at + * MAX_CLIENT_MSG_BYTES) instead of writing them to its local stderr. The + * receiver reads the string and writes it to the server's stderr (respecting + * the server log destination). Appended after STATUS_STATS so no existing + * status is renumbered. */ + STATUS_CLIENT_MSG, + /* Receiver-side partial transfer (protocol 2.30.0). Sent by the receiver as + * the terminal status INSTEAD of STATUS_OK when one or more entries failed + * per-entry without aborting the stream (currently a --devices mknod + * EPERM/EACCES). The transfer otherwise succeeded and every successfully + * stored file was acknowledged, so the sender may still remove + * --remove-source-files sources; the sender maps this to rsync's exit code + * 23 ("partial transfer due to error"), distinct from a fatal STATUS_ERROR. + * Appended after STATUS_CLIENT_MSG so no existing status is renumbered. */ + STATUS_PARTIAL }; void io_set_fds(int read_fd, int write_fd); @@ -341,6 +378,11 @@ bool receive_status(int file_descriptor, Status* status); * length-prefixed string. Over-long messages are sliced and NULL is treated * as "". Returns false if the status or the string could not be sent. */ bool send_error_detail(int file_descriptor, const char* message); +/* Send STATUS_CLIENT_MSG followed by a bounded (<= MAX_CLIENT_MSG_BYTES) + * length-prefixed string carrying a client diagnostic. Over-long messages are + * sliced and NULL is treated as "". Returns false if the status or the string + * could not be sent. */ +bool send_client_message(int file_descriptor, const char* message); /* Human-readable reason captured from the most recent STATUS_ERROR_DETAIL * received on this thread, or "" when the last status was a bare STATUS_ERROR * (or no detail was seen). Thread-local, and valid until the next non-keepalive diff --git a/tests/integration/test_differential_parity.py b/tests/integration/test_differential_parity.py index a5ee3d2..b2c15af 100644 --- a/tests/integration/test_differential_parity.py +++ b/tests/integration/test_differential_parity.py @@ -17,6 +17,7 @@ Run locally:: python3 -m pytest tests/integration/test_differential_parity.py -n 4 --dist=load -m parity """ import os +import re import shutil import sys import warnings @@ -127,6 +128,66 @@ def seed_filter_protect(_src, rroot, froot): _mk(os.path.join(root, "sub", "other2.txt"), b"nested dest-only other\n", _OLD_MTIME) +def seed_perdir_protect(_src, rroot, froot): + """Per-directory `.rsync-filter` carrying `P` rules on the source and both + destinations, plus destination-only extras. The `.log` extras must survive + --delete under every timing while the other extras go; the source's + `.rsync-filter` (which FastSync carries to the receiver) and the seeded + destination one (which rsync's receiver reads) are byte-identical.""" + for root in (_src, rroot, froot): + _mk(os.path.join(root, ".rsync-filter"), b"P extra.log\nP nested.log\n") + for root in (rroot, froot): + _mk(os.path.join(root, "extra.log"), b"dest-only protected\n", _OLD_MTIME) + _mk(os.path.join(root, "other.txt"), b"dest-only deleted\n", _OLD_MTIME) + _mk(os.path.join(root, "sub", "nested.log"), b"nested protected\n", _OLD_MTIME) + _mk(os.path.join(root, "sub", "other2.txt"), b"nested deleted\n", _OLD_MTIME) + + +def seed_perdir_exclude(_src, rroot, froot): + """Per-directory unqualified exclude (`-`): dual-sided, so it protects the + matching destination-only extra (and is opted back in by + --delete-excluded).""" + for root in (_src, rroot, froot): + _mk(os.path.join(root, ".rsync-filter"), b"- extra.log\n") + for root in (rroot, froot): + _mk(os.path.join(root, "extra.log"), b"dest-only excluded\n", _OLD_MTIME) + _mk(os.path.join(root, "other.txt"), b"dest-only deleted\n", _OLD_MTIME) + + +def seed_perdir_subdir_protect(_src, rroot, froot): + """A SUBDIRECTORY-owned `.rsync-filter` (its owner is not the transfer root): + the receiver must re-derive the `P` rules in the destination-relative + coordinate system, otherwise the destination-only nested extras are wrongly + deleted (silent data loss). A root-level extra is included so a too-broad + rule would over-protect. The file is seeded on both destinations because + rsync's receiver reads the per-directory file locally for delete-during.""" + for root in (_src, rroot, froot): + _mk(os.path.join(root, "sub", ".rsync-filter"), b"P nested.log\nP extra.log\n") + for root in (rroot, froot): + _mk(os.path.join(root, "sub", "nested.log"), b"dest-only protected\n", _OLD_MTIME) + _mk(os.path.join(root, "sub", "extra.log"), b"dest-only protected 2\n", _OLD_MTIME) + _mk(os.path.join(root, "sub", "other.txt"), b"dest-only deleted\n", _OLD_MTIME) + _mk(os.path.join(root, "root_extra.txt"), b"root dest-only deleted\n", _OLD_MTIME) + + +def seed_perdir_subdir_exclude(_src, rroot, froot): + """A SUBDIRECTORY-owned unqualified exclude (`-`): dual-sided, so it protects + the matching destination-only nested extra under plain --delete and is opted + back in by --delete-excluded.""" + for root in (_src, rroot, froot): + _mk(os.path.join(root, "sub", ".rsync-filter"), b"- nested.log\n") + for root in (rroot, froot): + _mk(os.path.join(root, "sub", "nested.log"), b"dest-only excluded\n", _OLD_MTIME) + _mk(os.path.join(root, "sub", "other.txt"), b"dest-only deleted\n", _OLD_MTIME) + + +def _seed_rules(content): + def seed(_src, _rroot, _froot): + _mk(os.path.join(_src, ".rules"), content) + + return seed + + def seed_max_delete(_src, rroot, froot): for root in (rroot, froot): _mk(os.path.join(root, "extra1.txt"), b"e1\n", _OLD_MTIME) @@ -271,6 +332,81 @@ _CASES = [ ["-a", "--delete-after", "--filter=P *.log"], seed=seed_filter_protect, server_args=DELETE, ci=True, ref="--filter P/--protect under the whole-tree --delete-after commit"), + # Per-directory merge rules (#315): the receiver must re-derive the + # protect/risk verdict from the carried per-directory rules, so a + # destination-only entry matching ONLY a per-directory rule is shielded. + H.Case("filter_perdir_protect", "filters", + ["-a", "-F", "--delete"], + seed=seed_perdir_protect, server_args=DELETE, ci=True, + ref="-F per-directory P rule under the default --delete timing"), + H.Case("filter_perdir_protect_during", "filters", + ["-a", "-F", "--delete-during"], + seed=seed_perdir_protect, server_args=DELETE, ci=True, + ref="-F per-directory P rule under --delete-during"), + H.Case("filter_perdir_protect_delay", "filters", + ["-a", "-F", "--delete-delay"], + seed=seed_perdir_protect, server_args=DELETE, ci=True, + ref="-F per-directory P rule under --delete-delay"), + H.Case("filter_perdir_protect_before", "filters", + ["-a", "-F", "--delete-before"], + seed=seed_perdir_protect, server_args=DELETE, ci=True, + ref="-F per-directory P rule under the whole-tree --delete-before commit"), + H.Case("filter_perdir_protect_after", "filters", + ["-a", "-F", "--delete-after"], + seed=seed_perdir_protect, server_args=DELETE, ci=True, + ref="-F per-directory P rule under the whole-tree --delete-after commit"), + H.Case("filter_perdir_exclude_protect", "filters", + ["-a", "-F", "--delete"], + seed=seed_perdir_exclude, server_args=DELETE, ci=True, + ref="-F per-directory exclude protects its destination mirror"), + H.Case("filter_perdir_exclude_deleted", "filters", + ["-a", "-F", "--delete", "--delete-excluded"], + seed=seed_perdir_exclude, server_args=DELETE, ci=True, + ref="-F per-directory exclude under --delete-excluded is at risk"), + # #316: a rule owned by a SUBDIRECTORY (not the transfer root) must be + # re-expressed in the receiver's destination-relative coordinate system, or + # the dest-only extras it protects are silently deleted. + H.Case("filter_perdir_subdir_protect", "filters", + ["-a", "-F", "--delete"], + seed=seed_perdir_subdir_protect, server_args=DELETE, ci=True, + ref="-F subdirectory-owned P rule under the default --delete timing"), + H.Case("filter_perdir_subdir_protect_during", "filters", + ["-a", "-F", "--delete-during"], + seed=seed_perdir_subdir_protect, server_args=DELETE, ci=True, + ref="-F subdirectory-owned P rule under --delete-during"), + H.Case("filter_perdir_subdir_protect_delay", "filters", + ["-a", "-F", "--delete-delay"], + seed=seed_perdir_subdir_protect, server_args=DELETE, ci=True, + ref="-F subdirectory-owned P rule under --delete-delay"), + H.Case("filter_perdir_subdir_protect_before", "filters", + ["-a", "-F", "--delete-before"], + seed=seed_perdir_subdir_protect, server_args=DELETE, ci=True, + ref="-F subdirectory-owned P rule under the whole-tree --delete-before commit"), + H.Case("filter_perdir_subdir_protect_after", "filters", + ["-a", "-F", "--delete-after"], + seed=seed_perdir_subdir_protect, server_args=DELETE, ci=True, + ref="-F subdirectory-owned P rule under the whole-tree --delete-after commit"), + H.Case("filter_perdir_subdir_exclude_protect", "filters", + ["-a", "-F", "--delete"], + seed=seed_perdir_subdir_exclude, server_args=DELETE, ci=True, + ref="-F subdirectory-owned exclude protects its destination mirror"), + H.Case("filter_perdir_subdir_exclude_deleted", "filters", + ["-a", "-F", "--delete", "--delete-excluded"], + seed=seed_perdir_subdir_exclude, server_args=DELETE, ci=True, + ref="-F subdirectory-owned exclude under --delete-excluded is at risk"), + # Merge-file modifiers (#315): e/n/w/- semantics match rsync 3.4.1. + H.Case("dir_merge_e", "filters", ["-a", "--filter=:e .rules"], + seed=_seed_rules(b"- *.log\n"), ci=True, + ref="dir-merge,e excludes the merge file itself"), + H.Case("dir_merge_n", "filters", ["-a", "--filter=:n .rules"], + seed=_seed_rules(b"- *.log\n"), ci=True, + ref="dir-merge,n does not inherit into subdirectories"), + H.Case("dir_merge_dash", "filters", ["-a", "--filter=:- .rules"], + seed=_seed_rules(b"*.log\n*.bin\n"), ci=True, + ref="dir-merge,- reads the file as bare exclude patterns"), + H.Case("dir_merge_w", "filters", ["-a", "--filter=:-w .rules"], + seed=_seed_rules(b"*.log *.bin\n"), ci=True, + ref="dir-merge,w word-splits bare patterns on whitespace"), # --- relative / dirs -------------------------------------------------- H.Case("relative_general", "basic", ["-a", "-R"], layout=H.MIRROR_ABS, @@ -666,6 +802,157 @@ def test_added_and_deleted_between_runs(parity_server_factory): _run_and_check(case_id, result) +def _seed_dest_tree(src, root): + """Copy `src`'s tree into `root` (the transfer mirror), preserving symlinks + and directory mtimes, so a second differential run starts from an existing + destination exactly like a seeded rsync run.""" + os.makedirs(root, exist_ok=True) + for dirpath, dirnames, filenames in os.walk(src): + rel = os.path.relpath(dirpath, src) + for name in dirnames: + s = os.path.join(dirpath, name) + d = os.path.join(root, rel, name) if rel != "." else os.path.join(root, name) + if os.path.islink(s): + continue + os.makedirs(d, exist_ok=True) + for name in filenames: + s = os.path.join(dirpath, name) + d = os.path.join(root, rel, name) if rel != "." else os.path.join(root, name) + os.makedirs(os.path.dirname(d), exist_ok=True) + if os.path.islink(s): + if os.path.lexists(d): + os.remove(d) + os.symlink(os.readlink(s), d) + else: + shutil.copy2(s, d) + if rel != ".": + os.utime(os.path.join(root, rel), None) + os.utime(root, None) + + +# A full rsync itemize code (11 columns) followed by the name. H._ITEMIZE_RE +# only matches created (`+`) entries, so the changed-attribute codes this test +# asserts need their own matcher. +_ITEMIZE_LINE_RE = re.compile(r"^[<>ch.*][fdLDS].{9} ") + + +def _itemize_dir_link_lines(text): + """The itemize lines for directory and symlink entries, excluding the + transfer-root `./` line (FastSync emits it unconditionally; a documented + residual).""" + out = [] + for line in (text or "").splitlines(): + line = line.rstrip() + if not line or not _ITEMIZE_LINE_RE.match(line): + continue + name = line.rsplit(" ", 1)[-1] + if name == "./": + continue + if name.endswith("/") or " -> " in line: + out.append(line) + return sorted(out) + + +@requires_rsync +@parity +def test_itemize_rerun_dirs_symlinks_matches_rsync(parity_server_factory): + """#314: a re-run reports directory/symlink destination state like rsync. + + On an unchanged tree FastSync emits no per-directory `cd+++++++++` (or + symlink) lines, and after a changed directory mtime / symlink target it + renders rsync's `.d..t......` / `cLc........` instead of `cd`/`cL`.""" + case_id = "itemize_rerun_dirs_symlinks" + src = os.path.join(TEST_DATA_DIR, "parity_itemds_src") + rdst = os.path.join(TEST_DATA_DIR, "parity_itemds_rdst") + fdst = os.path.join(TEST_DATA_DIR, "parity_itemds_fdst") + clean_dir(src) + _mk(os.path.join(src, "sub", "b.txt"), b"nested\n") + os.makedirs(os.path.join(src, "emptydir"), exist_ok=True) + os.symlink("a.txt", os.path.join(src, "link")) + _mk(os.path.join(src, "a.txt"), b"top\n") + clean_dir(rdst) + clean_dir(fdst) + server = parity_server_factory(SUPER) + rroot = rdst + froot = get_dest_received_dir(fdst, src) + _seed_dest_tree(src, rroot) + _seed_dest_tree(src, froot) + + # Unchanged re-run: no directory or symlink itemize lines from either tool. + rs = H.run_rsync(src, rdst, ["-a", "-i"]) + fs, _ = H.run_fastsync(src, fdst, ["-a", "-i", "--incremental"], server.port) + assert rs.returncode == 0, rs.stderr + assert fs.returncode == 0, fs.stderr + assert _itemize_dir_link_lines(rs.stdout) == [] + fast_unchanged = _itemize_dir_link_lines(fs.stdout) + assert fast_unchanged == [], f"unchanged re-run itemized dirs/links: {fast_unchanged}" + + # Change the directory mtime and the symlink target, then re-run. + _pin(os.path.join(src, "sub"), _OLD_MTIME) + os.remove(os.path.join(src, "link")) + os.symlink("b.txt", os.path.join(src, "link")) + rs = H.run_rsync(src, rdst, ["-a", "-i"]) + fs, _ = H.run_fastsync(src, fdst, ["-a", "-i", "--incremental"], server.port) + assert rs.returncode == 0, rs.stderr + assert fs.returncode == 0, fs.stderr + expected = _itemize_dir_link_lines(rs.stdout) + actual = _itemize_dir_link_lines(fs.stdout) + assert actual == expected, f"rsync={rs.stdout!r} fastsync={fs.stdout!r}" + assert any(line.endswith(" sub/") and line.startswith(".d..t") for line in actual), actual + assert any(line.startswith("cLc") and " -> b.txt" in line for line in actual), actual + + +def _deleted_breakdown_line(text): + for line in (text or "").splitlines(): + if line.startswith("Number of deleted files:"): + return " ".join(line.split()) + return "" + + +@requires_rsync +@parity +def test_stats_deleted_breakdown_matches_rsync(parity_server_factory): + """#316: `--stats` renders rsync's per-type `Number of deleted files` + breakdown for removed regular files, directories, symlinks and a special.""" + case_id = "stats_deleted_breakdown" + src = os.path.join(TEST_DATA_DIR, "parity_delbd_src") + rdst = os.path.join(TEST_DATA_DIR, "parity_delbd_rdst") + fdst = os.path.join(TEST_DATA_DIR, "parity_delbd_fdst") + clean_dir(src) + _mk(os.path.join(src, "keep.txt"), b"keep\n") + server = parity_server_factory(DELETE) + + def seed(_src, rroot, froot): + for root in (rroot, froot): + _mk(os.path.join(root, "extra1.txt"), b"e1\n", _OLD_MTIME) + _mk(os.path.join(root, "extradir", "inside.txt"), b"e2\n", _OLD_MTIME) + os.makedirs(os.path.join(root, "extradir"), exist_ok=True) + link = os.path.join(root, "extralink") + if not os.path.lexists(link): + os.symlink("keep.txt", link) + fifo = os.path.join(root, "extrafifo") + if not os.path.exists(fifo): + os.mkfifo(fifo) + + def extra(_src, _rroot, _froot, rs, fs): + rs_line = _deleted_breakdown_line(rs.stdout) + fs_line = _deleted_breakdown_line(fs.stdout) + if not rs_line: + return ["rsync printed no deleted-files line"] + if rs_line != fs_line: + return [f"deleted breakdown rsync={rs_line!r} fastsync={fs_line!r}"] + if "reg:" not in rs_line or "dir:" not in rs_line or \ + "link:" not in rs_line or "special:" not in rs_line: + return [f"breakdown missing a category: {rs_line!r}"] + return [] + + result = H.run_differential( + src, rdst, fdst, ["-a", "--delete", "--stats"], + ["-a", "--delete", "--stats", "--incremental"], server, + seed=seed, extra_check=extra) + _run_and_check(case_id, result, ref="--stats deleted per-type breakdown") + + @requires_rsync @parity def test_one_file_system(parity_server_factory): diff --git a/tests/integration/test_fault_injection.py b/tests/integration/test_fault_injection.py index 8c67f6d..1509735 100644 --- a/tests/integration/test_fault_injection.py +++ b/tests/integration/test_fault_injection.py @@ -36,7 +36,7 @@ from common import ( # noqa: E402 verify_transfer, ) -PROTOCOL_VERSION = b"2.29.0" +PROTOCOL_VERSION = b"2.30.0" STATUS_MANIFEST = 5 STATUS_OK = 0 diff --git a/tests/integration/test_features.py b/tests/integration/test_features.py index 5730af0..a93cfac 100644 --- a/tests/integration/test_features.py +++ b/tests/integration/test_features.py @@ -203,9 +203,9 @@ class TestDeviceSpecial: flags=["--devices"], port=port) finally: out, err = _stop_captured_server(server) - assert result.returncode != 0, ( - f"a failed device mknod must be a transfer error like rsync (got exit 0): " - f"{(out + err)[:300]}" + assert result.returncode == 23, ( + f"a failed device mknod must exit 23 (rsync partial transfer), got " + f"{result.returncode}: {(out + err)[:300]}" ) received = get_dest_received_dir(DEVICE_DEST, DEVICE_SOURCE) assert not os.path.lexists(os.path.join(received, "chardev")), ( @@ -215,6 +215,38 @@ class TestDeviceSpecial: f"receiver did not log the device creation error: out={out!r} err={err!r}" ) + @pytest.mark.skipif(os.geteuid() != 0, reason="requires root to create device nodes") + def test_devices_nonroot_partial_removes_transferred_sources(self): + """rsync parity for a partial receiver run under --remove-source-files: + the successfully transferred regular source is still removed, the + un-creatable device source is kept, and the client exits 23 (verified + against rsync 3.4.1: it removes ok.txt/ok2.txt, keeps the device, and + exits 23).""" + if os.geteuid() != 0 or shutil.which("setpriv") is None: + pytest.skip("requires root + setpriv to run the receiver unprivileged") + self._setup() + os.mknod(os.path.join(DEVICE_SOURCE, "chardev"), stat.S_IFCHR | 0o666, + os.makedev(1, 3)) + os.makedirs(DEVICE_DEST, exist_ok=True) + os.chmod(DEVICE_DEST, 0o777) + server, port = _start_captured_server( + prefix=["setpriv", "--reuid=65534", "--regid=65534", "--clear-groups"]) + try: + result, _ = run_client(DEVICE_SOURCE, DEVICE_DEST, + flags=["--devices", "--remove-source-files"], port=port) + finally: + out, err = _stop_captured_server(server) + assert result.returncode == 23, ( + f"a partial receiver run must exit 23, got {result.returncode}: " + f"{(out + err)[:300]}" + ) + assert not os.path.exists(os.path.join(DEVICE_SOURCE, "plain.txt")), ( + "a successfully transferred source must be removed even on a partial run" + ) + assert os.path.exists(os.path.join(DEVICE_SOURCE, "chardev")), ( + "the source device that failed to materialize must be kept" + ) + @pytest.mark.skipif(os.geteuid() != 0, reason="requires root to create device nodes") def test_devices_recreates_real_char_device(self, shared_server): """Root-only: a source char device node is recreated on the destination @@ -334,6 +366,57 @@ def setup_test_data(): shutil.rmtree(DEST_DIR, ignore_errors=True) +class TestClientStderrChannel: + """--stderr=client: the client's own diagnostics go to the peer's stderr.""" + + @pytest.mark.ci + def test_client_diagnostic_reaches_server_stderr(self): + """A client-side warning emitted during the transfer is forwarded over + the STATUS_CLIENT_MSG channel and printed on the server's stderr, not the + client's. A dangling symlink under -L is the deterministic trigger.""" + source = os.path.join(TEST_DATA_DIR, "client_msg_src") + dest = os.path.join(TEST_DATA_DIR, "client_msg_dst") + clean_dir(source) + clean_dir(dest) + with open(os.path.join(source, "plain.txt"), "wb") as f: + f.write(b"payload\n") + os.symlink("no-such-referent", os.path.join(source, "dangling")) + server, port = _start_captured_server() + try: + result, _ = run_client(source, dest, flags=["-L", "--stderr=client"], + port=port) + finally: + out, err = _stop_captured_server(server) + assert "symlink has no referent" in (out + err), ( + f"client diagnostic did not reach the server stderr: out={out!r} err={err!r}" + ) + assert "symlink has no referent" not in (result.stderr or ""), ( + f"client diagnostic must not also be written locally: {result.stderr!r}" + ) + + @pytest.mark.ci + def test_no_msgs2stderr_alias_uses_client_channel(self): + """--no-msgs2stderr is rsync's spelling of --stderr=client and now + forwards the client's diagnostics to the server too.""" + source = os.path.join(TEST_DATA_DIR, "client_msg_alias_src") + dest = os.path.join(TEST_DATA_DIR, "client_msg_alias_dst") + clean_dir(source) + clean_dir(dest) + with open(os.path.join(source, "plain.txt"), "wb") as f: + f.write(b"payload\n") + os.symlink("no-such-referent", os.path.join(source, "dangling")) + server, port = _start_captured_server() + try: + result, _ = run_client(source, dest, flags=["-L", "--no-msgs2stderr"], + port=port) + finally: + out, err = _stop_captured_server(server) + assert "symlink has no referent" in (out + err), ( + f"--no-msgs2stderr did not route to the server: out={out!r} err={err!r}" + ) + assert "symlink has no referent" not in (result.stderr or "") + + class TestDryRun: def test_trust_sender_transfer_completes(self, shared_server): """--trust-sender is a receiver-local policy (never sent to the peer). @@ -2810,6 +2893,37 @@ class TestItemizeChanges: result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["-i", "--dry-run"]) assert result.returncode == 0, f"dry-run -i failed: {result.stderr[:200]}" + def test_chunk_serialization_probes_ancestor_dir_state(self, shared_server): + """#314: --chunk-serialization + -i must probe ancestor directory state + so a pre-existing directory with a changed mtime itemizes as an + attribute change (`.d..t......`) instead of being rendered as created + (`cd+++++++++`).""" + source = os.path.join(TEST_DATA_DIR, "itemize_chunk_serial_src") + dest = os.path.join(TEST_DATA_DIR, "itemize_chunk_serial_dst") + clean_dir(source) + clean_dir(dest) + subdir = os.path.join(source, "sub") + os.makedirs(subdir) + with open(os.path.join(subdir, "file.txt"), "wb") as fh: + fh.write(b"payload\n") + + result, _ = run_client(source, dest, flags=["--preserve"], port=shared_server.port) + assert result.returncode == 0, f"seed sync failed: {result.stderr[:200]}" + + # Change only the source directory's mtime; its contents stay identical + # so only the directory's time attribute differs on the rerun. + os.utime(subdir, (1_000_000_000, 1_000_000_000)) + + result, _ = run_client(source, dest, + flags=["--preserve", "-i", "--chunk-serialization"], + port=shared_server.port) + assert result.returncode == 0, f"chunk-serialization -i failed: {result.stderr[:200]}" + dir_lines = [line for line in result.stdout.splitlines() if line.endswith(" sub/")] + assert dir_lines == [".d..t...... sub/"], ( + f"expected an attribute-change dir line, got {dir_lines!r}; " + f"full stdout={result.stdout!r}" + ) + def test_changed_file_on_second_incremental_run_prints_exactly_one_line(self, shared_server): """A changed file itemizes exactly once on an incremental rerun while unchanged files print nothing (no double emission).""" diff --git a/tests/integration/test_option_parity.py b/tests/integration/test_option_parity.py index 52a2540..1143b29 100644 --- a/tests/integration/test_option_parity.py +++ b/tests/integration/test_option_parity.py @@ -536,3 +536,78 @@ class TestFilterProtect: assert "extra.log" not in result.stdout, result.stdout assert os.path.exists(os.path.join(received, "extra.log")) assert os.path.exists(os.path.join(received, "other.txt")) + + @pytest.mark.ci + def test_perdir_protect_dest_only_matches_rsync(self): + """#315: a `P` rule inside a per-directory `.rsync-filter` is carried to + the receiver, so a destination-only extra matching ONLY that rule is + shielded under --delete. Both roots carry the same filter file (rsync's + receiver reads the destination one; FastSync carries the source's).""" + source = os.path.join(TEST_DATA_DIR, "fpdp_src") + dest = os.path.join(TEST_DATA_DIR, "fpdp_dst") + rdst = os.path.join(TEST_DATA_DIR, "fpdp_rdst") + clean_dir(source) + _write(os.path.join(source, "keep.txt"), b"keep\n") + _write(os.path.join(source, ".rsync-filter"), b"P extra.log\n") + clean_dir(rdst) + _write(os.path.join(rdst, ".rsync-filter"), b"P extra.log\n") + _write(os.path.join(rdst, "extra.log"), b"extra\n") + _write(os.path.join(rdst, "other.txt"), b"other\n") + + rsync_result = _rsync(["-aF", "--delete", source + "/", rdst + "/"]) + assert rsync_result.returncode == 0, rsync_result.stderr + assert os.path.exists(os.path.join(rdst, "extra.log")), "rsync did not protect extra.log" + assert not os.path.exists(os.path.join(rdst, "other.txt")) + + clean_dir(dest) + received = get_dest_received_dir(dest, source) + os.makedirs(received, exist_ok=True) + _write(os.path.join(received, ".rsync-filter"), b"P extra.log\n") + _write(os.path.join(received, "extra.log"), b"extra\n") + _write(os.path.join(received, "other.txt"), b"other\n") + with ServerManager() as server: + server.start(extra_args=["--allow-delete"]) + result, _ = run_client(source, dest, flags=["-aF", "--delete"], port=server.port) + assert result.returncode == 0, (result.stderr or result.stdout)[:300] + assert os.path.exists(os.path.join(received, "extra.log")), ( + "FastSync must protect a destination-only per-directory P match like rsync") + assert not os.path.exists(os.path.join(received, "other.txt")) + + @requires_rsync + @pytest.mark.ci + def test_perdir_protect_dry_run_enumeration(self, shared_server): + """#315: the -n/--dry-run would-delete enumeration also honors the + carried per-directory rules, matching rsync's `*deleting` set: a + destination-only entry matching only a `.rsync-filter` P rule is not + reported (nor removed).""" + source = os.path.join(TEST_DATA_DIR, "fpdp_nd_src") + dest = os.path.join(TEST_DATA_DIR, "fpdp_nd_dst") + rdst = os.path.join(TEST_DATA_DIR, "fpdp_nd_rdst") + clean_dir(source) + _write(os.path.join(source, "keep.txt"), b"keep\n") + _write(os.path.join(source, ".rsync-filter"), b"P extra.log\n") + received = get_dest_received_dir(dest, source) + clean_dir(rdst) + clean_dir(received) + for root in (rdst, received): + _write(os.path.join(root, "keep.txt"), b"keep\n") + _write(os.path.join(root, ".rsync-filter"), b"P extra.log\n") + _write(os.path.join(root, "extra.log"), b"extra\n") + _write(os.path.join(root, "other.txt"), b"other\n") + + rsync_result = _rsync(["-an", "-i", "-F", "--delete", source + "/", rdst + "/"]) + assert rsync_result.returncode == 0, rsync_result.stderr + rsync_del = sorted(l for l in rsync_result.stdout.splitlines() + if l.startswith("*deleting")) + assert rsync_del == ["*deleting other.txt"], f"unexpected rsync set: {rsync_del}" + + with ServerManager() as server: + server.start(extra_args=["--allow-delete"]) + result, _ = run_client(source, dest, flags=["-aF", "-n", "-i", "--delete"], + port=server.port) + assert result.returncode == 0, (result.stderr or result.stdout)[:300] + fs_del = sorted(l for l in (result.stdout or "").splitlines() + if l.startswith("*deleting")) + assert fs_del == rsync_del, f"rsync={rsync_del}\nfastsync={fs_del}" + assert os.path.exists(os.path.join(received, "extra.log")) + assert os.path.exists(os.path.join(received, "other.txt")) diff --git a/tests/integration/test_preflight.py b/tests/integration/test_preflight.py index 1763800..6397ae4 100644 --- a/tests/integration/test_preflight.py +++ b/tests/integration/test_preflight.py @@ -133,14 +133,14 @@ def _seed_protocol_source(source): class TestProtocol: @pytest.mark.ci def test_protocol_current_version_accepted(self, shared_server): - """--protocol=2.29.0 (the current PROTOCOL_VERSION) is accepted and the + """--protocol=2.30.0 (the current PROTOCOL_VERSION) is accepted and the transfer completes normally.""" source = os.path.join(TEST_DATA_DIR, "proto_ok_src") dest = os.path.join(TEST_DATA_DIR, "proto_ok_dst") shutil.rmtree(dest, ignore_errors=True) os.makedirs(dest) _seed_protocol_source(source) - result, _ = run_client(source, dest, flags=["--protocol=2.29.0"], + result, _ = run_client(source, dest, flags=["--protocol=2.30.0"], port=shared_server.port) assert result.returncode == 0, \ f"--protocol current run failed: {(result.stderr or result.stdout)[:400]}" @@ -157,7 +157,7 @@ class TestProtocol: shutil.rmtree(dest, ignore_errors=True) os.makedirs(dest) _seed_protocol_source(source) - for bad in ("2.28.0", "2.27.0", "2.26.0", "2.25.0", "2.24.0", "2.23.0", "2.22.0", "2.21.0", "2.20.0", + for bad in ("2.29.0", "2.28.0", "2.27.0", "2.26.0", "2.25.0", "2.24.0", "2.23.0", "2.22.0", "2.21.0", "2.20.0", "2.19.0", "2.18.0", "2.17.0", "2.15.0", "2.16.0", "216", "31"): result, _ = run_client(source, dest, flags=[f"--protocol={bad}"], port=shared_server.port) diff --git a/tests/test_client_cli.c b/tests/test_client_cli.c index d43a9a1..49c7f02 100644 --- a/tests/test_client_cli.c +++ b/tests/test_client_cli.c @@ -352,7 +352,7 @@ static void test_parse_args_protocol_accept_current() { Config* cfg = valid_client_config(); EXPECT_NOT_NULL(cfg); char* argv_equals[] = {"fastsync", "--source-dir", "/src", - "--dest-dir", "/dst", "--protocol=2.29.0"}; + "--dest-dir", "/dst", "--protocol=2.30.0"}; int positional_args[2]; int positional_count = 0; EXPECT_EQ_INT(parse_args(cfg, 6, argv_equals, positional_args, &positional_count), 0); @@ -362,7 +362,7 @@ static void test_parse_args_protocol_accept_current() { cfg = valid_client_config(); EXPECT_NOT_NULL(cfg); char* argv_space[] = {"fastsync", "--source-dir", "/src", "--dest-dir", - "/dst", "--protocol", "2.29.0"}; + "/dst", "--protocol", "2.30.0"}; positional_count = 0; EXPECT_EQ_INT(parse_args(cfg, 7, argv_space, positional_args, &positional_count), 0); EXPECT_EQ_STR(cfg->version, PROTOCOL_VERSION); @@ -372,10 +372,10 @@ static void test_parse_args_protocol_accept_current() { /* Any --protocol value other than the current PROTOCOL_VERSION must end in * failure (parse_args simply stores it; validate_config rejects it up front). */ static void test_parse_args_protocol_rejects_other_versions() { - static const char* const bad_versions[] = {"2.17", "2.16", "2.15.0", "2.16.0", "2.17.0", - "2.18.0", "2.19.0", "2.20.0", "2.21.0", "2.22.0", - "2.23.0", "2.24.0", "2.25.0", "2.26.0", "2.27.0", - "2.28.0", "216", "31", "abc", ""}; + static const char* const bad_versions[] = { + "2.17", "2.16", "2.15.0", "2.16.0", "2.17.0", "2.18.0", "2.19.0", + "2.20.0", "2.21.0", "2.22.0", "2.23.0", "2.24.0", "2.25.0", "2.26.0", + "2.27.0", "2.28.0", "2.29.0", "216", "31", "abc", ""}; for (size_t i = 0; i < sizeof(bad_versions) / sizeof(bad_versions[0]); i++) { Config* cfg = valid_client_config(); EXPECT_NOT_NULL(cfg); @@ -2323,9 +2323,9 @@ static void test_parse_args_8_bit_output() { } static void test_parse_args_stderr_modes() { - static const char* const modes[] = {"errors", "all", "e", "a"}; - static const LogStderrMode expected[] = {LOG_STDERR_ERRORS, LOG_STDERR_ALL, LOG_STDERR_ERRORS, - LOG_STDERR_ALL}; + static const char* const modes[] = {"errors", "all", "client", "e", "a", "c"}; + static const LogStderrMode expected[] = {LOG_STDERR_ERRORS, LOG_STDERR_ALL, LOG_STDERR_CLIENT, + LOG_STDERR_ERRORS, LOG_STDERR_ALL, LOG_STDERR_CLIENT}; for (size_t i = 0; i < sizeof(modes) / sizeof(modes[0]); i++) { Config* cfg = config_create(); char option[32]; @@ -2340,8 +2340,29 @@ static void test_parse_args_stderr_modes() { log_set_stderr_mode(LOG_STDERR_ERRORS); } +/* rsync's deprecated --msgs2stderr / --no-msgs2stderr spellings map to + * --stderr=all and --stderr=client respectively; the client-message channel + * that `client` needs now exists (protocol 2.30.0). */ +static void test_parse_args_msgs2stderr_aliases() { + Config* cfg = config_create(); + char* argv_all[] = {"fastsync", "--msgs2stderr", "/src", "/dst"}; + int positional_args[2]; + int positional_count = 0; + EXPECT_EQ_INT(parse_args(cfg, 4, argv_all, positional_args, &positional_count), 0); + EXPECT_EQ_INT(log_get_stderr_mode(), LOG_STDERR_ALL); + config_delete(cfg); + + cfg = config_create(); + char* argv_client[] = {"fastsync", "--no-msgs2stderr", "/src", "/dst"}; + positional_count = 0; + EXPECT_EQ_INT(parse_args(cfg, 4, argv_client, positional_args, &positional_count), 0); + EXPECT_EQ_INT(log_get_stderr_mode(), LOG_STDERR_CLIENT); + config_delete(cfg); + log_set_stderr_mode(LOG_STDERR_ERRORS); +} + static void test_parse_args_rejects_unsupported_stderr_modes() { - static const char* const modes[] = {"client", "c", "invalid"}; + static const char* const modes[] = {"invalid", "x", ""}; for (size_t i = 0; i < sizeof(modes) / sizeof(modes[0]); i++) { Config* cfg = config_create(); char option[32]; @@ -5242,6 +5263,7 @@ void test_client_cli() { test_parse_args_ignore_times(); test_parse_args_8_bit_output(); test_parse_args_stderr_modes(); + test_parse_args_msgs2stderr_aliases(); test_parse_args_rejects_unsupported_stderr_modes(); test_parse_args_secluded_args(); test_parse_args_chunk_serialization_long_form(); diff --git a/tests/test_config.c b/tests/test_config.c index 890de8e..0b94d00 100644 --- a/tests/test_config.c +++ b/tests/test_config.c @@ -2924,7 +2924,7 @@ static void golden_config_populate(Config* c) { array_list_add(c->filters, str_dup("- /sub/dir/")); } -/* The pinned golden frame (protocol 2.29.0). The values below are the only +/* The pinned golden frame (protocol 2.30.0). The values below are the only * thing that ties the generated table to the historical wire format; update * them ONLY with a PROTOCOL_VERSION bump and a documented reason. The 2.24.0 * delete-plan wave changed only the version string; 2.25.0 appended the @@ -2936,10 +2936,13 @@ static void golden_config_populate(Config* c) { * (project decision), so the frame grew by one int to 886 bytes. The 2.29.0 * symlink-xattr wave changes only the version string: the config-frame layout * is unchanged (use_xattrs already crosses the wire); the STATUS_SYMLINK frame - * body grows instead. The byte-exact values are recomputed for the merged - * layout. */ + * body grows instead. The 2.30.0 client-message/partial wave changes only the + * version string: the config-frame layout is unchanged (the new + * STATUS_CLIENT_MSG and STATUS_PARTIAL statuses are not part of this frame), so + * the length stays 886 and only the hash moves. The byte-exact values are + * recomputed for the merged layout. */ #define GOLDEN_WIRE_LEN 886 -#define GOLDEN_WIRE_HASH 17827864270611927842ULL +#define GOLDEN_WIRE_HASH 4169866417069573876ULL static unsigned long long fnv1a_64(const unsigned char* buf, size_t len) { unsigned long long h = 1469598103934665603ULL; @@ -3021,7 +3024,7 @@ static unsigned long long capture_wire_hash(const Config* cfg, size_t* out_len) return h; } -/* Byte-for-byte wire compatibility guard (protocol 2.29.0). The expected hash +/* Byte-for-byte wire compatibility guard (protocol 2.30.0). The expected hash * pins the pre-X-macro byte stream; the refactor MUST NOT change it. */ static void test_config_wire_golden() { if (is_running_under_valgrind()) diff --git a/tests/test_delete_plan.c b/tests/test_delete_plan.c index 9d67567..5a82858 100644 --- a/tests/test_delete_plan.c +++ b/tests/test_delete_plan.c @@ -11,8 +11,23 @@ #include #include #include +#include #include +/* Discards everything written to `fd` until EOF, so a sender that regresses to + * emitting an over-budget frame does not block forever on a full socket. */ +typedef struct { + int fd; +} DrainArg; + +static int drain_fd_thread(void* arg) { + DrainArg* drain = arg; + char buffer[8192]; + while (read(drain->fd, buffer, sizeof(buffer)) > 0) + ; + return 0; +} + /* Send one STATUS_DELETE_PLAN body (the leading status is consumed by the * caller/receiver entry point) describing `dir` with no kept children. */ static void send_plan_frame(int fd, const char* dir) { @@ -225,6 +240,7 @@ static void send_config_only_frame(int fd, const char* missing_path) { EXPECT_TRUE(send_int(fd, 0)); /* size-skipped */ EXPECT_TRUE(send_int(fd, 1)); /* missing args */ EXPECT_TRUE(send_wire_str(fd, missing_path)); + EXPECT_TRUE(send_int(fd, 0)); /* per-directory filter-rule block is empty */ EXPECT_TRUE(send_int(fd, 0)); /* apply = false */ EXPECT_TRUE(send_wire_str(fd, ".")); EXPECT_TRUE(send_int(fd, 0)); @@ -265,10 +281,201 @@ static void test_config_only_frame_applies_missing_args(void) { config_delete(config); } +/* The per-directory filter-rule block (protocol 2.30.0) must be bounded on + * receive: every count, the action/sides domain, the owner-directory syntax and + * the pattern length are validated so a hostile peer can neither overread nor + * allocate unboundedly. It also round-trips a valid group faithfully. */ +static void test_filter_dir_rules_receive_bounds(void) { + int p[2]; + FilterRuleList* out = NULL; + + /* Group count beyond the cap is rejected. */ + EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0); + EXPECT_TRUE(send_int(p[1], MAX_FILTER_RULES + 1)); + EXPECT_FALSE(delete_filter_dir_rules_receive(p[0], &out)); + EXPECT_NULL(out); + close(p[0]); + close(p[1]); + + /* A negative group count is rejected. */ + EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0); + EXPECT_TRUE(send_int(p[1], -1)); + EXPECT_FALSE(delete_filter_dir_rules_receive(p[0], &out)); + EXPECT_NULL(out); + close(p[0]); + close(p[1]); + + /* An empty block is valid and yields NULL. */ + EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0); + EXPECT_TRUE(send_int(p[1], 0)); + EXPECT_TRUE(delete_filter_dir_rules_receive(p[0], &out)); + EXPECT_NULL(out); + close(p[0]); + close(p[1]); + + /* An unknown action is a protocol error. */ + EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0); + EXPECT_TRUE(send_int(p[1], 1)); + EXPECT_TRUE(send_wire_str(p[1], "")); + EXPECT_TRUE(send_int(p[1], 1)); + EXPECT_TRUE(send_int(p[1], 999)); + EXPECT_FALSE(delete_filter_dir_rules_receive(p[0], &out)); + EXPECT_NULL(out); + close(p[0]); + close(p[1]); + + /* An absolute owner directory is rejected (confinement). */ + EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0); + EXPECT_TRUE(send_int(p[1], 1)); + EXPECT_TRUE(send_wire_str(p[1], "/etc")); + EXPECT_TRUE(send_int(p[1], 0)); + EXPECT_FALSE(delete_filter_dir_rules_receive(p[0], &out)); + EXPECT_NULL(out); + close(p[0]); + close(p[1]); + + /* An over-long pattern is rejected before allocation. */ + { + char* big = malloc(MAX_PROTECT_PATTERN_LEN + 2); + EXPECT_NOT_NULL(big); + memset(big, 'a', MAX_PROTECT_PATTERN_LEN + 1); + big[MAX_PROTECT_PATTERN_LEN + 1] = '\0'; + EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0); + EXPECT_TRUE(send_int(p[1], 1)); + EXPECT_TRUE(send_wire_str(p[1], "")); + EXPECT_TRUE(send_int(p[1], 1)); + EXPECT_TRUE(send_int(p[1], (int)FILTER_ACTION_EXCLUDE)); + EXPECT_TRUE(send_int(p[1], (int)(FILTER_SIDE_SENDER | FILTER_SIDE_RECEIVER))); + EXPECT_TRUE(send_int(p[1], 0)); + EXPECT_TRUE(send_int(p[1], 0)); + EXPECT_TRUE(send_int(p[1], 0)); + EXPECT_TRUE(send_int(p[1], 0)); + EXPECT_TRUE(send_wire_str(p[1], big)); + EXPECT_FALSE(delete_filter_dir_rules_receive(p[0], &out)); + EXPECT_NULL(out); + close(p[0]); + close(p[1]); + free(big); + } + + /* A valid group round-trips its owner, no-inherit flag and pattern. */ + EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0); + EXPECT_TRUE(send_int(p[1], 1)); + EXPECT_TRUE(send_wire_str(p[1], "sub")); + EXPECT_TRUE(send_int(p[1], 1)); + EXPECT_TRUE(send_int(p[1], (int)FILTER_ACTION_EXCLUDE)); + EXPECT_TRUE(send_int(p[1], (int)(FILTER_SIDE_SENDER | FILTER_SIDE_RECEIVER))); + EXPECT_TRUE(send_int(p[1], 0)); + EXPECT_TRUE(send_int(p[1], 0)); + EXPECT_TRUE(send_int(p[1], 0)); + EXPECT_TRUE(send_int(p[1], 1)); /* no_inherit */ + EXPECT_TRUE(send_wire_str(p[1], "*.log")); + EXPECT_TRUE(delete_filter_dir_rules_receive(p[0], &out)); + EXPECT_NOT_NULL(out); + EXPECT_EQ_INT(out->count, 1); + EXPECT_EQ_STR(out->items[0]->owner, "sub"); + EXPECT_EQ_STR(out->items[0]->pattern, "*.log"); + EXPECT_TRUE(out->items[0]->no_inherit); + filter_rule_list_free(out); + close(p[0]); + close(p[1]); +} + +/* The per-directory rule sender enforces exactly the receiver's limits: an + * over-long pattern and an over-budget owner+pattern total are rejected locally + * with a clear error instead of emitting a frame the peer would abort the + * transfer on. A valid block still round-trips. */ +static void test_filter_dir_rules_send_bounds(void) { + int p[2]; + + /* An over-long pattern is rejected before anything is written. */ + { + FilterRuleList* list = filter_rule_list_create(); + EXPECT_NOT_NULL(list); + FilterRule* rule = calloc(1, sizeof(FilterRule)); + EXPECT_NOT_NULL(rule); + rule->action = FILTER_ACTION_EXCLUDE; + rule->sides = FILTER_SIDE_SENDER | FILTER_SIDE_RECEIVER; + rule->owner = str_dup("sub"); + rule->pattern = malloc(MAX_PROTECT_PATTERN_LEN + 2); + EXPECT_NOT_NULL(rule->owner); + EXPECT_NOT_NULL(rule->pattern); + memset(rule->pattern, 'a', MAX_PROTECT_PATTERN_LEN + 1); + rule->pattern[MAX_PROTECT_PATTERN_LEN + 1] = '\0'; + EXPECT_TRUE(filter_rule_list_add(list, rule)); + EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0); + EXPECT_FALSE(delete_filter_dir_rules_send(p[1], list)); + close(p[0]); + close(p[1]); + filter_rule_list_free(list); + } + + /* A cumulative owner+pattern total over MAX_FILTER_BYTES is rejected. */ + { + FilterRuleList* list = filter_rule_list_create(); + EXPECT_NOT_NULL(list); + int per = MAX_PROTECT_PATTERN_LEN; + int need = MAX_FILTER_BYTES / per + 1; + EXPECT_TRUE(need < MAX_FILTER_RULES); + for (int i = 0; i < need; i++) { + FilterRule* rule = calloc(1, sizeof(FilterRule)); + EXPECT_NOT_NULL(rule); + rule->action = FILTER_ACTION_EXCLUDE; + rule->sides = FILTER_SIDE_SENDER | FILTER_SIDE_RECEIVER; + rule->owner = str_dup(""); + rule->pattern = malloc((size_t)per + 1); + EXPECT_NOT_NULL(rule->owner); + EXPECT_NOT_NULL(rule->pattern); + memset(rule->pattern, 'b', (size_t)per); + rule->pattern[per] = '\0'; + EXPECT_TRUE(filter_rule_list_add(list, rule)); + } + EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0); + DrainArg drain = {p[0]}; + thrd_t drainer; + EXPECT_EQ_INT(thrd_create(&drainer, drain_fd_thread, &drain), thrd_success); + bool sent = delete_filter_dir_rules_send(p[1], list); + close(p[1]); + thrd_join(drainer, NULL); + EXPECT_FALSE(sent); + close(p[0]); + filter_rule_list_free(list); + } + + /* A valid block still round-trips through send -> receive. */ + { + FilterRuleList* list = filter_rule_list_create(); + EXPECT_NOT_NULL(list); + FilterRule* rule = calloc(1, sizeof(FilterRule)); + EXPECT_NOT_NULL(rule); + rule->action = FILTER_ACTION_EXCLUDE; + rule->sides = FILTER_SIDE_SENDER | FILTER_SIDE_RECEIVER; + rule->owner = str_dup("sub"); + rule->pattern = str_dup("*.log"); + EXPECT_NOT_NULL(rule->owner); + EXPECT_NOT_NULL(rule->pattern); + EXPECT_TRUE(filter_rule_list_add(list, rule)); + EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0); + EXPECT_TRUE(delete_filter_dir_rules_send(p[1], list)); + FilterRuleList* out = NULL; + EXPECT_TRUE(delete_filter_dir_rules_receive(p[0], &out)); + EXPECT_NOT_NULL(out); + EXPECT_EQ_INT(out->count, 1); + EXPECT_EQ_STR(out->items[0]->owner, "sub"); + EXPECT_EQ_STR(out->items[0]->pattern, "*.log"); + filter_rule_list_free(out); + close(p[0]); + close(p[1]); + filter_rule_list_free(list); + } +} + void test_delete_plan(void) { test_delete_delay_refilled_dir_removed_recursively(); test_delete_delay_removed_file_counted(); test_delete_delay_max_delete_bounds_actual(); test_delete_delay_actual_removal_charges_budget(); test_config_only_frame_applies_missing_args(); + test_filter_dir_rules_receive_bounds(); + test_filter_dir_rules_send_bounds(); } diff --git a/tests/test_filter.c b/tests/test_filter.c index ae426f9..f4a190b 100644 --- a/tests/test_filter.c +++ b/tests/test_filter.c @@ -1,6 +1,7 @@ #include "test_filter.h" #include "filter.h" #include "test_utils.h" +#include "utils.h" #include #include #include @@ -64,44 +65,47 @@ static void test_filter_list_rejects_unsupported_modifiers() { } /* rsync accepts the merge-file modifiers e/n/w/- on merge and dir-merge rules. - * They must be consumed so they never leak into the merge filename. */ + * They must be consumed so they never leak into the merge filename, and their + * semantics (exclude-self, no-inherit, word-split, no-prefixes) must be + * applied while the file is read. */ static void test_filter_list_accepts_merge_modifiers() { char tmpl[] = "/tmp/fastsync_filter_mmod_XXXXXX"; EXPECT_TRUE(mkdtemp(tmpl) != NULL); - char path[512]; - snprintf(path, sizeof(path), "%s/rules", tmpl); - FILE* fp = fopen(path, "w"); + char prefixed[512]; + char bare[512]; + char words[512]; + snprintf(prefixed, sizeof(prefixed), "%s/prefixed", tmpl); + snprintf(bare, sizeof(bare), "%s/bare", tmpl); + snprintf(words, sizeof(words), "%s/words", tmpl); + FILE* fp = fopen(prefixed, "w"); EXPECT_NOT_NULL(fp); fputs("- *.tmp\n", fp); fclose(fp); + fp = fopen(bare, "w"); + EXPECT_NOT_NULL(fp); + fputs("*.log\n*.tmp\n", fp); + fclose(fp); + fp = fopen(words, "w"); + EXPECT_NOT_NULL(fp); + fputs("*.log *.tmp\n", fp); + fclose(fp); - /* merge with e/n/w/- consumes the modifiers and reads the right file. */ - static const char* const fmts[] = { - "merge,e %s", "merge,n %s", "merge,w %s", "merge,- %s", ".e %s", ".- %s", - }; - for (size_t i = 0; i < sizeof(fmts) / sizeof(fmts[0]); i++) { - FilterRuleList* list = filter_rule_list_create(); - EXPECT_NOT_NULL(list); - char rule[600]; - char err[256] = ""; - snprintf(rule, sizeof(rule), fmts[i], path); - bool ok = filter_rule_list_parse_append(list, rule, NULL, NULL, err, sizeof(err)); - if (!ok) - printf(" merge rule '%s' errored: %s\n", rule, err); - EXPECT_TRUE(ok); - EXPECT_EQ_INT(list->count, 1); - EXPECT_EQ_STR(list->items[0]->pattern, "*.tmp"); - filter_rule_list_free(list); - } - - /* dir-merge with e/n/w/- registers the basename without the modifiers. */ + /* dir-merge with e/n/w/- registers the basename without the modifiers and + * records the modifier flags; 'e' appends an exclude-self rule. */ static const struct { const char* rule; const char* want; + bool no_prefixes; + bool word_split; + bool no_inherit; + bool exclude_self; } drules[] = { - {"dir-merge,e .rules", ".rules"}, {"dir-merge,n .rules", ".rules"}, - {"dir-merge,w .rules", ".rules"}, {"dir-merge,- .rules", ".rules"}, - {":e .rules", ".rules"}, {":- .rules", ".rules"}, + {"dir-merge,e .rules", ".rules", false, false, false, true}, + {"dir-merge,n .rules", ".rules", false, false, true, false}, + {"dir-merge,w .rules", ".rules", false, true, false, false}, + {"dir-merge,- .rules", ".rules", true, false, false, false}, + {":e .rules", ".rules", false, false, false, true}, + {":- .rules", ".rules", true, false, false, false}, }; for (size_t i = 0; i < sizeof(drules) / sizeof(drules[0]); i++) { FilterRuleList* list = filter_rule_list_create(); @@ -112,11 +116,78 @@ static void test_filter_list_accepts_merge_modifiers() { printf(" dir-merge rule '%s' errored: %s\n", drules[i].rule, err); EXPECT_TRUE(ok); EXPECT_EQ_INT(list->dir_merge_count, 1); - EXPECT_EQ_STR(list->dir_merge_names[0], drules[i].want); + EXPECT_EQ_STR(list->dir_merges[0].name, drules[i].want); + EXPECT_EQ_INT(list->dir_merges[0].no_prefixes, drules[i].no_prefixes); + EXPECT_EQ_INT(list->dir_merges[0].word_split, drules[i].word_split); + EXPECT_EQ_INT(list->dir_merges[0].no_inherit, drules[i].no_inherit); + EXPECT_EQ_INT(list->dir_merges[0].exclude_self, drules[i].exclude_self); + if (drules[i].exclude_self) { + EXPECT_EQ_INT(list->count, 1); + EXPECT_EQ_STR(list->items[0]->pattern, ".rules"); + EXPECT_EQ_INT(list->items[0]->action, FILTER_ACTION_EXCLUDE); + } else { + EXPECT_EQ_INT(list->count, 0); + } filter_rule_list_free(list); } - unlink(path); + /* merge,n reads the file normally; no-inherit is meaningless for a single + * merge so the rule is not marked. */ + { + FilterRuleList* list = filter_rule_list_create(); + char err[256] = ""; + char rule[600]; + snprintf(rule, sizeof(rule), "merge,n %s", prefixed); + EXPECT_TRUE(filter_rule_list_parse_append(list, rule, NULL, NULL, err, sizeof(err))); + EXPECT_EQ_INT(list->count, 1); + EXPECT_EQ_STR(list->items[0]->pattern, "*.tmp"); + EXPECT_FALSE(list->items[0]->no_inherit); + filter_rule_list_free(list); + } + + /* merge,e adds an implicit exclude for the merge file's basename. */ + { + FilterRuleList* list = filter_rule_list_create(); + char err[256] = ""; + char rule[600]; + snprintf(rule, sizeof(rule), "merge,e %s", prefixed); + EXPECT_TRUE(filter_rule_list_parse_append(list, rule, NULL, NULL, err, sizeof(err))); + EXPECT_EQ_INT(list->count, 2); + EXPECT_EQ_STR(list->items[0]->pattern, "prefixed"); + EXPECT_EQ_INT(list->items[0]->action, FILTER_ACTION_EXCLUDE); + EXPECT_EQ_STR(list->items[1]->pattern, "*.tmp"); + filter_rule_list_free(list); + } + + /* merge,- reads the file as bare exclude patterns with no prefix parsing. */ + { + FilterRuleList* list = filter_rule_list_create(); + char err[256] = ""; + char rule[600]; + snprintf(rule, sizeof(rule), "merge,- %s", bare); + EXPECT_TRUE(filter_rule_list_parse_append(list, rule, NULL, NULL, err, sizeof(err))); + EXPECT_EQ_INT(list->count, 2); + EXPECT_EQ_STR(list->items[0]->pattern, "*.log"); + EXPECT_EQ_STR(list->items[1]->pattern, "*.tmp"); + filter_rule_list_free(list); + } + + /* merge,-w word-splits bare patterns on whitespace. */ + { + FilterRuleList* list = filter_rule_list_create(); + char err[256] = ""; + char rule[600]; + snprintf(rule, sizeof(rule), "merge,w- %s", words); + EXPECT_TRUE(filter_rule_list_parse_append(list, rule, NULL, NULL, err, sizeof(err))); + EXPECT_EQ_INT(list->count, 2); + EXPECT_EQ_STR(list->items[0]->pattern, "*.log"); + EXPECT_EQ_STR(list->items[1]->pattern, "*.tmp"); + filter_rule_list_free(list); + } + + unlink(prefixed); + unlink(bare); + unlink(words); rmdir(tmpl); } @@ -187,6 +258,59 @@ static void test_filter_list_accepts_supported_rules_and_modifiers() { } } +/* A "clear"/"!" inside a merge file resets the list to empty. Rules read after + * it must still be owned by the merge file's directory (and marked no-inherit + * when the dir-merge says so). The base index must follow the clear down: when + * it was captured before the clear, post-clear rules sat below it and were left + * globally owned by "" (and unmarked). */ +static void test_filter_merge_clear_then_owner() { + char tmpl[] = "/tmp/fastsync_filter_clear_XXXXXX"; + EXPECT_TRUE(mkdtemp(tmpl) != NULL); + char path[512]; + snprintf(path, sizeof(path), "%s/.rsync-filter", tmpl); + FILE* fp = fopen(path, "w"); + EXPECT_NOT_NULL(fp); + fputs("- *.tmp\n!\nP *.log\n", fp); + fclose(fp); + + FilterRuleList* list = filter_rule_list_create(); + EXPECT_NOT_NULL(list); + char err[256] = ""; + /* A pre-existing rule that the in-file clear must discard. */ + EXPECT_TRUE(filter_rule_list_parse_append(list, "- keep.txt", NULL, NULL, err, sizeof(err))); + EXPECT_EQ_INT(list->count, 1); + + FilterDirMerge spec = {.name = ".rsync-filter", .no_inherit = true}; + bool exists = false; + EXPECT_TRUE(filter_dir_merge_append(list, tmpl, &spec, "sub", NULL, &exists, err, sizeof(err))); + EXPECT_TRUE(exists); + /* Only the post-clear rule survives, owned by "sub" and no-inherit. */ + EXPECT_EQ_INT(list->count, 1); + EXPECT_EQ_STR(list->items[0]->pattern, "*.log"); + EXPECT_EQ_STR(list->items[0]->owner, "sub"); + EXPECT_TRUE(list->items[0]->no_inherit); + filter_rule_list_free(list); + + /* A parse failure after the clear must roll the list back to the post-clear + * base (empty here), freeing the post-clear rule rather than retaining it. */ + fp = fopen(path, "w"); + EXPECT_NOT_NULL(fp); + fputs("- *.tmp\n!\nP *.log\n-e bogus\n", fp); + fclose(fp); + list = filter_rule_list_create(); + EXPECT_NOT_NULL(list); + EXPECT_TRUE(filter_rule_list_parse_append(list, "- keep.txt", NULL, NULL, err, sizeof(err))); + EXPECT_EQ_INT(list->count, 1); + exists = false; + EXPECT_FALSE(filter_dir_merge_append(list, tmpl, &spec, "sub", NULL, &exists, err, sizeof(err))); + EXPECT_TRUE(exists); + EXPECT_EQ_INT(list->count, 0); + filter_rule_list_free(list); + + unlink(path); + rmdir(tmpl); +} + static void test_filter_list_merge_file_still_supported() { char tmpl[] = "/tmp/fastsync_filter_XXXXXX"; EXPECT_TRUE(mkdtemp(tmpl) != NULL); @@ -283,12 +407,85 @@ static void test_filter_rules_apply_supported_modifiers() { } } +static FilterRule* chain_rule(const char* owner, const char* pattern, FilterAction action, + bool no_inherit) { + FilterRule* rule = calloc(1, sizeof(FilterRule)); + if (!rule) + return NULL; + rule->action = action; + rule->sides = FILTER_SIDE_SENDER | FILTER_SIDE_RECEIVER; + rule->owner = str_dup(owner); + rule->pattern = str_dup(pattern); + rule->no_inherit = no_inherit; + if (!rule->owner || !rule->pattern) { + filter_rule_free(rule); + return NULL; + } + return rule; +} + +/* The receiver's per-directory chain: a containing directory's rules win over + * an ancestor's (deepest-first), root rules are inherited, and a no-inherit + * rule applies only to its own directory's direct children. */ +static void test_filter_dir_rules_chain(void) { + FilterRuleList* list = filter_rule_list_create(); + EXPECT_NOT_NULL(list); + FilterRule* root_log = chain_rule("", "*.log", FILTER_ACTION_EXCLUDE, false); + FilterRule* sub_keep = chain_rule("sub", "keep.log", FILTER_ACTION_INCLUDE, false); + FilterRule* sub_tmp = chain_rule("sub", "*.tmp", FILTER_ACTION_EXCLUDE, true); + EXPECT_NOT_NULL(root_log); + EXPECT_NOT_NULL(sub_keep); + EXPECT_NOT_NULL(sub_tmp); + EXPECT_TRUE(filter_rule_list_add(list, root_log)); + EXPECT_TRUE(filter_rule_list_add(list, sub_keep)); + EXPECT_TRUE(filter_rule_list_add(list, sub_tmp)); + + /* Root rule inherited by every directory (leaf match). */ + EXPECT_EQ_INT(filter_dir_rules_apply_side(list, "a.log", "a.log", false), FILTER_ACTION_PROTECT); + EXPECT_EQ_INT(filter_dir_rules_apply_side(list, "sub/a.log", "a.log", false), + FILTER_ACTION_PROTECT); + /* The deeper include overrides the inherited root exclude. */ + EXPECT_EQ_INT(filter_dir_rules_apply_side(list, "sub/keep.log", "keep.log", false), + FILTER_ACTION_RISK); + /* No-inherit applies directly in its owner... */ + EXPECT_EQ_INT(filter_dir_rules_apply_side(list, "sub/x.tmp", "x.tmp", false), + FILTER_ACTION_PROTECT); + /* ...but not below it. */ + EXPECT_EQ_INT(filter_dir_rules_apply_side(list, "sub/deep/x.tmp", "x.tmp", false), + FILTER_ACTION_NONE); + /* No matching rule. */ + EXPECT_EQ_INT(filter_dir_rules_apply_side(list, "sub/deep/plain.txt", "plain.txt", false), + FILTER_ACTION_NONE); + filter_rule_list_free(list); +} + +static void test_filter_rule_clone_copies_every_field(void) { + char err[128] = ""; + FilterRule* original = filter_rule_parse("-!p /a/*.o", NULL, err, sizeof(err)); + EXPECT_NOT_NULL(original); + FilterRule* copy = filter_rule_clone(original); + EXPECT_NOT_NULL(copy); + EXPECT_TRUE(copy != original); + EXPECT_EQ_INT(copy->action, original->action); + EXPECT_EQ_INT(copy->sides, original->sides); + EXPECT_EQ_INT(copy->anchored, original->anchored); + EXPECT_EQ_INT(copy->dir_only, original->dir_only); + EXPECT_EQ_INT(copy->negate, original->negate); + EXPECT_EQ_INT(copy->perishable, original->perishable); + EXPECT_EQ_STR(copy->pattern, original->pattern); + filter_rule_free(original); + filter_rule_free(copy); +} + void test_filter() { test_filter_list_rejects_xattr_modifier(); test_filter_list_rejects_unsupported_modifiers(); test_filter_list_accepts_merge_modifiers(); test_filter_list_accepts_supported_rules_and_modifiers(); test_filter_list_merge_file_still_supported(); + test_filter_merge_clear_then_owner(); test_filter_rule_parse_rejects_unsupported_and_keeps_supported(); test_filter_rules_apply_supported_modifiers(); + test_filter_dir_rules_chain(); + test_filter_rule_clone_copies_every_field(); } diff --git a/tests/test_format.c b/tests/test_format.c index 7343e26..820fb1c 100644 --- a/tests/test_format.c +++ b/tests/test_format.c @@ -64,6 +64,7 @@ static void test_dest_state_roundtrip() { memset(&out, 0, sizeof(out)); out.known = true; out.existed = true; + out.target_matches = true; out.size = 123456789ULL; out.mtime_sec = 1700000000; out.mtime_nsec = 123456789; @@ -76,6 +77,7 @@ static void test_dest_state_roundtrip() { EXPECT_TRUE(format_dest_state_receive(fds[1], &in)); EXPECT_TRUE(in.known); EXPECT_TRUE(in.existed); + EXPECT_TRUE(in.target_matches); EXPECT_TRUE(in.size == out.size); EXPECT_TRUE(in.mtime_sec == out.mtime_sec); EXPECT_TRUE(in.mtime_nsec == out.mtime_nsec); @@ -103,6 +105,10 @@ static void test_stats_roundtrip() { out.created_dir = 4; out.created_link = 2; out.created_special = 1; + out.deleted_reg = 9; + out.deleted_dir = 6; + out.deleted_link = 3; + out.deleted_special = 2; ReceiverStats in; memset(&in, 0, sizeof(in)); EXPECT_TRUE(format_stats_send(fds[0], &out)); @@ -115,6 +121,10 @@ static void test_stats_roundtrip() { EXPECT_TRUE(in.created_dir == out.created_dir); EXPECT_TRUE(in.created_link == out.created_link); EXPECT_TRUE(in.created_special == out.created_special); + EXPECT_TRUE(in.deleted_reg == out.deleted_reg); + EXPECT_TRUE(in.deleted_dir == out.deleted_dir); + EXPECT_TRUE(in.deleted_link == out.deleted_link); + EXPECT_TRUE(in.deleted_special == out.deleted_special); close(fds[0]); close(fds[1]); } diff --git a/tests/test_log.c b/tests/test_log.c index b5acedd..59e72f0 100644 --- a/tests/test_log.c +++ b/tests/test_log.c @@ -117,6 +117,54 @@ static void test_log_stderr_mode_all() { log_set_stderr_mode(LOG_STDERR_ERRORS); } +/* --stderr=client: an installed sink takes the message body and suppresses the + * local write; a declining sink (or no sink) falls back to stderr. */ +static char g_client_msg_capture[256]; + +static bool client_msg_capture_sink(const char* message) { + snprintf(g_client_msg_capture, sizeof(g_client_msg_capture), "%s", message); + return true; +} + +static bool client_msg_decline_sink(const char* message) { + (void)message; + return false; +} + +static void test_log_stderr_mode_client() { + int pipe_fds[2]; + EXPECT_EQ_INT(pipe(pipe_fds), 0); + int saved_stderr = dup(STDERR_FILENO); + EXPECT_TRUE(saved_stderr >= 0); + EXPECT_TRUE(dup2(pipe_fds[1], STDERR_FILENO) >= 0); + close(pipe_fds[1]); + + set_log_level(LOG_LEVEL_WARNING); + log_set_stderr_mode(LOG_STDERR_CLIENT); + + g_client_msg_capture[0] = '\0'; + log_set_client_msg_sink(client_msg_capture_sink); + log_message(LOG_LEVEL_ERROR, "routed to peer %d", 7); + fflush(stderr); + EXPECT_EQ_STR(g_client_msg_capture, "routed to peer 7"); + + /* A sink that declines makes the message fall back to local stderr. */ + log_set_client_msg_sink(client_msg_decline_sink); + log_message(LOG_LEVEL_ERROR, "fallback local"); + fflush(stderr); + + log_set_client_msg_sink(NULL); + log_set_stderr_mode(LOG_STDERR_ERRORS); + EXPECT_TRUE(dup2(saved_stderr, STDERR_FILENO) >= 0); + close(saved_stderr); + char output[256] = {0}; + ssize_t length = read(pipe_fds[0], output, sizeof(output) - 1); + close(pipe_fds[0]); + EXPECT_TRUE(length > 0); + EXPECT_TRUE(strstr(output, "fallback local") != NULL); + EXPECT_TRUE(strstr(output, "routed to peer") == NULL); +} + /* Test that log_message handles various format strings */ static void test_log_message_formats() { set_log_level(LOG_LEVEL_DEBUG); @@ -261,6 +309,48 @@ static void test_log_set_file_null_before_fclose(void) { EXPECT_TRUE(true); } +/* The server's client-message channel logs a peer-controlled body: it must be + * escaped so an interior newline/CR/ANSI escape cannot forge a log line or + * move the terminal cursor. With the default stderr mode a forwarded message + * is emitted as a warning on stdout. */ +static void test_log_client_message_sanitized(void) { + int pipe_fds[2]; + EXPECT_EQ_INT(pipe(pipe_fds), 0); + /* Drain anything earlier tests buffered on stdout before redirecting, so the + capture holds only this test's single log line. */ + fflush(stdout); + int saved_stdout = dup(STDOUT_FILENO); + EXPECT_TRUE(saved_stdout >= 0); + EXPECT_TRUE(dup2(pipe_fds[1], STDOUT_FILENO) >= 0); + close(pipe_fds[1]); + + set_log_level(LOG_LEVEL_WARNING); + log_set_stderr_mode(LOG_STDERR_ERRORS); + log_client_message("forged\n2026-01-01 [ERROR]: fake\x1b[31mred"); + fflush(stdout); + + EXPECT_TRUE(dup2(saved_stdout, STDOUT_FILENO) >= 0); + close(saved_stdout); + char output[512] = {0}; + ssize_t length = read(pipe_fds[0], output, sizeof(output) - 1); + close(pipe_fds[0]); + EXPECT_TRUE(length > 0); + /* The interior newline became an escaped octal, so the body stays on one + physical line: exactly one '\n' (the log terminator) is present. */ + int newlines = 0; + for (ssize_t i = 0; i < length; i++) { + if (output[i] == '\n') + newlines++; + } + EXPECT_EQ_INT(newlines, 1); + /* The ESC introducer is escaped too, so no raw ANSI sequence reaches the + terminal. */ + EXPECT_TRUE(strchr(output, '\x1b') == NULL); + EXPECT_TRUE(strstr(output, "forged") != NULL); + EXPECT_TRUE(strstr(output, "fake") != NULL); + log_set_stderr_mode(LOG_STDERR_ERRORS); +} + void test_log() { test_log_message_debug(); test_log_message_info(); @@ -271,6 +361,8 @@ void test_log() { test_log_set_level_error(); test_log_filtering(); test_log_stderr_mode_all(); + test_log_stderr_mode_client(); + test_log_client_message_sanitized(); test_log_message_formats(); test_log_debug_enabled_matches_gate(); test_log_concurrent_no_torn_lines(); diff --git a/tests/test_multiprocessing.c b/tests/test_multiprocessing.c index 5aa3be3..7581e41 100644 --- a/tests/test_multiprocessing.c +++ b/tests/test_multiprocessing.c @@ -153,7 +153,8 @@ static void test_receiver_deleted_paths_gated_by_report_deletes() { PipelineContextReceiver* ctx_info = pipeline_context_receiver_create(info, q_info, -1, NULL); EXPECT_NOT_NULL(ctx_info); EXPECT_NOT_NULL(ctx_info->deleted_paths); - receiver_record_deleted_path(ctx_info->deleted_paths, "d/old_extra"); + ReceiverDeleteContext delctx = {NULL, ctx_info->deleted_paths}; + receiver_record_deleted_path(&delctx, "d/old_extra", DELETE_ENTRY_REG); EXPECT_EQ_INT(ctx_info->deleted_paths->size, 1); EXPECT_EQ_STR((const char*)ctx_info->deleted_paths->items[0], "d/old_extra"); pipeline_context_receiver_destroy(ctx_info); diff --git a/tests/test_protocol.c b/tests/test_protocol.c index 5852e3f..164a9fc 100644 --- a/tests/test_protocol.c +++ b/tests/test_protocol.c @@ -228,7 +228,7 @@ static void test_send_receive_status() { /* An unknown wire status outside the enum range must be rejected as a protocol * error instead of being handed to the caller as an unexpected verdict. The - * last known enumerator (STATUS_STATS) must still be accepted, proving the + * last known enumerator (STATUS_PARTIAL) must still be accepted, proving the * validation does not reject legitimate statuses. */ static void test_receive_status_rejects_unknown() { int p[2]; @@ -236,7 +236,7 @@ static void test_receive_status_rejects_unknown() { ProtocolSession session; protocol_session_init(&session, p[0], p[1]); - Status bogus = (Status)(STATUS_STATS + 1); + Status bogus = (Status)(STATUS_PARTIAL + 1); EXPECT_EQ_INT((int)write(p[1], &bogus, sizeof(bogus)), (int)sizeof(bogus)); Status received = STATUS_OK; EXPECT_FALSE(protocol_receive_status(&session, &received)); @@ -245,12 +245,12 @@ static void test_receive_status_rejects_unknown() { EXPECT_EQ_INT((int)write(p[1], &negative, sizeof(negative)), (int)sizeof(negative)); EXPECT_FALSE(protocol_receive_status(&session, &received)); - Status top = STATUS_STATS; + Status top = STATUS_PARTIAL; EXPECT_EQ_INT((int)write(p[1], &top, sizeof(top)), (int)sizeof(top)); EXPECT_TRUE(protocol_receive_status(&session, &received)); - EXPECT_EQ_INT((int)received, (int)STATUS_STATS); + EXPECT_EQ_INT((int)received, (int)STATUS_PARTIAL); - Status timed_bogus = (Status)(STATUS_STATS + 7); + Status timed_bogus = (Status)(STATUS_PARTIAL + 7); EXPECT_EQ_INT((int)write(p[1], &timed_bogus, sizeof(timed_bogus)), (int)sizeof(timed_bogus)); EXPECT_FALSE(protocol_receive_status_timed(&session, &received, 5)); @@ -258,6 +258,43 @@ static void test_receive_status_rejects_unknown() { close(p[1]); } +/* STATUS_CLIENT_MSG carries a bounded, length-prefixed diagnostic string + * (protocol 2.30.0, --stderr=client). An over-long message must be sliced to + * MAX_CLIENT_MSG_BYTES rather than sent whole. */ +static void test_send_client_message_bounded() { + int p[2]; + EXPECT_EQ_INT(pipe(p), 0); + io_set_fds(p[0], p[1]); + io_set_bwlimit(0); + + const char message[] = "client diagnostic line"; + EXPECT_TRUE(send_client_message(0, message)); + Status received = STATUS_OK; + EXPECT_TRUE(receive_status(0, &received)); + EXPECT_EQ_INT((int)received, (int)STATUS_CLIENT_MSG); + char* body = receive_str(0); + EXPECT_NOT_NULL(body); + EXPECT_EQ_STR(body, message); + free(body); + + size_t big_len = MAX_CLIENT_MSG_BYTES + 100; + char* big = malloc(big_len + 1); + EXPECT_NOT_NULL(big); + memset(big, 'x', big_len); + big[big_len] = '\0'; + EXPECT_TRUE(send_client_message(0, big)); + EXPECT_TRUE(receive_status(0, &received)); + EXPECT_EQ_INT((int)received, (int)STATUS_CLIENT_MSG); + char* big_body = receive_str(0); + EXPECT_NOT_NULL(big_body); + EXPECT_EQ_INT((int)strlen(big_body), (int)MAX_CLIENT_MSG_BYTES); + free(big_body); + free(big); + + close(p[0]); + close(p[1]); +} + static void test_receive_n_data_truncated() { int p[2]; EXPECT_EQ_INT(pipe(p), 0); @@ -1250,6 +1287,7 @@ void test_protocol() { test_send_receive_int(); test_send_receive_status(); test_receive_status_rejects_unknown(); + test_send_client_message_bounded(); test_protocol_session_io_timeout(); test_protocol_server_io_timeout_floor(); test_send_receive_status_timed(); diff --git a/tests/test_server.c b/tests/test_server.c index 849dff2..9940948 100644 --- a/tests/test_server.c +++ b/tests/test_server.c @@ -708,6 +708,7 @@ static void test_late_manifest_abort_frees_keepset() { EXPECT_TRUE(send_int(p[1], 0)); /* protected-prefix section is empty */ EXPECT_TRUE(send_int(p[1], 0)); /* missing-args section is empty */ EXPECT_TRUE(send_int(p[1], 0)); /* synchronized-directories section is empty */ + EXPECT_TRUE(send_int(p[1], 0)); /* per-directory filter-rule block is empty */ EXPECT_TRUE(send_status(p[1], STATUS_ABORT)); DeleteManifest* pending = NULL; @@ -733,6 +734,7 @@ static void test_late_manifest_eof_frees_keepset() { EXPECT_TRUE(send_int(p[1], 0)); /* protected-prefix section is empty */ EXPECT_TRUE(send_int(p[1], 0)); /* missing-args section is empty */ EXPECT_TRUE(send_int(p[1], 0)); /* synchronized-directories section is empty */ + EXPECT_TRUE(send_int(p[1], 0)); /* per-directory filter-rule block is empty */ shutdown(p[1], SHUT_WR); DeleteManifest* pending = NULL; @@ -758,12 +760,14 @@ static void test_late_second_manifest_frees_both() { EXPECT_TRUE(send_int(p[1], 0)); /* protected-prefix section is empty */ EXPECT_TRUE(send_int(p[1], 0)); /* missing-args section is empty */ EXPECT_TRUE(send_int(p[1], 0)); /* synchronized-directories section is empty */ + EXPECT_TRUE(send_int(p[1], 0)); /* per-directory filter-rule block is empty */ EXPECT_TRUE(send_status(p[1], STATUS_MANIFEST)); EXPECT_TRUE(send_int(p[1], 1)); EXPECT_TRUE(send_str(p[1], "second.txt")); EXPECT_TRUE(send_int(p[1], 0)); /* protected-prefix section is empty */ EXPECT_TRUE(send_int(p[1], 0)); /* missing-args section is empty */ EXPECT_TRUE(send_int(p[1], 0)); /* synchronized-directories section is empty */ + EXPECT_TRUE(send_int(p[1], 0)); /* per-directory filter-rule block is empty */ DeleteManifest* pending = NULL; EXPECT_EQ_INT(run_pending_receiver(cfg, p[0], &pending), -1); @@ -799,6 +803,7 @@ static void test_receive_manifest_three_sections() { EXPECT_TRUE(send_int(p[1], 2)); EXPECT_TRUE(send_str(p[1], ".")); EXPECT_TRUE(send_str(p[1], "dir")); + EXPECT_TRUE(send_int(p[1], 0)); /* per-directory filter-rule block is empty */ DeleteManifest* manifest = receive_manifest_entries(p[0]); EXPECT_NOT_NULL(manifest); @@ -917,6 +922,7 @@ static void test_receiver_pending_commits_missing_args() { EXPECT_TRUE(send_str(p[1], "gone.txt")); EXPECT_TRUE(send_str(p[1], "never_here.txt")); EXPECT_TRUE(send_int(p[1], 0)); /* no synchronized directories */ + EXPECT_TRUE(send_int(p[1], 0)); /* per-directory filter-rule block is empty */ EXPECT_TRUE(send_status(p[1], STATUS_FINISHED)); /* NULL pending: the single-threaded commit path deletes at FINISHED. The @@ -1309,6 +1315,7 @@ static void test_dry_run_delete_plan_commit_does_not_delete() { EXPECT_TRUE(send_int(p[1], 0)); /* size-skipped prefixes */ EXPECT_TRUE(send_int(p[1], 1)); /* missing-args exact deletions */ EXPECT_TRUE(send_str(p[1], "victim.txt")); + EXPECT_TRUE(send_int(p[1], 0)); /* per-directory filter-rule block is empty */ EXPECT_TRUE(send_int(p[1], 1)); /* apply: a real plan */ EXPECT_TRUE(send_str(p[1], ".")); /* receive root plan */ EXPECT_TRUE(send_int(p[1], 0)); /* kept child directories */ diff --git a/tests/test_shared_utils.c b/tests/test_shared_utils.c index 5fafa7e..25425e1 100644 --- a/tests/test_shared_utils.c +++ b/tests/test_shared_utils.c @@ -349,8 +349,9 @@ static void test_walker_protect_rules_shield_dest_only() { FilterRuleList* rules = filter_base_build(rule_text, 3, false, false, err, sizeof(err)); EXPECT_NOT_NULL(rules); size_t deleted = 0; + DeleteProtectRules protect = {.base_rules = rules, .dir_rules = NULL}; DeleteWalkResult result = - delete_extras_limited(root, manifest, NULL, 100000, NULL, 0, rules, &deleted, NULL); + delete_extras_limited(root, manifest, NULL, 100000, NULL, 0, &protect, &deleted, NULL); EXPECT_EQ_INT((int)result, (int)DELETE_WALK_OK); EXPECT_TRUE(file_exists(root, "keep.txt")); EXPECT_FALSE(file_exists(root, "extra.log")); /* risk wins the first match */