The standalone filter_rule_parse() already rejected the rsync xattr-name
'x' modifier, but the list parser used by --filter/-f silently dropped the
flag for merge/dir-merge rules (and relied on a second parse for plain
rules). Reject it explicitly in filter_list_parse_append_depth() with the
same diagnostic, so '-x', 'merge,x' and 'dir-merge,x' all fail cleanly.
Also reject the unimplemented rsync merge modifiers 'e', 'n' and 'w'
instead of folding them into the pattern, which previously produced
misleading errors such as "could not read merge file 'n file'". Only a
token made up solely of modifier characters is treated as a modifier run,
so glued patterns ('-newfile', '-e2e') and mixed tokens ("H,!secret")
keep their historical parsing.
Adds tests/test_filter.c with focused rejection and supported-syntax
cases.
Three receiver security fixes from the audit:
1. --temp-dir symlink escape (High): file_open_temp_dir() opened the
client-controlled scratch dir with a bare open(), so a symlink planted
under the receive root let a peer redirect receiver scratch files
outside the authorized root. The opened dir is now judged by the REAL
path of its fd (via /proc/self/fd), and any target outside the
authorized receive root is refused with a logged error (EACCES). An
in-root symlink (the EXDEV cross-filesystem fallback case) still works,
and the no-root local batch path is unchanged.
2. setuid/setgid/sticky under SUPER_MODE_OFF (High): the special bits were
applied under --perms (and via --chmod) even when the connection forbade
super-user activities. FileAttrPolicy gains super_permitted, set by
file_attr_policy_from_config() from privilege_super_mode_permitted();
metadata_mode_for_policy(), the symlink path, the special-node creation
path, and the deferred directory-mode apply now strip the special bits
when it is false. Exact rsync semantics are preserved when permitted.
3. daemon umask (Low): daemonize() forced umask(0), so implied parent
directories created without -p were world-writable 0777. Set the
conventional daemon umask 022 instead (rsync never forces 0); -p/-a mode
preservation is unaffected because it restores modes via fchmod.
Tests: new unit tests for file_open_temp_dir confinement and the
masked/unmasked special-bit policy (incl. the --chmod path), a daemon
world-writable-dir regression test, an integration escape test, and a
root-only integration test asserting special bits are masked without
--allow-super. The old cross-filesystem test encoded the vulnerable
behavior (symlink target outside the root) and is replaced by the escape
test; the EXDEV fallback code is retained for in-root links.
rsync 3.4.1 resolves --partial-dir after option parsing and sets keep_partial,
so --partial-dir=DIR alone retains an interrupted transfer's partial file.
FastSync only used the partial dir when --partial was also given, silently
discarding it otherwise.
Set Config->partial in cli_finalize_config whenever partial_dir is set.
Following rsync, an explicit --no-partial does NOT win (verified on rsync
3.4.1 in either option order); --inplace is guarded because it writes the
destination in place with no partial staging.
Tests: CLI unit coverage for the implication/precedence/inplace guard, and a
deterministic integration case that blocks the final install (non-empty
directory at the destination) and asserts the staged partial survives under
--partial-dir alone.
MAX_DECOMPRESSED_SIZE was 100 MiB while the receiver advertises and the
sender compresses whole files up to MAX_RECEIVE_WHOLE_FILE_SIZE (256 MiB),
so -z on a 100-256 MiB regular file failed with 'Declared decompressed
size exceeds 104857600 bytes'. Define the internal bomb-guard ceiling in
terms of the protocol constant so the two bounds cannot drift, and add
unit coverage for a 130 MiB payload (accepted) and an over-ceiling
declared size (still rejected).
--info=mount now prints rsync's mount-point skip line (matching rsync
3.4.1, which emits it for repeated -xx and drops the mount-point dir);
--info=stats enables the same block as --stats; -x is repeatable.
--stats counts traversed directories for the Number of files breakdown
even when no directory metadata is captured (-r without -t/-p).
--debug enables real output for flist/del/hash/deltasum/recv/filter/send
at their natural FastSync events (synthetic categories stay inert).
--stats and --debug rows keep their documented residual status.
Transmit the complete --delete-during/--delete-delay per-directory plan
set before the first data frame, so a mid-transfer abort has already
applied every planned removal like rsync's generator; completed runs are
unchanged. Route -d/--dirs through the same per-directory plans: the
generator records only directories whose direct children it enumerated,
so extras directly inside a listed directory are removed while an
untraversed subdirectory's mirror is shielded (rsync's -d DIR/ --delete).
Also shields a -x mount point's untraversed destination content.
Buffer and sort each directory's inspected entries (non-directories
ascending, then directories ascending) and walk them depth-first via a
LIFO directory stack, so the sequential scanner's stream matches rsync
3.4.1's flist order. This makes the --info=name transfer order and the
--delete-during/--delete-delay deletion sequence byte-identical to rsync
(differential tests in test_parity_order.py); --threads stays unordered
(no rsync analogue) and is documented as such.
Adds LIFO queue_push/queue_pop over the existing ring buffer.
Resolve a relative --compare-dest/--copy-dest/--link-dest DIR against the
destination directory and append the file's transfer-relative name, as
rsync 3.4.1 does, instead of appending FastSync's source-mirrored wire
path (the historical spelling stays as a fallback for existing layouts).
Stop inheriting the ordinary delta engine's 16 KiB minimum and 10x size
ratio in the -y/--fuzzy candidate search: rsync's find_fuzzy has no
delta-size gate, so an oversized or sub-16-KiB sibling is now reused.
The ordinary delta path's bounds are unchanged.
Collect each directory's entries up front and process extraneous
subdirectories first (descending name, depth-first), then extraneous
files (descending name), then descend into kept subdirectories in
ascending order. Emit a trailing slash for deleted directories in
observers/dry-run output. This matches rsync's delete order for
--delete-before/--delete-after/--delete-delay and for dry-run listings.
The parent sends the file data body after the receiver's STATUS_NEXT, but
the forked child exited as soon as receive_incremental_check returned. The
parent's send_data could then race the child's exit into a spurious EPIPE
(seen in the coverage job as test_server.c:1222), or the reverse: the parent
could be descheduled past the child's exit.
Keep the child alive until the parent closes its write end (drain to EOF),
and close the parent's write end before waitpid so the child can observe EOF.
Applied to the three tests sharing the pattern: size-mismatch, FIFO
destination, and FIFO basis. Child exit status remains the authoritative
assertion.
Adds a unit test proving the config-only STATUS_DELETE_PLAN frame applies
--delete-missing-args exact deletions while walking no directory (the
--files-from-with-no-synced-dir fix).
- receiver stats: exclude basis-dir materializations (--link-dest/--copy-dest)
from created/literal tallies; rsync reports 0 for a basis hit, so a fresh
--link-dest --stats run now matches (differential test_link_dest_stats_matches_rsync)
- filter wire block: reject a pattern above the glob evaluation bound and lower
MAX_FILTER_RULES to 1024, so a crafted rule list cannot amplify delete-walk
glob work or install a rule that silently never protects
- negative tests for over-cap count and over-long pattern
- README: correct --delete default, --stats/--progress description, add
--delete-commit; RSYNC_COMPAT stale version labels/overclaim fixed
- plain --delete with no timing flag now selects delete-during (progressive
deletion, matching rsync and avoiding the full old+new tree peak)
- new long-only FastSync --delete-commit restores the old atomic behavior
(delete only after the whole transfer succeeds); timing-identical to
--delete-after, implemented via the same wire bool
- timing flags are mutually exclusive; --delete-commit conflicts with other
timings; --delete-before/--delete-during rows reworded per Phase-0 probes
- CHANGELOG migration note; tally unchanged 116/14/27
Probe shows the candidate choice is observable only as --stats bandwidth
counters (tree/exit always identical). FastSync already uses rsync's
fuzzy_distance/find_filename_suffix name heuristic; the residual is the
narrower delta eligibility window (>=16 KiB, <=10x) vs rsync's wider one.
Row -> caveat; tally 116/14/27.
Adds the exclude_protect_dest_only differential and flips --delete-excluded
to parity now that receiver-side rules protect a destination-only excluded
entry like rsync; tally 116/10/31.
- new bounded config-wire block (BLOCK_PROTECT_RULES) serializes the sender's
compiled filter rules to the receiver (bounded count + 256 KiB patterns;
strict action/sides validation)
- receiver evaluates protect/risk in the whole-tree extras walk and the
per-directory delete plans, so a dest-only entry matching 'P' is kept like
rsync; dry-run would-delete enumeration also honours it
- --filter flips to parity (115/11/31); per-dir merge receiver re-derivation
remains the documented residual
Probe shows the block-checksum choice is not observable in the parity surface:
%c, Matched/Literal data and the destination tree are invariant across
xxh64/xxh128/xxh3/md5/md4/sha1 and the transfer,pre-transfer form; only %C
changes, and it is byte-identical to rsync. FastSync's fixed xxHash32 block
strong sum is collision-safe within the payload cap. Row -> parity (114/11/32).
- when --progress/--info=progress is requested, a metadata-only pre-scan
builds the full file-list total and directory names so the to-chk
denominator counts every regular/dir/link/special entry like rsync
- per-directory/symlink/special name lines emitted; sequential and --threads
- reuses the --delete-during/delay pre-scan when present; non-progress runs
take no extra pass
- --progress stays a caveat (emission order still differs); single-file output
remains byte-identical
- PROTOCOL_VERSION 2.27.0 -> 2.28.0; STATUS_STATS gains literal_bytes and
the created reg/dir/link/special counters (golden wire updated)
- receiver reports which destination entries it newly created, including
implicitly-created parent directories below the logical transfer root, so
Number of created files carries rsync's per-type breakdown
- Literal data is now exact for a delta transfer (receiver counts the literal
fragments it stored)
- differential-tested vs rsync 3.4.1 for fresh-create, update and delta
- -n/--delete sends the same protected/size-skipped/scope as a real run, so
the read-only would-delete walk no longer over-reports (row -> caveat)
- --delete-delay charges --max-delete on actual removals and recursively
re-scans a refilled deferred directory at commit; independent deferred cap
- --info=name emits the leading ./ root line and name2 'is uptodate' lines
- differential tests promoted from residual pins to rsync parity assertions
The max_delete case allowlisted the tree aspect because the surviving extras after a partial --max-delete abort are deletion-order dependent. Under FASTSYNC_PARITY_STRICT a run where the orders coincide was reported as a stale entry (CI failure), while removing the entry made the order-dependent tree mismatch fail. Add a per-case 'compare_tree' flag: max_delete now asserts rc=25 plus the survivor count via extra_check instead of exact tree identity, so the allowlist can be empty. Burn-down reached zero.
- --delete-delay: state that the reported count advances on actual removal
while --max-delete is charged at plan/snapshot time (defer_add/planned).
A new differential shows rsync instead charges on actual removals and
recursively removes a queued directory, so the row moves to Caveat
(matrix 111/13/33) and the residual is pinned by tests.
- Add a deterministic unit test (plan-time budget charge), a FastSync
integration test (byte-barrier refill + --max-delete), and an rsync
differential for a refilled deferred directory.
- Soften the --fuzzy summary: the tree is byte-exact by design, so it is
pinned by the threshold suite, not a byte-level differential.
- README: describe what -m parity actually selects; fix the allowlist
example to the real max_delete entry.
- xdist-safe delete-timing fixture names (timing and --threads mode).
- Renumber the duplicate HANDOFF item 9 to 10; add the missing final
newline to test_checksum.c.
- Expose ignore_errors_allows_delete and unit-test the deletion gate
without a privileged source directory; update the stale deleted-count
doc comment.
No production behavior changes.
- cli: remove UB in --bwlimit scaling (range-check the double product before
casting, drop atoi for the +/-1 form) and add huge/boundary unit tests
- test: widen the CI throttle wall-clock band to [1.5, 4.5]s with a 2s
cross-tolerance so a loaded runner cannot flake it
- log: drop the unused LOG_INFO_BACKUP bit; --info=backup is accepted-but-
silent like the other rsync-only categories
- client_send: remove the duplicate delete_display_path forward declaration
- utils: add non-allocating utils_strip_transfer_root and use it from
scanner_note_nonreg and delete_display_path (was duplicated logic)
- scanner: lstat() instead of stat() when re-reading an empty dir's metadata
- file: drop the no-op else-if and the redundant ELOOP arm in
file_ensure_directory_secure (symlinks are refused anyway)
- format/stats: document literal_data as whole-file accurate (delta upper
bound) instead of claiming literal bytes sent
- docs: refresh stale protocol 2.26.0 labels to 2.27.0
- Initialize PipelineContextSender.delete_suppressed=false: an uninitialized
true silently skipped the --delete keep-set manifest under -m/--threads,
so destination extras were never removed.
- Allocate/install the receiver deleted-path observer only when
report_deletes is set (--info=del / -i / --out-format under --delete), cap
the retained list at MAX_MANIFEST_ENTRIES, and free already-created lists
on the receiver-pipeline create failure path.
- Validate report_deletes/report_stats/report_dest_info on receive.
- Correct stale comments (config.h report_deletes, delete_plan.h deleted
count, multiprocessing.h stats locking, utils.h observer placement).
- Tests: sender delete_suppressed init, report_deletes gating (unit), and
-m/--delete default delete-after keep-set removal (integration).
TapTap