The earlier modifier-rejection change rejected e/n/w on all rules, but rsync
3.4.1 accepts them (plus the '-' merge-only modifier) on merge and dir-merge
rules. Restrict the rejection to non-merge rules and consume the merge-file
modifiers (e/n/w/-) so they no longer leak into the merge filename.
- is_merge_rule()/is_merge_modifier_char() gate the merge-only modifiers.
- scan vs consume sets: e/n/w still count as modifier-run chars on every rule
(pure tokens like -new/-press stay rejected), but are only consumed on merge
rules, preserving mixed-token parsing such as H,!secret -> ecret.
- '-' is accepted/consumed only on merge/dir-merge (e.g. dir-merge,- .rules).
- x remains rejected everywhere with its dedicated message.
- e/n/w/- semantics remain unimplemented and are documented as accepted-but-
ignored in filter.h.
Tests: split the merge forms out of the rejection test into a new acceptance
test asserting the merge file is read and dir_merge_names keeps the modifier-
free basename; non-merge pure-modifier forms still rejected.
The standalone filter_rule_parse() already rejected the rsync xattr-name
'x' modifier, but the list parser used by --filter/-f silently dropped the
flag for merge/dir-merge rules (and relied on a second parse for plain
rules). Reject it explicitly in filter_list_parse_append_depth() with the
same diagnostic, so '-x', 'merge,x' and 'dir-merge,x' all fail cleanly.
Also reject the unimplemented rsync merge modifiers 'e', 'n' and 'w'
instead of folding them into the pattern, which previously produced
misleading errors such as "could not read merge file 'n file'". Only a
token made up solely of modifier characters is treated as a modifier run,
so glued patterns ('-newfile', '-e2e') and mixed tokens ("H,!secret")
keep their historical parsing.
Adds tests/test_filter.c with focused rejection and supported-syntax
cases.
Address findings from the four-agent review of PR #298:
- file_create: zero the new File.matched_bytes. It was uninitialized
malloc memory, so the receiver could sum a garbage value into
STATUS_STATS "Matched data" (nondeterministic --stats divergence and
an uninitialized-heap disclosure on the wire).
- send_append: load the source into memory before hashing/copying the
prefix and tail. Files >64 MiB without compression (and --sendfile
runs) are streamed without loading, so --append/--append-verify
dereferenced a NULL data->data and crashed.
- filter_file_append: clamp the rollback to the surviving rule count.
A "clear" rule in a merge file frees every rule including the
caller's; the old rollback rewound count to rules_before and
resurrected freed pointers for a double free / UAF. Also roll back
when set_rule_owner fails instead of leaving owner-less rules.
- filter_rule_parse: reject the xattr-name filter modifier (x), which
was parsed and silently reinterpreted as a filename rule (affecting
what --delete protects). The p modifier stays accepted (existing
grammar test).
- Remove two dead functions: compression_default_algo and
change_render_itemize_code.
- tests: free ctx->would_delete in the two test_multiprocessing manual
teardowns (ASan leak, 1648 bytes/run).
- docs: correct the RSYNC_COMPAT/HANDOFF tally (156 rows: 106/27/23),
downgrade --info/--debug to caveat with their silent categories, add
%C-vs-xxh64 and --delete-delay count caveats, refresh stale xattr
mode comments, and document -p special-bit (setuid/setgid/sticky)
parity plus its mitigations.
Blockers addressed together (shared scanner/delete-plan plumbing):
* #10: an empty in-scope source directory produced no plan keep entry, so the
receiver deleted the destination directory itself. The scanner now records
every traversed directory into a delete-plan sink, the plan sender keeps them,
and any directory whose plan the data stream never triggered is emitted after
the data so its extras are still removed. Differential tests cover
--delete-during and --delete-delay.
* #8: an invalid per-directory filter file was silently ignored when an earlier
merge file in the same directory existed; key the failure off the error text
(both sequential and parallel scanners) and fail the scan.
* #9: -R + --files-from receiver-protect rules recorded the source-relative
path; record the bare relative wire path in both scanners so the protected
destination mirror survives --delete.
* #5: the STATUS_STATS would-delete parser now validates each retained path and
enforces the shared MAX_MANIFEST_BYTES budget, and the --out-format dry-run
delete line is escaped like the itemize line.
* #11: drop the unused DELETE_PLAN_MAX_NAMES macro, log the delete-limit
warning once per session, roll back dir-merge names from a per-directory file
that fails to parse, and guard every filter error snprintf against err==NULL.
#10 leaves the empty directory itself kept and its extras removed, matching
rsync's final state on both per-directory timings.
Read per-directory filter files through the bounded reader, guard the rule
list's capacity doubling against INT_MAX/2 overflow, and escape the local
directory path before logging a read failure.
Address an independent c-review of the files-from/filter feature:
- .rsync-filter precedence now matches rsync: evaluate the innermost
(current) directory's rules first, then ancestors, then the command-line
base (--filter/-C), so a deeper file's '+' can re-include what a shallower
'-' excluded (regression tests in both scan modes; single-thread and -m).
- --files-from: a listed entry missing on disk and an empty list are now hard
errors surfaced pre-transfer in send_files, send_files_multithreaded,
dry-run and --list-only; '.' (whole tree) and empty listed dirs stay valid.
- scanner_path_relative now handles a transfer root of / (previously the
scanner aborted on children of /).
- Reject unsupported rsync filter syntax explicitly (no silent no-ops):
+/- modifiers other than '/' (! C s r p x) and rules beginning with ':'/'.'
/'!' (merge/dir-merge/list-clear shorthands). Docs updated.
- -0/--from0 NUL mode preserves entry bytes (no CR/LF trimming); only newline
mode trims. Absolute-entry error message no longer includes the newline.
- --no-from0/--no-cvs-exclude registered as negatable booleans.
- RSYNC_COMPAT rows updated for the precedence, rejection list, NUL-mode
detail and the documented O(entries x files) scalability bound of the
allow-set (Summary unchanged: 62/3/5/1/76 = 147).
Implement the RSYNC_COMPAT Phase-2 filter/parser feature group:
- --files-from=FILE (repeatable) plus -0/--from0 NUL delimiters: parse the
source file list relative to the source root into a shared read-only
allow-set; the scanner transfers listed files and the whole subtree of
listed directories and prunes everything else in single- and
multithreaded mode. Absolute/'..' entries and missing files are hard
CLI errors.
- --filter=RULE: rsync-style +/- rules (anchored '/', dir-only trailing '/',
word include/exclude forms) evaluated first-match-wins with a default of
include, as an independent layer from legacy --exclude/--include.
Unsupported directives (merge/hide/... ) are rejected explicitly. -f stays
sendfile.
- -C/--cvs-exclude: well-known rsync CVS default exclude set.
- -F: per-directory .rsync-filter files read during traversal and applied to
the owning directory's subtree (single + parallel), never transferred.
- Delete manifest still derives from what was actually sent.
Client-only config fields; no wire/protocol change. Adds unit coverage
(CLI parse, allow-set and filter scanning single+parallel) and integration
tests (TestFilesFrom, TestFilters). RSYNC_COMPAT matrix rows updated:
5 rows move to Implemented (Summary 62/3/5/1/76 = 147).
TapTap
opencode