A sequential scanner records an opendir failure of its seed/root directory as a
skippable io_error and would complete an EMPTY scan, whose keep-set manifest
would then delete every destination entry. The seed directory that maps to the
transfer root (relative path "") is now fatal regardless of --ignore-errors;
only subdirectories discovered during an otherwise-successful root scan are
skippable. The -m path never had this hole (its root open failure aborts
scanner creation), so sequential and -m now agree.
The scanners now record every entry pruned by user-selection rules
(--filter/-C/per-dir, --exclude/--include, --max-size/--min-size) as a
destination-relative protected path on a caller-supplied sink (thread-safe in
the parallel scanner); --files-from subset pruning and -R relative wire paths
are never recorded. The sender transmits these as manifest protected prefixes,
giving rsync's default --delete behavior (excluded mirrors survive) with
--delete-excluded opting back into deleting them. --ignore-errors makes an
unreadable source directory a recorded, non-fatal scan error: the run continues,
the deletion still runs, and the exit code reports the ignored error.
--prune-empty-dirs omits an empty source directory's explicit --dirs entry.
Empty directories were never transferred by recursive scans (rsync -m parity).
scan_root_entry str_dup'd the entry name for every root-level file but only
consumed it on the -R + --files-from path, leaking 1 string per root file in
normal parallel scans (found by CI ASan/valgrind).
B1: destination-root existence/creation mishandled two legitimate forms.
file_directory_exists_secure/file_ensure_directory_secure now normalize a
trailing-slash destination (so the last component is never an empty leaf)
and treat a destination equal to the already-open authorized root as present
(no spurious <root>/<basename> nested dir with --mkpath). Confinement and
O_NOFOLLOW probing are unchanged. Regression integration tests: existing
dest with trailing slash works with and without --mkpath; dest == authorized
root works and creates no stray nested dir.
W1: chunk serialize/deserialize round-trip unit test for a directory entry
mixed with a regular file, with and without metadata; --dirs coverage under
-s and -s -m.
W2: --list-only and --dry-run now print file_wire_path() so all outputs show
the -R transformed relative name, matching -i/--out-format.
W3: RSYNC_COMPAT -d/--dirs note: dirs are created immediately under
--delay-updates (only regular files are staged).
W4: --dirs generator flushes chunks by element count too, so a long
--files-from list of empty directories cannot exceed the per-chunk file cap.
N1: STATUS_MKDIR added to status_to_string.
N2: removed dead TestMkpath._transfer.
N3: removed redundant --no-implied-dirs OPTION_TABLE row.
N4: integration tests: --dirs --delete keeps the just-created empty dir (and
deletes extras); a listed dir colliding with a regular file at the dest fails
cleanly.
RSYNC_COMPAT Phase-2 row M: path-list construction and destination
directory creation while preserving traversal safety.
- -R/--relative with --files-from: transmit each listed entry under its
bare relative destination path (no source-root mirror). Files keep an
absolute local read path plus a separate wire/dest path (File.send_path);
manifest, incremental quick-check and change output follow the wire path,
so --delete and --remove-source-files stay consistent. -R without
--files-from is unchanged (full mirror).
- --no-implied-dirs: client-only, only meaningful with -R + --files-from.
A listed file whose parent dir is not itself (or via an ancestor)
explicitly listed cannot be placed; the run fails up front with a clear
error. No effect otherwise.
- --dirs/-d + --old-dirs/--old-d aliases: -d <dir> transmits the source
root as an explicit empty directory entry (STATUS_MKDIR frame); with
--files-from listed dirs are created empty and listed files transferred,
never descending. Works single-threaded, -m (sequential scanner in the
-m scan thread) and chunk-serialization (per-file type marker).
Directory entries appear in the delete manifest.
- --mkpath: new wire bool; server creates the destination root (and missing
leading components under its authorized root) at connection start. A
missing destination root is now rejected by default.
- Protocol bumped to 2.7.0 (mkpath wire field + STATUS_MKDIR + chunk type
marker). All receive paths funnel through file_save_to_disk_full which
creates directories via the secure confined mkdir engine; dir entries are
excluded from --remove-source-files outcome acknowledgements on both ends.
- Unit coverage: CLI parse (relative/dirs aliases/mkpath/no-implied-dirs),
config round-trip (relative + mkpath), scanner --dirs non-recursion and
-R send_path (sequential + parallel), receiver dir-entry save.
- Integration coverage: TestRelativeFilesFrom, TestNoImpliedDirs, TestDirs,
TestMkpath (single and -m).
- RSYNC_COMPAT: 4 rows move to Implemented (Summary 67/3/5/1/71 = 147).
Address an independent c-review of the files-from/filter feature:
- .rsync-filter precedence now matches rsync: evaluate the innermost
(current) directory's rules first, then ancestors, then the command-line
base (--filter/-C), so a deeper file's '+' can re-include what a shallower
'-' excluded (regression tests in both scan modes; single-thread and -m).
- --files-from: a listed entry missing on disk and an empty list are now hard
errors surfaced pre-transfer in send_files, send_files_multithreaded,
dry-run and --list-only; '.' (whole tree) and empty listed dirs stay valid.
- scanner_path_relative now handles a transfer root of / (previously the
scanner aborted on children of /).
- Reject unsupported rsync filter syntax explicitly (no silent no-ops):
+/- modifiers other than '/' (! C s r p x) and rules beginning with ':'/'.'
/'!' (merge/dir-merge/list-clear shorthands). Docs updated.
- -0/--from0 NUL mode preserves entry bytes (no CR/LF trimming); only newline
mode trims. Absolute-entry error message no longer includes the newline.
- --no-from0/--no-cvs-exclude registered as negatable booleans.
- RSYNC_COMPAT rows updated for the precedence, rejection list, NUL-mode
detail and the documented O(entries x files) scalability bound of the
allow-set (Summary unchanged: 62/3/5/1/76 = 147).
Implement the RSYNC_COMPAT Phase-2 filter/parser feature group:
- --files-from=FILE (repeatable) plus -0/--from0 NUL delimiters: parse the
source file list relative to the source root into a shared read-only
allow-set; the scanner transfers listed files and the whole subtree of
listed directories and prunes everything else in single- and
multithreaded mode. Absolute/'..' entries and missing files are hard
CLI errors.
- --filter=RULE: rsync-style +/- rules (anchored '/', dir-only trailing '/',
word include/exclude forms) evaluated first-match-wins with a default of
include, as an independent layer from legacy --exclude/--include.
Unsupported directives (merge/hide/... ) are rejected explicitly. -f stays
sendfile.
- -C/--cvs-exclude: well-known rsync CVS default exclude set.
- -F: per-directory .rsync-filter files read during traversal and applied to
the owning directory's subtree (single + parallel), never transferred.
- Delete manifest still derives from what was actually sent.
Client-only config fields; no wire/protocol change. Adds unit coverage
(CLI parse, allow-set and filter scanning single+parallel) and integration
tests (TestFilesFrom, TestFilters). RSYNC_COMPAT matrix rows updated:
5 rows move to Implemented (Summary 62/3/5/1/76 = 147).
Capture the transfer root's device (st_dev) at scanner creation and skip
descending into any subdirectory on a different device (a mount point).
Implemented sender/client-side only: sequential BFS and parallel (-m) root
scan apply the same scanner_same_filesystem decision; no wire/protocol change
and default behavior is unchanged. Unit tests cover the pure decision, same
device scanning in both modes, and CLI parsing; integration tests prove -x
leaves a single-filesystem tree byte-identical and, when root can mount a
tmpfs, skips a genuine cross-device subtree.
- Add log_perror() helper (context + strerror(errno)) to the log module
- Replace all bare perror() calls with log_perror() so errors are routed
through the unified logger (stderr sink + optional --log-file sink)
- Convert fprintf(stderr, "Error:/Warning: ...") in client code to
log_message(); raw fprintf kept only for progress/stats output
- parallel_scanner_init(): result queue + sync primitive setup with unwinding
- batch_files(): root-file chunk batching, reusable by other scan paths
- scan_root_directory()/scan_root_entry(): root-dir scanning
- spawn_parallel_workers(): worker thread creation with per-thread arg setup
Main function reduced from ~230 to ~40 lines
- Restore PROTOCOL_VERSION to a forward-compatible 2.1.0 and document wire format
- Add NULL guard to config_delete
- Add pipeline cancellation flag and cancellation-aware queue helper
- Join running threads before destroying pipeline contexts on creation failure
- Fix NULL dereference and memory leaks in manifest/chunk handling
- Fix TLS/TCP socket fd leak on connect error paths
- Add compression-level range validation (1-22)
- Close previous log file before opening a new one
- Use getline for unbounded pattern-file lines
- Fix thread-unsafe localtime() and add log level bounds check
- Fix file_load_data to clean up data on read size mismatch
- Add hard ceiling to decompression buffer growth
- Fix mkdir_r bounds check and restore glob comments
- Add send_str NULL guard and mutex-protect bandwidth limiter
- Update AGENTS.md for per-thread io_ssl contract
- send_n_data: void* data → const void* data
- send_str: char* data → const char* data
- send_data: Data* data → const Data* data
- Remove duplicate typedef struct ssl_st SSL in protocol.h
- Add NULL check after malloc in directory_scanner_create
- Fix ArrayList leak in directory_scanner_next when no files found
- Remove TSan from CI matrix (Docker 'unexpected memory mapping')
- Keep ASan which now passes clean
- Reformat all C/H files to match .clang-format (LLVM style)
- Fix 26 cppcheck const-correctness warnings (constParameterPointer,
constVariablePointer, constVariable)
- Update function declarations in headers to match const parameters
Feature changes across 15 files:
- -a/--archive: enables -c -m -M (no -s, which slows transfers)
- -n/--dry-run: scan + print without connecting or transferring
- -p <port>: custom SSH port (passed as -p to ssh via execvp)
- --exclude <pattern>: glob-based filename filtering in scanner
- --delete: sender collects file manifest; receiver deletes unlisted files
Protocol: added STATUS_MANIFEST, use_delete field in Config wire format.
New utilities: glob_match(), delete_extras() with recursive directory walk.
Scanner: accepts exclude patterns; skips matching entries.
SSH transport: switched from execlp to execvp for dynamic port arg.
Server + multiprocessing: handle STATUS_MANIFEST in both single and
multithreaded receive paths.
Builds clean, all 7 tests pass.
- Reduced test data from 50MB to 25MB (test.py)
- Increased SSH socketpair buffer to 1MB via setsockopt (transport_ssh.c)
- Made chunk size configurable: new Config field, --chunk-size CLI flag,
sent over wire, and used in scanner instead of DESIRED_CHUNK_SIZE
- Fixed DESIRED_CHUNK_SIZE macro parentheses (chunk.h)
TapTap