#291:
- Compile --exclude/--include/--exclude-from/--include-from into the SAME
ordered rule list as --filter/-f (first match wins), so the common
`--include='*.txt' --exclude='*'` idiom and include-alone semantics match
rsync. The legacy per-kind scanner arrays are no longer applied.
- -x/--one-file-system emits the cross-device mount-point directory entry
(empty) instead of dropping it, in both the sequential and parallel scanners.
- Stop passing the legacy arrays to the scanner; document -f is --filter.
#292:
- New src/shared/format.c/.h: rsync "big_num" (comma-grouped integers) and
decimal -h human sizes, %M/%t timestamp, and the STATUS_DEST_INFO codec.
- Receiver answers each STATUS_CHECK with a pre-transfer destination snapshot
(new report_dest_info wire field + STATUS_DEST_INFO, PROTOCOL_VERSION
2.23.0) so the sender can render true itemize columns.
- Itemize now emits rsync-correct update/type chars and c/s/t/p/o/g columns
for files, dirs, symlinks and hard links, comparing size/time/perms/owner/
group against the reported destination.
- --out-format gains %i %n %f %l %b %M %t %o %p %B %U %G %L; %f is the
relative display path, %M the YYYY/MM/DD-HH:MM:SS form, %b the literal
bytes sent.
- --list-only prints transfer-relative names, directory entries and ls-style
grouped sizes.
- --stats prints rsync's multi-line block on stdout; -h uses decimal units.
Tests: unit tests for the filter ordering, format primitives, itemize
columns; integration + differential tests against real rsync 3.4.1 for
itemize/out-format/list-only/selection and -x. Golden wire len/hash and
protocol version strings updated for 2.23.0.
Review fixes for Phase 7 Wave D.
#1 (HIGH): STATUS_DIR_TIMES entries no longer create directories. A new
receiver-only File.dir_time_only flag marks dir-time entries; file_save_to_disk_full
short-circuits them as FILE_SAVE_SKIPPED before any device/dir branch, so the sink
still accumulates metadata into the deferred DirTimeList but creates nothing. Empty
source dirs stay untransferred (-a), -m/--prune-empty-dirs semantics are preserved,
and a pre-existing regular file/symlink at an empty-dir mirror path no longer aborts
the transfer. dir_time_list_apply fstatat()s the leaf (AT_SYMLINK_NOFOLLOW) and skips
absent/non-directory paths QUIETLY; only a real existing directory is stamped.
Also initialize File.dir_time_only in file_create() (uninitialised garbage otherwise).
#2 (MED): send_dir_times() chunks entries into repeated STATUS_DIR_TIMES frames of at
most MAX_MANIFEST_ENTRIES, matching the receiver's per-frame bound; the tautological
> INT_MAX check is gone.
#3 (LOW): dir_time_list_add() assigns each grown array right after its realloc (no
dangling) and advances capacity only after both succeed.
#4 (LOW): RSYNC_COMPAT.md -- STATUS_MKDIR carries metadata, dir times are transmitted
via STATUS_DIR_TIMES and applied at the end, empty dirs are still never created; -m
rationale, -O row and Wave D notes updated. Summary counts untouched.
#5 (LOW): integration tests for the three #1 scenarios (empty-dir non-creation under
-a and -a -m, collision non-abort), scanner test now covers empty-dir capture, and
test_file_restore_symlink_metadata asserts the positive apply path when supported.
PROTOCOL_VERSION stays 2.17.0; config-frame layout unchanged.
Wave D of Phase 7. Make -O/--omit-dir-times and -J/--omit-link-times real by
preserving directory and symlink times, and mark --secluded-args as an explicit
Impossible/Divergence no-op.
Wire: PROTOCOL_VERSION 2.16.0 -> 2.17.0. Adds a terminal STATUS_DIR_TIMES frame
(int count + (wire path, metadata) pairs) sent after all file data and the
optional delete manifest. STATUS_MKDIR also carries metadata for --dirs entries.
Config-frame layout is unchanged.
Sender: the recursive scanner captures every traversed source directory (both
DirectoryScanner and the parallel scanner root + workers, appends mutex-guarded)
into a shared list; the single-threaded and -m paths transmit it last.
Receiver: a DirTimeList accumulates received directory metadata and applies it
with fd-relative no-follow utimensat only at the very end -- after all children,
after the commit-style --delete, and after --delay-updates publication -- in the
single-threaded success frame and in server.c after the -m threads join. -O skips
the application. Symlink metadata is applied at link creation with
utimensat/fchownat/fchmodat AT_SYMLINK_NOFOLLOW; -J suppresses only link times.
identity_apply_ownership_link shares the identity resolver with the fd path.
Docs: -O/-J rows -> Implemented; --secluded-args -> Impossible/Divergence;
--protocol accepted/rejected values and Phase-6/7 notes updated.
Tests: unit (scanner dir capture, DirTimeList apply, symlink metadata, protocol
version values) and integration (dir mtime round-trip + -O, symlink mtime
round-trip + -J, independent suppression), parameterized over single/multithread.
Unit: walker all-or-nothing bounds (exceeded -> nothing removed + distinct
result; exact bound -> deletes), protected-prefix skipping, config wire
round-trip for force_delete/delete_excluded/prune_empty_dirs/max_delete, CLI
parse/validation for the new flags. Integration (TestDeletePolicy): default
delete-excluded protection and --delete-excluded opt-out (single-thread, -m,
early --delete-before, excluded-dir subtrees), --max-delete all-or-nothing over
and at the limit, --force file-over-nonempty-dir replacement, --prune-empty-dirs
(--dirs mode + recursion-mode parity), and --ignore-errors keeping deletion
active across a genuine scan I/O error (run as an unprivileged user).
RSYNC_COMPAT Phase-2 row M: path-list construction and destination
directory creation while preserving traversal safety.
- -R/--relative with --files-from: transmit each listed entry under its
bare relative destination path (no source-root mirror). Files keep an
absolute local read path plus a separate wire/dest path (File.send_path);
manifest, incremental quick-check and change output follow the wire path,
so --delete and --remove-source-files stay consistent. -R without
--files-from is unchanged (full mirror).
- --no-implied-dirs: client-only, only meaningful with -R + --files-from.
A listed file whose parent dir is not itself (or via an ancestor)
explicitly listed cannot be placed; the run fails up front with a clear
error. No effect otherwise.
- --dirs/-d + --old-dirs/--old-d aliases: -d <dir> transmits the source
root as an explicit empty directory entry (STATUS_MKDIR frame); with
--files-from listed dirs are created empty and listed files transferred,
never descending. Works single-threaded, -m (sequential scanner in the
-m scan thread) and chunk-serialization (per-file type marker).
Directory entries appear in the delete manifest.
- --mkpath: new wire bool; server creates the destination root (and missing
leading components under its authorized root) at connection start. A
missing destination root is now rejected by default.
- Protocol bumped to 2.7.0 (mkpath wire field + STATUS_MKDIR + chunk type
marker). All receive paths funnel through file_save_to_disk_full which
creates directories via the secure confined mkdir engine; dir entries are
excluded from --remove-source-files outcome acknowledgements on both ends.
- Unit coverage: CLI parse (relative/dirs aliases/mkpath/no-implied-dirs),
config round-trip (relative + mkpath), scanner --dirs non-recursion and
-R send_path (sequential + parallel), receiver dir-entry save.
- Integration coverage: TestRelativeFilesFrom, TestNoImpliedDirs, TestDirs,
TestMkpath (single and -m).
- RSYNC_COMPAT: 4 rows move to Implemented (Summary 67/3/5/1/71 = 147).
Address an independent c-review of the files-from/filter feature:
- .rsync-filter precedence now matches rsync: evaluate the innermost
(current) directory's rules first, then ancestors, then the command-line
base (--filter/-C), so a deeper file's '+' can re-include what a shallower
'-' excluded (regression tests in both scan modes; single-thread and -m).
- --files-from: a listed entry missing on disk and an empty list are now hard
errors surfaced pre-transfer in send_files, send_files_multithreaded,
dry-run and --list-only; '.' (whole tree) and empty listed dirs stay valid.
- scanner_path_relative now handles a transfer root of / (previously the
scanner aborted on children of /).
- Reject unsupported rsync filter syntax explicitly (no silent no-ops):
+/- modifiers other than '/' (! C s r p x) and rules beginning with ':'/'.'
/'!' (merge/dir-merge/list-clear shorthands). Docs updated.
- -0/--from0 NUL mode preserves entry bytes (no CR/LF trimming); only newline
mode trims. Absolute-entry error message no longer includes the newline.
- --no-from0/--no-cvs-exclude registered as negatable booleans.
- RSYNC_COMPAT rows updated for the precedence, rejection list, NUL-mode
detail and the documented O(entries x files) scalability bound of the
allow-set (Summary unchanged: 62/3/5/1/76 = 147).
Implement the RSYNC_COMPAT Phase-2 filter/parser feature group:
- --files-from=FILE (repeatable) plus -0/--from0 NUL delimiters: parse the
source file list relative to the source root into a shared read-only
allow-set; the scanner transfers listed files and the whole subtree of
listed directories and prunes everything else in single- and
multithreaded mode. Absolute/'..' entries and missing files are hard
CLI errors.
- --filter=RULE: rsync-style +/- rules (anchored '/', dir-only trailing '/',
word include/exclude forms) evaluated first-match-wins with a default of
include, as an independent layer from legacy --exclude/--include.
Unsupported directives (merge/hide/... ) are rejected explicitly. -f stays
sendfile.
- -C/--cvs-exclude: well-known rsync CVS default exclude set.
- -F: per-directory .rsync-filter files read during traversal and applied to
the owning directory's subtree (single + parallel), never transferred.
- Delete manifest still derives from what was actually sent.
Client-only config fields; no wire/protocol change. Adds unit coverage
(CLI parse, allow-set and filter scanning single+parallel) and integration
tests (TestFilesFrom, TestFilters). RSYNC_COMPAT matrix rows updated:
5 rows move to Implemented (Summary 62/3/5/1/76 = 147).
Add a rootless unit test that reaches the actual st_dev skip branch in both
the sequential and parallel (-m) scanners: a symlink nested under the scan
root points at a directory on /dev/shm (a different device than the build
fs) and, under --copy-links semantics, -x must drop that subtree while a
plain scan includes it. Skips only when no cross-device target exists.
Integration OneFileSystem test now cleans both dest dirs up front and
reports a busy test mountpoint instead of ignoring the umount result.
RSYNC_COMPAT.md notes that cross-filesystem mount-point subdirectories are
dropped entirely (rsync parity).
Capture the transfer root's device (st_dev) at scanner creation and skip
descending into any subdirectory on a different device (a mount point).
Implemented sender/client-side only: sequential BFS and parallel (-m) root
scan apply the same scanner_same_filesystem decision; no wire/protocol change
and default behavior is unchanged. Unit tests cover the pure decision, same
device scanning in both modes, and CLI parsing; integration tests prove -x
leaves a single-filesystem tree byte-identical and, when root can mount a
tmpfs, skips a genuine cross-device subtree.
- Fix transport_tcp.c: initialize server->ssl_ctx to NULL
(prevents SSL_CTX_free on garbage when server_create_tls fails)
- Fix test_transport_tcp.c: client_create now sets fd=-1, family=AF_UNSPEC
- Fix test_transport_tcp.c: server address family may be AF_INET or AF_INET6
- Fix test_file_sendfile.c: send_path=false protocol includes file_type prefix
- Reformat all C/H files to match .clang-format (LLVM style)
- Fix 26 cppcheck const-correctness warnings (constParameterPointer,
constVariablePointer, constVariable)
- Update function declarations in headers to match const parameters
Feature changes across 15 files:
- -a/--archive: enables -c -m -M (no -s, which slows transfers)
- -n/--dry-run: scan + print without connecting or transferring
- -p <port>: custom SSH port (passed as -p to ssh via execvp)
- --exclude <pattern>: glob-based filename filtering in scanner
- --delete: sender collects file manifest; receiver deletes unlisted files
Protocol: added STATUS_MANIFEST, use_delete field in Config wire format.
New utilities: glob_match(), delete_extras() with recursive directory walk.
Scanner: accepts exclude patterns; skips matching entries.
SSH transport: switched from execlp to execvp for dynamic port arg.
Server + multiprocessing: handle STATUS_MANIFEST in both single and
multithreaded receive paths.
Builds clean, all 7 tests pass.
- Reduced test data from 50MB to 25MB (test.py)
- Increased SSH socketpair buffer to 1MB via setsockopt (transport_ssh.c)
- Made chunk size configurable: new Config field, --chunk-size CLI flag,
sent over wire, and used in scanner instead of DESIRED_CHUNK_SIZE
- Fixed DESIRED_CHUNK_SIZE macro parentheses (chunk.h)
- Remove FileReceive struct; use File everywhere with nullable FileMetadata*
- Remove struct stat from File; file size lives in Data->size (data_create_reserve)
- Add FileMetadata struct (mode, uid, gid, mtime) sent conditionally over wire
- Add -M / --preserve flag to client
- Restore permissions, ownership, timestamps on disk write
- Wire format uses per-file present flag for metadata (zero overhead when off)
TapTap