- Scope the --delete extras walk to directories synchronized by the
transfer: add a synchronized-directory section to the delete manifest
(protocol 2.23.0) so --files-from subsets no longer delete untransmitted
paths outside listed directory subtrees (data-loss fix).
- Separate --max-size/--min-size prune protection from --delete-excluded so
size-pruned source mirrors survive (rsync parity).
- Unlink extraneous destination symlinks instead of skipping them.
- Make --max-delete partial (delete up to N, skip the rest) and exit 25;
accept negative values as unlimited.
- Draw --delete-missing-args deletions from the shared --max-delete budget.
- Honor --force during --delay-updates publication.
Add unit and integration regression tests; update the pinned config wire
golden and version strings for the 2.23.0 manifest/status additions.
#289 checksum/compression:
- -c/--checksum now implies the incremental content quick-check (without
implying -t), so an unchanged file is skipped like rsync.
- --checksum-choice/--cc accepts xxh64/xxhash, xxh3, xxh128, md5 and auto;
md4/sha1/none and the two-name form are rejected by name.
- --compress-choice/--zc rejects lz4/zlib/zlibx by name (zstd/none/auto kept).
- --checksum-seed=0 is randomized per transfer and sent on the wire.
- --skip-compress uses rsync 3.4.1's default suffix list; slash separators and
dot-less suffixes are accepted.
- add --no-whole-file.
#295 timeouts/alloc/temp-dir:
- --timeout default 0 (disabled), --contimeout default 60; 0 disables both,
plus --no-timeout/--no-contimeout.
- --max-alloc=0 means no allocation limit (was rejected).
- --temp-dir accepts any dir, requires it to exist, and falls back to a
non-atomic copy on EXDEV instead of aborting.
#296 connectivity/daemon:
- -M/--remote-option is rejected for daemon/TCP destinations (SSH-only).
- --trust-sender clarified as receiver-local; server-path tests added.
- --stop-at accepts rsync's full date form (y-m-dTh:m etc.).
Adds unit and integration coverage; no wire-field change, PROTOCOL_VERSION stays
2.22.0.
Split FastSync's single use_metadata bundle into four independent rsync-parity attributes: preserve_perms, preserve_times, preserve_owner, preserve_group. use_metadata is now a derived transport bit (config_derived_use_metadata).
CLI: real -p/--perms, -t/--times, -o/--owner, -g/--group plus --no-perms/--no-times/--no-owner/--no-group (short and long) and --no-preserve; -a is now rsync -rlptgoD; --preserve = -pt; -A implies -p; -X does not; --chmod implies -p; --usermap/--groupmap/--chown imply owner/group per side; --incremental/--delta still auto-preserve unless negated.
Receiver: per-attribute FileAttrPolicy gating for files, dirs (modes applied at end of transfer), symlinks and specials; rsync -E read-bit rule; new files get source_mode & ~umask sanitized (no group/other write); per-side identity resolution; deferred directory metadata; batch dir-metadata replay; daemon modules without 'client owner = yes' no longer refuse plain -a but force super off (no ownership) with a warning.
Wire: PROTOCOL_VERSION 2.21.0 -> 2.22.0 (four appended config bools, golden 653 / 95530566005420798). FileMetadata/chunk/batch framing unchanged. Docs/CHANGELOG/CMake updated to 2.22.0.
A remote destination's user@host token is passed to ssh in option
position, so a host beginning with '-' (e.g. -oProxyCommand=...) was
parsed by ssh as an option, allowing arbitrary command execution.
- config_parse_ssh_dest now validates the user@host prefix and returns
-1 (with a clear logged error) for an empty host or a user/host that
starts with '-'; config_parse_transport_dest propagates the failure.
- transport_ssh.c's parse_remote_dest applies the same validation as
defense-in-depth, and ssh_build_client_argv inserts a '--'
end-of-options marker before the destination token.
- Unit tests cover -oProxyCommand=... / -prefixed hosts / empty host
rejection and the argv shape.
Extend _snapshot_tree to record mode, inode, xattrs, directories and
special nodes, and add coverage proving a server-contacting --dry-run
leaves the destination structurally identical for --delay-updates,
--backup, symlinks, hardlinks, FIFOs, and daemon modules (including a
read-only module). Add a regression test for the --read-batch --dry-run
refusal and for a missing/non-directory receive root failing a dry-run
exactly like a real run.
Fix stale version comments (2.20.0/633 -> 2.21.0/637) and RSYNC_COMPAT's
current --protocol value, and add a unit assertion that
--server-port/--port (and --server-host) set the dry-run routing bit.
Today a server rejection sends a bare STATUS_ERROR and the reason only
reaches the server log, so the client cannot say why a transfer was
refused. Add an optional, bounded server->client error-detail frame:
- Status gains STATUS_ERROR_DETAIL appended LAST so existing wire
values are unchanged.
- send_error_detail(fd, msg) sends STATUS_ERROR_DETAIL followed by the
existing length-prefixed string primitive, slicing over-long messages
to MAX_ERROR_DETAIL_BYTES (4096).
- receive_status() (and the timed/keepalive status readers) always
consume the detail body and map the status back to STATUS_ERROR,
capturing the text into a thread-local buffer exposed by
protocol_last_error(); a bare STATUS_ERROR leaves it cleared. Every
existing call site keeps working and the stream cannot desync.
- Upgrade the daemon module gate / config validation (config.c), the
final transfer failure (server.c) and receiver-side path/node
validation (file_receive.c) to send a concrete reason; surface it on
the client in client_send.c/config.c.
- Bump PROTOCOL_VERSION to 2.21.0 (CMake VERSION, CHANGELOG, docs) and
update the pinned config wire golden hash / CLI-version tests.
- Add tests/test_protocol_error.c covering mapping+capture, the
over-long bound, bare-error clearing, and thread-locality.
--dry-run now handshakes with a remote/daemon receiver and reports what
WOULD transfer/skip based on receiver state, mutating nothing on either
side.
- Serialize Config.dry_run into the wire config frame and append
STATUS_DRY_RUN_TRANSFER to the status enum (no renumbering); bump
PROTOCOL_VERSION/CMake VERSION/CHANGELOG/golden wire to 2.21.0.
- Receiver: receive_incremental_check_ex runs the normal read-only
decision and answers STATUS_OK (skip) or STATUS_DRY_RUN_TRANSFER
(would transfer) with no basis materialization/append/delta/full
transfer. All mutation sites are guarded by !dry_run: file store,
manifest deletes, --mkpath root creation, --delay-updates staging,
publication, directory-time application, and outcome acks.
- Client: send_dry_run_remote connects, sends the config, checks each
regular file and prints the would-transfer set + trailer; no file data
or delete manifest is sent. Plain local destinations keep the
client-side manifest.
The config_receive_{basis,skip,idmap}_count helpers wrote the
peer-controlled int through the Config member before range-checking it.
An over-cap basis_count therefore left config->basis_count huge while
config->basis_dirs was still NULL; config_receive()'s error path then
called config_delete(), whose basis loop dereferenced NULL and crashed
the daemon before authentication.
Read each count into a local, validate, and only then assign, leaving the
member untouched on failure. config_delete() also guards the basis loop
with the array pointer as defense in depth.
Add a regression test that feeds over-cap basis/idmap/skip counts and
asserts rejection without crashing, plus a direct config_delete() check
on the partial (count set, array NULL) state.
Address low-severity review findings on the X-macro config refactor:
1. The golden test only hashed config_send_wire_block(), so a
receive-side KIND that reads a different width/order could still
round-trip symmetrically. Add test_config_wire_golden_receive():
capture the same hash-pinned 633-byte frame and feed it through
config_receive(), asserting every field (config_wire_equal) plus the
derived use_delta/use_xattrs bits and representative bounded kinds.
Add test_config_wire_receive_bounds() for bounds the symmetric
round-trip cannot reach: an out-of-range BOOL (hand-built frame),
RAW_MAXALLOC zero, a malformed STR_MODULE, an over-cap
INT_IDMAPCOUNT, and an out-of-range INT_IDENTITY chown_uid.
2. golden_config_populate() set long runs of booleans to all-1, so an
adjacent swap within a run produced identical bytes. Alternate the
boolean values and make the fixture receiver-valid (chmod grammar
"u=rwx,go=rx" is the same 11 bytes; delta_max_file_size inside the
bound). Re-pin the golden: len stays 633, hash is now
9160991280011164139 (computed, not guessed).
3. Document in config.h and client_cli.c that the CLI option tables
remain hand-maintained and are deliberately not generated from the
wire-field X-macro (client-only fields, flag/alias/negation
semantics). No CLI-table rewrite.
PROTOCOL_VERSION stays "2.20.0"; src/shared/config.c is untouched and
the wire bytes are unchanged apart from the fixture's own new values.
test_config_wire_golden() serializes a fully-populated Config through
config_send_wire_block() and pins the exact frame to len=633 and FNV-1a
hash 6163263374908258816, captured from the pre-X-macro implementation.
Any field reorder, resize or codec change fails the test.
test_config_wire_roundtrip_all_fields() serializes/deserializes a defaults
Config and a fully-populated Config over a socketpair and compares every
serialized field. The comparison is itself generated from
CONFIG_WIRE_FIELDS (one CONFIG_CMP_<KIND> per table entry), so a new table
entry automatically extends coverage; it cannot fall out of sync. It
normalizes the receiver's NULL/"" canonicalization, the max_alloc server
clamp and the derived use_delta/use_xattrs bits.
Type Config.super_mode as SuperMode (a proper C enum) instead of a bare
int. The wire boundary still carries the mode as an int: send casts the
enum explicitly and receive reads a temporary int, validates the
AUTO..OFF range, then casts. Emitted bytes and accepted values are
unchanged. ModuleGateContext.super_mode_override keeps its -1 sentinel
as int with an explicit cast at the apply site.
Behavior preserved.
- A6: escape attacker-controlled file paths and the receive root in log
lines (file_receive, server, protocol DEBUG) with output_escape()
- A8: identity_set_active() returns bool and fails closed when a requested
usermap/groupmap cannot be deep-copied; handler refuses the connection
- remove the const cast and duplicate super_mode clamp from
server_module_gate via an explicit override the handler applies once
- release the identity snapshot on the queue_create failure path
- refactor identity_parse_copy_as to a single cleanup tail and drop the
duplicated group error format specifier
A1: daemon refuses every client-chosen ownership/super-user request
(--numeric-ids/--chown/--usermap/--groupmap/--fake-super/--copy-as/--super)
unless the selected module opts in with 'client owner = yes'.
A2: fake-super owner replay requires an explicit ownership identity policy.
A3: --super no longer implies --numeric-ids (ownership stays opt-in).
A5: a failed --copy-as chown marks the entry failed instead of reporting
success with the wrong owner.
Add the receiver-side --super / --no-super tri-state (Config->super_mode)
under the safe-subset + clear-refusal privilege model: FastSync never
elevates privileges, it only permits super-user attempts that are already
confined fd-relative below the authorized receive root.
- identity: privilege_super_permitted() gate (OFF=false, ON=true, AUTO follows
geteuid()==0); identity_apply_ownership/_link become no-ops when not
permitted; --super with no explicit identity policy implies raw numeric-id
preservation (explicit usermap/groupmap/chown/numeric-ids still win); warn
exactly once when --super is requested by a non-root receiver.
- file_receive: gate char/block device-node creation on the gate; FIFO/socket
handling is unchanged.
- wire: trailing super_mode int after the --iconv spec, validated 0..2 in
receive_privilege_options and validate_received_config; PROTOCOL_VERSION
2.17.0 -> 2.18.0; version-sensitive tests and docs updated.
- CLI: --super/--no-super parsed explicitly before the generic --no-* branch
(malformed --super=x rejected); usage text added.
- tests: config wire round-trip + invalid-value rejection, privilege-gate mode
unit test, CLI parse test, integration transfer + root-gated ownership
suppression/appliance tests.
- docs: RSYNC_COMPAT --super row + Wave E note, protocol mentions, README.
Force the receiver to apply the requested owner/group to every written
entry through the confined fd-relative identity path instead of switching
the process credentials (unsafe for the multithreaded receiver). An
unprivileged receiver refuses the transfer up front in server_module_gate,
before STATUS_OK, so no data is written with the wrong ownership.
- new Config fields copy_as_set/copy_as_uid/copy_as_gid + defaults
- identity_parse_copy_as (name/@N/* resolution, primary-gid default,
gid==uid fallback for numeric ids with no passwd entry); implies -M
- identity snapshot + highest-priority forcing in identity_resolve_targets
- identity_copy_as_refused() helper
- trailing config-frame block (presence int + two int32 ids, >=0 checked)
- PROTOCOL_VERSION 2.17.0 -> 2.18.0; version-sensitive tests updated
- unit tests for parse + wire round-trip/negative-id rejection
- integration TestCopyAs: unprivileged refusal + root chown assertion
- RSYNC_COMPAT.md --copy-as row updated (safe subset + divergence); README
protocol version refreshed
Capture+transmit source atime (pre-read stat; O_NOATIME sender guard) and birth
time (statx STATX_BTIME); receiver restores atime with mtime (crtime not settable
portably -> transmitted, explicitly not applied). --open-noatime is client-only.
-O/-J documented as accepted no-ops (FastSync never preserves dir/symlink times).
Wire: metadata frame gains atime/crtime val+sec+nsec; PROTOCOL_VERSION
2.11.0->2.12.0. Review fixes: gate atime capture to Linux (no epoch clobber on
non-Linux), close fd on fdopen failure, honest -O/-J status (Compat no-op).
Source files sharing (st_dev,st_ino) are recreated as hard links on the
destination; only the first member's data crosses the wire (siblings ride a
payload-less STATUS_HARDLINK frame). Ordering requires the single-FIFO-writer
receiver + forced sequential scan (documented). link()-failure falls back to a
byte-identical local copy. Rejects -s/--append. PROTOCOL_VERSION 2.11.0->2.12.0.
Review fixes: delete the dead HardLinkRegistry (ordering holds by FIFO writer),
and --existing no longer aborts when the first member is absent but the sibling
exists (leaves the sibling in place).
Receiver allocates the destination file's full size before streaming data
(posix_fallocate preferred, ftruncate fallback on EOPNOTSUPP/ENOSYS) so an
out-of-space transfer fails fast instead of partway. Additive config bool
crossing the wire; PROTOCOL_VERSION 2.10.0 -> 2.11.0. Threaded through all
store paths (atomic, inplace, partial/delay-updates staging, link-dest copy
fallback). Review hardening: explicit lseek(0) before the data write so
correctness does not depend on posix_fallocate leaving the fd offset unchanged.
Receiver-side ownership application, opt-in and privilege-gated:
- OFF for every existing transfer (plain -M/--preserve still never applies
ownership); only triggers on an explicit identity flag + receiver permission.
- EPERM/EACCES warn-and-continue (never aborts); other fchown errors escalate.
- fd-relative fchown after the file is written (symlink-safe, confined).
- New src/shared/identity.{c,h}; config fields numeric_ids / chown uid/gid /
usermap + groupmap id-pair tables cross the wire; PROTOCOL_VERSION 2.10.0
-> 2.11.0. CLI in client_cli.c; per-connection snapshot in server.c.
- Review fixes: EPERM/EACCES-only warn-and-continue, prominent root-receiver
notice, identity_clear_active on early server error paths, --numeric-ids
kept inert standalone (removed from activation trigger set).
- CLI: --append/--append-verify acceptance (parse + imply --incremental,
validate) and incompatibility rejection with -s and --whole-file; both
removed from the unimplemented reject list.
- Config: on-the-wire append/append_verify round-trip.
- Unit: append_resume_eligible / append_tail_length pure resume math.
- Integration (test_append.py): matching-prefix resume is byte-identical and
tail-only (wire bytes << source size); --append with a wrong prefix keeps
prefix+tail (rsync parity) while --append-verify detects the mismatch and
falls back to a byte-exact full transfer; --append with --inplace and -m.
Unit: CLI parse + imply relationships (delete-missing implies ignore, not
delete; valid with --delete and delete timing); delete_missing_args config wire
round-trip (ignore_missing_args confirmed client-only); three-section manifest
round-trip and third-section traversal rejection; manifest_delete_missing_args
exact-path semantics; commit-time parking. Existing two-section manifest frames
updated to the three-section format. Integration: default missing entry is a
hard pre-transfer error; --ignore-missing-args transfers the rest and succeeds
(all-missing transfers nothing; empty list still errors); --delete-missing-args
removes exactly the missing mirror and leaves unrelated extras unless --delete is
also present; filter-exclusion protection never blocks the explicit deletion;
--delete-before early timing composes; all parametrized single-threaded vs -m.
Unit: walker all-or-nothing bounds (exceeded -> nothing removed + distinct
result; exact bound -> deletes), protected-prefix skipping, config wire
round-trip for force_delete/delete_excluded/prune_empty_dirs/max_delete, CLI
parse/validation for the new flags. Integration (TestDeletePolicy): default
delete-excluded protection and --delete-excluded opt-out (single-thread, -m,
early --delete-before, excluded-dir subtrees), --max-delete all-or-nothing over
and at the limit, --force file-over-nonempty-dir replacement, --prune-empty-dirs
(--dirs mode + recursion-mode parity), and --ignore-errors keeping deletion
active across a genuine scan I/O error (run as an unprivileged user).
Unit: parse -y/--fuzzy and --no-fuzzy negation (order-independent), -W and
--no-delta leave fuzzy inert, --fuzzy implies --incremental/--delta, and
validate_config rejects fuzzy with -s (chunk serialization) and -f
(sendfile). Config: fuzzy survives the socketpair wire round-trip.
Integration (TestFuzzy, byte counts via a new CountingProxy helper): the
rename case transfers a 2 MiB file in a few percent of its size with byte-
exact output under --fuzzy, while the no-fuzzy, no-candidate, dissimilar-
sibling and --whole-file runs send the whole file; -y alias; -m parity;
dest-holds-unsuitable-file falls back to the sibling; --delay-updates and
--remove-source-files keep their semantics on fuzzy transfers.
- unit: config basis-path normalization (trailing slash, a//b, ./x/./y collapse;
degenerate inputs rejected).
- integration: same-size/same-mtime/different-content fixtures prove the xxHash
gate -- the basis changed.txt now has the SAME byte size as the source so the
size short-circuit can no longer mask the hash comparison, plus a dedicated
parametrized same-size mismatch test asserting link/copy never use a
content-mismatched basis and compare-dest transfers.
- basis-dir priority is first-match-wins: two link-dest dirs (inode of the
first), and compare-dest before link-dest stays sparse while the reverse
order hard-links.
- --size-only links a same-content basis file with a different mtime;
--ignore-times never links even an exact match.
- --delay-updates + --delete removes extras inside a nested .fastsync-stage
dir (regression guard) while keeping the real staging dir and basis tree.
- a basis run containing a file above the whole-file limit fails up front with
a clear error and transfers nothing.
TapTap