Collect each directory's entries up front and process extraneous
subdirectories first (descending name, depth-first), then extraneous
files (descending name), then descend into kept subdirectories in
ascending order. Emit a trailing slash for deleted directories in
observers/dry-run output. This matches rsync's delete order for
--delete-before/--delete-after/--delete-delay and for dry-run listings.
- new bounded config-wire block (BLOCK_PROTECT_RULES) serializes the sender's
compiled filter rules to the receiver (bounded count + 256 KiB patterns;
strict action/sides validation)
- receiver evaluates protect/risk in the whole-tree extras walk and the
per-directory delete plans, so a dest-only entry matching 'P' is kept like
rsync; dry-run would-delete enumeration also honours it
- --filter flips to parity (115/11/31); per-dir merge receiver re-derivation
remains the documented residual
- cli: remove UB in --bwlimit scaling (range-check the double product before
casting, drop atoi for the +/-1 form) and add huge/boundary unit tests
- test: widen the CI throttle wall-clock band to [1.5, 4.5]s with a 2s
cross-tolerance so a loaded runner cannot flake it
- log: drop the unused LOG_INFO_BACKUP bit; --info=backup is accepted-but-
silent like the other rsync-only categories
- client_send: remove the duplicate delete_display_path forward declaration
- utils: add non-allocating utils_strip_transfer_root and use it from
scanner_note_nonreg and delete_display_path (was duplicated logic)
- scanner: lstat() instead of stat() when re-reading an empty dir's metadata
- file: drop the no-op else-if and the redundant ELOOP arm in
file_ensure_directory_secure (symlinks are refused anyway)
- format/stats: document literal_data as whole-file accurate (delta upper
bound) instead of claiming literal bytes sent
- docs: refresh stale protocol 2.26.0 labels to 2.27.0
- Initialize PipelineContextSender.delete_suppressed=false: an uninitialized
true silently skipped the --delete keep-set manifest under -m/--threads,
so destination extras were never removed.
- Allocate/install the receiver deleted-path observer only when
report_deletes is set (--info=del / -i / --out-format under --delete), cap
the retained list at MAX_MANIFEST_ENTRIES, and free already-created lists
on the receiver-pipeline create failure path.
- Validate report_deletes/report_stats/report_dest_info on receive.
- Correct stale comments (config.h report_deletes, delete_plan.h deleted
count, multiprocessing.h stats locking, utils.h observer placement).
- Tests: sender delete_suppressed init, report_deletes gating (unit), and
-m/--delete default delete-after keep-set removal (integration).
- Wire: config frame gains report_deletes (protocol 2.26.0 -> 2.27.0); the
receiver lists actually-removed paths in the STATUS_STATS path list, so the
sender prints rsync's `deleting PATH` / `*deleting PATH` lines for a real
--delete run (and -i/out-format). Observers threaded through the manifest,
missing-args and per-directory delete engines; golden wire len/hash updated.
- bwlimit: throttle now paces like rsync 3.4.1 -- ~100ms burst capacity and the
sleep is no longer credited as refill, so 4 MiB at 1024/2048 KiB/s matches
rsync within ~4% (was ~2x too fast).
Add the STATUS_STATS end-of-transfer receiver report (matched/deleted
counters plus a would-delete path list) behind the report_stats wire
bool, and a read-only delete_extras_list walker. --stats now renders
true wire byte totals and the receiver-reported deleted count; a
server-contacting -n --delete prints transfer-relative '*deleting' lines
matching rsync's itemize layout.
Stream one delete plan per source directory from sender to receiver instead of
a single whole-tree keep-set manifest:
- --delete-during applies each directory's extras as its plan arrives, before
that directory's data (rsync's generator-order deletion).
- --delete-delay snapshots each directory's extras while the plan arrives and
commits the removals only after a fully-successful transfer, so files created
after the scan survive (matching rsync's delete-delay, not delete-after).
- Type conflicts (a destination file blocking a source directory, or vice
versa) are cleared immediately in both modes, so the nested write succeeds.
The plan carries the destination-relative directory, its kept child directory
names and its kept child file names; the first frame also carries the global
protected prefixes, size-skipped prefixes and --delete-missing-args paths.
--delete-before keeps the existing whole-tree early manifest; plain --delete and
--delete-after keep the end-of-transfer manifest commit.
Preserves the existing safety surface: protected/size-skipped prefixes and the
--delay-updates/basis skips are honored at any depth, deletion is scoped to the
synchronized directories (--files-from), MAX_SERVER_DELETE_COUNT and
--max-delete (partial + exit 25) are shared across plans, symlinks are never
followed, and paths are confined to the receive root.
- Scope the --delete extras walk to directories synchronized by the
transfer: add a synchronized-directory section to the delete manifest
(protocol 2.23.0) so --files-from subsets no longer delete untransmitted
paths outside listed directory subtrees (data-loss fix).
- Separate --max-size/--min-size prune protection from --delete-excluded so
size-pruned source mirrors survive (rsync parity).
- Unlink extraneous destination symlinks instead of skipping them.
- Make --max-delete partial (delete up to N, skip the rest) and exit 25;
accept negative values as unlimited.
- Draw --delete-missing-args deletions from the shared --max-delete budget.
- Honor --force during --delay-updates publication.
Add unit and integration regression tests; update the pinned config wire
golden and version strings for the 2.23.0 manifest/status additions.
Replace the recursive glob matcher with an iterative O(pattern*string)
dynamic program. The old recursion explored exponentially many paths for
overlapping '*'/'**' wildcards (e.g. '*a*a*...*b' against a long run of
'a'), a CPU DoS reachable from --exclude/--include patterns and
.rsync-filter. A differential fuzz against the original matcher confirms
identical results. Doc: has_path_traversal() is a lexical '..' check only.
Add utils_getdelim_bounded(): a getdelim-style reader that never allocates
beyond UTILS_MAX_LINE_LEN, used to cap untrusted list/filter line reads.
Tests: pathological glob completes quickly; bounded reader returns EFBIG on
an over-long record.
Document on utils_get_authorized_root_path() that the returned pointer is
borrowed and invalidated by the next authorized-root setter, that the fd
and path are not read atomically (non-reentrant), and that the fd remains
caller-owned. Add a matching single-threaded/set-before-threads note at
the accessor definitions in utils.c.
In server.c, drop the redundant utils_set_authorized_root(-1, NULL) after
a failed utils_set_authorized_root(): the setter already fail-closes the
state on allocation failure. The following close(root_fd) is unchanged.
utils_fd_peer_is_local now returns true only when getpeername SUCCEEDS and the
peer address classifies as loopback. A non-socket descriptor (pipe/socketpair)
or any getpeername error is NOT local, so the daemon auth gate fails closed
instead of treating an untestable --stdio pipe as trusted (daemon auth modules
are --daemon-only and the stdio path never loads a daemon config).
server_module_gate now requires --allow-unauthenticated for the loopback
plaintext auth path: a plaintext loopback connection without the operator
opt-in is refused at the config gate BEFORE server_auth_handshake, so no SCRAM
challenge is sent. Remote peers still require verified TLS regardless of the
flag; the handler keeps its defense-in-depth checks.
Docs state the exact policy (verified TLS with matching --client-cn, or
operator-opted-in loopback plaintext), drop the SSH/stdio auth-transport claim
(they are daemon-only), and add the loopback trust-boundary relay caveat and
the CN-only (no SAN) residual. Adds a unit-test negative for pipe/socketpair
and an integration test where a relay observes no challenge when the flag is
absent.
Daemon modules that declare 'auth users' no longer accept credentials over a
remote plaintext connection: server_module_gate refuses at the config gate,
before any SCRAM challenge is sent, unless the connection is verified TLS with
a client certificate matching --client-cn, or a local/SSH transport (loopback
TCP peer or the --stdio pipe). --allow-unauthenticated does not relax this.
The TLS client-CN comparison now uses credentials_secure_equal (S2). Clients
sending --password-file to a non-loopback daemon must use --tls; validate_config
rejects the plaintext case before any network I/O.
Adds utils_sockaddr_is_loopback / utils_fd_peer_is_local / utils_host_is_loopback
helpers with unit tests, a client validation unit test, and integration tests
for the client-side plaintext rejection and the wrong-CN gate refusal.
Implement rsync's append modes: when an existing destination file is SHORTER
than the source, the receiver negotiates a resume offset and only the tail is
transferred; the full file (retained prefix + tail) is rebuilt and installed
through the normal atomic store path, so the result is byte-identical to the
source whenever the prefix matches.
- --append: sends the tail without content-verifying the retained prefix
(rsync parity; the documented prefix-trust risk).
- --append-verify: verifies the retained prefix against the source's prefix
xxHash64 before appending and, on a mismatch, falls back to a clean full
transfer (never a corrupt prefix+tail blend).
New wire frames STATUS_APPEND / STATUS_APPEND_SIG / STATUS_APPEND_OK /
STATUS_APPEND_DATA; PROTOCOL_VERSION bumped 2.9.0 -> 2.10.0 (peers must match).
Both flags imply --incremental and are incompatible with -s (chunk
serialization) and --whole-file (rejected up front). Respects --inplace,
--partial/--partial-dir and --delay-updates via the shared store engine.
PROTOCOL_VERSION 2.9.0 -> 2.10.0. The STATUS_MANIFEST frame gains a third
section carrying destination-relative exact-delete paths (the missing
--files-from entries' mirrors); the config frame gains a delete_missing_args
bool (ignore_missing_args stays client-only). The receiver validates the third
section like the keep-set and commits it with manifest_delete_all():
manifest_delete_missing_args runs first (explicit user requests, never blocked
by protected-prefix exclusion protection; staging/basis protected; a non-empty
directory mirror removed only under --force/--delete, rsync parity) and then the
ordinary extras walk. Server --allow-delete gates it like --delete.
The walker's all-or-nothing guarantee holds only while the destination is not
concurrently modified (rehearsal and delete are separate walks). The
protected-prefix list shares the 16 MB MAX_MANIFEST_BYTES budget with the
keep-set and each section is capped at MAX_MANIFEST_ENTRIES; an over-budget
frame is rejected on the receiver with STATUS_ERROR rather than truncated.
delete_extras_limited now rehearses a finite-capped deletion before unlinking
anything (an identical fd-relative walk that counts files and directories) and
returns DELETE_WALK_LIMIT_EXCEEDED with nothing removed when the run would
exceed the cap, so --max-delete is enforced per run instead of truncating the
deletion. A directory that still holds entries the walker leaves in place
(protected excluded prefix, manifest-kept file, symlink) is left behind rather
than failing the whole deletion, matching rsync's leave-non-empty-dirs
behavior. Rehearsal/delete each open an independent file description so a
prior pass cannot drain the directory stream.
- copy-dest basis hits leaked the heap-allocated basis path: BASIS_DEST_COPY
did not transfer it (only LINK does) and returned before the basis cleanup.
basis_match_free is now called on every materialization return path (success
and send-failure) after content/link ownership is transferred.
- the --delete walker regression: the delay-updates staging name must be
protected only as a DIRECT child of the receive root, while basis dirs may
be skipped at any depth. delete_extras_limited now takes DeleteSkipEntry
entries carrying a top_level_only flag instead of a flat prefix list, so a
nested destination directory named .fastsync-stage is ordinary content again
(its extras are deleted) and a basis tree is still never removed.
Generalize the delete walker's protected-root-child skip into a prefix list.
The receiver now passes both the --delay-updates staging directory and every
basis-dir path, so a --delete run can never treat a basis snapshot (which a
--link-dest run just linked from) as destination content to remove.
Review fixes for --delay-updates:
- --delete no longer deletes the staged files: the delete walker gains a
skip_root_child parameter and receive_manifest passes DELAY_UPDATES_STAGING_DIR
when delay_updates is active, so deletion removes genuine extras while the
staging dir (a direct child of the receive root) is left for publication in
both single and -m modes.
- --backup-dir is rejected when it collides with the reserved internal staging
name .fastsync-stage (trailing slash normalized), in client validation and in
the received-config wire validation, preventing old backups from being
silently installed as new files.
- Staging dir is now held under an exclusive advisory flock for the whole
transfer (context lifetime): two simultaneous delayed transfers to one
destination root no longer share/destroy each other's staged data - the
second fails cleanly. Cleanup only touches the staging dir when this context
owns the lock, so a lock-contention failure cannot wipe a live session.
- Post-publish staging cleanup now returns/logs instead of discarding failures
(warning when the staging dir cannot be fully removed).
- Reworked the publish-failure integration test to exercise real mid-publish
semantics (top-level file published, nested rename fails, no rollback,
sources retained under --remove-source-files) and added integration tests for
--delete + --delay-updates ordering and reserved --backup-dir rejection.
- RSYNC_COMPAT note documents delete ordering, the reserved-name hazard, and
the concurrency guard.
receive_files() in src/server/server.c was a non-static, unprototyped,
zero-caller duplicate of receiver_process()/receiver_receive_files() in
src/server/receiver.c. Delete it along with the includes it uniquely pulled
(chunk.h, metadata.h, sys/stat.h, duplicate quoted unistd.h); remaining code
still uses file.h/config.h/protocol.h (via multiprocessing.h) directly.
mkdir_r() in src/shared/utils.c had zero callers in src/ and tests/; remove
the function and its declaration in utils.h, plus the unused libgen.h include.
- Reformat all C/H files to match .clang-format (LLVM style)
- Fix 26 cppcheck const-correctness warnings (constParameterPointer,
constVariablePointer, constVariable)
- Update function declarations in headers to match const parameters
Feature changes across 15 files:
- -a/--archive: enables -c -m -M (no -s, which slows transfers)
- -n/--dry-run: scan + print without connecting or transferring
- -p <port>: custom SSH port (passed as -p to ssh via execvp)
- --exclude <pattern>: glob-based filename filtering in scanner
- --delete: sender collects file manifest; receiver deletes unlisted files
Protocol: added STATUS_MANIFEST, use_delete field in Config wire format.
New utilities: glob_match(), delete_extras() with recursive directory walk.
Scanner: accepts exclude patterns; skips matching entries.
SSH transport: switched from execlp to execvp for dynamic port arg.
Server + multiprocessing: handle STATUS_MANIFEST in both single and
multithreaded receive paths.
Builds clean, all 7 tests pass.
TapTap