Commit Graph
237 Commits
Author SHA1 Message Date
TapTap ebab335488 Merge feat/p3-fuzzy: --fuzzy/-y/--no-fuzzy similar-file delta basis 2026-09-06 23:52:24 +02:00
TapTap 4f19f5bfe7 fix: satisfy cppcheck on the hard-bound walker test
CI / lint (pull_request) Successful in 31s
CI / sanitizers (address) (pull_request) Successful in 52s
CI / sanitizers (undefined) (pull_request) Successful in 51s
CI / fuzz-build (pull_request) Successful in 17s
CI / coverage (pull_request) Successful in 42s
CI / valgrind (pull_request) Successful in 36s
CI / build-and-test (pull_request) Successful in 12m49s
Drop the derived 'created = rootfd >= 0' that cppcheck flagged as always true
after the EXPECT_TRUE guard.
2026-09-06 23:33:21 +02:00
TapTap 5fc49a8503 test: pin the review findings
- ignore-errors scan-error test now runs single-threaded and under -m (the
  exact scan_directory_multithreaded path that had the use-after-free);
- new integration test: --delete/--delete-before --ignore-errors with an
  unreadable SOURCE ROOT (sequential and -m) must fail and delete NOTHING;
- new integration test: a destination-only file that merely matches an exclude
  rule is deleted under plain --delete (protection is sender-derived), while a
  source-excluded mirror is protected;
- delete-protects/delete-excluded coverage extended to --delete-after and
  --delete-delay;
- new unit test: the 100000-entry server hard bound is all-or-nothing (more
  extras than the bound -> nothing removed);
- new integration test: --force is inert under --delay-updates (documented);
- fixed the ignore-errors assertion message that stated the opposite of what it
  asserted.
2026-09-06 23:10:37 +02:00
TapTap 741d1f3b93 test: review follow-up coverage for --fuzzy
CI / lint (pull_request) Failing after 37s
CI / build-and-test (pull_request) Skipped
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
Unit (CLI): --fuzzy --no-incremental (either order) stays a valid plain-mode
config -- no forced handshake, no delta implication; --fuzzy --no-delta stays
covered.

Integration (TestFuzzy):
- worthless basis: a sibling that passes the name+size gates but shares no
  blocks makes the sender reply STATUS_NEXT; the whole file is consumed inside
  the delta handshake with byte-exact output (no protocol desync).
- non-displacement: an existing exact-path destination file INSIDE the delta
  size bounds (same size, different content, older mtime) is used as the delta
  basis instead of a byte-identical similar sibling (whole-file wire cost).
- asymmetric bases: basis larger than source (prefix reuse) and basis smaller
  than source (appended tail as literals) both reconstruct byte-exactly with a
  small delta.
- --no-fuzzy end-to-end equals the no-flag whole-file behavior.
CountingProxy closes its listener socket (fd hygiene).
2026-09-06 23:05:43 +02:00
TapTap 5f4c7ce638 fix: free walker-test root path after cleanup (ASan leak)
CI / lint (pull_request) Successful in 31s
CI / sanitizers (address) (pull_request) Successful in 40s
CI / sanitizers (undefined) (pull_request) Successful in 39s
CI / fuzz-build (pull_request) Successful in 18s
CI / coverage (pull_request) Successful in 36s
CI / valgrind (pull_request) Successful in 36s
CI / build-and-test (pull_request) Successful in 10m49s
make_walk_root() roots were removed from disk but never freed, leaking under
the address/valgrind sanitizer jobs.
2026-09-06 22:09:25 +02:00
TapTap 1de2677bb1 test: delete-policy unit and integration coverage
Unit: walker all-or-nothing bounds (exceeded -> nothing removed + distinct
result; exact bound -> deletes), protected-prefix skipping, config wire
round-trip for force_delete/delete_excluded/prune_empty_dirs/max_delete, CLI
parse/validation for the new flags.  Integration (TestDeletePolicy): default
delete-excluded protection and --delete-excluded opt-out (single-thread, -m,
early --delete-before, excluded-dir subtrees), --max-delete all-or-nothing over
and at the limit, --force file-over-nonempty-dir replacement, --prune-empty-dirs
(--dirs mode + recursion-mode parity), and --ignore-errors keeping deletion
active across a genuine scan I/O error (run as an unprivileged user).
2026-09-06 21:50:12 +02:00
TapTap 9c1ec6758c test: --fuzzy coverage (CLI, config wire, integration)
Unit: parse -y/--fuzzy and --no-fuzzy negation (order-independent), -W and
--no-delta leave fuzzy inert, --fuzzy implies --incremental/--delta, and
validate_config rejects fuzzy with -s (chunk serialization) and -f
(sendfile).  Config: fuzzy survives the socketpair wire round-trip.

Integration (TestFuzzy, byte counts via a new CountingProxy helper): the
rename case transfers a 2 MiB file in a few percent of its size with byte-
exact output under --fuzzy, while the no-fuzzy, no-candidate, dissimilar-
sibling and --whole-file runs send the whole file; -y alias; -m parity;
dest-holds-unsuitable-file falls back to the sibling; --delay-updates and
--remove-source-files keep their semantics on fuzzy transfers.
2026-09-06 21:04:23 +02:00
TapTap 01a5a93089 Merge feat/p3-basis-dest: alternate basis dirs (--compare-dest/--copy-dest/--link-dest) 2026-09-06 20:10:40 +02:00
TapTap 329fc60b14 test: make remove-source incremental-skip test deterministic
CI / lint (pull_request) Successful in 32s
CI / sanitizers (address) (pull_request) Successful in 42s
CI / sanitizers (undefined) (pull_request) Successful in 40s
CI / fuzz-build (pull_request) Successful in 17s
CI / coverage (pull_request) Successful in 35s
CI / valgrind (pull_request) Successful in 36s
CI / build-and-test (pull_request) Successful in 3m29s
The seed run did not preserve timestamps, so the destination copy's mtime was
the write time; the incremental --remove-source-files rerun only skipped the
file when both writes happened to land in the same whole second, making the
test flaky (observed intermittently in local full-suite runs and on CI).  Seed
with -M so the destination stores the source's exact mtime.
2026-09-06 19:53:22 +02:00
TapTap e0e0ea6eac test: xxHash equal-size gate, basis priority, size-only/ignore-times, oversize
- unit: config basis-path normalization (trailing slash, a//b, ./x/./y collapse;
  degenerate inputs rejected).
- integration: same-size/same-mtime/different-content fixtures prove the xxHash
  gate -- the basis changed.txt now has the SAME byte size as the source so the
  size short-circuit can no longer mask the hash comparison, plus a dedicated
  parametrized same-size mismatch test asserting link/copy never use a
  content-mismatched basis and compare-dest transfers.
- basis-dir priority is first-match-wins: two link-dest dirs (inode of the
  first), and compare-dest before link-dest stays sparse while the reverse
  order hard-links.
- --size-only links a same-content basis file with a different mtime;
  --ignore-times never links even an exact match.
- --delay-updates + --delete removes extras inside a nested .fastsync-stage
  dir (regression guard) while keeping the real staging dir and basis tree.
- a basis run containing a file above the whole-file limit fails up front with
  a clear error and transfers nothing.
2026-09-06 19:43:21 +02:00
TapTap c0c315cf48 test: cover -m late-deletion failure, receiver leak exits, timed ACK read
- Unit (leak guards): drive receiver_process_pending() past a parked keep-set
  into STATUS_ABORT, EOF, and a second manifest frame; each must return -1 with
  no manifest handed out. Verified leak-free under ASan.
- Unit: receive_status_timed reads a status and fails cleanly on EOF.
- Integration: test_late_flags_commit_only_after_success now parametrizes the
  -m path, proving a failed -m late-timing run preserves every extra and that
  the deferred manifest is dropped (never applied) when the writer fails.
2026-09-06 19:35:25 +02:00
TapTap 7eacf7c180 test: cover rsync delete-timing flags, config wire, and semantics
- CLI: each timing flag (+ --del alias) accepted and implies --delete;
  conflicting timings and a timing with --no-delete are rejected.
- Config: delete_during/delete_delay survive config_send/config_receive;
  two simultaneous timings are rejected by the receiver-side validation;
  config_delete_timing_early() mapping is unit-tested.
- Integration (TCP, single- and multithreaded): every flag removes extras on
  a successful transfer; early modes (--delete-before/--delete-during/--del)
  delete before data is applied so a destination file blocking a nested
  write is removed and the transfer succeeds, while plain --delete /
  --delete-after / --delete-delay keep it and fail with every extra intact
  (commit-style). Early timing also completes (without deleting) when the
  server refuses deletion.
2026-09-06 18:53:24 +02:00
TapTap 4e725517f0 feat: implement rsync delete timing (--delete-before/--delete-during/--delete-delay/--delete-after)
Deletion timing is now real and selected by the four rsync flags plus the
plain --delete default. Wire protocol bumps to 2.8.0: two new config
booleans (delete_during, delete_delay) are serialized and validated, joining
the existing delete_before/delete_after.

- Early modes (--delete-before, --delete-during/--del): the sender pre-scans
  the whole tree (paths only), transmits the keep-set manifest BEFORE any
  file data, and the receiver removes extras and acks STATUS_OK; the sender
  only streams data after the deletion committed. Deletion is thus performed
  even if a later transfer phase fails (rsync delete-before/during are
  destructive by definition). FastSync streams in a single scan so it cannot
  interleave per-directory like rsync delete-during; --delete-during selects
  the same engine mode as --delete-before (documented divergence).
- Late/commit modes (plain --delete, --delete-after, --delete-delay): the
  manifest closes the data stream and deletion is committed only after
  STATUS_FINISHED proves the whole transfer succeeded, preserving FastSync's
  commit-style safety. --delete-delay converges with --delete-after because
  FastSync never snapshots the destination during data flow (documented).
- The STATUS_MANIFEST frame is now self-delimiting and position-independent.
  Single-threaded receivers delete before the success frame; the -m receiver
  hands the keep-set to server.c, which commits the deletion only after the
  disk writer thread has drained (fixes a delete-vs-in-flight-temp race).
- Every timing flag implies --delete; at most one timing flag is allowed.
- Each timing flag implies --delete, matching rsync; conflicts are rejected.
2026-09-06 18:53:20 +02:00
TapTap 0d18b7fc87 test: integration coverage for compare/copy/link-dest basis directories
- compare-dest skips an exact basis match (leaving a sparse destination) and
  still transfers files the basis cannot satisfy; a content mismatch forces a
  normal transfer.
- copy-dest materializes the unchanged file as a real local copy (distinct
  inode) and transfers content mismatches.
- link-dest hard-links (asserted same inode/nlink to the DIR file) and falls
  back to a normal transfer on content mismatch.
- a missing basis dir is a clean full-transfer no-op for all three flags.
- link-dest works through -m multithreading and --delay-updates (staged and
  published as a real link, staging cleaned up).
- --delete removes genuine extras while leaving the basis dir untouched.
2026-09-06 18:47:37 +02:00
TapTap 1fc0cacc32 test: unit tests for basis-dir CLI parsing and config wire round-trip
- parse_args accepts each flag in both forms, keeps repetition order/types,
  implies --incremental + metadata, and rejects absolute/escaping/degenerate
  paths; validate_config rejects basis dirs combined with -s.
- config wire round-trips a mixed basis list and rejects escaping/absolute
  paths on the receiver side.
2026-09-06 18:47:33 +02:00
TapTap 4cf34026e7 test: drop racy queued_bytes==0 assert in byte-budget test
CI / lint (push) Successful in 25s
CI / sanitizers (address) (push) Successful in 40s
CI / sanitizers (undefined) (push) Successful in 40s
CI / fuzz-build (push) Successful in 17s
CI / coverage (push) Successful in 34s
CI / valgrind (push) Successful in 36s
CI / build-and-test (push) Successful in 3m6s
After the writer releases bytes, the blocked enqueuer may already have
admitted the next payload, so the transient queued_bytes==0 read is
scheduling-dependent (lost the race under ASan). The post-join state
(covers unblocking + budget re-limit) is deterministic.
2026-09-06 16:27:54 +02:00
TapTap 3275b6c978 fix: address c-review for -R/--dirs/--mkpath row
B1: destination-root existence/creation mishandled two legitimate forms.
file_directory_exists_secure/file_ensure_directory_secure now normalize a
trailing-slash destination (so the last component is never an empty leaf)
and treat a destination equal to the already-open authorized root as present
(no spurious <root>/<basename> nested dir with --mkpath). Confinement and
O_NOFOLLOW probing are unchanged. Regression integration tests: existing
dest with trailing slash works with and without --mkpath; dest == authorized
root works and creates no stray nested dir.

W1: chunk serialize/deserialize round-trip unit test for a directory entry
mixed with a regular file, with and without metadata; --dirs coverage under
-s and -s -m.
W2: --list-only and --dry-run now print file_wire_path() so all outputs show
the -R transformed relative name, matching -i/--out-format.
W3: RSYNC_COMPAT -d/--dirs note: dirs are created immediately under
--delay-updates (only regular files are staged).
W4: --dirs generator flushes chunks by element count too, so a long
--files-from list of empty directories cannot exceed the per-chunk file cap.
N1: STATUS_MKDIR added to status_to_string.
N2: removed dead TestMkpath._transfer.
N3: removed redundant --no-implied-dirs OPTION_TABLE row.
N4: integration tests: --dirs --delete keeps the just-created empty dir (and
deletes extras); a listed dir colliding with a regular file at the dest fails
cleanly.
2026-09-06 16:10:00 +02:00
TapTap c2cf231158 feat: implement -R/--relative, --no-implied-dirs, --dirs/-d, --mkpath
CI / lint (pull_request) Successful in 26s
CI / sanitizers (address) (pull_request) Failing after 41s
CI / sanitizers (undefined) (pull_request) Successful in 40s
CI / fuzz-build (pull_request) Successful in 18s
CI / coverage (pull_request) Successful in 35s
CI / valgrind (pull_request) Failing after 37s
CI / build-and-test (pull_request) Failing after 2m35s
RSYNC_COMPAT Phase-2 row M: path-list construction and destination
directory creation while preserving traversal safety.

- -R/--relative with --files-from: transmit each listed entry under its
  bare relative destination path (no source-root mirror). Files keep an
  absolute local read path plus a separate wire/dest path (File.send_path);
  manifest, incremental quick-check and change output follow the wire path,
  so --delete and --remove-source-files stay consistent. -R without
  --files-from is unchanged (full mirror).
- --no-implied-dirs: client-only, only meaningful with -R + --files-from.
  A listed file whose parent dir is not itself (or via an ancestor)
  explicitly listed cannot be placed; the run fails up front with a clear
  error. No effect otherwise.
- --dirs/-d + --old-dirs/--old-d aliases: -d <dir> transmits the source
  root as an explicit empty directory entry (STATUS_MKDIR frame); with
  --files-from listed dirs are created empty and listed files transferred,
  never descending. Works single-threaded, -m (sequential scanner in the
  -m scan thread) and chunk-serialization (per-file type marker).
  Directory entries appear in the delete manifest.
- --mkpath: new wire bool; server creates the destination root (and missing
  leading components under its authorized root) at connection start. A
  missing destination root is now rejected by default.
- Protocol bumped to 2.7.0 (mkpath wire field + STATUS_MKDIR + chunk type
  marker). All receive paths funnel through file_save_to_disk_full which
  creates directories via the secure confined mkdir engine; dir entries are
  excluded from --remove-source-files outcome acknowledgements on both ends.
- Unit coverage: CLI parse (relative/dirs aliases/mkpath/no-implied-dirs),
  config round-trip (relative + mkpath), scanner --dirs non-recursion and
  -R send_path (sequential + parallel), receiver dir-entry save.
- Integration coverage: TestRelativeFilesFrom, TestNoImpliedDirs, TestDirs,
  TestMkpath (single and -m).
- RSYNC_COMPAT: 4 rows move to Implemented (Summary 67/3/5/1/71 = 147).
2026-09-06 15:28:32 +02:00
TapTap a196522010 style: clang-format test_client_cli.c after merge resolution
CI / lint (push) Successful in 23s
CI / sanitizers (undefined) (push) Successful in 42s
CI / sanitizers (address) (push) Successful in 43s
CI / fuzz-build (push) Successful in 17s
CI / coverage (push) Successful in 34s
CI / valgrind (push) Successful in 36s
CI / build-and-test (push) Successful in 1m45s
2026-09-06 14:29:28 +02:00
TapTap 342dd152ef Merge feat/p2-files-from-filter: files-from/from0/filter/-F/-C
Client-side file-list selection and filter-rule layer (client-only; no wire
change). rsync-consistent inner-first per-dir filter precedence; listed-but-
missing files-from entries hard-error; root '/' scan regression fixed;
Summary recounted (63 implemented / 75 not).
c-review REQUEST CHANGES -> blockers fixed; PR #265.
2026-09-06 14:25:51 +02:00
TapTap 4295fefaa3 fix: --delay-updates delete/backup collisions, publish-failure test, staging lock
Review fixes for --delay-updates:

- --delete no longer deletes the staged files: the delete walker gains a
  skip_root_child parameter and receive_manifest passes DELAY_UPDATES_STAGING_DIR
  when delay_updates is active, so deletion removes genuine extras while the
  staging dir (a direct child of the receive root) is left for publication in
  both single and -m modes.
- --backup-dir is rejected when it collides with the reserved internal staging
  name .fastsync-stage (trailing slash normalized), in client validation and in
  the received-config wire validation, preventing old backups from being
  silently installed as new files.
- Staging dir is now held under an exclusive advisory flock for the whole
  transfer (context lifetime): two simultaneous delayed transfers to one
  destination root no longer share/destroy each other's staged data - the
  second fails cleanly.  Cleanup only touches the staging dir when this context
  owns the lock, so a lock-contention failure cannot wipe a live session.
- Post-publish staging cleanup now returns/logs instead of discarding failures
  (warning when the staging dir cannot be fully removed).
- Reworked the publish-failure integration test to exercise real mid-publish
  semantics (top-level file published, nested rename fails, no rollback,
  sources retained under --remove-source-files) and added integration tests for
  --delete + --delay-updates ordering and reserved --backup-dir rejection.
- RSYNC_COMPAT note documents delete ordering, the reserved-name hazard, and
  the concurrency guard.
2026-09-06 14:19:18 +02:00
TapTap 7f2180ef90 fix: filter precedence, files-from errors, NUL/CRLF and filter-rule rejections
Address an independent c-review of the files-from/filter feature:

- .rsync-filter precedence now matches rsync: evaluate the innermost
  (current) directory's rules first, then ancestors, then the command-line
  base (--filter/-C), so a deeper file's '+' can re-include what a shallower
  '-' excluded (regression tests in both scan modes; single-thread and -m).
- --files-from: a listed entry missing on disk and an empty list are now hard
  errors surfaced pre-transfer in send_files, send_files_multithreaded,
  dry-run and --list-only; '.' (whole tree) and empty listed dirs stay valid.
- scanner_path_relative now handles a transfer root of / (previously the
  scanner aborted on children of /).
- Reject unsupported rsync filter syntax explicitly (no silent no-ops):
  +/- modifiers other than '/' (! C s r p x) and rules beginning with ':'/'.'
  /'!' (merge/dir-merge/list-clear shorthands). Docs updated.
- -0/--from0 NUL mode preserves entry bytes (no CR/LF trimming); only newline
  mode trims. Absolute-entry error message no longer includes the newline.
- --no-from0/--no-cvs-exclude registered as negatable booleans.
- RSYNC_COMPAT rows updated for the precedence, rejection list, NUL-mode
  detail and the documented O(entries x files) scalability bound of the
  allow-set (Summary unchanged: 62/3/5/1/76 = 147).
2026-09-06 14:16:12 +02:00
TapTap f4c15a7b78 feat: add --files-from/-0, --filter, -C and -F filter layer
CI / lint (pull_request) Successful in 23s
CI / sanitizers (address) (pull_request) Successful in 40s
CI / sanitizers (undefined) (pull_request) Successful in 39s
CI / fuzz-build (pull_request) Successful in 17s
CI / coverage (pull_request) Successful in 34s
CI / build-and-test (pull_request) Successful in 1m28s
CI / valgrind (pull_request) Successful in 36s
Implement the RSYNC_COMPAT Phase-2 filter/parser feature group:
- --files-from=FILE (repeatable) plus -0/--from0 NUL delimiters: parse the
  source file list relative to the source root into a shared read-only
  allow-set; the scanner transfers listed files and the whole subtree of
  listed directories and prunes everything else in single- and
  multithreaded mode. Absolute/'..' entries and missing files are hard
  CLI errors.
- --filter=RULE: rsync-style +/- rules (anchored '/', dir-only trailing '/',
  word include/exclude forms) evaluated first-match-wins with a default of
  include, as an independent layer from legacy --exclude/--include.
  Unsupported directives (merge/hide/... ) are rejected explicitly. -f stays
  sendfile.
- -C/--cvs-exclude: well-known rsync CVS default exclude set.
- -F: per-directory .rsync-filter files read during traversal and applied to
  the owning directory's subtree (single + parallel), never transferred.
- Delete manifest still derives from what was actually sent.

Client-only config fields; no wire/protocol change. Adds unit coverage
(CLI parse, allow-set and filter scanning single+parallel) and integration
tests (TestFilesFrom, TestFilters). RSYNC_COMPAT matrix rows updated:
5 rows move to Implemented (Summary 62/3/5/1/76 = 147).
2026-09-06 13:43:34 +02:00
TapTap a2a82dd856 feat: implement --delay-updates receiver staging and publication
CI / lint (pull_request) Failing after 22s
CI / build-and-test (pull_request) Skipped
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
Stage every successfully written file under a private 0700 .fastsync-stage
directory inside the receive root and atomically publish all staged files
only after the whole protocol stream (manifest/delete handling included)
has completed, immediately before the success/outcome frame.  On any
abort/error before publication nothing is installed and staging is removed;
a publish failure aborts the transfer with best-effort cleanup of the
remainder (already-published files are not rolled back).  Crash leftovers
are wiped when the next delayed transfer starts.

Wire: new delay_updates config flag (selection-options block), protocol
version bumped to 2.6.0, client/server validation rejects --inplace.
CLI/usage/validation updated.  Works in single-threaded and -m modes
(exactly one write_thread stages files; the staged-file registry is
mutex-protected; publication runs once after both threads join).
--existing/--ignore-existing/--update decide against the final destination
at stage time; --backup is deferred to publication.  remove_source_files
outcomes are only sent after publication so skipped/unpublished sources are
never deleted.  Default (no flag) behavior is unchanged.

Tests: config wire round-trip, CLI parse, --inplace rejection, new
test_delay_updates unit suite (27 suites total), and integration
TestDelayUpdates covering single/-m parity, incremental reruns, remove
source files, receiver-skip ordering, and a deterministic publish-failure
abort path.
2026-09-06 13:20:03 +02:00
TapTap 8577f95550 fix: cppcheck cleanups (printf sentinel type, scanner test style)
CI / lint (push) Successful in 22s
CI / sanitizers (address) (push) Successful in 40s
CI / sanitizers (undefined) (push) Successful in 39s
CI / fuzz-build (push) Successful in 16s
CI / coverage (push) Successful in 34s
CI / build-and-test (push) Successful in 1m23s
CI / valgrind (push) Successful in 36s
2026-09-06 12:56:54 +02:00
TapTap 88ab4f65cb Merge feat/p2-itemize-output: structured change output (-i, --list-only, --out-format, --log-file-format)
Client-only per-file event/format model (change_list.c). Reviewed
(c-review APPROVE WITH NITS, all fixed); PR #263.
2026-09-06 12:52:46 +02:00
TapTap 9fbb86ca68 Merge feat/p2-temp-dir: implement --temp-dir
Receiver writes temps into a confined scratch dir and atomically renames
into place; EXDEV aborts; inplace/partial bypass; thread-safe temp names.
Uses existing wire field; no protocol bump. Reviewed (c-review APPROVE
WITH NITS, all fixed); PR #262.
2026-09-06 12:52:11 +02:00
TapTap 1b67f5ffcf docs: clarify %b semantics, temper thread-safety claim, harden %M scan; add -m coverage
CI / lint (pull_request) Successful in 20s
CI / sanitizers (undefined) (pull_request) Successful in 41s
CI / sanitizers (address) (pull_request) Successful in 41s
CI / fuzz-build (pull_request) Successful in 16s
CI / coverage (pull_request) Successful in 35s
CI / build-and-test (pull_request) Successful in 1m22s
CI / valgrind (pull_request) Successful in 36s
Address c-review nits on the itemize/output feature:
- RSYNC_COMPAT.md: state that %b is the source length (always == %l) because
  no wire-byte counter exists; keep Summary equal to the matrix (recounted:
  54 implemented / 84 not-implemented, 147 rows total - four rows flipped).
- change_list.h/.c: document bytes_sent == size; note itemize/out-format lines
  never interleave with each other but may interleave with legacy log
  messages sharing the stream; mark the %M stat() path best-effort.
- change_render_format scan in format_uses_mtime now mirrors the tokenizer
  (skips '%%' and unknown '%X' pairs) so a literal '%%M' no longer triggers
  the stat() fallback.
- Integration tests: --list-only under -m; a changed file on a second
  --incremental run emits exactly one '>f' line while unchanged files print
  nothing; --log-file + --log-file-format under -m.
2026-09-06 12:49:49 +02:00
TapTap 2d841405e6 test: cover -x cross-device skip without root; harden OneFileSystem tests
CI / lint (pull_request) Failing after 20s
CI / build-and-test (pull_request) Skipped
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
Add a rootless unit test that reaches the actual st_dev skip branch in both
the sequential and parallel (-m) scanners: a symlink nested under the scan
root points at a directory on /dev/shm (a different device than the build
fs) and, under --copy-links semantics, -x must drop that subtree while a
plain scan includes it. Skips only when no cross-device target exists.
Integration OneFileSystem test now cleans both dest dirs up front and
reports a busy test mountpoint instead of ignoring the umount result.
RSYNC_COMPAT.md notes that cross-filesystem mount-point subdirectories are
dropped entirely (rsync parity).
2026-09-06 12:49:26 +02:00
TapTap 00e8df4bcd fix: address c-review nits for --temp-dir engine
CI / lint (pull_request) Failing after 21s
CI / build-and-test (pull_request) Skipped
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
2026-09-06 12:48:07 +02:00
TapTap 7f5547e900 feat: add structured per-file change output model (-i, --list-only, --out-format, --log-file-format)
Implement rsync-style itemized output backed by one shared change-event
engine (src/client/change_list.c):
- -i/--itemize-changes prints ">f+++++++++ <path>" for files actually sent
  (single-threaded and -m); unchanged files print nothing.
- --list-only prints an ls-style listing of files that would be transferred
  without contacting the server or writing anything.
- --out-format=FORMAT prints a printf-style template per changed file
  (tokens %%f %%n %%l %%b %%M %%%%; unknown escapes preserved).
- --log-file-format=FMT logs each transferred file when --log-file is set.
Events are emitted from the per-file sender path shared by both transfer
modes, so the single sender thread is the only reporter (no races).
2026-09-06 12:34:10 +02:00
TapTap 19e6fc2205 feat: implement rsync --temp-dir for atomic receiver installs 2026-09-06 12:21:36 +02:00
TapTap 2bcc20aa08 feat: add -x/--one-file-system to stay within the source filesystem
Capture the transfer root's device (st_dev) at scanner creation and skip
descending into any subdirectory on a different device (a mount point).
Implemented sender/client-side only: sequential BFS and parallel (-m) root
scan apply the same scanner_same_filesystem decision; no wire/protocol change
and default behavior is unchanged. Unit tests cover the pure decision, same
device scanning in both modes, and CLI parsing; integration tests prove -x
leaves a single-filesystem tree byte-identical and, when root can mount a
tmpfs, skips a genuine cross-device subtree.
2026-09-06 12:18:43 +02:00
TapTap f7c6c91e13 fix: set *failed on delta oversize branch; add -m RSF skip test
CI / lint (push) Successful in 20s
CI / sanitizers (address) (push) Successful in 41s
CI / sanitizers (undefined) (push) Successful in 39s
CI / fuzz-build (push) Successful in 16s
CI / coverage (push) Successful in 34s
CI / build-and-test (push) Successful in 1m23s
CI / valgrind (push) Successful in 35s
Review nits from independent review of the four fix branches:
- receive_delta_file STATUS_NEXT oversize branch now sets *failed=true
- receive_incremental_check oversize branch returns NULL (receiver sends the
  single STATUS_ERROR) instead of double-sending
- add multithreaded -m --ignore-existing --remove-source-files integration
  coverage so the writer-thread outcome path is exercised
2026-09-05 13:16:42 +02:00
TapTap 39805e4eee Merge fix/quality-cleanup into dev
fixes #260 (dead code, CLI =form/missing-arg diagnostics)
2026-09-05 13:12:33 +02:00
TapTap 9ef78ef48b Merge fix/perf-delta into dev
fixes #259 (delta_compute hash index)
2026-09-05 13:12:33 +02:00
TapTap 68e7ed57d0 Merge fix/security-hardening into dev
fixes #254 (receiver queue byte-budget) #258 (inplace setuid/truncate)

# Conflicts:
#	src/shared/multiprocessing.c
2026-09-05 13:12:31 +02:00
TapTap 1d61a1426e fix: #260 accept --opt=value uniformly and report missing arguments
- Correct misleading doc comments on set_string_option /
  set_positive_int_option / set_nonneg_int_option (they return 0/-1,
  not true/false).
- find_table_option_with_equals() now matches every OPTION_TABLE value
  option (OPT_STRING/OPT_POS_INT/OPT_NONNEG_INT/OPT_ULL), so forms such
  as --max-size=2G, --min-size=1K, --suffix=.bak, --timeout=30,
  --max-depth=5, --backup-dir=X parse instead of dying as 'Unknown option'.
  --max-size/--min-size now accept rsync-style binary suffixes (0 remains a
  valid 'no limit' byte count). Existing special handling for
  --compress-choice, --compress-level, --modify-window=, --chmod=,
  --skip-compress=, --compress-threads= is preserved.
- Options that require a separate value (-p, --exclude, --include,
  --delta-block, --delta-max, --server-port, --bwlimit, --chunk-size,
  --log-file, --exclude-from, --include-from, -T, --skip-compress,
  --compress-threads) now emit an explicit 'missing argument' diagnostic
  instead of falling through to the generic 'Unknown option' branch when
  given as the final argv entry.
- Add unit tests covering the = forms (--max-size=2G, --min-size=1K,
  --suffix=.bak, --timeout=30, --max-depth=5, --backup-dir=X) and a clean
  'missing argument' (not 'Unknown option') diagnostic for trailing
  --exclude/--server-port/--skip-compress/-T.
2026-09-05 12:58:30 +02:00
TapTap 3704a6adaa test: integration coverage for #251 #252 #253 #257
- remove-source-files keeps sources skipped by --existing/--ignore-existing/
  --update (rsync reference behavior)
- --backup keeps <file>~, --suffix .bak, and --backup-dir backups
- --partial --partial-dir installs completed files in the destination
- a 100 MB file transfers end to end (>64 MiB whole-file cap regression)
2026-09-05 12:46:47 +02:00
TapTap 14c064a093 fix: #251 #252 #253 #255 #256 #257 receiver & remove-source correctness
#251 --remove-source-files deletes sources that were skipped receiver-side
     (--existing/--ignore-existing/--update). The receiver now reports a
     per-file outcome for every processed data file when the sender requests
     removal; the client only unlinks sources the receiver actually wrote.
     add remove_source_files to the wire config and bump the protocol to 2.5.0.
#252 --backup/--suffix/--backup-dir broken by NULL-vs-empty wire loss. Receivers
     canonicalize the empty wire string back to NULL for backup_dir, temp_dir,
     partial_dir and suffix, and --suffix is received unconditionally.
#253 --partial --partial-dir never installed completed files. file_save_to_disk
     now renames a fully written partial-dir file into the real destination.
#255 STATUS_CHECK read the entire old file before the size/mtime quick check.
     Old contents are only read when a checksum compare or delta needs them.
#256 receive_delta_file failure paths did not set *failed, so the caller sent
     STATUS_NEXT and waited for a body that never came. Every NULL return now
     marks the transfer failed.
#257 files >64 MiB could not transfer. Whole-file receive caps raised to the
     256 MiB connection/allocation ceiling (chunk caps stay 64 MiB) and the
     client ignores SIGPIPE so a server-side close surfaces as a clean error.

Unit tests added: config NULL-vs-empty round trip, incremental quick-check
skip/NEXT paths, delta oversize failure, partial-dir install, save-result
skip reporting.
2026-09-05 12:46:44 +02:00
TapTap 2234879b2f fix: #258 normalize mode and truncate on --inplace overwrite
The --inplace branch opened the destination with O_WRONLY|O_CREAT (no
O_TRUNC) and only restored metadata when the sender supplied it.  Two
flaws resulted:
1. An existing destination file kept its original mode when no metadata
   was sent, so setuid/setgid/sticky bits survived an overwrite (a root
   sync could leave a root-owned setuid binary controlled by a client).
2. A shorter payload left stale trailing bytes from the previous version
   because the file was never truncated to the new length.

In the inplace branch of file_to_disk_secure_impl:
- Always trim the file to the new payload length (ftruncate after the
  write) so stale trailing bytes can never survive; sparse targets keep
  their pre-size ftruncate.
- Always normalize the mode after a successful overwrite: apply the
  metadata-derived safe mode when metadata is present (as before), else
  fchmod to a safe default 0644, so setuid/setgid/sticky are cleared in
  both cases.
- The --update newer-destination check still runs before any truncation
  or chmod, preserving the skip semantics.

Adds unit tests in test_file.c: (a) setuid/sticky bits on an existing
destination are cleared after an inplace write with and without metadata,
(b) a shorter inplace payload leaves no trailing stale bytes.
2026-09-05 12:29:46 +02:00
TapTap 98120fc722 fix: #254 bound receiver queue by aggregate payload bytes
The per-connection memory budget (MAX_CONNECTION_MEMORY, 256 MiB) only
charged wire buffers via receive_data_limited.  Decompression buffers and
per-file chunk copies were not accounted for, and the multithreaded
receiver could enqueue up to 100 files (each up to 64 MiB uncompressed)
ahead of a slow disk writer, retaining ~6.4 GiB per connection.  A client
sending highly compressible chunks with little bandwidth could OOM the
host while the reserve never tripped.

Bound the receive pipeline by aggregate payload bytes instead of item
count alone:
- Export MAX_CONNECTION_MEMORY from protocol.h.
- PipelineContextReceiver tracks queued_bytes (payload bytes received but
  not yet released by the disk writer, i.e. queued or in the writer's
  hand) under the existing mutex.
- receiver enqueue now blocks while the queue is full by count OR when
  adding the file would push queued_bytes over the configured byte limit,
  applying backpressure to the sender instead of failing the transfer.
- The disk writer releases the byte budget after each file is freed and
  signals the not-full condition.
- The server sets the byte ceiling to
  MAX_CONNECTION_MEMORY - 2*MAX_CHUNK_SIZE so that the queued payloads
  plus the transient wire/decompression buffers of the one in-flight
  chunk stay within the per-connection budget.

The single-threaded receive path is already bounded: it writes files to
disk before reading the next chunk, so its transient is at most one
chunk's wire + decompressed + copied payload (~3 * MAX_CHUNK_SIZE, below
the budget).  Wire buffers remain charged exactly once by
receive_data_limited; this change does not double charge them.

Adds a deterministic unit test in test_multiprocessing.c proving that an
enqueue which would exceed the byte budget blocks until the writer
releases bytes.
2026-09-05 12:29:41 +02:00
TapTap 5fed0888aa perf: #259 delta_compute hash index over signature blocks 2026-09-05 12:21:26 +02:00
TapTap 9f8b58893b Apply clang-format 18 to merge-conflict resolution code
CI / lint (push) Successful in 15s
CI / sanitizers (address) (push) Successful in 37s
CI / sanitizers (undefined) (push) Successful in 37s
CI / fuzz-build (push) Successful in 16s
CI / coverage (push) Successful in 33s
CI / build-and-test (push) Successful in 1m18s
CI / valgrind (push) Successful in 34s
The Phase 1 merge conflict resolutions introduced formatting that failed
the CI lint job (clang-format 18.1.3). Reformatted with the exact CI
version; no functional changes.
2026-09-05 10:57:41 +02:00
TapTap 48dfd5dfa0 Fix incremental skip regression from modify-window nsec comparison
CI / lint (push) Failing after 3s
CI / build-and-test (push) Skipped
CI / sanitizers (address) (push) Skipped
CI / sanitizers (undefined) (push) Skipped
CI / fuzz-build (push) Skipped
CI / coverage (push) Skipped
CI / valgrind (push) Skipped
metadata_mtime_matches() required exact nanosecond equality at the default
modify_window=0, but destination write-time nsecs never match source
creation-time nsecs unless -M preserves metadata. Compare whole seconds
(rsync's default quick-check) at window 0; keep the subsecond-refined
tolerance for explicit window values.
2026-09-05 10:43:58 +02:00
TapTap 8384a1997b Merge remote-tracking branch 'origin/feat/rsync-delete-during-alias' into dev 2026-09-05 02:53:59 +02:00
TapTap a1981a78ad Merge remote-tracking branch 'origin/feat/rsync-old-dirs-aliases' into dev 2026-09-05 02:48:48 +02:00
TapTap b5646a0b9e Merge remote-tracking branch 'origin/feat/rsync-checksum-choice-alias' into dev 2026-09-05 02:44:04 +02:00
TapTap c6a341bd1e Merge remote-tracking branch 'origin/feat/rsync-partial-progress' into dev 2026-09-05 02:39:03 +02:00
TapTap c1e9509f16 Merge remote-tracking branch 'origin/feat/rsync-old-args' into dev 2026-09-05 02:34:29 +02:00