Commit Graph
402 Commits
Author SHA1 Message Date
TapTap 4e7e84f947 Merge feat/p4-metadata-capture: atimes/crtimes/open-noatime/omit-dir-times/omit-link-times
# Conflicts:
#	src/shared/config.c
#	tests/integration/test_features.py
2026-09-08 21:00:07 +02:00
TapTap e80888ce7b metadata times: -U/--atimes, -N/--crtimes, --open-noatime, -O/-J
CI / lint (pull_request) Successful in 52s
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
CI / build-and-test (pull_request) Successful in 1m19s
Capture+transmit source atime (pre-read stat; O_NOATIME sender guard) and birth
time (statx STATX_BTIME); receiver restores atime with mtime (crtime not settable
portably -> transmitted, explicitly not applied). --open-noatime is client-only.
-O/-J documented as accepted no-ops (FastSync never preserves dir/symlink times).
Wire: metadata frame gains atime/crtime val+sec+nsec; PROTOCOL_VERSION
2.11.0->2.12.0. Review fixes: gate atime capture to Linux (no epoch clobber on
non-Linux), close fd on fdopen failure, honest -O/-J status (Compat no-op).
2026-09-08 20:58:56 +02:00
TapTap f891cd0a6a hard-links: -H/--hard-links preserves inode relationships
CI / lint (pull_request) Successful in 50s
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
CI / build-and-test (pull_request) Successful in 1m19s
Source files sharing (st_dev,st_ino) are recreated as hard links on the
destination; only the first member's data crosses the wire (siblings ride a
payload-less STATUS_HARDLINK frame). Ordering requires the single-FIFO-writer
receiver + forced sequential scan (documented). link()-failure falls back to a
byte-identical local copy. Rejects -s/--append. PROTOCOL_VERSION 2.11.0->2.12.0.
Review fixes: delete the dead HardLinkRegistry (ordering holds by FIFO writer),
and --existing no longer aborts when the first member is absent but the sibling
exists (leaves the sibling in place).
2026-09-08 20:58:56 +02:00
TapTap cbb09e41ab identity: fix use-after-free in --usermap/--groupmap parse error path
CI / lint (push) Successful in 48s
CI / sanitizers (undefined) (push) Successful in 53s
CI / sanitizers (address) (push) Successful in 53s
CI / fuzz-build (push) Successful in 18s
CI / coverage (push) Successful in 42s
CI / valgrind (push) Successful in 37s
CI / build-and-test (push) Successful in 4m37s
identity_parse_map freed the str_dup'd list before logging the offending rule
( points into that buffer), causing an invalid read caught by CI valgrind
(MSAN/MSAN-style; the ONLY definite valgrind error in the suite).  Log before
freeing.  valgrind now reports 0 errors / 0 definite leaks in both the parent
and the forked wire-roundtrip child.
2026-09-08 18:49:23 +02:00
TapTap 30048dddef style: clang-format 18 (CI lint) on Wave-A sources
CI / lint (push) Successful in 48s
CI / sanitizers (address) (push) Successful in 47s
CI / sanitizers (undefined) (push) Successful in 48s
CI / fuzz-build (push) Successful in 19s
CI / coverage (push) Successful in 44s
CI / valgrind (push) Failing after 37s
CI / build-and-test (push) Successful in 4m36s
Reflow usage.c, config.c, file.c, test_client_cli.c, test_file.c so the
clang-format check passes.  Whitespace-only; no behavior change.
2026-09-08 18:38:50 +02:00
TapTap b283c8084c identity: keep --numeric-ids in the activate set (-M --numeric-ids)
CI / lint (push) Failing after 4s
CI / build-and-test (push) Skipped
CI / sanitizers (address) (push) Skipped
CI / sanitizers (undefined) (push) Skipped
CI / fuzz-build (push) Skipped
CI / coverage (push) Skipped
CI / valgrind (push) Skipped
identity_active_enabled() only gates identity_apply_ownership, which runs only
when metadata is present, so --numeric-ids must stay in the set: combined with
-M it activates raw-id application, while a standalone --numeric-ids (no
ownership-affecting flag) carries no metadata and correctly stays inert.  My
earlier review fix removed it and broke 'owner not applied' for -M --numeric-ids
(uid 0 instead of the source ids).  Revert that removal.
2026-09-08 18:37:07 +02:00
TapTap 279fc8468a Merge feat/p4-preallocate: --preallocate
# Conflicts:
#	tests/test_client_cli.c
#	tests/test_config.c
2026-09-08 18:25:49 +02:00
TapTap 362a6a5488 preallocate: --preallocate allocates dest space up front
CI / lint (pull_request) Failing after 3s
CI / build-and-test (pull_request) Skipped
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
Receiver allocates the destination file's full size before streaming data
(posix_fallocate preferred, ftruncate fallback on EOPNOTSUPP/ENOSYS) so an
out-of-space transfer fails fast instead of partway. Additive config bool
crossing the wire; PROTOCOL_VERSION 2.10.0 -> 2.11.0. Threaded through all
store paths (atomic, inplace, partial/delay-updates staging, link-dest copy
fallback). Review hardening: explicit lseek(0) before the data write so
correctness does not depend on posix_fallocate leaving the fd offset unchanged.
2026-09-08 18:23:48 +02:00
TapTap 53ce00b830 identity mapping: --numeric-ids / --usermap / --groupmap / --chown
CI / lint (pull_request) Failing after 3s
CI / build-and-test (pull_request) Skipped
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
Receiver-side ownership application, opt-in and privilege-gated:
- OFF for every existing transfer (plain -M/--preserve still never applies
  ownership); only triggers on an explicit identity flag + receiver permission.
- EPERM/EACCES warn-and-continue (never aborts); other fchown errors escalate.
- fd-relative fchown after the file is written (symlink-safe, confined).
- New src/shared/identity.{c,h}; config fields numeric_ids / chown uid/gid /
  usermap + groupmap id-pair tables cross the wire; PROTOCOL_VERSION 2.10.0
  -> 2.11.0. CLI in client_cli.c; per-connection snapshot in server.c.
- Review fixes: EPERM/EACCES-only warn-and-continue, prominent root-receiver
  notice, identity_clear_active on early server error paths, --numeric-ids
  kept inert standalone (removed from activation trigger set).
2026-09-08 18:23:43 +02:00
TapTap 4c98744014 Merge feat/p3-checksum-choice: --checksum-choice/--cc and --checksum-seed 2026-09-07 19:53:28 +02:00
TapTap f99e6e1edf Merge feat/p3-append: --append / --append-verify tail resume 2026-09-07 19:52:33 +02:00
TapTap c6758531f6 fix: reject xxh3 alias, pin handshake digest length to algorithm, simplify xxh64 copy, note FIPS md5
CI / lint (pull_request) Successful in 35s
CI / sanitizers (undefined) (pull_request) Successful in 52s
CI / sanitizers (address) (pull_request) Successful in 52s
CI / fuzz-build (pull_request) Successful in 18s
CI / coverage (pull_request) Successful in 42s
CI / valgrind (pull_request) Successful in 37s
CI / build-and-test (pull_request) Successful in 12m58s
2026-09-07 19:50:52 +02:00
TapTap 4b74f6a41d fix: treat an absent missing-mirror parent as a no-op, only claim real deletions
BLOCKER: an absent mirror whose PARENT directory does not exist on the
destination (a deeper --files-from missing entry, -R or full-mirror layout) was
treated as a hard failure because file_open_secure_parent returned -1 when the
parent was missing. That aborted the whole run, skipped the --delete extras
walk, and tore down an early --delete-before/during connection, contradicting
'missing mirror = no-op / all-missing succeeds'. A parent-open failure is now a
no-op when errno is ENOENT/ENOTDIR (matching file_remove_tree_secure); only a
genuine I/O error fails the run.

Also: an already-absent mirror reached via unlinkat-ENOENT no longer prints
'Deleted: <path>' (a no-op dressed as a deletion); the per-path 'Deleted:' line
is printed only when an entry was actually removed.
2026-09-07 18:32:47 +02:00
TapTap 0afda6b094 feat(checksum): --checksum-choice/--cc and --checksum-seed for whole-file digest
Adds real algorithm selection (xxh64 default, plus md5 via OpenSSL EVP) and a
64-bit seed for the per-file whole-file digest used by the --incremental/
--checksum handshake and basis-dir content verification. The seed also feeds
the delta path's per-block xxHash32 strong checksum (low 32 bits) so an
explicit seed deterministically changes those digests too. Sender and receiver
hash identically: the algorithm id and seed cross the config wire frame and the
STATUS_CHECK handshake now carries a length-prefixed, bounded digest instead of
a fixed 64-bit value. Unsupported algorithm names are rejected at parse time
(never a silent no-op). PROTOCOL_VERSION bumped 2.9.0 -> 2.10.0; defaults
(xxh64, seed 0) preserve prior byte-for-byte behavior.
2026-09-07 17:42:47 +02:00
TapTap 6ad3887aa1 feat: --append / --append-verify tail-only resume
Implement rsync's append modes: when an existing destination file is SHORTER
than the source, the receiver negotiates a resume offset and only the tail is
transferred; the full file (retained prefix + tail) is rebuilt and installed
through the normal atomic store path, so the result is byte-identical to the
source whenever the prefix matches.

- --append: sends the tail without content-verifying the retained prefix
  (rsync parity; the documented prefix-trust risk).
- --append-verify: verifies the retained prefix against the source's prefix
  xxHash64 before appending and, on a mismatch, falls back to a clean full
  transfer (never a corrupt prefix+tail blend).

New wire frames STATUS_APPEND / STATUS_APPEND_SIG / STATUS_APPEND_OK /
STATUS_APPEND_DATA; PROTOCOL_VERSION bumped 2.9.0 -> 2.10.0 (peers must match).
Both flags imply --incremental and are incompatible with -s (chunk
serialization) and --whole-file (rejected up front). Respects --inplace,
--partial/--partial-dir and --delay-updates via the shared store engine.
2026-09-07 15:43:29 +02:00
TapTap 97f2dc468b feat(sender): --ignore-missing-args / --delete-missing-args CLI and files-from preflight
New flags parse onto the config; --delete-missing-args implies
--ignore-missing-args (order-independent) and does NOT imply --delete (rsync:
independent of other delete processing). The files-from preflight now classifies
listed-but-missing entries instead of hard-failing: under the flags each is
skipped (logged + counted, never silent) and the run succeeds for the rest,
including the all-missing case; an empty list stays a hard error. Under
--delete-missing-args the missing entries' destination mirrors (bare relative
path with -R, full source mirror otherwise) ride the manifest's third section in
both the single-threaded and -m senders; --dirs listed-but-missing entries are
skipped in the scanner.
2026-09-07 14:34:25 +02:00
TapTap c9bd76e633 feat(shared): --delete-missing-args config/wire, manifest third section, receiver exact-path deletions
PROTOCOL_VERSION 2.9.0 -> 2.10.0. The STATUS_MANIFEST frame gains a third
section carrying destination-relative exact-delete paths (the missing
--files-from entries' mirrors); the config frame gains a delete_missing_args
bool (ignore_missing_args stays client-only). The receiver validates the third
section like the keep-set and commits it with manifest_delete_all():
manifest_delete_missing_args runs first (explicit user requests, never blocked
by protected-prefix exclusion protection; staging/basis protected; a non-empty
directory mirror removed only under --force/--delete, rsync parity) and then the
ordinary extras walk. Server --allow-delete gates it like --delete.
2026-09-07 14:34:19 +02:00
TapTap ebab335488 Merge feat/p3-fuzzy: --fuzzy/-y/--no-fuzzy similar-file delta basis 2026-09-06 23:52:24 +02:00
TapTap 87b58975de style(receiver): rework fuzzy DP suffix trim, silence cppcheck FP
CI / lint (pull_request) Successful in 37s
CI / sanitizers (undefined) (pull_request) Successful in 42s
CI / sanitizers (address) (pull_request) Successful in 44s
CI / fuzz-build (pull_request) Successful in 17s
CI / coverage (pull_request) Successful in 34s
CI / valgrind (pull_request) Successful in 36s
CI / build-and-test (pull_request) Successful in 7m19s
The suffix-trim loop using computed end offsets tripped cppcheck's
knownConditionTrueFalse value-range analysis (it unsoundly concluded the trims
always consume the whole middle).  Rewrite it with explicit moving end indices
and add an inline suppression with a rationale for the residual false
positive; cppcheck --error-exitcode=1 is clean again.  The trimming logic is
unchanged and was verified against a full DP reference over 200k random name
pairs.
2026-09-06 23:11:01 +02:00
TapTap bb54113254 docs: all-or-nothing TOCTOU caveat and two-section manifest budget
CI / lint (pull_request) Failing after 32s
CI / build-and-test (pull_request) Skipped
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
The walker's all-or-nothing guarantee holds only while the destination is not
concurrently modified (rehearsal and delete are separate walks).  The
protected-prefix list shares the 16 MB MAX_MANIFEST_BYTES budget with the
keep-set and each section is capped at MAX_MANIFEST_ENTRIES; an over-budget
frame is rejected on the receiver with STATUS_ERROR rather than truncated.
2026-09-06 23:10:42 +02:00
TapTap be37a509a5 perf(receiver): bound the fuzzy edit-distance cost, harden the open
Review follow-ups on the --fuzzy candidate scan:
- Allocate the two DP rows once per directory scan instead of once per
  candidate (4096-entry directories no longer do thousands of malloc pairs).
- Pre-prune before the DP with two cheap lower bounds on the edit distance:
  the name length gap and the count of characters of one basename absent from
  the other; a candidate whose gate (distance*2 <= longer) already fails on
  the max of those bounds is skipped without running the DP.
- Trim the common prefix and non-overlapping common suffix before the DP so
  it only runs over the differing middles.
- Note that the 4096 readdir cap bounds iterations, not per-entry DP cost,
  and that the seen set is filesystem-order dependent (winner stays
  deterministic via the total comparator).
- Open the chosen candidate with O_NONBLOCK so a name raced to a FIFO cannot
  block the receive thread forever in open(2); the existing fstat S_ISREG gate
  still rejects non-regular files.  (The pre-existing basis_open_regular has
  the same latent FIFO pattern and is intentionally left unchanged.)
- Free old_data in receive_delta_file's defensive NULL guard.
2026-09-06 23:05:32 +02:00
TapTap 356b5592e0 feat: add confined symlink-safe directory-tree removal helper
CI / lint (pull_request) Successful in 32s
CI / sanitizers (address) (pull_request) Failing after 42s
CI / sanitizers (undefined) (pull_request) Successful in 43s
CI / fuzz-build (pull_request) Successful in 18s
CI / coverage (pull_request) Successful in 35s
CI / valgrind (pull_request) Failing after 36s
CI / build-and-test (pull_request) Successful in 10m49s
file_remove_tree_secure() opens the final component O_NOFOLLOW below the
authorized root and recursively wipes it with an fd-relative walk (symlinks are
removed by name, never followed); --force uses it to clear a destination
directory that blocks an incoming regular file.
2026-09-06 21:50:25 +02:00
TapTap c4e0de8f08 feat: split delete-manifest frame into keep-set + protected prefixes; enforce --max-delete and --force on the receiver
The STATUS_MANIFEST frame now carries two count-delimited sections: the kept
paths and a protected-prefix list (excluded-on-source paths the walker must not
delete unless --delete-excluded opted out).  The receiver's DeleteManifest is
passed through the commit/early paths unchanged.  manifest_delete_extras
honors a client --max-delete (all-or-nothing) and produces a distinct error for
it versus the 100000-entry server bound.  --force clears a non-empty directory
that blocks an incoming regular file (confined, symlink-safe) via a new
file_remove_tree_secure helper.
2026-09-06 21:50:02 +02:00
TapTap 1c9b660ac0 feat: all-or-nothing bounded delete walker
delete_extras_limited now rehearses a finite-capped deletion before unlinking
anything (an identical fd-relative walk that counts files and directories) and
returns DELETE_WALK_LIMIT_EXCEEDED with nothing removed when the run would
exceed the cap, so --max-delete is enforced per run instead of truncating the
deletion.  A directory that still holds entries the walker leaves in place
(protected excluded prefix, manifest-kept file, symlink) is left behind rather
than failing the whole deletion, matching rsync's leave-non-empty-dirs
behavior.  Rehearsal/delete each open an independent file description so a
prior pass cannot drain the directory stream.
2026-09-06 21:49:57 +02:00
TapTap 3631df0a3e feat: add delete-policy config fields and CLI flags
Adds ignore_errors (client-only) and force_delete (wire) booleans, makes
max_delete default -1 (no client limit), and parses --delete-excluded,
--max-delete=NUM, --ignore-errors, --force, --prune-empty-dirs.  Bumps
PROTOCOL_VERSION 2.8.0 -> 2.9.0 for the new on-the-wire force_delete field.
2026-09-06 21:49:53 +02:00
TapTap cebd23a239 feat(receiver): --fuzzy similar-file delta basis
When a file must be transferred and the destination holds no usable content
at the exact path (file absent, or outside the delta engine's size bounds),
the receiver now searches the target's own destination directory for an
existing regular file with a similar basename and uses it as the delta basis
via the existing receiver-driven STATUS_DELTA_SIGNATURE handshake.  The
sender never learns the basis was another file, so no wire change beyond the
new config flag was required.

Heuristic (deterministic, simpler than rsync's, documented): candidates are
sibling entries confined below the root and opened O_NOFOLLOW (symlinks are
never followed, nothing outside the destination root is read); dotfiles,
directories, the target's own name and stage/temp names are excluded; size
gate is delta_should_attempt; name gate is a Levenshtein distance <= half
the longer basename; the closest candidate (size tie-break, then lexical) is
loaded; the scan is capped at 4096 entries.

Byte-exactness is independent of the basis: block matches are verified by
Adler-32 + xxHash32, delta_apply validates every reference, and a basis that
shares nothing makes the sender reply with a whole-file transfer.  When no
candidate qualifies the normal whole-file transfer runs unchanged.
2026-09-06 21:04:19 +02:00
TapTap f099a6e20f feat(config): add fuzzy flag and bump protocol to 2.9.0
-y/--fuzzy is receiver-side similar-file basis selection, so the receiver
must learn the flag: add a bool to Config, serialize it as a trailing field
on the config frame, and validate the received value.  Because the config
frame layout changed, PROTOCOL_VERSION moves to 2.9.0 (peers must match).
2026-09-06 21:04:11 +02:00
TapTap 01a5a93089 Merge feat/p3-basis-dest: alternate basis dirs (--compare-dest/--copy-dest/--link-dest) 2026-09-06 20:10:40 +02:00
TapTap 4bf4da37e5 fix: free basis path on copy-dest hits; keep --delete staging skip top-level-only
- copy-dest basis hits leaked the heap-allocated basis path: BASIS_DEST_COPY
  did not transfer it (only LINK does) and returned before the basis cleanup.
  basis_match_free is now called on every materialization return path (success
  and send-failure) after content/link ownership is transferred.
- the --delete walker regression: the delay-updates staging name must be
  protected only as a DIRECT child of the receive root, while basis dirs may
  be skipped at any depth.  delete_extras_limited now takes DeleteSkipEntry
  entries carrying a top_level_only flag instead of a flat prefix list, so a
  nested destination directory named .fastsync-stage is ordinary content again
  (its extras are deleted) and a basis tree is still never removed.
2026-09-06 19:43:11 +02:00
TapTap 08a5815ca7 refactor: unify basis-dir path validation and normalization
config_basis_path_valid and config_basis_append now share one normalizer
(basis_path_normalize): interior empty components (a//b) collapse, '.'
components and trailing slashes are dropped, and the stored form is exactly
the canonical relative path used by validation, the delete-walker prefix match
and the receiver's basis lookup.  Degenerate inputs (empty, absolute, '..',
'.' that normalizes to nothing) stay rejected.
2026-09-06 19:43:06 +02:00
TapTap ebfaced5c2 fix: wait for the early-delete ACK with an extended deadline
The receiver performs the whole bounded deletion walk (up to
MAX_SERVER_DELETE_COUNT unlinks) before answering the delete-before/during
manifest, so its STATUS_OK reply can take far longer than the default 60 s
per-message receive window. Waiting with the default would make the sender
abort AFTER the deletion had already committed on the receiver. Add a timed
receive variant (receive_status_timed / protocol_receive_n_data_timed) and use
it for the early-manifest ACK with a 1 h explicit deadline; connection errors
and EOF still abort immediately.
2026-09-06 19:35:21 +02:00
TapTap 4e725517f0 feat: implement rsync delete timing (--delete-before/--delete-during/--delete-delay/--delete-after)
Deletion timing is now real and selected by the four rsync flags plus the
plain --delete default. Wire protocol bumps to 2.8.0: two new config
booleans (delete_during, delete_delay) are serialized and validated, joining
the existing delete_before/delete_after.

- Early modes (--delete-before, --delete-during/--del): the sender pre-scans
  the whole tree (paths only), transmits the keep-set manifest BEFORE any
  file data, and the receiver removes extras and acks STATUS_OK; the sender
  only streams data after the deletion committed. Deletion is thus performed
  even if a later transfer phase fails (rsync delete-before/during are
  destructive by definition). FastSync streams in a single scan so it cannot
  interleave per-directory like rsync delete-during; --delete-during selects
  the same engine mode as --delete-before (documented divergence).
- Late/commit modes (plain --delete, --delete-after, --delete-delay): the
  manifest closes the data stream and deletion is committed only after
  STATUS_FINISHED proves the whole transfer succeeded, preserving FastSync's
  commit-style safety. --delete-delay converges with --delete-after because
  FastSync never snapshots the destination during data flow (documented).
- The STATUS_MANIFEST frame is now self-delimiting and position-independent.
  Single-threaded receivers delete before the success frame; the -m receiver
  hands the keep-set to server.c, which commits the deletion only after the
  disk writer thread has drained (fixes a delete-vs-in-flight-temp race).
- Every timing flag implies --delete; at most one timing flag is allowed.
- Each timing flag implies --delete, matching rsync; conflicts are rejected.
2026-09-06 18:53:20 +02:00
TapTap 8f846a43b8 feat: exclude basis directories from --delete
Generalize the delete walker's protected-root-child skip into a prefix list.
The receiver now passes both the --delay-updates staging directory and every
basis-dir path, so a --delete run can never treat a basis snapshot (which a
--link-dest run just linked from) as destination content to remove.
2026-09-06 18:47:29 +02:00
TapTap d38920c972 feat: receiver-side basis matching with link/copy materialization
The receiver's per-file incremental check now consults the ordered basis-dir
list whenever the destination is not already up to date.  An exact basis match
requires equal size, equal mtime (unless --size-only; --ignore-times disables
basis matching like rsync), and an equal content xxHash64 -- the sender sends
its xxHash for every file whenever basis dirs are configured (not only under
--checksum), so a hard link or local copy is only ever made from byte-identical
content.

On a match:
  - compare-dest: reply STATUS_OK and skip data only when the destination does
    not already hold the file (sparse, rsync parity).  A destination that holds
    a DIFFERENT version falls back to a normal transfer instead of rsync's
    delete, keeping the mirror complete.
  - copy-dest: reply STATUS_OK and hand a synthetic File (bytes read from the
    basis file, basis metadata) to the normal store sink, so the file is
    installed as a real local copy through the existing atomic temp+rename
    engine and honors --existing/--ignore-existing/--update/--backup/
    --delay-updates/--partial-dir unchanged.
  - link-dest: same, but File.basis_link records the basis path and the store
    engine calls the new file_to_disk_secure_link(): an atomic temp hard link +
    rename.  Cross-filesystem/refused links fall back to a byte-identical local
    copy (never a corrupt or partial file); the copy fallback applies metadata,
    while a successful link keeps the basis inode's own attributes so the basis
    file is never mutated.

Basis-materialized files carry File.skip so they are not acknowledged to a
--remove-source-files sender (the sender already saw STATUS_OK and keeps the
source).  The no-match path is byte-for-byte identical to the existing delta /
full-data transfer.
2026-09-06 18:47:26 +02:00
TapTap df890f76ea feat: basis-dir config/CLI for --compare-dest/--copy-dest/--link-dest
Replace the vestigial single compare_dest/copy_dest/link_dest Config fields with
an ordered BasisDest list (type + path per entry) that is serialized to the
receiver and interpreted relative to the destination root.  Paths must be
relative with no '.'/'..' components (confined like --backup-dir); trailing
slashes are normalized.  Wire layout changes, so PROTOCOL_VERSION -> 2.8.0.

CLI: each flag is parsed in both --flag=DIR and --flag DIR forms, is
repeatable, and keeps command-line order as basis priority.  Supplying any
basis dir implies --incremental (and therefore metadata) on the sender because
the unchanged decision is receiver-side; combining basis dirs with -s chunk
serialization is rejected in validate_config.  Usage text updated.
2026-09-06 18:47:20 +02:00
TapTap 3275b6c978 fix: address c-review for -R/--dirs/--mkpath row
B1: destination-root existence/creation mishandled two legitimate forms.
file_directory_exists_secure/file_ensure_directory_secure now normalize a
trailing-slash destination (so the last component is never an empty leaf)
and treat a destination equal to the already-open authorized root as present
(no spurious <root>/<basename> nested dir with --mkpath). Confinement and
O_NOFOLLOW probing are unchanged. Regression integration tests: existing
dest with trailing slash works with and without --mkpath; dest == authorized
root works and creates no stray nested dir.

W1: chunk serialize/deserialize round-trip unit test for a directory entry
mixed with a regular file, with and without metadata; --dirs coverage under
-s and -s -m.
W2: --list-only and --dry-run now print file_wire_path() so all outputs show
the -R transformed relative name, matching -i/--out-format.
W3: RSYNC_COMPAT -d/--dirs note: dirs are created immediately under
--delay-updates (only regular files are staged).
W4: --dirs generator flushes chunks by element count too, so a long
--files-from list of empty directories cannot exceed the per-chunk file cap.
N1: STATUS_MKDIR added to status_to_string.
N2: removed dead TestMkpath._transfer.
N3: removed redundant --no-implied-dirs OPTION_TABLE row.
N4: integration tests: --dirs --delete keeps the just-created empty dir (and
deletes extras); a listed dir colliding with a regular file at the dest fails
cleanly.
2026-09-06 16:10:00 +02:00
TapTap c2cf231158 feat: implement -R/--relative, --no-implied-dirs, --dirs/-d, --mkpath
CI / lint (pull_request) Successful in 26s
CI / sanitizers (address) (pull_request) Failing after 41s
CI / sanitizers (undefined) (pull_request) Successful in 40s
CI / fuzz-build (pull_request) Successful in 18s
CI / coverage (pull_request) Successful in 35s
CI / valgrind (pull_request) Failing after 37s
CI / build-and-test (pull_request) Failing after 2m35s
RSYNC_COMPAT Phase-2 row M: path-list construction and destination
directory creation while preserving traversal safety.

- -R/--relative with --files-from: transmit each listed entry under its
  bare relative destination path (no source-root mirror). Files keep an
  absolute local read path plus a separate wire/dest path (File.send_path);
  manifest, incremental quick-check and change output follow the wire path,
  so --delete and --remove-source-files stay consistent. -R without
  --files-from is unchanged (full mirror).
- --no-implied-dirs: client-only, only meaningful with -R + --files-from.
  A listed file whose parent dir is not itself (or via an ancestor)
  explicitly listed cannot be placed; the run fails up front with a clear
  error. No effect otherwise.
- --dirs/-d + --old-dirs/--old-d aliases: -d <dir> transmits the source
  root as an explicit empty directory entry (STATUS_MKDIR frame); with
  --files-from listed dirs are created empty and listed files transferred,
  never descending. Works single-threaded, -m (sequential scanner in the
  -m scan thread) and chunk-serialization (per-file type marker).
  Directory entries appear in the delete manifest.
- --mkpath: new wire bool; server creates the destination root (and missing
  leading components under its authorized root) at connection start. A
  missing destination root is now rejected by default.
- Protocol bumped to 2.7.0 (mkpath wire field + STATUS_MKDIR + chunk type
  marker). All receive paths funnel through file_save_to_disk_full which
  creates directories via the secure confined mkdir engine; dir entries are
  excluded from --remove-source-files outcome acknowledgements on both ends.
- Unit coverage: CLI parse (relative/dirs aliases/mkpath/no-implied-dirs),
  config round-trip (relative + mkpath), scanner --dirs non-recursion and
  -R send_path (sequential + parallel), receiver dir-entry save.
- Integration coverage: TestRelativeFilesFrom, TestNoImpliedDirs, TestDirs,
  TestMkpath (single and -m).
- RSYNC_COMPAT: 4 rows move to Implemented (Summary 67/3/5/1/71 = 147).
2026-09-06 15:28:32 +02:00
TapTap 342dd152ef Merge feat/p2-files-from-filter: files-from/from0/filter/-F/-C
Client-side file-list selection and filter-rule layer (client-only; no wire
change). rsync-consistent inner-first per-dir filter precedence; listed-but-
missing files-from entries hard-error; root '/' scan regression fixed;
Summary recounted (63 implemented / 75 not).
c-review REQUEST CHANGES -> blockers fixed; PR #265.
2026-09-06 14:25:51 +02:00
TapTap 4295fefaa3 fix: --delay-updates delete/backup collisions, publish-failure test, staging lock
Review fixes for --delay-updates:

- --delete no longer deletes the staged files: the delete walker gains a
  skip_root_child parameter and receive_manifest passes DELAY_UPDATES_STAGING_DIR
  when delay_updates is active, so deletion removes genuine extras while the
  staging dir (a direct child of the receive root) is left for publication in
  both single and -m modes.
- --backup-dir is rejected when it collides with the reserved internal staging
  name .fastsync-stage (trailing slash normalized), in client validation and in
  the received-config wire validation, preventing old backups from being
  silently installed as new files.
- Staging dir is now held under an exclusive advisory flock for the whole
  transfer (context lifetime): two simultaneous delayed transfers to one
  destination root no longer share/destroy each other's staged data - the
  second fails cleanly.  Cleanup only touches the staging dir when this context
  owns the lock, so a lock-contention failure cannot wipe a live session.
- Post-publish staging cleanup now returns/logs instead of discarding failures
  (warning when the staging dir cannot be fully removed).
- Reworked the publish-failure integration test to exercise real mid-publish
  semantics (top-level file published, nested rename fails, no rollback,
  sources retained under --remove-source-files) and added integration tests for
  --delete + --delay-updates ordering and reserved --backup-dir rejection.
- RSYNC_COMPAT note documents delete ordering, the reserved-name hazard, and
  the concurrency guard.
2026-09-06 14:19:18 +02:00
TapTap 7f2180ef90 fix: filter precedence, files-from errors, NUL/CRLF and filter-rule rejections
Address an independent c-review of the files-from/filter feature:

- .rsync-filter precedence now matches rsync: evaluate the innermost
  (current) directory's rules first, then ancestors, then the command-line
  base (--filter/-C), so a deeper file's '+' can re-include what a shallower
  '-' excluded (regression tests in both scan modes; single-thread and -m).
- --files-from: a listed entry missing on disk and an empty list are now hard
  errors surfaced pre-transfer in send_files, send_files_multithreaded,
  dry-run and --list-only; '.' (whole tree) and empty listed dirs stay valid.
- scanner_path_relative now handles a transfer root of / (previously the
  scanner aborted on children of /).
- Reject unsupported rsync filter syntax explicitly (no silent no-ops):
  +/- modifiers other than '/' (! C s r p x) and rules beginning with ':'/'.'
  /'!' (merge/dir-merge/list-clear shorthands). Docs updated.
- -0/--from0 NUL mode preserves entry bytes (no CR/LF trimming); only newline
  mode trims. Absolute-entry error message no longer includes the newline.
- --no-from0/--no-cvs-exclude registered as negatable booleans.
- RSYNC_COMPAT rows updated for the precedence, rejection list, NUL-mode
  detail and the documented O(entries x files) scalability bound of the
  allow-set (Summary unchanged: 62/3/5/1/76 = 147).
2026-09-06 14:16:12 +02:00
TapTap f4c15a7b78 feat: add --files-from/-0, --filter, -C and -F filter layer
CI / lint (pull_request) Successful in 23s
CI / sanitizers (address) (pull_request) Successful in 40s
CI / sanitizers (undefined) (pull_request) Successful in 39s
CI / fuzz-build (pull_request) Successful in 17s
CI / coverage (pull_request) Successful in 34s
CI / build-and-test (pull_request) Successful in 1m28s
CI / valgrind (pull_request) Successful in 36s
Implement the RSYNC_COMPAT Phase-2 filter/parser feature group:
- --files-from=FILE (repeatable) plus -0/--from0 NUL delimiters: parse the
  source file list relative to the source root into a shared read-only
  allow-set; the scanner transfers listed files and the whole subtree of
  listed directories and prunes everything else in single- and
  multithreaded mode. Absolute/'..' entries and missing files are hard
  CLI errors.
- --filter=RULE: rsync-style +/- rules (anchored '/', dir-only trailing '/',
  word include/exclude forms) evaluated first-match-wins with a default of
  include, as an independent layer from legacy --exclude/--include.
  Unsupported directives (merge/hide/... ) are rejected explicitly. -f stays
  sendfile.
- -C/--cvs-exclude: well-known rsync CVS default exclude set.
- -F: per-directory .rsync-filter files read during traversal and applied to
  the owning directory's subtree (single + parallel), never transferred.
- Delete manifest still derives from what was actually sent.

Client-only config fields; no wire/protocol change. Adds unit coverage
(CLI parse, allow-set and filter scanning single+parallel) and integration
tests (TestFilesFrom, TestFilters). RSYNC_COMPAT matrix rows updated:
5 rows move to Implemented (Summary 62/3/5/1/76 = 147).
2026-09-06 13:43:34 +02:00
TapTap a2a82dd856 feat: implement --delay-updates receiver staging and publication
CI / lint (pull_request) Failing after 22s
CI / build-and-test (pull_request) Skipped
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
Stage every successfully written file under a private 0700 .fastsync-stage
directory inside the receive root and atomically publish all staged files
only after the whole protocol stream (manifest/delete handling included)
has completed, immediately before the success/outcome frame.  On any
abort/error before publication nothing is installed and staging is removed;
a publish failure aborts the transfer with best-effort cleanup of the
remainder (already-published files are not rolled back).  Crash leftovers
are wiped when the next delayed transfer starts.

Wire: new delay_updates config flag (selection-options block), protocol
version bumped to 2.6.0, client/server validation rejects --inplace.
CLI/usage/validation updated.  Works in single-threaded and -m modes
(exactly one write_thread stages files; the staged-file registry is
mutex-protected; publication runs once after both threads join).
--existing/--ignore-existing/--update decide against the final destination
at stage time; --backup is deferred to publication.  remove_source_files
outcomes are only sent after publication so skipped/unpublished sources are
never deleted.  Default (no flag) behavior is unchanged.

Tests: config wire round-trip, CLI parse, --inplace rejection, new
test_delay_updates unit suite (27 suites total), and integration
TestDelayUpdates covering single/-m parity, incremental reruns, remove
source files, receiver-skip ordering, and a deterministic publish-failure
abort path.
2026-09-06 13:20:03 +02:00
TapTap 8577f95550 fix: cppcheck cleanups (printf sentinel type, scanner test style)
CI / lint (push) Successful in 22s
CI / sanitizers (address) (push) Successful in 40s
CI / sanitizers (undefined) (push) Successful in 39s
CI / fuzz-build (push) Successful in 16s
CI / coverage (push) Successful in 34s
CI / build-and-test (push) Successful in 1m23s
CI / valgrind (push) Successful in 36s
2026-09-06 12:56:54 +02:00
TapTap 88ab4f65cb Merge feat/p2-itemize-output: structured change output (-i, --list-only, --out-format, --log-file-format)
Client-only per-file event/format model (change_list.c). Reviewed
(c-review APPROVE WITH NITS, all fixed); PR #263.
2026-09-06 12:52:46 +02:00
TapTap 9fbb86ca68 Merge feat/p2-temp-dir: implement --temp-dir
Receiver writes temps into a confined scratch dir and atomically renames
into place; EXDEV aborts; inplace/partial bypass; thread-safe temp names.
Uses existing wire field; no protocol bump. Reviewed (c-review APPROVE
WITH NITS, all fixed); PR #262.
2026-09-06 12:52:11 +02:00
TapTap 00e8df4bcd fix: address c-review nits for --temp-dir engine
CI / lint (pull_request) Failing after 21s
CI / build-and-test (pull_request) Skipped
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
2026-09-06 12:48:07 +02:00
TapTap 7f5547e900 feat: add structured per-file change output model (-i, --list-only, --out-format, --log-file-format)
Implement rsync-style itemized output backed by one shared change-event
engine (src/client/change_list.c):
- -i/--itemize-changes prints ">f+++++++++ <path>" for files actually sent
  (single-threaded and -m); unchanged files print nothing.
- --list-only prints an ls-style listing of files that would be transferred
  without contacting the server or writing anything.
- --out-format=FORMAT prints a printf-style template per changed file
  (tokens %%f %%n %%l %%b %%M %%%%; unknown escapes preserved).
- --log-file-format=FMT logs each transferred file when --log-file is set.
Events are emitted from the per-file sender path shared by both transfer
modes, so the single sender thread is the only reporter (no races).
2026-09-06 12:34:10 +02:00
TapTap 19e6fc2205 feat: implement rsync --temp-dir for atomic receiver installs 2026-09-06 12:21:36 +02:00
TapTap 2bcc20aa08 feat: add -x/--one-file-system to stay within the source filesystem
Capture the transfer root's device (st_dev) at scanner creation and skip
descending into any subdirectory on a different device (a mount point).
Implemented sender/client-side only: sequential BFS and parallel (-m) root
scan apply the same scanner_same_filesystem decision; no wire/protocol change
and default behavior is unchanged. Unit tests cover the pure decision, same
device scanning in both modes, and CLI parsing; integration tests prove -x
leaves a single-filesystem tree byte-identical and, when root can mount a
tmpfs, skips a genuine cross-device subtree.
2026-09-06 12:18:43 +02:00
TapTap f7c6c91e13 fix: set *failed on delta oversize branch; add -m RSF skip test
CI / lint (push) Successful in 20s
CI / sanitizers (address) (push) Successful in 41s
CI / sanitizers (undefined) (push) Successful in 39s
CI / fuzz-build (push) Successful in 16s
CI / coverage (push) Successful in 34s
CI / build-and-test (push) Successful in 1m23s
CI / valgrind (push) Successful in 35s
Review nits from independent review of the four fix branches:
- receive_delta_file STATUS_NEXT oversize branch now sets *failed=true
- receive_incremental_check oversize branch returns NULL (receiver sends the
  single STATUS_ERROR) instead of double-sending
- add multithreaded -m --ignore-existing --remove-source-files integration
  coverage so the writer-thread outcome path is exercised
2026-09-05 13:16:42 +02:00