Commit Graph
18 Commits
Author SHA1 Message Date
TapTap f2c89b6e7c fix: mutex leak, errno-after-free, log_perror misuse, status validation
- multiprocessing: destroy mutex_progress on the dir_entries_mutex
  init-failure path (init >= 7); drop bogus log_perror
- delete_plan: capture errno before free() in apply_deferred_path
- queue/array_list: log_message instead of log_perror for non-errno
  conditions
- protocol: reject unknown wire Status values via status_is_valid() in
  receive_status, receive_status_timed and the keepalive reader; declare
  protocol_receive_status_timed in protocol.h
- protocol: %llu for unsigned long long debug counters
- tests: out-of-range status rejection test
2026-09-21 18:52:46 +02:00
TapTap b235721f8b delete: rsync-exact abort boundary and -d per-directory plans
Transmit the complete --delete-during/--delete-delay per-directory plan
set before the first data frame, so a mid-transfer abort has already
applied every planned removal like rsync's generator; completed runs are
unchanged.  Route -d/--dirs through the same per-directory plans: the
generator records only directories whose direct children it enumerated,
so extras directly inside a listed directory are removed while an
untraversed subdirectory's mirror is shielded (rsync's -d DIR/ --delete).
Also shields a -x mount point's untraversed destination content.
2026-09-20 14:22:22 +02:00
TapTap 9691dba6f0 delete: reproduce rsync traversal order for extras removal
Collect each directory's entries up front and process extraneous
subdirectories first (descending name, depth-first), then extraneous
files (descending name), then descend into kept subdirectories in
ascending order.  Emit a trailing slash for deleted directories in
observers/dry-run output.  This matches rsync's delete order for
--delete-before/--delete-after/--delete-delay and for dry-run listings.
2026-09-20 13:15:04 +02:00
TapTap 36fd0774e8 feat(delete): default --delete to rsync delete-during; add --delete-commit
- plain --delete with no timing flag now selects delete-during (progressive
  deletion, matching rsync and avoiding the full old+new tree peak)
- new long-only FastSync --delete-commit restores the old atomic behavior
  (delete only after the whole transfer succeeds); timing-identical to
  --delete-after, implemented via the same wire bool
- timing flags are mutually exclusive; --delete-commit conflicts with other
  timings; --delete-before/--delete-during rows reworded per Phase-0 probes
- CHANGELOG migration note; tally unchanged 116/14/27
2026-09-19 16:29:48 +02:00
TapTap 82959395fb feat(filter): receiver-side protect/risk engine for dest-only entries
- new bounded config-wire block (BLOCK_PROTECT_RULES) serializes the sender's
  compiled filter rules to the receiver (bounded count + 256 KiB patterns;
  strict action/sides validation)
- receiver evaluates protect/risk in the whole-tree extras walk and the
  per-directory delete plans, so a dest-only entry matching 'P' is kept like
  rsync; dry-run would-delete enumeration also honours it
- --filter flips to parity (115/11/31); per-dir merge receiver re-derivation
  remains the documented residual
2026-09-19 13:52:48 +02:00
TapTap 25062352f6 fix(parity): dry-run delete protections, delete-delay actual-removal budget, --info name2
- -n/--delete sends the same protected/size-skipped/scope as a real run, so
  the read-only would-delete walk no longer over-reports (row -> caveat)
- --delete-delay charges --max-delete on actual removals and recursively
  re-scans a refilled deferred directory at commit; independent deferred cap
- --info=name emits the leading ./ root line and name2 'is uptodate' lines
- differential tests promoted from residual pins to rsync parity assertions
2026-09-19 09:25:07 +02:00
TapTap eb7e3fd2e0 test(parity): option-wave rsync differentials + docs (109/21/26)
- tests/integration/test_option_parity.py: bwlimit parse matrix + throttle
  rate, --info flist/name/nonreg/del(dry+real+itemize)/remove, real-setpriv
  --ignore-errors, rsync-daemon -M forwarding evidence, filter protect/risk
  destination-only divergence pin.
- RSYNC_COMPAT.md: tally 109/21/26; --bwlimit and --ignore-errors -> Parity,
  --filter and -M -> Divergent with differential rationale, --info residual
  narrowed to the categories with no client-observable event.
- HANDOFF/README: protocol 2.27.0, option-wave summary.
2026-09-18 21:16:12 +02:00
TapTap 6a129b54d4 feat(parity): real --info=del deletion lines + rsync throttle pacing
- Wire: config frame gains report_deletes (protocol 2.26.0 -> 2.27.0); the
  receiver lists actually-removed paths in the STATUS_STATS path list, so the
  sender prints rsync's `deleting PATH` / `*deleting   PATH` lines for a real
  --delete run (and -i/out-format).  Observers threaded through the manifest,
  missing-args and per-directory delete engines; golden wire len/hash updated.
- bwlimit: throttle now paces like rsync 3.4.1 -- ~100ms burst capacity and the
  sleep is no longer credited as refill, so 4 MiB at 1024/2048 KiB/s matches
  rsync within ~4% (was ~2x too fast).
2026-09-18 21:06:20 +02:00
TapTap 9fa1696eff fix(parity): actual-removal delete-delay counts, rsync-accurate stats/progress/%C
- --delete-delay: count/track only entries actually removed; a directory
  refilled before commit (ENOTEMPTY) no longer inflates Number of deleted
  files or the --max-delete budget (unit + integration + rsync differential).
- --stats: per-type Number of files breakdown; only stored regular files
  count as transferred; transferred/literal byte totals and Total file size
  (symlink target lengths) now match rsync for whole-file transfers.
- --progress: print the leading ./ root line and include the root entry in
  the to-chk denominator (single-file output byte-identical to rsync).
- --out-format %C: use the selected transfer checksum and render every
  algorithm exactly like rsync; checksum_digest_file gains md4/sha1/none.
  Reclassify --out-format to Divergent (protocol-specific %b/delta-%c).
- Docs: RSYNC_COMPAT tally 107/25/24, HANDOFF update. No wire change.
2026-09-18 20:13:29 +02:00
TapTap bbecff9c04 fix(delete): keep the empty-scan safety guard file-only
Adding every traversed directory to the per-directory plan keep set must not
make an I/O-errored partial scan look non-empty.  Count only transmitted file
entries for delete_plan_sender_empty(), so a scan that hit an unreadable
directory and found no files still refuses to delete.
2026-09-17 01:34:26 +02:00
TapTap 6c6f02e5dd fix(parity): empty-dir delete, per-dir filter errors, -R protect, stats parser
Blockers addressed together (shared scanner/delete-plan plumbing):

* #10: an empty in-scope source directory produced no plan keep entry, so the
  receiver deleted the destination directory itself.  The scanner now records
  every traversed directory into a delete-plan sink, the plan sender keeps them,
  and any directory whose plan the data stream never triggered is emitted after
  the data so its extras are still removed.  Differential tests cover
  --delete-during and --delete-delay.
* #8: an invalid per-directory filter file was silently ignored when an earlier
  merge file in the same directory existed; key the failure off the error text
  (both sequential and parallel scanners) and fail the scan.
* #9: -R + --files-from receiver-protect rules recorded the source-relative
  path; record the bare relative wire path in both scanners so the protected
  destination mirror survives --delete.
* #5: the STATUS_STATS would-delete parser now validates each retained path and
  enforces the shared MAX_MANIFEST_BYTES budget, and the --out-format dry-run
  delete line is escaped like the itemize line.
* #11: drop the unused DELETE_PLAN_MAX_NAMES macro, log the delete-limit
  warning once per session, roll back dir-merge names from a per-directory file
  that fails to parse, and guard every filter error snprintf against err==NULL.

#10 leaves the empty directory itself kept and its extras removed, matching
rsync's final state on both per-directory timings.
2026-09-17 01:21:06 +02:00
TapTap e32733fbf6 feat(stats): populate receiver wire counters on both receive paths
The single-threaded and -m receivers never populated ReceiverStats.matched_data
or .deleted_files, so --stats always printed 0 for both even when rsync
reported nonzero.  Track the bytes reconstructed from the basis file while
applying a delta, and tally the delete-commit counts (manifest and
per-directory sessions) into the receiver stats.  The -m pipeline now carries
its own stats/would-delete fields and emits the STATUS_STATS frame before the
terminal success, so --threads finally reports the counters and renders
-n --delete  lines.

Also normalize the -n --delete would-delete enumeration's absolute basis
prefixes exactly like the real commit path (fixing an over-report) and fix the
basis_delete_relative off-by-one when the receive root is '/'.  Unit tests
cover the root mapping and the basis protection; integration tests cover
matched/deleted stats for both receivers and the --threads dry-run delete
lines.
2026-09-17 01:08:32 +02:00
TapTap b02799327d fix(delete): guard the per-directory delete commit against dry-run
delete_plan_session_commit() lacked the central no-mutation guard that
manifest_delete_all() has, so a server-contacting -n run (or a hostile plan
frame) could still remove --delete-missing-args mirrors on the per-directory
timing path.  Return DELETE_COMMIT_OK immediately when the session is a
dry-run, and gate the receiver/server commit call sites too.  Add a unit test
that streams a plan naming an existing destination file and asserts it
survives.
2026-09-17 01:02:22 +02:00
TapTap 946aa934cc fix(delete): scope -R per-directory delete walk to the transferred prefix
The -R prefix marker installed in synced_dirs was discarded when finalizing
the per-directory delete sender (--delete-during/--delete-delay), so the
up-front root plan was the receive root '.', whose keep list only held the
first prefix component.  The receiver then deleted destination content
outside the transferred prefix (e.g. unrelated/keep.txt), a data-loss bug;
rsync keeps it.

Confine the walk to the -R prefix: send that prefix's plan as the root plan,
only transmit plans at or below it, and never emit the receive root plan for
a scoped run.  Add a differential test covering both --delete-during and
--delete-delay.
2026-09-17 01:00:55 +02:00
TapTap dbf1b39d47 fix(delete): share the per-frame manifest byte budget across delete-plan sections 2026-09-16 23:36:46 +02:00
TapTap e771cc9da6 fix(delete): guard per-dir missing-args by server policy; fall back for --dirs
- Only honor the --delete-missing-args exact paths when the server's
  --allow-delete policy left delete_missing_args set.
- -d/--dirs does not recurse, so a per-directory plan would carry no child
  information and could delete the contents of an untraversed directory; fall
  back to the whole-tree end-of-transfer commit for that mode.
2026-09-16 23:26:14 +02:00
TapTap d6295d62ce test(delete): differential + timing regression tests for per-directory delete plans
- Compare --delete-during/--delete-delay final state against rsync 3.4.1.
- Force a mid-transfer failure through a byte-slicing proxy: --delete-during has
  removed the processed directory's extra, --delete-delay has not.
- Create a destination entry while the transfer is in flight: it survives
  --delete-delay's snapshot but is removed by --delete-after's fresh end scan.
- Cover the --delete-delay type-conflict case now matching rsync.
2026-09-16 22:50:51 +02:00
TapTap 448edc0432 feat(delete): per-directory delete plans for --delete-during/--delete-delay (protocol 2.24.0)
Stream one delete plan per source directory from sender to receiver instead of
a single whole-tree keep-set manifest:

- --delete-during applies each directory's extras as its plan arrives, before
  that directory's data (rsync's generator-order deletion).
- --delete-delay snapshots each directory's extras while the plan arrives and
  commits the removals only after a fully-successful transfer, so files created
  after the scan survive (matching rsync's delete-delay, not delete-after).
- Type conflicts (a destination file blocking a source directory, or vice
  versa) are cleared immediately in both modes, so the nested write succeeds.

The plan carries the destination-relative directory, its kept child directory
names and its kept child file names; the first frame also carries the global
protected prefixes, size-skipped prefixes and --delete-missing-args paths.
--delete-before keeps the existing whole-tree early manifest; plain --delete and
--delete-after keep the end-of-transfer manifest commit.

Preserves the existing safety surface: protected/size-skipped prefixes and the
--delay-updates/basis skips are honored at any depth, deletion is scoped to the
synchronized directories (--files-from), MAX_SERVER_DELETE_COUNT and
--max-delete (partial + exit 25) are shared across plans, symlinks are never
followed, and paths are confined to the receive root.
2026-09-16 22:45:26 +02:00