- receiver stats: exclude basis-dir materializations (--link-dest/--copy-dest)
from created/literal tallies; rsync reports 0 for a basis hit, so a fresh
--link-dest --stats run now matches (differential test_link_dest_stats_matches_rsync)
- filter wire block: reject a pattern above the glob evaluation bound and lower
MAX_FILTER_RULES to 1024, so a crafted rule list cannot amplify delete-walk
glob work or install a rule that silently never protects
- negative tests for over-cap count and over-long pattern
- README: correct --delete default, --stats/--progress description, add
--delete-commit; RSYNC_COMPAT stale version labels/overclaim fixed
- plain --delete with no timing flag now selects delete-during (progressive
deletion, matching rsync and avoiding the full old+new tree peak)
- new long-only FastSync --delete-commit restores the old atomic behavior
(delete only after the whole transfer succeeds); timing-identical to
--delete-after, implemented via the same wire bool
- timing flags are mutually exclusive; --delete-commit conflicts with other
timings; --delete-before/--delete-during rows reworded per Phase-0 probes
- CHANGELOG migration note; tally unchanged 116/14/27
Probe shows the candidate choice is observable only as --stats bandwidth
counters (tree/exit always identical). FastSync already uses rsync's
fuzzy_distance/find_filename_suffix name heuristic; the residual is the
narrower delta eligibility window (>=16 KiB, <=10x) vs rsync's wider one.
Row -> caveat; tally 116/14/27.
Adds the exclude_protect_dest_only differential and flips --delete-excluded
to parity now that receiver-side rules protect a destination-only excluded
entry like rsync; tally 116/10/31.
- new bounded config-wire block (BLOCK_PROTECT_RULES) serializes the sender's
compiled filter rules to the receiver (bounded count + 256 KiB patterns;
strict action/sides validation)
- receiver evaluates protect/risk in the whole-tree extras walk and the
per-directory delete plans, so a dest-only entry matching 'P' is kept like
rsync; dry-run would-delete enumeration also honours it
- --filter flips to parity (115/11/31); per-dir merge receiver re-derivation
remains the documented residual
Probe shows the block-checksum choice is not observable in the parity surface:
%c, Matched/Literal data and the destination tree are invariant across
xxh64/xxh128/xxh3/md5/md4/sha1 and the transfer,pre-transfer form; only %C
changes, and it is byte-identical to rsync. FastSync's fixed xxHash32 block
strong sum is collision-safe within the payload cap. Row -> parity (114/11/32).
- when --progress/--info=progress is requested, a metadata-only pre-scan
builds the full file-list total and directory names so the to-chk
denominator counts every regular/dir/link/special entry like rsync
- per-directory/symlink/special name lines emitted; sequential and --threads
- reuses the --delete-during/delay pre-scan when present; non-progress runs
take no extra pass
- --progress stays a caveat (emission order still differs); single-file output
remains byte-identical
- PROTOCOL_VERSION 2.27.0 -> 2.28.0; STATUS_STATS gains literal_bytes and
the created reg/dir/link/special counters (golden wire updated)
- receiver reports which destination entries it newly created, including
implicitly-created parent directories below the logical transfer root, so
Number of created files carries rsync's per-type breakdown
- Literal data is now exact for a delta transfer (receiver counts the literal
fragments it stored)
- differential-tested vs rsync 3.4.1 for fresh-create, update and delta
- -n/--delete sends the same protected/size-skipped/scope as a real run, so
the read-only would-delete walk no longer over-reports (row -> caveat)
- --delete-delay charges --max-delete on actual removals and recursively
re-scans a refilled deferred directory at commit; independent deferred cap
- --info=name emits the leading ./ root line and name2 'is uptodate' lines
- differential tests promoted from residual pins to rsync parity assertions
The max_delete case allowlisted the tree aspect because the surviving extras after a partial --max-delete abort are deletion-order dependent. Under FASTSYNC_PARITY_STRICT a run where the orders coincide was reported as a stale entry (CI failure), while removing the entry made the order-dependent tree mismatch fail. Add a per-case 'compare_tree' flag: max_delete now asserts rc=25 plus the survivor count via extra_check instead of exact tree identity, so the allowlist can be empty. Burn-down reached zero.
- --delete-delay: state that the reported count advances on actual removal
while --max-delete is charged at plan/snapshot time (defer_add/planned).
A new differential shows rsync instead charges on actual removals and
recursively removes a queued directory, so the row moves to Caveat
(matrix 111/13/33) and the residual is pinned by tests.
- Add a deterministic unit test (plan-time budget charge), a FastSync
integration test (byte-barrier refill + --max-delete), and an rsync
differential for a refilled deferred directory.
- Soften the --fuzzy summary: the tree is byte-exact by design, so it is
pinned by the threshold suite, not a byte-level differential.
- README: describe what -m parity actually selects; fix the allowlist
example to the real max_delete entry.
- xdist-safe delete-timing fixture names (timing and --threads mode).
- Renumber the duplicate HANDOFF item 9 to 10; add the missing final
newline to test_checksum.c.
- Expose ignore_errors_allows_delete and unit-test the deletion gate
without a privileged source directory; update the stale deleted-count
doc comment.
No production behavior changes.
- cli: remove UB in --bwlimit scaling (range-check the double product before
casting, drop atoi for the +/-1 form) and add huge/boundary unit tests
- test: widen the CI throttle wall-clock band to [1.5, 4.5]s with a 2s
cross-tolerance so a loaded runner cannot flake it
- log: drop the unused LOG_INFO_BACKUP bit; --info=backup is accepted-but-
silent like the other rsync-only categories
- client_send: remove the duplicate delete_display_path forward declaration
- utils: add non-allocating utils_strip_transfer_root and use it from
scanner_note_nonreg and delete_display_path (was duplicated logic)
- scanner: lstat() instead of stat() when re-reading an empty dir's metadata
- file: drop the no-op else-if and the redundant ELOOP arm in
file_ensure_directory_secure (symlinks are refused anyway)
- format/stats: document literal_data as whole-file accurate (delta upper
bound) instead of claiming literal bytes sent
- docs: refresh stale protocol 2.26.0 labels to 2.27.0
- Initialize PipelineContextSender.delete_suppressed=false: an uninitialized
true silently skipped the --delete keep-set manifest under -m/--threads,
so destination extras were never removed.
- Allocate/install the receiver deleted-path observer only when
report_deletes is set (--info=del / -i / --out-format under --delete), cap
the retained list at MAX_MANIFEST_ENTRIES, and free already-created lists
on the receiver-pipeline create failure path.
- Validate report_deletes/report_stats/report_dest_info on receive.
- Correct stale comments (config.h report_deletes, delete_plan.h deleted
count, multiprocessing.h stats locking, utils.h observer placement).
- Tests: sender delete_suppressed init, report_deletes gating (unit), and
-m/--delete default delete-after keep-set removal (integration).
- Wire: config frame gains report_deletes (protocol 2.26.0 -> 2.27.0); the
receiver lists actually-removed paths in the STATUS_STATS path list, so the
sender prints rsync's `deleting PATH` / `*deleting PATH` lines for a real
--delete run (and -i/out-format). Observers threaded through the manifest,
missing-args and per-directory delete engines; golden wire len/hash updated.
- bwlimit: throttle now paces like rsync 3.4.1 -- ~100ms burst capacity and the
sleep is no longer credited as refill, so 4 MiB at 1024/2048 KiB/s matches
rsync within ~4% (was ~2x too fast).
Each row's exact residual reproduced against rsync 3.4.1:
- basis dirs (--compare/copy/link-dest): FastSync xxHash-verifies a basis hit
while rsync --size-only installs the wrong same-size basis content.
- --delay-updates: the fixed .fastsync-stage name wipes an unrelated
destination entry of that name even without --delete; rsync leaves it.
- --fuzzy: deterministic name/size heuristic (10x window), not rsync's matcher.
- --dry-run: would-delete report over-reports the updated file and an
excluded-but-protected extra, and ordering differs.
Docs: RSYNC_COMPAT tally 109/16/31; HANDOFF item 9. clang-format + cppcheck +
ASan + full suite clean.
- --bwlimit: faithful port of rsync 3.4.1 parse_size_arg (default KiB/s,
binary K/M/G/T/P, decimal KB/MB, KiB/MiB, decimals, 0 = unlimited, 512-byte
floor, (size+512)/1024 quantization). Unit tests + docs.
- --info: wire del/remove/name/flist/nonreg/progress to real FastSync events in
rsync's line format (deleting PATH, sender removed NAME, name lines,
'sending incremental file list', skipping non-regular file "NAME"); name no
longer aliases copy; --info=progress drives the progress path and report_stats.
- --ignore-errors: match rsync's default -- a source I/O error skips deletion
unless --ignore-errors, while the readable tree still transfers and the run
exits 23. Covers all delete timings and both send paths.
- --iconv now matches rsync's push direction: the destination charset is the
client spec's REMOTE half, so a default receiver writes wire names verbatim;
a server's own --iconv LOCAL overrides it (daemon charset analog). Updated
unit + integration tests and added a default-server differential gate case.
- Empty-directory emission is gated behind a new ScannerOptions.emit_empty_dirs
set only by the real sender, so low-level scanner helpers keep the historical
file-only list.
- --temp-dir reclassified to Divergent: relative dirs match rsync exactly
(resolved under the destination), but an absolute path is deliberately
rejected by the confined receiver; differential test added.
- Docs/tally: 109 Parity / 22 Caveat / 25 Divergent.
- Recursive scans emit a directory entry for every traversed directory that
produced no transferred child, so empty source dirs (and dirs emptied by
filtering) are recreated like rsync; -m prunes them, --files-from/--list-only
never emit implicit dirs.
- A directory entry now replaces a destination regular file (rsync removes the
non-directory) instead of aborting; confined to the secure parent fd.
- -R --no-implied-dirs --files-from: stop refusing a listed file whose parent
is not listed; create the implied parent with default attributes (no source
metadata is captured for it), matching rsync 3.4.1.
- Differential gate: drop min_size/empty_dirs_recursive/dirs_plain allowlist
entries (dirs_plain now hands FastSync the same trailing-slash source as
rsync); add differential test for the files-from implied parent.
- Docs: -d row -> Parity, tally 108/24/24.
No wire change (PROTOCOL_VERSION stays 2.26.0).
- --delete-delay: count/track only entries actually removed; a directory
refilled before commit (ENOTEMPTY) no longer inflates Number of deleted
files or the --max-delete budget (unit + integration + rsync differential).
- --stats: per-type Number of files breakdown; only stored regular files
count as transferred; transferred/literal byte totals and Total file size
(symlink target lengths) now match rsync for whole-file transfers.
- --progress: print the leading ./ root line and include the root entry in
the to-chk denominator (single-file output byte-identical to rsync).
- --out-format %C: use the selected transfer checksum and render every
algorithm exactly like rsync; checksum_digest_file gains md4/sha1/none.
Reclassify --out-format to Divergent (protocol-specific %b/delta-%c).
- Docs: RSYNC_COMPAT tally 107/25/24, HANDOFF update. No wire change.
Add tests/integration/test_differential_parity.py plus a shared
parity_harness.py and a data-driven parity_caveats.py allowlist. The gate
runs real rsync 3.4.1 and FastSync over the same corpora, compares the
destination trees (paths, hashes, symlink targets, modes, hard-link
grouping) and the normalized -i/--stats/--out-format output, and fails on
any difference not listed in the allowlist. Stale allowlist entries warn
(or fail under FASTSYNC_PARITY_STRICT=1) so the residual list shrinks.
Register parity/parity_ci markers and wire a fast PR job (parity_ci) plus a
push-only full job (parity, strict) into .gitea/workflows/ci.yaml.
Document the gate and the allowlist workflow in tests/integration/README.md.
Add test_out_format_c_delta_mode_divergence documenting that FastSync's
delta %c (its own handshake bytes) cannot match rsync's (16-byte sum header
plus per-block checksums); only the whole-file case is aligned. The test
pins both values so a future parity improvement is noticed.
- Output parity: add `File list size` to the strict --stats differential and
document the row-#3 residual with test_stats_file_count_breakdown_residual
(rsync's `Number of files`/`Number of created files` type breakdown is not
reproducible from what the sender knows: no directory accounting and no
per-entry destination-created state). The row stays a caveat.
- Add --threads variants for the --stats and --progress/-P differentials.
The `-n --delete --threads` variant is a documented xfail: the threaded
dry-run path does not consume the receiver's STATUS_STATS delete list yet.
- Replace the 0.2s sleep flake in the delete-timing proxy with a socket
barrier: the hook now fires only after the server sends a reply (proving it
processed the preceding per-directory delete plan), using --incremental +
--ignore-times to guarantee a mid-transfer handshake reply.
Blockers addressed together (shared scanner/delete-plan plumbing):
* #10: an empty in-scope source directory produced no plan keep entry, so the
receiver deleted the destination directory itself. The scanner now records
every traversed directory into a delete-plan sink, the plan sender keeps them,
and any directory whose plan the data stream never triggered is emitted after
the data so its extras are still removed. Differential tests cover
--delete-during and --delete-delay.
* #8: an invalid per-directory filter file was silently ignored when an earlier
merge file in the same directory existed; key the failure off the error text
(both sequential and parallel scanners) and fail the scan.
* #9: -R + --files-from receiver-protect rules recorded the source-relative
path; record the bare relative wire path in both scanners so the protected
destination mirror survives --delete.
* #5: the STATUS_STATS would-delete parser now validates each retained path and
enforces the shared MAX_MANIFEST_BYTES budget, and the --out-format dry-run
delete line is escaped like the itemize line.
* #11: drop the unused DELETE_PLAN_MAX_NAMES macro, log the delete-limit
warning once per session, roll back dir-merge names from a per-directory file
that fails to parse, and guard every filter error snprintf against err==NULL.
#10 leaves the empty directory itself kept and its extras removed, matching
rsync's final state on both per-directory timings.
rsync's %c counts the block-checksum bytes received: even a whole-file
transfer with no basis receives rsync's 16-byte sum header (append and
inplace included), while a dry run receives nothing. FastSync's
whole-file path has no equivalent header, so report 16 for parity when
delta is inactive, keep 0 for dry runs, and keep the real received bytes
when delta is active (FastSync's signature framing differs, so delta %c
stays divergent). Add a strict %c/%l/%n differential against rsync and
turn the %b check into a real rsync differential (semantics: both count
wire bytes and exceed %l; the exact values are protocol-specific).
Accept the full rsync 3.4.1 --info (backup, del, flist, mount, nonreg,
progress, remove, symsafe) and --debug (acl, backup, bind, chdir, connect,
cmd, del, deltasum, dup, exit, filter, flist, fuzzy, genr, hash, hlink,
iconv, nstr, own, recv, send, time) vocabularies, plus the historical
syms/hl/owner aliases, with level suffixes. Categories FastSync already
emits (copy/name/misc/skip/stats and io/proto/pack/util) still set their
log flags; the rest are accepted but silent. Unknown names remain
rejected by name, matching rsync. Update the CLI unit tests and the
rsync-parity integration tests (previously they required del/filter to be
rejected).
- Assert FastSync's sparse/preallocate block accounting matches rsync 3.4.1
for a hole file (preallocate wins over sparse, exactly like rsync).
- Assert --ignore-existing is answered by the receiver before the sender
transmits the payload (CountingProxy wire volume near-zero).
- CountingProxy.run gains a bounded join_timeout so tests that only need the
client->server count do not wait for the server's idle socket.
- Fix the stale protocol 2.24.0 comment in test_config.c (golden is 2.26.0).
The single-threaded and -m receivers never populated ReceiverStats.matched_data
or .deleted_files, so --stats always printed 0 for both even when rsync
reported nonzero. Track the bytes reconstructed from the basis file while
applying a delta, and tally the delete-commit counts (manifest and
per-directory sessions) into the receiver stats. The -m pipeline now carries
its own stats/would-delete fields and emits the STATUS_STATS frame before the
terminal success, so --threads finally reports the counters and renders
-n --delete lines.
Also normalize the -n --delete would-delete enumeration's absolute basis
prefixes exactly like the real commit path (fixing an over-report) and fix the
basis_delete_relative off-by-one when the receive root is '/'. Unit tests
cover the root mapping and the basis protection; integration tests cover
matched/deleted stats for both receivers and the --threads dry-run delete
lines.
The -R prefix marker installed in synced_dirs was discarded when finalizing
the per-directory delete sender (--delete-during/--delete-delay), so the
up-front root plan was the receive root '.', whose keep list only held the
first prefix component. The receiver then deleted destination content
outside the transferred prefix (e.g. unrelated/keep.txt), a data-loss bug;
rsync keeps it.
Confine the walk to the -R prefix: send that prefix's plan as the root plan,
only transmit plans at or below it, and never emit the receive root plan for
a scoped run. Add a differential test covering both --delete-during and
--delete-delay.
Add tests/integration/test_codecs.py (accept/reject matrix and byte
differential against rsync 3.4.1 for every algorithm), extend the
checksum/compression unit tests with MD4/SHA1/none vectors and per-codec
round-trips, pin the new config golden (2.26.0), and update the version
strings and codec acceptance expectations.
A -R source prune (--exclude/--max-size) must record the destination wire
path below the reconstructed prefix so --delete protects it; the parallel
root scan and the sequential skip path used the source path instead.
A general -R transfer places its files below the reconstructed prefix, so
marking the whole receive root as the delete scope deleted unrelated
sibling directories (data loss; rsync keeps them). Use the prefix itself
as the root marker when it is non-empty, in both the single-threaded and
multithreaded pipelines.
Replace the aggregate stderr progress with rsync 3.4.1's per-file progress
block (name, 32 KiB first frame, final frame with (xfr#N, to-chk=X/Y)).
Add differential tests against real rsync for --out-format %C/%b, the
--progress frames, selected --stats lines and -n --delete lines.
- Compare --delete-during/--delete-delay final state against rsync 3.4.1.
- Force a mid-transfer failure through a byte-slicing proxy: --delete-during has
removed the processed directory's extra, --delete-delay has not.
- Create a destination entry while the transfer is in flight: it survives
--delete-delay's snapshot but is removed by --delete-after's fresh end scan.
- Cover the --delete-delay type-conflict case now matching rsync.
TapTap