feat: no-wire parity — delay-updates (#317), >256 MiB streaming (#318) #328

Merged
TapTap merged 5 commits from parity/no-wire into dev 2026-09-24 01:44:23 +02:00
Owner

No-wire parity cycle — closes #317, #318

No wire change (PROTOCOL_VERSION stays 2.30.0). Matrix: 119 ✅ / 16 ⚠️ / 22 ❌ = 157.

#317 — --delay-updates

  • Per-run unique staging directory (.fastsync-stage.<pid>.<entropy>) created with O_EXCL; a genuine destination entry with that name is never wiped (rsync-verified).
  • --delay-updates now implies --delete-after (normalized onto the existing wire bool), and both receivers publish all staged files before the deferred delete commit. --backup files are shielded from the delete-after pass; a blocking destination dir is cleared at publication under --delete/--force (rsync make-way).
  • New delay_updates/delay_updates_delete differential cases; row ❌ → ⚠️.

#318 — stream single files/bases above 256 MiB

  • The receiver no longer buffers a whole payload: the length-prefixed frame is read in bounded 64 KiB chunks and, above the streaming bound, written to a spool temp file (incrementally decompressed for zstd/zlib) and installed via the existing bounded basis-copy store path. Sender-side streaming compression added.
  • --compare-dest/--copy-dest/--link-dest no longer refuse an over-limit basis MISS (falls through to streaming); -y/--fuzzy streams the sibling signature/reconstruction instead of buffering.
  • Verified with a real 300 MiB transfer over loopback (sha256 match; peak RSS client 7 MB / receiver 3 MB with -z). New test_large_stream.py. Four ⚠️ rows (three basis dirs + fuzzy) had their over-256 MiB residuals dropped; they stay ⚠️ for their remaining caveats.

Verification

Strict -Werror, clang-format 18, cppcheck clean; unit 45/45 (ASan with leak detection, UBSan, valgrind); integration 944 passed; differential parity 84 passed.

## No-wire parity cycle — closes #317, #318 No wire change (`PROTOCOL_VERSION` stays 2.30.0). Matrix: **119 ✅ / 16 ⚠️ / 22 ❌ = 157**. ### #317 — `--delay-updates` - Per-run unique staging directory (`.fastsync-stage.<pid>.<entropy>`) created with `O_EXCL`; a genuine destination entry with that name is never wiped (rsync-verified). - `--delay-updates` now implies `--delete-after` (normalized onto the existing wire bool), and both receivers publish all staged files **before** the deferred delete commit. `--backup` files are shielded from the delete-after pass; a blocking destination dir is cleared at publication under `--delete`/`--force` (rsync make-way). - New `delay_updates`/`delay_updates_delete` differential cases; row ❌ → ⚠️. ### #318 — stream single files/bases above 256 MiB - The receiver no longer buffers a whole payload: the length-prefixed frame is read in bounded 64 KiB chunks and, above the streaming bound, written to a spool temp file (incrementally decompressed for zstd/zlib) and installed via the existing bounded basis-copy store path. Sender-side streaming compression added. - `--compare-dest`/`--copy-dest`/`--link-dest` no longer refuse an over-limit basis MISS (falls through to streaming); `-y/--fuzzy` streams the sibling signature/reconstruction instead of buffering. - Verified with a real 300 MiB transfer over loopback (sha256 match; peak RSS client 7 MB / receiver 3 MB with `-z`). New `test_large_stream.py`. Four ⚠️ rows (three basis dirs + fuzzy) had their over-256 MiB residuals dropped; they stay ⚠️ for their remaining caveats. ### Verification Strict `-Werror`, clang-format 18, cppcheck clean; unit 45/45 (ASan with leak detection, UBSan, valgrind); integration **944 passed**; differential parity **84 passed**.
TapTap added 5 commits 2026-09-24 01:39:07 +02:00
test: fix cppcheck style findings in cycle-F tests
CI / lint (pull_request) Successful in 1m53s
CI / parity-full (pull_request) Skipped
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
CI / parity-fast (pull_request) Successful in 17s
CI / build-and-test (pull_request) Successful in 54s
7408686370
TapTap merged commit a14fbb2c10 into dev 2026-09-24 01:44:23 +02:00
TapTap deleted branch parity/no-wire 2026-09-24 01:44:23 +02:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: TapTap/FastSync#328