feat: --checksum-choice/--cc and --checksum-seed #273

Closed
TapTap wants to merge 0 commits from feat/p3-checksum-choice into dev
Owner

Implements Phase-3 --checksum-choice/-cc and --checksum-seed as a real feature on the whole-file digest path.

What is supported (genuinely, no silent no-ops)

  • --checksum-choice/--cc (space and =NAME forms) select the per-file whole-file digest used by the --incremental/--checksum handshake and basis-dir content verification:
    • xxh64/default (exact xxHash64, seeded by --checksum-seed; xxhash accepted as rsync's spelling),
    • md5 via OpenSSL EVP (16-byte digest).
    • Any other name (md4/sha1/sha256/crc32/none/...) is rejected with a clear parse error - never ignored.
  • --checksum-seed=NUM: strict decimal 0..2^64-1; feeds the whole-file xxh64 (full 64-bit) and the delta path's per-block xxHash32 strong checksum (low 32 bits) so an explicit seed deterministically changes those digests on both endpoints. md5 ignores the seed (documented).

How it is wired

  • New src/shared/checksum.[ch]: digest abstraction (algo id + seed -> variable-length, bounded digest).
  • Config frame carries the algorithm id + seed (sender and receiver hash the old file identically).
  • The STATUS_CHECK handshake now sends a length-prefixed, bounded (1..16 byte) digest instead of a fixed 64-bit value; length validated on receive.
  • --checksum-seed also threads into delta_signature_create_seeded/delta_compute_seeded (same seed both ends), so a seeded delta stays byte-exact.
  • PROTOCOL_VERSION bumped 2.9.0 -> 2.10.0. Defaults (xxh64, seed 0) reproduce prior byte-for-byte behavior.

Divergences (documented in RSYNC_COMPAT.md)

  • FastSync defaults to seed 0 and never randomizes (rsync randomizes the seed when --checksum-seed is unset).
  • --checksum-choice selects only the WHOLE-FILE digest; the delta block strong checksum remains xxHash32 (independent per-block checksum, matching rsync).

Verification

  • Unit (28 suites) all pass; ASan build clean; STRICT_WARNINGS build clean; clang-format + cppcheck clean via the CI docker image.
  • Integration: 258 passed, 10 skipped, 1 xpassed (includes new unchanged-skip / same-size-mtime redetect for xxh64+md5, single-thread and -m, near-zero-data skip, deterministic seed, and a byte-exact seeded delta run).
Implements Phase-3 `--checksum-choice`/`-cc` and `--checksum-seed` as a real feature on the whole-file digest path. ## What is supported (genuinely, no silent no-ops) - `--checksum-choice`/`--cc` (space and `=NAME` forms) select the per-file whole-file digest used by the `--incremental`/`--checksum` handshake and basis-dir content verification: - `xxh64`/default (exact xxHash64, seeded by `--checksum-seed`; `xxhash` accepted as rsync's spelling), - `md5` via OpenSSL EVP (16-byte digest). - Any other name (md4/sha1/sha256/crc32/none/...) is rejected with a clear parse error - never ignored. - `--checksum-seed=NUM`: strict decimal 0..2^64-1; feeds the whole-file xxh64 (full 64-bit) and the delta path's per-block xxHash32 strong checksum (low 32 bits) so an explicit seed deterministically changes those digests on both endpoints. md5 ignores the seed (documented). ## How it is wired - New `src/shared/checksum.[ch]`: digest abstraction (algo id + seed -> variable-length, bounded digest). - Config frame carries the algorithm id + seed (sender and receiver hash the old file identically). - The `STATUS_CHECK` handshake now sends a length-prefixed, bounded (1..16 byte) digest instead of a fixed 64-bit value; length validated on receive. - `--checksum-seed` also threads into `delta_signature_create_seeded`/`delta_compute_seeded` (same seed both ends), so a seeded delta stays byte-exact. - `PROTOCOL_VERSION` bumped 2.9.0 -> 2.10.0. Defaults (xxh64, seed 0) reproduce prior byte-for-byte behavior. ## Divergences (documented in RSYNC_COMPAT.md) - FastSync defaults to seed 0 and never randomizes (rsync randomizes the seed when `--checksum-seed` is unset). - `--checksum-choice` selects only the WHOLE-FILE digest; the delta block strong checksum remains xxHash32 (independent per-block checksum, matching rsync). ## Verification - Unit (28 suites) all pass; ASan build clean; STRICT_WARNINGS build clean; clang-format + cppcheck clean via the CI docker image. - Integration: 258 passed, 10 skipped, 1 xpassed (includes new unchanged-skip / same-size-mtime redetect for xxh64+md5, single-thread and -m, near-zero-data skip, deterministic seed, and a byte-exact seeded delta run).
TapTap added 3 commits 2026-09-07 17:43:19 +02:00
Adds real algorithm selection (xxh64 default, plus md5 via OpenSSL EVP) and a
64-bit seed for the per-file whole-file digest used by the --incremental/
--checksum handshake and basis-dir content verification. The seed also feeds
the delta path's per-block xxHash32 strong checksum (low 32 bits) so an
explicit seed deterministically changes those digests too. Sender and receiver
hash identically: the algorithm id and seed cross the config wire frame and the
STATUS_CHECK handshake now carries a length-prefixed, bounded digest instead of
a fixed 64-bit value. Unsupported algorithm names are rejected at parse time
(never a silent no-op). PROTOCOL_VERSION bumped 2.9.0 -> 2.10.0; defaults
(xxh64, seed 0) preserve prior byte-for-byte behavior.
Unit: digest selection with known vectors (xxh64 seed 0/empty, md5 RFC vectors,
seed changes xxh64 but not md5, name mapping, bounded buffer rejection), CLI
parse (all spellings, = forms, --cc alias, unsupported rejected, seed strict
decimal), config wire round-trip and out-of-range algo rejection, and seeded
delta signature/compute symmetry (matching seed rebuilds, mismatched seed yields
no block matches). Integration: unchanged skip + same-size/mtime redetect for
xxh64 and md5 with and without -m, near-zero-data skip run, deterministic seed,
and a byte-exact seeded delta run.
docs: mark --checksum-choice and --checksum-seed implemented
CI / lint (pull_request) Failing after 35s
CI / build-and-test (pull_request) Skipped
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
a2ce0a8e41
Precise notes for both rows: supported algorithms (xxh64/xxhash, md5 via EVP),
rejection of unsupported names, seed semantics (full 64-bit for whole-file,
low 32 bits for the delta block hash, ignored by md5), the --cc alias, the
length-prefixed bounded handshake digest, the 2.9.0 -> 2.10.0 protocol bump,
default byte-for-byte preservation, and the divergence from rsync's randomized
seed (FastSync defaults to seed 0). Summary count line is intentionally untouched.
TapTap added 1 commit 2026-09-07 17:46:56 +02:00
fix: make digest-vector arrays const to satisfy cppcheck constVariable
CI / lint (pull_request) Successful in 35s
CI / sanitizers (undefined) (pull_request) Successful in 50s
CI / sanitizers (address) (pull_request) Successful in 52s
CI / fuzz-build (pull_request) Successful in 19s
CI / coverage (pull_request) Successful in 42s
CI / valgrind (pull_request) Successful in 36s
CI / build-and-test (pull_request) Successful in 12m58s
56981a0d9d
TapTap added 1 commit 2026-09-07 19:50:53 +02:00
fix: reject xxh3 alias, pin handshake digest length to algorithm, simplify xxh64 copy, note FIPS md5
CI / lint (pull_request) Successful in 35s
CI / sanitizers (undefined) (pull_request) Successful in 52s
CI / sanitizers (address) (pull_request) Successful in 52s
CI / fuzz-build (pull_request) Successful in 18s
CI / coverage (pull_request) Successful in 42s
CI / valgrind (pull_request) Successful in 37s
CI / build-and-test (pull_request) Successful in 12m58s
c6758531f6
Author
Owner

Merged into dev via local merge (2FA blocks server-side merge). missing-args 681d7a8, append f99e6e1, checksum-choice 4c98744, recount 829e760. Independent c-reviews: missing-args REQUEST CHANGES (absent-parent no-op blocker fixed); append REQUEST CHANGES (blocker verified already-handled + tested, sentinel guard for streamed/-f sources); checksum-choice APPROVE WITH NITS (xxh3 alias removed, digest length pinned). dev CI run #494: all jobs success. Closing without server merge.

Merged into dev via local merge (2FA blocks server-side merge). missing-args 681d7a8, append f99e6e1, checksum-choice 4c98744, recount 829e760. Independent c-reviews: missing-args REQUEST CHANGES (absent-parent no-op blocker fixed); append REQUEST CHANGES (blocker verified already-handled + tested, sentinel guard for streamed/-f sources); checksum-choice APPROVE WITH NITS (xxh3 alias removed, digest length pinned). dev CI run #494: all jobs success. Closing without server merge.
TapTap closed this pull request 2026-09-07 20:25:59 +02:00

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: TapTap/FastSync#273