FastSync/CHANGELOG.md

Changelog

All notable changes to FastSync are documented here. Versions match PROTOCOL_VERSION (printed by fastsync --version); the client and server must run the same version because the handshake is strict.

[2.23.0] - 2026-09-16

Added

• Rsync-parity wave. Closed the remaining CLI, filesystem, ownership, deletion, and output gaps against rsync 3.4.1.
  • Short options -r (--recursive), -b (--backup), -L (--copy-links), and -B (--block-size/--delta-block); rsync short-option clustering (-av, -aAX, -rlpt) and attached/inline values (--opt=value, -B1000, -essh, -MOPT). A value that starts with - is not mistaken for a cluster.
  • -c/--checksum now implies the incremental checksum quick-check (and, like rsync, does not imply -t).
  • --checksum-choice/--cc accepts xxh64/xxhash/xxh3/xxh128/md5/ auto and rejects md4/sha1/none and the two-name form by name; --checksum-seed=0 (the default) is randomized per transfer and the chosen seed is sent to the receiver.
  • --compress-choice/--zc accepts zstd/none/auto and rejects lz4/zlib/zlibx by name; --skip-compress defaults to rsync 3.4.1's built-in suffix list; --no-whole-file is accepted.
  • --timeout defaults to 0 (disabled) and --contimeout to 60 s (both 0 disables), matching rsync; --max-alloc=0 means no local limit.
  • --temp-dir is confined to the receive root (absolute/.. rejected by the receiver) and an EXDEV install falls back to a non-atomic copy.
  • --numeric-ids is documented as a mapping modifier only; --usermap/--groupmap support inclusive LOW-HIGH ranges, *, empty-FROM (unnamed ids), and receiver-resolved TO names; --chown conflicts with a map on the same side are rejected.
  • --fake-super records the resolved owner (never a real chown) and replays mode/time; directory ownership and directory xattrs/ACLs are preserved.
  • -l/--links stores symlink targets verbatim (absolute and ..-bearing included), matching rsync; --safe-links/--copy-unsafe-links are applied sender-side and --munge-links uses rsync's /rsyncd-munged/ marker; --trust-sender no longer affects symlink targets.
  • --specials recreates unix sockets with mknod(S_IFSOCK) (so -D covers the full rsync node set).
  • Deletion: the manifest carries a synchronized-directory section so --files-from subsets no longer delete untransmitted paths; --delete-excluded leaves size-pruned mirrors protected; extraneous destination symlinks are unlinked (never followed); --max-delete=N is partial (delete up to N, skip the rest, exit 25) and --delete-missing-args removals draw from the same budget; --force is honored during --delay-updates publication.
  • -x/--one-file-system emits the mount-point directory entry; the --include/--exclude layers are an ordered first-match rule list.
  • --chmod is a faithful port of rsync 3.4.1 (numeric/symbolic, D/F/X, s/t, append semantics, no -p implication, no sanitization).

Changed

• PROTOCOL_VERSION bumped 2.22.0 → 2.23.0: the delete manifest gains a synchronized-directory section and the terminal status gains STATUS_DELETE_LIMIT (client exit 25 on a --max-delete-capped commit).
• The 2.22.0 mode-masking divergence is removed. Under -p the source mode is copied exactly, including S_IWGRP/S_IWOTH and setuid/setgid/sticky; --chmod no longer implies -p. New files without -p still use source_mode & ~umask when metadata is present (else 0644), and new directories without -p still use the 0755 creation default.
• --protocol=NUM accepts only the current 2.23.0 version string.

Notes

• The rsync-compatibility matrix (RSYNC_COMPAT.md) now classifies every row as parity, caveat (works with a documented divergence), or divergent (not supported/no-op/impossible), replacing the previous misleading "N implemented / 0 divergence" summary. Durable documented divergences remain: receiver-side symlink target containment is not enforced by default (verbatim storage is rsync parity; use --safe-links), --temp-dir rejects absolute/foreign-filesystem paths, --copy-devices reads a bounded st_size, a broken referent under --copy-links exits 0, new directories without -p use 0755, --stats receiver-only counters are 0, and --password-file/--early-input/--hash-credentials/--iterations and the batch format are FastSync-native.

[2.22.0] - 2026-09-15

Added

• Per-attribute metadata preservation (protocol 2.22.0). The former single metadata bundle is split into four independent, rsync-compatible flags: -p/--perms, -t/--times, -o/--owner, and -g/--group, each applied independently on the receiver, with negations --no-perms/--no-times/ --no-owner/--no-group (short --no-p/--no-t/--no-o/--no-g) and --no-preserve clearing all four. -a/--archive is now full rsync -rlptgoD (owner and group included; their application stays privilege-gated). -A/--acls and --chmod imply -p, -X/--xattrs does not, -E/--executability sets only executability, and -U/-N do not imply -t. --incremental/--delta still auto-preserve perms+times unless the user explicitly negated them.
• Receiver applies directory modes under -p (at the end of the transfer, alongside the deferred directory times) and symlink mode under -p; -O suppresses directory times only.

Changed

• PROTOCOL_VERSION bumped 2.21.0 → 2.22.0: the binary config frame gains four appended booleans (preserve_perms/preserve_times/preserve_owner/ preserve_group) after omit_link_times. The fixed-width FileMetadata layout is unchanged; the receiver derives the metadata-frame gate (use_metadata) from the four attributes.

Notes

• Documented divergences from rsync: a client-supplied mode never grants group/other write (S_IWGRP|S_IWOTH are stripped for files, directories, symlinks, and specials; rsync's -p preserves them exactly); a brand-new file without -p gets source_mode & ~umask (sanitized) when metadata is present, else the historical fixed 0644; --chmod implies -p (rsync does not); -o/-g map by name on the receiver with a raw-numeric fallback (only numeric ids cross the wire); and a daemon module without client owner = yes does not refuse a plain -a/-o/-g but forces super-user activities off, applies no ownership, and logs a warning (explicit --chown/--usermap/ --groupmap/--numeric-ids/--copy-as/--super are still refused).

[2.21.0] - 2026-09-14

Added

• Optional server→client rejection detail (protocol 2.21.0). A rejected operation may now carry a bounded human-readable reason via STATUS_ERROR_DETAIL instead of a bare STATUS_ERROR, so the client can report why the server refused (daemon module gate, config validation, receiver-side path/node validation). receive_status() transparently maps the new status back to STATUS_ERROR for every existing call site and captures the reason into a thread-local buffer exposed by protocol_last_error(). The detail body is always consumed, so the stream cannot desynchronize, and messages are sliced to MAX_ERROR_DETAIL_BYTES (4096) on send.
• Server-contacting --dry-run (protocol 2.21.0). --dry-run now performs a real handshake with a remote/daemon receiver and reports exactly what WOULD change based on receiver state (existing destination files, mtimes, checksums, basis dirs). The wire config carries the dry-run intent (Config.dry_run) and the receiver answers each per-file check with STATUS_DRY_RUN_TRANSFER (would transfer) or STATUS_OK (already up to date); the sender prints the would-transfer set and its trailer without sending any file data. The receiver performs the normal read-only incremental decision but mutates nothing: no temp files, writes, renames, deletes, metadata/xattr/chown, or directory creation. A plain local destination (no explicit --server-port/remote) keeps the original client-side dry-run. Would-delete reporting for --delete* is deferred to a follow-up; dry-run never deletes.
• Daemon max connections per host (per-source-IP concurrent cap, default 0 = unlimited), auth lockout threshold (default 10; 0 disables) and auth lockout duration (default 300 s) config keys.
• fastsync-server --allow-super opt-in for a privileged standalone TCP server; without it a root standalone receiver forces super-user activities off (device nodes, --write-devices, ownership). The --stdio SSH argv is client-composed, so super activities always stay off there.

Changed

• Config wire fields are now declared once in an X-macro table (CONFIG_WIRE_FIELDS in src/shared/config.h) that generates the struct members, defaults, and the send/receive sequence, removing the manual six-site field sync. Wire bytes and PROTOCOL_VERSION are unchanged.
• receive_incremental_check() (the per-file STATUS_CHECK fast path) is split into small static helpers with a short linear orchestrator. Pure refactor: the wire byte stream and all cleanup are unchanged.
• authorized_root state has a single owner (utils.c) with read accessors; the duplicated statics in file.c and the server were removed.
• Data records its owning ProtocolSession so its memory charge is returned to the session that reserved it, regardless of the destroying thread.
• The receiver pipeline moved out of shared into server/receiver_pipeline.[ch]; the build now uses explicit fastsync_shared / fastsync_client_core / fastsync_server_core targets instead of a GLOB, and the client no longer links server code.
• The benchmark tool generates the requested random/compressible data mix accurately, verifies each transfer before recording it, computes correct percentiles, adds a MB/s column, handles tc/netem without requiring sudo when already root, builds into a dedicated build-bench/ directory, and adds a --warm incremental-transfer mode.
• The nix-shell dev environment provides the full toolchain (clang-format, cppcheck, pytest-xdist, OpenSSH, rsync, iproute2, valgrind, lcov) and no longer builds on entry.

Security

• Enforce the daemon's per-module max connections cap (0 = unlimited) and add the shared per-source max connections per host cap plus a cross-process auth lockout. Because the listener forks one child per connection, the counters live in an anonymous shared mapping created before the accept loop and reclaimed by the parent's SIGCHLD handler, so the per-module, per-source and auth-failure state is shared across every child (including after SIGKILL). The per-source table has a bounded lifetime (expired/idle entries are reclaimed, with a rate-limited warning when genuinely full), and the occupancy counters are re-derived from the shared slot table on every child exit. Trusted loopback peers are exempt (they share one address); clients behind a shared NAT/proxy share a single per-host budget and lockout, which is documented.
• Hardening from a full security audit:
  • Fail a truncated zstd frame instead of spinning forever (remote DoS).
  • Open receiver destination/basis/hard-link entries O_NONBLOCK so a client-planted FIFO cannot block a worker indefinitely.
  • Require a regular file before --inplace writes, closing a FIFO-hang and a raw-device write that bypassed the --write-devices gate.
  • Reject SSH destinations whose user/host begins with - and insert -- before the host token, closing -o ProxyCommand=… argument injection (RCE).
  • Gate client --force recursive removal behind the server --allow-delete policy.
  • Reject empty hosts allow/hosts deny/auth users values instead of silently meaning "unrestricted".
  • Restrict TLS 1.2 to AEAD suites and set server cipher preference; load the private key TOCTOU-safely from an O_NOFOLLOW fd; verify IP literals against IP SANs; guard client-cert CN truncation.
  • Make --dry-run content-blind: it neither reads destination files nor hashes basis files, removing a 1-bit content oracle against read only modules.
  • Bound glob matching (iterative DP, no exponential backtracking) and bound line reads for filter/--files-from/pattern files.
  • Gate system.posix_acl_* xattrs on --acls and charge decompression/chunk allocations against the per-connection memory budget.

Fixed

• Pre-auth NULL dereference in config_delete() when an over-long basis_count (and the analogous count fields) was received and then failed validation; received counts are now validated before being published.
• Leaked inherited Data in the forked compression-truncation unit test (valgrind definite leak).
• receive_status() no longer loses a captured rejection reason when owed keepalives are drained.

[2.20.0] - 2026-09-13

Security

• Cap cumulative DirTimeList growth and bound pre-auth config-string memory (remote memory-exhaustion DoS).
• Daemon host access control (hosts allow/hosts deny, IPv4/IPv6/CIDR), configurable global max connections, connection audit logging, and a bounded auth failure delay throttle. IPv4-mapped peers are normalized and invalid patterns are rejected at parse time (no silent fail-open).
• Honor --timeout for protocol I/O and bound idle/session time to defeat keepalive slowloris; child-safe signal handling in the forked daemon.
• Compiler/linker hardening (_FORTIFY_SOURCE, stack protector, PIE, RELRO) and pinned build dependencies.

Fixed

• Use-after-free in the basis-dir oversize preflight.
• Placeholder Data leaks, missing_args leak, scanner chunk leak.
• Thread-safe logging; single fd owner and cleanup epilogue in the server handler.

Performance

• Metadata now crosses the wire as one packed frame (protocol 2.20.0).
• Delete keep-set and --files-from lookups indexed (O(n*m) → O(n)).
• Reused per-thread zstd contexts; TCP_NODELAY by default.
• Byte-bounded sender queues; removed a redundant scanner stat().

[2.19.0] - 2026-09-12

Security

• Daemon authentication rewritten as SCRAM-SHA-256 challenge/response (STATUS_AUTH_CHALLENGE → STATUS_AUTH_RESPONSE → STATUS_AUTH_OK/STATUS_AUTH_FAILED), replacing the old replayable static SHA-256(password) bearer credential. Each proof is bound to a fresh per-connection server nonce plus a client nonce, so a captured response can never be reused.
• Salted verifier store. --password-file/--early-input now hold user:$fastsync$1$pbkdf2-sha256$<iters>$<salt>$<stored_key>$<server_key> (PBKDF2-HMAC-SHA256, default 600000 iterations, range 100000–10000000). The legacy user:SHA256HEX form is hard-rejected; there is no auto-upgrade. Generate stores offline with fastsync-server --hash-credentials FILE [--iterations N].
• Username-enumeration hardening. Unknown/off-list users are answered with a dummy verifier whose salt is a deterministic per-username value (HMAC-SHA256(dummy_key, username)), using the store-wide uniform iteration count and a constant-time full-length membership scan. The dummy key is persisted in an owner-only <store>.dummykey sidecar (atomic publish, exact mode 0600) so challenges are stable across restarts.
• Verified transport for auth-required modules. A module with auth users accepts credentials only over verified TLS whose client certificate matches --client-cn, or — when --allow-unauthenticated is explicitly set — plaintext from a loopback peer. Remote plaintext is refused before any challenge. Clients must use --tls to send --password-file credentials to a non-loopback daemon; --client-cn is mandatory with --tls.
• Secret hygiene. The plaintext password, derived keys, nonces/proofs and the dummy key are wiped from memory on every path and never logged.
• Carried-over hardening: -K TOCTOU-safe directory walk (openat(O_NOFOLLOW) per component), always shell-quoted SSH remote path, TLS compression/renegotiation disabled, race-free (open-then-fstat) --password-file/--early-input checks, log-injection escaping, and lazy protocol debug escaping.

Added

• fastsync-server --hash-credentials FILE [--iterations N] offline tool.
• <store>.dummykey sidecar (auto-created, owner-only, 0600).
• Integration tests for auth replay rejection, malformed frames, legacy-store refusal, and the loopback/TLS transport policy; fuzz targets for config receive and daemon-auth parsing.

Changed

• Protocol version 2.18.0 → 2.19.0 (breaking). The config-frame auth block is now [present][username] (digest removed) and the auth challenge/response frames are interleaved between the config frame and its STATUS_OK. A 2.19.0 client and a 2.18.0 server (or vice versa) fail cleanly at the handshake.
• Daemon modules declaring auth users require a configured credential store at startup (fail closed); operators regenerate stores from plaintext with --hash-credentials.

Notes

• First tagged release. FastSync implements rsync-compatible file synchronization over TCP and SSH with TLS (OpenSSL), streaming zstd compression, multithreaded transfers, and incremental sync. See RSYNC_COMPAT.md for the flag-parity matrix.