27 Commits
Author SHA1 Message Date
TapTap 1116da9f64 docs: correct rsync-parity claims and stale facts (#297)
CI / lint (pull_request) Successful in 1m47s
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
CI / build-and-test (pull_request) Successful in 40s
Reclassify every rsync-compatibility row as parity / caveat / divergent
(replacing the misleading 143-OK / 0-divergence summary), and document the
protocol 2.23.0 behavior:

- Split the conflated `-M, --preserve` row: `-M` is `--remote-option`,
  `--preserve` is the FastSync `-p`+`-t` alias.
- Fix `MAX_CONNECTION_MEMORY` (256 MiB, not 1 GB), `--rsync-path`
  (client-only, never crosses the wire), and the `-p` mode behavior
  (strict rsync parity; no masking).
- `--specials` now recreates sockets, so `-D` is real parity; fake-super
  records the resolved owner and replays mode/time (never real-chowns).
- Document short options/clustering, checksum/compression choices, seed
  randomization, timeout/max-alloc defaults, temp-dir confinement + EXDEV,
  identity/map parity, verbatim symlinks, delete scoping, `--max-delete`
  partial + exit 25, `--chmod`, output caveats, and server `--port`.
- Bump version refs to 2.23.0 and add the 2.23.0 CHANGELOG entry.

Docs-only; no source changes.
2026-09-16 01:48:13 +02:00
TapTap 684153350a Merge branch 'fix/parity-chmod' into feat/rsync-parity 2026-09-16 01:24:05 +02:00
TapTap ec206b02d0 chmod: match rsync 3.4.1 --chmod and remove mode masking (#293)
- --chmod no longer implies --preserve-perms; repeated --chmod options
  accumulate, and D/F/X selectors plus s/t special bits are supported with
  rsync's exact parse_chmod/tweak_mode semantics.
- Stop masking group/other write and setuid/setgid/sticky: -p copies the
  source mode exactly, no-p new entries use source&~umask, directories keep
  setgid/sticky, and special nodes follow the same rules.
- Apply ownership before mode on the fd path so a chown cannot clear the
  setuid/setgid bits -p just restored (rsync order).
- Update unit and integration tests, including differential checks against
  rsync 3.4.1.
2026-09-16 01:23:45 +02:00
TapTap 88bdfeeb58 fix(parity): receiver temp-dir confinement, server I/O floor, delete budget
Address review findings on feat/rsync-parity:
- confine --temp-dir below the receive root (reject absolute/.. like
  backup-dir/partial-dir); keep EXDEV non-atomic fallback
- floor server session I/O deadlines at SERVER_IO_TIMEOUT_SEC (60s) and
  install it on the socket layer at startup (slow-loris)
- charge each --delete-missing-args directory removal once and clamp the
  extras-walk remaining budget so it can never underflow past --max-delete
- normalize --compress-choice=auto to zstd client-side and accept it on
  receive so auto transfers no longer fail
- map received --max-alloc=0 to MAX_SERVER_ALLOC (receive path only)
- zero File.dest_state; include log-file-format in report_dest_info;
  add STATUS_DELETE_LIMIT name; recognize --skip-compress as a
  separate-value option; OOM-guard send_list_only root entry; drop the
  dead -M= branch; record the bare relative protected prefix for -R
  size-prunes in both scanners; refresh delete-manifest comment
- pin the rsync tarball sha256 and bump integrator image to v11

Tests: temp-dir rejection/relative/cross-device, server timeout floor,
delete-missing dir budget regression, compress-choice=auto e2e,
max-alloc=0 receive mapping, dest_state, report_dest_info modes,
skip-compress dash value, -M short forms, -R root size-prune mirror
protection (rsync 3.4.1 confirmed).
2026-09-16 01:11:59 +02:00
TapTap 3f5b0250f4 fix: ASan out-of-bounds argv in cli test, cppcheck uninit rate buffer 2026-09-15 23:53:49 +02:00
TapTap c41bfb2cdb test: align trust-sender tests with rsync-parity symlink storage 2026-09-15 23:44:14 +02:00
TapTap 1b2632f968 test: fix merged parity branches (4-section manifest fixtures, timeout-teardown) 2026-09-15 23:38:15 +02:00
TapTap 7dbca70a4b Merge branch 'feat/parity-output' into feat/rsync-parity
# Conflicts:
#	src/shared/config.h
#	src/shared/protocol.h
#	tests/test_config.c
2026-09-15 23:25:34 +02:00
TapTap 1a053f06e5 Merge branch 'feat/parity-ownership' into feat/rsync-parity
# Conflicts:
#	src/shared/config.h
#	tests/test_config.c
2026-09-15 23:24:58 +02:00
TapTap 58b3a33e82 Merge branch 'feat/parity-delete' into feat/rsync-parity 2026-09-15 23:24:24 +02:00
TapTap 17b0632098 Merge branch 'feat/parity-network' into feat/rsync-parity 2026-09-15 23:24:21 +02:00
TapTap 376e6500ab fix(delete): count recursive missing-arg removals per entry (#290)
A non-empty --delete-missing-args directory removed under --force/--delete
now has its contents deleted entry-by-entry through the budgeted walker, so
every deleted file/dir counts toward --max-delete exactly like rsync (a
capped run leaves the remaining entries and exits 25).
2026-09-15 23:23:52 +02:00
TapTap 82a1d5e240 fix(delete): match rsync deletion semantics (#290)
- Scope the --delete extras walk to directories synchronized by the
  transfer: add a synchronized-directory section to the delete manifest
  (protocol 2.23.0) so --files-from subsets no longer delete untransmitted
  paths outside listed directory subtrees (data-loss fix).
- Separate --max-size/--min-size prune protection from --delete-excluded so
  size-pruned source mirrors survive (rsync parity).
- Unlink extraneous destination symlinks instead of skipping them.
- Make --max-delete partial (delete up to N, skip the rest) and exit 25;
  accept negative values as unlimited.
- Draw --delete-missing-args deletions from the shared --max-delete budget.
- Honor --force during --delay-updates publication.

Add unit and integration regression tests; update the pinned config wire
golden and version strings for the 2.23.0 manifest/status additions.
2026-09-15 23:12:03 +02:00
TapTap 3eec5a4cc3 feat(parity): rsync 3.4.1 checksum/timeout/temp-dir/connectivity parity (#289 #295 #296)
#289 checksum/compression:
- -c/--checksum now implies the incremental content quick-check (without
  implying -t), so an unchanged file is skipped like rsync.
- --checksum-choice/--cc accepts xxh64/xxhash, xxh3, xxh128, md5 and auto;
  md4/sha1/none and the two-name form are rejected by name.
- --compress-choice/--zc rejects lz4/zlib/zlibx by name (zstd/none/auto kept).
- --checksum-seed=0 is randomized per transfer and sent on the wire.
- --skip-compress uses rsync 3.4.1's default suffix list; slash separators and
  dot-less suffixes are accepted.
- add --no-whole-file.

#295 timeouts/alloc/temp-dir:
- --timeout default 0 (disabled), --contimeout default 60; 0 disables both,
  plus --no-timeout/--no-contimeout.
- --max-alloc=0 means no allocation limit (was rejected).
- --temp-dir accepts any dir, requires it to exist, and falls back to a
  non-atomic copy on EXDEV instead of aborting.

#296 connectivity/daemon:
- -M/--remote-option is rejected for daemon/TCP destinations (SSH-only).
- --trust-sender clarified as receiver-local; server-path tests added.
- --stop-at accepts rsync's full date form (y-m-dTh:m etc.).

Adds unit and integration coverage; no wire-field change, PROTOCOL_VERSION stays
2.22.0.
2026-09-15 22:20:02 +02:00
TapTap 6144c7fc7f fix(identity): rsync ownership parity for numeric-ids, dirs, maps, fake-super (#286, #294)
- #286: --numeric-ids is a mapping modifier only; it no longer activates
  chown by itself (identity_active_enabled/owner/group predicates), and
  --fake-super stores the resolved mapping instead of real-chowning.
- #286: apply owner/group to directories via the deferred directory
  metadata path; capture+transmit+apply directory xattrs/ACLs (-aX/-aA),
  including default ACLs, in STATUS_MKDIR/STATUS_DIR_TIMES.
- #294: --usermap/--groupmap support inclusive ranges, '*', empty FROM
  (unnamed ids), and receiver-side TO name resolution; --chown mixing with
  a same-side map is rejected like rsync.
- Protocol 2.22.0 -> 2.23.0 (map wire entry gains from_hi + to_name;
  dir frames gain a bounded xattr block).
2026-09-15 22:10:02 +02:00
TapTap ea4ab661b4 fix(parity): rsync 3.4.1 symlink and special-node semantics (#287, #288)
#287:
- --safe-links: keep safe in-tree links AS symlinks and skip unsafe
  (absolute or ".."-escaping) ones, mirroring rsync's unsafe_symlink().
  Skipped links are recorded as delete-protected so --delete does not
  remove their destination mirror (no silent data loss).
- --copy-unsafe-links: preserve safe links as symlinks and dereference
  only unsafe ones.
- --munge-links: receiver-side rewrite storing /rsyncd-munged/-prefixed
  targets (rsync parity), replacing the no-op #SYMLINK sender prefix.
- -l: store the target verbatim, including absolute and ".." targets
  (rsync -l parity); the old receiver containment silently dropped them.

#288:
- --specials: recreate unix-domain sockets via mknod(S_IFSOCK), which
  Linux permits unprivileged; keep EEXIST/EPERM skip behavior.
- --copy-devices: copy a device's content into a regular file when
  requested; skip unrequested non-regular entries like rsync's default.
2026-09-15 21:57:48 +02:00
TapTap 84827ca617 feat(output): rsync 3.4.1 selection and output parity (#291, #292)
#291:
- Compile --exclude/--include/--exclude-from/--include-from into the SAME
  ordered rule list as --filter/-f (first match wins), so the common
  `--include='*.txt' --exclude='*'` idiom and include-alone semantics match
  rsync. The legacy per-kind scanner arrays are no longer applied.
- -x/--one-file-system emits the cross-device mount-point directory entry
  (empty) instead of dropping it, in both the sequential and parallel scanners.
- Stop passing the legacy arrays to the scanner; document -f is --filter.

#292:
- New src/shared/format.c/.h: rsync "big_num" (comma-grouped integers) and
  decimal -h human sizes, %M/%t timestamp, and the STATUS_DEST_INFO codec.
- Receiver answers each STATUS_CHECK with a pre-transfer destination snapshot
  (new report_dest_info wire field + STATUS_DEST_INFO, PROTOCOL_VERSION
  2.23.0) so the sender can render true itemize columns.
- Itemize now emits rsync-correct update/type chars and c/s/t/p/o/g columns
  for files, dirs, symlinks and hard links, comparing size/time/perms/owner/
  group against the reported destination.
- --out-format gains %i %n %f %l %b %M %t %o %p %B %U %G %L; %f is the
  relative display path, %M the YYYY/MM/DD-HH:MM:SS form, %b the literal
  bytes sent.
- --list-only prints transfer-relative names, directory entries and ls-style
  grouped sizes.
- --stats prints rsync's multi-line block on stdout; -h uses decimal units.

Tests: unit tests for the filter ordering, format primitives, itemize
columns; integration + differential tests against real rsync 3.4.1 for
itemize/out-format/list-only/selection and -x. Golden wire len/hash and
protocol version strings updated for 2.23.0.
2026-09-15 21:57:39 +02:00
TapTap 23552e823d feat(cli): rsync short-option clustering and inline/attached values (#285)
Implement rsync 3.4.1 client-CLI parity:
- cluster boolean shorts (-av, -aAX, -rlpt) and accept attached values
  (-B1048576, -essh, -Mfoo); add the -r, -b, -L and -B short aliases
  (-r is a faithful no-op since FastSync is always recursive)
- stop OPT_NOOP (-s/--secluded-args, -r/--recursive) from swallowing the
  next argv
- add inline --opt=value for every value-taking long option, including
  --exclude/--include/--exclude-from/--include-from/--log-file (#291)
- accept --port on the server CLI in addition to -p (#296)
- reject unknown flags naming the flag and stating it is unsupported

Unit tests cover clustering, attached/inline values, the OPT_NOOP
argument-consumption fix and rejected shorts.
2026-09-15 21:12:53 +02:00
TapTap d1a567f7e3 ci: pin fastsync-ci:v11 with rsync 3.4.1 + acl/attr for parity tests
Add POSIX ACL/xattr tooling (acl, attr), zstd/lz4/xxhash dev libs and build rsync 3.4.1 from source so drop-in parity tests can run inside CI. Bump all workflow/agent image references v10 -> v11.
2026-09-15 20:37:50 +02:00
TapTap 93c1fc3c1f test: create fault-injection destination root in seeding fixture
CI / lint (push) Successful in 1m29s
CI / lint (pull_request) Successful in 1m29s
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
CI / sanitizers (undefined) (push) Successful in 1m4s
CI / sanitizers (address) (push) Successful in 1m10s
CI / fuzz-build (push) Successful in 39s
CI / coverage (push) Successful in 58s
CI / build-and-test (pull_request) Successful in 1m55s
CI / valgrind (push) Successful in 3m23s
CI / build-and-test (push) Successful in 5m11s
The captured_config fixture assumed fault_dst already existed, relying on earlier tests in the same xdist worker creating it via _recover. Under --dist=load a worker can receive the capture test first, so the receiver rejected a missing destination root and the capture run failed. Create DEST_DIR in the autouse seeding fixture so test order/distribution cannot matter.
2026-09-15 20:02:00 +02:00
TapTap 4815b1b281 test: account for group/other-write sanitization in new-dest mode expectation
CI / lint (push) Successful in 1m28s
CI / lint (pull_request) Successful in 1m28s
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
CI / sanitizers (undefined) (push) Successful in 1m1s
CI / sanitizers (address) (push) Successful in 1m6s
CI / fuzz-build (push) Successful in 38s
CI / coverage (push) Successful in 1m3s
CI / build-and-test (pull_request) Successful in 1m56s
CI / build-and-test (push) Failing after 4m50s
CI / valgrind (push) Successful in 3m23s
2026-09-15 19:41:12 +02:00
TapTap ad7bc3348b Merge branch 'feat/preserve-attr-split' into dev
CI / lint (push) Successful in 1m29s
CI / lint (pull_request) Successful in 1m28s
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
CI / sanitizers (undefined) (push) Successful in 1m4s
CI / sanitizers (address) (push) Successful in 1m10s
CI / fuzz-build (push) Successful in 38s
CI / coverage (push) Successful in 57s
CI / build-and-test (pull_request) Failing after 1m55s
CI / build-and-test (push) Failing after 4m55s
CI / valgrind (push) Successful in 3m23s
2026-09-15 19:32:12 +02:00
TapTap 34970b961c feat: per-attribute preservation flags -p/-t/-o/-g with --no-* negations (protocol 2.22.0)
Split FastSync's single use_metadata bundle into four independent rsync-parity attributes: preserve_perms, preserve_times, preserve_owner, preserve_group. use_metadata is now a derived transport bit (config_derived_use_metadata).

CLI: real -p/--perms, -t/--times, -o/--owner, -g/--group plus --no-perms/--no-times/--no-owner/--no-group (short and long) and --no-preserve; -a is now rsync -rlptgoD; --preserve = -pt; -A implies -p; -X does not; --chmod implies -p; --usermap/--groupmap/--chown imply owner/group per side; --incremental/--delta still auto-preserve unless negated.

Receiver: per-attribute FileAttrPolicy gating for files, dirs (modes applied at end of transfer), symlinks and specials; rsync -E read-bit rule; new files get source_mode & ~umask sanitized (no group/other write); per-side identity resolution; deferred directory metadata; batch dir-metadata replay; daemon modules without 'client owner = yes' no longer refuse plain -a but force super off (no ownership) with a warning.

Wire: PROTOCOL_VERSION 2.21.0 -> 2.22.0 (four appended config bools, golden 653 / 95530566005420798). FileMetadata/chunk/batch framing unchanged. Docs/CHANGELOG/CMake updated to 2.22.0.
2026-09-15 19:32:02 +02:00
TapTap b3f7cad4db Merge docs/handoff: session handoff document
CI / lint (push) Successful in 1m29s
CI / lint (pull_request) Successful in 1m29s
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
CI / sanitizers (undefined) (push) Successful in 59s
CI / sanitizers (address) (push) Successful in 1m6s
CI / fuzz-build (push) Successful in 38s
CI / coverage (push) Successful in 56s
CI / build-and-test (pull_request) Successful in 1m55s
CI / valgrind (push) Successful in 3m24s
CI / build-and-test (push) Successful in 5m35s
2026-09-14 19:36:28 +02:00
TapTap cd8a84c0a2 docs: add session handoff (status, next steps, deferred security items) 2026-09-14 19:36:28 +02:00
TapTap 09c384d7d0 Merge branch 'docs/readme-refresh' into dev
CI / lint (push) Successful in 1m25s
CI / lint (pull_request) Successful in 1m24s
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
CI / sanitizers (undefined) (push) Successful in 1m1s
CI / sanitizers (address) (push) Successful in 1m7s
CI / fuzz-build (push) Successful in 36s
CI / coverage (push) Successful in 56s
CI / build-and-test (pull_request) Successful in 1m54s
CI / valgrind (push) Successful in 3m18s
CI / build-and-test (push) Successful in 5m34s
# Conflicts:
#	README.md
2026-09-14 18:52:54 +02:00
TapTap 81ad313ee5 docs: refresh README against implementation and guard against drift
Bring README.md and RSYNC_COMPAT.md in line with the actual code/CLI and add
an automated guard so they cannot silently drift again.

Waves A-E:
- Correct stale compatibility claims: archive is `-rlptD` (owner/group are
  opt-in via identity flags, not implied), and symlinks, hard links, xattrs,
  ACLs and `--dirs` are implemented.
- Remove documented-but-nonexistent features: the six unread FASTSYNC_* env
  vars, and `--client-cn` (server-only) from the client table.
- Repair the corrupted "Implementation Details" section (broken list numbering
  and emphasis) and correct it against the source.
- Sync the client and server option tables with usage.c / server_cli.c, and
  document server-contacting `--dry-run` (protocol 2.21.0).
- Hygiene: `# FastSync` heading, real build commands, consistent binary names,
  runnable TLS examples, daemon module keys.

Also align the client `--help` / archive log wording and the RSYNC_COMPAT
archive rows with the opt-in ownership model, and add
tests/integration/test_readme_consistency.py (marked `ci`) asserting every
documented FASTSYNC_* var is read in src/ and every documented client/server
flag appears in the corresponding `--help`.
2026-09-14 18:48:42 +02:00
88 changed files with 10447 additions and 2481 deletions

No files matched your search

+6 -6
View File
@@ -9,7 +9,7 @@ on:
jobs:
lint:
runs-on: ubuntu-latest
container: gitea.tap-tap.win/taptap/fastsync-ci:v10
container: gitea.tap-tap.win/taptap/fastsync-ci:v11
steps:
- name: Checkout
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
@@ -26,7 +26,7 @@ jobs:
# suite) run on merge to dev/main, so PR CI stays well under ~3 minutes.
build-and-test:
runs-on: ubuntu-latest
container: gitea.tap-tap.win/taptap/fastsync-ci:v10
container: gitea.tap-tap.win/taptap/fastsync-ci:v11
needs: lint
steps:
- name: Checkout
@@ -51,7 +51,7 @@ jobs:
sanitizers:
runs-on: ubuntu-latest
container: gitea.tap-tap.win/taptap/fastsync-ci:v10
container: gitea.tap-tap.win/taptap/fastsync-ci:v11
needs: lint
if: github.event_name == 'push'
strategy:
@@ -72,7 +72,7 @@ jobs:
fuzz-build:
runs-on: ubuntu-latest
container: gitea.tap-tap.win/taptap/fastsync-ci:v10
container: gitea.tap-tap.win/taptap/fastsync-ci:v11
needs: lint
if: github.event_name == 'push'
steps:
@@ -94,7 +94,7 @@ jobs:
coverage:
runs-on: ubuntu-latest
container: gitea.tap-tap.win/taptap/fastsync-ci:v10
container: gitea.tap-tap.win/taptap/fastsync-ci:v11
needs: lint
if: github.event_name == 'push'
steps:
@@ -118,7 +118,7 @@ jobs:
valgrind:
runs-on: ubuntu-latest
container: gitea.tap-tap.win/taptap/fastsync-ci:v10
container: gitea.tap-tap.win/taptap/fastsync-ci:v11
needs: lint
if: github.event_name == 'push'
steps:
+1 -1
View File
@@ -116,7 +116,7 @@ The project uses Gitea Actions. Key jobs:
jobs:
new-job:
runs-on: ubuntu-latest
container: gitea.tap-tap.win/taptap/fastsync-ci:v10
container: gitea.tap-tap.win/taptap/fastsync-ci:v11
steps:
- uses: actions/checkout@v4
- name: Configure
+1 -1
View File
@@ -16,7 +16,7 @@ Ask the user or determine from context:
- **Minor** (x.Y.0) — new features, backward compatible
- **Patch** (x.y.Z) — bug fixes, no protocol changes
Current version: `PROTOCOL_VERSION "2.21.0"` in `src/shared/config.h`
Current version: `PROTOCOL_VERSION "2.23.0"` in `src/shared/config.h`
### Step 2: Check Protocol Version
+8 -7
View File
@@ -4,18 +4,19 @@ FastSync is a high-performance file synchronization system written in C11. It su
## Dependency installation
**CI rule:** never add `apt-get install` / `pip install` steps to CI workflows — use the custom Docker image instead. The image is built from the repo-root `Dockerfile` and is the same image CI uses: `gitea.tap-tap.win/taptap/fastsync-ci:v10`. It contains the full toolchain: gcc/g++, CMake, libzstd-dev, libssl-dev, make, git, cppcheck, clang-format, python3 + pytest + pytest-xdist, openssh-client, and Node.js.
**CI rule:** never add `apt-get install` / `pip install` steps to CI workflows — use the custom Docker image instead. The image is built from the repo-root `Dockerfile` and is the same image CI uses: `gitea.tap-tap.win/taptap/fastsync-ci:v11`. It contains the full toolchain: gcc/g++, CMake, libzstd-dev, libssl-dev, make, git, cppcheck, clang-format, python3 + pytest + pytest-xdist, openssh-client, Node.js, plus `rsync` 3.4.1 (with zstd/xxhash/lz4), `acl` and `attr` (setfacl/getfacl, setfattr/getfattr) for drop-in parity tests.
**Host rule:** for local development, use `nix-shell` (see `README.md`) which provides zstd, OpenSSL, CMake, and gcc. The Docker image can also be used locally for CI parity.
```bash
# Use the prebuilt CI image directly (faster, guaranteed CI parity)
docker pull gitea.tap-tap.win/taptap/fastsync-ci:v10
docker tag gitea.tap-tap.win/taptap/fastsync-ci:v10 fastsync-ci:local
docker pull gitea.tap-tap.win/taptap/fastsync-ci:v11
docker tag gitea.tap-tap.win/taptap/fastsync-ci:v11 fastsync-ci:local
# Or build the image from the repo-root Dockerfile
# (Note: the prebuilt :v10 image reflects the previous Dockerfile state;
# rebuild from source to pick up any newly added packages like lcov/valgrind.)
# (Note: the prebuilt :v11 image is built from the current Dockerfile and
# includes rsync 3.4.1 plus acl/attr; rebuild from source after changing
# the Dockerfile.)
docker build -t fastsync-ci:local .
# Build, run unit tests, and run integration tests inside the container
@@ -66,14 +67,14 @@ When running the CI workflow via `tea` (the task execution agent), always set a
### If lint (clang-format) fails
Run clang-format in the CI Docker image to match the exact CI version:
```bash
docker run --rm -v "$PWD:/workspace" -w /workspace gitea.tap-tap.win/taptap/fastsync-ci:v10 \
docker run --rm -v "$PWD:/workspace" -w /workspace gitea.tap-tap.win/taptap/fastsync-ci:v11 \
sh -c 'find src/ tests/ -name "*.c" -o -name "*.h" | xargs clang-format -i'
```
### If cppcheck fails
Fix reported issues locally, then verify with:
```bash
docker run --rm -v "$PWD:/workspace" -w /workspace gitea.tap-tap.win/taptap/fastsync-ci:v10 \
docker run --rm -v "$PWD:/workspace" -w /workspace gitea.tap-tap.win/taptap/fastsync-ci:v11 \
sh -c 'cppcheck --enable=warning,style,performance,portability --suppress=missingIncludeSystem --error-exitcode=1 --inline-suppr src/ tests/'
```
+114
View File
@@ -4,6 +4,120 @@ All notable changes to FastSync are documented here. Versions match
`PROTOCOL_VERSION` (printed by `fastsync --version`); the client and server must
run the same version because the handshake is strict.
## [2.23.0] - 2026-09-16
### Added
- **Rsync-parity wave.** Closed the remaining CLI, filesystem, ownership,
deletion, and output gaps against rsync 3.4.1.
- Short options `-r` (`--recursive`), `-b` (`--backup`), `-L`
(`--copy-links`), and `-B` (`--block-size`/`--delta-block`); rsync
short-option clustering (`-av`, `-aAX`, `-rlpt`) and attached/inline values
(`--opt=value`, `-B1000`, `-essh`, `-MOPT`). A value that starts with `-`
is not mistaken for a cluster.
- `-c`/`--checksum` now implies the incremental checksum quick-check (and,
like rsync, does not imply `-t`).
- `--checksum-choice`/`--cc` accepts `xxh64`/`xxhash`/`xxh3`/`xxh128`/`md5`/
`auto` and rejects `md4`/`sha1`/`none` and the two-name form by name;
`--checksum-seed=0` (the default) is randomized per transfer and the chosen
seed is sent to the receiver.
- `--compress-choice`/`--zc` accepts `zstd`/`none`/`auto` and rejects
`lz4`/`zlib`/`zlibx` by name; `--skip-compress` defaults to rsync 3.4.1's
built-in suffix list; `--no-whole-file` is accepted.
- `--timeout` defaults to 0 (disabled) and `--contimeout` to 60 s (both `0`
disables), matching rsync; `--max-alloc=0` means no local limit.
- `--temp-dir` is confined to the receive root (absolute/`..` rejected by the
receiver) and an `EXDEV` install falls back to a non-atomic copy.
- `--numeric-ids` is documented as a mapping modifier only;
`--usermap`/`--groupmap` support inclusive `LOW-HIGH` ranges, `*`,
empty-`FROM` (unnamed ids), and receiver-resolved `TO` names; `--chown`
conflicts with a map on the same side are rejected.
- `--fake-super` records the *resolved* owner (never a real chown) and replays
mode/time; directory ownership and directory xattrs/ACLs are preserved.
- `-l`/`--links` stores symlink targets verbatim (absolute and `..`-bearing
included), matching rsync; `--safe-links`/`--copy-unsafe-links` are applied
sender-side and `--munge-links` uses rsync's `/rsyncd-munged/` marker;
`--trust-sender` no longer affects symlink targets.
- `--specials` recreates unix sockets with `mknod(S_IFSOCK)` (so `-D` covers
the full rsync node set).
- Deletion: the manifest carries a synchronized-directory section so
`--files-from` subsets no longer delete untransmitted paths;
`--delete-excluded` leaves size-pruned mirrors protected; extraneous
destination symlinks are unlinked (never followed); `--max-delete=N` is
partial (delete up to N, skip the rest, exit 25) and `--delete-missing-args`
removals draw from the same budget; `--force` is honored during
`--delay-updates` publication.
- `-x`/`--one-file-system` emits the mount-point directory entry; the
`--include`/`--exclude` layers are an ordered first-match rule list.
- `--chmod` is a faithful port of rsync 3.4.1 (numeric/symbolic, `D`/`F`/`X`,
`s`/`t`, append semantics, no `-p` implication, no sanitization).
### Changed
- `PROTOCOL_VERSION` bumped `2.22.0 → 2.23.0`: the delete manifest gains a
synchronized-directory section and the terminal status gains
`STATUS_DELETE_LIMIT` (client exit 25 on a `--max-delete`-capped commit).
- **The 2.22.0 mode-masking divergence is removed.** Under `-p` the source mode
is copied exactly, including `S_IWGRP`/`S_IWOTH` and setuid/setgid/sticky;
`--chmod` no longer implies `-p`. New files without `-p` still use
`source_mode & ~umask` when metadata is present (else `0644`), and new
directories without `-p` still use the `0755` creation default.
- `--protocol=NUM` accepts only the current `2.23.0` version string.
### Notes
- The rsync-compatibility matrix (`RSYNC_COMPAT.md`) now classifies every row
as **parity**, **caveat** (works with a documented divergence), or
**divergent** (not supported/no-op/impossible), replacing the previous
misleading "N implemented / 0 divergence" summary. Durable documented
divergences remain: receiver-side symlink target containment is not enforced
by default (verbatim storage is rsync parity; use `--safe-links`),
`--temp-dir` rejects absolute/foreign-filesystem paths, `--copy-devices`
reads a bounded `st_size`, a broken referent under `--copy-links` exits 0,
new directories without `-p` use `0755`, `--stats` receiver-only counters are
0, and `--password-file`/`--early-input`/`--hash-credentials`/`--iterations`
and the batch format are FastSync-native.
## [2.22.0] - 2026-09-15
### Added
- **Per-attribute metadata preservation (protocol 2.22.0).** The former single
metadata bundle is split into four independent, rsync-compatible flags:
`-p/--perms`, `-t/--times`, `-o/--owner`, and `-g/--group`, each applied
independently on the receiver, with negations `--no-perms`/`--no-times`/
`--no-owner`/`--no-group` (short `--no-p`/`--no-t`/`--no-o`/`--no-g`) and
`--no-preserve` clearing all four. `-a/--archive` is now full rsync
`-rlptgoD` (owner and group included; their application stays
privilege-gated). `-A/--acls` and `--chmod` imply `-p`, `-X/--xattrs` does
not, `-E/--executability` sets only executability, and `-U`/`-N` do not imply
`-t`. `--incremental`/`--delta` still auto-preserve perms+times unless the
user explicitly negated them.
- Receiver applies directory modes under `-p` (at the end of the transfer,
alongside the deferred directory times) and symlink mode under `-p`; `-O`
suppresses directory times only.
### Changed
- `PROTOCOL_VERSION` bumped `2.21.0 → 2.22.0`: the binary config frame gains
four appended booleans (`preserve_perms`/`preserve_times`/`preserve_owner`/
`preserve_group`) after `omit_link_times`. The fixed-width `FileMetadata`
layout is unchanged; the receiver derives the metadata-frame gate
(`use_metadata`) from the four attributes.
### Notes
- Documented divergences from rsync: a client-supplied mode never grants
group/other write (`S_IWGRP|S_IWOTH` are stripped for files, directories,
symlinks, and specials; rsync's `-p` preserves them exactly); a brand-new file
without `-p` gets `source_mode & ~umask` (sanitized) when metadata is present,
else the historical fixed `0644`; `--chmod` implies `-p` (rsync does not);
`-o`/`-g` map by name on the receiver with a raw-numeric fallback (only
numeric ids cross the wire); and a daemon module without `client owner = yes`
does not refuse a plain `-a`/`-o`/`-g` but forces super-user activities off,
applies no ownership, and logs a warning (explicit `--chown`/`--usermap`/
`--groupmap`/`--numeric-ids`/`--copy-as`/`--super` are still refused).
## [2.21.0] - 2026-09-14
### Added
+3 -1
View File
@@ -1,6 +1,6 @@
cmake_minimum_required(VERSION 3.22)
project(FastFileTransfer VERSION 2.21.0)
project(FastFileTransfer VERSION 2.23.0)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
set(CMAKE_C_STANDARD 11)
@@ -96,6 +96,7 @@ set(SHARED_SRCS
src/shared/file_send.c
src/shared/file_store.c
src/shared/filter.c
src/shared/format.c
src/shared/hardlink.c
src/shared/identity.c
src/shared/log.c
@@ -213,6 +214,7 @@ set(TEST_SRCS
tests/test_file.c
tests/test_file_list.c
tests/test_file_sendfile.c
tests/test_format.c
tests/test_fuzz_smoke.c
tests/test_glob.c
tests/test_hardlink.c
+15 -1
View File
@@ -2,8 +2,22 @@ FROM ubuntu:24.04
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc g++ make libc6-dev cmake libzstd-dev libssl-dev git ca-certificates curl cppcheck clang-format \
python3 python3-pip python3-venv openssl openssh-client \
lcov valgrind clang libclang-rt-18-dev && \
lcov valgrind clang libclang-rt-18-dev \
acl attr zlib1g-dev liblz4-dev libxxhash-dev && \
pip3 install --break-system-packages pytest pytest-xdist && \
curl -fsSL https://deb.nodesource.com/setup_20.x | bash - && \
apt-get install -y --no-install-recommends nodejs && \
rm -rf /var/lib/apt/lists/*
# rsync is used as the reference implementation for drop-in parity tests.
# Ubuntu 24.04 ships 3.2.7, so build the pinned 3.4.1 reference from source.
ARG RSYNC_VERSION=3.4.1
ARG RSYNC_SHA256=2924bcb3a1ed8b551fc101f740b9f0fe0a202b115027647cf69850d65fd88c52
RUN curl -fsSL "https://download.samba.org/pub/rsync/src/rsync-${RSYNC_VERSION}.tar.gz" -o /tmp/rsync.tar.gz && \
echo "${RSYNC_SHA256} /tmp/rsync.tar.gz" | sha256sum -c - && \
tar -xzf /tmp/rsync.tar.gz -C /tmp && \
cd "/tmp/rsync-${RSYNC_VERSION}" && \
./configure --enable-zstd --enable-xxhash --enable-lz4 && \
make -j"$(nproc)" && \
make install && \
rm -rf "/tmp/rsync-${RSYNC_VERSION}" /tmp/rsync.tar.gz
+66
View File
@@ -0,0 +1,66 @@
# FastSync — Session Handoff (2026-09-14)
## Current status
- **Release `v2.21.0`** tagged (`919a729`, "Release v2.21.0"); full CI green
(run 552: lint, build-and-test, ASan, UBSan, fuzz-build, coverage, valgrind).
`dev` has the release commit plus later doc-only merges (a README refresh and
this handoff).
- **Release PR #284 (`dev` -> `main`)** open, CI green (run 553).
`main` is protected: it needs review/approval to merge.
https://gitea.tap-tap.win/TapTap/FastSync/pulls/284
- **`PROTOCOL_VERSION` = `"2.23.0"`** (`src/shared/config.h`); CMake
`project(FastFileTransfer VERSION 2.23.0)`.
- Working tree clean; no wave worktrees remain.
## What landed this session
1. **Wave 8 (refactors):** Config X-macro wire table; single-owner `authorized_root`;
daemon per-module/per-host caps + cross-process auth lockout (`daemon_limits.[ch]`);
`Data` charge returns to its owning `ProtocolSession`.
2. **Wave 9 (protocol 2.21.0):** optional `STATUS_ERROR_DETAIL` rejection reasons;
server-contacting `--dry-run` (`STATUS_DRY_RUN_TRANSFER`, receiver mutates nothing).
3. **Security wave:** ran 5 parallel audits (wire parsing; daemon/transport/TLS/auth;
receiver confinement; client/CLI/SSH; crypto/memory/limits). Fixed all HIGH and the
confirmed MEDIUMs:
- SSH `-o ProxyCommand=…` argument injection (RCE) — reject leading `-`, insert `--`.
- Truncated zstd frame infinite CPU loop (remote DoS).
- FIFO receiver opens lacked `O_NONBLOCK` (indefinite hang).
- `--inplace` could write a FIFO/device (bypass of `--write-devices` gate).
- `--force` not gated by server `--allow-delete`.
- Privileged standalone server defaulted super activities on; added `--allow-super`
(never honored with `--stdio`).
- `--dry-run` content/hash oracle on `read only`/basis files removed.
- Empty `hosts allow`/`deny`/`auth users` now rejected.
- TLS: AEAD-only 1.2 + server preference, TOCTOU-safe key load, IP-SAN verify,
CN-truncation guard. Glob backtracking bounded; line reads bounded; ACL xattrs
gated on `--acls`; decompression/chunk memory charged; pre-auth `basis_count`
NULL-deref fixed.
4. **Tooling:** benchmark accuracy (data mix, verification, percentiles, `tc`,
`build-bench/`, `--warm` mode); `shell.nix` full toolchain and no build-on-entry;
docs state push-only / remote-source unsupported.
5. **Preserve-attribute split (protocol 2.22.0)** landed on `feat/preserve-attr-split`: per-attribute `-p/-t/-o/-g` + `--no-*` negations, `-a` = `-rlptgoD`, and the 2.21.0 → 2.22.0 wire bump.
6. **Rsync-parity wave (protocol 2.23.0)** on `feat/rsync-parity`: rsync short options/clustering/attached values (`-r`/`-b`/`-L`/`-B`, `-av`, `-aAX`, `-B1000`, `-essh`, `-MOPT`), `-c` checksum quick-check, `--checksum-choice`/`--compress-choice` validation and seed randomization, rsync timeout/max-alloc defaults, temp-dir confinement + `EXDEV` fallback, ownership/mapping parity (numeric-ids modifier, map ranges/`*`/empty-FROM, `--chown`+map conflicts, fake-super resolved-owner record), verbatim symlink storage with rsync `--safe-links`/`--munge-links`, socket recreation under `--specials`, `--chmod` 3.4.1 semantics, and delete scoping + `--max-delete` partial/exit-25. Wire: appended delete-manifest synchronized-directory section and `STATUS_DELETE_LIMIT`.
## Next steps
1. **Merge PR #284** (`dev` -> `main`) once reviewed (protected branch).
2. **Deferred security items** (documented, not implemented):
- Pre-auth config/daemon-auth handshake has no aggregate wall-clock deadline
(per-message timeout only) — slowloris holds connection slots.
- Per-source registry fails open when the shared table is full (per-module/global
caps and host ACLs still apply); consider fail-closed or larger/evicting table.
- SCRAM-like daemon auth has no TLS channel binding (and is not RFC 5802).
- `cleanup()` signal handler calls non-async-signal-safe teardown; daemon `umask(0)`.
- Wire protocol assumes homogeneous word size/endianness (lengths are native
`size_t`) — document or move to fixed-width framing.
3. **Out of scope / intentional:** pull (remote source) mode is **not** planned —
FastSync is push-only; see `RSYNC_COMPAT.md#direction`.
## Key facts / commands
- CI image: `gitea.tap-tap.win/taptap/fastsync-ci:v11` (alias `fastsync-ci:local`).
- Build/test: `cmake -B build -S . -DSTRICT_WARNINGS=ON && cmake --build build -j$(nproc) && ./build/tests`
then `python3 -m pytest tests/integration/ -n 4 --dist=load -m "not setpriv"`.
- Dev shell: `nix-shell` (provides clang-format, cppcheck, pytest-xdist, openssh,
rsync, iproute2, valgrind, lcov; does not build on entry).
- Gitea API token: supplied out-of-band via the `TOKEN` environment variable; it is
intentionally **not** recorded in this file.
- CI polling: `GET /api/v1/repos/TapTap/FastSync/actions/runs?limit=N`, match `head_sha`,
then `/actions/runs/<id>/jobs`.
+354 -159
View File
@@ -1,4 +1,4 @@
#FastSync
# FastSync
FastSync is a high-performance file synchronization tool designed to become a
drop-in replacement for common `rsync` workflows. It keeps the familiar
@@ -7,7 +7,7 @@ multithreading, streaming zstd compression, chunking, zero-copy TCP transfers,
and native TCP/TLS transports.
The release version is FastSync's client/server protocol version (printed by
`fastsync --version`); client and server must match. See
`./build/client --version`); client and server must match. See
[CHANGELOG.md](CHANGELOG.md) for the history.
The compatibility target is straightforward:
@@ -51,8 +51,9 @@ replacement for every rsync feature or protocol mode.
- Rsync-style source and destination arguments.
- SSH transport using `user@host:destination` paths below the remote authorized root.
- TCP client/server transfers.
- Dry runs, excludes, includes, size filters, backups, statistics, and
bandwidth limiting.
- Dry runs (server-contacting since protocol 2.21.0 for server-routed targets),
excludes, includes, size filters, backups, statistics, and bandwidth
limiting.
- Incremental size/mtime checks and optional xxHash64 content checks.
- FastSync-native delta transfer for changed files.
- Optional mode and timestamp preservation.
@@ -60,19 +61,40 @@ replacement for every rsync feature or protocol mode.
- Temporary-file writes with atomic rename by default.
- Path traversal checks and destination-root confinement.
### Not yet equivalent to rsync
### Boundaries and documented divergences
The items below summarize FastSync's rsync compatibility status — recently
closed gaps and the remaining known divergences. Each row of the detailed
matrix is classified as parity, caveat, or divergent in
[`RSYNC_COMPAT.md`](RSYNC_COMPAT.md).
- The FastSync wire protocol is not the rsync wire protocol.
- SSH mode requires `fastsync-server` on the remote host.
- Archive mode does not yet provide all of rsync's `-rlptgoD` behavior.
- Symlink transfer is incomplete; link targets are not yet recreated in all
modes.
- Owner/group, ACL, xattr, and hard-link handling is incomplete or
unavailable.
- Archive mode covers rsync's `-rlptgoD` behavior — links, permissions, times,
owner, group, devices, and special files — and does not imply compression or
multithreading (see [Client](#client)). Ownership application is still
privilege-gated: a receiver that cannot `chown` logs a warning and skips it.
Under `-p` the source mode is copied exactly, including setuid/setgid/sticky
and group/other-write bits (strict rsync parity; see
[`RSYNC_COMPAT.md`](RSYNC_COMPAT.md)).
- Symlink transfer stores targets **verbatim** (`-l`/`--links`), including
absolute and `..`-bearing targets, matching rsync. The receiver does not
enforce a containment predicate by default; `--safe-links` drops unsafe
targets on the sender, and `--munge-links` rewrites them with rsync's
`/rsyncd-munged/` marker. `--trust-sender` does not affect symlink targets.
A destination later consumed by a link-following tool can therefore follow a
link outside the receive root — use `--safe-links` for untrusted sources.
- Hard links (`-H`/`--hard-links`), extended attributes (`-X`/`--xattrs`), and
POSIX ACLs (`-A`/`--acls`) are preserved; owner/group is applied through
`-o`/`-g` (or an `-a`/`--archive` transfer), through the opt-in identity flags
(`--chown`/`--usermap`/`--groupmap`/`--numeric-ids`/`--copy-as`), and only when
the receiver has permission. See
[`RSYNC_COMPAT.md`](RSYNC_COMPAT.md) for the exact semantics and documented
divergences.
- Device and special-file preservation is implemented with documented
divergences: recreated device nodes require `CAP_MKNOD` on the receiver (a
non-root receiver skips the entry), and sockets cannot be recreated (FIFOs
are).
non-root receiver skips the entry), while FIFOs **and unix sockets** are
recreated (`--specials`).
- Sparse-file hole preservation (`-S`, `--sparse`) is implemented receiver-side:
long all-zero runs are written as holes (no wire change; the full file image
is already in memory).
@@ -80,78 +102,151 @@ replacement for every rsync feature or protocol mode.
the write atomic (temp + rename). With `--partial`, a failed/interrupted write
now retains the already-written temp at the destination path (best-effort) so
a later `--append`/`--append-verify` run can resume it.
- `--dirs` is not implemented. Its compatibility aliases `--old-dirs` and
`--old-d` are recognized but rejected explicitly rather than silently using
FastSync's recursive directory behavior.
- `-d`/`--dirs` and its aliases `--old-dirs`/`--old-d` transfer the named
directory entries without recursing into their contents.
- Short-option names are now rsync-parity (Phase 7 Wave A): FastSync's former
collisions were renamed (`-j`/`--threads`, `--preserve`, `--sendfile`,
`--chunk-serialization`, `--timeout`, `--ssh-port`), so `-m`, `-M`, `-f`,
`-s`, `-T`, `-p`, `-c`, `-a`, and `-z` follow rsync. See `RSYNC_COMPAT.md`.
`-s`, `-T`, `-p`, `-c`, `-a`, and `-z` follow rsync.
- Short-option clustering (`-av`, `-aAX`, `-rlpt`) and attached values
(`-B1000`, `-essh`, `-MOPT`, `--opt=value`) are accepted, matching rsync.
- `-r`, `-b`, `-L`, and `-B` are parsed with the rsync short names.
- `--stats` prints the counters FastSync can observe locally; receiver-only
counters (matched data, file-list bytes, deleted count) are reported as 0, and
`--progress` is an aggregate line rather than a per-file block.
The detailed flag matrix is maintained in
[`RSYNC_COMPAT.md`](RSYNC_COMPAT.md). It distinguishes implemented,
partial, alternate, and planned behavior.
[`RSYNC_COMPAT.md`](RSYNC_COMPAT.md). It reports each row as **parity**,
**caveat** (works with a documented divergence), or **divergent** (not
supported), rather than treating "parsed" as parity.
## Quick Start
### Build
`compile_commands.json` is a symlink to `build/compile_commands.json` and is used by clangd/editor tooling; its target is generated by the build, so it dangles until the first build.
```bash
cmake -B build -S .
cmake --build build -j$(nproc)
```
This produces `./build/client` and `./build/server`. `compile_commands.json` is a symlink to `build/compile_commands.json` and is used by clangd/editor tooling; its target is generated by the build, so it dangles until the first build.
### Client
| Argument | Description |
|----------|-------------|
| Positional | `<source> <dest>` — automatic SSH detection if dest contains `:` |
| `-c, --checksum` | Verify content by checksum instead of size+mtime |
| `-c, --checksum` | Verify content by checksum instead of size+mtime (implies the incremental checksum quick-check) |
| `--checksum-choice <alg>` | Whole-file checksum algorithm: `xxh64`/`xxhash` (default), `xxh3`, `xxh128`, `md5`, or `auto`; `md4`/`sha1`/`none` are rejected by name |
| `-z, --compress [level]` | Enable streaming zstd compression (level 1–22, default 5) |
| `-a, --archive` | rsync archive mode (`-rlptgoD`): links, metadata, devices and specials (not compression/multithreading) |
| `--compress-choice <alg>` | Compression algorithm: `zstd` (default), `none`, or `auto`; `lz4`/`zlib`/`zlibx` are rejected by name |
| `--skip-compress <list>` | Skip compression for suffixes (`/`- or `,`-separated); defaults to rsync 3.4.1's built-in suffix list |
| `-a, --archive` | rsync archive mode (`-rlptgoD`): links, perms, times, owner, group, devices and specials; ownership application stays privilege-gated (not compression/multithreading) |
| `-j, --threads[=N]` | Multithreading mode; `N` (1–256) sets the parallel scanner worker count, bare `-j`/`--threads` uses the default |
| `-m` | rsync `--prune-empty-dirs` (short form now rsync-parity) |
| `-r, --recursive` | Recurse into directories (FastSync is always recursive; accepted for rsync compatibility) |
| `-d, --dirs` | Transfer the named directory entries without recursing into their contents; aliases `--old-dirs`/`--old-d` |
| `-R, --relative` | With `--files-from`, preserve each listed entry's relative path below the destination root |
| `--chunk-serialization` | Chunk serialization (batch all files per chunk; long form only) |
| `-s` | rsync `--secluded-args` compatibility no-op (remote SSH argv is already injection-safe) |
| `--sendfile` | Sendfile zero-copy. Incompatible with compression / chunk serialization. TCP only. Long form only. |
| `--preserve` | Preserve supported file metadata (mode and mtime; ownership and atime are unsupported) |
| `-n, --dry-run` | Scan and print what would be transferred |
| `-p, --perms` | Preserve permission bits (part of the metadata bundle) |
| `--ssh-port <port>` | SSH port (default: 22) |
| `-v, --verbose` | Enable debug logging |
| `-q, --quiet` | Suppress non-error output |
| `--progress` | Show real-time transfer speed |
| `-P` | Enables partial-transfer mode + progress output; interrupted writes retain the already-written temp for resumption |
| `--delete` | Delete files on receiver not present in source (default timing: delete-after, i.e. only after the whole transfer succeeded) |
| `--preallocate` | Allocate destination file space up front (fail-fast on a full disk) |
| `--append` | Resume a shorter destination by appending only its tail (prefix not verified; requires `--incremental`) |
| `--append-verify` | Like `--append`, but verifies the retained prefix checksum first (falls back to a full transfer on mismatch) |
| `-W, --whole-file` | Transfer changed files without delta processing; `--no-whole-file` clears it |
| `-B <n>, --block-size <n>` | Delta block size in bytes (alias `--delta-block`) |
| `--checksum-seed <n>` | Seed for the whole-file xxHash digest; an unset/`0` seed is randomized per transfer, matching rsync |
| `-I, --ignore-times` | Transfer files even when size and mtime match |
| `--size-only` | Skip incremental files matching in size, ignoring mtime |
| `--preserve` | Preserve mode and mtime (`-p` + `-t`; add `-o`/`-g` for owner/group or `-U`/`--atimes` for atime; `-N`/`--crtimes` captures birth time but cannot apply it) |
| `-U, --atimes` | Preserve access times. Captured with the metadata payload; does not enable ownership. |
| `-N, --crtimes` | Capture birth time; cannot be applied (documented divergence) |
| `-p, --perms` | Preserve permission bits. Strict rsync parity: the source mode is copied exactly, including setuid/setgid/sticky and group/other-write bits |
| `-t, --times` | Preserve modification times |
| `-o, --owner` | Preserve the source owner (privilege-gated; mapped by name on the receiver with a numeric fallback) |
| `-g, --group` | Preserve the source group (privilege-gated; mapped by name on the receiver with a numeric fallback) |
| `--no-perms`, `--no-times`, `--no-owner`, `--no-group`, `--no-preserve` | Negate the per-attribute flags (short `--no-p`/`--no-t`/`--no-o`/`--no-g`; `--no-preserve` clears all four) |
| `-E, --executability` | Preserve executable permission bits |
| `-X, --xattrs` | Preserve user `user.*` extended attributes |
| `-A, --acls` | Preserve POSIX ACLs |
| `--chmod <changes>` | Modify transferred permissions (rsync syntax) |
| `--chown=USER:GROUP` | Override the ownership of transferred files |
| `--usermap=MAP` | Map usernames when applying ownership |
| `--groupmap=MAP` | Map group names when applying ownership |
| `--numeric-ids` | Apply source numeric uid/gid directly instead of mapping by name |
| `--copy-as=USER[:GROUP]` | Force every written entry to USER[:GROUP] (requires a privileged receiver) |
| `--fake-super` | Record the resolved owner plus mode/time in a reserved `user.fastsync.stat` xattr and replay mode/time; never performs a real chown |
| `--super` | Permit the receiver to attempt confined super-user activities (device nodes) |
| `-D` | Preserve device and special files (implies `--devices --specials`) |
| `--devices` | Recreate device nodes on the destination (privileged; skipped without `CAP_MKNOD`) |
| `--specials` | Recreate special files: FIFOs and unix sockets |
| `--remove-source-files` | Remove regular source files after a successful transfer |
| `--exclude <pattern>` | Exclude files matching glob pattern (repeatable) |
| `--exclude-from <file>` | Read exclude patterns from a file (one per line) |
| `--include <pattern>` | Only transfer files matching glob pattern (repeatable, whitelist) |
| `--include-from <file>` | Read include patterns from a file |
| `--files-from <file>` | Read the source file list from FILE (paths relative to the source root) |
| `--max-size <n>` | Skip files larger than n bytes |
| `--min-size <n>` | Skip files smaller than n bytes |
| `-x, --one-file-system` | Do not cross filesystem boundaries; the mount-point directory entry is emitted (empty at the destination) without descending |
| `--max-alloc <SIZE>` | Maximum single allocation (binary units: B, K, M, G, T, P, E; default 1G; `0` = no local limit, matching rsync) |
| `-u, --update` | Skip files newer than the source on the receiver |
| `--incremental` | Skip files unchanged since last transfer (size + mtime). Auto-enables `--preserve`. Incompatible with `--chunk-serialization`. |
| `--existing` | Skip files not already present at the destination; update existing files normally. |
| `--compare-dest <dir>` | Extra comparison basis: unchanged files are not transferred (requires/implies `--incremental`) |
| `--copy-dest <dir>` | Like `--compare-dest`, but copies the unchanged file from DIR into the destination |
| `--link-dest <dir>` | Like `--copy-dest`, but hard-links the unchanged file from DIR (repeatable; earlier DIRs win) |
| `--delete` | Delete files on receiver not present in source (default timing: delete-after, i.e. only after the whole transfer succeeded). Scoped to the synchronized directories, so `--files-from` subsets are safe |
| `--delete-before` | Delete extras before the transfer starts (implies `--delete`) |
| `--delete-during`, `--del` | Delete extras once the keep-set is known, before data is applied (implies `--delete`) |
| `--delete-delay` | Delete extras only after a successful transfer (implies `--delete`) |
| `--delete-after` | Explicit delete-after timing (implies `--delete`) |
| `--exclude <pattern>` | Exclude files matching glob pattern (repeatable) |
| `--exclude-from <file>` | Read exclude patterns from a file (one per line) |
| `--include <pattern>` | Only transfer files matching glob pattern (repeatable, whitelist) |
| `--max-size <n>` | Skip files larger than n bytes |
| `--min-size <n>` | Skip files smaller than n bytes |
| `--max-alloc <SIZE>` | Maximum single allocation (binary units: B, K, M, G, T, P, E; default 1G) |
| `--incremental` | Skip files unchanged since last transfer (size + mtime). Auto-enables `--preserve`. Incompatible with `--chunk-serialization`. |
| `--existing` | Skip files not already present at the destination; update existing files normally. |
| `--bwlimit <KB/s>` | Bandwidth limit in kilobytes per second |
| `--chunk-size <n>` | Chunk size in bytes (default: 10485760) |
| `--timeout <sec>` | Positive I/O timeout in seconds, applied to both the socket (`SO_RCVTIMEO`/`SO_SNDTIMEO`, built-in default 30 s) and the per-message protocol poll deadline (built-in default 60 s). Omit the option to keep both built-ins; `0` is rejected. The server side keeps the built-in 60 s protocol window (the value is not sent on the wire). |
| `--contimeout <sec>` | Connection timeout in seconds (default: 10) |
| `--backup` | Backup existing destination files before overwriting |
| `--backup-dir <dir>` | Target directory for backups (requires `--backup`) |
| `--stats` | Print transfer statistics at end (bytes, files, timing) |
| `-h, --human-readable` | Format transfer byte sizes with binary units |
| `--delete-excluded` | Also delete filter-excluded destination mirrors (size-pruned mirrors stay protected) |
| `--max-delete <n>` | Delete at most n destination entries; the rest are skipped and the run exits 25 (partial), matching rsync |
| `--delay-updates` | Put updated files into place only at the end of the transfer (`--force` is honored at publication) |
| `-T, --temp-dir <dir>` | Scratch directory for temp files before the atomic install; confined to the receive root (relative only), with an `EXDEV` non-atomic copy fallback |
| `-n, --dry-run` | Report what would be transferred without mutating the destination. Since protocol 2.21.0 a server-routed target contacts the receiver and reports would-transfer based on receiver state; a plain local destination keeps the client-side scan. Never mutates or deletes. |
| `-v, --verbose` | Enable debug logging |
| `-q, --quiet` | Suppress non-error output |
| `--progress` | Show a periodic aggregate transfer line (bytes sent, current rate); not rsync's per-file progress block |
| `-P` | Enables partial-transfer mode + progress output; interrupted writes retain the already-written temp for resumption |
| `--stats` | Print transfer statistics at end (bytes, files, timing). Receiver-only counters (matched data, file-list bytes, deleted count) are reported as 0 |
| `-i, --itemize-changes` | Print an rsync-style per-file change line |
| `--out-format=FORMAT` | Output format for changed files (`%f %n %l %b %M %%`) |
| `--list-only` | List source files instead of transferring |
| `--fsync` | Fsync every written file before publication |
| `-h, --human-readable` | Format transfer byte/rate counts with rsync's decimal (base-1000) units |
| `--max-depth <n>` | Maximum directory depth to recurse (0 = unlimited, default: 0) |
| `--log-file <path>` | Write log messages to file instead of stderr |
| `--write-batch=FILE` | Run the normal live transfer and also emit a self-contained batch file of the source tree |
| `--only-write-batch=FILE` | Emit the batch file only (no destination, no server) |
| `--read-batch=FILE` | Apply a batch file to the destination (no source, no server) |
| `--source-dir <path>` | Source directory (overrides `FASTSYNC_SOURCE_DIR`) |
| `--dest-dir <path>` | Server destination directory (overrides `FASTSYNC_DEST_DIR`) |
| `--save-to-disk` | Write received files to disk |
| `--server-host <ip>` | Server IP address (default: `127.0.0.1`) |
| `--server-port <n>` | Server port (default: `8080`) |
| `--ssh-port <port>` | SSH port (default: 22) |
| `-e, --rsh <command>` | Remote shell to launch for the SSH transport (default: `ssh`; may include arguments, e.g. `-e "ssh -p 2222"`) |
| `-M, --remote-option=OPT` | Append OPT to the remote server invocation over SSH (repeatable) |
| `--address <ip>` | Bind the outgoing client socket to this source address |
| `-4, --ipv4` | Force IPv4 for destination resolution |
| `-6, --ipv6` | Force IPv6 for destination resolution |
| `--sockopts=OPTS` | Comma-separated OPT=VAL socket options applied before connect (`TCP_NODELAY`, `SO_KEEPALIVE`, `SO_RCVBUF`, `SO_SNDBUF`, `SO_REUSEADDR`) |
| `--bwlimit <KB/s>` | Bandwidth limit in kilobytes per second |
| `--chunk-size <n>` | Chunk size in bytes (default: 10485760) |
| `--timeout <sec>` | I/O timeout in seconds, applied to both the socket (`SO_RCVTIMEO`/`SO_SNDTIMEO`) and the per-message protocol poll deadline. Default `0` = disabled (matching rsync); `0` disables it. `--no-timeout` is the negation. The value is not sent on the wire; the server side keeps its own safe floor. |
| `--contimeout <sec>` | Connection timeout in seconds (default: 60, matching rsync); `0` disables it (`--no-contimeout` is the negation) |
| `--stop-after=MINS` | Stop the transfer after MINS minutes (a positive integer); whatever was already transferred is kept |
| `--stop-at=TIME` | Stop at an absolute time (`HH:MM`, `HH:MM:SS`, or `now+N[smhd]`); an early stop skips the late `--delete` keep-set |
| `-b, --backup` | Backup existing destination files before overwriting |
| `--backup-dir <dir>` | Target directory for backups (requires `--backup`) |
| `--tls` | Enable TLS encryption |
| `--cert <path>` | TLS certificate file (PEM) |
| `--key <path>` | TLS private key file (PEM) |
| `--ca <path>` | TLS CA certificate file for verification (PEM) |
| `--client-cn <name>` | TLS client certificate common name; mandatory with `--tls` (a TLS connection always verifies the client CN) |
The exhaustive rsync flag matrix is in [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md).
**Per-message vs. connection timeouts.** `--timeout` bounds each individual protocol
send/receive (the `poll()` deadline), so a peer that stops mid-frame is dropped. It
@@ -190,60 +285,64 @@ transfer is never aborted.
| `FASTSYNC_SOURCE_DIR` | — | Source directory fallback |
| `FASTSYNC_DEST_DIR` | — | Destination directory fallback |
| `FASTSYNC_SAVE_TO_DISK` | `false` | Disk persistence fallback |
| `FASTSYNC_SSH_PORT` | `22` | Default SSH port |
| `FASTSYNC_SERVER_HOST` | `127.0.0.1` | Default server host |
| `FASTSYNC_SERVER_PORT` | `8080` | Default server port |
| `FASTSYNC_TLS_CERT` | — | Default TLS certificate path |
| `FASTSYNC_TLS_KEY` | — | Default TLS private key path |
| `FASTSYNC_TLS_CA` | — | Default TLS CA certificate path |
## Implementation Details
### Data Structures
1. **Chunk** — collection of files (~10 MB total by default)
2. **File** — path, content (`Data`), optional `FileMetadata` pointer
3. **FileMetadata** — `mode`, `uid`, `gid`, `mtime_sec`, `mtime_nsec`;
uid / gid are advisory wire fields and are never applied by the receiver;
atime is unsupported
4. **Config** — runtime parameters (transported over wire, TLS settings excluded). Includes `timeout`, `contimeout`, `quiet`, `backup`, `backup_dir`, `stats`, `max_depth`, `log_file`.
5. **Queue** — thread-safe bounded queue with condition variables
6. **DirectoryScanner** — recursive BFS traversal with exclude and include pattern support, max-depth enforcement
1. **Chunk** — collection of files (~10 MB total by default).
2. **File** — path, content (`Data`), optional `FileMetadata` pointer.
3. **FileMetadata** — `mode`, `uid`, `gid`, `mtime_sec`, `mtime_nsec` (plus
atime/crtime fields). `uid`/`gid` are applied only through the opt-in
identity path; atime is preserved with `-U`/`--atimes`; crtime is captured
but cannot be set on the destination.
4. **Config** — runtime parameters. Most cross the wire (TLS settings
excluded); `backup` and `backup_dir` are in the serialized wire table, while
`timeout`, `contimeout`, `quiet`, `stats`, `max_depth`, and `log_file` are
client-only.
5. **Queue** — thread-safe bounded queue with condition variables.
6. **DirectoryScanner** — recursive BFS traversal with exclude and include
pattern support, max-depth enforcement.
### Key Algorithms
1. **File scanning** — BFS directory traversal;
entries matched against exclude and include patterns,
max - depth enforced 2. * *Chunking ** — files accumulated until `chunk_size` threshold,
then flushed 3. *
*Compression ** — streaming zstd
via `ZSTD_compressStream2` / `ZSTD_decompressStream` 4. *
*Network protocol ** — status -
code - driven exchange with metadata packing,
keep - alive,
and abort support 5. * *Incremental check ** — client sends `STATUS_CHECK` + path + size +
mtime and,
with `--checksum`, XXH64 content checksum; server compares against destination. Can be batched via `STATUS_CHECK_BATCH` for reduced round-trips.
6. **Bandwidth limiting** — token-bucket algorithm with `nanosleep` throttling on 64 KB write chunks
7. **Metadata restoration** — `chmod()`, `chown()`, `utimensat()` on the receiving side
8. **`--delete`** — sender tracks all sent paths;
receiver walks destination tree and removes unlisted files / directories 9. *
*SSH transport *
* — `socketpair()` + `fork()` + `execvp("ssh",
...)` with `ControlMaster` and port support
10. *
*TLS transport ** — OpenSSL `SSL_CTX` with TLS
1.2 minimum,
mutual CA verification,
transparent `SSL_read`/`SSL_write` via `io_set_ssl()` 11. *
*Path traversal protection ** — `has_path_traversal()` rejects any file path
containing `..` components,
preventing directory escape attacks 12. *
*Connection limiting ** — server tracks active connections and rejects
new ones beyond `max_connections` (default 100)13. *
*Keep
- alive ** — idle connections receive periodic `STATUS_KEEPALIVE` to detect half
- open TCP connections 14. * *Abort handling ** — `SIGINT` sets an abort flag; the next protocol operation sends `STATUS_ABORT` for clean server cleanup
15. **Atomic writes** — files are written to a `.tmp` suffix then atomically renamed via `rename()`, preventing partial files
16. **Backup** — before overwriting, existing files are moved to `--backup-dir` (or same directory with `~` suffix) preserving the original
1. **File scanning** — BFS directory traversal; entries matched against exclude
and include patterns, with max-depth enforced.
2. **Chunking** — files accumulated until the `chunk_size` threshold (default
10 MiB) is reached, then flushed.
3. **Compression** — streaming zstd via `ZSTD_compressStream2()` /
`ZSTD_decompressStream()`.
4. **Network protocol** — status-code-driven exchange with metadata packing,
keep-alive, and abort support.
5. **Incremental check** — the client sends `STATUS_CHECK` + path + size +
mtime and, with `--checksum`, a whole-file content checksum (xxHash64 by
default, or md5 via `--checksum-choice=md5`/`--cc`, seeded by
`--checksum-seed`); the server compares against the destination. Can be
batched via `STATUS_CHECK_BATCH` for reduced round-trips.
6. **Bandwidth limiting** — token-bucket algorithm with sleep throttling on
64 KiB write chunks.
7. **Metadata restoration** — mode via `chmod()`/`fchmod()`, times via
`utimensat()`/`futimens()`, and ownership only with an identity flag via
fd-relative `fchown()`/`fchownat()`.
8. **`--delete`** — the sender tracks all sent paths; the receiver walks the
destination tree and removes unlisted files and directories.
9. **SSH transport** — `socketpair()` + `fork()` + `execvp("ssh", ...)` with
`ControlMaster` and port support.
10. **TLS transport** — OpenSSL `SSL_CTX` with TLS 1.2 minimum, mutual CA
verification, and transparent `SSL_read()`/`SSL_write()` via
`io_set_ssl()`.
11. **Path traversal protection** — `has_path_traversal()` rejects any file
path containing `..` components, preventing directory escape attacks.
12. **Connection limiting** — the server tracks active connections and rejects
new ones beyond `max_connections` (default 100).
13. **Keep-alive** — idle connections receive periodic `STATUS_KEEPALIVE` to
detect half-open TCP connections.
14. **Abort handling** — `SIGINT` sets an abort flag; the next protocol
operation sends `STATUS_ABORT` for clean server cleanup.
15. **Atomic writes** — files are written to a `.tmp` suffix then atomically
renamed via `rename()`, preventing partial files.
16. **Backup** — before overwriting, existing files are moved to `--backup-dir`
(or the same directory with a `~` suffix), preserving the original.
## Security Features
@@ -300,7 +399,8 @@ cmake --build build -j$(nproc)
### SSH transfer
The remote host must have `fastsync-server` available in `PATH`, or use
The remote host must have `fastsync-server` available in `PATH` (install or
copy the built `./build/server` there as `fastsync-server`), or use
`--fastsync-server-path`. SSH starts `fastsync-server --stdio` in its remote
working directory, so use a destination below that directory unless the
remote server is otherwise configured with a matching authorized root.
@@ -341,8 +441,13 @@ Plain TCP requires the explicit `--allow-unauthenticated` server option. Use TLS
authenticated network connections.
### TLS transfer
Server TLS requires `--cert`, `--key`, `--ca`, and `--client-cn`; the client
requires `--cert`, `--key`, and `--ca`.
```bash
./build/server --destination-root /path/to --tls --cert server.pem --key server-key.pem -p 8443
./build/server --destination-root /path/to --tls --cert server.pem --key server-key.pem \
--ca ca.pem --client-cn client -p 8443
./build/client --tls --cert client.pem --key client-key.pem --ca ca.pem \
--server-host example.com --server-port 8443 \
--source-dir /path/to/source --dest-dir /path/to/destination \
@@ -355,32 +460,32 @@ These examples show the intended rsync-style workflow. Options marked as
FastSync-native are optional performance or transport extensions.
```bash
#Basic synchronization
# Basic synchronization
./build/client /source/ /destination/
#Archive - style synchronization(current FastSync archive behavior)
# Archive-style synchronization (current FastSync archive behavior)
./build/client -a /source/ user@host:destination/
#Preview a transfer without changing the destination
# Preview a transfer without changing the destination
./build/client -n /source/ /destination/
#Exclude temporary and object files
# Exclude temporary and object files
./build/client --exclude '*.tmp' --exclude '*.o' \
/source/ user@host:destination/
#Remove destination entries not present in the source
# Remove destination entries not present in the source
./build/client --delete /source/ user@host:destination/
#Skip unchanged files using size and modification time
# Skip unchanged files using size and modification time
./build/client --incremental /source/ user@host:destination/
#Verify content when size and time are not sufficient
# Verify content when size and time are not sufficient
./build/client --incremental --checksum /source/ user@host:destination/
#Preserve supported mode and timestamp metadata
# Preserve supported mode and timestamp metadata
./build/client --preserve /source/ user@host:destination/
#Keep backups of overwritten destination files
# Keep backups of overwritten destination files
./build/client --backup --backup-dir backups \
/source/ user@host:destination/
```
@@ -395,9 +500,9 @@ features without changing the meaning of ordinary compatibility options.
| `-j`, `--threads[=N]` | Enable the multithreaded scanner/loader/sender pipeline. `N` (1–256) sets the parallel scanner worker count; bare `-j`/`--threads` uses the default. |
| `-z [level]`, `--compress [level]` | Enable streaming zstd compression, levels 1-22. |
| `--compress-level <n>` | Set the zstd compression level. |
| `--zc <alg>` | Alias for `--compress-choice`. FastSync supports `zstd` and `none`. |
| `--zc <alg>` | Alias for `--compress-choice`. FastSync supports `zstd`, `none`, and `auto`; `lz4`/`zlib`/`zlibx` are rejected by name. |
| `--zl <n>` | Alias for `--compress-level`. |
| `--skip-compress <list>` | Skip compression for comma-separated suffixes; incompatible with `--chunk-serialization`. |
| `--skip-compress <list>` | Skip compression for `/`- or `,`-separated suffixes; defaults to rsync 3.4.1's built-in list. Incompatible with `--chunk-serialization`. |
| `--compress-threads <n>` | Use `n` zstd compression workers. Requires compression and a zstd build with threaded support; the setting affects sender CPU work only. |
| `--chunk-size <bytes>` | Set the transfer chunk size. |
| `--chunk-serialization` | Enable FastSync chunk serialization (long form only; `-s` is rsync's `--secluded-args`). |
@@ -409,10 +514,10 @@ features without changing the meaning of ordinary compatibility options.
| `--server-port <port>` | Select the TCP server port (`--port <port>` and `--port=<port>` are rsync-friendly aliases). |
| `--tls` | Enable TLS for TCP transport. |
| `--bwlimit <KB/s>` | Apply token-bucket bandwidth limiting. |
| `--progress` | Show transfer progress and throughput. |
| `--stats` | Print transfer statistics. |
| `--timeout <seconds>` | Set the socket **and** per-message protocol I/O timeout (positive seconds). Omit to keep the built-in 30 s socket / 60 s protocol defaults. |
| `--contimeout <seconds>` | Set connection timeout. |
| `--progress` | Show a periodic aggregate transfer line (throughput; not a per-file block). |
| `--stats` | Print transfer statistics (receiver-only counters are 0). |
| `--timeout <seconds>` | Set the socket **and** per-message protocol I/O timeout. Default `0` = disabled (matching rsync); `0` disables it. |
| `--contimeout <seconds>` | Connection timeout (default 60, matching rsync); `0` disables it. |
Short-option conflicts with rsync have been resolved for the CLI namespace
(Phase 7): `-c` is now rsync's `--checksum`, `-m` is `--prune-empty-dirs`, `-M`
@@ -421,7 +526,8 @@ is `--remote-option`, `-f` is `--filter`, `-s` is `--secluded-args`, `-p` is
long-form-only or new shorts: multithreading is `-j`/`--threads`, metadata
is `--preserve`, sendfile is `--sendfile`, chunk serialization is
`--chunk-serialization`, timeout is `--timeout`, and SSH port is `--ssh-port`.
`-a`/`--archive` is now real rsync archive (`-rlptgoD`).
`-a`/`--archive` is now rsync archive `-rlptgoD` (owner/group implied, but the
receiver still needs privilege to apply them).
`--secluded-args` (and its short form `-s`) is accepted as a compatibility
no-op. It does not change FastSync's transport or protocol behavior, because
@@ -433,42 +539,93 @@ remote SSH argv is already built injection-safe.
| Option | Description |
|---|---|
| `-a`, `--archive` | rsync archive mode (`-rlptgoD`): links, metadata, devices and specials. |
| `-n`, `--dry-run` | Scan and report without writing files. |
| `--delete` | Request removal of destination entries absent from the source. The server must allow deletion. Default timing is delete-after: extras are removed only after the whole transfer succeeded. |
| `-a`, `--archive` | rsync archive mode (`-rlptgoD`): links, perms, times, owner, group, devices and specials; ownership application stays privilege-gated. |
| `-n`, `--dry-run` | Report what would be transferred without mutating the destination. Since protocol 2.21.0 a server-routed target contacts the receiver and reports would-transfer based on receiver state; a plain local destination keeps the client-side scan. Never mutates or deletes. |
| `--remove-source-files` | Remove regular source files after a successful transfer. |
| `--incremental` | Skip files matching destination size and mtime. Auto-enables `--preserve`. Incompatible with `--chunk-serialization`. |
| `-c, --checksum` | Verify content by checksum (implies the incremental quick-check). Algorithm selectable with `--checksum-choice`. |
| `--checksum-choice <alg>` | Whole-file checksum algorithm: `xxh64`/`xxhash` (default), `xxh3`, `xxh128`, `md5`, or `auto`. |
| `--checksum-seed <n>` | Seed for the whole-file xxHash digest; an unset/`0` seed is randomized per transfer, matching rsync. |
| `--size-only` | Skip incremental files matching in size, ignoring mtime. |
| `-I, --ignore-times` | Transfer files even when size and mtime match. |
| `-u, --update` | Skip files newer than the source on the receiver. |
| `-W, --whole-file` | Transfer changed files without delta processing (`--no-whole-file` clears it). |
| `-B <n>, --block-size <n>` | Delta block size in bytes (alias `--delta-block`). |
| `-d, --dirs` | Transfer the named directory entries without recursing into their contents (aliases `--old-dirs`/`--old-d`). |
| `-R, --relative` | With `--files-from`, preserve each listed entry's relative path below the destination root. |
| `--files-from <file>` | Read the source file list from FILE (paths relative to the source root). |
| `--delay-updates` | Put updated files into place only at the end of the transfer. |
| `--compare-dest <dir>` | Extra comparison basis: unchanged files are not transferred (requires/implies `--incremental`). |
| `--copy-dest <dir>` | Like `--compare-dest`, but copies the unchanged file from DIR into the destination. |
| `--link-dest <dir>` | Like `--copy-dest`, but hard-links the unchanged file from DIR (repeatable; earlier DIRs win). |
| `--preallocate` | Allocate destination file space up front (fail-fast on a full disk). |
| `--append` | Resume a shorter destination by appending only its tail (prefix not verified; requires `--incremental`). |
| `--append-verify` | Like `--append`, but verifies the retained prefix checksum first (falls back to a full transfer on mismatch). |
| `--delete` | Request removal of destination entries absent from the source. The server must allow deletion. Default timing is delete-after: extras are removed only after the whole transfer succeeded. Scoped to the synchronized directories, so `--files-from` subsets are safe. |
| `--delete-before` | Delete extras before the transfer starts (implies `--delete`). |
| `--delete-during`, `--del` | Delete extras once the keep-set manifest is known, before data is applied (implies `--delete`; early mode, same engine behaviour as `--delete-before`). |
| `--delete-delay` | Delete extras only after a successful transfer (implies `--delete`; commit mode, same behaviour as `--delete-after`). |
| `--delete-after` | Explicit delete-after timing: delete only after the transfer succeeded (implies `--delete`). |
| `--delete-excluded` | Also delete filter-excluded destination mirrors (size-pruned mirrors stay protected). |
| `--max-delete <n>` | Delete at most n destination entries; the rest are skipped and the run exits 25 (partial), matching rsync. |
| `--force` | Allow an incoming file/symlink to replace a destination directory (also during `--delay-updates` publication). |
| `--exclude <pattern>` | Exclude matching paths. Repeatable. |
| `--include <pattern>` | Include matching paths. Repeatable. |
| `--exclude-from <file>` | Read exclude patterns from a file. |
| `--include-from <file>` | Read include patterns from a file. |
| `--max-size <bytes>` | Skip files larger than the limit. |
| `--min-size <bytes>` | Skip files smaller than the limit. |
| `--max-depth <n>` | Limit recursive scanning depth;
zero means unlimited.| | `--incremental` | Skip files matching destination size and mtime.|
| `--checksum` | Include xxHash64 content checks in incremental comparisons.| | `--backup` |
Back up overwritten files.| | `--backup - dir<dir>` | Store backups under a separate directory.|
| `--suffix<suffix>` | Set the backup filename suffix.| | `--partial` |
Select partial - transfer handling. On failed/interrupted writes the
already-written temp file is retained (best-effort) for resumption.|
With `--partial --partial-dir <dir>`, completed files are written under the
partial directory and installed atomically. | | `--partial - dir<dir>` |
Set a relative partial - transfer directory below the server destination root.
Use with `--partial`. |
| `--max-alloc <SIZE>` | Maximum single allocation (binary units; default 1G; `0` = no local limit). |
| `--max-depth <n>` | Limit recursive scanning depth; zero means unlimited. |
| `-b, --backup` | Back up overwritten files. |
| `-T, --temp-dir <dir>` | Scratch directory for temp files before the atomic install (confined to the receive root; `EXDEV` falls back to a non-atomic copy). |
| `--backup-dir <dir>` | Store backups under a separate directory (requires `--backup`). |
| `--suffix <suffix>` | Set the backup filename suffix (default: `~`). |
| `--partial` | Select partial-transfer handling. On failed/interrupted writes the already-written temp file is retained (best-effort) for resumption. With `--partial --partial-dir <dir>`, completed files are written under the partial directory and installed atomically. |
| `--partial-dir <dir>` | Set a relative partial-transfer directory below the server destination root. Use with `--partial`. |
| `--inplace` | Write directly to the destination instead of using a temporary file. |
| `--fsync` | Fsync every written file before publication. |
| `--write-batch=FILE` | Run the normal live transfer and also emit a self-contained batch file of the source tree. |
| `--only-write-batch=FILE` | Emit the batch file only (no destination, no server). |
| `--read-batch=FILE` | Apply a batch file to the destination (no source, no server). |
| `--stop-after=MINS` | Stop the transfer after MINS minutes; whatever was already transferred is kept. |
| `--stop-at=TIME` | Stop at an absolute time (`HH:MM`, `HH:MM:SS`, or `now+N[smhd]`). An early stop skips the late `--delete` keep-set. |
### Metadata and links
| Option | Description |
|---|---|
| `--preserve` | Preserve supported file metadata, currently mode and modification time (long form only). |
| `-l`, `--links` | Request symlink preservation;
link-target transfer remains incomplete. |
| `--copy-links` | Copy symlink referents. |
| `--safe-links` | Skip symlinks that point outside the transfer tree. |
| `--preserve` | Preserve mode and mtime (long form only; equivalent to `-p` + `-t`). Add `-o`/`-g` for owner/group, `-U`/`--atimes` for atime, or an identity flag (`--chown`/`--usermap`/`--groupmap`/`--numeric-ids`/`--copy-as`) for mapped ownership. |
| `-U`, `--atimes` | Preserve access times. Captured with the metadata payload; does not enable ownership. |
| `-N`, `--crtimes` | Capture birth time and transmit it; it cannot be applied because no portable filesystem call can set a birth time (documented divergence). |
| `-p`, `--perms` | Preserve permission bits. One of the four per-attribute preserve flags (with `-t`/`-o`/`-g`); under `-p` the source mode is copied exactly (setuid/setgid/sticky and group/other-write included), matching rsync. |
| `-t`, `--times` | Preserve modification times. Independent of the other attributes; `-O`/`--omit-dir-times` suppresses directories only. |
| `-o`, `--owner` | Preserve the source owner (uid). Mapped by name on the receiver with a raw-numeric fallback (only numeric ids cross the wire); application is privilege-gated. |
| `-g`, `--group` | Preserve the source group (gid). Same name-mapping/numeric-fallback and privilege gating as `-o`. |
| `--no-perms`, `--no-times`, `--no-owner`, `--no-group` | Negate each per-attribute flag (also `--no-p`/`--no-t`/`--no-o`/`--no-g`); `--no-preserve` clears all four. |
| `-E`, `--executability` | Preserve executable permission bits. |
| `-X`, `--xattrs` | Preserve user `user.*` extended attributes. |
| `-A`, `--acls` | Preserve POSIX ACLs. |
| `--chmod <changes>` | Modify transferred permissions (rsync syntax, including `D`/`F`/`X` selectors and `s`/`t`); does not imply `-p`. |
| `--chown=USER:GROUP` | Override the ownership of transferred files (`USER:GROUP`, `USER`, or `:GROUP`); conflicts with `--usermap`/`--groupmap` on the same side. |
| `--usermap=MAP` | Map usernames when applying ownership (`FROM:TO` rules; names, ids, `LOW-HIGH` ranges, `*`, empty-`FROM`). |
| `--groupmap=MAP` | Map group names when applying ownership (same syntax as `--usermap`). |
| `--numeric-ids` | Mapping modifier: apply the source numeric uid/gid directly instead of mapping by name (combine with `-o`/`-g`, `-a`, or a map). |
| `--copy-as=USER[:GROUP]` | Force every written entry to USER[:GROUP]; requires a privileged receiver. |
| `--fake-super` | Record the resolved owner plus mode/time in a reserved `user.fastsync.stat` xattr and replay mode/time; never performs a real chown. |
| `--super` | Permit the receiver to attempt confined super-user activities (device nodes). |
| `--no-super` | Forbid those super-user activities even when the receiver is root. |
| `-l`, `--links` | Copy symlinks as symlinks; the target is stored verbatim (absolute and `..`-bearing targets included), matching rsync. |
| `-L`, `--copy-links` | Copy symlink referents (a broken referent exits 0). |
| `--safe-links` | Skip symlinks whose target points outside the transfer tree (applied on the sender). |
| `--copy-unsafe-links` | Copy unsafe symlink referents. |
| `--munge-links` | Rewrite stored symlink targets with rsync's `/rsyncd-munged/` marker. |
| `-k`, `--copy-dirlinks` | Treat a symlink to a directory as a real directory on the sender. |
| `-K`, `--keep-dirlinks` | Follow an existing destination symlink-to-directory (confined to the receive root). |
| `-H`, `--hard-links` | Preserve hard-link relationships across the transfer. |
| `-D` | Preserve device and special files (implies `--devices --specials`). |
| `--devices` | Recreate device nodes on the destination (privileged; skipped without `CAP_MKNOD`). |
| `--specials` | Recreate special files: FIFOs and unix sockets. |
| `-S`, `--sparse` | Sparse-file handling: receiver preserves holes (zero runs are written as holes; no wire change). |
### Output and logging
@@ -476,8 +633,12 @@ link-target transfer remains incomplete. |
| Option | Description |
|---|---|
| `-v`, `--verbose` | Enable debug logging. |
| `--progress` | Show live transfer progress. |
| `--stats` | Print transfer statistics. |
| `-q`, `--quiet` | Suppress non-error output. |
| `--progress` | Show a periodic aggregate transfer line (not a per-file block). |
| `--stats` | Print transfer statistics (receiver-only counters are reported as 0). |
| `-i`, `--itemize-changes` | Print an rsync-style per-file change line. |
| `--out-format=FORMAT` | Output format for changed files (`%f %n %l %b %M %%`). |
| `--list-only` | List source files instead of transferring. |
| `--log-file <path>` | Write log output to a file. |
| `-V`, `--version` | Print the FastSync protocol version. |
| `--help` | Print command usage. |
@@ -487,31 +648,56 @@ link-target transfer remains incomplete. |
| Option | Description |
|---|---|
| `--ssh-port <port>` | SSH port for the SSH transport (default: 22). Note the short `-p` is now rsync's `--perms`. |
| `--fastsync-server-path <path>` | Remote FastSync server path for SSH mode. |
| `-e`, `--rsh <command>` | Remote shell to launch for the SSH transport (default: `ssh`; may include arguments). |
| `--fastsync-server-path <path>` | Remote FastSync server path for SSH mode (client-only; never crosses the wire). |
| `--rsync-path <path>` | Alias for `--fastsync-server-path`. |
| `-M`, `--remote-option=OPT` | Append OPT to the remote server invocation over SSH (repeatable; rejected for daemon/TCP destinations). |
| `--trust-sender` | Receiver-local: trust the remote sender's file list and skip path re-validation (does not affect symlink targets). |
| `--timeout <sec>` | Socket + per-message I/O timeout; default `0` = disabled. |
| `--contimeout <sec>` | Connection timeout; default 60; `0` disables. |
| `--source-dir <path>` | Set the source directory explicitly. |
| `--dest-dir <path>` | Set the destination directory explicitly. |
| `--save-to-disk` | Enable server-side disk persistence. |
| `--server-host <host>` | TCP server address. |
| `--server-port <port>` | TCP server port. `--port <port>` / `--port=<port>` is an alias. |
| `--tls` | Enable TLS. Requires `--cert` and `--key`. |
| `--address <ip>` | Bind the outgoing client socket to this source address. |
| `-4`, `--ipv4` | Force IPv4 for destination resolution. |
| `-6`, `--ipv6` | Force IPv6 for destination resolution. |
| `--sockopts=OPTS` | Comma-separated OPT=VAL socket options applied before connect. |
| `--tls` | Enable TLS. Requires `--cert`, `--key`, and `--ca`. |
| `--cert <path>` | TLS certificate file. |
| `--key <path>` | TLS private key file. |
| `--ca <path>` | CA file for peer verification. |
| `--ca <path>` | CA file for peer verification (always required with `--tls`). |
## Server Options
| Option | Description |
|---|---|
| `--stdio` | Serve one SSH connection over standard input/output. |
| `-p <port>` | TCP listen port. |
| `--daemon` | Run as a persistent daemon listener using a module config file; the daemon default port is 873 (unlike `-p`, which defaults to 8080). |
| `--config=FILE` | Daemon config file (default: `~/.config/fastsync/fastsyncd.conf`, else `/etc/fastsyncd.conf`). Requires `--daemon`. |
| `--dparam=KEY=VALUE` | Override one global config key on the command line. Requires `--daemon`. |
| `--no-detach` | Stay in the foreground (default detaches to the background when running `--daemon`). |
| `-p, --port <port>` | TCP listen port (default: 8080, range: 1–65535). |
| `--tls` | Enable TLS. |
| `--cert <path>` | TLS certificate file. |
| `--key <path>` | TLS private key file. |
| `--ca <path>` | CA file for peer verification. |
| `--destination-root <path>` | Confine received files to this server-side root;
defaults to the current directory. |
| `--cert <path>` | TLS certificate file (PEM). |
| `--key <path>` | TLS private key file (PEM). |
| `--ca <path>` | CA file for peer verification (PEM). |
| `--client-cn <name>` | TLS client certificate CN; mandatory with `--tls` (the server verifies the client CN). |
| `--destination-root <path>` | Confine received files to this server-side root; defaults to the current directory. |
| `--address <addr>` | Bind the listening socket to this address. |
| `-4`, `--ipv4` | Bind an IPv4 socket (default). |
| `-6`, `--ipv6` | Bind an IPv6 socket. |
| `--allow-delete` | Permit client delete manifests. Deletion is refused by default. This also gates `--force` (which can recursively replace/remove a destination directory tree). |
| `--allow-super` | Standalone TCP listener only: keep super-user activities enabled for a **root** receiver. Without it a root standalone server forces `SUPER_MODE_OFF`, so client `--devices`/`--write-devices`/`--super` and client-chosen ownership requests are skipped/refused. Rejected with `--stdio` (the SSH remote argv is client-composed; use a forced command if the default must hold). No effect when not root. Daemon modules opt in per module with `client owner = yes`. |
| `--trust-sender` | Trust the remote sender's file list: skip the receiver's up-front path-traversal re-validation (fewer checks, faster, potentially unsafe; off by default). It does not affect symlink targets, which are stored verbatim either way. |
| `--no-super` | Operator veto: never attempt super-user activities (ownership, device nodes) even as root, and refuse any client `--copy-as`/`--super` request. |
| `--allow-unauthenticated` | Permit plaintext/anonymous network clients; an auth-required module still accepts only opted-in loopback plaintext. |
| `--iconv=LOCAL[,REMOTE]` | Declare this server's LOCAL charset for file-name conversion. |
| `--password-file=FILE` | Credential store for modules that declare `auth users`. Requires `--daemon`. |
| `--early-input=FILE` | Second credential store layered over `--password-file`. Requires `--daemon`. |
| `--hash-credentials <file>` | Read `<file>`'s `user:password` lines and print PBKDF2 credential-store lines to stdout, then exit. Cannot be combined with `--daemon` or `--stdio`. |
| `--iterations N` | PBKDF2 iteration count for `--hash-credentials` (default 600000, range 100000–10000000). Requires `--hash-credentials`. |
| `-v`, `--verbose` | Enable debug logging. |
| `--help` | Print server usage. |
@@ -540,8 +726,9 @@ and `address`, the global section accepts:
- `hosts allow` / `hosts deny` — comma- and/or whitespace-separated host access
patterns.
A `[module]` may also set `max connections` (0 = unlimited; enforced per module
across all connection children) and its own `hosts allow`/`hosts deny`.
A `[module]` requires `path`, and may also set `read only`, `client owner`,
`auth users`, `max connections` (0 = unlimited; enforced per module across all
connection children), and its own `hosts allow`/`hosts deny`.
The per-host cap and the shared auth lockout identify a source by its numeric
peer IP. **Loopback peers (127.0.0.0/8, IPv6 `::1`) are exempt**: every local
@@ -595,7 +782,7 @@ before the module list, before authentication, and the connecting peer address
## Protocol and Security
FastSync protocol version `2.21.0` is shared by the client and server. The
FastSync protocol version `2.23.0` is shared by the client and server. The
current protocol is sender-driven and includes configuration negotiation,
including the maximum allocation limit, incremental checks, checksums,
manifests, keep-alives, abort handling, per-file remove-source results, and
@@ -647,10 +834,9 @@ mandates `--client-cn`, so a TLS connection to an auth-required module always
has its client CN verified (`--client-cn` matches the certificate's CN only, not
a subjectAltName, which is acceptable for a private CA).
TLS provides encrypted TCP transport. Supplying `--ca` enables certificate
verification; without it, traffic is encrypted but peer identity is not
verified. Use certificate verification for deployments where authentication
matters. The default TCP transport is not encrypted.
TLS provides encrypted TCP transport. Both the client and the server require
`--ca` together with `--tls`, so peer certificates are always verified
(`SSL_VERIFY_PEER`, depth 4). The default TCP transport is not encrypted.
The receiver protects its destination root with path validation, `openat()`
directory traversal, `O_NOFOLLOW`, temporary files, and atomic renames. Delete
@@ -661,18 +847,24 @@ operations require the server's explicit `--allow-delete` policy.
The project will reach the drop-in replacement goal in stages:
1. Correct rsync option meanings, including short options, combined options,
and `--option=value` syntax.
and `--option=value` syntax — **done** in the rsync-parity wave: `-r`/`-b`/
`-L`/`-B`, short-option clustering (`-av`, `-aAX`, `-rlpt`), and attached
values (`-B1000`, `-essh`, `-MOPT`) all parse.
2. Add differential tests that compare FastSync and rsync contents, metadata,
links, deletes, filters, dry runs, and exit codes.
3. Make `-a` implement the expected recursive, links, permissions, times,
owner/group, and supported special-file behavior.
4. Complete symlink, sparse-file, metadata, delete-policy, and resumable-write
semantics.
3. `-a` implements full rsync `-rlptgoD`; under `-p` the source mode is copied
exactly (no masking). Ownership application stays privilege-gated, as in
rsync.
4. Symlink (verbatim storage), sparse-file, metadata, delete-policy (including
`--max-delete` partial + exit 25), and resumable-write semantics are
implemented; remaining work is the documented edge cases, which the
**Rsync-Parity Wave** section of `RSYNC_COMPAT.md` enumerates honestly.
5. Add rsync remote-shell and daemon protocol interoperability.
6. Keep FastSync performance options as negotiated, optional extensions.
The exhaustive implementation matrix and compatibility notes are in
[`RSYNC_COMPAT.md`](RSYNC_COMPAT.md).
[`RSYNC_COMPAT.md`](RSYNC_COMPAT.md); each row is classified as parity, caveat,
or divergent.
## Testing
@@ -709,10 +901,13 @@ rsync protocol or filesystem-semantic compatibility.
## Performance Guidance
- Use `-m` for workloads with many files or enough CPU parallelism.
- Use `-c` or `-z` when network bandwidth is more constrained than CPU.
- Use `-j`/`--threads` for workloads with many files or enough CPU parallelism
(`-m` is `--prune-empty-dirs`).
- Use `-z` when network bandwidth is more constrained than CPU (`-c` is
`--checksum`, not a bandwidth option).
- Tune `--chunk-size` for file sizes, memory limits, and network latency.
- Use `-f` for large uncompressed TCP transfers where zero-copy I/O helps.
- Use `--sendfile` for large uncompressed TCP transfers where zero-copy I/O
helps (`-f` is `--filter`).
- Use `--incremental` to avoid retransmitting unchanged files.
- Use `--delta` for changed files when both endpoints are FastSync peers.
- Use `--bwlimit` when sharing a link with other traffic.
+435 -265
View File
@@ -6,13 +6,34 @@ This document maps rsync's full feature set to FastSync's current implementation
| Status | Count | Description |
|--------|-------|-------------|
| ✅ Implemented | 143 | Feature works end-to-end |
| 🔀 Alt Arg | 0 | Functionality exists but under different flag/semantics |
| ⛔ Impossible/Divergence | 4 | Flag is a documented divergence or cannot be implemented on any portable filesystem call |
| ⚠️ Partial | 0 | Flag parsed/stored but behavior incomplete |
| 🔄 Compatibility No-op | 0 | Flag is accepted for CLI compatibility but has no effect |
| ❌ Not Implemented | 0 | Flag not recognized or no behavior |
| **Total** | **147** | |
| ✅ Parity | 83 | Reproduces rsync's semantics for this option's scope |
| ⚠️ Caveat | 63 | Fully wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
| ❌ Divergent | 4 | Rejected, an accepted no-op, or impossible on any portable filesystem call |
| **Total** | **150** | One row per rsync option/feature group; a row may name several spellings |
This matrix reports honest rsync parity, not "implemented" as a synonym for
"parsed". A ✅ row matches rsync for the option's scope. A ⚠️ row is real and
tested but diverges in at least one documented way — FastSync's push-only model,
its own wire protocol, the delete timings that approximate rsync's engine modes,
the safe-subset privilege model (`--super`/`--copy-as`), the stricter
xattr/ACL and temp-dir policies, and the output counters that rsync computes on
the generator side. An ❌ row is either rejected (`--stderr=client`, `--protocol`
with any value but the current one), an accepted no-op (`-s`/`--secluded-args`),
or impossible (`-N`/`--crtimes`). The counts are derived from the rows below;
update them together with the table.
**Recently closed parity gaps (protocol 2.23.0).** The rsync-parity wave wired up
the short options `-r`, `-b`, `-L`, `-B`; rsync short-option clustering
(`-av`, `-aAX`, `-rlpt`) and attached/inline values (`--opt=value`, `-B1000`,
`-essh`, `-MOPT`); `-c` now implies the checksum quick-check; `--checksum-choice`
accepts `xxh64`/`xxhash`/`xxh3`/`xxh128`/`md5`/`auto` and rejects `md4`/`sha1`/
`none` by name; `--compress-choice` accepts `zstd`/`none`/`auto`; `--checksum-seed=0`
is randomized per transfer; `--skip-compress` uses rsync's default suffix list;
`--timeout`/`--contimeout` match rsync's defaults; deletion gained
`--max-delete` partial semantics with exit 25; symlinks are stored verbatim; and
`--specials` recreates sockets. Every one of those still has an entry below with
its remaining caveats. See the **Rsync-Parity Wave (protocol 2.23.0)** section
near the end for the full list and the known limitations.
---
@@ -20,100 +41,100 @@ This document maps rsync's full feature set to FastSync's current implementation
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-a`, `--archive` | Archive mode is -rlptgoD | ✅ Implemented | Phase 7 Wave A: real rsync archive. `-a`/`--archive` now implies `--links` + metadata (perms/times/group/owner as FastSync's broad bundle) + `--devices` + `--specials`. FastSync is always recursive, so no `-r` is needed. It no longer implies compression or multithreading (those moved to `-z`/`-j`). The short-option namespace is now rsync-parity (see the Phase 7 note) |
| `-v`, `--verbose` | Increase verbosity | ✅ Implemented | Sets `log_level=DEBUG` |
| `-q`, `--quiet` | Suppress non-error messages | ✅ Implemented | Suppresses client output while preserving errors |
| `--help` | Show help | ✅ Implemented | Prints usage and exits; `-h` is not accepted |
| `-V`, `--version` | Print version | ✅ Implemented | |
| `--info=FLAGS` | Fine-grained info verbosity | ✅ Implemented | Supports `copy`, `misc`, `skip`, `stats`, `all`, and `none`; explicit flags override `--verbose`, and `none` suppresses info output; unsupported names are rejected |
| `--debug=FLAGS` | Fine-grained debug verbosity | ✅ Implemented | `io`, `proto`, `pack`, and `util` are supported; `--debug=help` lists flags; other rsync categories are rejected |
| `--stderr=MODE` | Change stderr output mode | ⛔ Impossible/Divergence | `errors` (default) and `all` are supported; `client` is rejected with a clear error (`--stderr=client is not supported`) because FastSync has no rsync client-message channel — the rejection itself is the documented behavior (Phase 7 Wave B decision). The modes that exist work; the missing rsync channel cannot be emulated without a wire change |
| `--no-motd` | Suppress daemon MOTD | ✅ Implemented | Client-only display switch (Wave C): the daemon still sends the configured `motd file` on a `host::module/path` connection; the client reads and discards the frame without showing it. Without the flag the MOTD is printed to stdout after the config/auth handshake and escaped so control bytes cannot inject terminal sequences |
| `--exclude=PATTERN` | Exclude files matching pattern | ✅ Implemented | Glob matching in scanner |
| `--include=PATTERN` | Include files matching pattern | ✅ Implemented | Glob matching in scanner |
| `-C`, `--cvs-exclude` | Auto-ignore CVS files | ✅ Implemented | Applies the well-known rsync default exclude set as exclude rules during scanning (RCS SCCS CVS CVS.adm RCSLOG cvslog.* tags TAGS .make.state .nse_depinfo *~ #* .#* ,* _$* *$ *.old *.bak *.BAK *.orig *.rej .del-* *.a *.olb *.o *.obj *.so *.exe *.Z *.elc *.ln core .svn/ .git/ .hg/ .bzr/); `.git/`-style repo dirs are pruned without descending |
| `-a`, `--archive` | Archive mode is -rlptgoD (rsync includes owner/group) | ✅ Parity | Phase 7 Wave A: real rsync archive. `-a`/`--archive` now implies `--links` + the four per-attribute preserve flags (perms/times/owner/group) + `--devices` + `--specials`, i.e. **`-rlptgoD`**. Owner/group **are** implied, but their application stays privilege-gated exactly like rsync: a receiver that cannot `chown` logs a warning and skips it (see the preserve-attribute split note below). FastSync is always recursive, so no `-r` is needed. It no longer implies compression or multithreading (those moved to `-z`/`-j`). The short-option namespace is now rsync-parity (see the Phase 7 note) |
| `-v`, `--verbose` | Increase verbosity | ✅ Parity | Sets `log_level=DEBUG` |
| `-q`, `--quiet` | Suppress non-error messages | ✅ Parity | Suppresses client output while preserving errors |
| `--help` | Show help | ✅ Parity | Prints usage and exits; `-h` is not accepted |
| `-V`, `--version` | Print version | ✅ Parity | |
| `--info=FLAGS` | Fine-grained info verbosity | ⚠️ Caveat | Supports `copy`, `misc`, `skip`, `stats`, `all`, and `none`; explicit flags override `--verbose`, and `none` suppresses info output; unsupported names are rejected |
| `--debug=FLAGS` | Fine-grained debug verbosity | ⚠️ Caveat | `io`, `proto`, `pack`, and `util` are supported; `--debug=help` lists flags; other rsync categories are rejected |
| `--stderr=MODE` | Change stderr output mode | ❌ Divergent | `errors` (default) and `all` are supported; `client` is rejected with a clear error (`--stderr=client is not supported`) because FastSync has no rsync client-message channel — the rejection itself is the documented behavior (Phase 7 Wave B decision). The modes that exist work; the missing rsync channel cannot be emulated without a wire change |
| `--no-motd` | Suppress daemon MOTD | ✅ Parity | Client-only display switch (Wave C): the daemon still sends the configured `motd file` on a `host::module/path` connection; the client reads and discards the frame without showing it. Without the flag the MOTD is printed to stdout after the config/auth handshake and escaped so control bytes cannot inject terminal sequences |
| `--exclude=PATTERN` | Exclude files matching pattern | ✅ Parity | Glob matching in scanner |
| `--include=PATTERN` | Include files matching pattern | ✅ Parity | Glob matching in scanner |
| `-C`, `--cvs-exclude` | Auto-ignore CVS files | ✅ Parity | Applies the well-known rsync default exclude set as exclude rules during scanning (RCS SCCS CVS CVS.adm RCSLOG cvslog.* tags TAGS .make.state .nse_depinfo *~ #* .#* ,* _$* *$ *.old *.bak *.BAK *.orig *.rej .del-* *.a *.olb *.o *.obj *.so *.exe *.Z *.elc *.ln core .svn/ .git/ .hg/ .bzr/); `.git/`-style repo dirs are pruned without descending |
## 2. Modifying Output
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `--stats` | Give transfer stats | ✅ Implemented | Prints file/byte counts |
| `-h`, `--human-readable` | Human-readable numbers | ✅ Implemented | Formats transfer byte sizes using binary units |
| `-i`, `--itemize-changes` | Per-file change summary | ✅ Implemented | Prints rsync-style `>f+++++++++` lines to stdout only for files actually sent (also under `-j`/`--threads`); unchanged files print nothing, matching single-`-i` behavior |
| `--progress` | Show progress | ✅ Implemented | Progress callback in sender |
| `-P` | Same as --partial --progress | ✅ Implemented | Phase 7 Wave B: `-P` parses to `--partial` + `--progress`. On a failed/interrupted write the receiver now retains the already-written temp file at the destination path (best-effort rename instead of unlink when configured), so a later `--append`/`--append-verify` run can resume it; `--partial-dir` still stages completed files under the confined partial dir and installs them atomically. The retention never runs when `--partial` is off, when no data was actually written, or under `--ignore-existing`/`--existing` (the destination is not ours to overwrite), and it only ever renames the already-written temp (never a corrupt blend; a failed rename falls back to the normal unlink). See the `-S`/`--sparse` interplay note (a retained sparse temp has full logical size) |
| `--out-format=FORMAT` | Custom output format | ✅ Implemented | Per-transfer template on stdout; tokens `%f` `%n` `%l` `%b` `%M` `%%` (`%b` is the source length, always `== %l`; post-compression/delta wire bytes are not counted); unknown escapes preserved |
| `--log-file=FILE` | Log to file | ✅ Implemented | `log_file` config field |
| `--log-file-format=FMT` | Log format | ✅ Implemented | Requires `--log-file`; writes one template line per transferred file using the same token set as `--out-format` (including `%b` `==` source length) |
| `--8-bit-output`, `-8` | Leave high-bit chars unescaped | ✅ Implemented | Applies to displayed paths and protocol debug output |
| `--list-only` | List files instead of copying | ✅ Implemented | `ls -l`-style listing of files that would be transferred; scans the source only, contacts no server, writes nothing; also works with `-n` |
| `--stats` | Give transfer stats | ⚠️ Caveat | Prints file/byte counts. **Divergence:** the receiver-only counters rsync derives during its generator pass (matched/unchanged data, file-list bytes, deleted-entry count) are reported as **0** by FastSync, and the byte total counts source bytes actually sent rather than the post-delta/post-compression wire volume. Counts that FastSync can observe locally (files, bytes, timing) are accurate |
| `-h`, `--human-readable` | Human-readable numbers | ✅ Parity | Formats transfer byte and rate counts using rsync's **decimal** (base-1000) units, matching rsync `-h` (e.g. `1.23M`), not binary units |
| `-i`, `--itemize-changes` | Per-file change summary | ✅ Parity | Prints rsync-style `>f+++++++++` lines to stdout only for files actually sent (also under `-j`/`--threads`); unchanged files print nothing, matching single-`-i` behavior |
| `--progress` | Show progress | ⚠️ Caveat | Prints a periodic **aggregate** transfer line (bytes sent and current rate), not rsync's per-file progress block. With `-P` the partial-file retention behavior is fully implemented; only the progress presentation differs |
| `-P` | Same as --partial --progress | ⚠️ Caveat | Phase 7 Wave B: `-P` parses to `--partial` + `--progress`. On a failed/interrupted write the receiver now retains the already-written temp file at the destination path (best-effort rename instead of unlink when configured), so a later `--append`/`--append-verify` run can resume it; `--partial-dir` still stages completed files under the confined partial dir and installs them atomically. The retention never runs when `--partial` is off, when no data was actually written, or under `--ignore-existing`/`--existing` (the destination is not ours to overwrite), and it only ever renames the already-written temp (never a corrupt blend; a failed rename falls back to the normal unlink). See the `-S`/`--sparse` interplay note (a retained sparse temp has full logical size) |
| `--out-format=FORMAT` | Custom output format | ⚠️ Caveat | Per-transfer template on stdout; tokens `%f` `%n` `%l` `%b` `%M` `%%` (`%b` is the source length, always `== %l`; post-compression/delta wire bytes are not counted); unknown escapes preserved |
| `--log-file=FILE` | Log to file | ✅ Parity | `log_file` config field |
| `--log-file-format=FMT` | Log format | ✅ Parity | Requires `--log-file`; writes one template line per transferred file using the same token set as `--out-format` (including `%b` `==` source length) |
| `--8-bit-output`, `-8` | Leave high-bit chars unescaped | ✅ Parity | Applies to displayed paths and protocol debug output |
| `--list-only` | List files instead of copying | ✅ Parity | `ls -l`-style listing of files that would be transferred; scans the source only, contacts no server, writes nothing; also works with `-n` |
## 3. File Selection
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `--exclude-from=FILE` | Read exclude patterns from file | ✅ Implemented | Reads patterns from file |
| `--include-from=FILE` | Read include patterns from file | ✅ Implemented | Reads patterns from file |
| `--filter=RULE` | Add file-filtering rule | ✅ Implemented | Long option only: rsync's short `-f` conflicts with FastSync sendfile (see FastSync-specific list), so `-f` is not reassigned. Supported subset: `+`/`-` include/exclude, implicit-exclude patterns, `include`/`exclude` word forms, a leading `/` anchor (to the transfer root, or to a `.rsync-filter` file's directory), and a trailing `/` for dir-only rules; first match wins with a default of include inside the filter layer. Filters are an independent layer from `--exclude`/`--include` (an entry must pass both). Rejected with a clear error (no silent no-ops): `merge`/`dir-merge`/`hide`/`show`/`protect`/`risk`/`clear` words, rules that begin with `:`/`.`/`!` (merge/dir-merge/list-clear shorthands), and include/exclude modifiers other than `/` (`! C s r p x`) |
| `--files-from=FILE` | Read source file list from file | ✅ Implemented | Entries are paths relative to the source root (leading `./` stripped, `..`/absolute entries rejected at parse time, blank lines ignored; NUL-delimited with `-0`). A listed regular file is transferred; a listed directory transfers its whole subtree (FastSync recursion is always on, unlike rsync's non-recursive default). Non-listed paths and their subtrees are pruned by the scanner. A listed entry that does not exist under the source (and an empty list) is a hard error reported before any transfer, unless `--ignore-missing-args` / `--delete-missing-args` is given (see the Safety & Security rows): those flags downgrade the listed-but-missing case to a skip and, for `--delete-missing-args`, a destination deletion; an empty list stays a hard error in every mode. Listing `.` (whole tree) and empty listed directories are fine. Scalability note: `file_list_affects` is O(list size) per scanned entry, so a very large `--files-from` list against a huge tree is quadratic; lists are typically small enough that this is acceptable, but it is the documented bound. The delete manifest still derives from what was actually sent, so `--delete` stays consistent with the subset |
| `-0`, `--from0` | Delimit *-from files with NULs | ✅ Implemented | `--files-from` entries become NUL-delimited; the flag may appear before or after `--files-from` on the command line. NUL mode preserves entry bytes exactly (trailing CR/LF are part of the name; only newline mode trims them) |
| `--max-size=SIZE` | Skip files larger than SIZE | ✅ Implemented | `max_size` in scanner |
| `--min-size=SIZE` | Skip files smaller than SIZE | ✅ Implemented | `min_size` in scanner |
| `-I`, `--ignore-times` | Don't skip files matching size+time | ✅ Implemented | `ignore_times` config field (crosses the wire). Disables the size+mtime quick-check in the `--incremental` per-file handshake and the basis-dir quick-match, forcing the file to be transferred rather than skipped as unchanged. Receiver-side policy: `match_by_metadata` (file_receive.c) is bypassed, so the receiver never replies `STATUS_OK` for a matching size+mtime. Requires `--incremental` to have the handshake to act on (rsync does its quick check by default; FastSync's `-I`/`--size-only`/`--modify-window` only take effect under `--incremental`, exactly like they take effect through the basis check) |
| `--size-only` | Skip based on size only | ✅ Implemented | With `--incremental`, ignores mtime |
| `-@`, `--modify-window=NUM` | Mod-time comparison accuracy | ✅ Implemented | Whole-second tolerance with nanosecond-aware comparisons |
| `--existing` | Skip creating new files on receiver | ✅ Implemented | Existing destination files continue through normal update handling |
| `--ignore-existing` | Skip updating existing files | ✅ Implemented | `ignore_existing` config field (crosses the wire; receiver-side policy). For a destination entry that already exists, the receiver skips the write: in the regular-file path, existing/delay-updates-staged, hardlink-sibling, and special/device handlers all return `FILE_SAVE_SKIPPED` without overwriting (passed as `no_replace` to the write engine), and `--backup` is disabled for skipped files. Note: it is applied at write time, so an existing dest whose size+mtime differ still has its data (or delta) transmitted before the write is discarded — functionally correct, bandwidth-suboptimal vs rsync, which short-circuits earlier. Like rsync, it does not apply to directories/symlinks (those return before the block). Combines with `-j`/`--threads` and `--delay-updates`. See Phase-4/— notes below |
| `--remove-source-files` | Sender removes regular files after confirmed transfer | ✅ Implemented | |
| `-x`, `--one-file-system` | Do not cross filesystem boundaries | ✅ Implemented | Sender scanner captures the root device and skips descending into mount-point crossings (`st_dev` differs); cross-filesystem mount-point subdirectories are dropped entirely, matching rsync |
| `-F` | Add the default `.rsync-filter` rules | ✅ Implemented | Reads one filter rule per line from each directory's `.rsync-filter` file during traversal and applies it to that directory's subtree; the current directory's rules are evaluated before its ancestors', so deeper files override shallower ones and per-directory files override the command-line `--filter`/`-C` base by default (matching rsync's first-match-wins precedence); `.rsync-filter` files are never transferred. The rsync `-FF` behavior (also `.cvsignore`) is out of scope; unsupported rule types inside the file abort with a clear error |
| `--exclude-from=FILE` | Read exclude patterns from file | ✅ Parity | Reads patterns from file |
| `--include-from=FILE` | Read include patterns from file | ✅ Parity | Reads patterns from file |
| `--filter=RULE` | Add file-filtering rule | ⚠️ Caveat | Long option only: rsync's short `-f` conflicts with FastSync sendfile (see FastSync-specific list), so `-f` is not reassigned. Supported subset: `+`/`-` include/exclude, implicit-exclude patterns, `include`/`exclude` word forms, a leading `/` anchor (to the transfer root, or to a `.rsync-filter` file's directory), and a trailing `/` for dir-only rules; first match wins with a default of include inside the filter layer. Filters are an independent layer from `--exclude`/`--include` (an entry must pass both). Rejected with a clear error (no silent no-ops): `merge`/`dir-merge`/`hide`/`show`/`protect`/`risk`/`clear` words, rules that begin with `:`/`.`/`!` (merge/dir-merge/list-clear shorthands), and include/exclude modifiers other than `/` (`! C s r p x`) |
| `--files-from=FILE` | Read source file list from file | ⚠️ Caveat | Entries are paths relative to the source root (leading `./` stripped, `..`/absolute entries rejected at parse time, blank lines ignored; NUL-delimited with `-0`). A listed regular file is transferred; a listed directory transfers its whole subtree (FastSync recursion is always on, unlike rsync's non-recursive default). Non-listed paths and their subtrees are pruned by the scanner. A listed entry that does not exist under the source (and an empty list) is a hard error reported before any transfer, unless `--ignore-missing-args` / `--delete-missing-args` is given (see the Safety & Security rows): those flags downgrade the listed-but-missing case to a skip and, for `--delete-missing-args`, a destination deletion; an empty list stays a hard error in every mode. Listing `.` (whole tree) and empty listed directories are fine. Scalability note: `file_list_affects` is O(list size) per scanned entry, so a very large `--files-from` list against a huge tree is quadratic; lists are typically small enough that this is acceptable, but it is the documented bound. Delete scoping (protocol 2.23.0): the manifest carries the set of synchronized directories, and the extras walk only visits those subtrees, so `--delete` with a `--files-from` subset no longer removes destination paths outside the listed directory subtrees (a data-loss fix matching rsync) |
| `-0`, `--from0` | Delimit *-from files with NULs | ✅ Parity | `--files-from` entries become NUL-delimited; the flag may appear before or after `--files-from` on the command line. NUL mode preserves entry bytes exactly (trailing CR/LF are part of the name; only newline mode trims them) |
| `--max-size=SIZE` | Skip files larger than SIZE | ✅ Parity | `max_size` in scanner |
| `--min-size=SIZE` | Skip files smaller than SIZE | ✅ Parity | `min_size` in scanner |
| `-I`, `--ignore-times` | Don't skip files matching size+time | ✅ Parity | `ignore_times` config field (crosses the wire). Disables the size+mtime quick-check in the `--incremental` per-file handshake and the basis-dir quick-match, forcing the file to be transferred rather than skipped as unchanged. Receiver-side policy: `match_by_metadata` (file_receive.c) is bypassed, so the receiver never replies `STATUS_OK` for a matching size+mtime. Requires `--incremental` to have the handshake to act on (rsync does its quick check by default; FastSync's `-I`/`--size-only`/`--modify-window` only take effect under `--incremental`, exactly like they take effect through the basis check) |
| `--size-only` | Skip based on size only | ✅ Parity | With `--incremental`, ignores mtime |
| `-@`, `--modify-window=NUM` | Mod-time comparison accuracy | ✅ Parity | Whole-second tolerance with nanosecond-aware comparisons |
| `--existing` | Skip creating new files on receiver | ✅ Parity | Existing destination files continue through normal update handling |
| `--ignore-existing` | Skip updating existing files | ⚠️ Caveat | `ignore_existing` config field (crosses the wire; receiver-side policy). For a destination entry that already exists, the receiver skips the write: in the regular-file path, existing/delay-updates-staged, hardlink-sibling, and special/device handlers all return `FILE_SAVE_SKIPPED` without overwriting (passed as `no_replace` to the write engine), and `--backup` is disabled for skipped files. Note: it is applied at write time, so an existing dest whose size+mtime differ still has its data (or delta) transmitted before the write is discarded — functionally correct, bandwidth-suboptimal vs rsync, which short-circuits earlier. Like rsync, it does not apply to directories/symlinks (those return before the block). Combines with `-j`/`--threads` and `--delay-updates`. See Phase-4/— notes below |
| `--remove-source-files` | Sender removes regular files after confirmed transfer | ✅ Parity | |
| `-x`, `--one-file-system` | Do not cross filesystem boundaries | ✅ Parity | Sender scanner captures the root device and does not descend into mount-point crossings (`st_dev` differs). **Protocol 2.23.0 matches rsync's entry emission:** the mount-point directory itself is emitted as a payload-less directory entry (so the destination gets an empty directory) while its contents are skipped; previously the crossing subdirectory was dropped entirely |
| `-F` | Add the default `.rsync-filter` rules | ⚠️ Caveat | Reads one filter rule per line from each directory's `.rsync-filter` file during traversal and applies it to that directory's subtree; the current directory's rules are evaluated before its ancestors', so deeper files override shallower ones and per-directory files override the command-line `--filter`/`-C` base by default (matching rsync's first-match-wins precedence); `.rsync-filter` files are never transferred. The rsync `-FF` behavior (also `.cvsignore`) is out of scope; unsupported rule types inside the file abort with a clear error |
## 4. Directory Options
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-r`, `--recursive` | Recurse into directories | ✅ Implemented | Default behavior |
| `-R`, `--relative` | Use relative path names | ✅ Implemented | Meaningful together with `--files-from` (FastSync's default full-tree scan always mirrors the full source argument path below the destination root, so -R does not change it). With `-R` + `--files-from` each listed entry is transmitted under its bare relative destination path: an entry `sub/x.txt` lands at `<dest>/sub/x.txt` (its leading components preserved) instead of under the `<dest>/<full source path>` mirror. Only the path sent on the wire changes; the client still reads the absolute source path, and the delete manifest derives from the sent (relative) paths so `--delete` and `--remove-source-files` stay consistent in both layouts. Works single-threaded and under `-j`/`--threads` (including chunk serialization) |
| `--no-implied-dirs` | Don't send implied dirs with -R | ✅ Implemented | Client-side, meaningful only with `-R` + `--files-from`. rsync would normally create the ancestor directories implied by a listed file so it can be written; with `--no-implied-dirs` a listed file whose parent directory is not itself (or via an ancestor) explicitly listed cannot be placed, and FastSync fails the whole run up front with a clear error (`--no-implied-dirs: cannot place file '...': parent directory '...' is not explicitly listed`). Listing the directory (or an ancestor of it, or the whole tree `.`) permits the file. In every other mode the option has no effect. FastSync has no per-entry skip channel, so the rsync "omit the file" case is surfaced as a hard pre-transfer error |
| `-d`, `--dirs`, `--old-dirs`, `--old-d` | Transfer dirs without recursing | ✅ Implemented | `-d <dir>` transmits an explicit directory entry for the source-root directory, so the destination mirror is created empty and nothing is descended into. With `--files-from` exactly the listed items are transferred: a listed directory is created empty (no descent) and a listed file is transferred with its content; the dest layout follows the same -R rules as plain files. A new wire frame (`STATUS_MKDIR`) carries each directory entry — the path and, when `--preserve`/`-a` (metadata mode) is negotiated, the directory's metadata; the receiver creates it with the same confined mkdir-parent semantics as regular writes, in single-threaded and `-j`/`--threads` receivers (chunk serialization carries a per-entry type marker). Directory entries appear in the delete manifest so `--delete` prunes correctly. Directory TIMES are transmitted (the `STATUS_DIR_TIMES` frame carries every traversed source directory's captured times, including `--dirs` entries) and applied by the receiver at the END of the transfer, after all children and the delete/publication phases, so a later child write cannot clobber a directory's mtime (`-O`/`--omit-dir-times` skips this application). FastSync divergences: directory modes/ownership are still not applied (only times are), and empty directories are still never created (a `STATUS_DIR_TIMES` entry is record-only), filter/`--exclude` rules are not re-applied to the listed dirs mode (there is no descent during which they would apply), and `-d` never creates the intermediate directories between the destination root and a listed file beyond the usual on-demand parent creation. Under `--delay-updates` only regular files are staged: directory entries are created immediately, so a delayed run that fails part way can leave the already-created empty directories behind (matching rsync, which also creates directories as it processes the file list and only delays regular-file data) |
| `--mkpath` | Create missing path components | ✅ Implemented | Wire option (client → server). At connection start the server creates the client's destination root directory (and any missing leading components below its own authorized root) when `--mkpath` is set, failing the connection cleanly if it cannot. Without `--mkpath` a destination root that does not exist yet is rejected up front (rsync semantics), so the flag is the only way to transfer into a not-yet-created destination directory. Creation is confined by the same secure mkdir walk as file writes (`O_NOFOLLOW`, no `..`) |
| `-r`, `--recursive` | Recurse into directories | ✅ Parity | Default behavior |
| `-R`, `--relative` | Use relative path names | ⚠️ Caveat | Meaningful together with `--files-from` (FastSync's default full-tree scan always mirrors the full source argument path below the destination root, so -R does not change it). With `-R` + `--files-from` each listed entry is transmitted under its bare relative destination path: an entry `sub/x.txt` lands at `<dest>/sub/x.txt` (its leading components preserved) instead of under the `<dest>/<full source path>` mirror. Only the path sent on the wire changes; the client still reads the absolute source path, and the delete manifest derives from the sent (relative) paths so `--delete` and `--remove-source-files` stay consistent in both layouts. Works single-threaded and under `-j`/`--threads` (including chunk serialization) |
| `--no-implied-dirs` | Don't send implied dirs with -R | ⚠️ Caveat | Client-side, meaningful only with `-R` + `--files-from`. rsync would normally create the ancestor directories implied by a listed file so it can be written; with `--no-implied-dirs` a listed file whose parent directory is not itself (or via an ancestor) explicitly listed cannot be placed, and FastSync fails the whole run up front with a clear error (`--no-implied-dirs: cannot place file '...': parent directory '...' is not explicitly listed`). Listing the directory (or an ancestor of it, or the whole tree `.`) permits the file. In every other mode the option has no effect. FastSync has no per-entry skip channel, so the rsync "omit the file" case is surfaced as a hard pre-transfer error |
| `-d`, `--dirs`, `--old-dirs`, `--old-d` | Transfer dirs without recursing | ⚠️ Caveat | `-d <dir>` transmits an explicit directory entry for the source-root directory, so the destination mirror is created empty and nothing is descended into. With `--files-from` exactly the listed items are transferred: a listed directory is created empty (no descent) and a listed file is transferred with its content; the dest layout follows the same -R rules as plain files. A new wire frame (`STATUS_MKDIR`) carries each directory entry — the path and, when `--preserve`/`-a` (metadata mode) is negotiated, the directory's metadata; the receiver creates it with the same confined mkdir-parent semantics as regular writes, in single-threaded and `-j`/`--threads` receivers (chunk serialization carries a per-entry type marker). Directory entries appear in the delete manifest so `--delete` prunes correctly. Directory TIMES are transmitted (the `STATUS_DIR_TIMES` frame carries every traversed source directory's captured times, including `--dirs` entries) and applied by the receiver at the END of the transfer, after all children and the delete/publication phases, so a later child write cannot clobber a directory's mtime (`-O`/`--omit-dir-times` skips this application). FastSync divergences: directory modes/ownership are still not applied (only times are), and empty directories are still never created (a `STATUS_DIR_TIMES` entry is record-only), filter/`--exclude` rules are not re-applied to the listed dirs mode (there is no descent during which they would apply), and `-d` never creates the intermediate directories between the destination root and a listed file beyond the usual on-demand parent creation. Under `--delay-updates` only regular files are staged: directory entries are created immediately, so a delayed run that fails part way can leave the already-created empty directories behind (matching rsync, which also creates directories as it processes the file list and only delays regular-file data) |
| `--mkpath` | Create missing path components | ✅ Parity | Wire option (client → server). At connection start the server creates the client's destination root directory (and any missing leading components below its own authorized root) when `--mkpath` is set, failing the connection cleanly if it cannot. Without `--mkpath` a destination root that does not exist yet is rejected up front (rsync semantics), so the flag is the only way to transfer into a not-yet-created destination directory. Creation is confined by the same secure mkdir walk as file writes (`O_NOFOLLOW`, no `..`) |
## 5. Transfer Modifications
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-u`, `--update` | Skip files newer on receiver | ✅ Implemented | `update` config field (crosses the wire; receiver-side policy, implies `-M` metadata). Before writing a regular file, the receiver checks `file_destination_is_newer_secure()` (via `stat_is_newer`, second-then-nanosecond strict `>` on the existing destination) and skips the write when the destination is newer than the source (`FILE_SAVE_SKIPPED`); equal-or-older destination (or a newer source) is transferred normally. Applied at write time on the regular-file, delay-updates-staged, hardlink-sibling, and special/device paths. Only regular destinations can be guarded (the newer-check requires `S_ISREG`), and like the other write-time policies it does not short-circuit the data transfer for a differing-size dest. `--remove-source-files` correctly respects the receiver's skip outcome so a skipped source is not removed |
| `--inplace` | Update files in-place | ✅ Implemented | Direct write mode |
| `--append` | Append data to shorter files | ✅ Implemented | Tail-only resume. When an existing destination file is SHORTER than the source, the receiver negotiates a resume offset with the sender and only the tail is transferred; the receiver rebuilds the full file (retained prefix + tail) and installs it through the normal atomic store path, so the result is byte-identical to the source whenever the retained prefix matches. Plain `--append` does NOT content-verify that prefix (rsync parity): a destination whose prefix differs from the source is resumed anyway, so the result (wrong prefix + correct tail) is NOT byte-identical and the file is effectively left corrupt — the documented rsync-parity risk (use `--append-verify` when the prefix cannot be trusted). Non-content attributes (permissions/ownership/mtime, via `-M`) are still applied. Requires the per-file `STATUS_CHECK` handshake, so it implies `--incremental`; it takes precedence over block delta for a growing file and falls back to delta/full when the destination is not shorter. Incompatible with `-s` (chunk serialization) and `--whole-file` (both rejected up front so the mode never silently degrades to a full transfer). Combines with `--inplace`, `--partial`/`--partial-dir`, and `--delay-updates` (the reconstructed full file flows through those paths unchanged). Divergence: rsync appends in place; FastSync reconstructs and atomically installs, so an interrupted or failed resume never leaves a half-written file at the destination (no corruption window), and `--append` is thus safe to use with the normal atomic path — not only with in-place writes |
| `--append-verify` | Append with old-data checksum | ✅ Implemented | Like `--append`, but the retained prefix IS verified before resuming: the sender transmits the source prefix checksum and the receiver compares it to the xxHash64 of the retained destination prefix; on a match only the tail is transferred, on a MISMATCH the run falls back to a clean full transfer so the result is always a byte-identical source copy (never a corrupt prefix+tail blend). Wire/protocol: the append handshake adds `STATUS_APPEND` / `STATUS_APPEND_SIG` / `STATUS_APPEND_OK` / `STATUS_APPEND_DATA` frames and `PROTOCOL_VERSION` was bumped **2.9.0 → 2.10.0** (peers must match, and both must be 2.10.0 or the run fails the version check). Same implications/incompatibilities as `--append`; when both spellings are given `--append-verify` wins (the safer semantics). See the Phase-3 append notes below |
| `-W`, `--whole-file` | Copy whole file (no delta) | ✅ Implemented | `whole_file` config field. Forces a full (whole-file) copy, disabling the block-level delta machinery: the sender only sends `STATUS_NEXT` + full data (client_send.c) and the receiver never requests a delta signature/reconstruction — the receiver's `try_delta = use_delta && !whole_file && ...` short-circuits. `whole_file` crosses the wire folded into `use_delta` (the wire carries `use_delta && !whole_file`), so no separate field/bump is needed. Delta is opt-in (`--delta` needs `--incremental`); `-W` additionally makes `--fuzzy` inert (no similar-file delta basis). `--append`/`--append-verify` are incompatible with `-W` and rejected up front (both sides). See the delta/append notes below |
| `--block-size=SIZE` | Force checksum block-size | ✅ Implemented | Phase 7 Wave B: `--block-size` is an alias for `--delta-block`; both set `config->delta_block_size` (default `DELTA_BLOCK_SIZE_DEFAULT`, bounds `DELTA_BLOCK_SIZE_MIN..MAX`, out-of-range values are rejected with the default kept). The value is genuinely honored by the delta engine end-to-end: `delta_signature_create_seeded(old, size, config->delta_block_size, seed)` on the sender and receiver, `delta_apply(old, ...)` with the same size, so a non-default block size changes the block count of every signature the harnesses exchange (verified by unit + integration tests) |
| `-u`, `--update` | Skip files newer on receiver | ✅ Parity | `update` config field (crosses the wire; receiver-side policy, implies metadata transmission). Before writing a regular file, the receiver checks `file_destination_is_newer_secure()` (via `stat_is_newer`, second-then-nanosecond strict `>` on the existing destination) and skips the write when the destination is newer than the source (`FILE_SAVE_SKIPPED`); equal-or-older destination (or a newer source) is transferred normally. Applied at write time on the regular-file, delay-updates-staged, hardlink-sibling, and special/device paths. Only regular destinations can be guarded (the newer-check requires `S_ISREG`), and like the other write-time policies it does not short-circuit the data transfer for a differing-size dest. `--remove-source-files` correctly respects the receiver's skip outcome so a skipped source is not removed |
| `--inplace` | Update files in-place | ✅ Parity | Direct write mode |
| `--append` | Append data to shorter files | ⚠️ Caveat | Tail-only resume. When an existing destination file is SHORTER than the source, the receiver negotiates a resume offset with the sender and only the tail is transferred; the receiver rebuilds the full file (retained prefix + tail) and installs it through the normal atomic store path, so the result is byte-identical to the source whenever the retained prefix matches. Plain `--append` does NOT content-verify that prefix (rsync parity): a destination whose prefix differs from the source is resumed anyway, so the result (wrong prefix + correct tail) is NOT byte-identical and the file is effectively left corrupt — the documented rsync-parity risk (use `--append-verify` when the prefix cannot be trusted). Non-content attributes (permissions/ownership/mtime, via `-M`) are still applied. Requires the per-file `STATUS_CHECK` handshake, so it implies `--incremental`; it takes precedence over block delta for a growing file and falls back to delta/full when the destination is not shorter. Incompatible with `-s` (chunk serialization) and `--whole-file` (both rejected up front so the mode never silently degrades to a full transfer). Combines with `--inplace`, `--partial`/`--partial-dir`, and `--delay-updates` (the reconstructed full file flows through those paths unchanged). Divergence: rsync appends in place; FastSync reconstructs and atomically installs, so an interrupted or failed resume never leaves a half-written file at the destination (no corruption window), and `--append` is thus safe to use with the normal atomic path — not only with in-place writes |
| `--append-verify` | Append with old-data checksum | ⚠️ Caveat | Like `--append`, but the retained prefix IS verified before resuming: the sender transmits the source prefix checksum and the receiver compares it to the xxHash64 of the retained destination prefix; on a match only the tail is transferred, on a MISMATCH the run falls back to a clean full transfer so the result is always a byte-identical source copy (never a corrupt prefix+tail blend). Wire/protocol: the append handshake adds `STATUS_APPEND` / `STATUS_APPEND_SIG` / `STATUS_APPEND_OK` / `STATUS_APPEND_DATA` frames and `PROTOCOL_VERSION` was bumped **2.9.0 → 2.10.0** (peers must match, and both must be 2.10.0 or the run fails the version check). Same implications/incompatibilities as `--append`; when both spellings are given `--append-verify` wins (the safer semantics). See the Phase-3 append notes below |
| `-W`, `--whole-file` | Copy whole file (no delta) | ✅ Parity | `whole_file` config field. Forces a full (whole-file) copy, disabling the block-level delta machinery: the sender only sends `STATUS_NEXT` + full data (client_send.c) and the receiver never requests a delta signature/reconstruction — the receiver's `try_delta = use_delta && !whole_file && ...` short-circuits. `whole_file` crosses the wire folded into `use_delta` (the wire carries `use_delta && !whole_file`), so no separate field/bump is needed. Delta is opt-in (`--delta` needs `--incremental`); `-W` additionally makes `--fuzzy` inert (no similar-file delta basis). `--append`/`--append-verify` are incompatible with `-W` and rejected up front (both sides). See the delta/append notes below |
| `--block-size=SIZE` | Force checksum block-size | ✅ Parity | Phase 7 Wave B: `--block-size` is an alias for `--delta-block`; both set `config->delta_block_size` (default `DELTA_BLOCK_SIZE_DEFAULT`, bounds `DELTA_BLOCK_SIZE_MIN..MAX`, out-of-range values are rejected with the default kept). The value is genuinely honored by the delta engine end-to-end: `delta_signature_create_seeded(old, size, config->delta_block_size, seed)` on the sender and receiver, `delta_apply(old, ...)` with the same size, so a non-default block size changes the block count of every signature the harnesses exchange (verified by unit + integration tests) |
## 6. Destination Handling
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-n`, `--dry-run` | Trial run with no changes | ✅ Implemented | Server-contacting since protocol 2.21.0. The final routing predicate is `dry_run_targets_server()` in `src/client/client_send.c`: any target a real run would reach over the wire selects the server-contacting path — an SSH transport, a daemon `host::module` destination, an explicit `--server-host` or `--server-port`/`--port`, TLS, or a source-bind `--address` — and the client handshakes with the receiver, which runs the normal read-only per-file check and answers `STATUS_DRY_RUN_TRANSFER`/`STATUS_OK` without mutating anything. A plain local destination (none of those) keeps the original client-side manifest that never dials the default `127.0.0.1:8080`. Would-delete reporting for `--delete*` is deferred (dry-run never deletes). |
| `-b`, `--backup` | Make backups of overwritten files | ✅ Implemented | Backup before overwrite |
| `--backup-dir=DIR` | Backup directory hierarchy | ✅ Implemented | `backup_dir` config field |
| `--suffix=SUFFIX` | Backup suffix (default ~) | ✅ Implemented | `suffix` config field |
| `--delay-updates` | Put updated files in place at end | ✅ Implemented | Successfully received files are staged under a private 0700 `.fastsync-stage` dir inside the receive root and atomically renamed into their final destinations only after the whole transfer (manifest/delete handling included) succeeds, just before the success/outcome frame is sent. The delete walker deliberately skips the staging dir at the receive root, so `--delete` removes genuine extras but never the staged files (deletion runs before publication; rsync's delete-after ordering is not implemented). `--existing`/`--ignore-existing`/`--update` decide against the final destination path at stage time; `--backup` moves the old file aside at publication. Incompatible with `--inplace` and with `--backup-dir=.fastsync-stage` (the internal staging name is reserved; both are rejected). The staging dir name is fixed, so two simultaneous delayed transfers to the same destination root are serialized with an exclusive advisory lock held for the whole transfer: the second session fails cleanly instead of corrupting the first. Aborting or failing before publication installs nothing and removes the staging tree; a crash between stage and publish leaves staged leftovers that the next delayed run wipes at start (process death releases the lock). A stage→publish failure aborts the transfer (best-effort cleanup of the not-yet-published staged files; already-published files are not rolled back). Works in single-threaded and `-j`/`--threads` modes |
| `-T`, `--temp-dir=DIR` | Create temporary files in DIR | ✅ Implemented | `--temp-dir` with the rsync short `-T` (Phase 7 Wave A; the timeout alias moved to long-only `--timeout`). Scratch dir is resolved under the receive root; temp copies use a unique name there and are atomically renamed into place. If the scratch dir and destination are on different filesystems the atomic rename fails with EXDEV and the file save fails, which aborts the whole transfer (FastSync has no per-file skip/resume on a save error; rsync's non-atomic copy fallback is deliberately not used). `--inplace` and `--partial-dir` writes bypass the scratch dir |
| `-n`, `--dry-run` | Trial run with no changes | ⚠️ Caveat | Server-contacting since protocol 2.21.0. The final routing predicate is `dry_run_targets_server()` in `src/client/client_send.c`: any target a real run would reach over the wire selects the server-contacting path — an SSH transport, a daemon `host::module` destination, an explicit `--server-host` or `--server-port`/`--port`, TLS, or a source-bind `--address` — and the client handshakes with the receiver, which runs the normal read-only per-file check and answers `STATUS_DRY_RUN_TRANSFER`/`STATUS_OK` without mutating anything. A plain local destination (none of those) keeps the original client-side manifest that never dials the default `127.0.0.1:8080`. Would-delete reporting for `--delete*` is deferred (dry-run never deletes). |
| `-b`, `--backup` | Make backups of overwritten files | ✅ Parity | Backup before overwrite |
| `--backup-dir=DIR` | Backup directory hierarchy | ✅ Parity | `backup_dir` config field |
| `--suffix=SUFFIX` | Backup suffix (default ~) | ✅ Parity | `suffix` config field |
| `--delay-updates` | Put updated files in place at end | ⚠️ Caveat | Successfully received files are staged under a private 0700 `.fastsync-stage` dir inside the receive root and atomically renamed into their final destinations only after the whole transfer (manifest/delete handling included) succeeds, just before the success/outcome frame is sent. The delete walker deliberately skips the staging dir at the receive root, so `--delete` removes genuine extras but never the staged files (deletion runs before publication; rsync's delete-after ordering is not implemented). `--existing`/`--ignore-existing`/`--update` decide against the final destination path at stage time; `--backup` moves the old file aside at publication, and **`--force` is honored at publication** (protocol 2.23.0): a staged regular file or symlink may replace a destination directory that blocks it. Incompatible with `--inplace` and with `--backup-dir=.fastsync-stage` (the internal staging name is reserved; both are rejected). The staging dir name is fixed, so two simultaneous delayed transfers to the same destination root are serialized with an exclusive advisory lock held for the whole transfer: the second session fails cleanly instead of corrupting the first. Aborting or failing before publication installs nothing and removes the staging tree; a crash between stage and publish leaves staged leftovers that the next delayed run wipes at start (process death releases the lock). A stage→publish failure aborts the transfer (best-effort cleanup of the not-yet-published staged files; already-published files are not rolled back). Works in single-threaded and `-j`/`--threads` modes |
| `-T`, `--temp-dir=DIR` | Create temporary files in DIR | ⚠️ Caveat | `--temp-dir` with the rsync short `-T` (the timeout alias moved to long-only `--timeout`). **Protocol 2.23.0 receiver policy: the scratch dir is confined to the receive root — a relative dir is resolved below it, and an absolute path or one containing `..` is rejected by the receiver** (an absolute/foreign-filesystem scratch dir was the divergence; rsync's standalone mode would follow an absolute `--temp-dir`, while its daemon also confines). Temp copies use a unique name there and are atomically renamed into place. **On `EXDEV` (scratch dir and destination on different filesystems) the receiver falls back to a non-atomic copy instead of aborting the transfer**, matching rsync. `--inplace` and `--partial-dir` writes bypass the scratch dir |
## 7. Deletion
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `--delete` | Delete extraneous files from dest | ✅ Implemented | `use_delete` config field. Deletion is always derived from the transmitted keep-set manifest of the paths the sender sent/keeps (never from unchecked input), runs through the symlink-safe walker bounded by `MAX_SERVER_DELETE_COUNT`, and skips the `.fastsync-stage` staging dir under `--delay-updates`. FastSync's default timing when no timing flag is given is **delete-after** (extras are removed only once the whole transfer succeeded) — intentionally NOT rsync's `--del`/delete-during default, to preserve FastSync's commit-style safety. By default the destination mirror of a path the source scan pruned (filter/exclude/size rules) is **protected** from deletion — matching rsync, which does not delete excluded files under `--delete`; `--delete-excluded` opts back into deleting them (see below). The bounded deletion is **all-or-nothing**: if the destination holds more extras than the effective bound (a client `--max-delete=NUM` or the 100000-entry server bound) nothing is deleted and the run fails with a distinct error instead of silently truncating |
| `--delete-before` | Delete before transfer | ✅ Implemented | Implies `--delete`. The sender runs a full source pre-scan (paths only) and transmits the keep-set manifest BEFORE any file data; the receiver validates it, removes every destination entry not listed (all-or-nothing bounded walk, staging-dir skip, protected prefixes honored), then acks `STATUS_OK`. The sender only starts streaming after the deletion committed, or aborts if the receiver reported a deletion error. By definition the deletions already happened when a later transfer phase fails — rsync's delete-before is destructive the same way; a subsequent failure does not restore the removed files. Divergence: the keep-set is the pre-scan snapshot, so a file that appears on the source between the pre-scan and the data pass is still transferred but was not protected from deletion |
| `--del`, `--delete-during` | Delete during transfer | ✅ Implemented | Both spellings accepted; imply `--delete`. FastSync streams the source in a single directory scan and has no per-directory generator pass, so deletions cannot be interleaved per-directory the way rsync's delete-during does. `--delete-during` therefore selects the same early engine mode as `--delete-before` (manifest transmitted before any data, extras removed and acknowledged before data is applied); observable success/failure behaviour equals `--delete-before`. That is the documented divergence from rsync, where `--del` is the default meaning of `--delete` |
| `--delete-delay` | Find deletions during, delete after | ✅ Implemented | Implies `--delete`. Commit-mode timing: extras are removed only after the whole transfer succeeded. rsync's delete-delay records the deletion list during its scan and applies it at the end; FastSync never snapshots the destination while data flows (the keep-set is the transmitted manifest and the destination is listed only at deletion time), so `--delete-delay` is implemented as the same end-of-transfer commit as `--delete-after` with identical safety. That is the documented divergence |
| `--delete-after` | Delete after transfer | ✅ Implemented | Implies `--delete`. The delete-after timing is also what plain `--delete` does: the keep-set manifest closes the data stream and the receiver commits the bounded deletion only after the terminal `STATUS_FINISHED` proves the whole transfer (every data frame received and stored) succeeded. A failed or aborted transfer removes nothing |
| `--delete-excluded` | Also delete excluded files | ✅ Implemented | `delete_excluded` config field. Under `--delete` FastSync now protects (rsync's default) the destination mirror of paths the sender's source scan pruned by user-selection rules — the `--filter`/`-F`/`-C` layer, the legacy `--exclude`/`--include` layer, and `--max-size`/`--min-size`. The sender transmits those concrete pruned paths as **protected prefixes** in the delete-manifest frame (see the Phase-3 notes below); the walker never descends into or removes them. `--delete-excluded` opts back in: the sender sends an empty protected list, so the excluded destination mirrors become ordinary extras and are removed. Divergences (documented): protection is derived only from what the source scan actually pruned — a stray destination-only file that happens to match an exclude rule is not protected (FastSync never re-applies rules to the destination, keeping deletion sender-derived), and `--files-from` subset pruning stays keep-set-only (an unlisted source path is treated as absent and its mirror is deletable, matching the `--files-from` delete note below). The two are orthogonal: `--delete-excluded` removes filter-excluded mirrors; it does not make `--files-from` prune things |
| `--max-delete=NUM` | Max files to delete | ✅ Implemented | `max_delete` config field (default -1 = no client limit; 0 = delete nothing). NUM bounds a `--delete` run with rsync's all-or-nothing semantics: the receiver rehearses the deletion first and, if the destination holds more than NUM extras, deletes NOTHING and fails the transfer with a distinct `--max-delete` error. A run at or below NUM deletes exactly the extras. NUM only applies together with `--delete` (it is inert otherwise, matching rsync). The hard server bound `MAX_SERVER_DELETE_COUNT` (100000) still caps the walk; a NUM above it never raises that cap, and exceeding the server bound is its own all-or-nothing error. Directories count toward the limit (each removed empty directory is one deletion), like rsync |
| `--ignore-errors` | Delete even with I/O errors | ✅ Implemented | Sender-side, client-only config field. rsync suppresses `--delete` when the transfer had I/O errors; FastSync's equivalent is a source-scan I/O error (an unreadable directory, e.g. EACCES): by default the scan aborts the run so no deletion happens. With `--ignore-errors` the scan continues past the unreadable directory, the readable tree is transferred and the deletion still runs (the mirror of the unreadable directory is treated as an extra). The run still exits non-zero (the error is reported, matching rsync's error status). Divergence: without the flag FastSync aborts the whole run on the scan error, whereas rsync transfers the rest of the tree and merely skips the deletion; both leave the deletion undone |
| `--force` | Force deletion of non-empty dirs | ✅ Implemented | `force_delete` receiver config field (crosses the wire). rsync's `--force` lets an incoming non-directory replace a destination directory; FastSync implements exactly that: when a regular file is written to a path that is currently a (possibly non-empty) destination directory, `--force` removes that directory tree first — confined to the receive root and symlink-safe (O_NOFOLLOW fd walk, symlinks removed by name, never followed) — so the atomic install can place the file. Without `--force` such a write fails and the run aborts. Divergence: `--force` acts on the immediate-install path only; under `--delay-updates` a blocking directory is not cleared (publication renames over regular files) |
| `-m`, `--prune-empty-dirs` | Prune empty dir chains | ✅ Implemented | `-m`/`--prune-empty-dirs` (Phase 7 Wave A freed the rsync short `-m`; FastSync multithreading is now `-j`/`--threads`). FastSync's recursive transfer records directory times but never CREATES an empty directory (a `STATUS_DIR_TIMES` entry is record-only, and `--dirs` empty entries are pruned by this flag), so empty directories are inherently never transferred (which is rsync's `-m` behavior) and truly-empty destination directory chains are removed by `--delete` regardless of this flag. The flag's additional real effect is on the `--dirs` explicit directory-entry generator: a plain `-d <empty-dir>` run omits the empty source directory's entry, so nothing is created at the destination (no `STATUS_MKDIR`, no `-i`/`--out-format` change line, and an existing empty mirror becomes an extra that `--delete` prunes). Explicitly `--files-from`-listed directories always pass through (documented `--files-from` behavior). A directory that still holds an excluded-but-protected file survives, matching the `--delete-excluded` default |
| `--delete` | Delete extraneous files from dest | ⚠️ Caveat | `use_delete` config field. Deletion is always derived from the transmitted keep-set manifest of the paths the sender sent/keeps (never from unchecked input), runs through the symlink-safe walker bounded by `MAX_SERVER_DELETE_COUNT`, and skips the `.fastsync-stage` staging dir under `--delay-updates`. FastSync's default timing when no timing flag is given is **delete-after** (extras are removed only once the whole transfer succeeded) — intentionally NOT rsync's `--del`/delete-during default, to preserve FastSync's commit-style safety. By default the destination mirror of a path the source scan pruned (filter/exclude/size rules) is **protected** from deletion — matching rsync, which does not delete excluded files under `--delete`; `--delete-excluded` opts back into deleting them (see below). Deletion is scoped to the **synchronized directories** sent in the manifest (protocol 2.23.0), so a `--files-from` subset no longer deletes untransmitted paths outside the listed directory subtrees. The walk is bounded: a client `--max-delete=NUM` (or the 100000-entry server bound) makes it **partial** — entries up to the bound are removed, the rest are skipped, and the client exits **25** (`RERR_PARTIAL`), matching rsync, rather than failing the transfer. Extraneous destination symlinks are unlinked by name (never followed); a directory still holding a kept/protected entry is left behind rather than failing |
| `--delete-before` | Delete before transfer | ⚠️ Caveat | Implies `--delete`. The sender runs a full source pre-scan (paths only) and transmits the keep-set manifest BEFORE any file data; the receiver validates it, removes every destination entry not listed (bounded walk, staging-dir skip, protected prefixes honored), then acks `STATUS_OK`. The sender only starts streaming after the deletion committed, or aborts if the receiver reported a deletion error. By definition the deletions already happened when a later transfer phase fails — rsync's delete-before is destructive the same way; a subsequent failure does not restore the removed files. Divergence: the keep-set is the pre-scan snapshot, so a file that appears on the source between the pre-scan and the data pass is still transferred but was not protected from deletion |
| `--del`, `--delete-during` | Delete during transfer | ⚠️ Caveat | Both spellings accepted; imply `--delete`. FastSync streams the source in a single directory scan and has no per-directory generator pass, so deletions cannot be interleaved per-directory the way rsync's delete-during does. `--delete-during` therefore selects the same early engine mode as `--delete-before` (manifest transmitted before any data, extras removed and acknowledged before data is applied); observable success/failure behaviour equals `--delete-before`. That is the documented divergence from rsync, where `--del` is the default meaning of `--delete` |
| `--delete-delay` | Find deletions during, delete after | ⚠️ Caveat | Implies `--delete`. Commit-mode timing: extras are removed only after the whole transfer succeeded. rsync's delete-delay records the deletion list during its scan and applies it at the end; FastSync never snapshots the destination while data flows (the keep-set is the transmitted manifest and the destination is listed only at deletion time), so `--delete-delay` is implemented as the same end-of-transfer commit as `--delete-after` with identical safety. That is the documented divergence |
| `--delete-after` | Delete after transfer | ✅ Parity | Implies `--delete`. The delete-after timing is also what plain `--delete` does: the keep-set manifest closes the data stream and the receiver commits the bounded deletion only after the terminal `STATUS_FINISHED` proves the whole transfer (every data frame received and stored) succeeded. A failed or aborted transfer removes nothing |
| `--delete-excluded` | Also delete excluded files | ⚠️ Caveat | `delete_excluded` config field. Under `--delete` FastSync protects (rsync's default) the destination mirror of paths the sender's source scan pruned by the user-selection rules — the `--filter`/`-F`/`-C` layer and the legacy `--exclude`/`--include` layer. The sender transmits those concrete pruned paths as **protected prefixes** in the delete-manifest frame (see the Phase-3 notes below); the walker never descends into or removes them. `--delete-excluded` opts back in: the sender sends an empty protected list, so the excluded destination mirrors become ordinary extras and are removed. **`--max-size`/`--min-size` pruned mirrors are a separate, always-on protection** (protocol 2.23.0, rsync parity): size-pruned source mirrors survive `--delete` even with `--delete-excluded`. Divergences (documented): protection is derived only from what the source scan actually pruned — a stray destination-only file that happens to match an exclude rule is not protected (FastSync never re-applies rules to the destination, keeping deletion sender-derived) |
| `--max-delete=NUM` | Max files to delete | ✅ Parity | `max_delete` config field (default -1 = no client limit; 0 = delete nothing). **Protocol 2.23.0 matches rsync's partial semantics:** the receiver deletes up to NUM entries (regular files, symlinks and empty directories; each directory removal counts as one) and then **stops deleting, skips the rest, and reports the run as partial**. The client prints a "deletions stopped due to `--max-delete` limit" message and exits **25** (rsync's `RERR_PARTIAL`), not a hard failure — the transfer itself succeeded. NUM only applies together with `--delete` (it is inert otherwise, matching rsync). A client NUM below the server hard bound `MAX_SERVER_DELETE_COUNT` (100000) replaces it; a NUM above it never raises that cap. Deleting an entire destination with no limit is still bounded by the server's 100000-entry ceiling. `--delete-missing-args` exact-path deletions and the ordinary extras walk draw from the same budget, matching rsync |
| `--ignore-errors` | Delete even with I/O errors | ⚠️ Caveat | Sender-side, client-only config field. rsync suppresses `--delete` when the transfer had I/O errors; FastSync's equivalent is a source-scan I/O error (an unreadable directory, e.g. EACCES): by default the scan aborts the run so no deletion happens. With `--ignore-errors` the scan continues past the unreadable directory, the readable tree is transferred and the deletion still runs (the mirror of the unreadable directory is treated as an extra). The run still exits non-zero (the error is reported, matching rsync's error status). Divergence: without the flag FastSync aborts the whole run on the scan error, whereas rsync transfers the rest of the tree and merely skips the deletion; both leave the deletion undone |
| `--force` | Force deletion of non-empty dirs | ⚠️ Caveat | `force_delete` receiver config field (crosses the wire). rsync's `--force` lets an incoming non-directory replace a destination directory; FastSync implements exactly that: when a regular file (or symlink) is written to a path that is currently a (possibly non-empty) destination directory, `--force` removes that directory tree first — confined to the receive root and symlink-safe (O_NOFOLLOW fd walk, symlinks removed by name, never followed) — so the install can place the file. **Protocol 2.23.0 honors `--force` on the `--delay-updates` publication path too**, not only the immediate-install path. Without `--force` such a write fails and the run aborts. Gated by the server `--allow-delete` policy (a client cannot use `--force` to remove a destination tree on a server that forbids deletion) |
| `-m`, `--prune-empty-dirs` | Prune empty dir chains | ✅ Parity | `-m`/`--prune-empty-dirs` (Phase 7 Wave A freed the rsync short `-m`; FastSync multithreading is now `-j`/`--threads`). FastSync's recursive transfer records directory times but never CREATES an empty directory (a `STATUS_DIR_TIMES` entry is record-only, and `--dirs` empty entries are pruned by this flag), so empty directories are inherently never transferred (which is rsync's `-m` behavior) and truly-empty destination directory chains are removed by `--delete` regardless of this flag. The flag's additional real effect is on the `--dirs` explicit directory-entry generator: a plain `-d <empty-dir>` run omits the empty source directory's entry, so nothing is created at the destination (no `STATUS_MKDIR`, no `-i`/`--out-format` change line, and an existing empty mirror becomes an extra that `--delete` prunes). Explicitly `--files-from`-listed directories always pass through (documented `--files-from` behavior). A directory that still holds an excluded-but-protected file survives, matching the `--delete-excluded` default |
**Deletion-timing implementation notes (Phase 3):** the delete flags above are
real. Two new config booleans (`delete_during`, `delete_delay`) join the already
@@ -133,64 +154,66 @@ noted in the rows above.
**Deletion-policy notes (Phase 3, delete-policy wave):** this wave made the
deletion family real — `--delete-excluded`, `--max-delete`, `--ignore-errors`,
`--force`, `--prune-empty-dirs` — and, to support them, the `STATUS_MANIFEST`
frame now carries **two sections**: the keep-set paths followed by a list of
**protected prefixes** (destination-relative paths the source scan pruned by
user-selection rules, which the walker must never delete unless
`--delete-excluded` opted out). Two config booleans were added for the wave:
`force_delete` (crosses the wire; the receiver clears a directory that blocks an
incoming file) and `ignore_errors` (client-only; the sender's scan continues
past an unreadable directory). `max_delete`'s default became -1 ("no client
limit"). These wire/layout changes bumped `PROTOCOL_VERSION` **2.8.0 → 2.9.0**
(peers must match). All four wire additions — `force_delete`,
`delete_excluded`, `prune_empty_dirs`, `max_delete` — round-trip unchanged and
are validated on receive.
frame carries the keep-set paths followed by a list of **protected prefixes**
(destination-relative paths the source scan pruned by user-selection rules,
which the walker must never delete unless `--delete-excluded` opted out).
Two config booleans were added for the wave: `force_delete` (crosses the wire;
the receiver clears a directory that blocks an incoming file) and
`ignore_errors` (client-only; the sender's scan continues past an unreadable
directory). `max_delete`'s default became -1 ("no client limit"). These
wire/layout changes bumped `PROTOCOL_VERSION` **2.8.0 → 2.9.0** (peers must
match). All four wire additions — `force_delete`, `delete_excluded`,
`prune_empty_dirs`, `max_delete` — round-trip unchanged and are validated on
receive.
**Missing-args note (Phase 3, missing-args wave):** `--ignore-missing-args` and
`--delete-missing-args` are implemented as described in the Safety & Security
rows. Wire impact: the `STATUS_MANIFEST` frame now carries a **third section** —
**Missing-args note (Phase 3, missing-args wave; extended in 2.23.0):**
`--ignore-missing-args` and `--delete-missing-args` are implemented as described
in the Safety & Security rows. Wire impact: the `STATUS_MANIFEST` frame carries
a list of destination-relative **exact-delete paths** (the missing entries'
mirrors) — and the config frame gained a `delete_missing_args` boolean
mirrors), and the config frame gained a `delete_missing_args` boolean
(`ignore_missing_args` stays client-only, exactly like `ignore_errors`). These
wire/layout changes bumped `PROTOCOL_VERSION` **2.9.0 → 2.10.0** (peers must
match). The receiver validates the third section identically to the keep-set
(non-empty, relative, traversal-free; `MAX_MANIFEST_ENTRIES` per section, a
single `MAX_MANIFEST_BYTES` budget shared across all three). On commit the
receiver runs the exact-path deletions FIRST (`manifest_delete_missing_args`:
confined per-path unlink/rmdir, deep removal only under `--force`/`--delete`,
staging/basis protected, never blocked by the protected-prefix list) and then
the ordinary extras walk when `--delete` is active (`manifest_delete_all`). A
client may request the exact-path deletions without `--delete`; the server's
`--allow-delete` policy gates them exactly like `--delete`, so an unauthorized
server ignores the request while the missing entries are still skipped.
match). The receiver validates the section identically to the keep-set (non-empty,
relative, traversal-free; the shared `MAX_MANIFEST_ENTRIES`/`MAX_MANIFEST_BYTES`
budget spans every section). On commit the receiver runs the exact-path deletions
FIRST (`manifest_delete_missing_args`: confined per-path unlink/rmdir, deep
removal only under `--force`/`--delete`, staging/basis protected, never blocked
by the protected-prefix list) and then the ordinary extras walk when `--delete`
is active (`manifest_delete_all`). A client may request the exact-path deletions
without `--delete`; the server's `--allow-delete` policy gates them exactly like
`--delete`, so an unauthorized server ignores the request while the missing
entries are still skipped.
The deletion walker is now **all-or-nothing**: before any unlink it rehearses
the deletion (an fd-relative walk identical to the delete pass, counting every
regular file it would unlink and every directory it would remove) and refuses to
start when the extras exceed the effective bound — a client `--max-delete=NUM`
below the hard bound, or the hard `MAX_SERVER_DELETE_COUNT` (100000) bound
itself. Previously the walker removed up to `MAX_SERVER_DELETE_COUNT` extras and
then reported an error (a truncated deletion); it now removes nothing and fails
with an error naming the bound. Directories count toward the bound. A directory
that still holds entries the walker leaves in place (a protected excluded file,
a kept manifest entry, a symlink) is left behind rather than failing the run —
matching rsync's "cannot delete non-empty directory" behaviour. The
all-or-nothing guarantee holds only while the destination is not concurrently
modified: rehearsal and delete are two separate walks, so a concurrent change
between them (another process adding or removing destination entries) can make
the actual deletion diverge from the counted set.
**Delete scoping and partial limits (protocol 2.23.0).** The `STATUS_MANIFEST`
frame now carries **four sections** — keep-set, protected prefixes, exact-delete
(missing-args) paths, and the set of **synchronized directories**. The extras
walk is scoped to the synchronized directories, so a `--files-from` subset no
longer deletes untransmitted destination paths outside the listed directory
subtrees (a data-loss fix, matching rsync). `--max-size`/`--min-size` pruned
source mirrors are protected independently of `--delete-excluded`. Extraneous
destination symlinks are unlinked by name (never followed). The `--max-delete`
budget is **partial**: the walker deletes up to the effective bound (a client
`--max-delete=NUM` below the hard bound, else the hard
`MAX_SERVER_DELETE_COUNT` = 100000) and then stops, skips the rest, and reports
the run as partial so the client exits **25** (`RERR_PARTIAL`) exactly like
rsync — it is a successful transfer with an incomplete deletion, not a hard
failure. The exact-path missing-args removals and the extras walk share that one
budget. A directory that still holds entries the walker leaves in place (a
protected excluded file, a kept manifest entry, a symlink) is left behind rather
than failing the run — matching rsync's "cannot delete non-empty directory"
behaviour.
Manifest size: the sender's keep-set and protected-prefix collections (streaming
or early pre-scan) are unbounded, but the receiver rejects a manifest beyond
`MAX_MANIFEST_ENTRIES` (1 048 576 entries, applied to EACH section — a frame can
therefore total up to 2 097 152 entries) / `MAX_MANIFEST_BYTES` (16 MB of paths,
counted across BOTH sections) as a hard protocol error. A heavily filtered
source whose exclusion list grows large thus fails the run cleanly on the
receiver (STATUS_ERROR) instead of being silently truncated. In the commit
modes this only means the deletion is refused after the data already arrived; in
the early modes (`--delete-before`/`--delete-during`) the manifest is the first
frame, so an oversized keep-set or protected list aborts the whole transfer
BEFORE any data is sent. Keep the source tree small enough for the receiver's
manifest caps when using the early timing.
Manifest size: the sender's collections (streaming or early pre-scan) are
unbounded, but the receiver rejects a manifest whose **aggregate** count exceeds
`MAX_MANIFEST_ENTRIES` (1 048 576 entries across ALL sections) or whose aggregate
path bytes exceed `MAX_MANIFEST_BYTES` (16 MB across all sections) as a hard
protocol error. A heavily filtered source whose exclusion list grows large thus
fails the run cleanly on the receiver (STATUS_ERROR) instead of being silently
truncated. In the commit modes this only means the deletion is refused after the
data already arrived; in the early modes (`--delete-before`/`--delete-during`)
the manifest is the first frame, so an oversized manifest aborts the whole
transfer BEFORE any data is sent. Keep the source tree small enough for the
receiver's manifest caps when using the early timing.
Early-delete ACK wait: after committing a large deletion (up to
`MAX_SERVER_DELETE_COUNT` removals) the receiver's `STATUS_OK`/`STATUS_ERROR`
@@ -238,33 +261,33 @@ why plain `--append` works on the normal atomic path, not only with `--inplace`.
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-M`, `--preserve` | Preserve file metadata | ✅ Implemented | Mode, uid, gid, mtime |
| `-p`, `--perms` | Preserve permissions | ✅ Implemented | Phase 7 Wave A: `-p`/`--perms` now preserve permission bits, folded into FastSync's broad metadata bundle (`--preserve`); the SSH port moved to `--ssh-port`. rsync-parity short form |
| `-o`, `--owner` | Preserve owner | ✅ Implemented | Part of -M |
| `-g`, `--group` | Preserve group | ✅ Implemented | Part of -M |
| `-t`, `--times` | Preserve modification times | ✅ Implemented | Part of -M |
| `-E`, `--executability` | Preserve executability | ✅ Implemented | Preserves executable permission bits (implies metadata preservation) |
| `--chmod=CHMOD` | Affect file permissions | ✅ Implemented | Supports numeric and symbolic `ugo` `rwx` changes; retains receiver safety masking |
| `-A`, `--acls` | Preserve ACLs | ✅ Implemented | Implemented on Linux via the POSIX-ACL xattr representation: the sender captures the `system.posix_acl_access` / `system.posix_acl_default` xattrs into the same bounded whitelisted set as `-X`, transmits them per-file, and the receiver re-applies them fd-relative. Setting an ACL the receiver is not permitted to set (non-root on a file it does not own, unsupported filesystem) is logged and skipped, never fatal. libacl is **not** required. Only the `system.posix_acl_*` namespaces plus `user.*` are ever applied; privileged namespaces are never applied (see the Phase-4 xattr/ACL notes below). Implies metadata transmission |
| `-X`, `--xattrs` | Preserve extended attributes | ✅ Implemented | Preserves unprivileged `user.*` extended attributes (Linux `listxattr`/`getxattr` on capture, `fsetxattr` on the written destination fd). Both capture (sender) and application (receiver) are restricted to the `user.*` namespace and the two POSIX ACL xattrs, so a client can **never** force a `security.*`/`trusted.*`/privileged attribute onto the destination; the receiver independently re-validates every incoming name against this whitelist and rejects anything else. Payloads are bounded (per-name ≤255B, per-value ≤1MiB, per-file count ≤256 total bytes ≤4MiB) on both ends, and an oversized/malformed frame is a clean protocol rejection (no OOM). Applied fd-relative to the exact written file. Implies metadata transmission. Incompatible with `-s` (chunk serialization), rejected up front (see the notes); a `--link-dest`/`-H` hard-link copy fallback re-applies the attributes so they are not dropped when a link is refused |
| `-H`, `--hard-links` | Preserve hard links | ✅ Implemented | Files on the source that share an inode (`st_dev`+`st_ino`, e.g. a `cp -al` tree) are re-created as hard links to one another on the destination, so duplicate links stay deduplicated and only the first member's data is sent (later members are transmitted as payload-less `STATUS_HARDLINK` frames). The receiver links each sibling to the first member's installed file with an atomic link + rename; on `link()` failure it falls back to a byte-identical local copy of the first member, never a partial/corrupt file. Requires the sequential scan for ordering (the first member is always emitted and installed before any sibling is linked). Works single-threaded and under `-j`/`--threads`, `--inplace`, `--delay-updates` (links staged and published by rename) and `--partial`. Crosses the wire (`preserve_hard_links` bool; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0**, peers must match). Incompatible with `-s` (chunk serialization) and `--append`/`--append-verify`, rejected up front with a distinct error. See the Phase-4 hard-links notes below |
| `-D` | Same as --devices --specials | ✅ Implemented | Implies `--devices --specials`. `-D` was unassigned in FastSync (verified: no collision), so it is free to imply both device-node and special-file preservation. See the `--devices`/`--specials` rows and the Phase-4 devices notes below |
| `--devices` | Preserve device files | ✅ Implemented | Recreates char/block device nodes on the destination via `mknod` instead of transferring content. Type + rdev are validated strictly (S_IFMT from the transmitted mode; major/minor range-checked, non-negative), and creation is **privilege-gated**: `mknod` needs `CAP_MKNOD`, so a non-root receiver (CI runs via setpriv as non-root) logs a warning and **skips the device entry safely** — the whole transfer never aborts just because the node could not be made. The node is created fd-relative below the receive root (`mknodat` on the confined secure parent), so it can never be placed outside the authorized root, never follows a symlink, and never replaces an existing directory. Only a char/block mode is honored. Crosses the wire (a new `STATUS_SPECIAL` frame carries the path + metadata mode + rdev; `PROTOCOL_VERSION` bumped **2.12.0 → 2.13.0**). Divergence: per-entry skip (not a hard error) when the receiver lacks `CAP_MKNOD`, documented in the Phase-4 devices notes |
| `--specials` | Preserve special files | ⛔ Impossible/Divergence | **FIFO recreation works**: FIFOs are recreated on the destination via `mkfifo` (unprivileged, so this is a real, assertable behavior under CI). **Only socket recreation is impossible**: a socket entry can be created only by `bind(2)` on a live socket, not by any filesystem call, so a source socket is skipped with an explicit note. That one unsupported node kind is why the flag is classified Impossible/Divergence even though FIFO recreation itself works; its normal path is otherwise complete. FIFO creation is privileged-gated only in the sense of graceful skip on any permission failure. Node creation is confined below the receive root (`mkfifoat` on the secure fd-relative parent; no `..`, no symlink follow). Crosses the wire like `--devices` (the `STATUS_SPECIAL` frame; `PROTOCOL_VERSION` bumped **2.12.0 → 2.13.0**). See the Phase-4 devices notes |
| `--copy-devices` | Copy device contents as file | ✅ Implemented | Copy a device's CONTENT into an ordinary regular file on the destination instead of recreating the node — non-privileged and safe. FastSync scans a device/FIFO as a regular file: its reported size (`st_size`, typically 0 for char devices and FIFOs) is copied, so a FIFO or a non-readable device becomes an empty (or size-bounded) regular file. The default data path is size-bounded and never blocks (it sends exactly `st_size` bytes, never an unbounded pseudo-device stream); with `--sendfile`, a non-regular source (FIFO/device) is detected from its `stat` mode and falls back to that same buffered read, so `--copy-devices --sendfile` cannot hang either. The run always succeeds and never crashes on such input. **Deliberate, safe divergence from rsync's dd-like unbounded device read.** See the Phase-4 devices notes |
| `--write-devices` | Write to devices as files | ✅ Implemented | Write the received data directly into an **existing** device node on the destination instead of creating a regular file. Restricted and best-effort: the destination must already exist and be a char/block device (opened only under the confined receive root, with `O_NOFOLLOW` + `O_NONBLOCK`); a missing, symlinked, FIFO-with-no-reader (`ENXIO`), non-device destination, or any write failure is **skipped with a warning** rather than allowed, so a run can never clobber the system, never blocks on a special-file target, and never aborts on an unusable target. See the Phase-4 devices notes |
| `-U`, `--atimes` | Preserve access times | ✅ Implemented | Captures the source access time (from the scanner's pre-read stat, so it is not clobbered by reading the file for transfer) and transmits it over the wire; the receiver restores it together with the mtime via `futimens`/`utimensat`. Implies metadata transmission (the times travel inside the `-M` metadata payload), but does not enable ownership application (that stays opt-in via the identity flags). Wire: new `atime` fields on the metadata frame + a `preserve_atimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** |
| `-N`, `--crtimes` | Preserve create times | ⛔ Impossible/Divergence | Birth-times cannot be set by any portable filesystem call (`utimensat`/`futimens` only set atime/mtime), so this row is an explicit **Impossible/Divergence** (Phase 7 Wave B). Capture + transmit stays: `statx(STATX_BTIME)` on Linux records the source birth time as a wire field; the receiver logs a debug note that it cannot be applied and continues — never failing the transfer and never pretending it worked. On platforms without `statx` it parses as a documented no-op (flag accepted; nothing is captured). Implies metadata transmission. Wire: new `crtime` fields + a `preserve_crtimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** (see the Phase-4 metadata-time notes) |
| `-O`, `--omit-dir-times` | Omit dirs from --times | ✅ Implemented | Real modifier now that FastSync preserves directory times. With metadata on, the scanner captures every traversed source directory's mtime (and atime under `-U`) and the sender transmits them in trailing `STATUS_DIR_TIMES` frame(s) **after all file data and the optional delete manifest** (chunked at the receiver's `MAX_MANIFEST_ENTRIES` per-frame cap); a dir-time entry only RECORDS metadata and never creates the directory, so empty source directories stay untransferred. The receiver defers applying them until its delete / `--delay-updates` publication phases have committed, so writing or removing a child never clobbers a parent directory's mtime (rsync applies directory times at the end for exactly this reason). When `-O` is set (the boolean crosses the wire) the receiver does not apply any of them; without `-O` an `-a`/`--preserve` transfer now restores directory times (reversing the old "never preserves dir times" divergence). Wire change: the terminal `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** |
| `-J`, `--omit-link-times` | Omit symlinks from --times | ✅ Implemented | Real modifier now that FastSync preserves symlink times. Symlink entries already carried their metadata on `STATUS_SYMLINK`; the receiver now applies it with **no-follow primitives only** (`utimensat(..., AT_SYMLINK_NOFOLLOW)`, plus best-effort `fchmodat(..., AT_SYMLINK_NOFOLLOW)` and policy-gated `fchownat(..., AT_SYMLINK_NOFOLLOW)`), so the link itself is stamped without ever dereferencing it, confined fd-relative below the authorized receive root. A symlink has no children, so the times are applied immediately at creation. When `-J` is set (the boolean crosses the wire) the receiver skips the timestamps (mode/ownership are unaffected); without `-J` an `-a`/`-l` transfer restores symlink mtimes. Wire change alongside `-O`: the shared `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** |
| `--super` | Receiver attempts super-user activities | ✅ Implemented | Phase 7 Wave E: receiver-side **safe-subset + clear-refusal** privilege model, tri-state `super_mode` (auto/on/off). `--super` **permits** the receiver to attempt super-user activities — ownership application and char/block device-node creation — that are already confined fd-relative below the authorized receive root; `--no-super` **forbids** them even when the receiver is root; the default (`auto`) preserves the pre-existing **best-effort** behavior of *attempting* them (not only when already root: an unprivileged attempt is refused by the kernel and skipped per entry, matching FastSync's history). The server additionally accepts an operator-level `--no-super` veto that forces `OFF` for every connection it accepts (so it also refuses any client `--copy-as`/`--super`); a **privileged (root) standalone TCP listener now also defaults to `OFF`** unless the operator opts in with the new server-only `--allow-super` flag (the flag is **rejected with `--stdio`**, whose remote argv is composed by the client and must never defeat the secure default; operators exposing `fastsync-server --stdio` over SSH need a forced command if the default must hold. An unprivileged receiver is unchanged, since the kernel refuses the confined attempts anyway; the `--daemon` path keeps its per-module `client owner = yes` opt-in); the `--fake-super` owner replay and the `--write-devices` write path are gated by the same policy. **FastSync never elevates**: no `setuid`/`seteuid`/`setgid` is ever called, and `--super` never bypasses the confinement floor (`file_open_secure_parent`, `O_NOFOLLOW`, root checks) — it only permits an attempt that is already confined. `--super` does **not** imply `--numeric-ids` and never enables client-chosen ownership on its own: ownership is applied only when an explicit identity policy (`--usermap`/`--groupmap`/`--chown`/`--numeric-ids`/`--copy-as`) is also given. A non-root receiver given `--super` logs exactly one warning at activation and each confined attempt is then refused by the kernel and skipped per entry (never aborts); `--no-super` suppresses ownership, char/block `mknod`, `--write-devices` and the fake-super owner replay, while unprivileged FIFO creation is unaffected. Wire: one trailing `super_mode` int on the config frame (validated 0..2), sent **before** the `--copy-as` block (fixed order: super int, then copy-as presence int + ids); `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Documented divergence from rsync:** rsync's `--super` runs the receiver with elevated privilege; FastSync only permits a confined attempt and never elevates |
| `--fake-super` | Store/recover privileged attrs via xattrs | ✅ Implemented | Phase 7 Wave B: full record **and replay**. The receiver writes the source `uid:gid:mode:mtime_sec:mtime_nsec` into a reserved `user.fastsync.stat` xattr on each written file (best-effort, fd-relative, format unchanged), then immediately re-applies it via `fake_super_restore_fd`: `fchown` (only where privileged — a non-root EPERM/EACCES is skipped silently, matching FastSync's identity philosophy), `fchmod`, and `futimens`. The OWNER leg is additionally skipped unless an explicit ownership identity policy (`--numeric-ids`/`--usermap`/`--groupmap`/`--chown`/`--copy-as`) is active — `--fake-super` on its own only *records* the source owner and must not act as an un-gated chown primitive — when `--no-super` forbids super-user activities (even for root), or when an active `--copy-as` is authoritative, so the recorded source owner can never override a forced `--copy-as` owner; the xattr record is still stored/replayed for a later privileged restore and mode/mtime still apply, so unprivileged `--fake-super` keeps working. The restored mode goes through the same sanitization as the normal metadata path (group/other write bits are never granted, so a recorded 0666 restores as 0644), so fake-super replay can never grant group/other-write that plain `--preserve` would refuse. Absence or a malformed record is a silent no-op, never fatal. The recording format diverges from rsync's `user.rsync.%stat%`; no cross-tool conversion is attempted. Implies metadata transmission so the source uid/gid/mode/mtime are available. Both it and `-X`/`-A` are incompatible with `-s` (chunk serialization), rejected up front |
| `--open-noatime` | Avoid changing access time when opening files | ✅ Implemented | Sender-side policy: the sender opens source files with `O_NOATIME` (Linux) when reading them for transfer, so the open/read does NOT bump the source's on-disk access time. Degrades safely when `O_NOATIME` is unavailable (not defined) or refused (`EPERM`, since it needs `CAP_FOWNER` or file ownership): the code falls back to a normal open, so the data always transfers — only the atime-bump is skipped. It does not itself capture/preserve atime; it only avoids modifying it. **Client-only, never crosses the wire.** Exposed as `file_open_for_read()` and applied to both the buffered data path and the sendfile path |
| `--numeric-ids` | Do not map uid/gid by name | ✅ Implemented | Ownership is applied through FastSync's opt-in identity path (see the Phase-4 identity notes below). `--numeric-ids` is a mapping-policy modifier: when applying ownership it uses the transmitted numeric uid/gid directly, skipping the name lookup. Without an ownership-affecting option it is inert (FastSync only applies ownership when the user opts in). It does not need `-M` to be parsed, but ownership is only applied when metadata (hence the source uid/gid) is actually transmitted (see the notes) |
| `--usermap=STRING` | Map usernames | ✅ Implemented | Opt-in ownership application. rsync subset implemented: comma-separated `FROM:TO` rules evaluated in order, first match wins; `FROM`/`TO` are group/user names (resolved on the SOURCE machine at parse time), `*` (FROM matches any id / TO = the receiving process's current euid), and an `@N` or bare `N` numeric id. Rules are carried over the wire as resolved numeric id pairs; the receiver applies a matching rule (else falls back to `--chown`, `--numeric-ids`, then a best-effort name lookup) via an fd-relative `fchown`. Malformed/unresolvable specs are rejected with a clear error, never a silent no-op. Implies metadata preservation so the source uid/gid travel. Only effective when the receiver can actually change ownership (root or membership); otherwise it warns and continues |
| `--groupmap=STRING` | Map group names | ✅ Implemented | Same rsync subset and semantics as `--usermap` but for the group (gid) side and the group databases. See the Phase-4 identity notes |
| `--chown=USER:GROUP` | Map owner and group | ✅ Implemented | Opt-in ownership override applied receiver-side. Forms: `USER:GROUP`, `USER` (owner only), `:GROUP` (group only); a `*` for USER/GROUP means the current/root user or group as appropriate; an `@N`/bare `N` numeric id is accepted. A `:` inside a name may be escaped as `\:`. Equivalent to a trailing `*:*` usermap+groupmap rule (so an explicit `--usermap`/`--groupmap` match wins). Malformed or unresolvable specs are clear parse errors. Implies metadata preservation. Only effective when the receiver has permission to chown; otherwise it warns and continues (rsync parity) |
| `--copy-as=USER[:GROUP]` | Perform the copy as another user/group | ✅ Implemented | Safe-subset implementation, an explicit divergence from rsync's **real identity switching**. rsync makes the receiving process actually assume USER/GROUP (setuid/setgid); FastSync's receiver is multithreaded, so a real credential drop would be unsafe and is never attempted — FastSync never calls `setuid`/`seteuid`/`setgid`. Instead the receiver FORCES the ownership of every entry it writes to `copy_as_uid`/`copy_as_gid` through the existing confined, fd-relative identity path (the same `fchown`/`fchownat` mechanism as `--chown`/`--usermap`/`--groupmap`; symlinks use `fchownat(..., AT_SYMLINK_NOFOLLOW)`, and directories — including intermediate parents created implicitly while writing a nested file — and char/block/FIFO nodes are owned no-follow too, so a directory never keeps the receiver's owner while its children get the target owner), with `--copy-as` at the **highest priority** — it beats usermap/groupmap/`--chown`/`--numeric-ids` and the best-effort name lookup. This REQUIRES a privileged (root) receiver: an unprivileged receiver REFUSES the whole transfer up front at the config handshake (`server_module_gate`, running inside `config_receive_with_validate` before the `STATUS_OK` ack) with a clear error and no file data exchanged — never a silent wrong-ownership result. A server running with an operator `--no-super` veto also refuses it; a privileged (root) standalone TCP listener refuses it by default too and only honors it after the operator passes `--allow-super` (the flag is rejected with `--stdio`, where the client-composed remote argv could otherwise defeat the default; a forced command is required if the default must hold), and a **daemon** refuses `--copy-as`, like every other client-chosen-ownership request (`--numeric-ids`/`--chown`/`--usermap`/`--groupmap`/`--fake-super`/explicit `--super`), unless the selected module opts in with `client owner = yes`; without that per-module opt-in a daemon must not honor an arbitrary client-selected owner (a root standalone listener honors these for its single operator-authorized root only when started with `--allow-super`). `--fake-super` interaction: `--copy-as` is authoritative, so the recorded source owner is never replayed over the forced target owner. If the ownership apply still fails with EPERM/EACCES (capability-restricted root, root-squash, read-only mount) the failure is logged at ERROR and the **entry is reported as failed** rather than written with the wrong owner, which fails the transfer (fail-fast) so overall success is never reported with the wrong owner. USER is resolved on the client against the user database (a name, an `@N`/bare `N` numeric id, or `*` meaning the client's current euid); when `:GROUP` is present it is resolved against the group database (`*` meaning the client's egid). **Group-default rule:** when the group is omitted FastSync uses the user's primary gid (`getpwuid(uid)->pw_gid`); a numeric id with no local passwd entry has no primary gid to look up, so `gid` falls back to `uid` (documented divergence). Malformed/empty/unresolvable specs are clear parse errors, never a silent no-op. Never elevates privileges and never bypasses the confined receive root. Implies metadata preservation (the source uid/gid must be transmitted). Wire: a new trailing config-frame block **sent after** the `--super` int (presence int, then the two int32 ids, both validated `>= 0` on receive; the ids are also rejected if they do not fit int32 at CLI parse time); `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0** |
| `--preserve` | (FastSync alias, not an rsync flag) | ✅ Parity | **FastSync-only alias** for `-p` + `-t` (mode + mtime), long-form only. It is not rsync's `--preserve` (rsync has no such option); the short `-M` that used to spell it is now rsync's `--remote-option`. The wire metadata also carries uid/gid for `-o`/`-g`/`-a`, and ownership is applied via `-o`/`-g`, `-a`, or an explicit identity flag (`--numeric-ids`/`--usermap`/`--groupmap`/`--chown`/`--copy-as`) |
| `-p`, `--perms` | Preserve permissions | ✅ Parity | Real per-attribute flag (protocol 2.22.0): `preserve_perms` applies the source mode independently of times/owner/group. **Strict rsync parity (protocol 2.23.0): the source mode is copied exactly, including setuid/setgid/sticky and group/other-write bits — there is no masking.** Without `-p`, a new file gets `source_mode & ~umask` when metadata is present (else the historical fixed `0644`); new directories without `-p` still use FastSync's `0755` creation default, because directory metadata is only applied when a directory attribute is requested. `-A/--acls` implies `-p`; `--chmod` does **not** imply `-p` (rsync parity) and applies its own unsanitized changes to the new mode. `-X/--xattrs` does not imply `-p`. The SSH port moved to `--ssh-port`. rsync-parity short form |
| `-o`, `--owner` | Preserve owner | ✅ Parity | Real per-attribute flag (`preserve_owner`): preserve the source uid, resolved on the receiver by name against its own user database with a raw-numeric fallback (only numeric ids cross the wire). `--usermap`/`--chown=USER` imply it. Application follows the `--super`/`--no-super` policy; a non-opted daemon module applies no ownership (see the Daemon Mode notes) |
| `-g`, `--group` | Preserve group | ✅ Parity | Real per-attribute flag (`preserve_group`): preserve the source gid, resolved by name on the receiver with a raw-numeric fallback. `--groupmap`/`--chown=:GROUP` imply it. Same privilege/super-policy gating as `-o` |
| `-t`, `--times` | Preserve modification times | ✅ Parity | Real per-attribute flag (`preserve_times`): apply the source mtime independently of the other attributes. `-O/--omit-dir-times` suppresses directories only and `-J/--omit-link-times` suppresses symlinks only; `-U`/`-N` do not imply it. `--preserve`/`-a` imply it, and `--incremental`/`--delta` auto-enable it unless `--no-times`/`--no-preserve` |
| `-E`, `--executability` | Preserve executability | ✅ Parity | Preserves executable permission bits (implies metadata preservation) |
| `--chmod=CHMOD` | Affect file permissions | ✅ Parity | Faithful port of rsync 3.4.1's `parse_chmod`/`tweak_mode`: numeric octal and symbolic `ugo`/`rwx` changes, `D`/`F` directory/file selectors, `X` (execute only on directories or already-executable files), `s`/`t` setuid/setgid/sticky, and append semantics — repeated clauses and repeated `--chmod` options accumulate in order (joined with commas). The changes are applied to the new mode **without sanitization** (matching rsync) and `--chmod` does **not** imply `-p` (rsync parity). Applied to files and directories on the receiver |
| `-A`, `--acls` | Preserve ACLs | ⚠️ Caveat | Implemented on Linux via the POSIX-ACL xattr representation: the sender captures the `system.posix_acl_access` / `system.posix_acl_default` xattrs into the same bounded whitelisted set as `-X`, transmits them per-file, and the receiver re-applies them fd-relative. Setting an ACL the receiver is not permitted to set (non-root on a file it does not own, unsupported filesystem) is logged and skipped, never fatal. libacl is **not** required. Only the `system.posix_acl_*` namespaces plus `user.*` are ever applied; privileged namespaces are never applied (see the Phase-4 xattr/ACL notes below). Implies metadata transmission |
| `-X`, `--xattrs` | Preserve extended attributes | ⚠️ Caveat | Preserves unprivileged `user.*` extended attributes (Linux `listxattr`/`getxattr` on capture, `fsetxattr` on the written destination fd). Both capture (sender) and application (receiver) are restricted to the `user.*` namespace and the two POSIX ACL xattrs, so a client can **never** force a `security.*`/`trusted.*`/privileged attribute onto the destination; the receiver independently re-validates every incoming name against this whitelist and rejects anything else. Payloads are bounded (per-name ≤255B, per-value ≤1MiB, per-file count ≤256 total bytes ≤4MiB) on both ends, and an oversized/malformed frame is a clean protocol rejection (no OOM). Applied fd-relative to the exact written file. Implies metadata transmission. Incompatible with `-s` (chunk serialization), rejected up front (see the notes); a `--link-dest`/`-H` hard-link copy fallback re-applies the attributes so they are not dropped when a link is refused |
| `-H`, `--hard-links` | Preserve hard links | ✅ Parity | Files on the source that share an inode (`st_dev`+`st_ino`, e.g. a `cp -al` tree) are re-created as hard links to one another on the destination, so duplicate links stay deduplicated and only the first member's data is sent (later members are transmitted as payload-less `STATUS_HARDLINK` frames). The receiver links each sibling to the first member's installed file with an atomic link + rename; on `link()` failure it falls back to a byte-identical local copy of the first member, never a partial/corrupt file. Requires the sequential scan for ordering (the first member is always emitted and installed before any sibling is linked). Works single-threaded and under `-j`/`--threads`, `--inplace`, `--delay-updates` (links staged and published by rename) and `--partial`. Crosses the wire (`preserve_hard_links` bool; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0**, peers must match). Incompatible with `-s` (chunk serialization) and `--append`/`--append-verify`, rejected up front with a distinct error. See the Phase-4 hard-links notes below |
| `-D` | Same as --devices --specials | ✅ Parity | Implies `--devices --specials`. `-D` was unassigned in FastSync (verified: no collision), so it is free to imply both device-node and special-file preservation. As of protocol 2.23.0 `--specials` genuinely covers **both FIFOs and unix sockets**, so `-D` covers the full rsync set. See the `--devices`/`--specials` rows and the Phase-4 devices notes below |
| `--devices` | Preserve device files | ⚠️ Caveat | Recreates char/block device nodes on the destination via `mknod` instead of transferring content. Type + rdev are validated strictly (S_IFMT from the transmitted mode; major/minor range-checked, non-negative), and creation is **privilege-gated**: `mknod` needs `CAP_MKNOD`, so a non-root receiver (CI runs via setpriv as non-root) logs a warning and **skips the device entry safely** — the whole transfer never aborts just because the node could not be made. The node is created fd-relative below the receive root (`mknodat` on the confined secure parent), so it can never be placed outside the authorized root, never follows a symlink, and never replaces an existing directory. Only a char/block mode is honored. Crosses the wire (a `STATUS_SPECIAL` frame carries the path + metadata mode + rdev). Divergence: per-entry skip (not a hard error) when the receiver lacks `CAP_MKNOD`, documented in the Phase-4 devices notes |
| `--specials` | Preserve special files | ✅ Parity | **FIFO and unix-socket recreation work** (protocol 2.23.0): FIFOs are recreated with `mkfifoat`, and sockets with `mknodat(..., S_IFSOCK)` — the latter is unprivileged on Linux because it materializes the socket *node*, not a live bound socket, so it is a real, assertable behavior under CI (it matches rsync, which also recreates a socket by `mknod`). Node creation is confined below the receive root (fd-relative parent; no `..`, no symlink follow) and type/rdev are validated strictly; a matching existing node is left in place and an unrelated entry is never replaced. Crosses the wire like `--devices` (the `STATUS_SPECIAL` frame). See the Phase-4 devices notes |
| `--copy-devices` | Copy device contents as file | ⚠️ Caveat | Copy a device's CONTENT into an ordinary regular file on the destination instead of recreating the node — non-privileged and safe. FastSync scans a device/FIFO as a regular file: its reported size (`st_size`, typically 0 for char devices and FIFOs) is copied, so a FIFO or a non-readable device becomes an empty (or size-bounded) regular file. The default data path is size-bounded and never blocks (it sends exactly `st_size` bytes, never an unbounded pseudo-device stream); with `--sendfile`, a non-regular source (FIFO/device) is detected from its `stat` mode and falls back to that same buffered read, so `--copy-devices --sendfile` cannot hang either. The run always succeeds and never crashes on such input. **Deliberate, safe divergence from rsync's dd-like unbounded device read.** See the Phase-4 devices notes |
| `--write-devices` | Write to devices as files | ⚠️ Caveat | Write the received data directly into an **existing** device node on the destination instead of creating a regular file. Restricted and best-effort: the destination must already exist and be a char/block device (opened only under the confined receive root, with `O_NOFOLLOW` + `O_NONBLOCK`); a missing, symlinked, FIFO-with-no-reader (`ENXIO`), non-device destination, or any write failure is **skipped with a warning** rather than allowed, so a run can never clobber the system, never blocks on a special-file target, and never aborts on an unusable target. See the Phase-4 devices notes |
| `-U`, `--atimes` | Preserve access times | ✅ Parity | Captures the source access time (from the scanner's pre-read stat, so it is not clobbered by reading the file for transfer) and transmits it over the wire; the receiver restores it together with the mtime via `futimens`/`utimensat`. Implies metadata transmission (the times travel inside the shared metadata payload), but does not enable ownership application (that stays opt-in via the identity flags). Wire: `atime` fields on the metadata frame + a `preserve_atimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** |
| `-N`, `--crtimes` | Preserve create times | ❌ Divergent | Birth-times cannot be set by any portable filesystem call (`utimensat`/`futimens` only set atime/mtime), so this row is an explicit **Divergent** entry (Phase 7 Wave B). Capture + transmit stays: `statx(STATX_BTIME)` on Linux records the source birth time as a wire field; the receiver logs a debug note that it cannot be applied and continues — never failing the transfer and never pretending it worked. On platforms without `statx` it parses as a documented no-op (flag accepted; nothing is captured). Implies metadata transmission. Wire: new `crtime` fields + a `preserve_crtimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** (see the Phase-4 metadata-time notes) |
| `-O`, `--omit-dir-times` | Omit dirs from --times | ✅ Parity | Real modifier now that FastSync preserves directory times. With metadata on, the scanner captures every traversed source directory's mtime (and atime under `-U`) and the sender transmits them in trailing `STATUS_DIR_TIMES` frame(s) **after all file data and the optional delete manifest** (chunked at the receiver's `MAX_MANIFEST_ENTRIES` per-frame cap); a dir-time entry only RECORDS metadata and never creates the directory, so empty source directories stay untransferred. The receiver defers applying them until its delete / `--delay-updates` publication phases have committed, so writing or removing a child never clobbers a parent directory's mtime (rsync applies directory times at the end for exactly this reason). When `-O` is set (the boolean crosses the wire) the receiver does not apply any of them; without `-O` an `-a`/`--preserve` transfer now restores directory times (reversing the old "never preserves dir times" divergence). Wire change: the terminal `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** |
| `-J`, `--omit-link-times` | Omit symlinks from --times | ✅ Parity | Real modifier now that FastSync preserves symlink times. Symlink entries already carried their metadata on `STATUS_SYMLINK`; the receiver now applies it with **no-follow primitives only** (`utimensat(..., AT_SYMLINK_NOFOLLOW)`, plus best-effort `fchmodat(..., AT_SYMLINK_NOFOLLOW)` and policy-gated `fchownat(..., AT_SYMLINK_NOFOLLOW)`), so the link itself is stamped without ever dereferencing it, confined fd-relative below the authorized receive root. A symlink has no children, so the times are applied immediately at creation. When `-J` is set (the boolean crosses the wire) the receiver skips the timestamps (mode/ownership are unaffected); without `-J` an `-a`/`-l` transfer restores symlink mtimes. Wire change alongside `-O`: the shared `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** |
| `--super` | Receiver attempts super-user activities | ⚠️ Caveat | Phase 7 Wave E: receiver-side **safe-subset + clear-refusal** privilege model, tri-state `super_mode` (auto/on/off). `--super` **permits** the receiver to attempt super-user activities — ownership application and char/block device-node creation — that are already confined fd-relative below the authorized receive root; `--no-super` **forbids** them even when the receiver is root; the default (`auto`) preserves the pre-existing **best-effort** behavior of *attempting* them (not only when already root: an unprivileged attempt is refused by the kernel and skipped per entry, matching FastSync's history). The server additionally accepts an operator-level `--no-super` veto that forces `OFF` for every connection it accepts (so it also refuses any client `--copy-as`/`--super`); a **privileged (root) standalone TCP listener now also defaults to `OFF`** unless the operator opts in with the new server-only `--allow-super` flag (the flag is **rejected with `--stdio`**, whose remote argv is composed by the client and must never defeat the secure default; operators exposing `fastsync-server --stdio` over SSH need a forced command if the default must hold. An unprivileged receiver is unchanged, since the kernel refuses the confined attempts anyway; the `--daemon` path keeps its per-module `client owner = yes` opt-in); the `--fake-super` owner replay and the `--write-devices` write path are gated by the same policy. **FastSync never elevates**: no `setuid`/`seteuid`/`setgid` is ever called, and `--super` never bypasses the confinement floor (`file_open_secure_parent`, `O_NOFOLLOW`, root checks) — it only permits an attempt that is already confined. `--super` does **not** imply `--numeric-ids` and never enables client-chosen ownership on its own: ownership is applied only when an explicit identity policy (`--usermap`/`--groupmap`/`--chown`/`--numeric-ids`/`--copy-as`) or a preserve-source request (`-o`/`-g`, or `-a`/`--archive`) is also given. A non-root receiver given `--super` logs exactly one warning at activation and each confined attempt is then refused by the kernel and skipped per entry (never aborts); `--no-super` suppresses ownership, char/block `mknod`, `--write-devices` and the fake-super owner replay, while unprivileged FIFO creation is unaffected. Wire: one trailing `super_mode` int on the config frame (validated 0..2), sent **before** the `--copy-as` block (fixed order: super int, then copy-as presence int + ids); `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Documented divergence from rsync:** rsync's `--super` runs the receiver with elevated privilege; FastSync only permits a confined attempt and never elevates |
| `--fake-super` | Store/recover privileged attrs via xattrs | ⚠️ Caveat | Full record **and replay** (protocol 2.23.0 parity update). The receiver writes the resolved `uid:gid:mode:mtime_sec:mtime_nsec` into a reserved `user.fastsync.stat` xattr on each written file (best-effort, fd-relative), then immediately re-applies the mode and times via `fake_super_restore_fd` (`fchmod` + `futimens`; absent/malformed records are a silent no-op, never fatal). **`--fake-super` never performs a real `chown`**: when an explicit ownership mapping (`--chown`/`--usermap`/`--groupmap`/`--copy-as`) is active the receiver records the *resolved* id, otherwise the source's own id, but the owner leg is always suppressed so recording can never defeat the flag; the record is retained for a later privileged restore. The replayed mode goes through the shared `metadata_mode_for_policy` helper, so under `-p` it is copied exactly (including group/other-write and special bits — strict rsync parity, no masking) and under `-E` it follows the rsync executability rule. Directory ownership and directory xattrs/ACLs are preserved alongside file entries (mode/owner are applied to directories under the same per-attribute policy and `-A`/`-X` carry the directory ACL/xattr block). Implies metadata transmission so the source uid/gid/mode/mtime are available. The recording format diverges from rsync's `user.rsync.%stat%`; no cross-tool conversion is attempted. Both it and `-X`/`-A` are incompatible with `-s` (chunk serialization), rejected up front |
| `--open-noatime` | Avoid changing access time when opening files | ✅ Parity | Sender-side policy: the sender opens source files with `O_NOATIME` (Linux) when reading them for transfer, so the open/read does NOT bump the source's on-disk access time. Degrades safely when `O_NOATIME` is unavailable (not defined) or refused (`EPERM`, since it needs `CAP_FOWNER` or file ownership): the code falls back to a normal open, so the data always transfers — only the atime-bump is skipped. It does not itself capture/preserve atime; it only avoids modifying it. **Client-only, never crosses the wire.** Exposed as `file_open_for_read()` and applied to both the buffered data path and the sendfile path |
| `--numeric-ids` | Do not map uid/gid by name | ✅ Parity | **A mapping modifier only:** when ownership is being applied it uses the transmitted numeric uid/gid directly, skipping the name lookup. It does **not** request ownership application on its own — combine it with `-o`/`-g`, `-a`, or an explicit map (`--chown`/`--usermap`/`--groupmap`) — and it does not need any metadata flag merely to parse. Ownership is only applied when metadata (hence the source uid/gid) is actually transmitted (see the Phase-4 identity notes) |
| `--usermap=STRING` | Map usernames | ⚠️ Caveat | Opt-in ownership application. rsync subset implemented (protocol 2.23.0): comma-separated `FROM:TO` rules evaluated in order, first match wins. `FROM` accepts a user name (resolved on the SOURCE machine at parse time), an `@N`/bare `N` numeric id, an inclusive `LOW-HIGH` **id range**, `*` (matches any id), or an **empty** field (matches ids with no name on the source). `TO` accepts a name (resolved on the **receiver**), an `@N`/bare `N` id, or `*` (the receiving process's current euid). Rules are carried over the wire as resolved numeric id pairs; the receiver applies a matching rule (else falls back to `--chown`, `--numeric-ids`, then a best-effort name lookup) via an fd-relative `fchown`, including directory entries. Malformed/unresolvable specs are rejected with a clear error, never a silent no-op. Implies metadata preservation so the source uid/gid travel. Only effective when the receiver can actually change ownership (root or membership); otherwise it warns and continues |
| `--groupmap=STRING` | Map group names | ⚠️ Caveat | Same rsync subset and semantics as `--usermap` (names, `@N`/bare `N`, inclusive ranges, `*`, empty-FROM for unnamed ids, receiver-resolved `TO` names) but for the group (gid) side and the group databases. See the Phase-4 identity notes |
| `--chown=USER:GROUP` | Map owner and group | ⚠️ Caveat | Opt-in ownership override applied receiver-side. Forms: `USER:GROUP`, `USER` (owner only), `:GROUP` (group only); a `*` for USER/GROUP means the current/root user or group as appropriate; an `@N`/bare `N` numeric id is accepted. A `:` inside a name may be escaped as `\:`. Equivalent to a trailing `*:*` usermap+groupmap rule (so an explicit `--usermap`/`--groupmap` match wins). **Protocol 2.23.0 makes `--chown` and `--usermap`/`--groupmap` mutually exclusive on the same side: combining them (in either order) is a clear configuration error** (`--usermap conflicts with prior --chown`), matching rsync and never an order-dependent silent winner. Malformed or unresolvable specs are clear parse errors. Implies metadata preservation. Only effective when the receiver has permission to chown; otherwise it warns and continues (rsync parity) |
| `--copy-as=USER[:GROUP]` | Perform the copy as another user/group | ⚠️ Caveat | Safe-subset implementation, an explicit divergence from rsync's **real identity switching**. rsync makes the receiving process actually assume USER/GROUP (setuid/setgid); FastSync's receiver is multithreaded, so a real credential drop would be unsafe and is never attempted — FastSync never calls `setuid`/`seteuid`/`setgid`. Instead the receiver FORCES the ownership of every entry it writes to `copy_as_uid`/`copy_as_gid` through the existing confined, fd-relative identity path (the same `fchown`/`fchownat` mechanism as `--chown`/`--usermap`/`--groupmap`; symlinks use `fchownat(..., AT_SYMLINK_NOFOLLOW)`, and directories — including intermediate parents created implicitly while writing a nested file — and char/block/FIFO nodes are owned no-follow too, so a directory never keeps the receiver's owner while its children get the target owner), with `--copy-as` at the **highest priority** — it beats usermap/groupmap/`--chown`/`--numeric-ids` and the best-effort name lookup. This REQUIRES a privileged (root) receiver: an unprivileged receiver REFUSES the whole transfer up front at the config handshake (`server_module_gate`, running inside `config_receive_with_validate` before the `STATUS_OK` ack) with a clear error and no file data exchanged — never a silent wrong-ownership result. A server running with an operator `--no-super` veto also refuses it; a privileged (root) standalone TCP listener refuses it by default too and only honors it after the operator passes `--allow-super` (the flag is rejected with `--stdio`, where the client-composed remote argv could otherwise defeat the default; a forced command is required if the default must hold), and a **daemon** refuses `--copy-as`, like every other client-chosen-ownership request (`--numeric-ids`/`--chown`/`--usermap`/`--groupmap`/`--fake-super`/explicit `--super`), unless the selected module opts in with `client owner = yes`; without that per-module opt-in a daemon must not honor an arbitrary client-selected owner (a root standalone listener honors these for its single operator-authorized root only when started with `--allow-super`). `--fake-super` interaction: `--copy-as` is authoritative, so the recorded source owner is never replayed over the forced target owner. If the ownership apply still fails with EPERM/EACCES (capability-restricted root, root-squash, read-only mount) the failure is logged at ERROR and the **entry is reported as failed** rather than written with the wrong owner, which fails the transfer (fail-fast) so overall success is never reported with the wrong owner. USER is resolved on the client against the user database (a name, an `@N`/bare `N` numeric id, or `*` meaning the client's current euid); when `:GROUP` is present it is resolved against the group database (`*` meaning the client's egid). **Group-default rule:** when the group is omitted FastSync uses the user's primary gid (`getpwuid(uid)->pw_gid`); a numeric id with no local passwd entry has no primary gid to look up, so `gid` falls back to `uid` (documented divergence). Malformed/empty/unresolvable specs are clear parse errors, never a silent no-op. Never elevates privileges and never bypasses the confined receive root. Implies metadata preservation (the source uid/gid must be transmitted). Wire: a new trailing config-frame block **sent after** the `--super` int (presence int, then the two int32 ids, both validated `>= 0` on receive; the ids are also rejected if they do not fit int32 at CLI parse time); `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0** |
**Phase-4 metadata-time notes:** `-U/--atimes`, `-N/--crtimes`,
`-O/--omit-dir-times`, `-J/--omit-link-times`, and `--open-noatime` are new.
@@ -326,8 +349,13 @@ match, exactly as prior phases did).
fatal.
- **`--fake-super`**: see the row above; the reserved key is `user.fastsync.stat`
with the documented `uid:gid:mode:mtime_sec:mtime_nsec` (mode octal) format.
It is honest but partial — there is no replay, and it does not interoperate
with rsync's `user.rsync.%stat%`.
**Replay exists**: after each stored record the receiver immediately re-applies
the recorded mode and times fd-relative (`fake_super_restore_fd`), but it
deliberately never performs a real `chown` — `--fake-super` only *records*
the resolved owner (the active `--chown`/`--usermap`/`--groupmap`/`--copy-as`
mapping when one is in effect, otherwise the source's own id) for a later
privileged restore. The recording format diverges from rsync's
`user.rsync.%stat%`; no cross-tool conversion is attempted.
- **Chunk serialization (`-s`) incompatibility:** the per-file xattr block rides
the streaming per-file frame, which `-s` replaces with a fixed buffer format,
so `-X` / `-A` combined with `-s` is rejected up front on both ends (mirroring
@@ -359,7 +387,7 @@ symlink timestamps (ownership/mode application is unaffected and stays governed
by the identity opt-in). Both config booleans already crossed the wire. See the
`-O`/`-J` rows and the Wave D note below.
**-U/-N and -M interaction:** because FastSync carries all metadata (mode, uid,
**-U/-N and metadata-bundle interaction:** because FastSync carries all metadata (mode, uid,
gid, mtime, and now atime/crtime) in one bounded payload that is only sent when
metadata transmission is on, `-U` and `-N` imply metadata transmission (the
times travel inside that payload). They do **not** enable ownership application,
@@ -458,19 +486,22 @@ CI runs the integration suite as a NON-ROOT user (via setpriv), so `mknod` fails
with `EPERM`. The receiver treats this as a graceful, logged *skip of the entry*
returned as a success/skip outcome — the whole transfer NEVER aborts just because
the environment cannot create the node. `mkfifo` (FIFOs) is unprivileged, so
`--specials` FIFO creation is a real, assertable behavior under CI; sockets cannot
be recreated by any standard filesystem call and are skipped with an explicit
note. The "device actually created" integration assertions are guarded to run
only as root. User-facing expectation: point `--devices` at devices and a
non-root receiver will faithfully skip them while transferring everything else.
`--specials` FIFO creation is a real, assertable behavior under CI. **Sockets are
recreated too** (protocol 2.23.0) with `mknodat(..., S_IFSOCK)`: Linux allows an
unprivileged `mknod` of a socket node because no live bound socket is created,
so a source socket materializes as a socket-type filesystem entry exactly as
rsync does. The "device actually created" integration assertions are guarded to
run only as root. User-facing expectation: point `--devices` at devices and a
non-root receiver will faithfully skip them while transferring everything else;
`--specials` recreates FIFOs and socket nodes for any receiver.
**Confinement & validation:** a special/device node is created with
`mknodat`/`mkfifoat` on the parent directory opened fd-relative below the receive
root (`file_open_secure_parent`: `O_NOFOLLOW`, no `..` components, root-checked),
so a node can never be created outside the authorized destination root and never
through a symlinked parent. The transmitted type is derived ONLY from the
validated S_IFMT bits of the metadata mode (char/block/FIFO honored, socket
skipped, regular/dir rejected as an invalid special), and the transmitted rdev is
validated S_IFMT bits of the metadata mode (char/block/FIFO and socket honored;
regular/dir rejected as an invalid special), and the transmitted rdev is
validated both on the wire (`file_receive_special`, `chunk_deserialize`) and at
the creation site (`file_special_rdev_valid`): a negative, oversize, or
non-device-carrying rdev is rejected outright (receiver aborts the frame), and a
@@ -496,13 +527,13 @@ warning + skip, never a system-clobbering write or an abort.
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-l`, `--links` | Copy symlinks as symlinks | ✅ Implemented | A symlink is transmitted as a real symlink: its target string crosses the wire (a new `STATUS_SYMLINK` frame / chunk entry type) and the receiver creates it with `symlinkat` beneath the receive root. This makes the previously-`-l`-included-but-targetless symlink handling complete. See the Phase-4 symlink-trust notes |
| `-L`, `--copy-links` | Transform symlink to referent | ✅ Implemented | `copy_links` config field |
| `--copy-unsafe-links` | Transform unsafe symlinks | ✅ Implemented | `copy_unsafe_links` config field |
| `--safe-links` | Ignore symlinks outside tree | ✅ Implemented | `safe_links` config field |
| `--munge-links` | Munge symlinks for safety | ✅ Implemented | Sender rewrites each transmitted symlink target with a `#SYMLINK/` marker; a target that could escape the receive root (absolute or containing `..`) is never transmitted (contained/skipped); the receiver strips the marker to restore the real target. See the Phase-4 symlink-trust notes |
| `-k`, `--copy-dirlinks` | Transform symlink to dir | ✅ Implemented | A symlink whose referent is a directory is dereferenced and recursed as a real directory; a symlink to a regular file stays a symlink. Sender-side only. See the Phase-4 symlink-trust notes |
| `-K`, `--keep-dirlinks` | Treat symlinked dir as dir | ✅ Implemented | On the receiver, an existing destination symlink-to-a-directory is used as that directory (followed) instead of being replaced; it is followed only when it resolves to a directory that stays beneath the receive root. See the Phase-4 symlink-trust notes |
| `-l`, `--links` | Copy symlinks as symlinks | ⚠️ Caveat | A symlink is transmitted as a real symlink: its target string crosses the wire (`STATUS_SYMLINK` / chunk entry type) and the receiver creates it with `symlinkat` beneath the receive root, never following the target. **Targets are stored verbatim (protocol 2.23.0), matching rsync `-l`: an absolute target or one containing `..` is copied exactly, and the receiver no longer enforces a containment predicate by default.** `--safe-links` is the sender-side opt-in that drops unsafe targets before transmission; `--trust-sender` does **not** affect symlink targets (it only relaxes the receiver's path-list re-validation). The *placement* path is still hard-confined (`has_path_traversal`, O_NOFOLLOW fd walk), and the link's own mode/times are applied with no-follow primitives. See the Phase-4 symlink-trust notes and the residual-risk note below |
| `-L`, `--copy-links` | Transform symlink to referent | ⚠️ Caveat | Sender-side: every symlink is replaced by its referent's content (`copy_links` config field). A referent that cannot be read, including a broken symlink, is treated as a non-error and the run exits 0 — where rsync exits 23 (`RERR_PARTIAL`). This is the documented status-code divergence |
| `--copy-unsafe-links` | Transform unsafe symlinks | ⚠️ Caveat | Sender-side: only symlinks whose target is unsafe (absolute or escaping via `..`, matching rsync's `unsafe_symlink()` semantics) are dereferenced into their referent; safe links stay symlinks. Same broken-referent exit-0 caveat as `-L` (`copy_unsafe_links` config field) |
| `--safe-links` | Ignore symlinks outside tree | ✅ Parity | Sender-side: a symlink whose target is unsafe is not transmitted at all (skipped), matching rsync's `--safe-links`. Because FastSync applies this while scanning the source, the receiver does not need to repeat it (`safe_links` config field) |
| `--munge-links` | Munge symlinks for safety | ✅ Parity | Sender rewrites each transmitted symlink target with rsync's `/rsyncd-munged/` prefix; the receiver strips the marker (only when the negotiated `munge_links` policy is on, so a source link that genuinely begins with the marker round-trips verbatim) and restores the exact real target. Unlike rsync, FastSync prefixes on the *sender* and un-munges on the receiver, but the wire result and the stored marker match rsync. See the Phase-4 symlink-trust notes |
| `-k`, `--copy-dirlinks` | Transform symlink to dir | ✅ Parity | A symlink whose referent is a directory is dereferenced and recursed as a real directory; a symlink to a regular file stays a symlink. Sender-side only. See the Phase-4 symlink-trust notes |
| `-K`, `--keep-dirlinks` | Treat symlinked dir as dir | ✅ Parity | On the receiver, an existing destination symlink-to-a-directory is used as that directory (followed) instead of being replaced; it is followed only when it resolves to a directory that stays beneath the receive root. See the Phase-4 symlink-trust notes |
**Phase-4 symlink-trust notes:** `-l/--links`, `-k/--copy-dirlinks`,
`-K/--keep-dirlinks`, and `--munge-links` form the "symlink trust boundaries"
@@ -518,17 +549,20 @@ was bumped **2.12.0 → 2.13.0** (peers must match, exactly as prior phases did)
**Per-flag semantics and divergences.**
- **`-l/--links`** copies a symlink as a symlink: the scanner `readlink`s the
target, the sender transmits it, and the receiver `symlinkat`s it. FastSync
`-l` never preserved symlink targets before (the flag was documented partial
and, in fact, tried to read the referent as file data); it now does, matching
rsync. Divergences: because the receiver enforces the symlink containment
predicate unconditionally, a plain `-l` sync **refuses to round-trip a
legitimate absolute symlink target** (it is dropped, never created pointing
outside the root — see the `--munge-links` note for the symmetric trust
boundary); a relative in-root target is copied as-is. As of P7 Wave D FastSync
also applies the symlink's own metadata with no-follow primitives
target, the sender transmits it, and the receiver `symlinkat`s it. **Targets
are stored verbatim (protocol 2.23.0), matching rsync `-l`:** an absolute
target or one containing `..` is copied exactly as-is. The receiver no longer
enforces the strict containment predicate on the link *value*; target policy
belongs to the sender (`--safe-links`/`--copy-unsafe-links`) exactly as in
rsync. The link's *placement* path is still hard-confined
(`has_path_traversal`, O_NOFOLLOW fd walk), and the link's own metadata is
applied with no-follow primitives
(`utimensat`/`fchownat`/`fchmodat` with `AT_SYMLINK_NOFOLLOW`), so `-J` is a
real omit switch rather than a no-op.
real omit switch rather than a no-op. **Residual risk:** because `-l` stores
targets verbatim and does not enforce containment, a destination later
consumed by a link-following tool can follow a link outside the receive root.
Use `--safe-links` when the source is not trusted; a destination that only
ever uses `openat`-style no-follow access is unaffected.
- **`-k/--copy-dirlinks`** (sender): a symlink whose referent is a directory is
dereferenced and recursed into as a real directory; a symlink to a regular
file (or any non-directory) is kept as a symlink. This is rsync's `-k`. When
@@ -547,40 +581,35 @@ was bumped **2.12.0 → 2.13.0** (peers must match, exactly as prior phases did)
divergence for `--delete` over an existing symlinked dir). Without `-K` the
destination symlink is not followed (the O_NOFOLLOW walk fails the write),
which is the safe default.
- **`--munge-links`** (sender security rewrite; crosses the wire so the receiver
unmunges): every transmitted symlink target is prefixed with the marker
`#SYMLINK/`; the receiver strips the marker (only when the negotiated
`munge_links` policy is on — a plain `-l` run never strips the prefix, so a
source symlink that genuinely begins with `#SYMLINK/` round-trips verbatim)
and restores the exact real target. The trust boundary is **symmetric and
enforced receiver-side**, independent of the sender: `file_symlink_at_secure`
refuses any target that `file_symlink_target_contained` rejects (absolute
`/...` or relative with a `..` component), and `file_save_to_disk_full`
contains such an entry (skipped) rather than materializing it. A deliberate confinement trade-off: because the receiver
enforces containment unconditionally, a plain `-l` (no `--munge-links`) sync
*refuses to round-trip a legitimate absolute symlink target* — such target is
dropped, never created pointing outside the root. This is a stricter subset of
rsync: rsync stores munged targets on the RECEIVING side and depends on both
ends running `--munge-links`; FastSync additionally enforces the containment
predicate at the receiver regardless of what the sender transmitted. When no
symlink is being transmitted (`-l`/`-k`/`-a` off) `--munge-links` has nothing
to rewrite and is inert. -*K/`--keep-dirlinks` policy is installed per
connection at config-accept (stable for the whole transfer, never racy under
`-j`/`--threads`), and only ever follows an in-root symlink-to-directory.*
- **`--munge-links`** (sender rewrite; crosses the wire so the receiver
unmunges): every transmitted symlink target is prefixed with rsync's marker
`SYMLINK_MUNGE_PREFIX` = `/rsyncd-munged/`; the receiver strips the marker
(only when the negotiated `munge_links` policy is on — a plain `-l` run never
strips the prefix, so a source symlink that genuinely begins with
`/rsyncd-munged/` round-trips verbatim) and restores the exact real target.
This matches rsync's stored marker and its both-ends-negotiated model, with the
prefix applied on the sender rather than the receiver. The link *value* is
otherwise stored verbatim; the *placement* path still goes through
`file_symlink_at_secure`'s confined fd walk (`has_path_traversal` on the
destination path, no symlink follow). When no symlink is being transmitted
(`-l`/`-k`/`-a` off) `--munge-links` has nothing to rewrite and is inert.
`-K`/`--keep-dirlinks` policy is installed per connection at config-accept
(stable for the whole transfer, never racy under `-j`/`--threads`), and only
ever follows an in-root symlink-to-directory.
**Compatibility (byte-identical when all three are absent):** `-k`, `-K` and
`--munge-links` are opt-in. Without them the scanner's link handling, the wire
frames, and the receiver's writes are unchanged for every other option set, so a
run that previously worked continues to behave identically. `-l/--links` itself
now transmits targets (the prior behavior was broken/partial); its status moved
`⚠️ Partial → ✅ Implemented`.
**Compatibility:** `-k`, `-K` and `--munge-links` are opt-in. Without them the
scanner's link handling, the wire frames, and the receiver's writes are unchanged
for every other option set. `--safe-links`/`--copy-unsafe-links` are applied
sender-side; `--trust-sender` no longer changes how symlink targets are stored
(it only skips the receiver's path-list re-validation). `-l/--links` stores
targets verbatim, matching rsync.
## 10. Sparse & Device
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-S`, `--sparse` | Sparse block handling | ✅ Implemented | Phase 7 Wave B: real hole preservation with no wire change. The receiver's sparse-aware writer (`write_all_sparse`, next to `write_all` in `src/shared/file.c` and `src/shared/file_store.c`) walks the in-memory file image and emits any all-zero run ≥ 4096 bytes as a hole via `lseek(SEEK_CUR)` (the pre-size `ftruncate` guarantees the offset bookkeeping and logical size), `ftruncate(size)` after the last run pins the final size even with a hole tail. Wired into both the atomic temp+rename store and `--inplace` when `sparse` is set; the non-sparse path is byte-identical to before. **Sparse wins over `--preallocate`** (posix_fallocate is skipped when sparse is set, so the holes are not re-allocated). Interplay note: under `--partial` a retained sparse temp already has the full logical size (trailing content is holes), so `--append`'s "shorter destination" resume does not re-run; the retained file is still valid and a normal re-transfer (or `-W`/delta) repairs it — documented so the combination is never surprising |
| `--preallocate` | Allocate dest files before writing | ✅ Implemented | The receiver preallocates the destination file's full expected space before any data is written, so a transfer that would overflow disk fails fast at allocation time (a clean error, not a half-written file) and the file is laid out contiguously, avoiding fragmentation. Crosses the wire (the config frame carries a `preallocate` boolean; `PROTOCOL_VERSION` bumped **2.10.0 → 2.11.0**, peers must match) so the sender knows the receiver will preallocate and the receiver performs it. **Allocation approach:** `posix_fallocate()` is preferred because it reserves *real* disk blocks (true fail-fast on ENOSPC), falling back to plain `ftruncate()` only when the filesystem reports the allocation is unsupported (`EOPNOTSUPP`/`ENOSYS`); `ftruncate` still extends the logical size so the intent degrades gracefully. **Fallback/error semantics:** `EOPNOTSUPP`/`ENOSYS` → clean fallback to `ftruncate` (best-effort, preallocates the logical size and never fails a transfer on filesystems that lack `posix_fallocate`); a genuine allocation failure (`ENOSPC`/`EDQUOT`/`EFBIG`/…) aborts the file/receive with a distinct `preallocate failed ... transfer aborted` error — it does **not** fall back to a normal non-preallocated write, preserving the fail-fast purpose. **Size-known requirement:** preallocation only runs when the final size is already known up front (the normal regular-file case); unknown-length data is skipped (never failed). **Orthogonality:** applies uniformly across the atomic temp+rename store path, `--inplace`, `--partial`/`--partial-dir`, `--delay-updates` (the staged temp file is preallocated before data flows) and the `--link-dest` copy fallback; it neither implies nor conflicts with `-s`, `--append`, or delta. rsync-divergence: rsync signals that `--preallocate` is ignored with `--sparse`; FastSync gives **sparse precedence** — when both are set, `posix_fallocate` is skipped so the holes the sparse writer creates are not re-allocated (the `ftruncate` presize sizing stays), matching the intent of "sparse wins". See the Phase-4 preallocate notes below |
| `-S`, `--sparse` | Sparse block handling | ✅ Parity | Phase 7 Wave B: real hole preservation with no wire change. The receiver's sparse-aware writer (`write_all_sparse`, next to `write_all` in `src/shared/file.c` and `src/shared/file_store.c`) walks the in-memory file image and emits any all-zero run ≥ 4096 bytes as a hole via `lseek(SEEK_CUR)` (the pre-size `ftruncate` guarantees the offset bookkeeping and logical size), `ftruncate(size)` after the last run pins the final size even with a hole tail. Wired into both the atomic temp+rename store and `--inplace` when `sparse` is set; the non-sparse path is byte-identical to before. **Sparse wins over `--preallocate`** (posix_fallocate is skipped when sparse is set, so the holes are not re-allocated). Interplay note: under `--partial` a retained sparse temp already has the full logical size (trailing content is holes), so `--append`'s "shorter destination" resume does not re-run; the retained file is still valid and a normal re-transfer (or `-W`/delta) repairs it — documented so the combination is never surprising |
| `--preallocate` | Allocate dest files before writing | ⚠️ Caveat | The receiver preallocates the destination file's full expected space before any data is written, so a transfer that would overflow disk fails fast at allocation time (a clean error, not a half-written file) and the file is laid out contiguously, avoiding fragmentation. Crosses the wire (the config frame carries a `preallocate` boolean; `PROTOCOL_VERSION` bumped **2.10.0 → 2.11.0**, peers must match) so the sender knows the receiver will preallocate and the receiver performs it. **Allocation approach:** `posix_fallocate()` is preferred because it reserves *real* disk blocks (true fail-fast on ENOSPC), falling back to plain `ftruncate()` only when the filesystem reports the allocation is unsupported (`EOPNOTSUPP`/`ENOSYS`); `ftruncate` still extends the logical size so the intent degrades gracefully. **Fallback/error semantics:** `EOPNOTSUPP`/`ENOSYS` → clean fallback to `ftruncate` (best-effort, preallocates the logical size and never fails a transfer on filesystems that lack `posix_fallocate`); a genuine allocation failure (`ENOSPC`/`EDQUOT`/`EFBIG`/…) aborts the file/receive with a distinct `preallocate failed ... transfer aborted` error — it does **not** fall back to a normal non-preallocated write, preserving the fail-fast purpose. **Size-known requirement:** preallocation only runs when the final size is already known up front (the normal regular-file case); unknown-length data is skipped (never failed). **Orthogonality:** applies uniformly across the atomic temp+rename store path, `--inplace`, `--partial`/`--partial-dir`, `--delay-updates` (the staged temp file is preallocated before data flows) and the `--link-dest` copy fallback; it neither implies nor conflicts with `-s`, `--append`, or delta. rsync-divergence: rsync signals that `--preallocate` is ignored with `--sparse`; FastSync gives **sparse precedence** — when both are set, `posix_fallocate` is skipped so the holes the sparse writer creates are not re-allocated (the `ftruncate` presize sizing stays), matching the intent of "sparse wins". See the Phase-4 preallocate notes below |
**Preallocate notes (Phase 4, preallocate wave):** `--preallocate` is implemented as a real receiver-side allocation of the destination file's space before data is written. It is a plain boolean config flag that crosses the wire (serialized in the config frame's selection-options block, mirroring `--inplace`/`--append`/`--force`), so the run requires matching ends: `PROTOCOL_VERSION` was bumped **2.10.0 → 2.11.0** (peers must match or the version check fails). The allocation is performed on the exact destination fd, immediately after it is opened, before any bytes are streamed; `posix_fallocate` (and the `ftruncate` fallback) leave the fd's file offset untouched, so the subsequent data write at offset 0 is unaffected and complete. Because FastSync writes each file's byte payload in one in-memory batch, the "full expected size" is exactly the known `data_size`, which is what gets preallocated. Unknown-length/streamed payloads are skipped rather than failed. A failed allocation logs a distinct `preallocate failed` error and aborts the file (the atomic temp is unlinked, the inplace target is left untrimmed) so the run fails cleanly and never silently degrades to a non-preallocated write — preserving rsync's fail-fast intent on a full disk.
@@ -589,49 +618,50 @@ now transmits targets (the prior behavior was broken/partial); its status moved
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `--checksum` | Skip based on checksum | ✅ Implemented | With `--incremental`, compares per-file whole-file content digests to skip unchanged files. The digest algorithm is `xxh64` with seed 0 by default and is selectable via `--checksum-choice`/`--cc` (xxh64/xxhash or md5) and `--checksum-seed=NUM` (see those rows); `-c` remains compression |
| `--checksum-choice=STR`, `--cc=STR` | Choose checksum algorithm | ✅ Implemented | Real algorithm selection for the per-file whole-file digest used by the `--incremental`/`--checksum` handshake and by the basis-dir content verification. FastSync genuinely supports `xxh64` (the default, exact xxHash64, seeded by `--checksum-seed`) and `md5` (via OpenSSL EVP); `xxhash` is accepted as rsync's spelling of xxHash64. Any other name (md4/sha1/sha256/crc32/none/…) is rejected with a clear error at parse time — never a silent no-op. `--cc` is the alias (`--cc=ALG` and space forms both parse). The algorithm id and seed cross the wire with the config frame, so the receiver hashes its on-disk old file with the SAME algorithm+seed the sender used and both agree on a match; the sender's digest and the receiver's comparison live in the per-file `STATUS_CHECK` handshake, which now carries a length-prefixed, bounded (1..16 byte) digest instead of a fixed 64-bit value, and the receiver pins the received length to the negotiated algorithm's digest length (defense-in-depth: a mismatched/malicious length only forces a safe re-transfer). Note: `md5` is a FIPS-non-approved algorithm, so under an OpenSSL build with FIPS mode enabled `--checksum-choice=md5` fails loudly rather than silently falling back. Protocol/layout: `PROTOCOL_VERSION` bumped **2.9.0 → 2.10.0** (peers must match). Defaults preserve the pre-existing behavior byte-for-byte (xxh64, seed 0). Like rsync, the choice only takes effect where a whole-file digest is actually computed (`--checksum` on, or a basis-dir flag); it does not itself enable `--checksum`. Closely-related divergence: the delta BLOCK strong checksum (§11 delta) stays xxHash32 — `--checksum-choice` selects only the whole-file digest, matching rsync where the per-block checksum is independent of the whole-file checksum choice |
| `--compare-dest=DIR` | Compare dest files relative to DIR | ✅ Implemented | DIR is a receiver-side basis relative to the destination root (confined below it; absolute/`..`/`.` rejected, `//` collapsed and trailing `/` dropped). On the receiver's per-file check (implies `--incremental`) an exact match = same size + mtime (unless `--size-only`; `-I` disables matching) **and** equal xxHash64 of the sender's file; a match suppresses the data transfer. compare-dest never copies: it only skips a file the destination does **not** already hold (sparse destination, rsync parity), and is consulted before the normal delta/full paths. Repeatable; searched in command-line order, first match wins. Divergences: when the destination already holds a *different* version rsync deletes it but FastSync instead transfers the data (keeps the mirror complete; never deletes without `--delete`); attribute-only differences on a match are not re-applied (data is skipped so the sender never sends metadata); content is verified by xxHash64, stricter than rsync's default quick check. Sizing: FastSync's whole-file payload limit is 256 MiB on **every** transfer path (not basis-specific); rsync applies basis dirs to arbitrary sizes, so FastSync refuses a basis run whose source contains a larger file up front with a clear error before any transfer. Wire: a basis-count field is always present on the config frame (protocol 2.9.0, so clients and servers must both be 2.9.0) |
| `--copy-dest=DIR` | Include copies of unchanged files | ✅ Implemented | Same basis rules as `--compare-dest`, but an exact match materializes a **local copy** of the DIR file into the destination (via the normal atomic temp+rename store path, so `--existing`/`--ignore-existing`/`--update`/`--backup`/`--delay-updates` all still apply) instead of transferring data. Repeatable; command-line order = priority. Content is xxHash64-verified before the copy. Divergences: a basis-hit destination keeps the basis file's own mode/uid/gid and mtime (the sender sends no metadata on a skip), so with `--size-only` its mtime can differ from the source and attribute-only differences are copied with the basis attributes rather than rsync's "copy + fix attributes". Requires `--incremental` (implied); incompatible with `-s`. Wire: protocol 2.9.0 |
| `--link-dest=DIR` | Hardlink to files when unchanged | ✅ Implemented | Same basis rules as `--copy-dest`, but an exact match installs an atomic **hard link** to the DIR file (temp hard link + rename) so no data or disk space is used; where the link is impossible (basis on another filesystem, filesystem refuses links) it falls back cleanly to a byte-identical local copy, never a corrupt/partial file. `--delay-updates` stages the link and publishes by rename, so the final entry stays a real hard link. Repeatable (searched in command-line order, first match wins). Content is xxHash64-verified before linking. Divergences and caveats: an already up-to-date destination file is not re-linked to a basis file (only files that would otherwise be written are linked); a link keeps the basis inode's own mode/uid/gid and mtime — metadata is never written through the shared inode (that would mutate the basis file), so a later `--inplace` run that rewrites such a destination path **will mutate the basis snapshot** through the shared inode (use `--copy-dest` when the destination must stay independently writable); with `--size-only` the linked mtime can differ from the source; a `--remove-source-files` source satisfied by a basis dir is treated as skipped and therefore **retained** (never removed); basis dirs are excluded from `--delete`. Requires `--incremental` (implied); incompatible with `-s`. Wire: protocol 2.9.0 |
| `-y`, `--fuzzy`, `--no-fuzzy` | Find similar file for basis | ✅ Implemented | `-y/--fuzzy` is a pure bandwidth optimization on the existing receiver-driven delta path: when a file must be transferred and the destination holds no usable content at the exact path (file absent, or the destination file is outside the delta engine's size bounds), the receiver searches the SAME destination directory for an existing regular file whose basename is similar to the incoming name and uses it as the delta basis, so the sender transmits only the differences instead of the whole file. The output is always byte-exact regardless of which (or whether any) basis is chosen. Decision location: the receiver performs the candidate search inside `receive_incremental_check` and sends the normal `STATUS_DELTA_SIGNATURE`; the sender never learns the basis was a different file, so no new frame type or sender logic was needed — only the config frame grew a `fuzzy` boolean, so `PROTOCOL_VERSION` was bumped **2.8.0 → 2.9.0** (peers must match). Similarity heuristic (deterministic, simpler than rsync's deliberately-fuzzy matching, and documented precisely): candidates are the target's sibling entries in its destination directory, opened `O_NOFOLLOW`/`AT_SYMLINK_NOFOLLOW` under the confined root (symlinks never followed; nothing outside the destination root is ever read or hashed); dotfiles, directories, the target's own name, and the `.fastsync-stage`/temp scratch names are excluded; like the ordinary delta path, the block signature the receiver transmits is derived from on-disk content it may not otherwise send, so a negotiated `--fuzzy` run exposes the destination's sibling files (at block granularity) to the sender as a known-plaintext oracle — the same information class as the normal delta handshake over the file being replaced; the size gate is the delta engine's own bounds (both files ≥ 16 KiB, ≤ `--delta-max`, ratio ≤ 10×) rather than rsync's ~1.5× size window; the name gate is a Levenshtein edit distance between the basenames accepted only when ≤ half the length of the longer basename; the single best candidate (smallest distance, tie-break size closest to the incoming file then lexicographically smaller basename) is read; the directory scan is capped at 4096 entries so a pathological directory cannot stall a transfer. When fuzzy applies: only to files the receiver would otherwise send whole — the destination's own file is always preferred as the delta basis when it exists and fits the delta size bounds, so fuzzy does NOT replace an existing-but-different destination basis; FastSync's 10× delta size-ratio bound means an existing destination file that is too far away in size still lets the fuzzy search run. When no similar candidate exists the transfer falls back to the normal whole-file transfer. rsync-divergence note: rsync's own matching uses a fuzzy name/size rule set; FastSync implements the closest safe deterministic approximation above. Because FastSync's delta machinery is off by default (rsync's is on), `--fuzzy` implies `--incremental` + `--delta` (unless `--whole-file`/`-W` or an explicit `--no-delta` switched delta off, in which case fuzzy is inert — matching rsync where `--whole-file` makes fuzzy irrelevant). Unlike the basis-dir options, `--fuzzy` honors an explicit `--no-incremental` (it does not force the handshake back on); an explicit `--no-incremental` also suppresses the delta implication so no invalid `--delta requires --incremental` config results. `--no-fuzzy` negates it. All surrounding semantics are untouched: a fuzzy-reconstructed file is stored as a normal file, so `--remove-source-files`, itemize/`-i`, `--stats`, `--backup`, `--delay-updates`, `--existing`/`--ignore-existing`/`--update` behave exactly as for a whole-file transfer (the fuzzy delta does not skip the file) |
| `--checksum` | Skip based on checksum | ✅ Parity | `-c`/`--checksum` compares per-file whole-file content digests to skip unchanged files. **As of protocol 2.23.0 the short `-c` implies the checksum quick-check**, so a plain `-c` run verifies content rather than only affecting the `--incremental` handshake. The digest algorithm is `xxh64` by default and is selectable via `--checksum-choice`/`--cc` (`xxh64`/`xxhash`/`xxh3`/`xxh128`/`md5`/`auto`) and `--checksum-seed=NUM` (see those rows) |
| `--checksum-choice=STR`, `--cc=STR` | Choose checksum algorithm | ⚠️ Caveat | Real algorithm selection for the per-file whole-file digest used by the `--incremental`/`--checksum` handshake and by the basis-dir content verification. **Protocol 2.23.0 accepts `xxh64` (the default), `xxhash` (rsync's spelling of xxHash64), `xxh3`, `xxh128`, `md5`, and `auto` (which selects FastSync's default).** rsync choices FastSync does not implement — `md4`, `sha1`, `none`, and the two-name `transfer,pre-transfer` form — are **rejected by name** with a clear error at parse time, never a silent no-op. `--cc` is the alias (`--cc=ALG` and space forms both parse). The algorithm id and seed cross the wire with the config frame, so the receiver hashes its on-disk old file with the SAME algorithm+seed the sender used and both agree on a match; the sender's digest and the receiver's comparison live in the per-file `STATUS_CHECK` handshake, which carries a length-prefixed, bounded (1..16 byte) digest, and the receiver pins the received length to the negotiated algorithm's digest length (defense-in-depth: a mismatched/malicious length only forces a safe re-transfer). Digest lengths: `xxh64`/`xxh3` = 8 bytes, `xxh128`/`md5` = 16. Note: `md5` is a FIPS-non-approved algorithm, so under an OpenSSL build with FIPS mode enabled `--checksum-choice=md5` fails loudly rather than silently falling back. `PROTOCOL_VERSION` has moved well past the original 2.10.0 digest-frame bump. Like rsync, the choice only takes effect where a whole-file digest is actually computed (`--checksum` on, or a basis-dir flag). Closely-related divergence: the delta BLOCK strong checksum stays xxHash32 — `--checksum-choice` selects only the whole-file digest, matching rsync where the per-block checksum is independent of the whole-file choice |
| `--compare-dest=DIR` | Compare dest files relative to DIR | ⚠️ Caveat | DIR is a receiver-side basis relative to the destination root (confined below it; absolute/`..`/`.` rejected, `//` collapsed and trailing `/` dropped). On the receiver's per-file check (implies `--incremental`) an exact match = same size + mtime (unless `--size-only`; `-I` disables matching) **and** equal xxHash64 of the sender's file; a match suppresses the data transfer. compare-dest never copies: it only skips a file the destination does **not** already hold (sparse destination, rsync parity), and is consulted before the normal delta/full paths. Repeatable; searched in command-line order, first match wins. Divergences: when the destination already holds a *different* version rsync deletes it but FastSync instead transfers the data (keeps the mirror complete; never deletes without `--delete`); attribute-only differences on a match are not re-applied (data is skipped so the sender never sends metadata); content is verified by xxHash64, stricter than rsync's default quick check. Sizing: FastSync's whole-file payload limit is 256 MiB on **every** transfer path (not basis-specific); rsync applies basis dirs to arbitrary sizes, so FastSync refuses a basis run whose source contains a larger file up front with a clear error before any transfer. Wire: a basis-count field is always present on the config frame (protocol 2.9.0, so clients and servers must both be 2.9.0) |
| `--copy-dest=DIR` | Include copies of unchanged files | ⚠️ Caveat | Same basis rules as `--compare-dest`, but an exact match materializes a **local copy** of the DIR file into the destination (via the normal atomic temp+rename store path, so `--existing`/`--ignore-existing`/`--update`/`--backup`/`--delay-updates` all still apply) instead of transferring data. Repeatable; command-line order = priority. Content is xxHash64-verified before the copy. Divergences: a basis-hit destination keeps the basis file's own mode/uid/gid and mtime (the sender sends no metadata on a skip), so with `--size-only` its mtime can differ from the source and attribute-only differences are copied with the basis attributes rather than rsync's "copy + fix attributes". Requires `--incremental` (implied); incompatible with `-s`. Wire: protocol 2.9.0 |
| `--link-dest=DIR` | Hardlink to files when unchanged | ⚠️ Caveat | Same basis rules as `--copy-dest`, but an exact match installs an atomic **hard link** to the DIR file (temp hard link + rename) so no data or disk space is used; where the link is impossible (basis on another filesystem, filesystem refuses links) it falls back cleanly to a byte-identical local copy, never a corrupt/partial file. `--delay-updates` stages the link and publishes by rename, so the final entry stays a real hard link. Repeatable (searched in command-line order, first match wins). Content is xxHash64-verified before linking. Divergences and caveats: an already up-to-date destination file is not re-linked to a basis file (only files that would otherwise be written are linked); a link keeps the basis inode's own mode/uid/gid and mtime — metadata is never written through the shared inode (that would mutate the basis file), so a later `--inplace` run that rewrites such a destination path **will mutate the basis snapshot** through the shared inode (use `--copy-dest` when the destination must stay independently writable); with `--size-only` the linked mtime can differ from the source; a `--remove-source-files` source satisfied by a basis dir is treated as skipped and therefore **retained** (never removed); basis dirs are excluded from `--delete`. Requires `--incremental` (implied); incompatible with `-s`. Wire: protocol 2.9.0 |
| `-y`, `--fuzzy`, `--no-fuzzy` | Find similar file for basis | ⚠️ Caveat | `-y/--fuzzy` is a pure bandwidth optimization on the existing receiver-driven delta path: when a file must be transferred and the destination holds no usable content at the exact path (file absent, or the destination file is outside the delta engine's size bounds), the receiver searches the SAME destination directory for an existing regular file whose basename is similar to the incoming name and uses it as the delta basis, so the sender transmits only the differences instead of the whole file. The output is always byte-exact regardless of which (or whether any) basis is chosen. Decision location: the receiver performs the candidate search inside `receive_incremental_check` and sends the normal `STATUS_DELTA_SIGNATURE`; the sender never learns the basis was a different file, so no new frame type or sender logic was needed — only the config frame grew a `fuzzy` boolean, so `PROTOCOL_VERSION` was bumped **2.8.0 → 2.9.0** (peers must match). Similarity heuristic (deterministic, simpler than rsync's deliberately-fuzzy matching, and documented precisely): candidates are the target's sibling entries in its destination directory, opened `O_NOFOLLOW`/`AT_SYMLINK_NOFOLLOW` under the confined root (symlinks never followed; nothing outside the destination root is ever read or hashed); dotfiles, directories, the target's own name, and the `.fastsync-stage`/temp scratch names are excluded; like the ordinary delta path, the block signature the receiver transmits is derived from on-disk content it may not otherwise send, so a negotiated `--fuzzy` run exposes the destination's sibling files (at block granularity) to the sender as a known-plaintext oracle — the same information class as the normal delta handshake over the file being replaced; the size gate is the delta engine's own bounds (both files ≥ 16 KiB, ≤ `--delta-max`, ratio ≤ 10×) rather than rsync's ~1.5× size window; the name gate is a Levenshtein edit distance between the basenames accepted only when ≤ half the length of the longer basename; the single best candidate (smallest distance, tie-break size closest to the incoming file then lexicographically smaller basename) is read; the directory scan is capped at 4096 entries so a pathological directory cannot stall a transfer. When fuzzy applies: only to files the receiver would otherwise send whole — the destination's own file is always preferred as the delta basis when it exists and fits the delta size bounds, so fuzzy does NOT replace an existing-but-different destination basis; FastSync's 10× delta size-ratio bound means an existing destination file that is too far away in size still lets the fuzzy search run. When no similar candidate exists the transfer falls back to the normal whole-file transfer. rsync-divergence note: rsync's own matching uses a fuzzy name/size rule set; FastSync implements the closest safe deterministic approximation above. Because FastSync's delta machinery is off by default (rsync's is on), `--fuzzy` implies `--incremental` + `--delta` (unless `--whole-file`/`-W` or an explicit `--no-delta` switched delta off, in which case fuzzy is inert — matching rsync where `--whole-file` makes fuzzy irrelevant). Unlike the basis-dir options, `--fuzzy` honors an explicit `--no-incremental` (it does not force the handshake back on); an explicit `--no-incremental` also suppresses the delta implication so no invalid `--delta requires --incremental` config results. `--no-fuzzy` negates it. All surrounding semantics are untouched: a fuzzy-reconstructed file is stored as a normal file, so `--remove-source-files`, itemize/`-i`, `--stats`, `--backup`, `--delay-updates`, `--existing`/`--ignore-existing`/`--update` behave exactly as for a whole-file transfer (the fuzzy delta does not skip the file) |
## 12. Compression
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-z`, `--compress` | Compress file data | ✅ Implemented | Always uses zstd (rsync supports multiple algorithms — a documented divergence, selectable via `--compress-choice`). Phase 7 Wave A: `-z` is now the compression short form; `-c` is rsync's `--checksum` |
| `--compress-choice=STR`, `--zc=STR` | Choose compression algorithm | ✅ Implemented | FastSync supports `zstd` and `none` |
| `--compress-level=NUM`, `--zl=NUM` | Set compression level | ✅ Implemented | 1-22, default 5 |
| `--compress-threads=NUM` | Set compression threads | ✅ Implemented | `compression_threads` config field (client-only; does not cross the wire). Sets the number of worker threads used by the zstd compression pool to NUM (1..64; 0/garbage/oversized rejected up front). Accepted in both `--compress-threads=NUM` and two-argument `--compress-threads NUM` forms. Composes with `-z`/compression; under the `-j`/`--threads` multithreaded pipeline it parallelizes compressed chunk encoding. See test_tcp.py `-z --compress-threads=2` and test_client_cli.c |
| `--skip-compress=LIST` | Skip compress for suffixes | ✅ Implemented | Comma-separated, case-insensitive suffix list; empty list skips none; incompatible with FastSync chunk serialization (`-s`) |
| `-z`, `--compress` | Compress file data | ⚠️ Caveat | Streaming zstd (rsync supports multiple algorithms — a documented divergence, selectable via `--compress-choice`). `-z` is the compression short form; `-c` is rsync's `--checksum`. `--skip-compress` applies rsync 3.4.1's default suffix list when no list is given |
| `--compress-choice=STR`, `--zc=STR` | Choose compression algorithm | ⚠️ Caveat | FastSync supports `zstd` (default), `none`, and `auto`. rsync's other compiled-in choices (`lz4`, `zlib`, `zlibx`) are **rejected by name** at parse time with a clear error, never silently ignored. `--zc` is the alias |
| `--compress-level=NUM`, `--zl=NUM` | Set compression level | ✅ Parity | 1-22, default 5 |
| `--compress-threads=NUM` | Set compression threads | ✅ Parity | `compression_threads` config field (client-only; does not cross the wire). Sets the number of worker threads used by the zstd compression pool to NUM (1..64; 0/garbage/oversized rejected up front). Accepted in both `--compress-threads=NUM` and two-argument `--compress-threads NUM` forms. Composes with `-z`/compression; under the `-j`/`--threads` multithreaded pipeline it parallelizes compressed chunk encoding. See test_tcp.py `-z --compress-threads=2` and test_client_cli.c |
| `--skip-compress=LIST` | Skip compress for suffixes | ⚠️ Caveat | Comma-separated (or `/`-separated, as in rsync) case-insensitive suffix list; a leading dot is optional; an empty list skips none. **When the option is omitted, rsync 3.4.1's built-in default suffix list applies** (`3g2 3gp 7z aac … zip zst`); an explicit list replaces that default entirely, matching rsync. A user-supplied list is a client-side compression choice; incompatible with FastSync chunk serialization (`-s`) |
## 13. Connectivity
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-e`, `--rsh=COMMAND` | Remote shell to use | ✅ Implemented | `-e`/`--rsh` (and `--rsh=COMMAND`) select the remote-shell program used to build the SSH child argv, overriding the default `ssh`. The command is whitespace-split into the leading argv words so rsync's `-e "ssh -p 2222"` works; the standard `-o` family, an optional `-p` port, `user@host` and the quoted remote command (`fastsync-server --stdio`) follow. Stored in the `rsh_command` config field. **Client-only, never crosses the wire** (it is a launch concern, not a handshake property) |
| `--rsync-path=PROGRAM` | rsync binary on remote | ✅ Implemented | Alias for `--fastsync-server-path`: both write the `fastsync_server_path` config field used as the remote-side server program (always quoted as one remote-shell word), which CROSSES the wire as before. Kept separate from `--rsh`, which names the local connecting program |
| `--port=PORT`, `--port PORT` | Alternate daemon port | ✅ Implemented | rsync's daemon-port flag is an alias for `--server-port`: both spellings (and `--server-port=PORT`) map to the client-side `server_port` config field. The client connects to a TCP/TLS server (incl. `host::module/path` daemon destinations) on that port, and the `fastsync-server --daemon` listener's port is taken from its config's `port` key (default 873) or overridden by `--dparam port=` / `-p` |
| `--sockopts=OPTIONS` | Custom TCP options | ✅ Implemented | Comma-separated allowlist of `OPT=VAL` applied via `setsockopt` after `socket()` before `connect()`/`bind()`. Only `TCP_NODELAY`, `SO_KEEPALIVE`, `SO_REUSEADDR` (0/1) and `SO_RCVBUF`/`SO_SNDBUF` (byte count) are accepted; an unknown option name or a bad value is rejected up front, never silently ignored. A value is required for every option (`OPT=VAL`; a bare name is an error). Applied to the outgoing TCP and TLS client socket; absent by default. `SockOptEntry`/`sockopts` config fields. Local socket concern: never crosses the wire |
| `--blocking-io` | Use blocking I/O for remote shell | ✅ Implemented | With `--blocking-io` the SSH-transport socketpair socket is left without `SO_RCVTIMEO`/`SO_SNDTIMEO`, so the transfer blocks naturally; by default it gets the same read/write timeout as the TCP transport (see `--timeout`). `blocking_io` config bool. **Client-only, never crosses the wire** |
| `--outbuf=N\|L\|B` | Set output buffering | ✅ Implemented | `N` (none/unbuffered) → `_IONBF`, `L` (line) → `_IOLBF`, `B` (block, the default) → `_IOFBF` via `setvbuf` on stdout and stderr. Garbage values are rejected. `outbuf` config field (`OutbufMode`). **Client-only, never crosses the wire** |
| `--address=ADDRESS` | Bind address for outgoing socket | ✅ Implemented | Binds the outgoing client socket to a local source address before `connect()` (resolved with the same `-4`/`-6` family hints as the destination). Local socket concern: never crosses the wire |
| `-4`, `--ipv4` | Prefer IPv4 | ✅ Implemented | Forces `AF_INET` in the `getaddrinfo` hints for client destination/source resolution and the server bind (see the Phase 5, Wave B note). Mutually exclusive with `-6` |
| `-6`, `--ipv6` | Prefer IPv6 | ✅ Implemented | Forces `AF_INET6` in the `getaddrinfo` hints for client destination/source resolution and the server bind. Mutually exclusive with `-4` |
| `--remote-option=OPT`, `-M` | Send an option only to the remote side | ✅ Implemented | Each value is appended to the remote server invocation over SSH as an individually single-quote-escaped shell word in `ssh_build_remote_command()`. Values are validated (non-empty, no control characters) and shell metacharacters cannot break out of the quoting (`;`, `&`, `|`, <code>`</code>, `$`, `(`, `)`, quotes are neutralized), so a value cannot inject an arbitrary remote command and a subsequent `--` on the client line cannot be turned into one. The options never cross the binary config frame. Phase 7 Wave A: the short `-M` form is now available (as `-M OPT` and `-M=OPT`), matching rsync; metadata mode moved to long-only `--preserve` |
| `-e`, `--rsh=COMMAND` | Remote shell to use | ✅ Parity | `-e`/`--rsh` (and `--rsh=COMMAND`) select the remote-shell program used to build the SSH child argv, overriding the default `ssh`. The command is whitespace-split into the leading argv words so rsync's `-e "ssh -p 2222"` works; the standard `-o` family, an optional `-p` port, `user@host` and the quoted remote command (`fastsync-server --stdio`) follow. Stored in the `rsh_command` config field. **Client-only, never crosses the wire** (it is a launch concern, not a handshake property) |
| `--rsync-path=PROGRAM` | rsync binary on remote | ✅ Parity | Alias for `--fastsync-server-path`: both write the `fastsync_server_path` config field used as the remote-side server program. The path is always quoted as one remote-shell word in the SSH argv. **Client-only: `fastsync_server_path` never crosses the wire** (it is a launch concern, not a handshake property), matching rsync, where `--rsync-path` likewise names the remote program locally. Kept separate from `--rsh`, which names the local connecting program |
| `--port=PORT`, `--port PORT` | Alternate daemon port | ✅ Parity | rsync's daemon-port flag is an alias for `--server-port`: both spellings (and `--server-port=PORT`) map to the client-side `server_port` config field. The client connects to a TCP/TLS server (incl. `host::module/path` daemon destinations) on that port, and the `fastsync-server --daemon` listener's port is taken from its config's `port` key (default 873) or overridden by `--dparam port=` / `-p` |
| `--sockopts=OPTIONS` | Custom TCP options | ✅ Parity | Comma-separated allowlist of `OPT=VAL` applied via `setsockopt` after `socket()` before `connect()`/`bind()`. Only `TCP_NODELAY`, `SO_KEEPALIVE`, `SO_REUSEADDR` (0/1) and `SO_RCVBUF`/`SO_SNDBUF` (byte count) are accepted; an unknown option name or a bad value is rejected up front, never silently ignored. A value is required for every option (`OPT=VAL`; a bare name is an error). Applied to the outgoing TCP and TLS client socket; absent by default. `SockOptEntry`/`sockopts` config fields. Local socket concern: never crosses the wire |
| `--blocking-io` | Use blocking I/O for remote shell | ✅ Parity | With `--blocking-io` the SSH-transport socketpair socket is left without `SO_RCVTIMEO`/`SO_SNDTIMEO`, so the transfer blocks naturally; by default it gets the same read/write timeout as the TCP transport (see `--timeout`). `blocking_io` config bool. **Client-only, never crosses the wire** |
| `--timeout=SEC`, `--contimeout=SEC` | Set I/O / connect timeouts | ✅ Parity | Protocol 2.23.0 matches rsync's defaults: **`--timeout` defaults to 0 (I/O deadlines disabled) and `--contimeout` to 60 s; `0` disables either.** A positive `--timeout` bounds both the socket (`SO_RCVTIMEO`/`SO_SNDTIMEO`) and the per-message protocol poll deadline on the client; the server floors its session deadline so a client `0` can never hold a session open forever. `--no-timeout`/`--no-contimeout` are the negations. Both are client-side deadlines and are not sent on the wire |
| `--outbuf=N\|L\|B` | Set output buffering | ✅ Parity | `N` (none/unbuffered) → `_IONBF`, `L` (line) → `_IOLBF`, `B` (block, the default) → `_IOFBF` via `setvbuf` on stdout and stderr. Garbage values are rejected. `outbuf` config field (`OutbufMode`). **Client-only, never crosses the wire** |
| `--address=ADDRESS` | Bind address for outgoing socket | ✅ Parity | Binds the outgoing client socket to a local source address before `connect()` (resolved with the same `-4`/`-6` family hints as the destination). Local socket concern: never crosses the wire |
| `-4`, `--ipv4` | Prefer IPv4 | ✅ Parity | Forces `AF_INET` in the `getaddrinfo` hints for client destination/source resolution and the server bind (see the Phase 5, Wave B note). Mutually exclusive with `-6` |
| `-6`, `--ipv6` | Prefer IPv6 | ✅ Parity | Forces `AF_INET6` in the `getaddrinfo` hints for client destination/source resolution and the server bind. Mutually exclusive with `-4` |
| `--remote-option=OPT`, `-M` | Send an option only to the remote side | ⚠️ Caveat | Each value is appended to the remote server invocation over SSH as an individually single-quote-escaped shell word in `ssh_build_remote_command()`. Values are validated (non-empty, no control characters) and shell metacharacters cannot break out of the quoting (`;`, `&`, `\|`, <code>`</code>, `$`, `(`, `)`, quotes are neutralized), so a value cannot inject an arbitrary remote command and a subsequent `--` on the client line cannot be turned into one. The short `-M` form (`-M OPT`, `-M=OPT`, and rsync-style attached `-MOPT`) is available, matching rsync; metadata mode moved to long-only `--preserve`. **Divergence:** `-M` is only meaningful for the SSH transport (`user@host:path`); a daemon (`host::module/path`) or local TCP destination **rejects** it (there is no remote command line to append to), whereas rsync applies it to its own remote process on every transport. The options never cross the binary config frame |
## 14. Daemon Mode
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `--daemon` | Run as rsync daemon | ✅ Implemented | Wave A: a real persistent listener. `fastsync-server --daemon --config FILE` (plus `--no-detach` to stay foreground; without it the listener detaches to the background after binding) reads a FastSync-native module config file and serves each connection confined to the requested module's `path` root (never a client-chosen root; every client-chosen-ownership/super-user request (`--numeric-ids`/`--chown`/`--usermap`/`--groupmap`/`--fake-super`/`--copy-as`/explicit `--super`) is refused unless the module opts in with `client owner = yes`, and the operator `--no-super` veto is honored). TCP/TLS via the existing `--tls` stack; plaintext still requires `--allow-unauthenticated` (same secure default as the standalone server). Client destinations use rsync's `host::module/path` form. Wire/protocol: the config frame gained a trailing daemon-module string and `PROTOCOL_VERSION` was bumped **2.14.0 → 2.15.0** (see the Daemon Mode notes below). Daemon mode is built in FastSync's own protocol/config grammar, not rsync's SMB/daemon option encoding |
| `--config=FILE` | Alternate rsyncd.conf file | ✅ Implemented | Wave A: selects the daemon config file. Default when omitted (in `--daemon` mode): `~/.config/fastsync/fastsyncd.conf` if it exists, else `/etc/fastsyncd.conf`. The grammar is FastSync-native (documented in the Daemon Mode notes below) and strictly rejects unknown keys so a typo can never silently change what a module serves; requires `--daemon` |
| `--dparam=OVERRIDE` | Override global daemon config | ✅ Implemented | Wave A: overrides one global scalar from the command line (`--dparam port=8734` and `--dparam=KEY=VALUE` both work). Limited to the global keys the grammar defines (`port`, `motd file`, `address`, `max connections`, `max connections per host`, `auth failure delay`, `auth lockout threshold`, `auth lockout duration`, `hosts allow`, `hosts deny`); keys are case-insensitive and unknown keys/invalid values are rejected. Requires `--daemon` |
| `--no-detach` | Don't detach from parent | ✅ Implemented | Wave A: with `--daemon`, keeps the listener in the foreground (what integration tests use). Without it the daemonizes (fork/setsid, stdio redirected to /dev/null) after the listening socket is bound. Requires `--daemon` |
| `--password-file=FILE` | Read daemon password from file | ✅ Implemented | A7 daemon auth. Client: `--password-file` supplies `user:password` for a `host::module/path` destination (the username is taken from this file, so `user@host::module` stays rejected); the literal password is held client-side only for the SCRAM handshake and wiped at teardown. Server (`fastsync-server --daemon --password-file FILE`): the salted-PBKDF2 verifier store that modules with `auth users` are verified against. **Neither the password nor any replayable bearer value crosses the wire or is stored server-side** — the store holds a per-user salt plus derived keys, and the daemon proves the secret with a per-connection nonce challenge. The file must be private to its owner: both the client and server verify the exact inode they read (open-then-`fstat`, so the check cannot be raced) and refuse a `--password-file`/`--early-input` that is not owned by the current user or grants any group/other permission bit (mode 0600), mirroring the TLS private-key check. A process-substitution pipe (`--early-input <(vault ...)`) is still accepted when it satisfies those checks. See the Daemon Mode notes below for the file formats and the plaintext/TLS caveat |
| `--early-input=FILE` | Use FILE for daemon early exec | ✅ Implemented | Server-only (requires `--daemon`): a second credential-store file, same new-format grammar as `--password-file`, read before the listener accepts connections (a secrets-manager / process-substitution source). Its entries layer over `--password-file`: byte-identical verifiers dedupe, a conflicting verifier for the same user is a startup error. A daemon whose modules declare `auth users` must be given at least one of the two, or it refuses to start (fail closed) |
| `--hash-credentials=FILE`, `--iterations N` | Hash a plaintext credential file | ✅ Implemented | Server-only offline tool (A7): reads the `user:password` lines of FILE (same owner-only 0600 check) and prints one new-format store line per entry to stdout, then exits. `--iterations` sets the PBKDF2 work factor (default 600000, range 100000–10000000). Dependency-free and does not run a listener. Use its output as `--password-file` for `--daemon`. There is no auto-upgrade: a legacy store line is hard-rejected by the loader and must be regenerated |
| `--daemon` | Run as rsync daemon | ⚠️ Caveat | Wave A: a real persistent listener. `fastsync-server --daemon --config FILE` (plus `--no-detach` to stay foreground; without it the listener detaches to the background after binding) reads a FastSync-native module config file and serves each connection confined to the requested module's `path` root (never a client-chosen root; every client-chosen-ownership/super-user request (`--numeric-ids`/`--chown`/`--usermap`/`--groupmap`/`--fake-super`/`--copy-as`/explicit `--super`) is refused unless the module opts in with `client owner = yes`, and the operator `--no-super` veto is honored). TCP/TLS via the existing `--tls` stack; plaintext still requires `--allow-unauthenticated` (same secure default as the standalone server). Client destinations use rsync's `host::module/path` form. Wire/protocol: the config frame gained a trailing daemon-module string and `PROTOCOL_VERSION` was bumped **2.14.0 → 2.15.0** (see the Daemon Mode notes below). Daemon mode is built in FastSync's own protocol/config grammar, not rsync's SMB/daemon option encoding |
| `--config=FILE` | Alternate rsyncd.conf file | ⚠️ Caveat | Wave A: selects the daemon config file. Default when omitted (in `--daemon` mode): `~/.config/fastsync/fastsyncd.conf` if it exists, else `/etc/fastsyncd.conf`. The grammar is FastSync-native (documented in the Daemon Mode notes below) and strictly rejects unknown keys so a typo can never silently change what a module serves; requires `--daemon` |
| `--dparam=OVERRIDE` | Override global daemon config | ⚠️ Caveat | Wave A: overrides one global scalar from the command line (`--dparam port=8734` and `--dparam=KEY=VALUE` both work). Limited to the global keys the grammar defines (`port`, `motd file`, `address`, `max connections`, `max connections per host`, `auth failure delay`, `auth lockout threshold`, `auth lockout duration`, `hosts allow`, `hosts deny`); keys are case-insensitive and unknown keys/invalid values are rejected. Requires `--daemon` |
| `--no-detach` | Don't detach from parent | ✅ Parity | Wave A: with `--daemon`, keeps the listener in the foreground (what integration tests use). Without it the daemonizes (fork/setsid, stdio redirected to /dev/null) after the listening socket is bound. Requires `--daemon` |
| `--password-file=FILE` | Read daemon password from file | ⚠️ Caveat | A7 daemon auth. Client: `--password-file` supplies `user:password` for a `host::module/path` destination (the username is taken from this file, so `user@host::module` stays rejected); the literal password is held client-side only for the SCRAM handshake and wiped at teardown. Server (`fastsync-server --daemon --password-file FILE`): the salted-PBKDF2 verifier store that modules with `auth users` are verified against. **Neither the password nor any replayable bearer value crosses the wire or is stored server-side** — the store holds a per-user salt plus derived keys, and the daemon proves the secret with a per-connection nonce challenge. The file must be private to its owner: both the client and server verify the exact inode they read (open-then-`fstat`, so the check cannot be raced) and refuse a `--password-file`/`--early-input` that is not owned by the current user or grants any group/other permission bit (mode 0600), mirroring the TLS private-key check. A process-substitution pipe (`--early-input <(vault ...)`) is still accepted when it satisfies those checks. See the Daemon Mode notes below for the file formats and the plaintext/TLS caveat |
| `--early-input=FILE` | Use FILE for daemon early exec | ⚠️ Caveat | Server-only (requires `--daemon`): a second credential-store file, same new-format grammar as `--password-file`, read before the listener accepts connections (a secrets-manager / process-substitution source). Its entries layer over `--password-file`: byte-identical verifiers dedupe, a conflicting verifier for the same user is a startup error. A daemon whose modules declare `auth users` must be given at least one of the two, or it refuses to start (fail closed) |
| `--hash-credentials=FILE`, `--iterations N` | Hash a plaintext credential file | ⚠️ Caveat | Server-only offline tool (A7): reads the `user:password` lines of FILE (same owner-only 0600 check) and prints one new-format store line per entry to stdout, then exits. `--iterations` sets the PBKDF2 work factor (default 600000, range 100000–10000000). Dependency-free and does not run a listener. Use its output as `--password-file` for `--daemon`. There is no auto-upgrade: a legacy store line is hard-rejected by the loader and must be regenerated |
**Daemon Mode notes (Wave A protocol 2.15.0; A7 auth protocol 2.19.0; MOTD no bump):** FastSync daemon mode is supported in FastSync's own protocol/config grammar, not rsync's SMB/daemon option encoding.
@@ -639,7 +669,7 @@ now transmits targets (the prior behavior was broken/partial); its status moved
- **Host access control (`hosts allow`/`hosts deny`):** both keys accept a comma- and/or whitespace-separated list of patterns and may appear globally and/or per module (multiple config-file lines append; a `--dparam` override replaces). Supported patterns are `*` (match all), an IPv4 or IPv6 literal (`10.0.0.1`, `2001:db8::1`), and an IPv4/IPv6 CIDR (`10.0.0.0/8`, `2001:db8::/32`). Hostname patterns are **not** supported: because the peer is always a numeric address and no reverse DNS is performed, a hostname/glob pattern would silently never match, so it is rejected at load time (fail-closed) instead of being accepted as a dead rule. An IPv4 peer on a dual-stack IPv6 listener is normalized from its `::ffff:a.b.c.d` form so IPv4 patterns match it. rsync-like semantics: a matching `hosts deny` rejects; if any `hosts allow` entries exist, a peer matching none of them is rejected; deny takes precedence over allow. The daemon enforces the global list first, then the selected module's list, **before authentication** in `server_module_gate`, with an audit log line naming the peer, the module and the outcome. The numeric peer address is obtained with `getpeername`+`inet_ntop` (`utils_fd_peer_ip`, handling both address families); when it cannot be obtained a module with any ACL fails closed (refused), while an ACL-free module continues and logs at debug. A malformed pattern (e.g. an out-of-range CIDR prefix) is a parse error at load time.
- **Connection caps, shared registry and auth lockout:** the global `max connections` key (default 100) is plumbed into the listener (`transport_tcp.c`), which rejects a connection once the accept-loop parent's active-child count reaches it; the IPv4/IPv6 peer is logged for every accepted connection. Because the listener forks one child per connection, the per-module `max connections` cap, the global `max connections per host` cap, and the auth-failure counter live in a fixed-size registry carved from an anonymous shared mapping (`daemon_limits.c`, `mmap(MAP_SHARED|MAP_ANONYMOUS)`) created by the parent before the accept loop, so every forked child shares the same counters (C11 atomics only — never a pthread lock, which can deadlock in a forked child). The parent reserves a registry slot per accepted connection and the child records the selected module and source IP once known; the parent's `SIGCHLD` handler reclaims the slot when the child dies (including `SIGKILL`) and re-derives the per-module and per-source occupancy counts from the surviving REGISTERED slots, so a child killed mid-registration cannot leak a count. The per-source table has a bounded lifetime: an entry with no live connection is reclaimed after its lockout expires or it has been idle (300 s); if the table is genuinely full the per-source cap/lockout fails open for new sources (per-module cap and ACLs still apply) with a rate-limited warning. The per-module cap (0 = unlimited) is enforced after the module lookup and before auth; per-source identity reuses the normalized numeric peer address (`utils_fd_peer_ip`, IPv4-mapped IPv6 collapsed to IPv4), and a trusted loopback peer (127.0.0.0/8 / `::1`, `utils_fd_peer_is_local`) is exempt from the per-source cap and the auth lockout because all local clients share one address (the per-module/global caps still apply). Clients behind a shared NAT/proxy address likewise share one per-source budget and lockout counter. A failed authentication increments the shared per-source failure count and, once `auth lockout threshold` (default 10; 0 disables) is reached, the source is refused for `auth lockout duration` seconds (default 300) before any challenge is sent, even when the next attempt is handled by a different forked child; a successful authentication clears the counter. On a failed authentication the per-connection child still sleeps the global `auth failure delay` (default 500 ms, 0 disables, capped at 5000) via `nanosleep`, rate-limiting online guessing without delaying a success. A missing registry (allocation failure) degrades to the global cap and host ACLs rather than refusing to start.
- **Module selection & confinement:** the client requests a module with an rsync-style `host::module[/path]` destination. The module name crosses the wire as a trailing string on the config frame (bumping `PROTOCOL_VERSION` 2.14.0 → 2.15.0; the bump is required because the config-frame layout changed and the strict same-version handshake is what prevents a peer from desynchronizing on the new trailing field). The daemon looks the module up in ITS OWN config and uses the module's `path` as the authorized root through the exact same `configure_authorization` confinement the standalone server applies to `--destination-root` (`file_open_secure_parent`, `has_path_traversal`, `path_is_within`); the client never supplies the root, every client-chosen-ownership/super-user request is refused unless the module declares `client owner = yes` (the daemon's per-module opt-in, see below), and the operator `--no-super` veto forces super-user activities off for every daemon connection. The client's `/path` part is relative inside the module and is rejected if absolute or if it contains `..`. Unknown modules are refused before any data moves (the run fails cleanly at the config handshake). An absolute destination and a module request against a non-daemon server are also refused.
- **`client owner` (client-chosen-ownership opt-in):** by default a daemon module refuses every request that would let the client pick an owner or ask for super-user activities — `--numeric-ids`, `--chown`, `--usermap`/`--groupmap`, `--fake-super`, `--copy-as`, and an explicit `--super` — at the config handshake (before `STATUS_OK`), because a daemon has no per-module opt-in for client-chosen ownership and any anonymous client could otherwise force arbitrary owner ids inside the module root. `client owner = yes` opts a single module in, allowing those requests within that module's root (a root standalone TCP listener honors them for its single operator-authorized root only when started with `--allow-super`; the flag is rejected with `--stdio`, whose client-composed remote argv must never opt back into super mode). Without the opt-in the daemon also forces super-user **device** activity off for that connection — char/block device-node creation (`--devices`) and `--write-devices` — even under the default `AUTO` mode, so a non-opted module can never be made to `mknod` or write a raw device; those entries are skipped (not refused) so an ordinary `-a` push still succeeds without device nodes. The opt-in does **not** lift the privilege requirement: `--copy-as` still needs a root receiver, and the operator `--no-super` veto still forces super-user activities off for every connection. The daemon logs a prominent startup warning for each `client owner = yes` module so the operator's deliberate choice is visible.
- **`client owner` (client-chosen-ownership opt-in):** by default a daemon module refuses every request that would let the client pick an owner or ask for super-user activities — `--numeric-ids`, `--chown`, `--usermap`/`--groupmap`, `--fake-super`, `--copy-as`, and an explicit `--super` — at the config handshake (before `STATUS_OK`), because a daemon has no per-module opt-in for client-chosen ownership and any anonymous client could otherwise force arbitrary owner ids inside the module root. A plain preserve-source request (`-a`/`-o`/`-g`) is **not** refused: the module forces super-user activities off for that connection, so no ownership is applied, and it logs a warning that the requested ownership will not be applied (the transfer itself still succeeds). `client owner = yes` opts a single module in, allowing those requests within that module's root (a root standalone TCP listener honors them for its single operator-authorized root only when started with `--allow-super`; the flag is rejected with `--stdio`, whose client-composed remote argv must never opt back into super mode). Without the opt-in the daemon also forces super-user **device** activity off for that connection — char/block device-node creation (`--devices`) and `--write-devices` — even under the default `AUTO` mode, so a non-opted module can never be made to `mknod` or write a raw device; those entries are skipped (not refused) so an ordinary `-a` push still succeeds without device nodes. The opt-in does **not** lift the privilege requirement: `--copy-as` still needs a root receiver, and the operator `--no-super` veto still forces super-user activities off for every connection. The daemon logs a prominent startup warning for each `client owner = yes` module so the operator's deliberate choice is visible.
- **`read only` safe default:** every network transfer FastSync currently supports is a push that writes under the module root, so a `read only` module refuses the connection (clear server log "module is read only"; the client exits non-zero, nothing is transferred). A future pull/list operation can be opened up when it exists; the knob is already stored.
- **Direction — remote source / pull is intentionally unsupported:** FastSync is push-only. The first positional argument is always a **local** source directory and the second is the destination; only the destination is parsed for remote syntax (`user@host:path` SSH, `host::module[/path]` daemon). A remote source such as `fastsync user@host:src ./local` is deliberately **not** implemented: rsync has no pull flag (direction is positional), so supporting a remote source is an optional feature rather than a compatibility requirement, and it would require a protocol role reversal (server as sender, client as receiver) across both transports. FastSync documents this as an intentional limitation rather than a missing rsync option. <a id="direction"></a>
- **`auth users` (A7 SCRAM-SHA-256 authentication):** a module that declares `auth users` requires the client to present credentials. The config frame carries ONLY the username; the daemon answers an auth-required module with `STATUS_AUTH_CHALLENGE` (PBKDF2 iteration count, 16-byte salt, 32-byte server nonce), the client answers with `STATUS_AUTH_RESPONSE` (fresh 32-byte client nonce + a 32-byte ClientProof), and the daemon accepts only when the proof verifies **and** the username is **on the module's `auth users` list** and has a store entry, replying `STATUS_AUTH_OK` with a 32-byte ServerSignature the client verifies before proceeding. Verification is constant-time over fixed 32-byte keys (the compare runs even for a miss), username membership uses a constant-time full-length scan, and an unknown/off-list user still receives a challenge and runs the same math against a dummy verifier: a deterministic per-username salt (`HMAC-SHA256(store dummy key, username)`), the store-wide uniform iteration count and dummy keys. Re-probing the same unknown username therefore yields an identical salt and iteration count while a different username yields a different salt, so there is no user-enumeration or timing oracle. The daemon logs the username but **never the password, proof or keys**. A module WITHOUT `auth users` stays open (legitimate rsync configuration); credentials sent to such a module are ignored. Read-only is orthogonal: even a correctly authenticated push to a `read only` module is still refused (all FastSync network transfers write). Fail-closed policy: a daemon whose config declares `auth users` on any module refuses to start unless a credential store was given (`--password-file` and/or `--early-input`); a missing or empty store is never silently treated as "open". A failed handshake (missing credentials, unknown/off-list user, wrong proof or malformed data) yields a single generic `STATUS_AUTH_FAILED` and the daemon closes before any data moves. The dummy key is persisted in an owner-only `<store_path>.dummykey` sidecar (auto-created on first load, mode 0600) so the dummy salt stays stable across daemon restarts, closing the restart-gated enumeration channel. The sidecar is secret material and must be protected like the credential store (owner-only 0600, included with the store in backups and rotation). It must be preserved across restarts for that guarantee; if it cannot be created (a process-substitution/FIFO store path such as `/dev/fd/N`, a read-only filesystem, a missing directory, or a create/write/fsync/link/fchmod failure), the daemon logs a warning and uses a transient per-run key, so unknown-user challenges change across restarts and the cross-restart guarantee does not hold for that deployment. One residual is accepted: the store iteration count is observable pre-auth by design, since the miss path must match a hit. **Transport policy (hardening A7-3/S1):** an auth-required module accepts credentials only when either (a) the connection is an encrypted, verified TLS connection whose client certificate matches `--client-cn`, or (b) the connection is plaintext from a loopback TCP peer **and** the operator explicitly passed `--allow-unauthenticated`. A remote plaintext peer, and a loopback plaintext peer without that flag, are refused at the config gate before any challenge is sent; `--allow-unauthenticated` never permits remote plaintext auth (remote peers still require verified TLS). Daemon modules are a `--daemon`-only feature — the SSH `--stdio` path never loads a daemon config and is not an auth transport for them. Because the loopback allowance trusts whichever peer the kernel reports as `127.0.0.1`, it assumes nothing relays remote connections to the daemon: a local TCP forwarder or TLS-terminating proxy in front of an auth-module listener makes remote clients appear as loopback and bypasses the mutual-TLS identity check, so do not front an auth-module listener with such a relay.
@@ -654,37 +684,37 @@ now transmits targets (the prior behavior was broken/partial); its status moved
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| Path escape detection | Ensure files stay within root | ✅ Implemented | `has_path_traversal()` + realpath |
| Symlink-safe delete | Skip symlinks in delete walk | ✅ Implemented | `delete_extras_walk()` |
| Protocol version check | Verify compatible versions | ✅ Implemented | `config_receive()` |
| Max data/string/chunk sizes | Prevent OOM attacks | ✅ Implemented | Per-message limits |
| Per-connection memory limit | 1GB per connection | ✅ Implemented | `MAX_CONNECTION_MEMORY` |
| `--max-alloc=SIZE` | Limit a single memory allocation | ✅ Implemented | Caps the largest single allocation; binary units, default 1G |
| `--trust-sender` | Trust remote sender's file list | ✅ Implemented | Long-form-only, receiver-local policy that never crosses the wire. The receiver skips its redundant up-front re-validation of the incoming file list (empty/`..` path rejection and the escaping-symlink-target containment), trusting the sender instead of double-checking (fewer checks, faster, potentially unsafe, matching rsync). Off by default. The low-level fd-relative confinement primitives (`file_open_secure_parent`, the O_NOFOLLOW parent walk, leaf/destination confinement) are deliberately KEPT even under `--trust-sender`, so a hostile sender still cannot write or link outside the authorized root (see Phase-5 notes below) |
| `--old-args` | Disable modern arg protection | ✅ Implemented | SSH-only; accepted for CLI compatibility but is now a **documented no-op**: FastSync always single-quote-escapes the remote server path and each `--remote-option` value (`ssh_build_remote_command`), so a metacharacter-bearing `--rsync-path` can never be interpreted by the remote shell. The flag no longer disables that quoting (the old raw-construction behavior was an injection foot-gun and is removed); the safety-relevant behavior is identical either way |
| `--ignore-missing-args` | Ignore missing source args | ✅ Implemented | FastSync has a single source-root argument (which always exists), so the "explicitly requested source arguments" are the `--files-from` entries and the flags only ever apply there (inert without `--files-from`, like `-R`). Without the flag a listed-but-missing entry stays a hard pre-transfer error (nothing is transferred). With it each missing entry is skipped: nothing is sent for it, it never enters the keep-set, and the run succeeds for the rest — an all-missing non-empty list succeeds transferring nothing, matching rsync. `--dirs` + `--files-from` missing entries are skipped the same way. Every skipped entry is logged and a per-run warning names the count, so the handling is never a silent no-op. Divergences: an EMPTY `--files-from` file stays a hard error in every mode (no argument was requested at all; rsync likewise reports "no source files specified"); missing-arg skipping only applies to the pre-transfer list validation, so an entry that is present at preflight and vanishes mid-transfer still fails (matching rsync, whose flag "does not affect subsequent vanished-file errors"); `--no-ignore-missing-args` is not a supported negation |
| `--delete-missing-args` | Delete missing source args | ✅ Implemented | Implies `--ignore-missing-args` (order-independent) and additionally removes each missing entry's destination mirror receiver-side. The mirror is computed exactly like a present sibling's wire path: the bare relative entry under `-R`, otherwise the full source-mirror path below the destination root. rsync parity, verified against the man page: it does **not** imply `--delete` generally and is "independent of any other type of delete processing" — unrelated destination extras are untouched unless `--delete` is also present. Composition with `--delete` + timing: the exact-path deletions commit with the manifest, early for `--delete-before`/`--delete-during`, else only after a fully-successful transfer (delete-after/commit). A non-empty directory mirror is removed only when `--force` or `--delete` is in effect (otherwise it is left with a warning and the run continues, like rsync); an absent mirror is a no-op. `--force` is deletion authority and is therefore gated by the server `--allow-delete` policy exactly like `--delete`/`--delete-missing-args`: without it the receiver clears the flag, so a client cannot use `--force` to recursively replace or remove a destination directory tree. An explicitly listed missing arg is a user request, not an excluded file: its deletion is never blocked by the filter-exclusion protection of excluded destination mirrors (a mirror sitting inside a filter-excluded directory is still removed). Safety/policy: gated by the server `--allow-delete` policy like `--delete`; the request paths cross the wire only in the delete-manifest frame and are confined by the same receiver validation as the keep-set (non-empty, relative, traversal-free, bounded by the per-section/per-frame manifest caps); the `--delay-updates` staging directory and basis snapshots are protected exactly as in the extras walker. Divergence: the missing-args deletions are not counted toward `--max-delete` (they are explicit per-path requests, not discovered extras). See the Phase-3 wire note below for the `PROTOCOL_VERSION` bump |
| Path escape detection | Ensure files stay within root | ✅ Parity | `has_path_traversal()` + realpath |
| Symlink-safe delete | Skip symlinks in delete walk | ✅ Parity | `delete_extras_walk()` |
| Protocol version check | Verify compatible versions | ✅ Parity | `config_receive()` |
| Max data/string/chunk sizes | Prevent OOM attacks | ✅ Parity | Per-message limits |
| Per-connection memory limit | Cap memory per connection | ✅ Parity | `MAX_CONNECTION_MEMORY` is **256 MiB per connection** (256 * 1024 * 1024 bytes), charged across protocol reservations and decompression/chunk allocations. This is a FastSync-internal bound with no direct rsync analogue |
| `--max-alloc=SIZE` | Limit a single memory allocation | ✅ Parity | Caps the largest single allocation; binary units, default 1G |
| `--trust-sender` | Trust remote sender's file list | ⚠️ Caveat | Long-form-only, receiver-local policy that never crosses the wire. The receiver skips its redundant up-front re-validation of the incoming file list (empty/`..` path rejection), trusting the sender instead of double-checking (fewer checks, faster, potentially unsafe, matching rsync). Off by default. **It no longer affects symlink targets** (protocol 2.23.0): targets are stored verbatim under `-l` regardless of `--trust-sender`; the flag only relaxes the receiver's path-list checks. The low-level fd-relative confinement primitives (`file_open_secure_parent`, the O_NOFOLLOW parent walk, leaf/destination confinement) are deliberately KEPT even under `--trust-sender`, so a hostile sender still cannot write or link outside the authorized root (see Phase-5 notes below) |
| `--old-args` | Disable modern arg protection | ⚠️ Caveat | SSH-only; accepted for CLI compatibility but is now a **documented no-op**: FastSync always single-quote-escapes the remote server path and each `--remote-option` value (`ssh_build_remote_command`), so a metacharacter-bearing `--rsync-path` can never be interpreted by the remote shell. The flag no longer disables that quoting (the old raw-construction behavior was an injection foot-gun and is removed); the safety-relevant behavior is identical either way |
| `--ignore-missing-args` | Ignore missing source args | ⚠️ Caveat | FastSync has a single source-root argument (which always exists), so the "explicitly requested source arguments" are the `--files-from` entries and the flags only ever apply there (inert without `--files-from`, like `-R`). Without the flag a listed-but-missing entry stays a hard pre-transfer error (nothing is transferred). With it each missing entry is skipped: nothing is sent for it, it never enters the keep-set, and the run succeeds for the rest — an all-missing non-empty list succeeds transferring nothing, matching rsync. `--dirs` + `--files-from` missing entries are skipped the same way. Every skipped entry is logged and a per-run warning names the count, so the handling is never a silent no-op. Divergences: an EMPTY `--files-from` file stays a hard error in every mode (no argument was requested at all; rsync likewise reports "no source files specified"); missing-arg skipping only applies to the pre-transfer list validation, so an entry that is present at preflight and vanishes mid-transfer still fails (matching rsync, whose flag "does not affect subsequent vanished-file errors"); `--no-ignore-missing-args` is not a supported negation |
| `--delete-missing-args` | Delete missing source args | ✅ Parity | Implies `--ignore-missing-args` (order-independent) and additionally removes each missing entry's destination mirror receiver-side. The mirror is computed exactly like a present sibling's wire path: the bare relative entry under `-R`, otherwise the full source-mirror path below the destination root. rsync parity, verified against the man page: it does **not** imply `--delete` generally and is "independent of any other type of delete processing" — unrelated destination extras are untouched unless `--delete` is also present. Composition with `--delete` + timing: the exact-path deletions commit with the manifest, early for `--delete-before`/`--delete-during`, else only after a fully-successful transfer (delete-after/commit). A non-empty directory mirror is removed only when `--force` or `--delete` is in effect (otherwise it is left with a warning and the run continues, like rsync); an absent mirror is a no-op. `--force` is deletion authority and is therefore gated by the server `--allow-delete` policy exactly like `--delete`/`--delete-missing-args`: without it the receiver clears the flag, so a client cannot use `--force` to recursively replace or remove a destination directory tree. An explicitly listed missing arg is a user request, not an excluded file: its deletion is never blocked by the filter-exclusion protection of excluded destination mirrors (a mirror sitting inside a filter-excluded directory is still removed). Safety/policy: gated by the server `--allow-delete` policy like `--delete`; the request paths cross the wire only in the delete-manifest frame and are confined by the same receiver validation as the keep-set (non-empty, relative, traversal-free, bounded by the per-section/per-frame manifest caps); the `--delay-updates` staging directory and basis snapshots are protected exactly as in the extras walker. Protocol 2.23.0 parity: the missing-args exact-path removals and the ordinary extras walk **draw from one shared `--max-delete` budget**, so a capped run stops part-way and exits 25 exactly like rsync. See the Phase-3 wire note below for the `PROTOCOL_VERSION` bump |
## 16. Batch Operations
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `--write-batch=FILE` | Write batched update to file | ✅ Implemented | Phase-6 residual-batch (client-only): runs the normal live transfer AND additionally emits a self-contained single-file batch of the whole source tree. The batch is a magic/format-version header followed by length-prefixed `chunk_serialize` blobs (full file images), replayable byte-identically by `--read-batch` on another machine with no source/server. `--write-batch` drives the single-threaded transfer path (the multithreaded path consumes the config before the separate batch scan pass). See the Phase-6 batch note below |
| `--only-write-batch=FILE` | Write batch without updating dest | ✅ Implemented | Phase-6 residual-batch: emits the self-contained batch FILE only — NO destination update, NO server connection. Requires a source (scans it and serializes the full tree to FILE). Same single-file format as `--write-batch`, so the file is re-appliable via `--read-batch=FILE DEST`. See the Phase-6 batch note below |
| `--read-batch=FILE` | Read batched update from file | ✅ Implemented | Phase-6 residual-batch: applies a previously written batch FILE locally to the destination. NO source and NO server — positional args are the destination only. Reads the magic/version header, then length-prefixed records, `chunk_deserialize`, and applies each via the confined `file_save_to_disk_full` path (same O_NOFOLLOW / `..`-rejection / root-confinement as the network receiver, so an attacker-controlled batch cannot escape the destination root). Malformed/truncated/oversized/traversal records are rejected cleanly. See the Phase-6 batch note below |
| `--write-batch=FILE` | Write batched update to file | ⚠️ Caveat | Phase-6 residual-batch (client-only): runs the normal live transfer AND additionally emits a self-contained single-file batch of the whole source tree. The batch is a magic/format-version header followed by length-prefixed `chunk_serialize` blobs (full file images), replayable byte-identically by `--read-batch` on another machine with no source/server. `--write-batch` drives the single-threaded transfer path (the multithreaded path consumes the config before the separate batch scan pass). See the Phase-6 batch note below |
| `--only-write-batch=FILE` | Write batch without updating dest | ⚠️ Caveat | Phase-6 residual-batch: emits the self-contained batch FILE only — NO destination update, NO server connection. Requires a source (scans it and serializes the full tree to FILE). Same single-file format as `--write-batch`, so the file is re-appliable via `--read-batch=FILE DEST`. See the Phase-6 batch note below |
| `--read-batch=FILE` | Read batched update from file | ⚠️ Caveat | Phase-6 residual-batch: applies a previously written batch FILE locally to the destination. NO source and NO server — positional args are the destination only. Reads the magic/version header, then length-prefixed records, `chunk_deserialize`, and applies each via the confined `file_save_to_disk_full` path (same O_NOFOLLOW / `..`-rejection / root-confinement as the network receiver, so an attacker-controlled batch cannot escape the destination root). Malformed/truncated/oversized/traversal records are rejected cleanly. See the Phase-6 batch note below |
## 17. Advanced
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `--stop-after=MINS` | Stop after N minutes | ✅ Implemented | Client-only sender stop deadline (Phase 6): computing `--stop-after=MINS` (a positive minute count; 0/negative/garbage rejected) and `--stop-at=TIME` (`HH:MM`, `HH:MM:SS`, or `now+N[smhd]`; a past time stops immediately). The transfer stops ELEGANTLY at the next chunk boundary: everything already fully sent is kept and applied, the run returns 0, and --delete (late/delete-after timing) does NOT wipe the destination — when the scan is cut short the partial keep-set manifest is suppressed with a warning (the delete walk is skipped rather than acting on an incomplete keep-set, so unscanned source mirrors survive). `--delete-before`/`--delete-during` still run their complete pre-scan (which ignores the deadline). Local client-only fields: never serialized into the wire config frame, so no PROTOCOL_VERSION bump. `--stop-after` uses CLOCK_MONOTONIC; `--stop-at` uses the wall clock. Works single-threaded and under `-j`/`--threads` (multithreaded). Divergence: rsync computes `--stop-after` from the run start; FastSync likewise. When both are given, the earlier of the two deadlines wins (checked per iteration). See the Phase-6 stop notes below |
| `--stop-at=TIME` | Stop at specified time | ✅ Implemented | Same feature as `--stop-after` (deadline transfer stop), absolute wall-clock form (`HH:MM[:SS]` or `now+N[smhd]`). See the row above and the Phase-6 stop notes |
| `--fsync` | Fsync every written file before publication | ✅ Implemented | |
| `--protocol=NUM` | Force older protocol version | ✅ Implemented | Forces the wire protocol version for this transfer. FastSync has exactly ONE wire format (`PROTOCOL_VERSION`, currently 2.21.0) with no downgrade/backward-compat code paths, so `--protocol=2.21.0` is accepted (it sets the version claim the client sends, which the server already requires to match exactly) and **every other value is rejected up front** with a clear error before any connection — it does not and cannot speak an older or virtual wire format. Divergence from rsync (which negotiates a range and downgrades to an integer 0..31): FastSync's honest contract is force-to-the-one-supported-value; a genuine downgrade would require a per-version compatibility layer that does not exist. Client-only; the server-side exact-match check is unchanged. `--protocol=2.20.0`/`2.19.0`/`2.18.0`/`2.18`/`2.17.0`/`2.16.0`/`2.15.0`/`216`/`31`/garbage are all rejected. See the Phase-6 protocol note below |
| `--iconv=CONVERT_SPEC` | Charset conversion | ✅ Implemented | Charset conversion of FILE NAMES (not content) at the protocol boundary via iconv(3): `--iconv=LOCAL[,REMOTE]` — the sender converts each local filename LOCAL→REMOTE before transmitting, and the receiver converts each wire filename REMOTE→LOCAL before creating/writing. The full CONVERT_SPEC is serialized into the config frame as a new trailing string field so the peer knows the wire charset; **PROTOCOL_VERSION bumped 2.15.0 → 2.16.0**. `LOCAL[,REMOTE]` parse: single charset ⇒ LOCAL==REMOTE (identity both ways); garbage rejected up front. Validation probes BOTH directions (a spec that only opens one way is refused, as is a NUL-emitting target charset like utf-16/utf-32/ucs-2, since filenames cannot contain NUL). An unrepresentable name (EILSEQ/EINVAL) fails that path cleanly with a logged `--iconv: cannot convert file name ...` and is never written mangled/truncated. Conversion is applied at EVERY wire-path site (regular/MKDIR/hardlink path+target/symlink path+target/SPECIAL, the delete manifest, the incremental-check path, and the `-s`/`chunk_serialize` embedded blob path), on both client and server (`--iconv` is also a server/daemon option). Zero overhead when unset. See the Phase-6 iconv notes below |
| `--checksum-seed=NUM` | Set checksum seed | ✅ Implemented | Sets the seed for FastSync's whole-file xxHash64 digest (full 64-bit seed) and for the delta path's per-block xxHash32 strong checksum (low 32 bits of the seed). An explicit seed deterministically changes every computed digest on BOTH endpoints (sender and receiver share the seed via the config frame, protocol 2.10.0), so identical runs with the same seed skip the same files and a changed seed changes the digests — the explicit-seed path that makes xxHash comparisons deterministic. `--checksum-choice=md5` has no seed and ignores it (documented). The value is a strict decimal 0..2⁶⁴-1 (blank, signed, or non-numeric values are rejected). Like rsync, a seed only matters where a digest is actually computed (`--checksum` or a basis-dir run, or a delta transfer); it does not by itself enable `--checksum`/`--delta`. Divergence from rsync: the default is seed 0, and FastSync never randomizes the seed (rsync uses a random per-transfer seed when `--checksum-seed` is unset); FastSync's unset default therefore reproduces its historical byte-for-byte behavior |
| `--secluded-args`, `-s` | Use protocol to send args | ⛔ Impossible/Divergence | Accepted for CLI compatibility (including the rsync short `-s`, Phase 7 Wave A) but a documented **no-op / divergence**. rsync's `-s` protects arguments from shell expansion by shipping them over the protocol; FastSync never passes remote arguments through a shell expansion boundary in the first place — its SSH transport builds the remote argv as **single-quote-escaped shell words** (`ssh_build_remote_command`), so the injection/leak that `-s` guards against does not exist and there is nothing to "seclude". Implementing a true arg-send protocol would mean replacing the argv-based SSH launch with an in-band argument channel, a large redesign of the transport that buys no security here. Chunk serialization remains the long-only `--chunk-serialization`. |
| `--no-OPTION` | Turn off implied option | ✅ Supported | Supported boolean FastSync options and archive-implied options; unsafe or value-taking options are rejected. |
| `--stop-after=MINS` | Stop after N minutes | ✅ Parity | Client-only sender stop deadline (Phase 6): computing `--stop-after=MINS` (a positive minute count; 0/negative/garbage rejected) and `--stop-at=TIME` (`HH:MM`, `HH:MM:SS`, or `now+N[smhd]`; a past time stops immediately). The transfer stops ELEGANTLY at the next chunk boundary: everything already fully sent is kept and applied, the run returns 0, and --delete (late/delete-after timing) does NOT wipe the destination — when the scan is cut short the partial keep-set manifest is suppressed with a warning (the delete walk is skipped rather than acting on an incomplete keep-set, so unscanned source mirrors survive). `--delete-before`/`--delete-during` still run their complete pre-scan (which ignores the deadline). Local client-only fields: never serialized into the wire config frame, so no PROTOCOL_VERSION bump. `--stop-after` uses CLOCK_MONOTONIC; `--stop-at` uses the wall clock. Works single-threaded and under `-j`/`--threads` (multithreaded). Divergence: rsync computes `--stop-after` from the run start; FastSync likewise. When both are given, the earlier of the two deadlines wins (checked per iteration). See the Phase-6 stop notes below |
| `--stop-at=TIME` | Stop at specified time | ⚠️ Caveat | Same feature as `--stop-after` (deadline transfer stop), absolute wall-clock form (`HH:MM[:SS]` or `now+N[smhd]`). See the row above and the Phase-6 stop notes |
| `--fsync` | Fsync every written file before publication | ✅ Parity | |
| `--protocol=NUM` | Force older protocol version | ❌ Divergent | Forces the wire protocol version for this transfer. FastSync has exactly ONE wire format (`PROTOCOL_VERSION`, currently 2.23.0) with no downgrade/backward-compat code paths, so `--protocol=2.23.0` is accepted (it sets the version claim the client sends, which the server already requires to match exactly) and **every other value is rejected up front** with a clear error before any connection — it does not and cannot speak an older or virtual wire format. Divergence from rsync (which negotiates a range and downgrades to an integer 0..31): FastSync's honest contract is force-to-the-one-supported-value; a genuine downgrade would require a per-version compatibility layer that does not exist. Client-only; the server-side exact-match check is unchanged. `--protocol=2.21.0`/`2.20.0`/`2.19.0`/`2.18.0`/`2.18`/`2.17.0`/`2.16.0`/`2.15.0`/`216`/`31`/garbage are all rejected. See the Phase-6 protocol note below |
| `--iconv=CONVERT_SPEC` | Charset conversion | ⚠️ Caveat | Charset conversion of FILE NAMES (not content) at the protocol boundary via iconv(3): `--iconv=LOCAL[,REMOTE]` — the sender converts each local filename LOCAL→REMOTE before transmitting, and the receiver converts each wire filename REMOTE→LOCAL before creating/writing. The full CONVERT_SPEC is serialized into the config frame as a new trailing string field so the peer knows the wire charset; **PROTOCOL_VERSION bumped 2.15.0 → 2.16.0**. `LOCAL[,REMOTE]` parse: single charset ⇒ LOCAL==REMOTE (identity both ways); garbage rejected up front. Validation probes BOTH directions (a spec that only opens one way is refused, as is a NUL-emitting target charset like utf-16/utf-32/ucs-2, since filenames cannot contain NUL). An unrepresentable name (EILSEQ/EINVAL) fails that path cleanly with a logged `--iconv: cannot convert file name ...` and is never written mangled/truncated. Conversion is applied at EVERY wire-path site (regular/MKDIR/hardlink path+target/symlink path+target/SPECIAL, the delete manifest, the incremental-check path, and the `-s`/`chunk_serialize` embedded blob path), on both client and server (`--iconv` is also a server/daemon option). Zero overhead when unset. See the Phase-6 iconv notes below |
| `--checksum-seed=NUM` | Set checksum seed | ✅ Parity | Sets the seed for FastSync's whole-file xxHash digest (full 64-bit seed) and for the delta path's per-block xxHash32 strong checksum (low 32 bits of the seed). **As of protocol 2.23.0 a seed of `0` — the default when the flag is unset — is randomized per transfer and the chosen seed is sent to the receiver**, exactly like rsync, so two runs against different content do not share a predictable seed; an explicit non-zero seed is used verbatim, so an explicit seed deterministically reproduces every computed digest on BOTH endpoints (the seed crosses in the config frame). `--checksum-choice=md5` has no seed and ignores it (documented). The value is a strict decimal 0..2⁶⁴-1 (blank, signed, or non-numeric values are rejected). Like rsync, a seed only matters where a digest is actually computed (`--checksum` or a basis-dir run, or a delta transfer); it does not by itself enable `--checksum`/`--delta` |
| `--secluded-args`, `-s` | Use protocol to send args | ❌ Divergent | Accepted for CLI compatibility (including the rsync short `-s`, Phase 7 Wave A) but a documented **no-op / divergence**. rsync's `-s` protects arguments from shell expansion by shipping them over the protocol; FastSync never passes remote arguments through a shell expansion boundary in the first place — its SSH transport builds the remote argv as **single-quote-escaped shell words** (`ssh_build_remote_command`), so the injection/leak that `-s` guards against does not exist and there is nothing to "seclude". Implementing a true arg-send protocol would mean replacing the argv-based SSH launch with an in-band argument channel, a large redesign of the transport that buys no security here. Chunk serialization remains the long-only `--chunk-serialization`. |
| `--no-OPTION` | Turn off implied option | ✅ Parity | Supported boolean FastSync options and archive-implied options; unsafe or value-taking options are rejected. |
---
@@ -692,7 +722,7 @@ now transmits targets (the prior behavior was broken/partial); its status moved
**Phase 5 notes (remote-option wave):** `--remote-option=OPT` (long form only) and `--trust-sender` landed here.
- `--remote-option` is CLIENT-only and never serialized into the binary config frame. On the SSH transport the client forwards each value to the remote server by appending it to the remote command line in `ssh_build_remote_command()`, after ` --stdio`, as an individually single-quoted shell word (`'...'` with `'\''` for embedded quotes). Values are validated at CLI parse time (non-empty; no ASCII control characters) and rejected otherwise, and a non-conforming value is refused again in the command builder, so shell metacharacters (`;`, `&`, `|`, backticks, `$()`, quotes) can never break out of the quoting to inject an unrelated remote command — including after a client-side `--` separator, whose arguments are never forwarded anyway. Because the remote options affect the *remote server invocation*, not the transmitted config, the wire frame layout is unchanged, but `PROTOCOL_VERSION` was bumped **2.13.0 → 2.14.0** as the Phase-5 lockstep release marker (a 2.14 client against a 2.13 server fails the version check cleanly rather than the old server rejecting an unfamiliar forwarded argv later). Divergence: rsync's short `-M` form of `--remote-option` was intentionally NOT implemented at that time because `-M` was FastSync metadata mode; **Phase 7 Wave A later freed `-M` for `--remote-option` and moved metadata to long-only `--preserve`** (see the Sending Options table).
- `--trust-sender` is a receiver-local policy: it never crosses the wire (the sender's value is never serialized, so a wire peer can never enable it). On the receiving process it skips the up-front re-validation of the incoming file list (empty/`..` path rejection and the escaping-symlink-target containment), trusting the sender's list instead of double-checking — fewer checks, faster, and potentially unsafe, matching rsync. It is OFF by default (`config.trust_sender`). As a deliberate safety floor, the low-level fd-relative confinement primitives are NOT disabled: `file_open_secure_parent()` (O_NOFOLLOW walk, `..` rejection, root containment) and leaf/destination confinement still hold, so even under `--trust-sender` a hostile sender cannot write or create a symlink outside the authorized root — the relaxation only removes the redundant list-layer double-checks, never the root-confinement guarantees.
- `--trust-sender` is a receiver-local policy: it never crosses the wire (the sender's value is never serialized, so a wire peer can never enable it). On the receiving process it skips the up-front re-validation of the incoming file list (empty/`..` path rejection), trusting the sender's list instead of double-checking — fewer checks, faster, and potentially unsafe, matching rsync. It is OFF by default (`config.trust_sender`). Since protocol 2.23.0 it does **not** gate symlink-target handling: `-l` stores targets verbatim either way. As a deliberate safety floor, the low-level fd-relative confinement primitives are NOT disabled: `file_open_secure_parent()` (O_NOFOLLOW walk, `..` rejection, root containment) and leaf/destination confinement still hold, so even under `--trust-sender` a hostile sender cannot write or place a *path* outside the authorized root — the relaxation only removes the redundant list-layer double-checks, never the root-confinement guarantees for paths and placements.
The estimates below cover the currently unimplemented features in this document. They assume one engineer familiar with the codebase, include implementation and focused tests, and exclude production rollout time. A feature should not be marked implemented until its behavior is tested in both local and SSH/TCP paths where applicable.
@@ -796,7 +826,7 @@ These are the hardest compatibility items because they require durable formats o
**Phase 6, Wave B (iconv) shipping note (PROTOCOL 2.15.0 → 2.16.0):** `--iconv=LOCAL[,REMOTE]` converts file NAMES at the wire boundary (never content). The full CONVERT_SPEC is serialized into the config frame as a new trailing string field (empty→NULL canonicalized), so both ends share the same wire charset interpretation; this required the PROTOCOL bump because the frame is a strict ordered sequence and a peer that does not parse the new trailing field would desynchronize. Each end derives LOCAL (its own charset) and REMOTE (the wire charset): the sender opens LOCAL→REMOTE and converts every transmitted filename; the receiver opens REMOTE→LOCAL and converts every received filename before creating/writing. Conversion is applied at every wire-path site (regular/MKDIR/hardlink path+target/symlink path+target/SPECIAL, the delete manifest keep/protected/missing entries, the incremental-check path, and the embedded `-s`/chunk-blob path). A name it cannot convert (EILSEQ/EINVAL) is failed cleanly with a logged `--iconv: cannot convert file name ...` and is never written truncated/mangled. Validation probes both directions up front (both the sender local→remote and the receiver remote→local, and, for a server/daemon with its own `--iconv`, the client-REMOTE→server-LOCAL pair) so an unusable spec is rejected before the connection rather than mid-transfer, and NUL-emitting target charsets (utf-16/utf-32/ucs-2) are refused because filenames cannot contain NUL. Divergence documented upstream: the receiver does NOT half-swap; the wire charset always comes from the sender's REMOTE half, so a server whose local charset differs from the client's LOCAL must declare it with its own `--iconv`. Conversion is process-global and runs on a single thread per process (sender thread / receiver-loop thread), initialized before worker threads start and freed after they join.
**Phase 6, Wave C (protocol-version) shipping note (no PROTOCOL_VERSION change):** `--protocol=NUM` lets the client force the wire protocol version for a transfer. FastSync's protocol is a single lockstep format: the config frame is a strict ordered sequence and the server requires the client's version string to equal `PROTOCOL_VERSION` exactly (`config_receive_with_validate`, src/shared/config.c) — there are no older-format code paths and no downgrade/negotiation machinery, so a lower/higher/virtual version can never be spoken. The honest contract is therefore: `--protocol=2.21.0` (the current `PROTOCOL_VERSION`, as of the combined error-detail + server-contacting dry-run wave) is accepted and stored into the client's `version` claim (which `config_send` already transmits), and every other value — `2.20.0`, `2.19.0`, `2.18.0`, `2.18`, `2.17.0`, `2.16.0`, `2.15.0`, `3.0.0`, rsync-integer spellings like `216`/`31`, garbage, empty — is rejected up front in `validate_config()` before any connection, with a clear error that FastSync supports only its current wire protocol and cannot speak an older or virtual one. Implementation is client-only: a server-side `--protocol` is intentionally not added because the server has no negotiation (it only enforces exact match), and it could only ever be the current version. This preserves (and slightly tightens) existing validation: the client now also refuses to launch with a version it cannot actually speak, rather than only the server rejecting it later. A genuine downgrade would require a per-version compatibility layer for every frame/feature added since (append 2.10, preallocate 2.11, hardlinks 2.12, devices/specials/symlink-trust/xattr 2.13, remote-option 2.14, daemon module/auth 2.15, iconv 2.16, dir/symlink times 2.17, privilege flags --super/--copy-as 2.18, SCRAM daemon auth 2.19, packed metadata 2.20) and is intentionally out of scope — documented divergences from rsync's integer-negotiated downgrade remain.
**Phase 6, Wave C (protocol-version) shipping note (no PROTOCOL_VERSION change):** `--protocol=NUM` lets the client force the wire protocol version for a transfer. FastSync's protocol is a single lockstep format: the config frame is a strict ordered sequence and the server requires the client's version string to equal `PROTOCOL_VERSION` exactly (`config_receive_with_validate`, src/shared/config.c) — there are no older-format code paths and no downgrade/negotiation machinery, so a lower/higher/virtual version can never be spoken. The honest contract is therefore: `--protocol=2.23.0` (the current `PROTOCOL_VERSION`, as of the rsync-parity wave) is accepted and stored into the client's `version` claim (which `config_send` already transmits), and every other value — `2.22.0`, `2.21.0`, `2.20.0`, `2.19.0`, `2.18.0`, `2.18`, `2.17.0`, `2.16.0`, `2.15.0`, `3.0.0`, rsync-integer spellings like `216`/`31`, garbage, empty — is rejected up front in `validate_config()` before any connection, with a clear error that FastSync supports only its current wire protocol and cannot speak an older or virtual one. Implementation is client-only: a server-side `--protocol` is intentionally not added because the server has no negotiation (it only enforces exact match), and it could only ever be the current version. This preserves (and slightly tightens) existing validation: the client now also refuses to launch with a version it cannot actually speak, rather than only the server rejecting it later. A genuine downgrade would require a per-version compatibility layer for every frame/feature added since (append 2.10, preallocate 2.11, hardlinks 2.12, devices/specials/symlink-trust/xattr 2.13, remote-option 2.14, daemon module/auth 2.15, iconv 2.16, dir/symlink times 2.17, privilege flags --super/--copy-as 2.18, SCRAM daemon auth 2.19, packed metadata 2.20, error-detail/dry-run 2.21, preserve-attribute split 2.22, rsync-parity wave 2.23) and is intentionally out of scope — documented divergences from rsync's integer-negotiated downgrade remain.
**Phase-1/2 selection-and-update status correction (docs):** `-I/--ignore-times`, `--size-only`, `-@/--modify-window`, `--existing`, `--ignore-existing`, `-u/--update`, `-W/--whole-file`, and `--compress-threads` were previously listed as not-implemented in this document but are in fact fully implemented and tested on `dev`. This pass corrects the matrix to match the code. The realistic model of these is that FastSync is a *sender-driven* whole-tree copy, so the size+mtime quick-check and all three receiver-policy skips (`--existing`, `--ignore-existing`, `-u`) are evaluated against the **destination** on the receiver side, and their booleans cross the wire in the config frame. `-I`/`--size-only`/`--modify-window` modify the `--incremental` per-file `STATUS_CHECK` handshake's match predicate (`-I` disables the mtime leg and forces transfer; `--size-only` drops only the mtime leg; `--modify-window` adds tolerance to `metadata_mtime_matches`); they require `--incremental` (or a basis dir) to have a handshake to affect, mirroring how they only matter where a quick-check exists in rsync. `--existing`/`--ignore-existing`/`-u` are receiver write-time policies (skipping the write / newer-destination guard) applied across the regular-file, `--delay-updates`-staged, hardlink-sibling, and special/device paths; `-u` implies `-M` metadata and uses a second-then-nanosecond strict `>` newer check; both correctly influence `--remove-source-files` (a skipped source is not removed). `-W/--whole-file` disables block-level delta (opt-in via `--delta`), folded into the wire `use_delta` so no protocol bump was needed, and makes `--fuzzy` inert; `--append`/`--append-verify` are rejected with `-W`. `--compress-threads=NUM` (1..64, client-only, never crosses the wire) sizes the zstd compression worker pool. No code was changed by this correction; the implementation had landed in earlier merge waves (feat/ignore-times, feat/ignore-existing via the newer `file_to_disk_secure_no_replace`/`linkat EEXIST` path, feat/size-only, feat/modify-window, feat/whole-file, feat/update, compression-threads).
@@ -806,7 +836,7 @@ These are the hardest compatibility items because they require durable formats o
These are the last compatibility items and the closing phase toward rsync flag parity. Per the project decision: every rsync flag (short **and** long) that is *possible* gets real rsync-parity behavior; anything physically impossible becomes an explicit **Impossible/Divergence** status (accepted for CLI compatibility, safely inert, with coverage tests proving that); and the two privilege flags (`--super`, `--copy-as`) adopt the deliberately-scoped **safe-subset + clear-refusal** model rather than blind elevation. The remaining `⚠️ Partial`, `🔄 Compatibility No-op`, `🔀 Alt Arg`, and `❌ Not Implemented` rows in the Summary are this phase's scope. All Wave A renames are **client-side only** (the wire config fields `use_compression`/`use_metadata`/`use_sendfile`/`use_chunk_serialization` are unchanged), so they require **no `PROTOCOL_VERSION` bump**.
**Wave A — CLI namespace parity (rename colliding FastSync short flags) — ✅ implemented.** This freed the short letters rsync needs and made the three `🔀 Alt Arg` rows real. `-c`→`--checksum`, `-m`→`--prune-empty-dirs`, `-M`→`--remote-option`, `-f`→`--filter`, `-s`→`--secluded-args`, `-p`→`--perms`, `-T`→`--temp-dir`, `-a`/`--archive`→real `-rlptgoD`. FastSync's own flags moved to long-form-only or new shorts: `-j`/`--threads` (multithreading), `--preserve` (metadata), `--sendfile`, `--chunk-serialization`, `--timeout`, `--ssh-port`. The server's independent little CLI keeps `-p` as its port. All client-side, no wire change, no `PROTOCOL_VERSION` bump. Unit tests 37/37, full integration 400 passed, cppcheck and clang-format clean. Known Wave-A limitation: `--no-perms`/`--no-compress`-style negation of the newly-aliased shorts is not wired into the negatable set (only the long-form `--preserve`/`--compress`/`--no-links` negations exist); `--archive --no-perms` is consequently not supported yet — a minor deviation from rsync, acceptable for Wave A.
**Wave A — CLI namespace parity (rename colliding FastSync short flags) — ✅ implemented.** This freed the short letters rsync needs and made the three `🔀 Alt Arg` rows real. `-c`→`--checksum`, `-m`→`--prune-empty-dirs`, `-M`→`--remote-option`, `-f`→`--filter`, `-s`→`--secluded-args`, `-p`→`--perms`, `-T`→`--temp-dir`, `-a`/`--archive`→real `-rlptD`. FastSync's own flags moved to long-form-only or new shorts: `-j`/`--threads` (multithreading), `--preserve` (metadata), `--sendfile`, `--chunk-serialization`, `--timeout`, `--ssh-port`. The server's independent little CLI keeps `-p` as its port. All client-side, no wire change, no `PROTOCOL_VERSION` bump. Unit tests 37/37, full integration 400 passed, cppcheck and clang-format clean. Known Wave-A limitation: `--no-perms`/`--no-compress`-style negation of the newly-aliased shorts was not wired into the negatable set (only the long-form `--preserve`/`--compress`/`--no-links` negations existed), so `--archive --no-perms` was initially unsupported — a minor deviation from rsync. The preserve-attribute split wave below resolves the preservation side: `--no-perms`/`--no-times`/`--no-owner`/`--no-group` and `--no-preserve` now work, so `--archive --no-perms` is supported.
| FastSync flag today | rsync wants that name | Proposed rename |
|---------------------|----------------------|-----------------|
@@ -817,29 +847,169 @@ These are the last compatibility items and the closing phase toward rsync flag p
| `-s` / `--chunk-serialization` | `-s` = `--secluded-args`/`--protect-args` | → `--chunk-serialization` (long-only) |
| `-p` (SSH port) | `-p` = `--perms` | → `--port` (long-only; `--server-port` already exists) |
| `-T` / `--timeout` | `-T` = `--temp-dir` | → `--timeout` (long-only) |
| `-a` / `--archive` (= `-c -m -M`) | `-a` = `-rlptgoD` | → becomes **real rsync `-a`** after the renames |
| `-a` / `--archive` (= `-c -m -M`) | `-a` = `-rlptD` | → becomes **real rsync `-a`** after the renames |
**Wave B — Output & filesystem completion (✅ implemented).** `-S`/`--sparse` (`⚠️→✅`): real hole preservation — a sparse-aware writer (`write_all_sparse`) skips all-zero runs ≥ 4096 bytes with `lseek(SEEK_CUR)` and `ftruncate`s the final size, wired into both the atomic temp+rename store and `--inplace` receiver-side with **no wire change** (the full file image is already in memory; the ftruncate presize is kept). `-P` (`⚠️→✅`): interrupted-write retention — on a save failure after data reached the temp fd, `--partial` now renames the already-written temp to the destination path (best-effort; falls through to the normal unlink on failure, never retains when `--partial` is off) so a later `--append`/`--append-verify` run can resume. `--block-size=SIZE` (`⚠️→✅`): promoted after verification — `--block-size` is now an alias for `--delta-block`, both set `config->delta_block_size`, which the delta engine already honored end-to-end (`delta_signature_create_seeded` + `delta_apply`); out-of-range values keep the default. `--fake-super` (`⚠️→✅`): added `fake_super_restore_fd` to parse and re-apply the recorded `user.fastsync.stat` record fd-relative (fchown best-effort/non-root skipped, fchmod, futimens); a save under `--fake-super` now re-applies the recorded attrs instead of only recording them, with the recording format unchanged. `--stderr=client` (`⚠️→⛔ Impossible/Divergence`): FastSync has no rsync client-message channel, and `client` is rejected at CLI parse — the rejection is the documented behavior (unit-tested). `-N`/`--crtimes` (`⚠️→⛔ Impossible/Divergence`): birth-times cannot be set by any portable fs call (`utimensat` sets only atime/mtime); capture/transmit stays, setting is impossible, the flag is accepted and safely inert. Review-hardening (post-eval): fake-super replay applies the mode through the same sanitization as the normal metadata path (group/other write bits are never granted); `--sparse` takes precedence over `--preallocate` (posix_fallocate skipped so holes survive); `--partial` retention is disabled for `--no_replace` (ignore/existing) and only marks a write-attempt after the actual write begins; `--block-size=SIZE`/`--delta-block=SIZE` inline forms are accepted.
**Wave B — Output & filesystem completion (✅ implemented).** `-S`/`--sparse` (`⚠️→✅`): real hole preservation — a sparse-aware writer (`write_all_sparse`) skips all-zero runs ≥ 4096 bytes with `lseek(SEEK_CUR)` and `ftruncate`s the final size, wired into both the atomic temp+rename store and `--inplace` receiver-side with **no wire change** (the full file image is already in memory; the ftruncate presize is kept). `-P` (`⚠️→✅`): interrupted-write retention — on a save failure after data reached the temp fd, `--partial` now renames the already-written temp to the destination path (best-effort; falls through to the normal unlink on failure, never retains when `--partial` is off) so a later `--append`/`--append-verify` run can resume. `--block-size=SIZE` (`⚠️→✅`): promoted after verification — `--block-size` is now an alias for `--delta-block`, both set `config->delta_block_size`, which the delta engine already honored end-to-end (`delta_signature_create_seeded` + `delta_apply`); out-of-range values keep the default. `--fake-super` (`⚠️→✅`): added `fake_super_restore_fd` to parse and re-apply the recorded `user.fastsync.stat` record fd-relative (mode/time only — protocol 2.23.0: **never a real chown**; the resolved owner is recorded for a later privileged restore); a save under `--fake-super` now re-applies the recorded attrs instead of only recording them, with the recording format unchanged. `--stderr=client` (`⚠️→❌ Divergent`): FastSync has no rsync client-message channel, and `client` is rejected at CLI parse — the rejection is the documented behavior (unit-tested). `-N`/`--crtimes` (`⚠️→❌ Divergent`): birth-times cannot be set by any portable fs call (`utimensat` sets only atime/mtime); capture/transmit stays, setting is impossible, the flag is accepted and safely inert. Review-hardening (post-eval): fake-super replay applies the mode through the shared `metadata_mode_for_policy` helper (protocol 2.23.0: exactly the source mode under `-p`, with no masking); `--sparse` takes precedence over `--preallocate` (posix_fallocate skipped so holes survive); `--partial` retention is disabled for `--no_replace` (ignore/existing) and only marks a write-attempt after the actual write begins; `--block-size=SIZE`/`--delta-block=SIZE` inline forms are accepted.
**Wave C — Devices & special files (finalize statuses + tests) (✅ implemented).** The four special-file rows are finalized with coverage tests. `--devices`, `--copy-devices`, and `--write-devices` are **✅ Implemented**, each with a documented, safety-driven divergence: device-node creation is privilege-gated, so a receiver without `CAP_MKNOD` skips that entry with a warning (a per-entry skip, never a transfer failure); `--copy-devices` copies a device/FIFO's reported size into an ordinary regular file (a size-bounded safe divergence from rsync's unbounded dd-like read); `--write-devices` writes only into an existing char/block node under the confined receive root and skips every unusable target rather than clobbering or aborting. `--specials` is classified **⛔ Impossible/Divergence** for one reason only: **FIFO recreation works** (unprivileged `mkfifo`, asserted under CI), but **sockets cannot be recreated by any standard filesystem call**, so a source socket is skipped with an explicit note. Tests assert FIFO recreation, the safe socket skip, the regular-file result of `--copy-devices`, the skipped/missing and non-device `--write-devices` targets, and (root-gated) real device-node creation; a root runner additionally drops the receiver to an unprivileged user to assert the `CAP_MKNOD` skip is graceful.
**Wave C — Devices & special files (finalize statuses + tests) (✅ implemented).** The four special-file rows are finalized with coverage tests. `--devices`, `--copy-devices`, and `--write-devices` are **✅ Implemented**, each with a documented, safety-driven divergence: device-node creation is privilege-gated, so a receiver without `CAP_MKNOD` skips that entry with a warning (a per-entry skip, never a transfer failure); `--copy-devices` copies a device/FIFO's reported size into an ordinary regular file (a size-bounded safe divergence from rsync's unbounded dd-like read); `--write-devices` writes only into an existing char/block node under the confined receive root and skips every unusable target rather than clobbering or aborting. `--specials` reclassified from **⛔ Impossible/Divergence** to **✅ Parity** in protocol 2.23.0: **FIFO recreation works** (unprivileged `mkfifo`) **and unix sockets are recreated** with `mknod(S_IFSOCK)`, which Linux permits unprivileged (the flag previously assumed sockets were impossible — see the `--specials` row). Tests assert FIFO recreation, socket recreation, the regular-file result of `--copy-devices`, the skipped/missing and non-device `--write-devices` targets, and (root-gated) real device-node creation; a root runner additionally drops the receiver to an unprivileged user to assert the `CAP_MKNOD` skip is graceful.
**Wave D — Times superstructure & arg-protection no-ops (✅ implemented, `--secluded-args` ⛔).** `-O`/`--omit-dir-times` and `-J`/`--omit-link-times` are now **real modifiers** (both `🔄 → ✅ Implemented`), reversing the old "never preserves directory/symlink times" divergence:
**Wave D — Times superstructure & arg-protection no-ops (✅ implemented, `--secluded-args` ❌).** `-O`/`--omit-dir-times` and `-J`/`--omit-link-times` are now **real modifiers** (both `🔄 → ✅ Implemented`), reversing the old "never preserves directory/symlink times" divergence:
- **Directory times.** The recursive scanner captures every traversed source directory's metadata (mtime, plus atime under `-U`) into a per-transfer list — two paths are covered: the sequential `DirectoryScanner` captures each opened directory (including the transfer root), and the parallel scanner captures both the root in `parallel_scanner_create_with_options` and each worker's subdirectories in `open_next_directory` (appends are guarded by a mutex shared with the sender's pipeline context). The sender transmits them in trailing `STATUS_DIR_TIMES` frames (each: int count + count × (wire path, metadata) pairs) sent **after all file data and after the optional delete manifest**, just before `STATUS_FINISHED`. A tree larger than `MAX_MANIFEST_ENTRIES` (1 048 576) directories is chunked into repeated frames, each within the receiver's per-frame bound. A dir-time entry is RECORD-ONLY (`file->dir_time_only`): `file_save_to_disk_full` returns `FILE_SAVE_SKIPPED` without creating anything, so a source directory that was empty (or pruned by `-m/--prune-empty-dirs`) is never resurrected. The receiver accumulates received directory metadata in a `DirTimeList` and applies it only at the very end — after the entire stream, after the commit-style `--delete` deletion, and after `--delay-updates` publication — because creating or removing a child bumps the parent's mtime. Application is fd-relative/walk-confined (`file_open_secure_parent` + `utimensat(..., AT_SYMLINK_NOFOLLOW)`) and best-effort per entry: an absent path (an intentionally uncreated empty dir) is skipped QUIETLY and only a real existing directory is stamped. `-O` (config boolean, already on the wire) makes the receiver skip the whole set. The single-threaded sink applies in `receiver_send_success_frame`; the `-j`/`--threads` sink accumulates in `write_thread` and server.c applies after both threads join and the deletion commits.
- **Symlink times/owner/mode.** `STATUS_SYMLINK` already carried metadata; the receiver now applies it with no-follow primitives only: `utimensat(..., AT_SYMLINK_NOFOLLOW)`, best-effort `fchmodat(..., AT_SYMLINK_NOFOLLOW)` (honest no-op where unsupported, e.g. Linux), and policy-gated `fchownat(..., AT_SYMLINK_NOFOLLOW)` via a new `identity_apply_ownership_link` that shares the identity resolver with the fd path. `-J` suppresses only the timestamps; ownership stays governed by the identity opt-in (`--numeric-ids`/`--usermap`/`--groupmap`/`--chown`) exactly like regular files. A symlink has no children, so this is applied immediately at creation.
- **Wire:** the shared `STATUS_DIR_TIMES` frame (and metadata on `STATUS_MKDIR` for `--dirs` entries) is a frame-sequence change, so `PROTOCOL_VERSION` was bumped **2.16.0 → 2.17.0**; every version-sensitive test (`--protocol` accepted/rejected values) was updated. The config-frame layout itself is unchanged (the omit booleans already crossed). Non-metadata and `--no-preserve` transfers send no `STATUS_DIR_TIMES` frame and no directory metadata, keeping them byte-identical.
`--secluded-args` (`🔄 → ⛔ Impossible/Divergence`): a true arg-send protocol would replace the argv-based SSH launch with an in-band channel, and FastSync already builds the remote SSH argv injection-safe (single-quote-escaped shell words), so there is no argument-leak to close; the already-safe behavior is documented in the row and no transport change is made.
`--secluded-args` (`🔄 → ❌ Divergent`): a true arg-send protocol would replace the argv-based SSH launch with an in-band channel, and FastSync already builds the remote SSH argv injection-safe (single-quote-escaped shell words), so there is no argument-leak to close; the already-safe behavior is documented in the row and no transport change is made.
**Wave E (LAST) — Privilege: `--super`/`--no-super` and `--copy-as=USER[:GROUP]` (✅ implemented).** FastSync adopts a **safe-subset + clear-refusal** privilege model: it never blind-elevates and never calls `setuid`/`seteuid`/`setgid`. All privileged operations remain fd-relative and confined below the authorized receive root.
`--super`/`--no-super` set a receiver-side tri-state `Config->super_mode` (`SUPER_MODE_AUTO`/`ON`/`OFF`). `privilege_super_permitted()` / `privilege_super_mode_permitted()` (src/shared/identity.c) return true for `ON` and `AUTO` (AUTO preserves FastSync's historical best-effort attempt, where the kernel refuses an unprivileged call and the caller skips it) and false only for `OFF`. The gate covers every super-user activity FastSync performs: ownership application (`identity_apply_ownership`/`_link`), char/block device-node creation (`file_save_special_to_disk`), writes into an existing device (`--write-devices`), and the `--fake-super` owner replay. Unprivileged FIFO creation is deliberately unaffected. `--super` does **not** imply `--numeric-ids`: ownership is applied only when an explicit identity policy (`--usermap`/`--groupmap`/`--chown`/`--numeric-ids`/`--copy-as`) is also given. `--no-super` suppresses those activities even for a root receiver. A non-root receiver given `--super` logs one warning at activation (`identity_set_active`); each confined attempt is then refused by the kernel and skipped, never aborting. The confinement floor is unchanged (`file_open_secure_parent`, `O_NOFOLLOW`, root/path checks). Operator control: the server CLI accepts `--no-super`, a veto that forces `OFF` for every connection, refuses any client `--copy-as`, and neutralizes an explicit `--super` (the connection is accepted but no super-user activity is attempted). A privileged (root) standalone TCP listener instead defaults to `OFF` and requires the server-only `--allow-super` opt-in to attempt any super-user activity (the flag is rejected with `--stdio`, whose client-composed remote argv must never defeat the default; use a forced command if the default must hold); a non-root server is unchanged. On a daemon, a module that has not opted in with `client owner = yes` additionally has super-user device activity forced off (see the Daemon Mode notes).
`--super`/`--no-super` set a receiver-side tri-state `Config->super_mode` (`SUPER_MODE_AUTO`/`ON`/`OFF`). `privilege_super_permitted()` / `privilege_super_mode_permitted()` (src/shared/identity.c) return true for `ON` and `AUTO` (AUTO preserves FastSync's historical best-effort attempt, where the kernel refuses an unprivileged call and the caller skips it) and false only for `OFF`. The gate covers every super-user activity FastSync performs: ownership application (`identity_apply_ownership`/`_link`), char/block device-node creation (`file_save_special_to_disk`), writes into an existing device (`--write-devices`), and the `--fake-super` owner replay. Unprivileged FIFO creation is deliberately unaffected. `--super` does **not** imply `--numeric-ids`: ownership is applied only when an explicit identity policy (`--usermap`/`--groupmap`/`--chown`/`--numeric-ids`/`--copy-as`) or a preserve-source request (`-o`/`-g`, or `-a`/`--archive`) is also given. `--no-super` suppresses those activities even for a root receiver. A non-root receiver given `--super` logs one warning at activation (`identity_set_active`); each confined attempt is then refused by the kernel and skipped, never aborting. The confinement floor is unchanged (`file_open_secure_parent`, `O_NOFOLLOW`, root/path checks). Operator control: the server CLI accepts `--no-super`, a veto that forces `OFF` for every connection, refuses any client `--copy-as`, and neutralizes an explicit `--super` (the connection is accepted but no super-user activity is attempted). A privileged (root) standalone TCP listener instead defaults to `OFF` and requires the server-only `--allow-super` opt-in to attempt any super-user activity (the flag is rejected with `--stdio`, whose client-composed remote argv must never defeat the default; use a forced command if the default must hold); a non-root server is unchanged. On a daemon, a module that has not opted in with `client owner = yes` additionally has super-user device activity forced off (see the Daemon Mode notes).
`--copy-as=USER[:GROUP]` is the safe subset. FastSync's receiver is multithreaded, so a real credential switch is unsafe; instead the receiver forces the ownership of **every entry it writes** — regular files, symlinks, directories (including implicitly-created parents), and special nodes — to the resolved target ids through the confined fd-relative identity path. USER is resolved on the client (name, `@N`/bare N, or `*` = client euid); when `:GROUP` is omitted the user's primary gid is used (falling back to `gid == uid` for a numeric id with no local passwd entry). It requires a privileged (root) receiver: an unprivileged receiver refuses the whole transfer at the config handshake, before `STATUS_OK`, so no data is ever written with the wrong ownership. A `--copy-as` chown failure on a capability-restricted root is logged at ERROR (never silently downgraded). `--copy-as` implies metadata (`--no-preserve` is rejected) and `--fake-super` cannot override it. Daemon policy: a `--daemon` receiver refuses **every** client-chosen-ownership / super-user request — `--numeric-ids`, `--chown`, `--usermap`/`--groupmap`, `--fake-super`, `--copy-as`, and explicit `--super` — unless the selected module opts in with `client owner = yes`; without that per-module opt-in any client could force arbitrary ownership inside the module root (a root standalone TCP listener, which serves one operator-authorized root, honors these requests only when started with `--allow-super`; the flag is rejected with `--stdio`). A `--copy-as` chown failure on a capability-restricted root marks the entry as failed rather than reporting success with the wrong owner.
**Wire:** two trailing config-frame blocks after the `--iconv` spec, in fixed order — `send_privilege_options`/`receive_privilege_options` (one `super_mode` int, validated `0..2`), then `send_copy_as_options`/`receive_copy_as_options` (presence int + two int32 ids, validated `>= 0`, with `copy_as_set ⇒ use_metadata`). `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Divergences from rsync:** rsync's `--super` elevates the receiver and `--copy-as` actually switches its credentials; FastSync never elevates and only permits/forwards confined attempts, and `--copy-as` forces ownership rather than switching identity.
**Post-Phase-7 Summary (after Waves A–E).** ✅143 / 🔀0 / ⛔4 / ⚠️0 / 🔄0 / ❌0 = 147. The 3 `🔀 Alt Arg` rows (`-a`, `-p`, `-z`) are ✅ (Wave A). All 10 prior `⚠️ Partial` rows are resolved to ✅ (`-S`, `-P`, `--block-size`, `--fake-super`, `--devices`, `--copy-devices`, `--write-devices`) or ⛔ (`--stderr=client`, `-N/--crtimes`, `--specials` for the impossible socket case). The 3 `🔄 Compatibility No-op` rows are resolved: `-O`/`-J` are now real ✅ (Wave D), `--secluded-args` is ⛔. The **Impossible/Divergence** bucket holds the 4 physically-impossible/divergent flags: `--stderr=client`, `-N/--crtimes`, `--specials` (sockets), `--secluded-args`. The last two `❌ Not Implemented` rows — `--super` and `--copy-as=USER[:GROUP]` — are now ✅ (Wave E). **No `❌ Not Implemented` rows remain.**
**Current honest status (protocol 2.23.0).** ✅ Parity 83 / ⚠️ Caveat 63 / ❌ Divergent 4 = 150 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. The reclassification makes the differences explicit and the rsync-parity wave closed the genuine gaps (short options, clustering, checksum/compression choices, seed randomization, timeout defaults, delete scoping and partial limits, verbatim symlink storage, socket recreation, `--chmod`, and more — see the next section). The four ❌ rows are `--stderr=client` (no rsync client-message channel), `-N/--crtimes` (no portable setter), `--protocol=NUM` (only the current wire version is accepted), and `-s/--secluded-args` (accepted no-op). `--specials` is now ✅ because sockets are recreated with `mknod(S_IFSOCK)`. **No `❌ Not Implemented` rows remain.**
**Preserve-attribute split (protocol 2.21.0 → 2.22.0) — ✅ implemented.** FastSync splits the former single metadata bundle into four independent, rsync-compatible per-attribute flags — `-p/--perms`, `-t/--times`, `-o/--owner`, `-g/--group` — each with a negation (`--no-perms`/`--no-times`/`--no-owner`/`--no-group`, short `--no-p`/`--no-t`/`--no-o`/`--no-g`), plus `--no-preserve` clearing all four. `-a/--archive` is now full rsync `-rlptgoD` (owner and group included, though their application stays privilege-gated), `-A/--acls` implies `-p`, `-X/--xattrs` does not, `-E/--executability` sets only executability, and `-U`/`-N` do not imply `-t`. `--incremental`/`--delta` still auto-preserve perms+times unless the user explicitly negated them. Wire: the binary config frame gains four appended booleans (`preserve_perms`/`preserve_times`/`preserve_owner`/`preserve_group`) after `omit_link_times`, so `PROTOCOL_VERSION` is bumped **2.21.0 → 2.22.0**; the fixed-width `FileMetadata` layout is unchanged and the receiver gates the metadata frame on a derived `use_metadata`. Receiver behavior: each attribute is applied independently, directory modes are applied under `-p` (at the end of the transfer, alongside dir times), symlink mode under `-p`, and `-O/--omit-dir-times` suppresses directory times only. Documented divergences as of 2.22.0, **all but (d)/(e) removed by the rsync-parity wave (protocol 2.23.0)**: (a) the mode-masking divergence is **gone** — under `-p` the source mode is now copied exactly, including `S_IWGRP`/`S_IWOTH` and setuid/setgid/sticky; (b) a brand-new file without `-p` still gets `source_mode & ~umask` when metadata is present (else the historical fixed `0644`), and a new *directory* without `-p` still uses FastSync's `0755` default; (c) the `--chmod`-implies-`-p` divergence is **gone** — `--chmod` no longer implies `-p` (rsync parity); (d) `-o`/`-g` map by name on the receiver with a raw-numeric fallback (only numeric ids cross the wire); (e) a daemon module without `client owner = yes` does not refuse a plain `-a`/`-o`/`-g` — it forces super off, applies no ownership, and logs a warning, while explicit `--chown`/`--usermap`/`--groupmap`/`--numeric-ids`/`--copy-as`/`--super` are still refused.
## Rsync-Parity Wave (protocol 2.23.0)
This wave closed the remaining CLI, filesystem, ownership, deletion, and output
gaps against rsync 3.4.1. It is a wire change: `PROTOCOL_VERSION` moved
**2.22.0 → 2.23.0** because the delete manifest gained a synchronized-directory
section and the terminal status gained `STATUS_DELETE_LIMIT` (see the deletion
notes above). Everything below is implemented and covered by unit and
integration tests unless it is explicitly listed as a limitation.
### CLI parsing
- **Short options now parsed:** `-r` (`--recursive`), `-b` (`--backup`),
`-L` (`--copy-links`), and `-B` (`--block-size`/`--delta-block`) are accepted
as rsync spells them.
- **rsync short-option clustering:** a token is expanded before parsing, so
`-av` → `-a -v`, `-aAX` → `-a -A -X`, `-rlpt` → `-r -l -p -t`, and so on.
A value-taking short option consumes the remainder of its token
(`-B1000` → `-B 1000`, `-essh` → `-e ssh`, `-MOPT` → `-M OPT`), with an
optional leading `=` dropped (`-B=1000`); a value-taking option written alone
takes the next argv entry, which is copied verbatim so a value that happens to
start with `-` (e.g. `--filter "- *.tmp"`) is not mistaken for a cluster.
- **Inline/attached long values:** `--opt=value` is accepted uniformly, and each
expanded token is mapped back to its original argv index so positional
arguments stay correct.
- **`-c` implies the checksum quick-check.** `-c`/`--checksum` sets the
incremental checksum comparison rather than doing nothing on its own; like
rsync, `-c` does not imply `-t`.
### Checksums and compression
- **`--checksum-choice`/`--cc`** accepts `xxh64` (default), `xxhash`, `xxh3`,
`xxh128`, `md5`, and `auto`; `md4`, `sha1`, `none`, and the two-name
`transfer,pre-transfer` form are **rejected by name**.
- **`--checksum-seed=0` is randomized per transfer** (the chosen seed is sent to
the receiver), matching rsync; an explicit non-zero seed is used verbatim.
- **`--compress-choice`/`--zc`** accepts `zstd` (default), `none`, and `auto`;
rsync's `lz4`/`zlib`/`zlibx` are **rejected by name**.
- **`--skip-compress`** uses rsync 3.4.1's built-in default suffix list when no
list is supplied; an explicit list replaces it.
- **`--no-whole-file`** is accepted as the rsync spelling that clears
`-W`/`--whole-file`.
### Timeouts and limits
- **`--timeout` defaults to 0 (disabled) and `--contimeout` to 60 s; `0`
disables either**, matching rsync.
- **`--max-alloc=0` means "no local allocation limit"** (rsync semantics). A
standalone server still keeps its own ceiling for the peer it serves.
### Filesystem and deletion semantics
- **`--temp-dir` is confined to the receive root on the receiver:** a relative
dir resolves below it; an absolute path or one containing `..` is rejected.
An `EXDEV` install falls back to a non-atomic copy instead of aborting.
- **Deletion scoping:** the manifest carries the synchronized directories, so
the extras walk only visits their subtrees; `--files-from` subsets no longer
delete untransmitted paths outside the listed directories.
- **`--delete-excluded`** removes filter-excluded mirrors but never
`--max-size`/`--min-size`-pruned mirrors (separate, always-on protection).
- **Destination symlinks** are unlinked by name, never followed; a directory
still holding one survives.
- **`--max-delete=N` is partial:** delete up to N, skip the rest, exit **25**.
`--delete-missing-args` removals draw from the same budget.
- **`--force` is honored during `--delay-updates` publication.**
- **`-x`/`--one-file-system` emits the mount-point directory entry** (an empty
directory at the destination) without descending into it.
- **`--include`/`--exclude` are an ordered first-match rule list**, evaluated
like `--filter`/`-F`/`-C` (first match wins), so an earlier rule can override a
later one.
### Ownership and metadata
- **`--numeric-ids` is a mapping modifier only** — it changes *how* ids map, not
*whether* ownership is applied; combine it with `-o`/`-g`, `-a`, or an
explicit map.
- **`--usermap`/`--groupmap`** support names, `@N`/bare `N` ids, inclusive
`LOW-HIGH` ranges, `*`, empty-`FROM` (unnamed ids), and receiver-resolved `TO`
names.
- **`--chown` conflicts with `--usermap`/`--groupmap` on the same side** and is a
clear configuration error (matching rsync) instead of an order-dependent
winner.
- **`--fake-super` never real-chowns.** It records the *resolved* owner (the
active mapping, else the source id) in `user.fastsync.stat` for a later
privileged restore and replays only mode/times. Directory ownership and
directory xattrs/ACLs are preserved alongside file entries.
- **`--chmod`** implements rsync's `D`/`F`/`X` selectors, `s`/`t`, append
semantics, does not imply `-p`, and applies its changes without sanitization.
### Symlinks and special files
- **`-l`/`--links` stores symlink targets verbatim** (absolute and `..`-bearing
targets included), matching rsync. `--safe-links`, `--copy-unsafe-links`, and
`--munge-links` (which now uses rsync's `/rsyncd-munged/` marker) match rsync
and are applied sender-side.
- **`--specials` recreates unix sockets** with `mknodat(..., S_IFSOCK)`, so
`-D`/`--devices --specials` now covers the full rsync node set.
- **`--copy-devices`** is implemented (see its caveat below).
### Output
- **`-i`/`--out-format`** print rsync-style change lines; **`--list-only`**
scans the source only and contacts no server; **`-h`** uses rsync's decimal
units; **`--progress`** is an aggregate line; **`--stats`** prints the counters
FastSync can observe locally (receiver-only counters are 0).
- **Server `--port`** is an alias of the `-p <port>` TCP listen port
(`--dparam port=` overrides the daemon config).
### Known intentional divergences and limitations
These remain after the wave; they are the reasons a row above is ⚠️.
- **Symlink target containment is not enforced receiver-side by default.**
Verbatim storage is rsync parity, but a destination later consumed by a
link-following tool can follow a link outside the receive root. Use
`--safe-links` when the source is untrusted. `--trust-sender` does **not**
affect symlink targets.
- **`--temp-dir` absolute/foreign-filesystem paths are rejected by the
receiver** (rsync's daemon also confines; standalone rsync differs).
- **`--copy-devices` reads a bounded `st_size`** rather than rsync's unbounded
device read.
- **A broken symlink referent under `--copy-links`/`--copy-unsafe-links` exits 0**
where rsync exits 23.
- **New directories without `-p` still use FastSync's `0755` creation default**
rather than `source & ~umask`; directory metadata is only applied when a
directory attribute is requested.
- **`--stats` receiver-only counters** (matched data, file-list bytes, deleted
count) are reported as 0; `--progress` is an aggregate line, not per-file.
- **`--password-file`/`--early-input`/`--hash-credentials`/`--iterations` are
FastSync-native** (SCRAM/PBKDF2), not rsync semantics; the batch format is not
rsync-interoperable.
- **xattr/ACL namespace policy** permits only `user.*` and
`system.posix_acl_*` when `-A` is negotiated (stricter than rsync).
- **`--stop-at` remains a FastSync-flexible parser** (client-only, not
serialized); `--stop-after` matches rsync.
- **Push-only model and a non-rsync wire protocol** remain by design;
`--protocol` accepts only the current version and `-s`/`--secluded-args` is an
accepted no-op.
## Packed Metadata Frame (protocol 2.20.0)
+1 -1
View File
@@ -54,6 +54,6 @@ pkgs.mkShell {
echo "FastSync dev shell ready."
echo " Build: cmake -B build -S . && cmake --build build -j\$(nproc)"
echo " Unit: ./build/tests"
echo " CI parity: docker run --rm --user \"\$(id -u):\$(id -g)\" -v \"\$PWD:/workspace\" -w /workspace gitea.tap-tap.win/taptap/fastsync-ci:v10 ..."
echo " CI parity: docker run --rm --user \"\$(id -u):\$(id -g)\" -v \"\$PWD:/workspace\" -w /workspace gitea.tap-tap.win/taptap/fastsync-ci:v11 ..."
'';
}
+348 -155
View File
@@ -7,21 +7,7 @@
#include <string.h>
#include <sys/stat.h>
#include <time.h>
/* Itemize code emitted for a transferred regular file.
*
* Layout (rsync-compatible 11-char item): `>f` marks a regular file that was
* transferred to the remote host; the trailing nine markers are, in order,
* c(hecksum) s(ize) t(ime) p(erms) o(wner) g(roup) u(ser/acl) a(ttrs) x(attrs).
* Every marker is `+` (FastSync does not compare each attribute on the
* receiving side, so a sent file is reported as fully updated). Files that
* are already up to date print no line at all, matching rsync's single -i
* which only itemizes changes.
*
* Because the scanner only yields regular-file transfer candidates, `>d`
* (directory) lines are never produced; directories are not transferred as
* items by FastSync. */
#define ITEMIZE_SENT_FILE ">f+++++++++"
#include <unistd.h>
typedef struct {
char* data;
@@ -80,103 +66,14 @@ static bool strbuf_append(StrBuf* buf, const char* text) {
return true;
}
static bool strbuf_append_ull(StrBuf* buf, unsigned long long value) {
char digits[32];
int written = snprintf(digits, sizeof(digits), "%llu", value);
if (written < 0 || (size_t)written >= sizeof(digits))
return false;
return strbuf_append(buf, digits);
}
static bool strbuf_append_longlong(StrBuf* buf, long long value) {
char digits[32];
int written = snprintf(digits, sizeof(digits), "%lld", value);
if (written < 0 || (size_t)written >= sizeof(digits))
return false;
return strbuf_append(buf, digits);
}
bool change_list_enabled(const Config* config) {
return config != NULL && (config->itemize_changes || config->out_format != NULL ||
(config->log_file != NULL && config->log_file_format != NULL));
}
char* change_render_itemize(const ChangeEvent* event) {
if (event == NULL || event->decision != CHANGE_SENT)
return str_dup("");
const char* code = event->is_directory ? ">d+++++++++" : ITEMIZE_SENT_FILE;
StrBuf line = {0};
bool ok = strbuf_append(&line, code) && strbuf_append(&line, " ") &&
strbuf_append(&line, event->path != NULL ? event->path : "");
if (!ok) {
strbuf_free(&line);
return NULL;
}
return line.data;
}
/* ---- Itemize code ---- */
static const char* leaf_name(const char* path) {
if (path == NULL)
return "";
const char* slash = strrchr(path, '/');
return slash != NULL && slash[1] != '\0' ? slash + 1 : path;
}
char* change_render_format(const char* format, const ChangeEvent* event) {
if (format == NULL)
return NULL;
StrBuf line = {0};
bool ok = true;
for (const char* p = format; *p != '\0' && ok;) {
if (*p != '%') {
ok = strbuf_append_char(&line, *p);
p++;
continue;
}
char token = p[1];
if (token == '\0') {
ok = strbuf_append_char(&line, '%');
break;
}
switch (token) {
case '%':
ok = strbuf_append_char(&line, '%');
break;
case 'f':
ok = strbuf_append(&line, event->path != NULL ? event->path : "");
break;
case 'n':
ok = strbuf_append(&line, leaf_name(event->path));
break;
case 'l':
ok = strbuf_append_ull(&line, event->size);
break;
case 'b':
ok = strbuf_append_ull(&line, event->bytes_sent);
break;
case 'M':
ok = strbuf_append_longlong(&line, (long long)event->mtime_sec);
break;
default:
/* Unknown escape sequences are preserved verbatim. */
ok = strbuf_append_char(&line, '%') && strbuf_append_char(&line, token);
break;
}
p += 2;
}
if (!ok) {
strbuf_free(&line);
return NULL;
}
if (line.data == NULL) {
line.data = str_dup("");
if (!line.data)
return NULL;
}
return line.data;
}
/* Format a mode as an `ls -l` permission string, e.g. `-rw-r--r--`. */
/* Format the permission bits as an `ls -l` string, e.g. `-rw-r--r--`. */
static void mode_to_ls_string(mode_t mode, char out[11]) {
out[0] = S_ISDIR(mode) ? 'd'
: S_ISLNK(mode) ? 'l'
@@ -198,29 +95,102 @@ static void mode_to_ls_string(mode_t mode, char out[11]) {
out[10] = '\0';
}
char* change_render_list_line(mode_t mode, unsigned long long size, time_t mtime,
const char* path) {
char permission[11];
mode_to_ls_string(mode, permission);
char date[32];
struct tm broken_down;
if (localtime_r(&mtime, &broken_down) != NULL) {
if (strftime(date, sizeof(date), "%Y/%m/%d %H:%M:%S", &broken_down) == 0)
snprintf(date, sizeof(date), "?");
} else {
snprintf(date, sizeof(date), "?");
static char itemize_type_char(const ChangeEvent* event) {
if (event->is_directory)
return 'd';
if (event->is_symlink)
return 'L';
if (event->is_special) {
if (S_ISCHR(event->mode) || S_ISBLK(event->mode))
return 'D';
return 'S';
}
return 'f';
}
static bool times_match(const Config* config, const ChangeEvent* event) {
if (!event->dest.known || !event->dest.existed)
return false;
if (event->mtime_sec == event->dest.mtime_sec)
return event->mtime_nsec == event->dest.mtime_nsec;
long long delta = (long long)event->mtime_sec - (long long)event->dest.mtime_sec;
if (delta < 0)
delta = -delta;
return delta <= (long long)config->modify_window;
}
/* Fill the 11-character itemize code (10 chars + NUL). `created` means the
* destination entry did not exist, so every attribute marker is `+`. */
static void itemize_code(const Config* config, const ChangeEvent* event, char code[12]) {
bool known = event->dest.known;
bool created = !known || !event->dest.existed;
char update;
if (event->is_hardlink)
update = 'h';
else if (created)
update = (event->is_directory || event->is_symlink || event->is_special) ? 'c' : '>';
else
update = '>';
code[0] = update;
code[1] = itemize_type_char(event);
if (created) {
for (int i = 0; i < 9; i++)
code[2 + i] = '+';
code[11] = '\0';
return;
}
bool size_diff = event->size != event->dest.size;
bool time_diff = !times_match(config, event);
bool perms_diff = (event->mode & 07777) != (event->dest.mode & 07777);
bool owner_diff = event->uid != (uid_t)event->dest.uid;
bool group_diff = event->gid != (gid_t)event->dest.gid;
code[2] = '.'; /* checksum: no destination digest available */
code[3] = size_diff ? 's' : '.';
code[4] = time_diff ? 't' : '.';
code[5] = (config->preserve_perms && perms_diff) ? 'p' : '.';
code[6] = (config->preserve_owner && owner_diff) ? 'o' : '.';
code[7] = (config->preserve_group && group_diff) ? 'g' : '.';
code[8] = '.'; /* reserved */
code[9] = '.'; /* acl: not compared */
code[10] = '.';
code[11] = '\0';
}
char* change_render_itemize_code(const Config* config, const ChangeEvent* event) {
if (event == NULL || event->decision != CHANGE_SENT)
return str_dup("");
char code[12];
itemize_code(config, event, code);
return str_dup(code);
}
/* rsync %n: the transfer-relative name, with a trailing slash for directories. */
static bool append_name(StrBuf* buf, const ChangeEvent* event) {
if (!strbuf_append(buf, event->name != NULL ? event->name : ""))
return false;
if (event->is_directory && (event->name == NULL || event->name[0] == '\0' ||
event->name[strlen(event->name) - 1] != '/'))
return strbuf_append_char(buf, '/');
return true;
}
/* rsync %L: " -> target" for a symlink, " => target" for a hard link, else "". */
static bool append_link_suffix(StrBuf* buf, const ChangeEvent* event) {
if (event->is_symlink && event->symlink_target != NULL)
return strbuf_append(buf, " -> ") && strbuf_append(buf, event->symlink_target);
if (event->is_hardlink && event->hardlink_target != NULL)
return strbuf_append(buf, " => ") && strbuf_append(buf, event->hardlink_target);
return true;
}
char* change_render_itemize(const Config* config, const ChangeEvent* event) {
if (event == NULL || event->decision != CHANGE_SENT)
return str_dup("");
char code[12];
itemize_code(config, event, code);
StrBuf line = {0};
char size_field[32];
int written = snprintf(size_field, sizeof(size_field), "%llu", size);
if (written < 0 || (size_t)written >= sizeof(size_field)) {
strbuf_free(&line);
return NULL;
}
bool ok = strbuf_append(&line, permission) && strbuf_append_char(&line, ' ') &&
strbuf_append(&line, size_field) && strbuf_append_char(&line, ' ') &&
strbuf_append(&line, date) && strbuf_append_char(&line, ' ') &&
strbuf_append(&line, path != NULL ? path : "");
bool ok = strbuf_append(&line, code) && strbuf_append_char(&line, ' ') &&
append_name(&line, event) && append_link_suffix(&line, event);
if (!ok) {
strbuf_free(&line);
return NULL;
@@ -228,6 +198,141 @@ char* change_render_list_line(mode_t mode, unsigned long long size, time_t mtime
return line.data;
}
/* ---- --out-format / --log-file-format ---- */
char* change_render_format(const char* format, const Config* config, const ChangeEvent* event) {
if (format == NULL || event == NULL)
return NULL;
StrBuf line = {0};
bool ok = true;
for (const char* p = format; *p != '\0' && ok;) {
if (*p != '%') {
ok = strbuf_append_char(&line, *p);
p++;
continue;
}
char token = p[1];
if (token == '\0') {
ok = strbuf_append_char(&line, '%');
break;
}
switch (token) {
case '%':
ok = strbuf_append_char(&line, '%');
break;
case 'i': {
char code[12];
itemize_code(config, event, code);
ok = strbuf_append(&line, code);
break;
}
case 'f':
ok = strbuf_append(&line, event->path != NULL ? event->path : "");
break;
case 'n':
ok = append_name(&line, event);
break;
case 'L':
ok = append_link_suffix(&line, event);
break;
case 'l': {
char digits[32];
int written = snprintf(digits, sizeof(digits), "%llu", event->size);
ok = written >= 0 && (size_t)written < sizeof(digits) && strbuf_append(&line, digits);
} break;
case 'b': {
char digits[32];
int written = snprintf(digits, sizeof(digits), "%llu", event->bytes_sent);
ok = written >= 0 && (size_t)written < sizeof(digits) && strbuf_append(&line, digits);
} break;
case 'M': {
char when[32];
if (format_rsync_datetime(event->mtime_sec, true, when, sizeof(when)))
ok = strbuf_append(&line, when);
} break;
case 't': {
char when[32];
if (format_rsync_datetime(time(NULL), false, when, sizeof(when)))
ok = strbuf_append(&line, when);
} break;
case 'o':
ok = strbuf_append(&line, "send");
break;
case 'p': {
char digits[32];
int written = snprintf(digits, sizeof(digits), "%ld", (long)getpid());
ok = written >= 0 && (size_t)written < sizeof(digits) && strbuf_append(&line, digits);
} break;
case 'B': {
char permission[11];
mode_to_ls_string(event->mode, permission);
ok = strbuf_append(&line, permission + 1);
} break;
case 'U': {
char digits[32];
int written = snprintf(digits, sizeof(digits), "%u", (unsigned)event->uid);
ok = written >= 0 && (size_t)written < sizeof(digits) && strbuf_append(&line, digits);
} break;
case 'G': {
char digits[32];
int written = snprintf(digits, sizeof(digits), "%u", (unsigned)event->gid);
ok = written >= 0 && (size_t)written < sizeof(digits) && strbuf_append(&line, digits);
} break;
default:
/* Unknown escape sequences are preserved verbatim. */
ok = strbuf_append_char(&line, '%') && strbuf_append_char(&line, token);
break;
}
p += 2;
}
if (!ok) {
strbuf_free(&line);
return NULL;
}
if (line.data == NULL) {
line.data = str_dup("");
if (!line.data)
return NULL;
}
return line.data;
}
/* ---- --list-only ---- */
char* change_render_list_line(const Config* config, const ChangeEvent* event) {
(void)config;
if (event == NULL)
return NULL;
char permission[11];
mode_to_ls_string(event->mode, permission);
char date[32];
if (!format_rsync_datetime(event->mtime_sec, false, date, sizeof(date)))
snprintf(date, sizeof(date), "?");
StrBuf line = {0};
char size_field[40];
char grouped[32];
if (!format_big_num(event->size, false, grouped, sizeof(grouped))) {
strbuf_free(&line);
return NULL;
}
int written = snprintf(size_field, sizeof(size_field), "%15s", grouped);
if (written < 0 || (size_t)written >= sizeof(size_field)) {
strbuf_free(&line);
return NULL;
}
const char* name = event->name != NULL && event->name[0] != '\0' ? event->name : ".";
bool ok = strbuf_append(&line, permission) && strbuf_append(&line, size_field) &&
strbuf_append_char(&line, ' ') && strbuf_append(&line, date) &&
strbuf_append_char(&line, ' ') && strbuf_append(&line, name);
if (!ok) {
strbuf_free(&line);
return NULL;
}
return line.data;
}
/* ---- Event emission ---- */
static void print_escaped_line(FILE* stream, const char* line, bool eight_bit_output) {
char* escaped = output_escape(line, eight_bit_output);
if (escaped != NULL) {
@@ -247,15 +352,16 @@ void change_emit(const Config* config, const ChangeEvent* event) {
bool to_stdout = config->itemize_changes || config->out_format != NULL;
bool to_log = config->log_file != NULL && config->log_file_format != NULL;
if (to_stdout) {
char* line = config->out_format != NULL ? change_render_format(config->out_format, event)
: change_render_itemize(event);
char* line = config->out_format != NULL
? change_render_format(config->out_format, config, event)
: change_render_itemize(config, event);
if (line != NULL) {
print_escaped_line(stdout, line, config->eight_bit_output);
free(line);
}
}
if (to_log) {
char* line = change_render_format(config->log_file_format, event);
char* line = change_render_format(config->log_file_format, config, event);
if (line != NULL) {
print_escaped_line(config->log_file, line, config->eight_bit_output);
free(line);
@@ -266,9 +372,6 @@ void change_emit(const Config* config, const ChangeEvent* event) {
static bool format_uses_mtime(const char* format) {
if (format == NULL)
return false;
/* Mirror change_render_format's tokenizer: "%%" is a literal percent (so
* "%%M" does NOT expand %M) and unknown "%X" escapes consume both chars.
* This keeps the optional stat() fallback below in step with the renderer. */
for (const char* p = format; *p != '\0';) {
if (*p != '%') {
p++;
@@ -284,46 +387,136 @@ static bool format_uses_mtime(const char* format) {
return false;
}
/* Relative path of an entry below the transfer root (no leading slash). Uses
* the sender-side send_path override when present (bare-relative -R layout). */
static char* relative_name(const Config* config, const File* file) {
const char* full = file_wire_path(file);
if (file->send_path != NULL)
return str_dup(full != NULL ? full : "");
const char* root = config->send_directory;
if (root == NULL || full == NULL)
return str_dup(full != NULL ? full : "");
size_t root_len = strlen(root);
while (root_len > 1 && root[root_len - 1] == '/')
root_len--;
if (strncmp(root, full, root_len) == 0) {
if (full[root_len] == '\0')
return str_dup("");
if (full[root_len] == '/')
return str_dup(full + root_len + 1);
}
return str_dup(full);
}
/* rsync %f long form: the source argument as typed (leading '/' removed,
* trailing '/' removed, leading "./" removed) joined to the relative name. */
static char* display_name(const Config* config, const char* name) {
const char* root = config->send_directory;
if (root == NULL)
return str_dup(name != NULL ? name : "");
const char* p = root;
while (*p == '/')
p++;
if (p[0] == '.' && p[1] == '/')
p += 2;
size_t root_len = strlen(p);
while (root_len > 0 && p[root_len - 1] == '/')
root_len--;
size_t name_len = name != NULL ? strlen(name) : 0;
if (root_len == 0 && name_len == 0)
return str_dup("");
char* out = malloc(root_len + (root_len > 0 && name_len > 0 ? 1 : 0) + name_len + 1);
if (!out)
return NULL;
size_t offset = 0;
if (root_len > 0) {
memcpy(out, p, root_len);
offset = root_len;
}
if (root_len > 0 && name_len > 0)
out[offset++] = '/';
if (name_len > 0)
memcpy(out + offset, name, name_len);
out[offset + name_len] = '\0';
return out;
}
static void fill_event_from_file(const Config* config, const File* file, ChangeEvent* event,
char** name_out, char** path_out) {
char* name = relative_name(config, file);
char* path = display_name(config, name);
event->name = name;
event->path = path;
*name_out = name;
*path_out = path;
if (file->metadata != NULL) {
event->mtime_sec = file->metadata->mtime_sec;
event->mtime_nsec = file->metadata->mtime_nsec;
event->mode = file->metadata->mode;
event->uid = file->metadata->uid;
event->gid = file->metadata->gid;
} else if (format_uses_mtime(config->out_format) || format_uses_mtime(config->log_file_format)) {
struct stat st;
if (file->path != NULL && stat(file->path, &st) == 0) {
event->mtime_sec = st.st_mtime;
event->mtime_nsec = st.st_mtim.tv_nsec;
}
}
}
void change_emit_file_sent(const Config* config, const File* file) {
if (file == NULL || !change_list_enabled(config))
return;
ChangeEvent event;
memset(&event, 0, sizeof(event));
/* The displayed path is the one transmitted (with -R + --files-from this is
the bare relative destination path); the metadata fallback below still
stats the local absolute path. */
event.path = file_wire_path(file);
event.decision = CHANGE_SENT;
event.is_directory = false;
event.is_symlink = false;
event.is_special = false;
event.is_hardlink = false;
event.size = file->data != NULL ? file->data->size : 0;
/* FastSync has no wire-byte counter yet, so %b reports the source length
* that had to be delivered (always equal to %l); the actual bytes written
* to the socket (compressed/delta) are not measured. */
event.bytes_sent = event.size;
if (file->metadata != NULL) {
event.mtime_sec = file->metadata->mtime_sec;
} else if (format_uses_mtime(config->out_format) || format_uses_mtime(config->log_file_format)) {
/* Best-effort fallback for %M when no metadata was captured (no -M): the
* path is stat()ed just to fill the field, and any failure leaves 0. */
struct stat st;
if (file->path != NULL && stat(file->path, &st) == 0)
event.mtime_sec = st.st_mtime;
event.dest = file->dest_state;
if (file->is_symlink) {
event.is_symlink = true;
event.symlink_target = file->symlink_target;
event.size = file->symlink_target != NULL ? strlen(file->symlink_target) : 0;
event.bytes_sent = 0;
} else if (file->is_special) {
event.is_special = true;
event.bytes_sent = 0;
} else if (file->link_group != 0 && !file->link_first) {
event.is_hardlink = true;
event.hardlink_target = file->hardlink_target;
event.bytes_sent = 0;
} else {
/* Literal payload bytes delivered; compressed/delta wire bytes are not
* separately counted. */
event.bytes_sent = event.size;
}
change_emit(config, &event);
char* name = NULL;
char* path = NULL;
fill_event_from_file(config, file, &event, &name, &path);
if (name != NULL && path != NULL)
change_emit(config, &event);
free(name);
free(path);
}
/* Build and emit a CHANGE_SENT event for an explicit directory entry (-d). */
void change_emit_dir_sent(const Config* config, const File* file) {
if (file == NULL || !change_list_enabled(config))
return;
ChangeEvent event;
memset(&event, 0, sizeof(event));
event.path = file_wire_path(file);
event.decision = CHANGE_SENT;
event.is_directory = true;
event.size = 0;
event.bytes_sent = 0;
if (file->metadata != NULL)
event.mtime_sec = file->metadata->mtime_sec;
change_emit(config, &event);
event.dest = file->dest_state;
char* name = NULL;
char* path = NULL;
fill_event_from_file(config, file, &event, &name, &path);
if (name != NULL && path != NULL)
change_emit(config, &event);
free(name);
free(path);
}
+35 -24
View File
@@ -3,6 +3,7 @@
#include "config.h"
#include "file_types.h"
#include "format.h"
#include <stdbool.h>
#include <sys/stat.h>
#include <time.h>
@@ -26,42 +27,52 @@ typedef enum {
} ChangeDecision;
typedef struct {
const char* path; /* full source path */
const char* path; /* long-form display path (rsync %f) */
const char* name; /* transfer-relative path (rsync %n), no trailing slash */
ChangeDecision decision;
bool is_directory;
unsigned long long size; /* source file length in bytes */
/* The number of bytes reported for a sent file. FastSync has no wire-byte
* counter, so this is always the source length (== size / %l); actual
* post-compression/delta bytes on the wire are not counted. */
unsigned long long bytes_sent;
time_t mtime_sec; /* 0 when unknown */
bool is_symlink;
bool is_special;
bool is_hardlink; /* a hard-link sibling (linked, no data sent) */
const char* symlink_target;
const char* hardlink_target;
unsigned long long size; /* source file length in bytes */
unsigned long long bytes_sent; /* literal data bytes actually transferred */
time_t mtime_sec;
long mtime_nsec;
mode_t mode;
uid_t uid;
gid_t gid;
/* Receiver-reported pre-transfer destination state (OutputDestState.known is
* false when no report was requested/received). */
OutputDestState dest;
} ChangeEvent;
/* True when any output mode is active and per-file events matter. */
bool change_list_enabled(const Config* config);
/* Render the rsync-style itemize line for a transferred file:
* `>f+++++++++ <path>`
* The 11-char code is `>f` (regular file transferred to the remote host)
* followed by c/s/t/p/o/g/u/a/x markers that are all `+` (value will be set
* / differs) because FastSync does not separately compare checksums, size,
* mtime, perms, owner, group, uid, acl, or xattr on the receiving side, so a
* sent file is reported as fully updated. Up-to-date files print no line
* (rsync single `-i` only shows changes). Caller frees the result. */
char* change_render_itemize(const ChangeEvent* event);
/* Render the rsync-style itemize line for a transferred item
* (`%i %n%L`): `>f+++++++++ sub/b.txt`. Caller frees the result. */
char* change_render_itemize(const Config* config, const ChangeEvent* event);
/* Expand an --out-format/--log-file-format template. Tokens:
* %f full source path %b "bytes sent" == the source length (%l);
* %n leaf (base) name actual post-compression/delta wire bytes
* %l file length in bytes are not counted
* %M mtime in whole seconds %% a literal percent sign
/* Render only the 11-character itemize code (rsync %i). Caller frees. */
char* change_render_itemize_code(const Config* config, const ChangeEvent* event);
/* Expand an --out-format/--log-file-format template. Supported tokens:
* %i itemize code %n transfer-relative name (dir: trailing /)
* %f long display path %l file length in bytes
* %b bytes actually sent %M mtime (YYYY/MM/DD-HH:MM:SS)
* %t current time %o operation ("send"/"del.")
* %p pid %B permission bits without the type char
* %U uid %G gid
* %L " -> target" / " => target" %% a literal percent sign
* Unknown %X sequences are preserved verbatim. Caller frees the result. */
char* change_render_format(const char* format, const ChangeEvent* event);
char* change_render_format(const char* format, const Config* config, const ChangeEvent* event);
/* Render one --list-only long-listing entry:
* `-rw-r--r-- 12 2026/09/06 10:00:00 <path>`
* `-rw-r--r-- 12 2026/09/06 10:00:00 sub/b.txt`
* (ls -l style columns; mtime in the local time zone). Caller frees it. */
char* change_render_list_line(mode_t mode, unsigned long long size, time_t mtime, const char* path);
char* change_render_list_line(const Config* config, const ChangeEvent* event);
/* Emit an event to every active destination:
* stdout: --itemize-changes line, or the --out-format expansion when set;
+643 -175
View File
File diff suppressed because it is too large. Load diff
+331 -88
View File
@@ -11,6 +11,7 @@
#include "file.h"
#include "file_list.h"
#include "filter.h"
#include "format.h"
#include "hardlink.h"
#include "metadata.h"
#include "motd.h"
@@ -69,32 +70,61 @@ static int progress_thread_fn(void* arg);
static const char* display_bytes(unsigned long long bytes, bool human_readable, char* buffer,
size_t buffer_size) {
if (human_readable && format_human_bytes(bytes, buffer, buffer_size))
if (human_readable && format_human_size_decimal(bytes, buffer, buffer_size))
return buffer;
snprintf(buffer, buffer_size, "%.1f MB", (double)bytes / (double)BYTES_PER_MIB);
return buffer;
}
/* Print the canonical `--stats` line. Shared by the single-threaded and
multithreaded send paths so both honor --stats, --human-readable and --quiet
identically; `start` marks the beginning of the transfer for the rate. */
/* rsync byte count: human-readable decimal when -h was given, otherwise a
* comma-grouped integer (rsync's big_num in the C locale). */
static const char* stats_bytes(const Config* config, unsigned long long bytes, char* buffer,
size_t buffer_size) {
if (!format_big_num(bytes, config->human_readable, buffer, buffer_size))
snprintf(buffer, buffer_size, "%llu", bytes);
return buffer;
}
/* Print the rsync `--stats` block on stdout. FastSync is a push sender, so a
few receiver-only counters (matched data, file-list bytes, deletion count)
are not observable and are reported as 0; the labels and layout match rsync
3.4.1. Shared by the single-threaded and multithreaded send paths. */
static void report_transfer_stats(const Config* config, int total_files,
unsigned long long total_bytes, time_t start) {
if (!config->stats || config->quiet)
return;
double elapsed = difftime(time(NULL), start);
double rate = elapsed > 0.0 ? (double)total_bytes / ((double)BYTES_PER_MIB * elapsed) : 0.0;
double rate = elapsed > 0.0 ? (double)total_bytes / elapsed : 0.0;
char total_buffer[32];
char rate_buffer[32] = {0};
char human_rate[32] = {0};
const char* total = stats_bytes(config, total_bytes, total_buffer, sizeof(total_buffer));
const char* rate_str = rate_buffer;
if (config->human_readable) {
char total_buffer[32];
char rate_buffer[32];
fprintf(stderr, "Stats: %d files, %s, %s/s\n", total_files,
display_bytes(total_bytes, true, total_buffer, sizeof(total_buffer)),
display_bytes((unsigned long long)(rate * (double)BYTES_PER_MIB), true, rate_buffer,
sizeof(rate_buffer)));
if (!format_human_size_decimal((unsigned long long)rate, human_rate, sizeof(human_rate)))
snprintf(human_rate, sizeof(human_rate), "0");
rate_str = human_rate;
} else {
fprintf(stderr, "Stats: %d files, %.1f MB, %.1f MB/s\n", total_files,
(double)total_bytes / (double)BYTES_PER_MIB, rate);
snprintf(rate_buffer, sizeof(rate_buffer), "%.2f", rate);
}
printf("\n");
printf("Number of files: %d\n", total_files);
printf("Number of created files: %d\n", total_files);
printf("Number of deleted files: 0\n");
printf("Number of regular files transferred: %d\n", total_files);
printf("Total file size: %s bytes\n", total);
printf("Total transferred file size: %s bytes\n", total);
printf("Literal data: %s bytes\n", total);
printf("Matched data: 0 bytes\n");
printf("File list size: 0\n");
printf("File list generation time: 0.000 seconds\n");
printf("File list transfer time: 0.000 seconds\n");
printf("Total bytes sent: %s\n", total);
printf("Total bytes received: 0\n");
printf("\n");
printf("sent %s bytes received 0 bytes %s bytes/sec\n", total, rate_str);
printf("total size is %s speedup is %.2f\n", total, 1.0);
fflush(stdout);
}
/* Compiled scanner inputs that are shared read-only across scanner instances
@@ -146,10 +176,16 @@ static bool prepare_scanner(const Config* config, int num_threads, PreparedScann
options->preserve_xattrs = config->preserve_xattrs;
options->preserve_acls = config->preserve_acls;
options->chunk_size = config->chunk_size;
options->exclude_patterns = config->exclude_patterns;
options->exclude_count = config->exclude_count;
options->include_patterns = config->include_patterns;
options->include_count = config->include_count;
/* --exclude/--include are compiled, in command-line order, into the SAME
* ordered filter rule list as --filter/-f (see config_add_selection_rule), so
* the legacy per-kind arrays are deliberately NOT passed to the scanner:
* doing so would re-apply them with the old "excludes first, then includes as
* a mandatory whitelist" precedence and defeat rsync's first-match-wins
* ordering. The arrays remain populated purely for the Config API surface. */
options->exclude_patterns = NULL;
options->exclude_count = 0;
options->include_patterns = NULL;
options->include_count = 0;
options->max_size = config->max_size;
options->min_size = config->min_size;
options->max_depth = config->max_depth;
@@ -175,10 +211,13 @@ static bool prepare_scanner(const Config* config, int num_threads, PreparedScann
options->ignore_missing_args = config->ignore_missing_args || config->delete_missing_args;
options->excluded_paths = NULL;
options->excluded_mutex = NULL;
options->size_skipped_paths = NULL;
options->synced_dirs = NULL;
options->hardlinks = NULL;
/* P7 Wave D: capture source directory times whenever metadata rides the
wire. Whether they are APPLIED is decided receiver-side (-O skips). */
options->capture_dir_times = config->use_metadata;
/* P7 Wave D: capture source directory metadata when a directory attribute is
requested (-p for modes, -t for times unless -O omits them). Whether they
are APPLIED is decided receiver-side. */
options->capture_dir_times = dir_metadata_should_capture(config);
options->dir_entries = NULL;
options->dir_entries_mutex = NULL;
if (config->preserve_hard_links) {
@@ -466,7 +505,7 @@ static void receive_daemon_motd(Client* client, const Config* config) {
static Client* connect_transfer_client(const Config* config) {
if (config->transport == TRANSPORT_SSH) {
if (config->use_sendfile) {
log_message(LOG_LEVEL_ERROR, "-f/--sendfile is not supported with SSH transport");
log_message(LOG_LEVEL_ERROR, "--sendfile is not supported with SSH transport");
return NULL;
}
return client_connect_ssh(config->ssh_destination, config->ssh_port,
@@ -653,7 +692,10 @@ static void mark_sender_done(PipelineContextSender* context) {
file it processed, in send order: STATUS_NEXT means the file was written,
STATUS_OK means the file was skipped/unchanged. Skipped sources are marked
so the later removal pass keeps them. */
static bool finalize_transfer(Client* client, const Config* config, ArrayList* remove_sources) {
static bool finalize_transfer(Client* client, const Config* config, ArrayList* remove_sources,
bool* delete_limit_out) {
if (delete_limit_out)
*delete_limit_out = false;
if (!send_status(client->file_descriptor, STATUS_FINISHED))
return false;
if (config->remove_source_files && remove_sources) {
@@ -676,6 +718,15 @@ static bool finalize_transfer(Client* client, const Config* config, ArrayList* r
Status status;
if (!receive_status(client->file_descriptor, &status))
return false;
/* A capped --max-delete commit is a successful transfer that the client must
report with rsync's exit code 25 (not an error). */
if (status == STATUS_DELETE_LIMIT) {
log_message(LOG_LEVEL_ERROR,
"Deletions stopped due to --max-delete limit; some deletions were skipped");
if (delete_limit_out)
*delete_limit_out = true;
return true;
}
if (status != STATUS_OK) {
log_server_rejection("Receiver reported transfer failure");
return false;
@@ -781,30 +832,53 @@ static int send_dry_run_manifest(const Config* config) {
}
typedef struct {
char* path;
char* name; /* transfer-relative name ("" == the source root) */
mode_t mode;
unsigned long long size;
time_t mtime;
long mtime_nsec;
bool is_dir;
bool is_symlink;
char* link_target;
} ListEntry;
static void list_entries_destroy(ListEntry* entries, size_t count) {
if (entries == NULL)
return;
for (size_t i = 0; i < count; i++)
free(entries[i].path);
for (size_t i = 0; i < count; i++) {
free(entries[i].name);
free(entries[i].link_target);
}
free(entries);
}
static int compare_list_entries(const void* left, const void* right) {
const ListEntry* a = (const ListEntry*)left;
const ListEntry* b = (const ListEntry*)right;
return strcmp(a->path, b->path);
return strcmp(a->name, b->name);
}
/* --list-only: print an ls-style listing of the files that WOULD be
/* Relative path of an entry below `root` ("" for the root itself). Mirrors
* change_list's relative_name for list-only rendering. */
static char* list_relative_name(const char* root, const char* full) {
if (root == NULL || full == NULL)
return str_dup(full != NULL ? full : "");
size_t root_len = strlen(root);
while (root_len > 1 && root[root_len - 1] == '/')
root_len--;
if (strncmp(root, full, root_len) == 0) {
if (full[root_len] == '\0')
return str_dup("");
if (full[root_len] == '/')
return str_dup(full + root_len + 1);
}
return str_dup(full);
}
/* --list-only: print an ls-style listing of the entries that WOULD be
* transferred and exit without contacting the server or writing anything.
* Directory lines are not printed because the scanner only yields regular
* transfer candidates. Returns 0 on success, 1 on error. */
* Names are transfer-relative (rsync prints `a.txt`, `sub/b.txt`, `.`) and
* directory entries are included. Returns 0 on success, 1 on error. */
static int send_list_only(const Config* config) {
int skipped = 0;
if (!files_from_list_check(config, NULL, &skipped))
@@ -813,6 +887,7 @@ static int send_list_only(const Config* config) {
if (!prepare_scanner(config, 0, &prepared))
return 1;
prepared.options.use_metadata = true; /* capture mode + mtime for the listing */
prepared.options.list_dirs = true;
DirectoryScanner* scanner =
directory_scanner_create_with_options(config->send_directory, &prepared.options);
if (!scanner) {
@@ -822,9 +897,33 @@ static int send_list_only(const Config* config) {
ListEntry* entries = NULL;
size_t count = 0;
size_t capacity = 0;
Chunk* chunk;
bool oom = false;
while ((chunk = directory_scanner_next(scanner)) != NULL) {
/* rsync lists the source root itself (as "."). Only when the source is a
* directory and no --files-from subset is in effect. */
if (config->files_from_set == NULL && config->send_directory != NULL) {
struct stat st;
if (stat(config->send_directory, &st) == 0 && S_ISDIR(st.st_mode)) {
capacity = 64;
entries = calloc(capacity, sizeof(ListEntry));
if (entries == NULL) {
oom = true;
} else if ((entries[0].name = str_dup("")) == NULL) {
/* A NULL name would be dereferenced by qsort/render: fail the listing. */
oom = true;
} else {
entries[0].mode = st.st_mode;
entries[0].mtime = st.st_mtime;
entries[0].mtime_nsec = st.st_mtim.tv_nsec;
entries[0].size = (unsigned long long)st.st_size;
entries[0].is_dir = true;
count = 1;
}
}
}
Chunk* chunk;
while (!oom && (chunk = directory_scanner_next(scanner)) != NULL) {
for (int i = 0; i < chunk->element_count; i++) {
File* f = chunk->items[i];
if (f == NULL)
@@ -841,34 +940,47 @@ static int send_list_only(const Config* config) {
break;
}
entries = grown;
memset(entries + capacity, 0, (new_capacity - capacity) * sizeof(ListEntry));
capacity = new_capacity;
}
char* path = str_dup(file_wire_path(f));
if (!path) {
char* name = list_relative_name(config->send_directory, file_wire_path(f));
if (!name) {
oom = true;
break;
}
mode_t mode = 0;
time_t mtime = 0;
long mtime_nsec = 0;
if (f->metadata != NULL) {
mode = f->metadata->mode;
mtime = f->metadata->mtime_sec;
mtime_nsec = f->metadata->mtime_nsec;
} else {
struct stat st;
if (stat(f->path, &st) == 0) {
if (lstat(f->path, &st) == 0) {
mode = st.st_mode;
mtime = st.st_mtime;
mtime_nsec = st.st_mtim.tv_nsec;
}
}
entries[count].path = path;
entries[count].name = name;
entries[count].mode = mode;
entries[count].mtime = mtime;
entries[count].size = f->data != NULL ? f->data->size : 0;
entries[count].mtime_nsec = mtime_nsec;
if (f->is_symlink)
entries[count].size = f->symlink_target != NULL ? strlen(f->symlink_target) : 0;
else if (f->is_dir) {
struct stat dir_st;
entries[count].size = stat(f->path, &dir_st) == 0 ? (unsigned long long)dir_st.st_size : 0;
} else
entries[count].size = f->data != NULL ? f->data->size : 0;
entries[count].is_dir = f->is_dir;
entries[count].is_symlink = f->is_symlink;
entries[count].link_target =
f->is_symlink && f->symlink_target ? str_dup(f->symlink_target) : NULL;
count++;
}
chunk_destroy(chunk);
if (oom)
break;
}
bool failed = oom || directory_scanner_failed(scanner) || directory_scanner_had_io_error(scanner);
directory_scanner_destroy(scanner);
@@ -882,8 +994,18 @@ static int send_list_only(const Config* config) {
if (count > 1)
qsort(entries, count, sizeof(ListEntry), compare_list_entries);
for (size_t i = 0; i < count; i++) {
char* line = change_render_list_line(entries[i].mode, entries[i].size, entries[i].mtime,
entries[i].path);
ChangeEvent event;
memset(&event, 0, sizeof(event));
event.name = entries[i].name;
event.path = entries[i].name;
event.mode = entries[i].mode;
event.size = entries[i].size;
event.mtime_sec = entries[i].mtime;
event.mtime_nsec = entries[i].mtime_nsec;
event.is_directory = entries[i].is_dir;
event.is_symlink = entries[i].is_symlink;
event.symlink_target = entries[i].link_target;
char* line = change_render_list_line(config, &event);
if (line != NULL) {
char* escaped = output_escape(line, config->eight_bit_output);
printf("%s\n", escaped != NULL ? escaped : line);
@@ -895,21 +1017,25 @@ static int send_list_only(const Config* config) {
return 0;
}
/* Send the delete manifest (keep-set paths plus the protected excluded
prefixes and the --delete-missing-args exact-delete paths) to the server.
Returns 0 on success, -1 on failure. When --delete-excluded is given
`protected` is empty: excluded destination mirrors are then ordinary extras
and are removed. When --delete-missing-args is active `missing_args` holds
the destination mirrors of missing --files-from entries: each is an explicit
receiver-side deletion request, independent of the extras walk. A NULL
keep-set / protected / missing list transmits an empty section. All three
sections are unbounded on the sender; the receiver enforces
/* Send the delete manifest to the server. Returns 0 on success, -1 on
failure. It carries FOUR sections: the keep-set paths, the protected
excluded prefixes, the --delete-missing-args exact-delete paths, and the
destination-relative directories the sender synchronized this run.
When --delete-excluded is given `protected` is empty: excluded destination
mirrors are then ordinary extras and are removed. When
--delete-missing-args is active `missing_args` holds the destination mirrors
of missing --files-from entries: each is an explicit receiver-side deletion
request, independent of the extras walk. `synced_dirs` confines the extras
walk to entries directly inside a synchronized directory. A NULL
keep-set / protected / missing / dirs list transmits an empty section. All
four sections are unbounded on the sender; the receiver enforces
MAX_MANIFEST_ENTRIES per section and a single MAX_MANIFEST_BYTES budget
shared across the sections, rejecting (with STATUS_ERROR) an over-budget
frame. A heavily filtered source whose exclusion list is large therefore
fails the run cleanly on the receiver rather than being truncated. */
static int send_delete_manifest(int fd, ArrayList* manifest, ArrayList* protected_prefixes,
ArrayList* missing_args) {
ArrayList* size_skipped, ArrayList* missing_args,
ArrayList* synced_dirs) {
if (!send_status(fd, STATUS_MANIFEST))
return -1;
int keep_count = manifest ? manifest->size : 0;
@@ -919,12 +1045,24 @@ static int send_delete_manifest(int fd, ArrayList* manifest, ArrayList* protecte
if (!send_wire_str(fd, (char*)manifest->items[i]))
return -1;
}
int protected_count = protected_prefixes ? protected_prefixes->size : 0;
/* The receiver has ONE protected-prefix section; filter-excluded prefixes
(dropped under --delete-excluded) and size-pruned prefixes (always
protected) are concatenated into it. */
int protected_count =
(protected_prefixes ? protected_prefixes->size : 0) + (size_skipped ? size_skipped->size : 0);
if (!send_int(fd, protected_count))
return -1;
for (int i = 0; i < protected_count; i++) {
if (!send_wire_str(fd, (char*)protected_prefixes->items[i]))
return -1;
if (protected_prefixes) {
for (int i = 0; i < protected_prefixes->size; i++) {
if (!send_wire_str(fd, (char*)protected_prefixes->items[i]))
return -1;
}
}
if (size_skipped) {
for (int i = 0; i < size_skipped->size; i++) {
if (!send_wire_str(fd, (char*)size_skipped->items[i]))
return -1;
}
}
int missing_count = missing_args ? missing_args->size : 0;
if (!send_int(fd, missing_count))
@@ -933,6 +1071,13 @@ static int send_delete_manifest(int fd, ArrayList* manifest, ArrayList* protecte
if (!send_wire_str(fd, (char*)missing_args->items[i]))
return -1;
}
int dirs_count = synced_dirs ? synced_dirs->size : 0;
if (!send_int(fd, dirs_count))
return -1;
for (int i = 0; i < dirs_count; i++) {
if (!send_wire_str(fd, (char*)synced_dirs->items[i]))
return -1;
}
return 0;
}
@@ -952,11 +1097,12 @@ static int send_delete_manifest(int fd, ArrayList* manifest, ArrayList* protecte
#define DELETE_ACK_KEEPALIVE_SEC 10
static bool send_delete_manifest_early(Client* client, ArrayList* manifest,
ArrayList* protected_prefixes, ArrayList* missing_args) {
ArrayList* protected_prefixes, ArrayList* size_skipped,
ArrayList* missing_args, ArrayList* synced_dirs) {
if (!client || !manifest)
return false;
if (send_delete_manifest(client->file_descriptor, manifest, protected_prefixes, missing_args) !=
0)
if (send_delete_manifest(client->file_descriptor, manifest, protected_prefixes, size_skipped,
missing_args, synced_dirs) != 0)
return false;
Status ack;
/* The wait is long (up to an hour) and runs inline on this thread: a helper
@@ -1051,6 +1197,19 @@ static int incremental_check(Client* client, File* file, const Config* config,
Status s;
if (!receive_status(client->file_descriptor, &s))
return -1;
/* Output parity: when dest-info reporting is negotiated the receiver sends
* the pre-transfer destination snapshot BEFORE its ordinary verdict. Consume
* it here so the following status read stays in sync. */
if (config->report_dest_info) {
if (s != STATUS_DEST_INFO ||
!format_dest_state_receive(client->file_descriptor, &file->dest_state)) {
log_message(LOG_LEVEL_ERROR, "Unexpected reply to the destination-state report");
send_status(client->file_descriptor, STATUS_ERROR);
return -1;
}
if (!receive_status(client->file_descriptor, &s))
return -1;
}
if (s == STATUS_ERROR) {
log_server_rejection("Server reported error for file");
return -1;
@@ -1428,7 +1587,11 @@ static bool send_directory_entry(const Client* client, File* file, const Config*
if (!send_status(client->file_descriptor, STATUS_MKDIR) ||
!send_wire_str(client->file_descriptor, file_wire_path(file)))
return false;
return !config->use_metadata || metadata_send(client->file_descriptor, file->metadata);
if (config->use_metadata && !metadata_send(client->file_descriptor, file->metadata))
return false;
/* Directory xattrs/ACLs (-X/-A) ride the same trailing block as regular files
when the xattr transport was negotiated. */
return !config->use_xattrs || xattr_send(client->file_descriptor, file->xattrs);
}
/* P7 Wave D: transmit every captured source directory's metadata in terminal
@@ -1444,7 +1607,8 @@ static bool send_directory_entry(const Client* client, File* file, const Config*
stays within the receiver's bound, and a frame that would exceed it is never
emitted. */
static bool send_dir_times(const Client* client, const Config* config, ArrayList* dir_entries) {
if (!client || !config || !config->use_metadata || !dir_entries || dir_entries->size == 0)
if (!client || !config || !dir_metadata_should_capture(config) || !dir_entries ||
dir_entries->size == 0)
return true;
int fd = client->file_descriptor;
int index = 0;
@@ -1459,6 +1623,9 @@ static bool send_dir_times(const Client* client, const Config* config, ArrayList
return false;
if (!send_wire_str(fd, file_wire_path(file)) || !metadata_send(fd, file->metadata))
return false;
/* Directory xattrs/ACLs travel with the deferred directory metadata. */
if (config->use_xattrs && !xattr_send(fd, file->xattrs))
return false;
}
index += chunk;
}
@@ -1759,7 +1926,8 @@ static int send_chunks_multithreaded(void* pipeline_context) {
/* The keep-set manifest was prebuilt by a path-only pre-scan. Transmit it
and wait for the receiver to delete extras before streaming any data. */
if (!send_delete_manifest_early(client, context->manifest, context->excluded_paths,
context->missing_args)) {
context->size_skipped_paths, context->missing_args,
context->synced_dirs)) {
pipeline_cancel(context);
disconnect_transfer_client(client);
mark_sender_done(context);
@@ -1866,13 +2034,15 @@ static int send_chunks_multithreaded(void* pipeline_context) {
goto send_fail;
}
if (send_delete_manifest(client->file_descriptor, context->manifest, context->excluded_paths,
context->missing_args) != 0)
context->size_skipped_paths, context->missing_args,
context->synced_dirs) != 0)
goto send_fail;
} else if (context->config->delete_missing_args && !context->early_delete) {
/* --delete-missing-args without --delete: no keep-set is built, but the
exact-delete paths still ride the same manifest frame (commit once the
transfer succeeded). */
if (send_delete_manifest(client->file_descriptor, NULL, NULL, context->missing_args) != 0)
if (send_delete_manifest(client->file_descriptor, NULL, NULL, NULL, context->missing_args,
NULL) != 0)
goto send_fail;
}
/* P7 Wave D: transmit the captured directory times last. The scanner thread
@@ -1882,11 +2052,12 @@ static int send_chunks_multithreaded(void* pipeline_context) {
if (!context->scan_stopped_early &&
!send_dir_times(client, context->config, context->dir_entries))
goto send_fail;
bool ok = finalize_transfer(client, context->config, context->remove_source_files);
bool delete_limit = false;
bool ok = finalize_transfer(client, context->config, context->remove_source_files, &delete_limit);
context->delete_limit = delete_limit;
if (!ok && context->config->use_delete)
log_message(LOG_LEVEL_ERROR,
"server reported a deletion failure (--delete); see the server log for the "
"reason (a --max-delete limit that the run would exceed deletes nothing)");
"server reported a deletion failure (--delete); see the server log for the reason");
if (ok)
remove_transferred_sources(context->config, context->remove_source_files);
mtx_lock(&context->mutex_progress);
@@ -1929,11 +2100,18 @@ static int scan_directory_multithreaded(void* pipeline_context) {
prepared.options.dir_entries = context->dir_entries;
prepared.options.dir_entries_mutex = &context->dir_entries_mutex;
/* The keep-set manifest for the late modes is built from this data pass, so
the parallel scanner records the protected excluded prefixes here. The
early modes already transmitted the pre-scan keep-set and its protected
list, so the data pass must not append to it again. */
if (!context->early_delete)
the parallel scanner records the protected excluded prefixes and the
synchronized directories here (the size-prune protection is collected in
every mode). The early modes already transmitted the pre-scan keep-set and
its protected lists, so the data pass must not append to them again. */
if (!context->early_delete) {
prepared.options.excluded_paths = context->excluded_paths;
/* The root marker for a full recursive transfer is already in the list; do
not let the scanner append every directory to it. */
if (context->config->files_from_set != NULL)
prepared.options.synced_dirs = context->synced_dirs;
}
prepared.options.size_skipped_paths = context->size_skipped_paths;
bool dirs_mode = prepared.options.dirs;
/* -H also selects the sequential scanner (see the comment at the branch),
* so the loop below must choose the scanner by which object exists, not by
@@ -2240,6 +2418,9 @@ int send_files(Config* config) {
ArrayList* dir_entries = NULL;
/* Protected excluded prefixes (delete-excluded default protection). */
ArrayList* excluded = NULL;
/* Size-pruned prefixes (always protected) and synchronized directories. */
ArrayList* size_skipped = NULL;
ArrayList* synced_dirs = NULL;
bool delete_early = config->use_delete && config_delete_timing_early(config);
bool send_failed = false;
bool had_scan_io = false;
@@ -2250,7 +2431,7 @@ int send_files(Config* config) {
receive_daemon_motd(client, config);
if (!prepare_scanner(config, 0, &prepared))
goto send_fail;
if (config->use_metadata) {
if (dir_metadata_should_capture(config)) {
dir_entries = array_list_create(file_destroy);
if (!dir_entries)
goto send_fail;
@@ -2263,11 +2444,31 @@ int send_files(Config* config) {
by user-selection rules so the receiver protects their destination mirrors
from --delete (rsync's default). Only scans that build the keep-set get the
sink attached (prescan for early timing, the streaming data pass otherwise). */
if (config->use_delete && !config->delete_excluded) {
excluded = array_list_create(free);
if (!excluded)
if (config->use_delete) {
if (!config->delete_excluded) {
excluded = array_list_create(free);
if (!excluded)
goto send_fail;
prepared.options.excluded_paths = excluded;
}
size_skipped = array_list_create(free);
synced_dirs = array_list_create(free);
if (!size_skipped || !synced_dirs)
goto send_fail;
prepared.options.excluded_paths = excluded;
prepared.options.size_skipped_paths = size_skipped;
/* Only a --files-from subset confines the extras walk to the directories
the scan synchronized; a full recursive transfer deletes throughout the
receive root, so mark the root itself (the "." sentinel) and let the
scanner record nothing extra. */
if (config->files_from_set == NULL) {
char* root_marker = str_dup(".");
if (!root_marker || !array_list_add(synced_dirs, root_marker)) {
free(root_marker);
goto send_fail;
}
} else {
prepared.options.synced_dirs = synced_dirs;
}
}
/* The late-timing modes (plain --delete / --delete-after / --delete-delay)
build the manifest while streaming and send it after the last data frame.
@@ -2294,13 +2495,16 @@ int send_files(Config* config) {
"with an empty keep-set (--delete)");
prescan_ok = false;
} else {
early_ok = send_delete_manifest_early(client, early_manifest, excluded, missing_args);
early_ok = send_delete_manifest_early(client, early_manifest, excluded, size_skipped,
missing_args, synced_dirs);
}
}
array_list_delete(early_manifest);
/* The keep-set (and its protected prefixes) are already on the wire; the
data pass must not append to the exclusion list again. */
/* The keep-set (and its protected prefixes and synchronized directories) are
already on the wire; the data pass must not append to those lists again. */
prepared.options.excluded_paths = NULL;
prepared.options.size_skipped_paths = NULL;
prepared.options.synced_dirs = NULL;
if (!prescan_ok || !early_ok)
goto send_fail;
} else if (config->use_delete) {
@@ -2444,7 +2648,8 @@ int send_files(Config* config) {
--delete-missing-args exact-path deletions only after the transfer
succeeds. In the early modes (--delete-before/--delete-during) the
manifest already went out up front, so nothing is re-sent here. */
if (send_delete_manifest(client->file_descriptor, manifest, excluded, missing_args) != 0) {
if (send_delete_manifest(client->file_descriptor, manifest, excluded, size_skipped,
missing_args, synced_dirs) != 0) {
if (manifest) {
array_list_delete(manifest);
manifest = NULL;
@@ -2462,11 +2667,11 @@ int send_files(Config* config) {
applying them until after its own deletion/publication phase. */
if (!send_dir_times(client, config, dir_entries))
goto send_fail;
bool ok = finalize_transfer(client, config, remove_sources);
bool delete_limit = false;
bool ok = finalize_transfer(client, config, remove_sources, &delete_limit);
if (!ok && config->use_delete)
log_message(LOG_LEVEL_ERROR,
"server reported a deletion failure (--delete); see the server log for the "
"reason (a --max-delete limit that the run would exceed deletes nothing)");
"server reported a deletion failure (--delete); see the server log for the reason");
if (ok)
remove_transferred_sources(config, remove_sources);
if (config->show_progress && !config->quiet)
@@ -2475,8 +2680,13 @@ int send_files(Config* config) {
log_info_message(LOG_INFO_STATS, "Transfer summary: %d files, %.1f MB", total_files,
(double)total_bytes / (double)BYTES_PER_MIB);
/* --ignore-errors: an unreadable source directory was skipped but the run
still completed (and deleted); report the run as errored like rsync does. */
ret = (ok && !had_scan_io) ? 0 : 1;
still completed (and deleted); report the run as errored like rsync does.
A --max-delete-capped commit is a successful transfer that rsync reports
with exit code 25. */
if (!ok || had_scan_io)
ret = 1;
else
ret = delete_limit ? 25 : 0;
send_fail:
/* Single cleanup path for all exits. The manifest is intentionally deleted
@@ -2485,6 +2695,10 @@ send_fail:
array_list_delete(manifest);
if (excluded)
array_list_delete(excluded);
if (size_skipped)
array_list_delete(size_skipped);
if (synced_dirs)
array_list_delete(synced_dirs);
if (missing_args)
array_list_delete(missing_args);
if (remove_sources)
@@ -2590,16 +2804,41 @@ int send_files_multithreaded(Config** config_ptr) {
return 1;
}
}
/* Size-pruned mirrors stay protected under every mode (even
--delete-excluded); synchronized directories confine the walk. A full
recursive transfer marks the receive root itself (".") so the walk is not
confined; only a --files-from subset records concrete directories. */
context->size_skipped_paths = array_list_create(free);
context->synced_dirs = array_list_create(free);
if (!context->size_skipped_paths || !context->synced_dirs) {
pipeline_context_sender_destroy(context);
return 1;
}
if (config->files_from_set == NULL) {
char* root_marker = str_dup(".");
if (!root_marker || !array_list_add(context->synced_dirs, root_marker)) {
free(root_marker);
pipeline_context_sender_destroy(context);
return 1;
}
}
if (config_delete_timing_early(config)) {
/* --delete-before/--delete-during: build the complete keep-set manifest
(paths only, nothing loaded or sent) up front so the sender thread can
transmit it before the first data byte. The path-only pre-scan also
fills the protected excluded prefixes. */
fills the protected excluded prefixes and synchronized directories. */
PreparedScanner prepared;
memset(&prepared, 0, sizeof(prepared));
bool prepared_ok = prepare_scanner(config, config->scanner_threads, &prepared);
if (prepared_ok && context->excluded_paths)
prepared.options.excluded_paths = context->excluded_paths;
if (prepared_ok) {
if (context->excluded_paths)
prepared.options.excluded_paths = context->excluded_paths;
prepared.options.size_skipped_paths = context->size_skipped_paths;
/* The root marker for a full recursive transfer is already in the list;
only a --files-from subset needs the scanner to record directories. */
if (config->files_from_set != NULL)
prepared.options.synced_dirs = context->synced_dirs;
}
bool prebuilt = prepared_ok && scan_paths_only(config, &prepared.options, context->manifest,
&context->scan_had_io_error);
prepared_scanner_destroy(&prepared);
@@ -2681,9 +2920,13 @@ int send_files_multithreaded(Config** config_ptr) {
scan_io = context->scan_had_io_error;
mtx_unlock(&context->mutex_scanner);
bool sender_ok = sender_result == thrd_success;
bool delete_limit = context->delete_limit;
/* --ignore-errors: the run completed (and deleted) past an unreadable source
directory; report it as errored like rsync does. */
directory; report it as errored like rsync does. A --max-delete-capped
commit is a successful transfer that rsync reports with exit code 25. */
pipeline_context_sender_destroy(context);
client_set_abort_armed(false);
return sender_ok && !scan_io ? 0 : 1;
if (!sender_ok || scan_io)
return 1;
return delete_limit ? 25 : 0;
}
+10
View File
@@ -60,6 +60,16 @@ bool validate_config(const Config* config) {
log_message(LOG_LEVEL_ERROR, "-f/--sendfile is not supported with SSH transport");
return false;
}
/* -M/--remote-option appends an option to the REMOTE server's argv, which
* only exists on the SSH (user@host:path) transport. A daemon
* (host::module/path) or local TCP destination has no remote command line,
* so the option would be silently ignored; reject it by name instead. */
if (config->remote_option_count > 0 && config->transport != TRANSPORT_SSH) {
log_message(LOG_LEVEL_ERROR,
"-M/--remote-option is only valid with the SSH transport (user@host:path); it "
"cannot be used with a daemon (host::module/path) or local TCP destination");
return false;
}
/* -4 and -6 are mutually exclusive: a socket address family cannot be both. */
if (config->ipv4 && config->ipv6) {
log_message(LOG_LEVEL_ERROR, "-4/--ipv4 and -6/--ipv6 are mutually exclusive");
+341 -126
View File
@@ -98,17 +98,51 @@ static DirEntry* dir_entry_create(const char* path, int depth, FilterNode* conte
return de;
}
static bool safe_relative_link(const char* source_root, const char* containing_dir,
const char* link_target) {
char root[PATH_MAX];
if (!realpath(source_root, root))
return false;
char* joined = path_cat(containing_dir, link_target);
char resolved[PATH_MAX];
bool safe = joined && realpath(joined, resolved) && strncmp(root, resolved, strlen(root)) == 0 &&
(resolved[strlen(root)] == '\0' || resolved[strlen(root)] == '/');
free(joined);
return safe;
/* How rsync's readlink_stat()/generator resolves one source symlink. */
typedef enum {
LINK_ACTION_SKIP, /* not transferred (no link option) */
LINK_ACTION_SKIP_PROTECTED, /* ignored as unsafe by --safe-links; rsync keeps
it in the transfer, so its destination mirror
must be protected from --delete */
LINK_ACTION_DEREF, /* follow the referent (--copy-links, an unsafe
target under --copy-unsafe-links, or -k dir) */
LINK_ACTION_CARRY, /* transmit the link itself (-l) */
} LinkAction;
/* Apply rsync's symlink-resolution precedence to one S_ISLNK entry:
* --copy-links dereferences every symlink;
* --copy-unsafe-links dereferences only targets unsafe_symlink() flags;
* -k/--copy-dirlinks dereferences only a symlink whose referent is a dir;
* --safe-links (receiver-side in rsync; modelled here) ignores an unsafe
* target that would otherwise be carried; with --munge-links
* every stored target becomes absolute, so --safe-links then
* ignores every symlink, exactly as rsync documents;
* -l/--links carries the link.
* `link_rel` is the symlink's transfer-relative path (incl. name) and is used
* only for the lexical unsafe test. `target` receives the raw link value. */
static LinkAction scanner_link_action(const ScannerOptions* options, const char* path,
const char* link_rel, char* target, size_t target_size) {
if (!options->follow_symlinks && !options->copy_links && !options->safe_links &&
!options->copy_unsafe_links && !options->copy_dirlinks)
return LINK_ACTION_SKIP;
ssize_t length = readlink(path, target, target_size - 1);
if (length < 0)
return LINK_ACTION_SKIP;
target[length] = '\0';
bool unsafe = file_symlink_unsafe(target, link_rel);
if (options->copy_links || (options->copy_unsafe_links && unsafe))
return LINK_ACTION_DEREF;
if (options->copy_dirlinks) {
struct stat ref;
if (stat(path, &ref) == 0 && S_ISDIR(ref.st_mode))
return LINK_ACTION_DEREF;
}
if (options->safe_links && (unsafe || options->munge_links))
return LINK_ACTION_SKIP_PROTECTED;
if (!options->follow_symlinks || target[0] == '\0')
return LINK_ACTION_SKIP;
return LINK_ACTION_CARRY;
}
typedef struct {
@@ -122,9 +156,13 @@ typedef struct {
bool is_symlink;
char* link_target;
/* True when the entry was pruned by a user selection rule (--filter/-C/per-dir
rules, the --exclude/--include layer, or --max-size/--min-size) rather than
skipped for another reason (unreadable, symlink policy, not applicable). */
rules or the --exclude/--include layer) rather than skipped for another
reason (unreadable, symlink policy, not applicable). */
bool excluded;
/* True when the entry was skipped specifically by --max-size/--min-size.
Size pruning protects the destination mirror even under --delete-excluded,
so it is recorded into a separate sink from `excluded`. */
bool size_excluded;
} ScannerEntry;
/* --one-file-system (-x) decision. Only directories can carry a different
@@ -134,6 +172,26 @@ bool scanner_same_filesystem(bool one_file_system, dev_t root_device, dev_t entr
return !one_file_system || entry_device == root_device;
}
/* Build a payload-less directory File carrying the captured metadata (when
* requested). Used by -x mount-point emission and --list-only directory
* entries. Returns NULL on allocation failure. */
static File* scanner_build_dir_file(const char* path, const struct stat* stats,
const ScannerOptions* options) {
File* dir = file_create(path);
if (dir == NULL)
return NULL;
dir->is_dir = true;
if (options->use_metadata) {
dir->metadata =
file_metadata_create(dir->path, stats, options->preserve_atimes, options->preserve_crtimes);
if (!dir->metadata) {
file_destroy(dir);
return NULL;
}
}
return dir;
}
/* Relative path of an on-disk path below `root`. The transfer root may be
* given with a trailing slash; the returned rel path never has one and is ""
* for the root itself. A root of "/" is handled (its children start at "/").
@@ -210,24 +268,35 @@ static void scanner_assign_hardlink(DirectoryScanner* scanner, HardLinkTable* ta
}
}
/* Phase 4 special/devices: detect a device (char/block), FIFO or socket entry
and, when the matching --devices/--specials flag asks it be preserved,
convert the File into a node to recreate (is_special, empty payload) with its
device rdev captured from the source stat. When the entry is not preserved
(or --copy-devices instead copies its content as an ordinary regular file)
the File is left as a normal data file. Returns true when converted. */
static bool scanner_prepare_special(bool preserve_devices, bool preserve_specials, File* file,
const struct stat* stats) {
/* Phase 4 special/devices decision for one non-regular entry, matching rsync:
- a char/block device is RECREATED as a node under -D/--devices, unless
--copy-devices asks for its content to be copied into a regular file;
- a FIFO/socket is RECREATED under --specials;
- when the matching flag is absent the entry is SKIPPED ("skipping
non-regular file"), exactly like rsync's default, instead of being
silently copied as a zero-length regular file;
- anything else (regular/directory) is left to the normal data path. */
typedef enum {
SCANNER_SPECIAL_REGULAR, /* ordinary file: transfer content */
SCANNER_SPECIAL_RECREATE, /* is_special node to recreate on the receiver */
SCANNER_SPECIAL_SKIP, /* non-regular entry not requested: skip */
} ScannerSpecial;
static ScannerSpecial scanner_prepare_special(bool preserve_devices, bool preserve_specials,
bool copy_devices, File* file,
const struct stat* stats) {
if (!file || !stats)
return false;
return SCANNER_SPECIAL_REGULAR;
bool is_device = S_ISCHR(stats->st_mode) || S_ISBLK(stats->st_mode);
bool is_fifo = S_ISFIFO(stats->st_mode);
bool is_socket = S_ISSOCK(stats->st_mode);
if (!is_device && !is_fifo && !is_socket)
return false;
return SCANNER_SPECIAL_REGULAR;
if (is_device && copy_devices)
return SCANNER_SPECIAL_REGULAR; /* copy device content as a regular file */
bool preserve = is_device ? preserve_devices : preserve_specials;
if (!preserve)
return false;
return SCANNER_SPECIAL_SKIP;
file->is_special = true;
file->data->size = 0;
file->data->data = NULL;
@@ -235,7 +304,7 @@ static bool scanner_prepare_special(bool preserve_devices, bool preserve_special
file->rdev_major = (int32_t)major(stats->st_rdev);
file->rdev_minor = (int32_t)minor(stats->st_rdev);
}
return true;
return SCANNER_SPECIAL_RECREATE;
}
/* Append `rel` to the caller's exclusion sink, taking `mtx` when shared across
@@ -257,19 +326,49 @@ static bool excluded_sink_append(ArrayList* list, mtx_t* mtx, const char* rel) {
return ok;
}
/* Record one pruned-by-user-selection filesystem path in the scanner's
exclusion sink (see ScannerOptions.excluded_paths). The stored form is the
entry's wire/destination-relative path (a single leading '/' removed, exactly
how manifest keep entries are stored), so the receiver's walker prefixes
match the destination layout. An allocation failure is a fatal scan error. */
static void scanner_record_excluded(DirectoryScanner* scanner, const char* fs_path) {
if (!scanner->options.excluded_paths || !fs_path)
/* Record one pruned filesystem path in a delete-protection sink. The stored
form is the entry's wire/destination-relative path (a single leading '/'
removed, exactly how manifest keep entries are stored), so the receiver's
walker prefixes match the destination layout. An allocation failure is a
fatal scan error. */
static void scanner_record_protected(DirectoryScanner* scanner, const char* fs_path,
ArrayList* sink) {
if (!sink || !fs_path)
return;
const char* rel = *fs_path == '/' ? fs_path + 1 : fs_path;
if (!excluded_sink_append(scanner->options.excluded_paths, scanner->options.excluded_mutex, rel))
if (!excluded_sink_append(sink, scanner->options.excluded_mutex, rel))
scanner->failed = true;
}
/* A user-selection exclusion (--filter/-C/per-dir or --exclude/--include). */
static void scanner_record_excluded(DirectoryScanner* scanner, const char* fs_path) {
scanner_record_protected(scanner, fs_path, scanner->options.excluded_paths);
}
/* A --max-size/--min-size prune (always protected, even under --delete-excluded). */
static void scanner_record_size_skipped(DirectoryScanner* scanner, const char* fs_path) {
scanner_record_protected(scanner, fs_path, scanner->options.size_skipped_paths);
}
/* Record a directory the scan synchronized. `fs_path` is its absolute path and
`rel` its path relative to the transfer root ("" for the root); the stored
form matches the wire layout (the bare relative path in -R+--files-from, else
the source path with a leading '/' removed, with "." for the receive root).
Returns false on allocation failure. */
static bool scanner_record_synced_dir(const ScannerOptions* options, const char* fs_path,
const char* rel, bool relative_mode) {
if (!options->synced_dirs)
return true;
if (!file_list_dir_in_scope(options->file_list, rel))
return true;
const char* dest = relative_mode ? rel : fs_path;
if (dest[0] == '/')
dest++;
if (dest[0] == '\0')
dest = ".";
return excluded_sink_append(options->synced_dirs, options->excluded_mutex, dest);
}
/* Merge the open directory's own .rsync-filter rules into the inherited
* context, returning the context used for this directory's entries. On a parse
* error the scanner is marked failed. Returns 0 on success, -1 on failure. */
@@ -306,11 +405,13 @@ static int open_directory_filter_context(DirectoryScanner* scanner, const Filter
return 0;
}
/* Inspect symlinks, resolve the entry type, and apply file filters once for both scanners. */
static int scanner_inspect_entry(const ScannerOptions* options, const char* source_root,
const char* containing_dir, const char* name,
ScannerEntry* entry) {
/* Inspect symlinks, resolve the entry type, and apply file filters once for both scanners.
* `link_rel` is the entry's path relative to the transfer root (including its
* name), used for the lexical rsync unsafe-symlink test. */
static int scanner_inspect_entry(const ScannerOptions* options, const char* containing_dir,
const char* link_rel, const char* name, ScannerEntry* entry) {
entry->excluded = false;
entry->size_excluded = false;
entry->is_symlink = false;
entry->link_target = NULL;
entry->path = path_cat(containing_dir, name);
@@ -322,72 +423,49 @@ static int scanner_inspect_entry(const ScannerOptions* options, const char* sour
free(entry->path);
return 0;
}
bool is_symlink = S_ISLNK(link_stats.st_mode);
if (!is_symlink)
if (!S_ISLNK(link_stats.st_mode))
goto regular;
/* Symlink: choose between dereferencing (---copy-links / --safe-links /
--copy-unsafe-links, plus -k for symlinks-to-directories) and carrying the
link through as a symlink (-l, and -k for symlinks-to-files). No link
option means the symlink is skipped entirely (pre-existing behavior). */
const bool any_link_option = options->follow_symlinks || options->copy_links ||
options->safe_links || options->copy_unsafe_links ||
options->copy_dirlinks;
if (!any_link_option)
goto skip;
char link_target[4096];
ssize_t length = readlink(entry->path, link_target, sizeof(link_target) - 1);
if (length < 0)
switch (scanner_link_action(options, entry->path, link_rel, link_target, sizeof(link_target))) {
case LINK_ACTION_SKIP:
goto skip;
link_target[length] = '\0';
if (options->safe_links) {
if (link_target[0] == '/' || !safe_relative_link(source_root, containing_dir, link_target))
goto skip;
}
if (options->copy_unsafe_links && !options->copy_links) {
if (link_target[0] != '/')
goto skip;
}
bool emit_symlink = false;
if (options->copy_links) {
emit_symlink = false; /* --copy-links dereferences every referent */
} else if (options->safe_links || options->copy_unsafe_links) {
emit_symlink = false; /* preserve pre-existing dereference behavior */
} else if (options->copy_dirlinks) {
struct stat ref;
if (stat(entry->path, &ref) == 0 && S_ISDIR(ref.st_mode))
emit_symlink = false; /* -k: symlink to a directory recurses as a dir */
else
emit_symlink = true; /* -k: symlink to a file stays a symlink */
} else if (options->follow_symlinks) {
emit_symlink = true; /* -l: copy symlink as symlink */
}
if (!emit_symlink) {
if (stat(entry->path, &entry->stats) != 0)
case LINK_ACTION_SKIP_PROTECTED:
/* --safe-links ignored the link, but rsync still counts it as present in
the transfer, so its destination mirror survives --delete. Record it as
an excluded path (the same delete-protection channel as a filter prune). */
entry->excluded = true;
goto skip;
case LINK_ACTION_DEREF:
if (stat(entry->path, &entry->stats) != 0) {
/* rsync reports "symlink has no referent" and continues (exit 23); we
surface the same condition rather than silently dropping the entry. */
char* escaped = output_escape(entry->path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "symlink has no referent: %s",
escaped ? escaped : "<allocation failed>");
free(escaped);
goto skip;
}
entry->is_directory = S_ISDIR(entry->stats.st_mode);
if (entry->is_directory)
return 1;
goto apply_filters;
case LINK_ACTION_CARRY:
break;
}
/* Carry the link as a symlink. --munge-links containment: a target that
could escape the receive root (absolute or containing "..") is never
transmitted -- the entry is merely skipped ("contained"). */
if (link_target[0] == '\0' ||
(options->munge_links && !file_symlink_target_contained(link_target)))
goto skip;
/* Carry the link as a symlink. --munge-links is applied by the RECEIVER (it
prefixes every stored target with /rsyncd-munged/); when the SOURCE already
holds a munged value the sender strips it so the receiver re-munges a clean
target, round-tripping a munged tree exactly like rsync. */
entry->is_symlink = true;
entry->stats = link_stats;
entry->is_directory = false;
entry->link_target =
options->munge_links ? file_symlink_munge(link_target) : str_dup(link_target);
entry->link_target = str_dup(link_target);
if (!entry->link_target)
goto skip;
if (options->munge_links)
file_symlink_unmunge(entry->link_target);
goto apply_filters;
regular:
@@ -420,6 +498,7 @@ apply_filters:
if ((options->max_size > 0 && (unsigned long long)entry->stats.st_size > options->max_size) ||
(options->min_size > 0 && (unsigned long long)entry->stats.st_size < options->min_size)) {
entry->excluded = true;
entry->size_excluded = true;
goto skip;
}
return 1;
@@ -586,7 +665,8 @@ static Chunk* chunk_data_to_chunk(ArrayList* chunk_data) {
* allocation failure is fatal and reported to the caller. */
static bool scanner_capture_dir_time(ArrayList* dir_entries, mtx_t* mutex, const char* root_path,
const char* fs_path, bool relative_mode, bool preserve_atimes,
bool preserve_crtimes) {
bool preserve_crtimes, bool preserve_xattrs,
bool preserve_acls) {
if (!dir_entries || !root_path || !fs_path)
return true;
struct stat st;
@@ -613,6 +693,11 @@ static bool scanner_capture_dir_time(ArrayList* dir_entries, mtx_t* mutex, const
file_destroy(file);
return false;
}
/* Directory xattrs/ACLs (-X/-A): captured here so the deferred
STATUS_DIR_TIMES frame can carry them and the receiver can re-apply them
fd-relative (a regular file's per-file block never covered directories). */
if (preserve_xattrs || preserve_acls)
file->xattrs = xattr_capture_path(fs_path, preserve_acls);
if (relative_mode) {
file->send_path = rel;
rel = NULL;
@@ -699,11 +784,24 @@ static int open_next_directory(DirectoryScanner* scanner) {
scanner->current_path = NULL;
return -1;
}
/* A successfully opened directory is synchronized for --delete: record it
so the receiver confines its extras walk to these (and the root sentinel
".") instead of the whole receive root. */
if (!scanner_record_synced_dir(&scanner->options, scanner->current_path, scanner->current_rel,
scanner->relative_mode)) {
closedir(scanner->current_dir);
scanner->current_dir = NULL;
free(scanner->current_path);
scanner->current_path = NULL;
scanner->failed = true;
return -1;
}
if (scanner->options.capture_dir_times &&
!scanner_capture_dir_time(scanner->options.dir_entries, scanner->options.dir_entries_mutex,
scanner->root_path, scanner->current_path, scanner->relative_mode,
scanner->options.preserve_atimes,
scanner->options.preserve_crtimes)) {
!scanner_capture_dir_time(
scanner->options.dir_entries, scanner->options.dir_entries_mutex, scanner->root_path,
scanner->current_path, scanner->relative_mode, scanner->options.preserve_atimes,
scanner->options.preserve_crtimes, scanner->options.preserve_xattrs,
scanner->options.preserve_acls)) {
closedir(scanner->current_dir);
scanner->current_dir = NULL;
free(scanner->current_path);
@@ -753,6 +851,7 @@ static File* dirs_root_dir_file(DirectoryScanner* scanner) {
return NULL;
}
}
scanner_capture_xattrs(scanner, file);
return file;
}
@@ -798,30 +897,57 @@ static File* dirs_file_for_entry(DirectoryScanner* scanner, const char* entry) {
return NULL;
}
struct stat effective = link_stats;
bool emit_symlink = false;
char* symlink_target = NULL;
if (S_ISLNK(link_stats.st_mode)) {
/* A symlink is transferred (following its referent) only when a link
resolution option is active, mirroring the regular scanner. */
bool resolve = scanner->options.follow_symlinks || scanner->options.copy_links ||
scanner->options.safe_links || scanner->options.copy_unsafe_links;
if (!resolve || stat(abs_path, &effective) != 0) {
/* Resolve the listed symlink with the same precedence as the recursive
scanner: dereference or carry the link. */
char link_target[4096];
LinkAction action =
scanner_link_action(&scanner->options, abs_path, entry, link_target, sizeof(link_target));
if (action == LINK_ACTION_SKIP || action == LINK_ACTION_SKIP_PROTECTED) {
free(abs_path);
return NULL;
}
if (action == LINK_ACTION_DEREF) {
if (stat(abs_path, &effective) != 0) {
free(abs_path);
return NULL;
}
} else {
emit_symlink = true;
symlink_target = str_dup(link_target);
if (!symlink_target) {
free(abs_path);
scanner->failed = true;
return NULL;
}
if (scanner->options.munge_links)
file_symlink_unmunge(symlink_target);
}
}
bool is_dir = S_ISDIR(effective.st_mode);
bool is_file = S_ISREG(effective.st_mode);
if (!is_dir && !is_file) {
if (!emit_symlink && !is_dir && !is_file) {
free(symlink_target);
free(abs_path);
return NULL;
}
File* file = file_create(abs_path);
free(abs_path);
if (!file) {
free(symlink_target);
scanner->failed = true;
return NULL;
}
file->is_dir = is_dir;
file->data->size = is_file ? (unsigned long long)effective.st_size : 0;
if (emit_symlink) {
file->is_symlink = true;
file->symlink_target = symlink_target;
symlink_target = NULL;
} else {
file->is_dir = is_dir;
file->data->size = is_file ? (unsigned long long)effective.st_size : 0;
}
if (scanner->relative_mode) {
file->send_path = str_dup(entry);
if (!file->send_path) {
@@ -839,6 +965,7 @@ static File* dirs_file_for_entry(DirectoryScanner* scanner, const char* entry) {
return NULL;
}
}
scanner_capture_xattrs(scanner, file);
return file;
}
@@ -978,24 +1105,38 @@ Chunk* directory_scanner_next(DirectoryScanner* scanner) {
continue;
ScannerEntry inspected;
int inspection = scanner_inspect_entry(&scanner->options, scanner->current_path,
scanner->current_path, entry->d_name, &inspected);
char* link_rel = child_rel_path(scanner->current_rel, entry->d_name);
if (!link_rel) {
scanner->failed = true;
break;
}
int inspection = scanner_inspect_entry(&scanner->options, scanner->current_path, link_rel,
entry->d_name, &inspected);
free(link_rel);
if (inspection < 0) {
scanner->failed = true;
break;
}
if (inspection == 0) {
/* The entry was pruned by a user selection rule (exclude/include/size) or
skipped for another reason; only the user-selection prunes protect the
corresponding destination mirror from --delete. */
/* A user-selection exclude protects its destination mirror from --delete
unless --delete-excluded; a size prune is always protected. Other
skips (unreadable, symlink policy) protect nothing. Under -R +
--files-from the protected prefix must be the entry's bare relative
wire path, not its source path (which would not match the destination
layout and would leave the mirror deletable). */
if (inspected.excluded) {
char* abs_path = path_cat(scanner->current_path, entry->d_name);
if (!abs_path) {
char* protected_path = scanner->relative_mode
? child_rel_path(scanner->current_rel, entry->d_name)
: path_cat(scanner->current_path, entry->d_name);
if (!protected_path) {
scanner->failed = true;
break;
}
scanner_record_excluded(scanner, abs_path);
free(abs_path);
if (inspected.size_excluded)
scanner_record_size_skipped(scanner, protected_path);
else
scanner_record_excluded(scanner, protected_path);
free(protected_path);
}
continue;
}
@@ -1044,9 +1185,31 @@ Chunk* directory_scanner_next(DirectoryScanner* scanner) {
free(rel_copy);
if (!scanner_same_filesystem(scanner->options.one_file_system, scanner->root_dev,
stats.st_dev)) {
/* rsync's -x/--one-file-system emits the mount-point directory entry
itself (so the destination gets an empty directory) but does NOT
descend into it. Build a payload-less directory File and hand it to
the caller; never enqueue it for traversal. */
File* mount = scanner_build_dir_file(cur_path, &stats, &scanner->options);
if (mount == NULL || !array_list_add(chunk_data, mount)) {
file_destroy(mount);
free(cur_path);
scanner->failed = true;
break;
}
free(cur_path);
continue;
}
/* --list-only: list directory entries too (rsync prints them), even
though a real transfer never sends them explicitly. */
if (scanner->options.list_dirs) {
File* dir = scanner_build_dir_file(cur_path, &stats, &scanner->options);
if (dir == NULL || !array_list_add(chunk_data, dir)) {
file_destroy(dir);
free(cur_path);
scanner->failed = true;
break;
}
}
int next_depth = scanner->current_depth + 1;
if (scanner->options.max_depth <= 0 || next_depth < scanner->options.max_depth) {
DirEntry* de = dir_entry_create(cur_path, next_depth, scanner->current_node);
@@ -1084,9 +1247,16 @@ Chunk* directory_scanner_next(DirectoryScanner* scanner) {
rel_copy = NULL;
}
/* --devices/--specials: a device/FIFO/socket entry marked for preservation
becomes a node to recreate (is_special, no data, rdev captured). */
scanner_prepare_special(scanner->options.preserve_devices, scanner->options.preserve_specials,
file, &stats);
becomes a node to recreate (is_special, no data, rdev captured); an
unrequested non-regular entry is skipped (rsync default). */
ScannerSpecial special = scanner_prepare_special(scanner->options.preserve_devices,
scanner->options.preserve_specials,
scanner->options.copy_devices, file, &stats);
if (special == SCANNER_SPECIAL_SKIP) {
free(rel_copy);
file_destroy(file);
continue;
}
if (scanner->options.hardlinks && S_ISREG(stats.st_mode))
scanner_assign_hardlink(scanner, scanner->options.hardlinks, file, &stats);
if (scanner->options.use_metadata)
@@ -1335,24 +1505,33 @@ static void scan_root_entry(const ScannerOptions* options, const FilterNode* roo
ParallelScanner* ps) {
ScannerEntry inspected;
int inspection =
scanner_inspect_entry(options, root_directory, root_directory, entry->d_name, &inspected);
scanner_inspect_entry(options, root_directory, entry->d_name, entry->d_name, &inspected);
if (inspection < 0) {
ps->failed = true;
return;
}
if (inspection == 0) {
if (inspected.excluded && options->excluded_paths) {
/* A root-level user-selection prune protects the destination mirror of
the same-named wire path (at the root the bare name is the wire path in
every layout). */
char* abs_path = path_cat(root_directory, entry->d_name);
if (!abs_path) {
ps->failed = true;
} else {
const char* rel = *abs_path == '/' ? abs_path + 1 : abs_path;
if (!excluded_sink_append(options->excluded_paths, options->excluded_mutex, rel))
ArrayList* sink = NULL;
if (inspected.excluded)
sink = inspected.size_excluded ? options->size_skipped_paths : options->excluded_paths;
if (sink) {
/* A root-level prune protects the destination mirror of the entry's wire
path: under -R + --files-from that is the bare relative name, otherwise
it is the full source path with a leading '/' removed (matching the
send_path/file_wire_path the scanner hands the sender). */
if (options->relative && options->file_list != NULL) {
if (!excluded_sink_append(sink, options->excluded_mutex, entry->d_name))
ps->failed = true;
free(abs_path);
} else {
char* abs_path = path_cat(root_directory, entry->d_name);
if (!abs_path) {
ps->failed = true;
} else {
const char* rel = *abs_path == '/' ? abs_path + 1 : abs_path;
if (!excluded_sink_append(sink, options->excluded_mutex, rel))
ps->failed = true;
free(abs_path);
}
}
}
return;
@@ -1386,7 +1565,28 @@ static void scan_root_entry(const ScannerOptions* options, const FilterNode* roo
if (is_dir) {
free(rel);
if (!scanner_same_filesystem(options->one_file_system, root_dev, st.st_dev)) {
/* -x/--one-file-system: emit the mount-point directory entry (empty) but
do not descend into it (see the sequential scanner for the same rule). */
File* mount = file_create(cur_path);
free(cur_path);
if (mount == NULL) {
ps->failed = true;
return;
}
mount->is_dir = true;
if (options->use_metadata) {
mount->metadata = file_metadata_create(mount->path, &st, options->preserve_atimes,
options->preserve_crtimes);
if (!mount->metadata) {
file_destroy(mount);
ps->failed = true;
return;
}
}
if (!array_list_add(root_files, mount)) {
file_destroy(mount);
ps->failed = true;
}
return;
}
if (!array_list_add(subdirs, cur_path)) {
@@ -1415,7 +1615,13 @@ static void scan_root_entry(const ScannerOptions* options, const FilterNode* roo
file->send_path = rel;
rel = NULL;
}
scanner_prepare_special(options->preserve_devices, options->preserve_specials, file, &st);
ScannerSpecial special = scanner_prepare_special(
options->preserve_devices, options->preserve_specials, options->copy_devices, file, &st);
if (special == SCANNER_SPECIAL_SKIP) {
free(rel);
file_destroy(file);
return;
}
if (options->hardlinks && S_ISREG(st.st_mode)) {
int gid;
bool is_first;
@@ -1465,6 +1671,14 @@ static bool scan_root_directory(ParallelScanner* ps, const char* root_directory,
log_perror("Could not open root directory for parallel scan");
return false;
}
/* The parallel scanner opens the transfer root directly (not through
open_next_directory), so record it as synchronized here. */
if (!scanner_record_synced_dir(options, root_directory, "",
options->relative && options->file_list != NULL)) {
closedir(dir);
ps->failed = true;
return false;
}
const struct dirent* entry;
while ((entry = readdir(dir)) != NULL) {
if (strcmp(entry->d_name, ".") == 0 || strcmp(entry->d_name, "..") == 0)
@@ -1629,7 +1843,8 @@ ParallelScanner* parallel_scanner_create_with_options(const char* root_directory
if (options->capture_dir_times &&
!scanner_capture_dir_time(options->dir_entries, options->dir_entries_mutex, root_directory,
root_directory, options->relative && options->file_list != NULL,
options->preserve_atimes, options->preserve_crtimes)) {
options->preserve_atimes, options->preserve_crtimes,
options->preserve_xattrs, options->preserve_acls)) {
array_list_delete(root_files);
array_list_delete(subdirs);
parallel_scanner_destroy(ps);
+28 -9
View File
@@ -65,6 +65,10 @@ typedef struct {
bool per_dir_filters; /* -F: read .rsync-filter per directory */
bool dirs; /* -d/--dirs: transfer dir entries, no recursion */
bool relative; /* -R/--relative (dest rel paths, with --files-from) */
/* --list-only: emit an is_dir File for every traversed directory (the listing
* includes directory entries, matching rsync). Client-only; never set on a
* real transfer, which relies on implicit parent creation. */
bool list_dirs;
/* --prune-empty-dirs (long only): in --dirs mode an empty source directory's
explicit entry is omitted from the transfer file list (so nothing is
created at the destination and it can be pruned by --delete); explicitly
@@ -73,17 +77,32 @@ typedef struct {
bool prune_empty_dirs;
/* Delete-excluded protection sink (optional): when non-NULL the scanner
* appends the destination-relative path of every entry it prunes because a
* USER SELECTION rule excluded it (--filter/-C/per-dir rules, the legacy
* --exclude/--include layer, and --max-size/--min-size). The sender turns
* this list into the manifest's protected prefixes so `--delete` leaves the
* destination mirror of excluded source paths alone (rsync's default), and
* empties it when --delete-excluded opts back into deleting them. NOT
* recorded for --files-from subset pruning (whose delete semantics stay
* keep-set-only) or for -R/--files-from relative wire paths. When
* `excluded_mutex` is non-NULL it is taken around every append (the parallel
* scanner shares one list across its worker threads). */
* USER SELECTION rule excluded it (--filter/-C/per-dir rules and the legacy
* --exclude/--include layer). The sender turns this list into the manifest's
* protected prefixes so `--delete` leaves the destination mirror of excluded
* source paths alone (rsync's default), and drops it when --delete-excluded
* opts back into deleting them. NOT recorded for --files-from subset pruning
* (whose delete semantics derive from the synchronized-directory set) or for
* -R/--files-from relative wire paths. When `excluded_mutex` is non-NULL it
* is taken around every append (the parallel scanner shares one list across
* its worker threads). */
ArrayList* excluded_paths;
mtx_t* excluded_mutex;
/* Size-prune protection sink (optional): when non-NULL the scanner appends
* the destination-relative path of every entry it skipped because of
* --max-size/--min-size. rsync never deletes a size-skipped source mirror,
* even under --delete-excluded, so the sender always transmits this list as
* protected prefixes (unlike excluded_paths, which --delete-excluded drops).
* Guarded by `excluded_mutex` like excluded_paths. */
ArrayList* size_skipped_paths;
/* Synchronized-directory sink (optional): when non-NULL the scanner appends
* the destination-relative path of every directory it is about to traverse
* that lies inside a --files-from listed directory (or of every traversed
* directory when there is no list). The sender sends this set with the delete
* manifest so the receiver confines its extras walk to synchronized
* directories, exactly like rsync; the receive root is the "." sentinel.
* Guarded by `excluded_mutex`. */
ArrayList* synced_dirs;
/* --ignore-errors: an unreadable directory during the scan is recorded as an
* I/O error and skipped instead of aborting the scan. Client-only. */
bool ignore_io_errors;
+96 -55
View File
@@ -20,11 +20,16 @@ void print_usage(void) {
printf("Options:\n");
printf(" -c, --checksum Verify content by checksum instead of size+mtime\n");
printf(" -z, --compress [level] Enable compression (level 1-22, default 5)\n");
printf(" -a, --archive rsync archive mode (-rlptgoD): links, metadata,\n");
printf(" devices and specials (not compression/multithreading)\n");
printf(" -a, --archive rsync archive mode (-rlptgoD): links, perms, times,\n");
printf(" owner, group, devices and specials; not\n");
printf(" compression/multithreading\n");
printf(" -r, --recursive Recurse into directories (FastSync is always recursive)\n");
printf(" -n, --dry-run Show what would be transferred\n");
printf(" --remove-source-files Remove regular source files after successful transfer\n");
printf(" -p, --perms Preserve permission bits (part of the metadata bundle)\n");
printf(" -p, --perms Preserve permission bits\n");
printf(" -t, --times Preserve modification times\n");
printf(" -o, --owner Preserve owner (uid)\n");
printf(" -g, --group Preserve group (gid)\n");
printf(" --ssh-port <port> SSH port (default: 22)\n");
printf(" -e, --rsh <command> Remote shell to launch on the client for the SSH\n");
printf(" transport (default: ssh). The command may include\n");
@@ -54,6 +59,8 @@ void print_usage(void) {
printf(" Emit the batch file only (no destination, no server)\n");
printf(" --read-batch=FILE Apply the batch file to the destination (no source, no\n");
printf(" server); takes only the destination as an argument\n");
printf(" NOTE: the FastSync batch format is NOT interoperable with rsync's batch\n");
printf(" files (different container format); do not mix the two tools.\n");
printf(" --delete Delete files on receiver not in source\n");
printf(" (default timing: delete only after the whole\n");
printf(" transfer has succeeded)\n");
@@ -100,10 +107,10 @@ void print_usage(void) {
printf(" parent directory is not itself listed\n");
printf(" --mkpath Create the destination root directory on the server when it\n");
printf(" does not exist yet\n");
printf(" --exclude <pattern> Exclude files matching pattern\n");
printf(" --include <pattern> Only include files matching pattern\n");
printf(" --exclude-from <file> Read exclude patterns from file\n");
printf(" --include-from <file> Read include patterns from file\n");
printf(" --exclude <pattern>, --exclude=<pattern> Exclude files matching pattern\n");
printf(" --include <pattern>, --include=<pattern> Only include files matching pattern\n");
printf(" --exclude-from <file>, --exclude-from=<file> Read exclude patterns from file\n");
printf(" --include-from <file>, --include-from=<file> Read include patterns from file\n");
printf(" --files-from <file> Read the source file list from FILE (paths relative to the "
"source root)\n");
printf(" -0, --from0 Entries in --files-from are NUL-delimited\n");
@@ -113,7 +120,8 @@ void print_usage(void) {
printf(" -F Apply per-directory .rsync-filter files during the scan\n");
printf(" --max-size <n> Skip files larger than n bytes\n");
printf(" --min-size <n> Skip files smaller than n bytes\n");
printf(" --max-alloc <SIZE> Maximum single allocation (default: 1G)\n");
printf(" --max-alloc <SIZE> Maximum single allocation (default: 1G; 0 = no limit,\n");
printf(" matching rsync)\n");
printf(" --incremental Skip files unchanged since last transfer\n");
printf(" --size-only Skip incremental files matching in size, ignoring mtime\n");
printf(" -I, --ignore-times Transfer files even when size and mtime match\n");
@@ -128,20 +136,24 @@ void print_usage(void) {
printf(" --link-dest <dir> Like --copy-dest, but hard-links the unchanged file from DIR\n");
printf(" into the destination (repeatable; earlier DIRs win)\n");
printf(" --checksum-choice, --cc <alg> Whole-file checksum algorithm for --incremental/\n");
printf(" --checksum compares (xxh64/xxhash or md5; default xxh64 with\n");
printf(" seed 0). The seed comes from --checksum-seed\n");
printf(" --checksum-seed <num> Seed for the whole-file xxHash64 digest (and the delta\n");
printf(" block strong hash, low 32 bits); md5 ignores the seed. The\n");
printf(" digest algorithm and seed must match on sender and receiver\n");
printf(" --checksum compares. Accepted: xxh64 (aka xxhash), xxh3,\n");
printf(" xxh128, md5, or auto (default xxh64). rsync choices FastSync\n");
printf(" does not implement (md4, sha1, none) and the two-name\n");
printf(" transfer,pre-transfer form are rejected by name\n");
printf(" --checksum-seed <num> Seed for the whole-file xxHash digest (and the delta\n");
printf(" block strong hash, low 32 bits); md5 ignores the seed. A seed\n");
printf(" of 0 (the default) is randomized per transfer, exactly like\n");
printf(" rsync, and the chosen seed is sent to the receiver\n");
printf(" --delta Delta transfer for changed files (requires --incremental)\n");
printf(" -W, --whole-file Transfer changed files without delta processing\n");
printf(" --no-whole-file rsync spelling that clears -W/--whole-file\n");
printf(" -y, --fuzzy Use a similar-named file already in the destination\n");
printf(" directory as the delta basis when the destination has no\n");
printf(" usable file at the exact path (saves bandwidth; implies\n");
printf(" --incremental and --delta; inert with --whole-file,\n");
printf(" --no-delta, or --no-incremental)\n");
printf(" --no-fuzzy Disable --fuzzy\n");
printf(" --delta-block <n>, --block-size <n>\n");
printf(" -B <n>, --block-size <n>, --delta-block <n>\n");
printf(" Delta block size in bytes (default: %d)\n", DELTA_BLOCK_SIZE_DEFAULT);
printf(" --delta-max <n> Max file size for delta transfer (default: %llu)\n",
DELTA_MAX_FILE_SIZE);
@@ -152,7 +164,8 @@ void print_usage(void) {
printf(" --chunk-serialization Enable chunk serialization (long form only)\n");
printf(" -s, --secluded-args Protect-args compatibility option (no effect; remote\n");
printf(" SSH argv is already built injection-safe)\n");
printf(" --sendfile Enable sendfile zero-copy (TCP only; long form only)\n");
printf(" --sendfile Enable sendfile zero-copy (TCP only; long form only;\n");
printf(" -f is bound to --filter, not --sendfile)\n");
printf(" --compress-choice <alg> Compression algorithm (default: zstd)\n");
printf(" --zc <alg> Alias for --compress-choice\n");
printf(" -v, --verbose Enable debug logging\n");
@@ -160,8 +173,16 @@ void print_usage(void) {
printf(" --debug=FLAGS Fine-grained debug logging (use --debug=help for flags)\n");
printf(" --info=FLAGS Fine-grained info: copy,misc,skip,stats,all,none\n");
printf(" none suppresses info even with --verbose\n");
printf(" --preserve Preserve file metadata (long form only)\n");
printf(" --preserve Preserve permissions and times (= -pt; long form only)\n");
printf(" --no-perms Negate -p/--perms\n");
printf(" --no-times Negate -t/--times\n");
printf(" --no-owner Negate -o/--owner\n");
printf(" --no-group Negate -g/--group\n");
printf(" --no-preserve Disable metadata preservation (negates --preserve)\n");
printf(" -E, --executability Preserve executable permission bits\n");
printf(" -U, --atimes Preserve access times\n");
printf(" -N, --crtimes Capture birth time; cannot be applied (documented\n");
printf(" divergence)\n");
printf(" -X, --xattrs Preserve user extended attributes (user.* only;\n");
printf(" privileged security.*/trusted.* namespaces are\n");
printf(" never captured or applied)\n");
@@ -176,28 +197,32 @@ void print_usage(void) {
printf(" (char/block device-node creation, --write-devices)\n");
printf(" within the confined receive root. Never elevates\n");
printf(" privileges and never bypasses confinement; ownership\n");
printf(" is still applied only with an explicit identity flag\n");
printf(" (--numeric-ids/--chown/--usermap/--groupmap/--copy-as)\n");
printf(" is still applied only with -o/--owner, -g/--group, or an\n");
printf(" explicit identity flag (--chown/--usermap/--groupmap/\n");
printf(" --copy-as); --numeric-ids only changes how ids map\n");
printf(" --no-super Forbid those super-user activities even when the\n");
printf(" receiver is running as root\n");
printf(" --chmod <changes> Modify transferred permissions (rsync syntax)\n");
printf(" --numeric-ids Do not map uid/gid by name: use the source numeric\n");
printf(" ids directly when applying ownership\n");
printf(
" --chmod <changes> Modify new/transferred permissions (rsync syntax; implies no -p)\n");
printf(" --numeric-ids Map uid/gid by id instead of by name (a modifier, not\n");
printf(" an ownership request: combine with -o/-g or a map)\n");
printf(" --usermap=MAP Map usernames when applying ownership: comma-separated\n");
printf(" FROM:TO rules, first match wins. FROM/TO are names\n");
printf(" (resolved on the source machine), * (match any /\n");
printf(" current user), or @N numeric ids. e.g. *:nobody\n");
printf(" FROM:TO rules, first match wins. FROM is a name (from\n");
printf(" the source), an id, an inclusive LOW-HIGH range, *\n");
printf(" (any id), or empty (ids with no name). TO is an id, *\n");
printf(" (current user), or a name resolved on the receiver.\n");
printf(" e.g. 0-99:nobody,*:normal (cannot mix with --chown)\n");
printf(" --groupmap=MAP Map group names when applying ownership (same syntax)\n");
printf(" --chown=USER:GROUP Override the ownership of transferred files. Forms:\n");
printf(" USER:GROUP, USER (owner only), :GROUP (group only); a\n");
printf(" value of * means the current/root user as appropriate.\n");
printf(" Names resolve on the source machine; @N for numerics.\n");
printf(" (Metadata is enabled with --preserve; -M now means\n");
printf(" rsync's --remote-option.)\n");
printf(" (Implies owner/group metadata; -M now means rsync's\n");
printf(" --remote-option.)\n");
printf(" --copy-as=USER[:GROUP] Force every written entry (files, dirs, symlinks\n");
printf(" and special nodes) to USER[:GROUP], resolved on the\n");
printf(" source machine like --chown. Requires a privileged\n");
printf(" (root) receiver and implies --preserve; an\n");
printf(" (root) receiver and implies owner/group metadata; an\n");
printf(" unprivileged receiver refuses the transfer. Never\n");
printf(" switches process credentials (safe-subset; see\n");
printf(" RSYNC_COMPAT.md). A daemon refuses it.\n");
@@ -209,9 +234,10 @@ void print_usage(void) {
printf(" --server-port <n> Server port (default: 8080)\n");
printf(" --port <n> Alias for --server-port\n");
printf(" --password-file <f> Authenticate a host::module/path daemon destination.\n");
printf(" The file's first user:password line supplies the\n");
printf(" username and password (only a SHA-256 digest of the\n");
printf(" password is sent; keep the file mode 0600)\n");
printf(" FastSync-native SCRAM/PBKDF2 credential scheme (NOT\n");
printf(" rsync's --password-file): the file's first user:password\n");
printf(" line supplies the username and password; no password or\n");
printf(" reusable digest is sent (keep the file mode 0600)\n");
printf(" --no-motd Suppress display of the daemon's MOTD (the server\n");
printf(" still sends it; the client just does not show it)\n");
printf(" --bwlimit <KB/s> Bandwidth limit in kilobytes per second\n");
@@ -219,21 +245,25 @@ void print_usage(void) {
printf(" --cert <path> TLS certificate file (PEM)\n");
printf(" --key <path> TLS private key file (PEM)\n");
printf(" --ca <path> TLS CA certificate file (PEM)\n");
printf(" --timeout <sec> I/O timeout in seconds (default: 30; long form only)\n");
printf(" --contimeout <sec> Connection timeout in seconds (default: 10)\n");
printf(" --timeout <sec> I/O timeout in seconds (default: 0 = disabled, matching\n");
printf(" rsync). 0 disables it; --no-timeout is the same\n");
printf(" --contimeout <sec> Connection timeout in seconds (default: 60, matching\n");
printf(" rsync); 0 disables it (--no-contimeout)\n");
printf(" --stop-after=MINS Stop the transfer after MINS minutes (a positive\n");
printf(" integer); whatever was already transferred is kept\n");
printf(" --stop-at=TIME Stop at an absolute time: HH:MM, HH:MM:SS, or\n");
printf(" now+N[smhd] (a time already in the past stops the\n");
printf(" transfer immediately; client-only). An early stop\n");
printf(" skips the late --delete keep-set so it cannot delete\n");
printf(" source mirrors that were not yet scanned\n");
printf(" --stop-at=TIME Stop at an absolute time. Accepts rsync's date form\n");
printf(" (Y-M-DTh:m, Y/M/DTh:m, abbreviable fields such as 12-31,\n");
printf(" 14:00, :59, 1) plus FastSync's HH:MM[:SS] and now+N[smhd]\n");
printf(" (a time already in the past stops the transfer\n");
printf(" immediately; client-only). An early stop skips the late\n");
printf(" --delete keep-set so it cannot delete source mirrors that\n");
printf(" were not yet scanned\n");
printf(" --address <ip> Bind the outgoing client socket to this source address\n");
printf(" -4, --ipv4 Force IPv4 for destination resolution\n");
printf(" -6, --ipv6 Force IPv6 for destination resolution\n");
printf(" --sockopts=OPTS Comma-separated OPT=VAL socket options applied before connect:\n");
printf(" TCP_NODELAY, SO_KEEPALIVE, SO_RCVBUF, SO_SNDBUF, SO_REUSEADDR\n");
printf(" --backup Backup existing files before overwriting\n");
printf(" -b, --backup Backup existing files before overwriting\n");
printf(" --backup-dir <dir> Directory for backups (requires --backup)\n");
printf(" --suffix <str> Backup suffix (default: ~)\n");
printf(" --stats Print transfer statistics at end\n");
@@ -244,30 +274,40 @@ void print_usage(void) {
printf(" -h, --human-readable Print byte sizes in human-readable form\n");
printf(" --max-depth <n> Maximum directory depth (0=unlimited)\n");
printf(" -x, --one-file-system Do not cross filesystem boundaries\n");
printf(" --log-file <path> Write log messages to file\n");
printf(" --log-file <path>, --log-file=<path> Write log messages to file\n");
printf(" --stderr=MODE Route logging to stderr: errors or all\n");
printf(" --partial Keep partial files on interrupted transfer\n");
printf(" --partial-dir <dir> Directory for partial files\n");
printf(" -T, --temp-dir <dir> Scratch dir for temp files before atomic install\n");
printf(" -T, --temp-dir <dir> Scratch dir for temp files before atomic install.\n");
printf(" Relative dirs resolve below the destination root; absolute\n");
printf(" dirs are used as-is (rsync semantics). The dir must\n");
printf(" already exist; a different filesystem falls back to a\n");
printf(" non-atomic copy instead of aborting\n");
printf(" --fastsync-server-path <path>\n");
printf(" Path to fastsync-server on remote (default: fastsync-server)\n");
printf(" --old-args Accepted for rsync CLI compatibility; no effect (the\n");
printf(" remote server path is always safely quoted now)\n");
printf(" -M, --remote-option=OPT Append OPT to the REMOTE server invocation over SSH\n");
printf(" (repeatable; each value is single-quote-escaped on the remote\n");
printf(" command line; empty values and values with control characters\n");
printf(" are rejected; -M OPT, -M=OPT and --remote-option=OPT work)\n");
printf(" --trust-sender Trust the remote sender's file list: the receiver skips its\n");
printf(" own up-front path-traversal/containment re-validation of the\n");
printf(" incoming file list (fewer checks, faster, potentially unsafe).\n");
printf(" Local receiver policy: never sent to the peer, off by default\n");
printf(" -M, --remote-option=OPT Append OPT to the REMOTE server invocation. SSH\n");
printf(" transport ONLY (user@host:path): a daemon (host::module) or\n");
printf(" local TCP destination rejects it (no remote command line to\n");
printf(" append to). Repeatable; each value is single-quote-escaped on\n");
printf(" the remote command line; empty values and values with control\n");
printf(" characters are rejected; -M OPT, -M=OPT and\n");
printf(" --remote-option=OPT work\n");
printf(" --trust-sender RECEIVER-LOCAL policy: trust the remote sender's file list\n");
printf(" and skip the receiver's own up-front path-traversal/\n");
printf(" containment re-validation of the incoming list (fewer checks,\n");
printf(" faster, potentially unsafe). It is never sent to the peer, so\n");
printf(" for a push it must be enabled on the receiving SERVER\n");
printf(" (fastsync-server --trust-sender) or forwarded with\n");
printf(" -M--trust-sender; the client flag alone has no effect\n");
printf(" -l, --links Copy symlinks as symlinks\n");
printf(" --copy-links Transform symlinks into referent files\n");
printf(" --safe-links Skip symlinks that point outside transfer tree\n");
printf(" --copy-unsafe-links Only transform unsafe symlinks into referent files\n");
printf(" -L, --copy-links Transform symlinks into referent files\n");
printf(" --safe-links Skip symlinks whose target points outside the tree\n");
printf(" --copy-unsafe-links Copy unsafe symlinks (outside tree) as referent files\n");
printf(" -k, --copy-dirlinks Transform symlinks to directories into real dirs\n");
printf(" -K, --keep-dirlinks Keep an existing symlink-to-dir as that dir\n");
printf(" --munge-links Munge symlink targets on the wire (sender)\n");
printf(" --munge-links Munge stored symlink targets (/rsyncd-munged/) on the receiver\n");
printf(" -H, --hard-links Preserve hard-link relationships across the transfer\n");
printf(" -S, --sparse Handle sparse files efficiently\n");
printf(
@@ -275,8 +315,7 @@ void print_usage(void) {
printf(
" --devices Recreate device nodes on the destination (privileged; skipped when\n");
printf(" the receiver lacks CAP_MKNOD)\n");
printf(" --specials Recreate special files (FIFOs) on the destination (sockets "
"skipped)\n");
printf(" --specials Recreate special files (FIFOs, sockets) on the destination\n");
printf(" --copy-devices Copy a source device's content as a regular file instead\n");
printf(" --write-devices Write received data into an existing destination device node\n");
printf(" --inplace Update files in-place (no temp+rename)\n");
@@ -289,7 +328,9 @@ void print_usage(void) {
printf(" --fsync Fsync every written file before publication\n");
printf(" --compress-level <n> Compression level (default: 5)\n");
printf(" --zl <n> Alias for --compress-level\n");
printf(" --skip-compress=LIST Skip compression for comma-separated suffixes\n");
printf(" --skip-compress=LIST Skip compression for suffixes in LIST (separated by\n");
printf(" '/' as in rsync, or ','); a leading dot is optional. The\n");
printf(" default is rsync 3.4.1's built-in skip-compress list\n");
printf(" --compress-threads <n> Compression worker threads (requires zstd threaded support)\n");
printf(" --no-OPTION Disable a supported boolean option\n");
printf(" --help Show this help\n");
+40 -20
View File
@@ -43,17 +43,19 @@ void receiver_outcomes_destroy(ReceiverOutcomes* outcomes) {
/* End-of-transfer success frame. When --remove-source-files was negotiated
each processed data file is acknowledged first (STATUS_NEXT = written,
STATUS_OK = skipped) so the sender never removes a source the receiver did
not actually store. The frame always ends with a plain STATUS_OK. */
bool receiver_send_final_success(int fd, const Config* config, const ReceiverOutcomes* outcomes) {
not actually store. The frame ends with `final_status` (STATUS_OK, or
STATUS_DELETE_LIMIT when a --max-delete commit was capped). */
bool receiver_send_final_success(int fd, const Config* config, const ReceiverOutcomes* outcomes,
Status final_status) {
if (!config->remove_source_files)
return send_status(fd, STATUS_OK);
return send_status(fd, final_status);
size_t count = outcomes ? outcomes->count : 0;
for (size_t i = 0; i < count; i++) {
Status per_file = outcomes->entries[i] == FILE_SAVE_WRITTEN ? STATUS_NEXT : STATUS_OK;
if (!send_status(fd, per_file))
return false;
}
return send_status(fd, STATUS_OK);
return send_status(fd, final_status);
}
static bool receiver_process_chunk(Chunk* chunk, const ReceiverSink* sink) {
@@ -346,15 +348,19 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver
moment it arrives, before any file data. Delete now and acknowledge
so the sender only starts streaming once the deletion committed (or
failed). This is the rsync delete-before/delete-during window: a
later transfer failure does not restore these deletions. */
bool deletion_ok = (config->use_delete || config->delete_missing_args)
? manifest_delete_all(config, manifest)
: true;
later transfer failure does not restore these deletions. A
--max-delete-capped commit still succeeds and the transfer proceeds;
the terminal success frame reports the cap. */
DeleteCommitResult deletion = (config->use_delete || config->delete_missing_args)
? manifest_delete_all(config, manifest)
: DELETE_COMMIT_OK;
delete_manifest_free(manifest);
if (!deletion_ok) {
if (deletion == DELETE_COMMIT_ERROR) {
send_status(file_descriptor, STATUS_ERROR);
goto fail;
}
if (deletion == DELETE_COMMIT_LIMIT_REACHED && sink->note_delete_limit)
sink->note_delete_limit(sink->context);
if (!send_status(file_descriptor, STATUS_OK))
goto fail;
} else if (config->use_delete || config->delete_missing_args) {
@@ -407,13 +413,15 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver
*pending_manifest = deferred_manifest;
deferred_manifest = NULL;
} else {
bool deletion_ok = manifest_delete_all(config, deferred_manifest);
DeleteCommitResult deletion = manifest_delete_all(config, deferred_manifest);
delete_manifest_free(deferred_manifest);
deferred_manifest = NULL;
if (!deletion_ok) {
if (deletion == DELETE_COMMIT_ERROR) {
send_status(file_descriptor, STATUS_ERROR);
goto fail;
}
if (deletion == DELETE_COMMIT_LIMIT_REACHED && sink->note_delete_limit)
sink->note_delete_limit(sink->context);
}
}
if (sink->send_success) {
@@ -454,6 +462,9 @@ typedef struct {
after the whole transfer (and its delete/publication phases) has run so a
child write never clobbers a directory mtime. */
DirTimeList dir_times;
/* Set when a --max-delete commit was capped; the terminal frame then carries
STATUS_DELETE_LIMIT so the sender exits 25 like rsync. */
bool delete_limit_reached;
} ReceiverSaveContext;
static bool receiver_save_file(File* file, void* context_pointer) {
@@ -470,12 +481,13 @@ static bool receiver_save_file(File* file, void* context_pointer) {
} else {
result = file_save_to_disk_full(context->config->receive_root_directory, file, context->config);
}
/* A directory's times are deferred, never applied inline: collect the
metadata now and apply it at the end. -O/--omit-dir-times is honored by
dir_time_list_apply's caller (see receiver_send_success_frame). */
/* A directory's metadata is deferred, never applied inline: collect it now
and apply it at the end. -O/--omit-dir-times and --preserve_perms/-times
are honored by dir_metadata_list_apply's caller (see
receiver_send_success_frame). */
if (result != FILE_SAVE_ERROR && file->is_dir && file->metadata &&
dir_times_should_capture(context->config) &&
!dir_time_list_add(&context->dir_times, file->path, file->metadata)) {
dir_metadata_should_capture(context->config) &&
!dir_time_list_add(&context->dir_times, file->path, file->metadata, file->xattrs)) {
file_destroy(file);
return false;
}
@@ -493,12 +505,18 @@ static bool receiver_save_file(File* file, void* context_pointer) {
return result != FILE_SAVE_ERROR;
}
static void receiver_note_delete_limit(void* context_pointer) {
ReceiverSaveContext* context = context_pointer;
context->delete_limit_reached = true;
}
static bool receiver_send_success_frame(int fd, void* context_pointer) {
ReceiverSaveContext* context = context_pointer;
Status final_status = context->delete_limit_reached ? STATUS_DELETE_LIMIT : STATUS_OK;
/* Server-contacting --dry-run: nothing was staged or written, so there is
nothing to publish and no directory times to stamp. */
if (context->config->dry_run)
return receiver_send_final_success(fd, context->config, &context->outcomes);
return receiver_send_final_success(fd, context->config, &context->outcomes, final_status);
/* --delay-updates: the whole protocol stream (including manifest/delete
handling, which ran inside receiver_process) has succeeded and every
staged file was fully written. Publish them atomically now, before the
@@ -514,14 +532,16 @@ static bool receiver_send_success_frame(int fd, void* context_pointer) {
phases have committed, so it is finally safe to stamp directory times.
This runs after the deferred deletion because receiver_process commits it
before calling this success frame. */
dir_time_list_apply(&context->dir_times, context->config->receive_root_directory);
return receiver_send_final_success(fd, context->config, &context->outcomes);
dir_metadata_list_apply(&context->dir_times, context->config->receive_root_directory,
context->config);
return receiver_send_final_success(fd, context->config, &context->outcomes, final_status);
}
int receiver_receive_files(Config* config, int file_descriptor) {
ReceiverSaveContext context = {.config = config, .outcomes = {0}};
dir_time_list_init(&context.dir_times);
ReceiverSink sink = {receiver_save_file, &context, true, true, receiver_send_success_frame};
ReceiverSink sink = {receiver_save_file, &context, true, true, receiver_send_success_frame,
receiver_note_delete_limit};
int ret = receiver_process(config, file_descriptor, &sink);
if (ret != 0 && config->delay_updates && config->delay_context)
delay_updates_cleanup(config->delay_context);
+14 -2
View File
@@ -4,6 +4,7 @@
#include "config.h"
#include "file.h"
#include "file_receive.h"
#include "protocol.h"
#include <stdbool.h>
#include <time.h>
@@ -21,6 +22,12 @@ typedef struct {
typedef bool (*ReceiverSuccessFrame)(int fd, void* context);
/* Records that a --max-delete commit stopped with extras left over, so the
caller's terminal success frame can carry STATUS_DELETE_LIMIT instead of
STATUS_OK. The commit runs on the receiver thread, so the flag is stored in
the sink's own context rather than in a shared global. */
typedef void (*ReceiverNoteDeleteLimit)(void* context);
typedef struct {
ReceiverFileSink store_file;
void* context;
@@ -28,13 +35,18 @@ typedef struct {
bool send_success;
/* Emits the end-of-transfer success frame. When the sender requested
--remove-source-files this includes one per-file status per processed
data file followed by the final STATUS_OK; otherwise just STATUS_OK. */
data file followed by the final status; otherwise just the final status. */
ReceiverSuccessFrame send_success_frame;
/* Optional; may be NULL when the sink has no --max-delete handling. */
ReceiverNoteDeleteLimit note_delete_limit;
} ReceiverSink;
bool receiver_outcomes_append(ReceiverOutcomes* outcomes, unsigned char code);
void receiver_outcomes_destroy(ReceiverOutcomes* outcomes);
bool receiver_send_final_success(int fd, const Config* config, const ReceiverOutcomes* outcomes);
/* Send the terminal success frame. `final_status` is usually STATUS_OK, or
STATUS_DELETE_LIMIT when a --max-delete commit was capped. */
bool receiver_send_final_success(int fd, const Config* config, const ReceiverOutcomes* outcomes,
Status final_status);
int receiver_process(Config* config, int file_descriptor, const ReceiverSink* sink);
/* receiver_process with an escape hatch for the commit-style (late) deletion:
+14 -3
View File
@@ -27,6 +27,7 @@ PipelineContextReceiver* pipeline_context_receiver_create(Config* config, Queue*
context->queued_bytes = 0;
context->max_queue_bytes = 0;
context->deferred_manifest = NULL;
context->delete_limit_reached = false;
atomic_init(&context->cancelled, false);
int init = 0;
if (mtx_init(&context->mutex, mtx_plain) != thrd_success)
@@ -135,6 +136,15 @@ static bool receiver_enqueue_file(File* file, void* context_pointer) {
return pipeline_context_receiver_enqueue_file(context, file);
}
/* Early delete modes (--delete-before/--delete-during) commit the manifest
inside receiver_process_pending on this thread; record a capped commit so
server.c's terminal frame can report STATUS_DELETE_LIMIT. The plain bool is
safe: receive_thread writes it before the main thread joins the thread. */
static void receiver_pipeline_note_delete_limit(void* context_pointer) {
PipelineContextReceiver* context = (PipelineContextReceiver*)context_pointer;
context->delete_limit_reached = true;
}
static void receiver_thread_fail(PipelineContextReceiver* context) {
mtx_lock(&context->mutex);
atomic_store(&context->cancelled, true);
@@ -152,7 +162,8 @@ int receive_thread(void* pipeline_context) {
const Config* config = context->config;
mtx_unlock(&context->mutex);
ReceiverSink sink = {receiver_enqueue_file, context, false, false, NULL};
ReceiverSink sink = {
receiver_enqueue_file, context, false, false, NULL, receiver_pipeline_note_delete_limit};
if (receiver_process_pending((Config*)config, file_descriptor, &sink,
&context->deferred_manifest) != 0) {
receiver_thread_fail(context);
@@ -220,8 +231,8 @@ int write_thread(void* pipeline_context) {
write would clobber them); accumulate the metadata here and let the
caller apply it once every writer has drained. */
if (!dry_run && result != FILE_SAVE_ERROR && file->is_dir && file->metadata &&
dir_times_should_capture(context->config) &&
!dir_time_list_add(&context->dir_times, file->path, file->metadata)) {
dir_metadata_should_capture(context->config) &&
!dir_time_list_add(&context->dir_times, file->path, file->metadata, file->xattrs)) {
file_destroy(file);
pipeline_context_receiver_note_bytes_released(context, file_bytes);
mtx_lock(&context->mutex);
+4
View File
@@ -41,6 +41,10 @@ typedef struct PipelineContextReceiver {
transfer truly succeeded. NULL in the early delete modes (which delete at
the manifest). */
DeleteManifest* deferred_manifest;
/* Set by server.c when the deferred delete commit hit the --max-delete
budget; the terminal success frame then carries STATUS_DELETE_LIMIT
(rsync exit 25) while the transfer itself still succeeds. */
bool delete_limit_reached;
/* P7 Wave D: directory metadata collected by write_thread from received
directory entries. Only write_thread mutates it (before it joins); the
caller (server.c) applies it after the delete/delay-updates phase. */
+49 -14
View File
@@ -389,8 +389,12 @@ static const char* module_gate_check_ownership(const Config* config, const Daemo
ModuleGateContext* gate_ctx) {
if (module->client_owner)
return NULL;
/* Ownership: refuse the whole transfer up front (a clear failure). */
if (identity_ownership_requested(config)) {
/* Ownership: refuse the whole transfer up front (a clear failure) for a
* client-CHOSEN owner/group request. A plain -o/-g/-a preserve-source
* request is deliberately not in this narrow set: it falls through to the
* super-mode override below, which forces all ownership activity off for this
* connection so no chown happens (the transfer itself still succeeds). */
if (identity_explicit_ownership_requested(config)) {
log_message(LOG_LEVEL_ERROR,
"daemon module '%s' refuses client-chosen ownership/super-user activities "
"(no `client owner = yes` opt-in); refusing",
@@ -737,13 +741,22 @@ void handler(int file_descriptor) {
* received config. */
if (gate_ctx.super_mode_override != -1)
config->super_mode = (SuperMode)gate_ctx.super_mode_override;
/* If the client requested ownership but the effective super mode forbids it
* (operator --no-super, a privileged standalone receiver's secure default, or
* a daemon module without `client owner = yes`), say so ONCE per connection so
* a successful -a/-o/-g transfer is not mistaken for preserved ownership. */
if (config->super_mode == SUPER_MODE_OFF && identity_ownership_requested(config))
log_message(LOG_LEVEL_WARNING,
"requested ownership will NOT be applied: super-user activities are disabled "
"for this connection (operator veto, or module without `client owner = yes`)");
protocol_set_8_bit_output(config->eight_bit_output);
/* Server-side per-message protocol deadline for every frame from here on.
* `timeout` is not serialized, so this is the server's own config (the server
* has no --timeout CLI and defaults it to 0): the built-in 60 s window stays
* in effect. A client's --timeout tightens only that client's own protocol
* I/O and the server's socket read/write timeout is the transport default. */
protocol_session_set_io_timeout(&session, config->timeout);
* has no --timeout CLI and defaults it to 0). A client's --timeout tightens
* only that client's own protocol I/O; the server floors its own deadline at
* SERVER_IO_TIMEOUT_SEC so a silent peer can never hold a session slot
* forever (the socket layer gets the same floor at startup). */
protocol_session_set_io_timeout(&session, protocol_server_io_timeout_sec(config->timeout));
const char* authorized_root = utils_get_authorized_root_path();
if (!authorized_root) {
log_message(LOG_LEVEL_ERROR, "No server-side destination root configured");
@@ -892,7 +905,8 @@ void handler(int file_descriptor) {
goto done;
}
protocol_session_set_max_alloc(&context->session, config->max_alloc);
protocol_session_set_io_timeout(&context->session, config->timeout);
protocol_session_set_io_timeout(&context->session,
protocol_server_io_timeout_sec(config->timeout));
atomic_store(&context->session.total_allocated_bytes,
atomic_load(&session.total_allocated_bytes));
pipeline_context_receiver_set_queue_byte_limit(context, RECEIVER_QUEUE_MAX_BYTES);
@@ -937,8 +951,13 @@ void handler(int file_descriptor) {
--delay-updates run; the walker skips the staging directory. A
server-contacting --dry-run deletes nothing (no manifest is sent). */
if (context->deferred_manifest) {
if (!manifest_delete_all(config, context->deferred_manifest)) {
DeleteCommitResult deletion = manifest_delete_all(config, context->deferred_manifest);
if (deletion == DELETE_COMMIT_ERROR) {
transfer_ok = false;
} else if (deletion == DELETE_COMMIT_LIMIT_REACHED) {
/* The transfer still succeeds; the terminal frame reports the capped
deletion so the sender exits 25 like rsync. */
context->delete_limit_reached = true;
}
delete_manifest_free(context->deferred_manifest);
context->deferred_manifest = NULL;
@@ -959,10 +978,11 @@ void handler(int file_descriptor) {
to stamp directory times; a directory's mtime must not be clobbered by
its children or by an extra removal. */
if (transfer_ok)
dir_time_list_apply(&context->dir_times, config->receive_root_directory);
dir_metadata_list_apply(&context->dir_times, config->receive_root_directory, config);
}
if (transfer_ok) {
if (!receiver_send_final_success(file_descriptor, config, &context->outcomes))
Status final_status = context->delete_limit_reached ? STATUS_DELETE_LIMIT : STATUS_OK;
if (!receiver_send_final_success(file_descriptor, config, &context->outcomes, final_status))
transfer_ok = false;
} else {
send_error_detail(file_descriptor, "transfer failed on receiver");
@@ -1029,8 +1049,9 @@ static void print_server_usage(void) {
printf(" hosts allow, hosts deny)\n");
printf(" --no-detach Stay in the foreground (default detaches to\n");
printf(" background when running --daemon)\n");
printf(" --password-file=FILE Credential store for modules that declare\n");
printf(" 'auth users' (line format:\n");
printf(" --password-file=FILE FastSync-native SCRAM/PBKDF2 credential store (NOT\n");
printf(" rsync's auth scheme) for modules that declare 'auth\n");
printf(" users' (line format:\n");
printf(" user:$fastsync$1$pbkdf2-sha256$iters$salt$stored$server,\n");
printf(" generated by --hash-credentials). Legacy\n");
printf(" user:SHA256HEX lines are rejected. Requires\n");
@@ -1039,7 +1060,7 @@ static void print_server_usage(void) {
printf(" --early-input=FILE Second credential store layered over\n");
printf(" --password-file (same format); usually a secrets-\n");
printf(" manager/process-substitution file. Requires --daemon\n");
printf(" -p <port> TCP port (default: 8080, range: 1-65535)\n");
printf(" -p, --port <port> TCP port (default: 8080, range: 1-65535)\n");
printf(" --tls Enable TLS encryption\n");
printf(" --cert <path> TLS certificate file (PEM)\n");
printf(" --key <path> TLS private key file (PEM)\n");
@@ -1050,7 +1071,9 @@ static void print_server_usage(void) {
printf(" -4, --ipv4 Bind an IPv4 socket (default)\n");
printf(" -6, --ipv6 Bind an IPv6 socket\n");
printf(" --allow-delete Permit manifest deletion\n");
printf(" --trust-sender Trust the remote sender's file list\n");
printf(" --trust-sender Trust the remote sender's file list (receiver-local;\n");
printf(" this server-side flag is the only one that matters -- a\n");
printf(" client --trust-sender is never sent to the server)\n");
printf(" --no-super Operator veto: never attempt super-user activities\n");
printf(" (ownership, device nodes) even as root, and refuse\n");
printf(" any client --copy-as/--super request\n");
@@ -1127,10 +1150,18 @@ static bool daemonize(void) {
if (chdir("/") != 0)
log_message(LOG_LEVEL_WARNING, "daemon: chdir to / failed: %s", strerror(errno));
umask(0);
/* Refresh the cached umask: main() captured the launch umask before this
* (single-threaded) umask(0), and file_mode_base() must see the daemon's
* actual umask. */
file_umask_capture();
return true;
}
int main(int argc, char* argv[]) {
/* Capture the process umask now, while still single-threaded: the cached
* value is what file_mode_base() uses, and reading it later would race with
* receiver threads creating files. */
file_umask_capture();
ServerCliOptions opts;
char cli_err[512];
int parse_result = server_cli_parse(argc, argv, &opts, cli_err, sizeof(cli_err));
@@ -1191,6 +1222,10 @@ int main(int argc, char* argv[]) {
server_iconv_spec = opts.iconv_spec;
signal(SIGINT, cleanup);
signal(SIGTERM, cleanup);
/* Server-owned socket deadline floor: the client default --timeout=0 would
* otherwise leave accepted sockets without SO_RCVTIMEO/SO_SNDTIMEO and let a
* silent peer hold a connection (and its process slot) forever. */
tcp_set_timeouts(SERVER_IO_TIMEOUT_SEC, SERVER_IO_TIMEOUT_SEC);
if (opts.stdio_mode) {
/* SSH authenticates the stdio transport outside of FastSync. */
+13 -7
View File
@@ -192,14 +192,20 @@ int server_cli_parse(int argc, char* argv[], ServerCliOptions* opts, char* err,
inline_value = argv[++i];
}
opts->iconv_spec = inline_value;
} else if (arg_is(argv[i], "-p")) {
if (i + 1 >= argc) {
set_error(err, err_size, "missing argument for -p");
return -1;
} else if (arg_is(argv[i], "-p") || arg_has_value(argv[i], "--port", &inline_value)) {
if (inline_value) {
opts->port_set = true;
if (parse_port_arg(inline_value, &opts->port, err, err_size) != 0)
return -1;
} else {
if (i + 1 >= argc) {
set_error(err, err_size, "missing argument for %s", argv[i]);
return -1;
}
opts->port_set = true;
if (parse_port_arg(argv[++i], &opts->port, err, err_size) != 0)
return -1;
}
opts->port_set = true;
if (parse_port_arg(argv[++i], &opts->port, err, err_size) != 0)
return -1;
} else {
if (arg_has_value(argv[i], "--config", &inline_value)) {
if (!inline_value) {
+55 -20
View File
@@ -2,6 +2,7 @@
#include "data.h"
#include "file.h"
#include "file_receive.h"
#include "identity.h"
#include "log.h"
#include <errno.h>
#include <stdlib.h>
@@ -10,11 +11,15 @@
/* Serialization metadata mode for the batch stream, captured from the config at
* batch_write_header time. The header persists it into the file so a batch is
* self-describing: batch_read_apply re-reads it from the file (not from the
* reading config), so a batch written with -M is applied identically by an
* invoking process regardless of its own -M setting. The batch driver is a
* single sequential scan pass within one thread, so this module-level flag is
* safe. */
* self-describing about whether per-entry metadata was CAPTURED in the stream:
* batch_read_apply re-reads it from the file (not from the reading config) to
* decode the chunk records correctly. Which attributes are actually APPLIED,
* however, comes from the INVOKING process's per-attribute config (the
* FileAttrPolicy and the dir-metadata gate), so a batch written with -M is NOT
* automatically applied identically by an invoking process with a different
* -p/-t/-o/-g: --read-batch must be invoked with the same -p/-t/-o/-g as the
* write side (rsync requires the same options). The batch driver is a single
* sequential scan pass within one thread, so this module-level flag is safe. */
static bool batch_metadata_mode = false;
static bool write_all_bytes(int fd, const void* data, size_t size) {
@@ -91,22 +96,37 @@ int batch_read_apply(int fd, const Config* config, const char* dest_root) {
if (fd < 0 || dest_root == NULL || dest_root[0] == '\0')
return -1;
/* Directory metadata is deferred to the end of the apply (a child write would
* otherwise clobber its parent's mtime/mode). The batch header's single
* metadata bit only says whether metadata is present in the stream; which
* attributes are APPLIED comes from the invoking process's config, so
* --read-batch must be invoked with the same -p/-t/-o/-g as the write side
* (rsync requires the same options). The identity snapshot is activated so
* -o/-g and the explicit ownership flags can apply. */
DirTimeList dir_times;
dir_time_list_init(&dir_times);
int result = -1;
if (!identity_set_active(config)) {
log_message(LOG_LEVEL_ERROR, "batch: could not activate the identity policy");
goto done;
}
char magic[BATCH_MAGIC_LEN];
bool eof = false;
if (!read_exact(fd, magic, BATCH_MAGIC_LEN, &eof) || eof ||
memcmp(magic, BATCH_MAGIC, BATCH_MAGIC_LEN) != 0) {
log_message(LOG_LEVEL_ERROR, "batch: malformed header (bad magic)");
return -1;
goto done;
}
unsigned char version;
if (!read_exact(fd, &version, 1, &eof) || eof || version != BATCH_FORMAT_VERSION) {
log_message(LOG_LEVEL_ERROR, "batch: malformed header (bad or missing format version)");
return -1;
goto done;
}
unsigned char mode;
if (!read_exact(fd, &mode, 1, &eof) || eof || (mode != 0 && mode != 1)) {
log_message(LOG_LEVEL_ERROR, "batch: malformed header (bad metadata flag)");
return -1;
goto done;
}
bool use_metadata = mode == 1;
@@ -114,47 +134,62 @@ int batch_read_apply(int fd, const Config* config, const char* dest_root) {
unsigned long long length;
if (!read_exact(fd, &length, sizeof(length), &eof)) {
log_message(LOG_LEVEL_ERROR, "batch: truncated length prefix");
return -1;
goto done;
}
if (eof)
break; /* clean end of stream */
if (length == 0 || length > BATCH_MAX_RECORD) {
log_message(LOG_LEVEL_ERROR, "batch: rejected record length %llu (valid range 1..%llu)",
length, (unsigned long long)BATCH_MAX_RECORD);
return -1;
goto done;
}
char* record = (char*)malloc((size_t)length);
if (record == NULL) {
log_message(LOG_LEVEL_ERROR, "batch: could not allocate a %llu-byte record", length);
return -1;
goto done;
}
if (!read_exact(fd, record, (size_t)length, &eof) || eof) {
log_message(LOG_LEVEL_ERROR, "batch: truncated chunk record");
free(record);
return -1;
goto done;
}
Data* data = data_create(record, (size_t)length);
if (data == NULL)
return -1; /* data_create frees `record` on failure */
goto done; /* data_create frees `record` on failure */
Chunk* chunk = chunk_deserialize(data, use_metadata);
data_destroy(data);
if (chunk == NULL) {
log_message(LOG_LEVEL_ERROR, "batch: rejected malformed chunk record");
return -1;
goto done;
}
for (int i = 0; i < chunk->element_count; i++) {
File* file = chunk->items[i];
chunk->items[i] = NULL;
if (file == NULL)
continue;
FileSaveResult result = file_save_to_disk_full(dest_root, file, config);
file_destroy(file);
if (result == FILE_SAVE_ERROR) {
FileSaveResult save = file_save_to_disk_full(dest_root, file, config);
/* Accumulate directory metadata (when it applies) before the File is
* destroyed; applied once the whole stream has been consumed. */
if (save != FILE_SAVE_ERROR && file->is_dir && file->metadata &&
dir_metadata_should_capture(config) &&
!dir_time_list_add(&dir_times, file->path, file->metadata, file->xattrs)) {
file_destroy(file);
chunk_destroy(chunk);
return -1;
goto done;
}
file_destroy(file);
if (save == FILE_SAVE_ERROR) {
chunk_destroy(chunk);
goto done;
}
}
chunk_destroy(chunk);
}
return 0;
}
dir_metadata_list_apply(&dir_times, dest_root, config);
result = 0;
done:
identity_clear_active();
dir_time_list_free(&dir_times);
return result;
}
+26 -1
View File
@@ -21,6 +21,20 @@ bool checksum_digest(ChecksumAlgo algo, uint64_t seed, const void* data, size_t
return true;
}
if (algo == CHECKSUM_ALGO_XXH3) {
uint64_t digest = XXH3_64bits_withSeed(data, size, seed);
memcpy(out, &digest, sizeof(digest));
*out_len = sizeof(digest);
return true;
}
if (algo == CHECKSUM_ALGO_XXH128) {
XXH128_hash_t digest = XXH3_128bits_withSeed(data, size, seed);
memcpy(out, &digest, sizeof(digest));
*out_len = sizeof(digest);
return true;
}
if (algo == CHECKSUM_ALGO_MD5) {
/* md5 takes no seed; the caller's seed is deliberately ignored (documented
* in RSYNC_COMPAT.md). OpenSSL's one-shot EVP_Digest needs a non-NULL
@@ -45,6 +59,10 @@ int checksum_algo_from_name(const char* name) {
return -1;
if (strcasecmp(name, "xxh64") == 0 || strcasecmp(name, "xxhash") == 0)
return (int)CHECKSUM_ALGO_XXH64;
if (strcasecmp(name, "xxh3") == 0)
return (int)CHECKSUM_ALGO_XXH3;
if (strcasecmp(name, "xxh128") == 0)
return (int)CHECKSUM_ALGO_XXH128;
if (strcasecmp(name, "md5") == 0)
return (int)CHECKSUM_ALGO_MD5;
return -1;
@@ -54,6 +72,10 @@ const char* checksum_algo_name(ChecksumAlgo algo) {
switch (algo) {
case CHECKSUM_ALGO_XXH64:
return "xxh64";
case CHECKSUM_ALGO_XXH3:
return "xxh3";
case CHECKSUM_ALGO_XXH128:
return "xxh128";
case CHECKSUM_ALGO_MD5:
return "md5";
}
@@ -61,13 +83,16 @@ const char* checksum_algo_name(ChecksumAlgo algo) {
}
bool checksum_algo_valid(int algo) {
return algo == (int)CHECKSUM_ALGO_XXH64 || algo == (int)CHECKSUM_ALGO_MD5;
return algo == (int)CHECKSUM_ALGO_XXH64 || algo == (int)CHECKSUM_ALGO_MD5 ||
algo == (int)CHECKSUM_ALGO_XXH3 || algo == (int)CHECKSUM_ALGO_XXH128;
}
uint8_t checksum_digest_len(ChecksumAlgo algo) {
switch (algo) {
case CHECKSUM_ALGO_XXH64:
case CHECKSUM_ALGO_XXH3:
return 8;
case CHECKSUM_ALGO_XXH128:
case CHECKSUM_ALGO_MD5:
return 16;
}
+15 -6
View File
@@ -9,10 +9,17 @@
* seeded with --checksum-seed. The ids are the values actually placed on the
* wire (config frame), so they must be kept stable and validated on receive.
* CHECKSUM_ALGO_XXH64 == 0 is the default and is byte-for-byte what FastSync
* computed before these options existed (xxHash64 with seed 0). */
typedef enum { CHECKSUM_ALGO_XXH64 = 0, CHECKSUM_ALGO_MD5 = 1 } ChecksumAlgo;
* computed before these options existed (xxHash64 with seed 0). The set mirrors
* the algorithms rsync 3.4.1 can be built with; the ones FastSync does not
* implement (md4, sha1, none) are rejected by name at parse time. */
typedef enum {
CHECKSUM_ALGO_XXH64 = 0,
CHECKSUM_ALGO_MD5 = 1,
CHECKSUM_ALGO_XXH3 = 2,
CHECKSUM_ALGO_XXH128 = 3
} ChecksumAlgo;
/* md5 digest is 16 bytes, the longest supported. */
/* xxh128 digest is 16 bytes, the longest supported. */
#define CHECKSUM_MAX_DIGEST_LEN 16
/* Compute the whole-file digest of the first `size` bytes of `data`.
@@ -29,8 +36,10 @@ bool checksum_digest(ChecksumAlgo algo, uint64_t seed, const void* data, size_t
size_t out_capacity, size_t* out_len);
/* Resolve a --checksum-choice string (case-insensitive) to an algorithm id.
* Accepts "xxh64" and "xxhash" (both map to CHECKSUM_ALGO_XXH64, rsync's
* xxhash spelling) and "md5". Returns -1 for any unsupported name. */
* Accepts "xxh64"/"xxhash", "xxh3", "xxh128" and "md5". "auto", rsync's
* default automatic choice, is resolved to the default by the caller (it is not
* a distinct algorithm here). Returns -1 for any name FastSync does not
* implement (md4/sha1/none included). */
int checksum_algo_from_name(const char* name);
/* Canonical name of an algorithm (used in CLI error messages). */
@@ -39,7 +48,7 @@ const char* checksum_algo_name(ChecksumAlgo algo);
/* True when `algo` is a supported id (used by config receive validation). */
bool checksum_algo_valid(int algo);
/* Digest length in bytes for an algorithm (xxx64 = 8, md5 = 16). */
/* Digest length in bytes for an algorithm (xxh64/xxh3 = 8, md5/xxh128 = 16). */
uint8_t checksum_digest_len(ChecksumAlgo algo);
#endif /* CHECKSUM_H */
+170 -77
View File
@@ -1,90 +1,183 @@
#include "chmod.h"
#include "file.h"
#include <stddef.h>
#include <string.h>
static bool parse_clause(mode_t* mode, const char* begin, const char* end) {
const char* p = begin;
unsigned who = 0;
while (p < end && strchr("ugoa", *p)) {
if (*p == 'a')
who = 7;
else
who |= *p == 'u' ? 1U : (*p == 'g' ? 2U : 4U);
p++;
}
if (who == 0)
who = 7;
if (p == end || (*p != '+' && *p != '-' && *p != '='))
return false;
char operation = *p++;
mode_t bits = 0;
while (p < end) {
mode_t bit;
switch (*p++) {
case 'r':
bit = 4;
break;
case 'w':
bit = 2;
break;
case 'x':
bit = 1;
break;
default:
return false;
}
bits |= bit;
}
for (unsigned class_index = 0; class_index < 3; class_index++) {
unsigned class_bit = 1U << class_index;
if (!(who & class_bit))
continue;
mode_t shift = (mode_t)((2U - class_index) * 3U);
mode_t mask = (mode_t)(7U << shift);
mode_t class_bits = (mode_t)(bits << shift);
if (operation == '+')
*mode |= class_bits;
else if (operation == '-')
*mode &= ~class_bits;
else
*mode = (*mode & ~mask) | class_bits;
}
return true;
}
/* rsync's --chmod parser (parse_chmod + tweak_mode). A single clause is
* applied as it is completed, so repeated clauses and repeated --chmod options
* (joined with commas by the CLI) accumulate exactly like rsync. The D/F
* selectors restrict a clause to directories/files; X adds execute only to
* directories or files that were already executable. */
#define CHMOD_BITS 07777
#define CHMOD_FLAG_X_KEEP (1U << 0)
#define CHMOD_FLAG_DIRS_ONLY (1U << 1)
#define CHMOD_FLAG_FILES_ONLY (1U << 2)
enum chmod_op { CHMOD_OP_ADD = 1, CHMOD_OP_SUB, CHMOD_OP_EQ, CHMOD_OP_SET };
enum chmod_state {
CHMOD_STATE_ERROR,
CHMOD_STATE_1ST_HALF,
CHMOD_STATE_2ND_HALF,
CHMOD_STATE_OCTAL
};
bool chmod_apply(mode_t mode, const char* spec, mode_t* result) {
if (!spec || !*spec || !result)
return false;
bool numeric = true;
size_t length = strlen(spec);
if (length > 4)
numeric = false;
for (size_t i = 0; i < length && numeric; i++)
numeric = spec[i] >= '0' && spec[i] <= '7';
if (numeric) {
if (length == 0 || length > 4)
return false;
mode_t parsed = 0;
for (size_t i = 0; i < length; i++)
parsed = (mode_t)((parsed << 3) | (spec[i] - '0'));
*result = parsed;
return true;
}
const mode_t nonperm = mode & ~(mode_t)CHMOD_BITS;
const bool initially_executable = (mode & 0111) != 0;
mode_t changed = mode;
const char* begin = spec;
while (*begin) {
const char* end = strchr(begin, ',');
if (!end)
end = begin + strlen(begin);
if (!parse_clause(&changed, begin, end))
return false;
if (*end == '\0')
int state = CHMOD_STATE_1ST_HALF;
unsigned where = 0;
int what = 0, op = 0, topbits = 0, topoct = 0, flags = 0;
const char* p = spec;
while (state != CHMOD_STATE_ERROR) {
if (*p == '\0' || *p == ',') {
int bits;
if (!op) {
state = CHMOD_STATE_ERROR;
break;
}
if (where)
bits = (int)(where * (unsigned)what);
else {
where = 0111;
bits = (int)((where * (unsigned)what) & ~(unsigned)file_process_umask());
}
int mode_and, mode_or;
switch (op) {
case CHMOD_OP_ADD:
mode_and = CHMOD_BITS;
mode_or = bits + topoct;
break;
case CHMOD_OP_SUB:
mode_and = CHMOD_BITS - bits - topoct;
mode_or = 0;
break;
case CHMOD_OP_EQ:
mode_and = CHMOD_BITS - (int)(where * 7U) - (topoct ? topbits : 0);
mode_or = bits + topoct;
break;
default:
mode_and = 0;
mode_or = bits;
break;
}
bool is_dir = S_ISDIR(nonperm);
if (!((flags & CHMOD_FLAG_DIRS_ONLY) && !is_dir) &&
!((flags & CHMOD_FLAG_FILES_ONLY) && is_dir)) {
changed &= (mode_t)mode_and;
if ((flags & CHMOD_FLAG_X_KEEP) && !initially_executable && !is_dir)
changed |= (mode_t)(mode_or & ~0111);
else
changed |= (mode_t)mode_or;
}
if (*p == '\0')
break;
p++;
state = CHMOD_STATE_1ST_HALF;
where = 0;
what = op = topoct = topbits = flags = 0;
continue;
}
switch (state) {
case CHMOD_STATE_1ST_HALF:
switch (*p) {
case 'D':
if (flags & CHMOD_FLAG_FILES_ONLY) {
state = CHMOD_STATE_ERROR;
break;
}
flags |= CHMOD_FLAG_DIRS_ONLY;
break;
case 'F':
if (flags & CHMOD_FLAG_DIRS_ONLY) {
state = CHMOD_STATE_ERROR;
break;
}
flags |= CHMOD_FLAG_FILES_ONLY;
break;
case 'u':
where |= 0100;
topbits |= 04000;
break;
case 'g':
where |= 0010;
topbits |= 02000;
break;
case 'o':
where |= 0001;
break;
case 'a':
where |= 0111;
break;
case '+':
op = CHMOD_OP_ADD;
state = CHMOD_STATE_2ND_HALF;
break;
case '-':
op = CHMOD_OP_SUB;
state = CHMOD_STATE_2ND_HALF;
break;
case '=':
op = CHMOD_OP_EQ;
state = CHMOD_STATE_2ND_HALF;
break;
default:
if (*p >= '0' && *p <= '7' && !where) {
op = CHMOD_OP_SET;
state = CHMOD_STATE_OCTAL;
where = 1;
what = *p - '0';
} else {
state = CHMOD_STATE_ERROR;
}
break;
}
break;
begin = end + 1;
if (!*begin)
return false;
case CHMOD_STATE_2ND_HALF:
switch (*p) {
case 'r':
what |= 4;
break;
case 'w':
what |= 2;
break;
case 'X':
flags |= CHMOD_FLAG_X_KEEP;
/* fall through */
case 'x':
what |= 1;
break;
case 's':
if (topbits)
topoct |= topbits;
else
topoct = 04000;
break;
case 't':
topoct |= 01000;
break;
default:
state = CHMOD_STATE_ERROR;
break;
}
break;
default:
if (*p >= '0' && *p <= '7') {
what = what * 8 + (*p - '0');
if (what > CHMOD_BITS)
state = CHMOD_STATE_ERROR;
} else {
state = CHMOD_STATE_ERROR;
}
break;
}
p++;
}
*result = changed;
if (state == CHMOD_STATE_ERROR)
return false;
*result = (changed & (mode_t)CHMOD_BITS) | nonperm;
return true;
}
+4 -1
View File
@@ -4,7 +4,10 @@
#include <stdbool.h>
#include <sys/stat.h>
/* Apply the supported rsync --chmod syntax to a permission mode. */
/* Apply rsync's --chmod syntax to a permission mode, including the D/F/X
* selectors and the s/t special bits. `mode` should carry the file type bits
* (S_IFDIR/S_IFREG) so D/F/X can be evaluated; the type bits are preserved in
* `result`. A spec may contain comma-separated clauses, which accumulate. */
bool chmod_apply(mode_t mode, const char* spec, mode_t* result);
#endif
+41 -10
View File
@@ -14,23 +14,54 @@
#define INITIAL_DECOMPRESS_BUF_SIZE (1024 * 1024)
#define MAX_DECOMPRESSED_SIZE (100ULL * 1024 * 1024) /* 100 MB hard ceiling */
static char* SKIP_COMPRESSION_EXTENSIONS[] = {".jpg", ".jpeg", ".png", ".gif", ".mp4", ".mkv",
".zip", ".gz", ".xz", ".zst", NULL};
/* rsync 3.4.1's built-in skip-compress suffix list (the `--skip-compress`
* defaults, in the man page's order). rsync stores it as space-separated
* "*.suffix" globs; FastSync matches the plain suffix after the final dot, so
* the leading "*." is omitted here. A user --skip-compress list replaces this
* default entirely (matching rsync). */
#define DEFAULT_SKIP_COMPRESS_SUFFIXES \
"3g2 3gp 7z aac ace apk avi bz2 deb dmg ear f4v flac flv gpg gz iso jar jpeg jpg lrz lz lz4 " \
"lzma " \
"lzo m1a m1v m2a m2ts m2v m4a m4b m4p m4r m4v mka mkv mov mp1 mp2 mp3 mp4 mpa mpeg mpg mpv mts " \
"odb odf odg odi odm odp ods odt oga ogg ogm ogv ogx opus otg oth otp ots ott oxt png qt rar " \
"rpm " \
"rz rzip spx squashfs sxc sxd sxg sxm sxw sz tbz tbz2 tgz tlz ts txz tzo vob war webm webp xz " \
"z " \
"zip zst"
/* Case-insensitive match of a bare suffix (no leading dot) against a
* space-separated suffix list. */
static bool suffix_in_list(const char* name, const char* list) {
size_t name_len = strlen(name);
while (*list) {
while (*list == ' ')
list++;
const char* start = list;
while (*list && *list != ' ')
list++;
size_t len = (size_t)(list - start);
if (len == name_len && strncasecmp(name, start, len) == 0)
return true;
}
return false;
}
bool compression_should_skip_with_suffixes(const char* path, char* const* suffixes, int count) {
if (!path)
return false;
const char* dot = strrchr(path, '.');
if (!dot)
if (!dot || dot[1] == '\0')
return false;
if (count < 0) {
suffixes = SKIP_COMPRESSION_EXTENSIONS;
count = 0;
while (SKIP_COMPRESSION_EXTENSIONS[count])
count++;
}
const char* name = dot + 1;
/* count < 0 (the user gave no --skip-compress) selects rsync's built-in
* default list; a non-negative count is the user's explicit list. */
if (count < 0)
return suffix_in_list(name, DEFAULT_SKIP_COMPRESS_SUFFIXES);
for (int i = 0; i < count; i++) {
if (strcasecmp(dot, suffixes[i]) == 0)
const char* suffix = suffixes[i];
if (suffix[0] == '.')
suffix++;
if (strcasecmp(name, suffix) == 0)
return true;
}
return false;
+86 -21
View File
@@ -20,6 +20,8 @@
static void config_set_defaults(Config* config) {
config->scanner_threads = 0;
config->metadata_explicitly_disabled = false;
config->preserve_perms_explicit_off = false;
config->preserve_times_explicit_off = false;
config->show_progress = false;
config->compression_threads = 0;
config->ssh_port = 22;
@@ -43,12 +45,14 @@ static void config_set_defaults(Config* config) {
config->server_port = 8080;
config->server_port_set = false;
config->server_host_set = false;
/* 0 means "--timeout not given": the transport keeps its own built-in 30 s
* socket timeout (tcp_set_timeouts ignores non-positive values) and the
* protocol layer keeps its built-in 60 s per-message deadline. A positive
* value overrides BOTH (see protocol_session_set_io_timeout). */
/* rsync defaults: --timeout=0 (I/O timeouts disabled) and --contimeout=60.
* A value of 0 disables the client's own deadline on both the socket layer
* (tcp_set_timeouts) and the protocol layer
* (protocol_session_set_io_timeout); a positive value sets it. A server
* session floors the deadline at SERVER_IO_TIMEOUT_SEC so 0 can never hold a
* connection open forever. */
config->timeout = 0;
config->contimeout = 10;
config->contimeout = 60;
config->quiet = false;
config->stats = false;
config->max_depth = 0;
@@ -194,7 +198,9 @@ static bool validate_received_config(const Config* config) {
valid_wire_bool(config->dry_run) && checksum_algo_valid(config->checksum_algo) &&
identity_wire_valid(config) && valid_wire_bool(config->preserve_atimes) &&
valid_wire_bool(config->preserve_crtimes) && valid_wire_bool(config->omit_dir_times) &&
valid_wire_bool(config->omit_link_times) && valid_wire_bool(config->munge_links) &&
valid_wire_bool(config->omit_link_times) && valid_wire_bool(config->preserve_perms) &&
valid_wire_bool(config->preserve_times) && valid_wire_bool(config->preserve_owner) &&
valid_wire_bool(config->preserve_group) && valid_wire_bool(config->munge_links) &&
valid_wire_bool(config->keep_dirlinks) && valid_wire_bool(config->fake_super) &&
(!config->copy_as_set || (config->copy_as_uid >= 0 && config->copy_as_gid >= 0)) &&
(!config->use_compression ||
@@ -203,8 +209,9 @@ static bool validate_received_config(const Config* config) {
config->delta_block_size >= DELTA_BLOCK_SIZE_MIN &&
config->delta_block_size <= DELTA_BLOCK_SIZE_MAX &&
config->delta_max_file_size <= DELTA_MAX_FILE_SIZE && config->modify_window >= 0 &&
config->max_delete >= -1 && config->skip_compress_count >= 0 &&
config->skip_compress_count <= MAX_SKIP_COMPRESS_SUFFIXES && config->max_alloc > 0 &&
config->max_delete >= -1 && config->max_alloc <= MAX_SERVER_ALLOC &&
config->skip_compress_count >= 0 &&
config->skip_compress_count <= MAX_SKIP_COMPRESS_SUFFIXES &&
(!config->chmod_spec || !*config->chmod_spec ||
chmod_apply(0, config->chmod_spec, &(mode_t){0})) &&
config->super_mode >= SUPER_MODE_AUTO && config->super_mode <= SUPER_MODE_OFF;
@@ -292,11 +299,31 @@ const char* config_invariants_error(const Config* config) {
"timing; at most one may be given and each implies --delete";
if (config->iconv_spec && !charset_spec_valid(config->iconv_spec))
return "--iconv requires LOCAL[,REMOTE] charset names supported by iconv";
if ((config->preserve_perms || config->preserve_times || config->preserve_owner ||
config->preserve_group || config->preserve_atimes || config->preserve_crtimes ||
config->use_executability) &&
!config->use_metadata)
return "a preservation attribute requires metadata transmission";
if (config->copy_as_set && !config->use_metadata)
return "--copy-as requires metadata preservation and cannot be combined with --no-preserve";
return NULL;
}
bool config_derived_use_metadata(const Config* config) {
if (!config)
return false;
if (config->preserve_perms || config->preserve_times || config->preserve_owner ||
config->preserve_group || config->preserve_atimes || config->preserve_crtimes ||
config->use_executability || config->preserve_xattrs || config->preserve_acls ||
config->fake_super || config->preserve_devices || config->preserve_specials ||
config->copy_devices || config->write_devices ||
(config->chmod_spec && config->chmod_spec[0]) || config->copy_as_set ||
config->chown_uid_set || config->chown_gid_set || config->usermap_count > 0 ||
config->groupmap_count > 0 || config->update)
return true;
return (config->use_incremental || config->use_delta) && !config->metadata_explicitly_disabled;
}
bool config_has_basis(const Config* config) {
return config && config->basis_count > 0;
}
@@ -725,10 +752,18 @@ void config_delete(Config* config) {
free(config->skip_compress_suffixes[i]);
free(config->skip_compress_suffixes);
}
free(config->usermap);
if (config->usermap) {
for (int i = 0; i < config->usermap_count; i++)
free(config->usermap[i].to_name);
free(config->usermap);
}
config->usermap = NULL;
config->usermap_count = 0;
free(config->groupmap);
if (config->groupmap) {
for (int i = 0; i < config->groupmap_count; i++)
free(config->groupmap[i].to_name);
free(config->groupmap);
}
config->groupmap = NULL;
config->groupmap_count = 0;
if (config->filters) {
@@ -756,11 +791,14 @@ void config_delete(Config* config) {
* ------------------------------------------------------------------------- */
/* --max-alloc: raw 64-bit value, clamped server-side and installed as the
* session allocation ceiling. A zero value is rejected. */
* session allocation ceiling. A received 0 is rsync's "no alloc limit"; on the
* receive path it is mapped to the server ceiling so a client can never disable
* it (client-side 0 remains unlimited). Any value above the ceiling is clamped
* to it. */
static bool config_receive_max_alloc(int fd, unsigned long long* value) {
if (!receive_n_data(fd, value, sizeof(*value)) || *value == 0)
if (!receive_n_data(fd, value, sizeof(*value)))
return false;
if (*value > MAX_SERVER_ALLOC)
if (*value == 0 || *value > MAX_SERVER_ALLOC)
*value = MAX_SERVER_ALLOC;
protocol_session_set_max_alloc(NULL, *value);
return true;
@@ -960,7 +998,8 @@ static bool receive_basis_entries(int fd, Config* c, ConfigStringBudget* budget)
static bool send_identity_entries(int fd, const IdentityMap* map, int count) {
for (int i = 0; i < count; i++) {
if (!send_int(fd, map[i].from) || !send_int(fd, map[i].to))
if (!send_int(fd, map[i].from) || !send_int(fd, map[i].from_hi) || !send_int(fd, map[i].to) ||
!send_str(fd, map[i].to_name ? map[i].to_name : ""))
return false;
}
return true;
@@ -968,20 +1007,32 @@ static bool send_identity_entries(int fd, const IdentityMap* map, int count) {
static bool receive_identity_entries(int fd, ConfigStringBudget* budget, int count,
IdentityMap** out) {
(void)budget;
if (count <= 0)
return true;
IdentityMap* map = calloc((size_t)count, sizeof(IdentityMap));
if (!map)
return false;
for (int i = 0; i < count; i++) {
if (!receive_int(fd, &map[i].from) || !receive_int(fd, &map[i].to)) {
free(map);
return false;
if (!receive_int(fd, &map[i].from) || !receive_int(fd, &map[i].from_hi) ||
!receive_int(fd, &map[i].to))
goto fail;
char* name = config_receive_str(fd, budget);
if (!name)
goto fail;
if (name[0] == '\0') {
free(name);
map[i].to_name = NULL;
} else {
map[i].to_name = name;
}
}
*out = map;
return true;
fail:
for (int i = 0; i < count; i++)
free(map[i].to_name);
free(map);
return false;
}
/* ---------------------------------------------------------------------------
@@ -1104,6 +1155,7 @@ CONFIG_DEFINE_SEND(send_daemon_auth, CONFIG_WIRE_DAEMON_AUTH_FIELDS)
CONFIG_DEFINE_SEND(send_iconv_spec, CONFIG_WIRE_ICONV_FIELDS)
CONFIG_DEFINE_SEND(send_privilege_options, CONFIG_WIRE_PRIVILEGE_FIELDS)
CONFIG_DEFINE_SEND(send_copy_as_options, CONFIG_WIRE_COPY_AS_FIELDS)
CONFIG_DEFINE_SEND(send_output_options, CONFIG_WIRE_OUTPUT_FIELDS)
CONFIG_DEFINE_RECV(receive_core_fields, CONFIG_WIRE_CORE_FIELDS)
CONFIG_DEFINE_RECV(receive_delta_fields, CONFIG_WIRE_DELTA_FIELDS)
@@ -1122,6 +1174,7 @@ CONFIG_DEFINE_RECV(receive_daemon_auth, CONFIG_WIRE_DAEMON_AUTH_FIELDS)
CONFIG_DEFINE_RECV(receive_iconv_spec, CONFIG_WIRE_ICONV_FIELDS)
CONFIG_DEFINE_RECV(receive_privilege_options, CONFIG_WIRE_PRIVILEGE_FIELDS)
CONFIG_DEFINE_RECV(receive_copy_as_options, CONFIG_WIRE_COPY_AS_FIELDS)
CONFIG_DEFINE_RECV(receive_output_options, CONFIG_WIRE_OUTPUT_FIELDS)
#undef XSEND
#undef XRECV
@@ -1238,7 +1291,8 @@ bool config_send_wire_block(int file_descriptor, const Config* config) {
send_daemon_module(file_descriptor, config) && send_daemon_auth(file_descriptor, config) &&
send_iconv_spec(file_descriptor, config) &&
send_privilege_options(file_descriptor, config) &&
send_copy_as_options(file_descriptor, config);
send_copy_as_options(file_descriptor, config) &&
send_output_options(file_descriptor, config);
}
bool config_send(int file_descriptor, const Config* config) {
@@ -1308,10 +1362,12 @@ Config* config_receive_with_validate(int file_descriptor, ConfigValidateFunc val
!receive_daemon_auth(file_descriptor, config, &budget) ||
!receive_iconv_spec(file_descriptor, config, &budget) ||
!receive_privilege_options(file_descriptor, config, &budget) ||
!receive_copy_as_options(file_descriptor, config, &budget))
!receive_copy_as_options(file_descriptor, config, &budget) ||
!receive_output_options(file_descriptor, config, &budget))
goto error;
if (config->compress_choice[0] != '\0' && strcmp(config->compress_choice, "zstd") != 0 &&
strcmp(config->compress_choice, "none") != 0) {
strcmp(config->compress_choice, "none") != 0 &&
strcmp(config->compress_choice, "auto") != 0) {
char* escaped_choice = output_escape(config->compress_choice, config->eight_bit_output);
log_message(LOG_LEVEL_ERROR, "Unsupported compression choice: %s",
escaped_choice ? escaped_choice : "<allocation failed>");
@@ -1322,6 +1378,15 @@ Config* config_receive_with_validate(int file_descriptor, ConfigValidateFunc val
free(escaped_choice);
goto error;
}
/* Defensive: an older/hostile client may still send "auto"; canonicalize it
to zstd (its effective choice) so the stored value is always concrete. */
if (strcmp(config->compress_choice, "auto") == 0) {
char* canonical = str_dup("zstd");
if (!canonical)
goto error;
free(config->compress_choice);
config->compress_choice = canonical;
}
if (!validate_received_config(config)) {
log_message(LOG_LEVEL_ERROR, "Invalid configuration received from client");
send_error_detail(file_descriptor, "invalid configuration received from client");
+141 -23
View File
@@ -39,15 +39,20 @@ typedef struct BasisDest {
char* path; /* relative to the destination root (receiver-confined) */
} BasisDest;
/* One resolved FROM:TO identity-mapping rule (--usermap / --groupmap). Both
* fields are numeric ids. IDENTITY_MATCH_ANY (-1) in `from` is rsync's '*'
* wildcard (matches any transmitted id); IDENTITY_CURRENT (-1) in `to` makes
* the receiver resolve the receiving process's own current euid/egid at apply
* time. Names are resolved to numbers at parse time on the client (see
* identity.h for the exact subset). */
/* One FROM:TO identity-mapping rule (--usermap / --groupmap). `from`/`from_hi`
* describe the sender-side FROM matcher (a single id when from_hi == from, an
* inclusive LOW-HIGH range, IDENTITY_MATCH_ANY for rsync's '*', or
* IDENTITY_MATCH_UNNAMED for rsync's empty FROM). `to` is the receiver-side TO
* numeric id (IDENTITY_CURRENT = the receiving process's own euid/egid) UNLESS
* `to_name` is non-NULL, in which case the receiver resolves the name against
* its own account database at apply time (rsync resolves TO names on the
* receiver) and `to` is ignored. FROM names/ranges/globs are resolved on the
* client (the sender) exactly as rsync matches them against sender names. */
typedef struct {
int32_t from;
int32_t from_hi;
int32_t to;
char* to_name;
} IdentityMap;
/* --sockopts=OPTIONS allowlist. Only these option names are accepted; anything
@@ -76,7 +81,7 @@ typedef struct {
typedef enum SuperMode { SUPER_MODE_AUTO = 0, SUPER_MODE_ON = 1, SUPER_MODE_OFF = 2 } SuperMode;
/* ===========================================================================
* Config wire-field table (single source of truth for protocol 2.21.0).
* Config wire-field table (single source of truth for protocol 2.23.0).
*
* Every field below crosses the wire. The table is the ONLY place a
* serialized field is named: config.h expands CONFIG_WIRE_FIELDS() to declare
@@ -216,7 +221,11 @@ typedef enum SuperMode { SUPER_MODE_AUTO = 0, SUPER_MODE_ON = 1, SUPER_MODE_OFF
X(preserve_atimes, bool, false, BOOL) \
X(preserve_crtimes, bool, false, BOOL) \
X(omit_dir_times, bool, false, BOOL) \
X(omit_link_times, bool, false, BOOL)
X(omit_link_times, bool, false, BOOL) \
X(preserve_perms, bool, false, BOOL) \
X(preserve_times, bool, false, BOOL) \
X(preserve_owner, bool, false, BOOL) \
X(preserve_group, bool, false, BOOL)
#define CONFIG_WIRE_SYMLINK_TRUST_FIELDS(X) \
X(munge_links, bool, false, BOOL) \
@@ -237,6 +246,13 @@ typedef enum SuperMode { SUPER_MODE_AUTO = 0, SUPER_MODE_ON = 1, SUPER_MODE_OFF
X(copy_as_uid, int32_t, 0, COPY_AS_ID) \
X(copy_as_gid, int32_t, 0, COPY_AS_ID)
/* Output-parity wave (protocol 2.23.0). report_dest_info tells the receiver to
* answer every per-file STATUS_CHECK with a STATUS_DEST_INFO snapshot of the
* pre-transfer destination entry (see protocol.h). It is set by the client
* only when -i/--itemize-changes or --out-format asks for per-file change
* output; the transfer decision itself is unchanged. */
#define CONFIG_WIRE_OUTPUT_FIELDS(X) X(report_dest_info, bool, false, BOOL)
/* All serialized fields, in exact wire order. Concatenating the per-segment
* lists here is what keeps the declaration order = the wire order. */
#define CONFIG_WIRE_FIELDS(X) \
@@ -257,7 +273,8 @@ typedef enum SuperMode { SUPER_MODE_AUTO = 0, SUPER_MODE_ON = 1, SUPER_MODE_OFF
CONFIG_WIRE_DAEMON_AUTH_FIELDS(X) \
CONFIG_WIRE_ICONV_FIELDS(X) \
CONFIG_WIRE_PRIVILEGE_FIELDS(X) \
CONFIG_WIRE_COPY_AS_FIELDS(X)
CONFIG_WIRE_COPY_AS_FIELDS(X) \
CONFIG_WIRE_OUTPUT_FIELDS(X)
typedef struct Config {
/* -j/--threads=N: number of parallel scanner worker threads for the -m
@@ -266,6 +283,15 @@ typedef struct Config {
* concern and is NEVER serialized into the wire config frame. */
int scanner_threads;
bool metadata_explicitly_disabled;
/* CLIENT-ONLY (never serialized; not in CONFIG_WIRE_FIELDS). Set when the
* user explicitly turned an attribute off with --no-perms / --no-times (long
* or short form). --incremental/--delta historically auto-enabled mode and
* mtime preservation; these flags let cli_finalize_config restore that
* behavior while still honoring the explicit per-attribute negation. A
* later -p/-t re-enables the attribute directly, so the flag only prevents
* the incremental/delta implication, never a POSITIVE request. */
bool preserve_perms_explicit_off;
bool preserve_times_explicit_off;
bool show_progress;
int compression_threads;
int ssh_port;
@@ -301,12 +327,14 @@ typedef struct Config {
char* tls_cert;
char* tls_key;
char* tls_ca;
/* --timeout: per-message I/O deadline in seconds. 0 (the default/unset
* sentinel) leaves the transport's built-in 30 s socket timeout and the
* protocol's built-in 60 s per-message deadline in place; a positive value
* overrides both. See protocol_session_set_io_timeout. */
/* --timeout: per-message I/O deadline in seconds. 0 (rsync's default)
* disables the deadline entirely on the client's own socket and protocol
* layers; a positive value sets it. A server session never inherits the
* disabled value: it applies the SERVER_IO_TIMEOUT_SEC floor (see
* protocol_server_io_timeout_sec and tcp_set_timeouts). */
int timeout;
/* --contimeout: connect()/accept timeout, transport layer only. */
/* --contimeout: connect()/accept timeout in seconds (rsync's default 60);
* 0 disables it. Transport layer only. */
int contimeout;
bool quiet;
bool stats;
@@ -563,13 +591,17 @@ typedef struct Config {
* targets and, with -K, follows an in-root destination symlink-to-directory);
* -k/--copy-dirlinks is sender-only and is never serialized. */
/* numeric_ids */
/* --numeric-ids: no name lookup, use the transmitted numeric ids raw. */
/* --numeric-ids: a mapping MODIFIER only -- no name lookup, use the
* transmitted numeric ids raw. It does NOT by itself request ownership. */
/* chown_uid_set */
/* --chown USER (owner) override; IDENTITY_CURRENT = the receiver's euid. */
/* chown_gid_set */
/* --chown :GROUP (group) override; IDENTITY_CURRENT = the receiver's egid. */
/* usermap */
/* --usermap / --groupmap entries, in order (first match wins). */
/* --usermap / --groupmap entries, in order (first match wins). Each entry's
* from/from_hi are a single id, an inclusive range, IDENTITY_MATCH_ANY ('*'),
* or IDENTITY_MATCH_UNNAMED (empty FROM); to_name carries a receiver-resolved
* TO name (rsync resolves TO names on the receiving side). */
/* preserve_atimes */
/* -U/--atimes: preserve source access times on the destination. */
/* preserve_crtimes */
@@ -579,10 +611,29 @@ typedef struct Config {
/* -O/--omit-dir-times: do not apply mtimes to directories. */
/* omit_link_times */
/* -J/--omit-link-times: do not apply times to symlinks. */
/* preserve_perms */
/* -p/--perms: preserve the source permission bits (mode). One of the four
* per-attribute preservation flags split out of the former single
* use_metadata bundle; --chmod and -A/--acls also imply it. */
/* preserve_times */
/* -t/--times: preserve source modification times. Split out of the former
* use_metadata bundle; --preserve and -a/--archive imply it. */
/* preserve_owner */
/* -o/--owner: preserve the source owner (uid). Split out of the former
* use_metadata bundle; --usermap/--chown (and, when a uid is requested,
* --copy-as) imply it. Owner application still requires receiver privilege
* and is gated separately by the identity flags. */
/* preserve_group */
/* -g/--group: preserve the source group (gid). Split out of the former
* use_metadata bundle; --groupmap/--chown (and, when a gid is requested,
* --copy-as) imply it. */
/* fake_super */
/* --fake-super: receiver-only. When set, each written file additionally gets
* a reserved user.fastsync.stat xattr recording the source uid/gid/mode/mtime
* so a later privileged restore could re-apply them. Crosses the wire. */
* a reserved user.fastsync.stat xattr recording the RESOLVED uid/gid (the
* source's own when no ownership request is active, else the --chown/--usermap
* result) plus mode/mtime so a later privileged restore could re-apply them.
* It NEVER real-chowns: the point is to record the source ownership on an
* unprivileged receiver. Crosses the wire. */
/* module */
/* Daemon module selection (Wave A, protocol 2.15.0). Client-composed from a
* host::module/path destination; NULL or "" means "no module" (the ordinary
@@ -623,7 +674,7 @@ typedef struct Config {
* fd-relative confinement (file_open_secure_parent, O_NOFOLLOW, root checks);
* --super only permits an attempt that is already confined. Crosses the wire
* as a trailing int so the receiver can enforce the policy. See
* privilege_super_permitted() and identity_ownership_requested() in
* privilege_super_permitted() and identity_explicit_ownership_requested() in
* identity.h. */
/* copy_as_set */
/* --copy-as=USER[:GROUP] (P7 Wave E, protocol 2.18.0). Safe-subset
@@ -803,8 +854,65 @@ typedef struct Config {
* unknown status, or the unconsumed detail body, and the strict same-version
* handshake (config_receive rejects a mismatched version before parsing
* anything else) is what keeps a 2.21 client and a 2.20 server from ever
* reaching that state. */
#define PROTOCOL_VERSION "2.21.0"
* reaching that state.
*
* Preserve-Attribute Split Wave: 2.21.0 -> 2.22.0.
*
* WHY the bump, grounded in the wire: this wave splits the former single
* use_metadata bundle into four independent rsync-compatible preservation
* attributes (preserve_perms / preserve_times / preserve_owner /
* preserve_group) so -p/-t/-o/-g (and their --no-* negations) become real
* drop-in flags. The binary config frame gains four serialized bools appended
* to CONFIG_WIRE_METADATA_TIMES_FIELDS after omit_link_times, in this fixed
* order: preserve_perms, preserve_times, preserve_owner, preserve_group. Any
* config-frame layout change must bump the protocol version: a peer that does
* not parse the new trailing bytes would desynchronize on the frame boundary,
* and the strict same-version handshake (config_receive rejects a mismatched
* version before parsing anything else) is what keeps a 2.22 client and a 2.21
* server from ever reaching that state. The fixed-width FileMetadata layout is
* UNCHANGED: the receiver still gates attribute application on use_metadata,
* which is now DERIVED from these attributes by config_derived_use_metadata().
*
* Rsync-Parity Wave: 2.22.0 -> 2.23.0.
*
* WHY the bump, grounded in the wire. Several independent changes land in this
* protocol version:
*
* (1) Ownership parity (#286/#294): each --usermap/--groupmap wire entry grows
* from two int32s to [from][from_hi][to][to_name]; `from_hi` carries an
* inclusive LOW-HIGH range (== from for a single/any/unnamed matcher) and the
* trailing string carries a TO NAME for the receiver to resolve (rsync resolves
* TO names on the receiving side). The STATUS_MKDIR and STATUS_DIR_TIMES frames
* also gain a bounded per-entry xattr block when -X/-A is negotiated, so
* directory xattrs/ACLs (including default ACLs) are preserved like regular-file
* xattrs.
*
* (2) Delete semantics (#290): the delete-manifest frame gains a fourth trailing
* section -- a synchronized-directory count followed by that many
* destination-relative directory paths (the receive root is "."). The receiver
* confines its extras walk to these directories, so `--files-from` with
* `--delete` only removes inside listed directory subtrees (rsync parity)
* instead of deleting every untransmitted path under the receive root. The
* frame stream also gains STATUS_DELETE_LIMIT, the terminal success status sent
* instead of STATUS_OK when a --max-delete commit removes up to the bound and
* skips the rest (the sender then exits 25 like rsync).
*
* Any config-frame layout or frame-sequence change must bump the protocol
* version: a 2.22 peer would desynchronize on the new entry bytes, the extra
* trailing section or the unknown status, and the strict same-version handshake
* (config_receive rejects a mismatched version before parsing anything else) is
* what keeps a 2.23 client and a 2.22 server from ever reaching that state.
*
* (3) Output parity (#291/#292): -i/--itemize-changes and --out-format must
* compare the source against the PRE-TRANSFER destination entry (new vs
* modified, and which of size/time/perms/owner/group differ), but FastSync's
* push sender never sees the destination. The receiver therefore answers a
* per-file STATUS_CHECK with a new STATUS_DEST_INFO frame (a fixed-width
* snapshot of the old entry) before its ordinary verdict when the config frame
* carries the new report_dest_info bool appended after the --copy-as block.
* This is both a config-frame layout change (one trailing bool) and a frame
* sequence change (the new status). */
#define PROTOCOL_VERSION "2.23.0"
#define DEFAULT_CHUNK_SIZE (10 * 1024 * 1024)
/* Upper bound on total basis-dir entries (rsync caps --link-dest at 20). */
#define MAX_BASIS_DIRS 64
@@ -829,9 +937,11 @@ typedef struct Config {
/* Identity-mapping sentinels and bounds (see identity.h for semantics).
* IDENTITY_MATCH_ANY is a usermap/groupmap FROM '*' (matches any id);
* IDENTITY_CURRENT is a chown / map TO '*' (resolve to the receiver's current
* euid/egid at apply time). */
* IDENTITY_MATCH_UNNAMED is a FROM with an empty token (rsync's "ids with no
* name on the sender"); IDENTITY_CURRENT is a chown / map TO '*' (resolve to
* the receiver's current euid/egid at apply time). */
#define IDENTITY_MATCH_ANY (-1)
#define IDENTITY_MATCH_UNNAMED (-2)
#define IDENTITY_CURRENT (-1)
#define MAX_IDENTITY_MAP 128
@@ -918,6 +1028,14 @@ bool config_has_valid_delete_timing(const Config* config);
* validate_received_config() so the receiver enforces exactly the same
* invariants it relies on (the server is the trust boundary). */
const char* config_invariants_error(const Config* config);
/* Single source of truth for the DERIVED transport bit (use_metadata): true
* when any configured preservation/ownership option requires the metadata
* frame to travel. Returns false when no such option is set (a bare run).
* This is a pure predicate over the config; the client lowers it into
* Config->use_metadata at the end of parsing so every implication (devices,
* executability, identity maps, incremental/delta, ...) is centralized here
* rather than scattered as direct writes. */
bool config_derived_use_metadata(const Config* config);
/* True when at least one --compare-dest/--copy-dest/--link-dest was set. */
bool config_has_basis(const Config* config);
/* Append one basis-dir entry. Returns 0 on success, -1 on allocation failure. */
+14
View File
@@ -264,6 +264,20 @@ static bool delay_publish_entry(DelayUpdatesContext* context, const Config* conf
const StagedFileEntry* entry) {
if (!delay_publish_backup(context, config, entry))
return false;
/* --force: an incoming regular file/symlink may replace a destination
DIRECTORY (possibly non-empty). The immediate-install path handles this in
file_receive; a --delay-updates run stages elsewhere and only discovers the
blocking directory here, so clear it before the rename (rsync's
"could not make way for new regular file" without --force). */
if (config && config->force_delete && file_directory_exists_secure(entry->final_path)) {
if (!file_remove_tree_secure(entry->final_path)) {
char* escaped = output_escape(entry->final_path, false);
log_message(LOG_LEVEL_ERROR, "could not remove destination directory blocking '%s': %s",
escaped ? escaped : "<allocation failed>", strerror(errno));
free(escaped);
return false;
}
}
if (!file_rename_secure(entry->staged_path, entry->final_path)) {
if (errno == EXDEV) {
char* escaped = output_escape(entry->final_path, false);
+262 -91
View File
@@ -6,6 +6,7 @@
#include <fcntl.h>
#include <libgen.h>
#include <limits.h>
#include <pthread.h>
#include <stdatomic.h>
#include <stdio.h>
#include <stdlib.h>
@@ -72,6 +73,58 @@ static unsigned long long next_temp_sequence(void) {
return atomic_fetch_add_explicit(&sequence, 1, memory_order_relaxed);
}
/* Process-wide umask, captured exactly once. Reading the umask requires a
* get+set round trip (umask(0); umask(old)); doing that per write would be racy
* in the multithreaded receiver, so the value is captured at process startup by
* file_umask_capture() (called at the top of main(), before any threads exist).
* The pthread_once fallback keeps a caller that never called the capture (e.g. a
* unit test) correct. */
static unsigned g_process_umask;
static atomic_bool g_process_umask_captured;
static pthread_once_t g_process_umask_once = PTHREAD_ONCE_INIT;
static void file_capture_umask_now(void) {
mode_t mask = umask(0);
umask(mask);
g_process_umask = (unsigned)mask;
atomic_store_explicit(&g_process_umask_captured, true, memory_order_release);
}
static void file_capture_umask_once(void) {
if (atomic_load_explicit(&g_process_umask_captured, memory_order_acquire))
return;
file_capture_umask_now();
}
/* Re-captures the umask. Must only be called while the process is still
* single-threaded (startup, or the daemon's post-fork setup after umask(0)),
* so a later re-capture can refresh the cached value before any receiver
* thread exists. */
void file_umask_capture(void) {
file_capture_umask_now();
}
unsigned file_process_umask(void) {
if (!atomic_load_explicit(&g_process_umask_captured, memory_order_acquire))
pthread_once(&g_process_umask_once, file_capture_umask_once);
return g_process_umask;
}
/* Base mode applied when the policy does not take the source mode wholesale
* (i.e. --perms is off). A pre-existing destination keeps its own mode; a
* brand-new file is created like rsync: source_mode & 0777 & ~umask (special
* bits are not part of a mode-preserving transfer without -p). Only when no
* metadata is available at all does the historical fixed 0644 default apply.
* The -E rule (and no-op for a plain -t) is layered on top of this base. */
static mode_t file_mode_base(const FileMetadata* metadata, bool existing_known,
mode_t existing_mode) {
if (existing_known)
return existing_mode;
if (metadata)
return metadata->mode & 0777 & ~(mode_t)file_process_umask();
return S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH;
}
bool file_checksum(File* file, ChecksumAlgo algo, uint64_t seed, uint8_t* out, size_t out_capacity,
size_t* out_len) {
if (!file || !out || !out_len || !file->data)
@@ -124,6 +177,7 @@ File* file_create(const char* path) {
file->rdev_major = 0;
file->rdev_minor = 0;
file->xattrs = NULL;
file->dest_state = (OutputDestState){0};
return file;
}
@@ -357,10 +411,69 @@ bool file_get_trust_sender(void) {
return file_trust_sender;
}
/* True when `target` is a lexical symlink target that can never escape the
* receive root once created beneath it: relative (not absolute) and containing
* no ".." path component. Used by --munge-links' sender-side containment: an
* escaping target is never transmitted (the entry is skipped/contained). */
/* rsync 3.4.1 unsafe_symlink(): true when `target` (the link's destination
* string) points outside the transfer tree rooted at the symlink's own
* location. `link_path` is the symlink's path relative to the top of the
* transfer (including its name). This is a purely lexical test matching
* rsync's util1.c: absolute/empty targets are always unsafe; leading "../"
* components are counted against the symlink's own directory depth; a ".."
* that would climb above the transfer root is unsafe. rsync 3.4.1 additionally
* rejects any INTERNAL "/../" component and a trailing "/..". */
bool file_symlink_unsafe(const char* target, const char* link_path) {
if (!target || target[0] == '\0' || target[0] == '/')
return true;
const char* rest = target;
while (strncmp(rest, "../", 3) == 0) {
rest += 3;
while (*rest == '/')
rest++;
}
if (strstr(rest, "/../") != NULL)
return true;
size_t target_len = strlen(target);
if (target_len > 3 && strcmp(&target[target_len - 3], "/..") == 0)
return true;
int depth = 0;
const char* name;
const char* slash;
const char* src = link_path ? link_path : "";
for (name = src; (slash = strchr(name, '/')) != NULL; name = slash + 1) {
if (*name == '.' && (name[1] == '/' || (name[1] == '.' && name[2] == '/'))) {
if (name[1] == '.')
depth = 0;
} else {
depth++;
}
while (slash[1] == '/')
slash++;
}
if (*name == '.' && name[1] == '.' && name[2] == '\0')
depth = 0;
for (name = target; (slash = strchr(name, '/')) != NULL; name = slash + 1) {
if (*name == '.' && (name[1] == '/' || (name[1] == '.' && name[2] == '/'))) {
if (name[1] == '.') {
if (--depth < 0)
return true;
}
} else {
depth++;
}
while (slash[1] == '/')
slash++;
}
if (*name == '.' && name[1] == '.' && name[2] == '\0')
depth--;
return depth < 0;
}
/* Strict lexical helper: true when `target` is relative (not absolute) and
* contains no ".." component at all, so it can never escape the directory it
* is created in. This is stricter than rsync's unsafe_symlink() (which allows
* an in-tree ".."); the scanner/receiver use file_symlink_unsafe()/--safe-links
* for rsync parity, and this helper is retained for callers that want the
* ".."-free guarantee. */
bool file_symlink_target_contained(const char* target) {
if (!target || target[0] == '\0' || target[0] == '/')
return false;
@@ -391,8 +504,9 @@ bool file_symlink_unmunge(char* target) {
return true;
}
/* Owned copy of `target` prefixed with SYMLINK_MUNGE_PREFIX (the sender-side
* --munge-links rewriting). Returns NULL on allocation failure. */
/* Owned copy of `target` prefixed with SYMLINK_MUNGE_PREFIX (the receiver-side
* --munge-links rewriting, matching rsync's receiver). Returns NULL on
* allocation failure. */
char* file_symlink_munge(const char* target) {
if (!target)
return NULL;
@@ -413,23 +527,14 @@ char* file_symlink_munge(const char* target) {
* the target is ever followed. The final component is never dereferenced: an
* existing non-directory entry at `path` is unlinked by name before the link is
* placed; an existing directory there is left untouched (returns false, so a
* caller can treat it as a collision). As a receiver-side trust-boundary
* invariant, `target` must be file_symlink_target_contained() (relative and
* ".."-free): an absolute or escaping target is rejected outright (returns
* false) so a malicious sender can never materialize a symlink that points
* outside the receive root. */
* caller can treat it as a collision). The link VALUE `target` is copied
* verbatim, matching rsync -l (which stores absolute and ".."-bearing targets
* as-is); target policy is the caller's job -- the scanner applies
* --safe-links/--copy-unsafe-links, and the receiver applies --munge-links.
* The PLACEMENT path is always confined below the authorized root. */
bool file_symlink_at_secure(const char* path, const char* target) {
/* The link itself (`path`) is always kept below the authorized root. The
TARGET may point anywhere: normally only a contained (relative, ".."-free)
target is permitted so a malicious sender can never plant a symlink that
later dereferences outside the root. Under --trust-sender that target
containment check is relaxed (the receiver trusts the sender and copies the
link verbatim, matching rsync -l), but path/leaf confinement is never
disabled, so the link still cannot be placed outside the tree. */
if (!path || !target || has_path_traversal(path))
return false;
if (!file_trust_sender && !file_symlink_target_contained(target))
return false;
char* leaf = NULL;
int parent_fd = file_open_secure_parent(path, &leaf, true);
if (parent_fd < 0)
@@ -855,29 +960,48 @@ int file_open_private_dir(const char* dir_path) {
return fd;
}
/* Open a --temp-dir scratch directory exactly as rsync does: the directory must
* already exist and is used as given (an absolute path is used verbatim, a
* relative one was already resolved against the destination root by the
* caller). Unlike file_open_private_dir this neither creates it nor confines
* it below the receive root, because rsync accepts any temp dir -- including
* one outside the destination tree or on another filesystem. Returns an
* O_DIRECTORY|O_CLOEXEC fd, or -1 on error. */
int file_open_temp_dir(const char* dir_path) {
if (!dir_path)
return -1;
return open(dir_path, O_RDONLY | O_DIRECTORY | O_CLOEXEC);
}
/* After the content and mode/times are restored on the just-written file, apply
* the per-file xattrs (-X/-A) and, for --fake-super, park the source's
* uid/gid/mode/mtime in the reserved xattr. All fd-relative (confined to the
* destination file) and best-effort: a per-attribute or privilege failure is
* logged and skipped, never fatal. */
static void restore_extra_fd(int fd, const FileMetadata* metadata, const FileXattrList* xattrs,
bool fake_super) {
bool fake_super, FileAttrPolicy policy) {
xattr_apply_fd(fd, xattrs);
if (fake_super && metadata) {
fake_super_store_fd(fd, (uint32_t)metadata->uid, (uint32_t)metadata->gid,
(uint32_t)metadata->mode, metadata->mtime_sec, metadata->mtime_nsec);
/* Replay: re-apply the recorded uid/gid/mode/mtime fd-relative so a save
under --fake-super restores the attrs (when privileged) instead of only
recording them. Best-effort; fake_super_restore_fd silently skips a
non-root fchown EPERM/EACCES and never fatal. */
fake_super_restore_fd(fd);
/* Record the ownership that WOULD have been applied: when an explicit
ownership request (--chown/--usermap/--groupmap/--copy-as or -o/-g) is
active, the resolved mapping; otherwise the source's own id. The real
chown is suppressed (identity_apply_ownership early-returns under
--fake-super) so recording never defeats the flag. Mode/mtime are still
replayed (policy-gated) so unprivileged --fake-super keeps working. */
uint32_t store_uid;
uint32_t store_gid;
identity_resolve_storage_ids((int32_t)metadata->uid, (int32_t)metadata->gid, &store_uid,
&store_gid);
fake_super_store_fd(fd, store_uid, store_gid, (uint32_t)metadata->mode, metadata->mtime_sec,
metadata->mtime_nsec);
fake_super_restore_fd(fd, policy);
}
}
static bool file_to_disk_secure_impl(const char* path, const void* data,
unsigned long long data_size, bool inplace, bool sparse,
bool preallocate, const FileMetadata* metadata,
bool preserve_executability, bool update, bool no_replace,
FileAttrPolicy policy, bool update, bool no_replace,
bool use_fsync, const char* temp_dir,
const FileXattrList* xattrs, bool fake_super,
bool keep_partial) {
@@ -887,6 +1011,17 @@ static bool file_to_disk_secure_impl(const char* path, const void* data,
return false;
int fd = -1;
bool ok = false;
/* Set when a --temp-dir install fails with EXDEV: rsync then falls back to a
* non-atomic write directly in the destination directory (see the tail of
* this function). */
bool cross_device_fallback = false;
/* The base mode applied when --perms is off (neither the source mode nor an
* exec-only change is taken wholesale): a pre-existing destination keeps its
* own mode (special bits dropped), while a brand-new file uses
* source&~umask when metadata is available (see file_mode_base) or 0644 when
* there is none. Captured from the destination probe before the write. */
mode_t existing_mode = S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH;
bool existing_mode_known = false;
if (inplace) {
/* --inplace writes directly into the destination; a scratch --temp-dir
does not apply and must never redirect these writes. */
@@ -897,10 +1032,16 @@ static bool file_to_disk_secure_impl(const char* path, const void* data,
super-mode gate (a client-controlled device write). fstatat with
AT_SYMLINK_NOFOLLOW does not follow a symlink and does not block. */
struct stat pre_stat;
if (fstatat(dirfd, leaf, &pre_stat, AT_SYMLINK_NOFOLLOW) == 0 && !S_ISREG(pre_stat.st_mode)) {
close(dirfd);
free(leaf);
return false;
if (fstatat(dirfd, leaf, &pre_stat, AT_SYMLINK_NOFOLLOW) == 0) {
if (!S_ISREG(pre_stat.st_mode)) {
close(dirfd);
free(leaf);
return false;
}
/* Capture the old destination mode before the overwrite so a no--p/-E
* write can restore it (the write itself may clear setuid/setgid). */
existing_mode = pre_stat.st_mode & 0777;
existing_mode_known = true;
}
/* O_NONBLOCK: a no-op for a regular file, but a raced-in FIFO cannot block
the open before the post-open S_ISREG re-check rejects it. */
@@ -953,15 +1094,25 @@ static bool file_to_disk_secure_impl(const char* path, const void* data,
/* Normalize the mode: apply the metadata-derived safe mode when the
sender supplied metadata (setuid/setgid/sticky are never honored);
otherwise fall back to a safe default so dangerous bits on an
existing destination cannot survive an overwrite. */
existing destination cannot survive an overwrite. When the policy
requests neither -p nor -E the source mode is deliberately ignored
and the pre-existing destination mode (or 0644 for a new file) is
restored instead. The exec-bits-only -E change is likewise applied
on top of that destination-derived base, not the scratch file's
0600. */
if (ok) {
if (metadata)
ok = file_restore_metadata_fd(fd, metadata, preserve_executability);
else if (fchmod(fd, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH) != 0)
if (metadata) {
if (!policy.perms &&
fchmod(fd, file_mode_base(metadata, existing_mode_known, existing_mode)) != 0)
ok = false;
if (ok)
ok = file_restore_metadata_fd(fd, metadata, policy);
} else if (fchmod(fd, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH) != 0) {
ok = false;
}
}
if (ok)
restore_extra_fd(fd, metadata, xattrs, fake_super);
restore_extra_fd(fd, metadata, xattrs, fake_super, policy);
if (ok && use_fsync)
ok = fsync(fd) == 0;
}
@@ -974,25 +1125,31 @@ static bool file_to_disk_secure_impl(const char* path, const void* data,
install failure (partial data may exist, --partial may retain it) from a
pre-write validation failure (nothing to retain). */
bool write_attempted = false;
if (update && metadata) {
/* This check protects the normal atomic path as far as possible. A
concurrent replacement can still occur before the final rename. */
struct stat destination_stat;
if (fstatat(dirfd, leaf, &destination_stat, AT_SYMLINK_NOFOLLOW) == 0 &&
S_ISREG(destination_stat.st_mode) && stat_is_newer(&destination_stat, metadata)) {
close(dirfd);
free(leaf);
return true;
}
/* Probe the destination ONCE up front: it both drives the --update check
and records the pre-existing mode the no--p/-E fallback preserves. */
struct stat destination_stat;
bool destination_is_regular =
fstatat(dirfd, leaf, &destination_stat, AT_SYMLINK_NOFOLLOW) == 0 &&
S_ISREG(destination_stat.st_mode);
if (destination_is_regular) {
existing_mode = destination_stat.st_mode & 0777;
existing_mode_known = true;
}
if (update && metadata && destination_is_regular &&
stat_is_newer(&destination_stat, metadata)) {
close(dirfd);
free(leaf);
return true;
}
/* Scratch directory for the temporary working copy. When NULL the temp
file is created in the destination directory, exactly as historically. */
int scratch_dirfd = -1;
if (temp_dir) {
scratch_dirfd = file_open_private_dir(temp_dir);
scratch_dirfd = file_open_temp_dir(temp_dir);
if (scratch_dirfd < 0) {
int saved_errno = errno;
log_message(LOG_LEVEL_ERROR, "could not open --temp-dir scratch directory '%s': %s",
log_message(LOG_LEVEL_ERROR,
"--temp-dir '%s' could not be opened (rsync requires it to already exist): %s",
temp_dir, strerror(saved_errno));
close(dirfd);
free(leaf);
@@ -1061,10 +1218,19 @@ static bool file_to_disk_secure_impl(const char* path, const void* data,
? file_store_write_sparse(fd, (const unsigned char*)data, data_size)
: write_all(fd, data, data_size);
}
if (ok && metadata)
ok = file_restore_metadata_fd(fd, metadata, preserve_executability);
if (ok) {
if (metadata) {
if (!policy.perms &&
fchmod(fd, file_mode_base(metadata, existing_mode_known, existing_mode)) != 0)
ok = false;
if (ok)
ok = file_restore_metadata_fd(fd, metadata, policy);
} else if (fchmod(fd, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH) != 0) {
ok = false;
}
}
if (ok)
restore_extra_fd(fd, metadata, xattrs, fake_super);
restore_extra_fd(fd, metadata, xattrs, fake_super, policy);
if (ok && use_fsync)
ok = fsync(fd) == 0;
}
@@ -1081,17 +1247,16 @@ static bool file_to_disk_secure_impl(const char* path, const void* data,
errno != ENOENT)
ok = false;
} else {
/* Cross-device (or otherwise impossible) link: rsync falls back to
writing the file directly in the destination directory. Record
it and retry below with no scratch dir. */
if (scratch_dirfd >= 0 && errno == EXDEV)
log_message(LOG_LEVEL_ERROR,
"temp dir is on a different filesystem than the destination; cannot "
"link file into place (EXDEV); no fallback copy is attempted");
cross_device_fallback = true;
ok = false;
}
} else if (renameat(scratch_dirfd >= 0 ? scratch_dirfd : dirfd, tmp, dirfd, leaf) != 0) {
if (scratch_dirfd >= 0 && errno == EXDEV)
log_message(LOG_LEVEL_ERROR,
"temp dir is on a different filesystem than the destination; cannot "
"atomically install file (EXDEV); no fallback copy is attempted");
cross_device_fallback = true;
ok = false;
}
}
@@ -1124,43 +1289,49 @@ static bool file_to_disk_secure_impl(const char* path, const void* data,
close(fd);
close(dirfd);
free(leaf);
if (cross_device_fallback) {
/* rsync semantics: a --temp-dir on another filesystem must not abort the
write. Retry once with no scratch dir so the file is written and
installed non-atomically in the destination directory. */
log_message(LOG_LEVEL_WARNING,
"temp dir is on a different filesystem than the destination; falling back to a "
"non-atomic copy into the destination directory");
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
policy, update, no_replace, use_fsync, NULL, xattrs, fake_super,
keep_partial);
}
return ok;
}
bool file_to_disk_secure(const char* path, const void* data, unsigned long long data_size,
bool inplace, bool sparse, bool preallocate, const FileMetadata* metadata,
bool preserve_executability, const char* temp_dir) {
FileAttrPolicy policy, const char* temp_dir) {
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
preserve_executability, false, false, false, temp_dir, NULL,
false, false);
policy, false, false, false, temp_dir, NULL, false, false);
}
bool file_to_disk_secure_update(const char* path, const void* data, unsigned long long data_size,
bool inplace, bool sparse, bool preallocate,
const FileMetadata* metadata, bool preserve_executability,
const FileMetadata* metadata, FileAttrPolicy policy,
const char* temp_dir) {
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
preserve_executability, true, false, false, temp_dir, NULL, false,
false);
policy, true, false, false, temp_dir, NULL, false, false);
}
bool file_to_disk_secure_with_fsync(const char* path, const void* data,
unsigned long long data_size, bool inplace, bool sparse,
bool preallocate, const FileMetadata* metadata,
bool preserve_executability, bool use_fsync,
const char* temp_dir) {
FileAttrPolicy policy, bool use_fsync, const char* temp_dir) {
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
preserve_executability, false, false, use_fsync, temp_dir, NULL,
false, false);
policy, false, false, use_fsync, temp_dir, NULL, false, false);
}
bool file_to_disk_secure_no_replace(const char* path, const void* data,
unsigned long long data_size, bool sparse, bool preallocate,
const FileMetadata* metadata, bool preserve_executability,
const FileMetadata* metadata, FileAttrPolicy policy,
const char* temp_dir) {
return file_to_disk_secure_impl(path, data, data_size, false, sparse, preallocate, metadata,
preserve_executability, false, true, false, temp_dir, NULL, false,
false);
policy, false, true, false, temp_dir, NULL, false, false);
}
/* Receiver write-path variant that also applies the per-file xattrs (-X/-A)
@@ -1170,13 +1341,12 @@ bool file_to_disk_secure_no_replace(const char* path, const void* data,
* failed write's temp. See file_to_disk_secure_impl for the semantics. */
bool file_to_disk_secure_attrs(const char* path, const void* data, unsigned long long data_size,
bool inplace, bool sparse, bool preallocate,
const FileMetadata* metadata, bool preserve_executability,
bool update, bool no_replace, bool use_fsync,
const FileXattrList* xattrs, bool fake_super, bool keep_partial,
const char* temp_dir) {
const FileMetadata* metadata, FileAttrPolicy policy, bool update,
bool no_replace, bool use_fsync, const FileXattrList* xattrs,
bool fake_super, bool keep_partial, const char* temp_dir) {
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
preserve_executability, update, no_replace, use_fsync, temp_dir,
xattrs, fake_super, keep_partial);
policy, update, no_replace, use_fsync, temp_dir, xattrs,
fake_super, keep_partial);
}
/* Atomic --link-dest install. The destination is replaced (via a temporary
@@ -1197,7 +1367,7 @@ bool file_to_disk_secure_attrs(const char* path, const void* data, unsigned long
static bool file_to_disk_secure_link_impl(const char* path, const char* basis_path,
const void* data, unsigned long long data_size,
bool preallocate, const FileMetadata* metadata,
bool preserve_executability, bool use_fsync,
FileAttrPolicy policy, bool use_fsync,
const FileXattrList* xattrs, bool fake_super,
const char* temp_dir) {
if (!path || !basis_path)
@@ -1209,11 +1379,12 @@ static bool file_to_disk_secure_link_impl(const char* path, const char* basis_pa
int scratch_dirfd = -1;
if (temp_dir) {
scratch_dirfd = file_open_private_dir(temp_dir);
scratch_dirfd = file_open_temp_dir(temp_dir);
if (scratch_dirfd < 0) {
int saved_errno = errno;
log_message(LOG_LEVEL_ERROR, "could not open --temp-dir scratch directory '%s': %s", temp_dir,
strerror(saved_errno));
log_message(LOG_LEVEL_ERROR,
"--temp-dir '%s' could not be opened (rsync requires it to already exist): %s",
temp_dir, strerror(saved_errno));
close(dirfd);
free(leaf);
return false;
@@ -1286,8 +1457,8 @@ static bool file_to_disk_secure_link_impl(const char* path, const char* basis_pa
/* The basis file could not be linked in (missing, cross-device, refused
by the filesystem). Write a byte-identical local copy instead. */
return file_to_disk_secure_attrs(path, data, data_size, false, false, preallocate, metadata,
preserve_executability, false, false, use_fsync, xattrs,
fake_super, false, temp_dir);
policy, false, false, use_fsync, xattrs, fake_super, false,
temp_dir);
}
if (scratch_dirfd >= 0)
@@ -1299,25 +1470,25 @@ static bool file_to_disk_secure_link_impl(const char* path, const char* basis_pa
bool file_to_disk_secure_link(const char* path, const char* basis_path, const void* data,
unsigned long long data_size, bool preallocate,
const FileMetadata* metadata, bool preserve_executability,
bool use_fsync, const char* temp_dir) {
const FileMetadata* metadata, FileAttrPolicy policy, bool use_fsync,
const char* temp_dir) {
return file_to_disk_secure_link_impl(path, basis_path, data, data_size, preallocate, metadata,
preserve_executability, use_fsync, NULL, false, temp_dir);
policy, use_fsync, NULL, false, temp_dir);
}
bool file_to_disk_secure_link_attrs(const char* path, const char* basis_path, const void* data,
unsigned long long data_size, bool preallocate,
const FileMetadata* metadata, bool preserve_executability,
const FileMetadata* metadata, FileAttrPolicy policy,
bool use_fsync, const FileXattrList* xattrs, bool fake_super,
const char* temp_dir) {
return file_to_disk_secure_link_impl(path, basis_path, data, data_size, preallocate, metadata,
preserve_executability, use_fsync, xattrs, fake_super,
temp_dir);
policy, use_fsync, xattrs, fake_super, temp_dir);
}
bool file_write_to_disk(const char* path, const void* data, unsigned long long data_size,
bool inplace, bool sparse) {
if (!path || (!data && data_size != 0) || has_path_traversal(path))
return false;
return file_to_disk_secure(path, data, data_size, inplace, sparse, false, NULL, false, NULL);
FileAttrPolicy policy = {false, false, false, false};
return file_to_disk_secure(path, data, data_size, inplace, sparse, false, NULL, policy, NULL);
}
+50 -28
View File
@@ -28,17 +28,36 @@ void file_metadata_destroy(void* metadata);
/* --open-noatime process-wide sender policy; see file.c. */
void file_set_open_noatime(bool enable);
bool file_get_open_noatime(void);
/* Capture the process umask ONCE, before any threads are created. Call this at
* the very top of main() in both entry points so the cached value is read while
* the process is still single-threaded: reading the umask needs a get+set round
* trip (umask(0); umask(old)), which would race against receiver threads
* creating files if it happened during the first write. Idempotent and safe to
* call more than once. */
void file_umask_capture(void);
/* Process-wide umask, captured once (thread-safe). Used to derive the mode of
* a brand-new destination like rsync: source_mode & 0777 & ~umask. Falls back
* to file_umask_capture() (behind pthread_once) if capture was never called. */
unsigned file_process_umask(void);
/* Open `path` read-only for transfer, honouring --open-noatime when set. */
int file_open_for_read(const char* path);
bool file_write_to_disk(const char* path, const void* data, unsigned long long data_size,
bool inplace, bool sparse);
/* Symlink trust-boundary helpers (Phase 4, symlink wave). --munge-links
* sender-side marker: every transmitted symlink target is prefixed with this
* while the flag is on; the receiver strips it to restore the real target. */
#define SYMLINK_MUNGE_PREFIX "#SYMLINK/"
/* Symlink trust-boundary helpers (Phase 4, symlink wave; rsync parity).
* --munge-links is a RECEIVER-side rewrite: rsync prefixes every stored symlink
* target with this marker, making the link unusable while the referenced
* directory does not exist. A SENDER receiving a munged source strips it back
* off before transmitting (so a munged tree round-trips through the receiver's
* re-munging). */
#define SYMLINK_MUNGE_PREFIX "/rsyncd-munged/"
char* file_symlink_munge(const char* target);
/* rsync 3.4.1 unsafe_symlink(): true when `target` escapes the transfer tree
* rooted at `link_path` (the symlink's transfer-relative path incl. its name).
* Absolute/empty targets and targets climbing above the transfer root (via
* "..") are unsafe, as are internal "/../" components and trailing "/..". */
bool file_symlink_unsafe(const char* target, const char* link_path);
/* True when a lexical target is relative and contains no ".." component, so it
* can never escape the receive root once created beneath it. */
bool file_symlink_target_contained(const char* target);
@@ -46,8 +65,9 @@ bool file_symlink_target_contained(const char* target);
* returns true when a marker was removed. */
bool file_symlink_unmunge(char* target);
/* Create a symlink at `path` -> `target`, confined below the authorized root
* (O_NOFOLLOW parent walk, symlinkat; the target is never followed). Returns
* false when a directory already occupies `path`. */
* (O_NOFOLLOW parent walk, symlinkat; the target is never followed). The link
* value is copied verbatim (rsync -l); only the placement path is confined.
* Returns false when a directory already occupies `path`. */
bool file_symlink_at_secure(const char* path, const char* target);
/* --keep-dirlinks (-K) receiver process-wide policy: allow an in-root existing
* symlink-to-directory to be followed as a directory. */
@@ -75,37 +95,40 @@ bool file_rename_secure(const char* old_path, const char* new_path);
regular file. See the .c for the exact success semantics. */
bool file_remove_tree_secure(const char* path);
/* Open a private 0700 directory (creating it on demand) that must live below
the authorized root. Used for the --temp-dir scratch directory and the
--delay-updates staging directory. */
the authorized root. Used for the --delay-updates staging directory. */
int file_open_private_dir(const char* dir_path);
/* Open an existing --temp-dir scratch directory as-is (absolute or relative;
no creation, no root confinement), matching rsync's --temp-dir handling. */
int file_open_temp_dir(const char* dir_path);
/* The file_to_disk_secure* variants write a temporary copy in the destination
directory and atomically rename it over `path`. temp_dir is an absolute,
root-confined scratch directory (already validated by the caller): when it
is non-NULL the temporary copy is instead created there (with a name unique
across the whole scratch directory) and atomically renamed into the
destination directory once fully written and fsynced. A rename across
filesystems (EXDEV) fails the write with an error; the file is never
silently copied into place. Pass NULL for the historical same-directory
behavior. --inplace writes never use temp_dir. */
directory and atomically rename it over `path`. temp_dir is a scratch
directory (an absolute path, or one the caller already resolved against the
destination root): when it is non-NULL the temporary copy is instead created
there (with a name unique across the whole scratch directory) and atomically
renamed into the destination directory once fully written and fsynced. When
that rename/link fails with EXDEV (the scratch dir is on another filesystem)
the write falls back to a non-atomic copy directly in the destination
directory, matching rsync. Pass NULL for the same-directory behavior.
--inplace writes never use temp_dir. */
bool file_to_disk_secure(const char* path, const void* data, unsigned long long data_size,
bool inplace, bool sparse, bool preallocate, const FileMetadata* metadata,
bool preserve_executability, const char* temp_dir);
FileAttrPolicy policy, const char* temp_dir);
bool file_to_disk_secure_with_fsync(const char* path, const void* data,
unsigned long long data_size, bool inplace, bool sparse,
bool preallocate, const FileMetadata* metadata,
bool preserve_executability, bool use_fsync,
const char* temp_dir);
FileAttrPolicy policy, bool use_fsync, const char* temp_dir);
/* With update enabled, an existing newer destination is left untouched. The
check is descriptor-based for inplace writes; atomic replacement still has
an unavoidable final rename race without filesystem locking. */
bool file_to_disk_secure_update(const char* path, const void* data, unsigned long long data_size,
bool inplace, bool sparse, bool preallocate,
const FileMetadata* metadata, bool preserve_executability,
const FileMetadata* metadata, FileAttrPolicy policy,
const char* temp_dir);
bool file_to_disk_secure_no_replace(const char* path, const void* data,
unsigned long long data_size, bool sparse, bool preallocate,
const FileMetadata* metadata, bool preserve_executability,
const FileMetadata* metadata, FileAttrPolicy policy,
const char* temp_dir);
/* Receiver write-path variant that also applies per-file xattrs (-X/-A) and the
* --fake-super stat xattr fd-relative before the final rename. `update` /
@@ -113,10 +136,9 @@ bool file_to_disk_secure_no_replace(const char* path, const void* data,
* enables --partial best-effort retention of a failed write's temp. */
bool file_to_disk_secure_attrs(const char* path, const void* data, unsigned long long data_size,
bool inplace, bool sparse, bool preallocate,
const FileMetadata* metadata, bool preserve_executability,
bool update, bool no_replace, bool use_fsync,
const FileXattrList* xattrs, bool fake_super, bool keep_partial,
const char* temp_dir);
const FileMetadata* metadata, FileAttrPolicy policy, bool update,
bool no_replace, bool use_fsync, const FileXattrList* xattrs,
bool fake_super, bool keep_partial, const char* temp_dir);
/* Atomic --link-dest install: replace `path` with a hard link to `basis_path`
(via a temp name + rename); fall back to a byte-identical local copy from
`data` when the link is impossible (EXDEV/EPERM/unsupported filesystem).
@@ -125,15 +147,15 @@ bool file_to_disk_secure_attrs(const char* path, const void* data, unsigned long
never re-allocated). */
bool file_to_disk_secure_link(const char* path, const char* basis_path, const void* data,
unsigned long long data_size, bool preallocate,
const FileMetadata* metadata, bool preserve_executability,
bool use_fsync, const char* temp_dir);
const FileMetadata* metadata, FileAttrPolicy policy, bool use_fsync,
const char* temp_dir);
/* Like file_to_disk_secure_link, but the byte-copy fallback also applies the
* per-file xattrs (-X/-A) and --fake-super stat xattr (fd-relative). On a
* successful hard link no attributes are applied (the shared inode already
* carries the basis's). */
bool file_to_disk_secure_link_attrs(const char* path, const char* basis_path, const void* data,
unsigned long long data_size, bool preallocate,
const FileMetadata* metadata, bool preserve_executability,
const FileMetadata* metadata, FileAttrPolicy policy,
bool use_fsync, const FileXattrList* xattrs, bool fake_super,
const char* temp_dir);
+38
View File
@@ -0,0 +1,38 @@
#ifndef FILE_ATTR_H
#define FILE_ATTR_H
#include "config.h"
#include <stdbool.h>
#include <sys/stat.h>
/*
* Per-attribute receiver policy for applying a transmitted FileMetadata. This
* is the split-out replacement for the former single use_metadata bundle: each
* flag is applied independently, matching rsync's -p/-t/-o/-g/-E/-U semantics.
* `use_metadata` remains the transport/presence gate (whether the metadata frame
* travelled at all); this struct decides which attributes are ACTUALLY applied.
*
* It lives in its own header (rather than metadata.h) because xattr.h's
* fake_super_restore_fd() takes one and metadata.h <-> file_types.h form an
* include cycle that must not be entered from xattr.h.
*
* The mode leg is: perms wins over executability; an exec-bits-only change is
* made only when perms is off; when neither is set the receiver deliberately
* sets no source mode. file.c then substitutes the pre-existing destination
* mode for a brand-new destination with metadata it uses the sanitized
* source-mode-&-umask base (S_IWGRP|S_IWOTH cleared), and the fixed 0644
* default only when no metadata is available at all, so a no--p overwrite
* does not lose the destination's perms.
*/
typedef struct FileAttrPolicy {
bool perms; /* config->preserve_perms: apply the source mode bits */
bool times; /* config->preserve_times: apply the source mtime */
bool atimes; /* config->preserve_atimes (-U): apply the source atime */
bool executability; /* config->use_executability (-E): exec-bits-only mode */
} FileAttrPolicy;
/* Build the per-attribute policy from a connection's Config. A NULL config
* yields the all-off policy (no attribute application). */
FileAttrPolicy file_attr_policy_from_config(const Config* config);
#endif
+26
View File
@@ -240,3 +240,29 @@ bool file_list_affects(const FileListSet* set, const char* rel) {
entry (binary search for the first entry at or after `rel` + '/'). */
return path_index_has_descendant(&set->index, rel);
}
bool file_list_dir_in_scope(const FileListSet* set, const char* rel) {
if (!set || set->whole_tree)
return true;
if (!rel || rel[0] == '\0')
return false;
/* `rel` itself is listed, or one of its ancestor prefixes is an exact listed
directory (a listed prefix of a directory path is necessarily a
directory). */
size_t len = strlen(rel);
while (len > 0) {
const char* slash = NULL;
for (size_t i = len; i-- > 0;) {
if (rel[i] == '/') {
slash = rel + i;
break;
}
}
if (!slash)
break;
len = (size_t)(slash - rel);
if (path_index_contains_n(&set->index, rel, len))
return true;
}
return path_index_contains(&set->index, rel);
}
+10
View File
@@ -40,4 +40,14 @@ void file_list_destroy(FileListSet* set);
* this returns true, files are transferred only when it returns true. */
bool file_list_affects(const FileListSet* set, const char* rel);
/* True when the DIRECTORY `rel` (path relative to the source root) is inside a
* listed directory subtree: `rel` itself is a listed entry, or one of `rel`'s
* ancestor directory prefixes is an exact listed entry. Unlike
* file_list_affects this does NOT treat an ancestor of a listed entry as
* affected, so an implied parent directory of a listed file is not synchronized
* (rsync deletes nothing in it). With no set or a whole-tree set every
* directory is in scope. This is the delete-walker's "synchronized directory"
* predicate. */
bool file_list_dir_in_scope(const FileListSet* set, const char* rel);
#endif
+402 -151
View File
@@ -19,6 +19,7 @@
#include "delay_updates.h"
#include "delta.h"
#include "file.h"
#include "format.h"
#include "identity.h"
#include "log.h"
#include "metadata.h"
@@ -52,7 +53,7 @@ static FileSaveResult file_stage_delayed_update(const char* root_directory,
if (!config)
return FILE_SAVE_ERROR;
bool sparse = config->preserve_sparse;
bool preserve_executability = config->use_executability;
FileAttrPolicy policy = file_attr_policy_from_config(config);
if (config->existing && !file_path_exists_secure(destination_path))
return FILE_SAVE_SKIPPED;
@@ -91,13 +92,12 @@ static FileSaveResult file_stage_delayed_update(const char* root_directory,
bool ok;
if (file->basis_link) {
ok = file_to_disk_secure_link(staged_path, file->basis_link, file->data->data, file->data->size,
config->preallocate, metadata, preserve_executability,
config->use_fsync, NULL);
config->preallocate, metadata, policy, config->use_fsync, NULL);
} else {
ok = file_to_disk_secure_attrs(staged_path, file->data->data, file->data->size, false, sparse,
config->preallocate, metadata, preserve_executability, false,
false, config->use_fsync, file->xattrs, config->fake_super,
false, NULL);
ok =
file_to_disk_secure_attrs(staged_path, file->data->data, file->data->size, false, sparse,
config->preallocate, metadata, policy, false, false,
config->use_fsync, file->xattrs, config->fake_super, false, NULL);
}
if (!ok) {
free(staged_path);
@@ -229,7 +229,7 @@ static FileSaveResult file_save_hardlink_sibling(const char* root_directory, con
}
bool preallocate = cfg && cfg->preallocate;
bool preserve_executability = cfg && cfg->use_executability;
FileAttrPolicy policy = file_attr_policy_from_config(cfg);
bool use_fsync = cfg && cfg->use_fsync;
if (cfg->delay_updates) {
@@ -265,9 +265,9 @@ static FileSaveResult file_save_hardlink_sibling(const char* root_directory, con
}
FileXattrList* sibling_xattrs =
cfg->use_xattrs ? xattr_capture_path(staged_first, cfg->preserve_acls) : NULL;
bool ok = file_to_disk_secure_link_attrs(
staged_sibling, staged_first, content, content_size, preallocate, file->metadata,
preserve_executability, use_fsync, sibling_xattrs, cfg ? cfg->fake_super : false, NULL);
bool ok = file_to_disk_secure_link_attrs(staged_sibling, staged_first, content, content_size,
preallocate, file->metadata, policy, use_fsync,
sibling_xattrs, cfg ? cfg->fake_super : false, NULL);
xattr_list_free(sibling_xattrs);
free(content);
if (ok)
@@ -295,13 +295,31 @@ static FileSaveResult file_save_hardlink_sibling(const char* root_directory, con
free(destination_path);
return absent_result;
}
const char* temp_dir = (cfg && cfg->temp_dir) ? cfg->temp_dir : NULL;
/* Resolve a relative --temp-dir under the destination root, exactly as the
* primary save path does; an absolute or `..`-escaping value is rejected. */
char* resolved_temp = NULL;
if (cfg->temp_dir) {
if (cfg->temp_dir[0] == '/' || has_path_traversal(cfg->temp_dir)) {
free(content);
free(first_disk);
free(destination_path);
return FILE_SAVE_ERROR;
}
resolved_temp = path_cat(root_directory, cfg->temp_dir);
if (!resolved_temp) {
free(content);
free(first_disk);
free(destination_path);
return FILE_SAVE_ERROR;
}
}
FileXattrList* sibling_xattrs =
cfg->use_xattrs ? xattr_capture_path(first_disk, cfg->preserve_acls) : NULL;
bool ok = file_to_disk_secure_link_attrs(
destination_path, first_disk, content, content_size, preallocate, file->metadata,
preserve_executability, use_fsync, sibling_xattrs, cfg ? cfg->fake_super : false, temp_dir);
destination_path, first_disk, content, content_size, preallocate, file->metadata, policy,
use_fsync, sibling_xattrs, cfg ? cfg->fake_super : false, resolved_temp);
xattr_list_free(sibling_xattrs);
free(resolved_temp);
free(content);
free(first_disk);
free(destination_path);
@@ -359,14 +377,6 @@ static FileSaveResult file_save_special_to_disk(const char* root_directory, cons
log_message(LOG_LEVEL_ERROR, "Special node has no device/FIFO/socket mode");
return FILE_SAVE_ERROR;
}
if (is_sock) {
/* No standard filesystem call recreates a socket; best-effort unsupported. */
char* escaped_path = output_escape(file->path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "socket not recreated: %s (unsupported; skipped)",
escaped_path ? escaped_path : "<allocation failed>");
free(escaped_path);
return FILE_SAVE_SKIPPED;
}
if (is_char || is_blk) {
if (!config || !config->preserve_devices)
return FILE_SAVE_SKIPPED;
@@ -384,7 +394,10 @@ static FileSaveResult file_save_special_to_disk(const char* root_directory, cons
free(escaped_path);
return FILE_SAVE_SKIPPED;
}
} else if (is_fifo) {
} else if (is_fifo || is_sock) {
/* FIFOs and unix sockets are recreated by --specials. mknod(S_IFSOCK)
works unprivileged on Linux (the node carries no live socket), so unlike
a socket bound to a live fd it can be materialized. */
if (!config || !config->preserve_specials)
return FILE_SAVE_SKIPPED;
}
@@ -434,10 +447,18 @@ static FileSaveResult file_save_special_to_disk(const char* root_directory, cons
} else if (is_blk) {
create_mode = S_IFBLK;
rdev = makedev((unsigned)file->rdev_major, (unsigned)file->rdev_minor);
} else if (is_sock) {
create_mode = S_IFSOCK;
} else {
create_mode = S_IFIFO;
}
mode_t perms = mode & 0777;
const char* node_kind = (is_char || is_blk) ? "device" : (is_fifo ? "FIFO" : "socket");
/* Under -p/--perms rsync copies the source's permission and special bits; a
* kernel that denies setuid/setgid/sticky reports the failure rather than
* having them masked here. Without -p the node is created like any other new
* entry: source_mode & 0777 & ~umask. */
mode_t perms = config->preserve_perms ? (mode & (mode_t)(S_ISUID | S_ISGID | S_ISVTX | 0777))
: (mode & 0777 & ~(mode_t)file_process_umask());
int rc = is_fifo ? mkfifoat(parent_fd, leaf, perms)
: mknodat(parent_fd, leaf, create_mode | perms, rdev);
@@ -448,7 +469,7 @@ static FileSaveResult file_save_special_to_disk(const char* root_directory, cons
struct stat st;
if (fstatat(parent_fd, leaf, &st, AT_SYMLINK_NOFOLLOW) == 0 &&
((is_char && S_ISCHR(st.st_mode)) || (is_blk && S_ISBLK(st.st_mode)) ||
(is_fifo && S_ISFIFO(st.st_mode)))) {
(is_fifo && S_ISFIFO(st.st_mode)) || (is_sock && S_ISSOCK(st.st_mode)))) {
close(parent_fd);
free(leaf);
free(destination);
@@ -456,7 +477,7 @@ static FileSaveResult file_save_special_to_disk(const char* root_directory, cons
}
char* escaped_path = output_escape(file->path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "refusing to replace existing entry with %s: %s (skipped)",
is_fifo ? "FIFO" : "device", escaped_path ? escaped_path : "<allocation failed>");
node_kind, escaped_path ? escaped_path : "<allocation failed>");
free(escaped_path);
} else if (errno == EPERM || errno == EACCES) {
/* Missing CAP_MKNOD / parent write permission: the environment cannot
@@ -465,14 +486,12 @@ static FileSaveResult file_save_special_to_disk(const char* root_directory, cons
log_message(LOG_LEVEL_WARNING,
"skipping %s: cannot create %s node (%s)\n"
" --devices/--specials node creation needs privilege (CAP_MKNOD)",
escaped_path ? escaped_path : "<allocation failed>", is_fifo ? "FIFO" : "device",
strerror(errno));
escaped_path ? escaped_path : "<allocation failed>", node_kind, strerror(errno));
free(escaped_path);
} else {
char* escaped_path = output_escape(file->path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "failed to create %s %s: %s (skipped)",
is_fifo ? "FIFO" : "device", escaped_path ? escaped_path : "<allocation failed>",
strerror(errno));
log_message(LOG_LEVEL_WARNING, "failed to create %s %s: %s (skipped)", node_kind,
escaped_path ? escaped_path : "<allocation failed>", strerror(errno));
free(escaped_path);
}
close(parent_fd);
@@ -481,11 +500,23 @@ static FileSaveResult file_save_special_to_disk(const char* root_directory, cons
return FILE_SAVE_SKIPPED;
}
/* Apply mtime on the fresh node (utimensat, no-follow). */
struct timespec times[2] = {
{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
{.tv_sec = file->metadata->mtime_sec, .tv_nsec = file->metadata->mtime_nsec}};
utimensat(parent_fd, leaf, times, AT_SYMLINK_NOFOLLOW);
/* Apply times on the fresh node (utimensat, no-follow) per the negotiated
* per-attribute policy: mtime only under -t, atime only under -U. The slot
* not requested stays UTIME_OMIT so it is left untouched. */
FileAttrPolicy policy = file_attr_policy_from_config(config);
if (policy.times || (policy.atimes && file->metadata->atime_valid)) {
struct timespec times[2] = {{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
{.tv_sec = 0, .tv_nsec = UTIME_OMIT}};
if (policy.times) {
times[1].tv_sec = file->metadata->mtime_sec;
times[1].tv_nsec = file->metadata->mtime_nsec;
}
if (policy.atimes && file->metadata->atime_valid) {
times[0].tv_sec = file->metadata->atime_sec;
times[0].tv_nsec = file->metadata->atime_nsec;
}
utimensat(parent_fd, leaf, times, AT_SYMLINK_NOFOLLOW);
}
/* P7 Wave E: apply the negotiated ownership to the node ITSELF. A FIFO is
created unprivileged, but --copy-as and explicit identity policies own
every entry (a char/block node path is already privilege-gated above). The
@@ -592,7 +623,7 @@ FileSaveResult file_save_to_disk_full(const char* root_directory, const File* fi
bool backup_enabled = config && config->backup && !config->ignore_existing;
bool inplace = config && config->inplace;
bool sparse = config && config->preserve_sparse;
bool preserve_executability = config && config->use_executability;
FileAttrPolicy policy = file_attr_policy_from_config(config);
const char* backup_suffix = (config && config->suffix) ? config->suffix : "~";
const char* backup_dir = (config && config->backup_dir) ? config->backup_dir : NULL;
const char* partial_dir = (config && config->partial_dir) ? config->partial_dir : NULL;
@@ -696,26 +727,23 @@ FileSaveResult file_save_to_disk_full(const char* root_directory, const File* fi
char* link_path = path_cat(root_directory, file->path);
if (!link_path)
return FILE_SAVE_ERROR;
/* Restore the real target by stripping the sender's --munge-links marker.
Only unmunge when the policy was negotiated: a plain -l run must preserve
a source symlink whose target genuinely begins with the marker verbatim. */
/* The link value is stored verbatim (rsync -l parity: absolute and
".."-bearing targets are preserved; the scanner's --safe-links /
--copy-unsafe-links decide which links are sent at all). --munge-links
is a RECEIVER-side rewrite: the stored target is prefixed with
/rsyncd-munged/, making the link unusable while the referenced directory
does not exist -- exactly as rsync's receiver munges. Only the link's
own placement path is confined below the receive root. */
bool munge = config && config->munge_links;
char* target = str_dup(file->symlink_target);
bool ok = target != NULL;
if (ok && config && config->munge_links)
file_symlink_unmunge(target);
/* Receiver-side trust boundary (independent of the sender): a target that
could escape the receive root (absolute, or relative-with-"..") is never
materialized. It is contained (the entry is skipped) rather than failing
the whole transfer, so a hostile sender can inject a broken symlink but
can never redirect it outside the root. --trust-sender deliberately
relaxes this receiver-side re-validation: a trusted sender's escaping
symlink target is copied verbatim (rsync -l parity). The low-level
leaf/destination confinement in file_symlink_at_secure still ensures the
link itself is placed inside the authorized root. */
if (ok && !file_get_trust_sender() && !file_symlink_target_contained(target))
ok = false;
if (ok && munge) {
char* munged = file_symlink_munge(target);
free(target);
target = munged;
ok = target != NULL;
}
if (!ok) {
/* Skip the escaping/empty target (contained) rather than abort. */
free(target);
free(link_path);
return FILE_SAVE_SKIPPED;
@@ -734,8 +762,11 @@ FileSaveResult file_save_to_disk_full(const char* root_directory, const File* fi
(utimensat/lchown/fchmodat AT_SYMLINK_NOFOLLOW). -J/--omit-link-times
suppresses the timestamps; ownership stays gated by the identity policy.
A symlink has no children, so this can be applied immediately. */
if (ok && config && config->use_metadata)
ok = file_restore_symlink_metadata(link_path, file->metadata, config->omit_link_times);
if (ok && config && config->use_metadata) {
FileAttrPolicy link_policy = file_attr_policy_from_config(config);
ok = file_restore_symlink_metadata(link_path, file->metadata, link_policy,
config->omit_link_times);
}
free(link_path);
return ok ? FILE_SAVE_WRITTEN : FILE_SAVE_ERROR;
}
@@ -748,11 +779,13 @@ FileSaveResult file_save_to_disk_full(const char* root_directory, const File* fi
return file_save_hardlink_sibling(root_directory, file, config);
}
/* These options arrive from the client. They are names below the server
root, never independent filesystem roots. --temp-dir is confined exactly
like --backup-dir/--partial-dir: an absolute or `..`-escaping scratch
directory is rejected outright so nothing is ever created outside the
authorized destination root. */
/* These options arrive from the client. --backup-dir, --partial-dir and
--temp-dir are names below the server root, never independent filesystem
roots: an absolute or `..`-escaping value is rejected outright (rsync's
daemon confines temp-dir to the module the same way). A relative temp dir
is resolved under the receive root below; if that resolution still lands on
a different filesystem than the destination the install falls back to a
non-atomic copy (see file_to_disk_secure_impl), never an abort. */
if ((backup_dir && (backup_dir[0] == '/' || has_path_traversal(backup_dir))) ||
(partial_dir && (partial_dir[0] == '/' || has_path_traversal(partial_dir))) ||
(temp_dir && (temp_dir[0] == '/' || has_path_traversal(temp_dir))))
@@ -880,11 +913,13 @@ FileSaveResult file_save_to_disk_full(const char* root_directory, const File* fi
}
/* A configured --temp-dir sends the temporary working copy to a scratch
directory resolved below the receive root; the engine then atomically
renames the completed file into the final destination directory. The
partial-dir flow already keeps its working copy in a separate directory
and --inplace writes directly, so neither diverts through the scratch
dir (matching rsync, where --inplace/--partial-dir supersede --temp-dir). */
directory; the engine then atomically renames the completed file into the
final destination directory. A relative temp dir is resolved under the
receive root and must already exist (an absolute or `..`-escaping value was
rejected above); the engine falls back to a non-atomic copy on EXDEV. The
partial-dir flow already keeps its working copy in a separate directory and
--inplace writes directly, so neither diverts through the scratch dir
(matching rsync, where --inplace/--partial-dir supersede --temp-dir). */
char* confined_temp = NULL;
bool use_temp_dir = temp_dir != NULL && !inplace && !use_partial_root;
if (use_temp_dir) {
@@ -904,16 +939,15 @@ FileSaveResult file_save_to_disk_full(const char* root_directory, const File* fi
policy decision. */
bool ok;
if (config && file->basis_link) {
ok = file_to_disk_secure_link_attrs(disk_path, file->basis_link, file->data->data,
file->data->size, config->preallocate, metadata,
preserve_executability, config->use_fsync, file->xattrs,
config->fake_super, confined_temp);
ok = file_to_disk_secure_link_attrs(
disk_path, file->basis_link, file->data->data, file->data->size, config->preallocate,
metadata, policy, config->use_fsync, file->xattrs, config->fake_super, confined_temp);
} else {
/* The plain no-replace / update / with-fsync engines, plus per-file xattr
(-X/-A) and --fake-super application on the written fd. */
ok = file_to_disk_secure_attrs(
disk_path, file->data->data, file->data->size, inplace, sparse,
config && config->preallocate, metadata, preserve_executability, config && config->update,
config && config->preallocate, metadata, policy, config && config->update,
config && config->ignore_existing, config && config->use_fsync, file->xattrs,
config ? config->fake_super : false, config ? config->partial : false, confined_temp);
}
@@ -1842,6 +1876,33 @@ static IncrementalCheckOutcome incremental_check_open_destination(IncrementalChe
return INCREMENTAL_CONTINUE;
}
/* Output parity (protocol 2.23.0): when the wire config asked for it, report a
snapshot of the pre-transfer destination entry BEFORE the ordinary verdict so
the sender can render rsync-accurate -i/--out-format columns. A missing
destination is reported explicitly (existed=false) rather than omitted, so
the sender can distinguish "new" from "unknown". */
static IncrementalCheckOutcome incremental_check_report_dest_info(IncrementalCheckState* state) {
if (!state->config->report_dest_info)
return INCREMENTAL_CONTINUE;
OutputDestState info;
memset(&info, 0, sizeof(info));
info.known = true;
info.existed = state->has_old_file;
if (state->has_old_file) {
info.size = (unsigned long long)state->old_st.st_size;
info.mtime_sec = (long long)state->old_st.st_mtime;
#ifdef __linux__
info.mtime_nsec = state->old_st.st_mtim.tv_nsec;
#endif
info.mode = (uint32_t)state->old_st.st_mode;
info.uid = (int32_t)state->old_st.st_uid;
info.gid = (int32_t)state->old_st.st_gid;
}
if (!send_status(state->fd, STATUS_DEST_INFO) || !format_dest_state_send(state->fd, &info))
return INCREMENTAL_ERROR;
return INCREMENTAL_CONTINUE;
}
/* Metadata-only (and, when --checksum forces it, content) up-to-date decision.
Loads the old contents only when a checksum comparison or delta needs them. */
static IncrementalCheckOutcome incremental_check_quick_skip(IncrementalCheckState* state,
@@ -2286,6 +2347,10 @@ File* receive_incremental_check_ex(int fd, const Config* config, bool* skipped,
if (outcome == INCREMENTAL_ERROR)
goto done;
outcome = incremental_check_report_dest_info(&state);
if (outcome == INCREMENTAL_ERROR)
goto done;
outcome = incremental_check_quick_skip(&state, &try_delta);
if (outcome == INCREMENTAL_ERROR)
goto done;
@@ -2401,10 +2466,18 @@ File* file_receive(const Config* config, int file_descriptor) {
return file;
}
/* ---- P7 Wave D: deferred directory times ---- */
/* ---- P7 Wave D: deferred directory metadata ---- */
bool dir_times_should_capture(const Config* config) {
return config->use_metadata && !config->omit_dir_times;
bool dir_metadata_should_capture(const Config* config) {
/* Directory metadata is captured when a directory attribute is actually
* requested: -p/--perms (directory modes), -t/--times (directory mtimes,
* unless -O/--omit-dir-times suppresses them), -o/-g (directory ownership),
* or -X/-A (directory xattrs/ACLs). --atimes/-U alone does not pull
* directory metadata (matching the original dir-time bundle). */
return config && config->use_metadata &&
(config->preserve_perms || (config->preserve_times && !config->omit_dir_times) ||
config->preserve_owner || config->preserve_group || config->preserve_xattrs ||
config->preserve_acls);
}
void dir_time_list_init(DirTimeList* list) {
@@ -2412,6 +2485,7 @@ void dir_time_list_init(DirTimeList* list) {
return;
list->paths = NULL;
list->entries = NULL;
list->xattrs = NULL;
list->count = 0;
list->capacity = 0;
list->bytes = 0;
@@ -2420,18 +2494,23 @@ void dir_time_list_init(DirTimeList* list) {
void dir_time_list_free(DirTimeList* list) {
if (!list)
return;
for (size_t i = 0; i < list->count; i++)
for (size_t i = 0; i < list->count; i++) {
free(list->paths[i]);
xattr_list_free(list->xattrs ? list->xattrs[i] : NULL);
}
free(list->paths);
free(list->entries);
free(list->xattrs);
list->paths = NULL;
list->entries = NULL;
list->xattrs = NULL;
list->count = 0;
list->capacity = 0;
list->bytes = 0;
}
bool dir_time_list_add(DirTimeList* list, const char* wire_path, const FileMetadata* metadata) {
bool dir_time_list_add(DirTimeList* list, const char* wire_path, const FileMetadata* metadata,
const FileXattrList* xattrs) {
if (!list || !wire_path || !metadata)
return true; /* nothing to remember; never a hard error */
/* Cumulative, not per-frame: the sender may stream a tree across unbounded
@@ -2440,9 +2519,14 @@ bool dir_time_list_add(DirTimeList* list, const char* wire_path, const FileMetad
transfer, which becomes a clean protocol error). */
size_t path_len = strlen(wire_path);
/* Charge the whole per-entry cost (path copy + pointer slot + metadata
struct), not just the path, so the array growth is bounded by the same
cumulative budget. */
size_t entry_cost = path_len + sizeof(FileMetadata) + sizeof(char*);
struct + captured xattrs), not just the path, so the array growth is
bounded by the same cumulative budget. */
size_t xattr_cost = 0;
if (xattrs) {
for (int i = 0; i < xattrs->count; i++)
xattr_cost += strlen(xattrs->items[i].name) + xattrs->items[i].value_len + sizeof(FileXattr);
}
size_t entry_cost = path_len + sizeof(FileMetadata) + 2 * sizeof(char*) + xattr_cost;
if (list->count >= MAX_DIR_TIME_ENTRIES || entry_cost > MAX_DIR_TIME_BYTES - list->bytes)
return false;
if (list->count == list->capacity) {
@@ -2451,10 +2535,9 @@ bool dir_time_list_add(DirTimeList* list, const char* wire_path, const FileMetad
return false;
/* Assign each grown array as soon as its realloc succeeds: the old block is
already freed by then, so discarding the pointer would dangle. capacity
is advanced only after BOTH reallocs succeed, so a partial failure leaves
capacity no larger than the entries allocation (the paths array may be
over-allocated, which is harmless) -- never a mismatched list the next
add could write past. */
is advanced only after ALL reallocs succeed, so a partial failure leaves
capacity no larger than the smallest allocation -- never a mismatched
list the next add could write past. */
char** grown_paths = realloc(list->paths, new_capacity * sizeof(char*));
if (!grown_paths)
return false;
@@ -2463,20 +2546,40 @@ bool dir_time_list_add(DirTimeList* list, const char* wire_path, const FileMetad
if (!grown_entries)
return false;
list->entries = grown_entries;
FileXattrList** grown_xattrs = realloc(list->xattrs, new_capacity * sizeof(FileXattrList*));
if (!grown_xattrs)
return false;
list->xattrs = grown_xattrs;
list->capacity = new_capacity;
}
char* copy = str_dup(wire_path);
if (!copy)
return false;
FileXattrList* xattr_copy = xattr_list_clone(xattrs);
if (xattrs && !xattr_copy) {
free(copy);
return false;
}
list->paths[list->count] = copy;
list->entries[list->count] = *metadata;
list->xattrs[list->count] = xattr_copy;
list->count++;
list->bytes += entry_cost;
return true;
}
void dir_time_list_apply(const DirTimeList* list, const char* root_directory) {
if (!list || !root_directory)
void dir_metadata_list_apply(const DirTimeList* list, const char* root_directory,
const Config* config) {
if (!list || !root_directory || !config)
return;
bool apply_times = config->preserve_times && !config->omit_dir_times;
bool apply_mode = config->preserve_perms;
bool apply_xattrs = config->use_xattrs;
/* Ownership is applied through the active identity snapshot (which no-ops
* unless an ownership request is active), and xattrs only when -X/-A was
* negotiated. Times/mode keep their own per-attribute gates. */
bool have_any = apply_times || apply_mode || apply_xattrs || identity_active_enabled();
if (!have_any)
return;
for (size_t i = 0; i < list->count; i++) {
char* dir_path = path_cat(root_directory, list->paths[i]);
@@ -2484,7 +2587,8 @@ void dir_time_list_apply(const DirTimeList* list, const char* root_directory) {
continue;
char* leaf = NULL;
/* The parent walk is fd-relative and O_NOFOLLOW, so a symlink planted in a
parent component can never redirect the utimensat outside the root. */
parent component can never redirect the utimensat/chmod outside the
root. */
int parent_fd = file_open_secure_parent(dir_path, &leaf, false);
if (parent_fd < 0) {
free(dir_path);
@@ -2493,8 +2597,8 @@ void dir_time_list_apply(const DirTimeList* list, const char* root_directory) {
/* A dir-time entry only records metadata: the directory is (deliberately)
not created from it, so an empty source directory (or one pruned by
-m/--prune-empty-dirs) may well not exist here. Skip absent paths
QUIETLY rather than warning for every one, and apply the times only to a
real directory that does exist. AT_SYMLINK_NOFOLLOW keeps a same-named
QUIETLY rather than warning for every one, and apply the metadata only to
a real directory that does exist. AT_SYMLINK_NOFOLLOW keeps a same-named
symlink from being followed; a pre-existing regular file/symlink is not a
directory, so it is left completely untouched. */
struct stat st;
@@ -2504,19 +2608,62 @@ void dir_time_list_apply(const DirTimeList* list, const char* root_directory) {
free(dir_path);
continue;
}
struct timespec times[2] = {
{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
{.tv_sec = list->entries[i].mtime_sec, .tv_nsec = list->entries[i].mtime_nsec}};
if (list->entries[i].atime_valid) {
times[0].tv_sec = list->entries[i].atime_sec;
times[0].tv_nsec = list->entries[i].atime_nsec;
/* One O_DIRECTORY|O_NOFOLLOW fd drives ownership/mode/xattr application so
none of them can follow a same-named symlink planted after the fstatat. */
int dir_fd = openat(parent_fd, leaf, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
/* Ownership first: a chown clears setuid/setgid, so it must precede mode. */
if (dir_fd >= 0)
identity_apply_ownership(dir_fd, (int32_t)list->entries[i].uid,
(int32_t)list->entries[i].gid);
if (apply_times) {
struct timespec times[2] = {
{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
{.tv_sec = list->entries[i].mtime_sec, .tv_nsec = list->entries[i].mtime_nsec}};
if (config->preserve_atimes && list->entries[i].atime_valid) {
times[0].tv_sec = list->entries[i].atime_sec;
times[0].tv_nsec = list->entries[i].atime_nsec;
}
if (utimensat(parent_fd, leaf, times, AT_SYMLINK_NOFOLLOW) != 0) {
char* escaped_path = output_escape(dir_path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "Failed to set directory timestamps on %s: %s",
escaped_path ? escaped_path : "<allocation failed>", strerror(errno));
free(escaped_path);
}
}
if (utimensat(parent_fd, leaf, times, AT_SYMLINK_NOFOLLOW) != 0) {
char* escaped_path = output_escape(dir_path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "Failed to set directory timestamps on %s: %s",
escaped_path ? escaped_path : "<allocation failed>", strerror(errno));
free(escaped_path);
if (apply_mode) {
mode_t dir_mode = list->entries[i].mode;
bool mode_ready = true;
if (config->chmod_spec && *config->chmod_spec &&
!chmod_apply(dir_mode, config->chmod_spec, &dir_mode)) {
char* escaped_path = output_escape(dir_path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "Failed to apply --chmod to directory %s",
escaped_path ? escaped_path : "<allocation failed>");
free(escaped_path);
mode_ready = false;
}
if (mode_ready) {
/* rsync -p copies the source directory mode exactly, including
* group/other write and the setgid/sticky bits. */
mode_t safe_mode = dir_mode & (mode_t)(S_ISUID | S_ISGID | S_ISVTX | 0777);
if (dir_fd < 0) {
char* escaped_path = output_escape(dir_path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "Failed to open directory %s to set its mode: %s",
escaped_path ? escaped_path : "<allocation failed>", strerror(errno));
free(escaped_path);
} else if (fchmod(dir_fd, safe_mode) != 0) {
char* escaped_path = output_escape(dir_path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "Failed to set directory mode on %s: %s",
escaped_path ? escaped_path : "<allocation failed>", strerror(errno));
free(escaped_path);
}
}
}
/* xattrs/ACLs last: a mode change can rewrite the ACL mask, so the ACL
xattrs must be (re)applied after fchmod. */
if (apply_xattrs && dir_fd >= 0 && list->xattrs)
xattr_apply_fd(dir_fd, list->xattrs[i]);
if (dir_fd >= 0)
close(dir_fd);
close(parent_fd);
free(leaf);
free(dir_path);
@@ -2556,6 +2703,11 @@ File* file_receive_directory(int file_descriptor, const Config* config) {
return NULL;
}
}
/* Directory xattrs/ACLs (-X/-A) ride after the metadata when negotiated. */
if (config && !receive_file_xattrs(file, file_descriptor, config)) {
file_destroy(file);
return NULL;
}
return file;
}
@@ -2592,6 +2744,12 @@ File* file_receive_dir_time(int file_descriptor, const Config* config) {
return NULL;
}
}
/* Directory xattrs/ACLs (-X/-A) ride after the metadata, mirroring the
sender's send_dir_times(). */
if (config && !receive_file_xattrs(file, file_descriptor, config)) {
file_destroy(file);
return NULL;
}
return file;
}
@@ -2767,8 +2925,10 @@ File* file_receive_special(int file_descriptor) {
/* Read a delete-manifest frame (the STATUS_MANIFEST leading code has already
been consumed): a keep-set entry count followed by that many
destination-relative paths, then a protected-prefix count followed by that
many destination-relative prefixes, then (protocol 2.10.0+) a missing-args
count followed by that many destination-relative delete paths. The frame is
many destination-relative prefixes, then a missing-args count followed by that
many destination-relative delete paths, then (protocol 2.23.0) a
synchronized-directory count followed by that many destination-relative
directory paths (the receive root is the "." sentinel). The frame is
self-delimiting (the counts are authoritative), so the caller decides what to
do next and continues reading the following STATUS_* frame. Every section is
validated identically: an entry must be non-empty, relative and traversal-free
@@ -2813,7 +2973,8 @@ DeleteManifest* receive_manifest_entries(int fd) {
manifest->keeps = array_list_create(free);
manifest->protected = array_list_create(free);
manifest->missing = array_list_create(free);
if (!manifest->keeps || !manifest->protected || !manifest->missing) {
manifest->dirs = array_list_create(free);
if (!manifest->keeps || !manifest->protected || !manifest->missing || !manifest->dirs) {
delete_manifest_free(manifest);
send_status(fd, STATUS_ERROR);
return NULL;
@@ -2822,7 +2983,8 @@ DeleteManifest* receive_manifest_entries(int fd) {
size_t manifest_entries = 0;
if (!receive_manifest_section(fd, manifest->keeps, &manifest_bytes, &manifest_entries) ||
!receive_manifest_section(fd, manifest->protected, &manifest_bytes, &manifest_entries) ||
!receive_manifest_section(fd, manifest->missing, &manifest_bytes, &manifest_entries)) {
!receive_manifest_section(fd, manifest->missing, &manifest_bytes, &manifest_entries) ||
!receive_manifest_section(fd, manifest->dirs, &manifest_bytes, &manifest_entries)) {
delete_manifest_free(manifest);
return NULL;
}
@@ -2835,18 +2997,33 @@ void delete_manifest_free(DeleteManifest* manifest) {
array_list_delete(manifest->keeps);
array_list_delete(manifest->protected);
array_list_delete(manifest->missing);
array_list_delete(manifest->dirs);
free(manifest);
}
/* Shared --max-delete budget for one receiver-side deletion commit. Both the
--delete-missing-args exact-path removals and the ordinary extras walk draw
from the same tally, matching rsync (whose --max-delete counts every deleted
file or directory). `max_delete` is SIZE_MAX for an unlimited budget. */
typedef struct {
size_t max_delete;
size_t deleted;
size_t skipped;
bool limit_hit;
} DeleteBudgetState;
/* Remove every destination entry under the receive root that is not in the
keep-set, bounded by MAX_SERVER_DELETE_COUNT (or a smaller client
--max-delete=NUM, which is all-or-nothing), using the symlink-safe delete
walker. With --delay-updates the not-yet-published staging directory is a
keep-set, bounded by the shared budget (a smaller client --max-delete=NUM
replaces the server hard bound; rsync deletes up to the bound and skips the
rest). With --delay-updates the not-yet-published staging directory is a
direct child of the receive root and must not be treated as a set of extras;
the manifest's protected prefixes (paths excluded on the source) and the
the manifest's protected prefixes (paths excluded on the source), the
size-pruned prefixes (--max-size/--min-size, always protected) and the
alternate basis directories are never destination content and are skipped at
any depth. Prints a notice and returns true on success. */
bool manifest_delete_extras(const Config* config, DeleteManifest* manifest) {
any depth. Returns true unless a traversal/unlink error aborted the walk;
the budget's limit_hit/skipped fields report a cap-stopped run. */
static bool delete_extras_budgeted(const Config* config, DeleteManifest* manifest,
DeleteBudgetState* budget) {
if (!config || !manifest || !manifest->keeps)
return false;
fprintf(stderr, "Deleting files not in manifest...\n");
@@ -2858,9 +3035,10 @@ bool manifest_delete_extras(const Config* config, DeleteManifest* manifest) {
at any depth: they are extra comparison snapshots the user pointed at,
not destination content, and deleting them would destroy the very files a
--link-dest run just linked into place;
- the sender-side protected prefixes (source paths excluded by filters), at
any depth, so an excluded destination mirror survives --delete unless
--delete-excluded opts back into removing it. */
- the sender-side protected prefixes (source paths excluded by filters and
paths pruned by --max-size/--min-size), at any depth, so their destination
mirror survives --delete unless --delete-excluded opts back into removing
the filter-excluded ones (size-pruned entries are always protected). */
int skip_count = (config->delay_updates ? 1 : 0) + config->basis_count +
(manifest->protected ? manifest->protected->size : 0);
DeleteSkipEntry* skips = NULL;
@@ -2885,30 +3063,26 @@ bool manifest_delete_extras(const Config* config, DeleteManifest* manifest) {
idx++;
}
}
/* A client --max-delete=NUM smaller than the server's hard bound replaces it
for this run; both still bound the walk. The walker is all-or-nothing, so
a run that would delete more than the bound removes nothing and fails with
an error that names the bound that was hit. */
bool user_limited =
config->max_delete >= 0 && (size_t)config->max_delete < MAX_SERVER_DELETE_COUNT;
size_t cap = user_limited ? (size_t)config->max_delete : MAX_SERVER_DELETE_COUNT;
size_t deleted_count = 0;
DeleteWalkResult result = delete_extras_limited(config->receive_root_directory, manifest->keeps,
cap, skips, skip_count, &deleted_count);
/* Clamp rather than subtract: an accounting bug where deleted already exceeds
max_delete must never underflow into an effectively unlimited budget. */
size_t remaining;
if (budget->max_delete == SIZE_MAX)
remaining = SIZE_MAX;
else if (budget->deleted >= budget->max_delete)
remaining = 0;
else
remaining = budget->max_delete - budget->deleted;
size_t deleted = 0;
size_t skipped = 0;
DeleteWalkResult result =
delete_extras_limited(config->receive_root_directory, manifest->keeps, manifest->dirs,
remaining, skips, skip_count, &deleted, &skipped);
free(skips);
if (result == DELETE_WALK_LIMIT_EXCEEDED) {
if (user_limited) {
log_message(LOG_LEVEL_ERROR,
"deletion stopped: the destination holds more than --max-delete=%d extraneous "
"entries; no files were deleted",
config->max_delete);
} else {
log_message(LOG_LEVEL_ERROR,
"deletion stopped: the destination holds more than %u extraneous entries "
"(server deletion limit); no files were deleted",
(unsigned)MAX_SERVER_DELETE_COUNT);
}
return false;
budget->deleted += deleted;
budget->skipped += skipped;
if (result == DELETE_WALK_LIMIT_REACHED) {
budget->limit_hit = true;
return true;
}
if (result != DELETE_WALK_OK) {
log_message(LOG_LEVEL_ERROR, "deletion failed while removing extraneous files");
@@ -2926,10 +3100,12 @@ bool manifest_delete_extras(const Config* config, DeleteManifest* manifest) {
removed recursively only when --delete or --force is in effect (rsync parity:
the man page says a non-empty directory mirror is only deleted with --force
or --delete); otherwise it is left with a warning and the run continues. A
mirror that does not exist is a no-op. Returns false only on a genuine error
(a confinement failure on a validated path or an I/O error), which fails the
run. */
bool manifest_delete_missing_args(const Config* config, DeleteManifest* manifest) {
mirror that does not exist is a no-op. Each removal draws from the shared
--max-delete budget: once it is exhausted the remaining requests are skipped
and counted. Returns false only on a genuine error (a confinement failure on
a validated path or an I/O error), which fails the run. */
static bool delete_missing_args_budgeted(const Config* config, DeleteManifest* manifest,
DeleteBudgetState* budget) {
if (!config || !manifest)
return false;
if (!manifest->missing || manifest->missing->size == 0)
@@ -3003,6 +3179,16 @@ bool manifest_delete_missing_args(const Config* config, DeleteManifest* manifest
free(full);
continue;
}
/* An entry that exists is one deletion: skip it (and count it) when the
shared --max-delete budget is already exhausted. */
if (budget->deleted >= budget->max_delete) {
budget->limit_hit = true;
budget->skipped++;
close(parent_fd);
free(leaf);
free(full);
continue;
}
bool removed = false;
if (S_ISDIR(st.st_mode)) {
if (unlinkat(parent_fd, leaf, AT_REMOVEDIR) == 0) {
@@ -3013,10 +3199,40 @@ bool manifest_delete_missing_args(const Config* config, DeleteManifest* manifest
free(leaf);
leaf = NULL;
if (config->use_delete || config->force_delete) {
if (!file_remove_tree_secure(full))
/* Remove the contents entry-by-entry through the budgeted extras
walker so every deleted file/dir counts toward --max-delete (rsync
parity); the now-empty directory itself costs one more. A run that
hits the cap leaves the remaining entries in place. */
ArrayList* no_keeps = array_list_create(free);
/* Never let an accounting slip (deleted > max_delete) underflow the
remaining budget into SIZE_MAX, which would grant unlimited
deletions. */
size_t remaining =
budget->deleted >= budget->max_delete ? 0 : budget->max_delete - budget->deleted;
size_t contents_deleted = 0;
size_t contents_skipped = 0;
DeleteWalkResult walk =
no_keeps ? delete_extras_limited(full, no_keeps, NULL, remaining, NULL, 0,
&contents_deleted, &contents_skipped)
: DELETE_WALK_ERROR;
if (no_keeps)
array_list_delete(no_keeps);
budget->deleted += contents_deleted;
budget->skipped += contents_skipped;
if (walk == DELETE_WALK_LIMIT_REACHED) {
budget->limit_hit = true;
} else if (walk != DELETE_WALK_OK) {
ok = false;
else
} else if (budget->deleted >= budget->max_delete) {
budget->limit_hit = true;
budget->skipped++;
} else if (file_remove_tree_secure(full)) {
/* The shared `if (removed)` tail charges this directory exactly
once; counting it here too would consume two budget units. */
removed = true;
} else {
ok = false;
}
} else {
char* escaped = output_escape(rel, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING,
@@ -3036,6 +3252,7 @@ bool manifest_delete_missing_args(const Config* config, DeleteManifest* manifest
}
}
if (removed) {
budget->deleted++;
char* escaped = output_escape(rel, log_get_8_bit_output());
fprintf(stderr, " Deleted: %s\n", escaped ? escaped : "<allocation failed>");
free(escaped);
@@ -3051,24 +3268,58 @@ bool manifest_delete_missing_args(const Config* config, DeleteManifest* manifest
return ok;
}
/* Public wrappers used outside the commit path (and by unit tests): no
--max-delete budget. */
bool manifest_delete_extras(const Config* config, DeleteManifest* manifest) {
DeleteBudgetState budget = {
.max_delete = SIZE_MAX, .deleted = 0, .skipped = 0, .limit_hit = false};
return delete_extras_budgeted(config, manifest, &budget);
}
bool manifest_delete_missing_args(const Config* config, DeleteManifest* manifest) {
DeleteBudgetState budget = {
.max_delete = SIZE_MAX, .deleted = 0, .skipped = 0, .limit_hit = false};
return delete_missing_args_budgeted(config, manifest, &budget);
}
/* Commit every deletion family the manifest carries. The --delete-missing-args
exact-path deletions run FIRST: they are explicit user requests and must not
be blocked by the extras walker's filter-exclusion protection (a protected
leftover inside a missing-argument directory must not make that user-requested
removal fail). The ordinary extras walk then runs when --delete is active.
Returns true when there was nothing to do or every requested deletion
committed. */
bool manifest_delete_all(const Config* config, DeleteManifest* manifest) {
Both draw from one --max-delete budget; the result reports a cap-stopped
(partial) commit distinctly so the client can exit 25 like rsync. */
DeleteCommitResult manifest_delete_all(const Config* config, DeleteManifest* manifest) {
if (!config || !manifest)
return false;
return DELETE_COMMIT_ERROR;
/* Central no-mutation guard: a dry-run never deletes. No manifest is sent on
the dry-run path, but a hostile/buggy peer could; treat it as a no-op so
the receiver can never remove anything. */
if (config->dry_run)
return true;
if (config->delete_missing_args && !manifest_delete_missing_args(config, manifest))
return false;
if (config->use_delete && !manifest_delete_extras(config, manifest))
return false;
return true;
return DELETE_COMMIT_OK;
/* A client --max-delete=NUM smaller than the server's hard bound replaces it
for this run; both still bound the commit. */
bool user_limited =
config->max_delete >= 0 && (size_t)config->max_delete < MAX_SERVER_DELETE_COUNT;
DeleteBudgetState budget = {.max_delete = user_limited ? (size_t)config->max_delete
: MAX_SERVER_DELETE_COUNT,
.deleted = 0,
.skipped = 0,
.limit_hit = false};
if (config->delete_missing_args && !delete_missing_args_budgeted(config, manifest, &budget))
return DELETE_COMMIT_ERROR;
if (config->use_delete && !delete_extras_budgeted(config, manifest, &budget))
return DELETE_COMMIT_ERROR;
if (budget.limit_hit) {
if (user_limited) {
log_message(LOG_LEVEL_ERROR, "Deletions stopped due to --max-delete limit (%zu skipped)",
budget.skipped);
} else {
log_message(LOG_LEVEL_ERROR,
"Deletions stopped due to the server deletion limit of %u (%zu skipped)",
(unsigned)MAX_SERVER_DELETE_COUNT, budget.skipped);
}
return DELETE_COMMIT_LIMIT_REACHED;
}
return DELETE_COMMIT_OK;
}
+47 -21
View File
@@ -40,31 +40,39 @@ File* receive_incremental_check_ex(int fd, const Config* config, bool* skipped,
* parent's mtime). -O/--omit-dir-times skips the application entirely. The
* list owns deep copies of the paths and metadata; freed on every path. */
typedef struct {
char** paths; /* owned, destination-relative wire paths */
FileMetadata* entries; /* owned, parallel to paths */
char** paths; /* owned, destination-relative wire paths */
FileMetadata* entries; /* owned, parallel to paths */
FileXattrList** xattrs; /* owned, parallel to paths; NULL when none */
size_t count;
size_t capacity;
size_t bytes; /* cumulative strlen of every retained path */
} DirTimeList;
/* Capture gate shared by the sender-side and receiver-side sinks: directory
* metadata is accumulated only when --times/--metadata is in effect and
* -O/--omit-dir-times does not suppress it. Kept here, next to the accumulator
* it guards, so both call sites express the same condition. */
bool dir_times_should_capture(const Config* config);
* metadata is accumulated only when a directory attribute is requested
* (-p/--perms for directory modes, or -t/--times for directory mtimes with
* -O/--omit-dir-times not suppressing them) and metadata rides the wire. Kept
* here, next to the accumulator it guards, so both call sites express the same
* condition. */
bool dir_metadata_should_capture(const Config* config);
void dir_time_list_init(DirTimeList* list);
void dir_time_list_free(DirTimeList* list);
/* Deep-copy one directory's path + metadata into the list. Returns false on
* allocation failure OR when the cumulative entry/byte caps would be exceeded
* (the caller fails the transfer). */
bool dir_time_list_add(DirTimeList* list, const char* wire_path, const FileMetadata* metadata);
/* Apply every accumulated directory's mtime (and atime when captured) beneath
* `root_directory`, confined fd-relative. Best-effort per entry: an absent
* directory (an empty/pruned source dir that was deliberately not created) or a
* non-directory at the path is skipped QUIETLY, an unreachable one with a
* warning, and never fatal. */
void dir_time_list_apply(const DirTimeList* list, const char* root_directory);
/* Deep-copy one directory's path + metadata (and, when non-NULL, its captured
* xattr/ACL block) into the list. Returns false on allocation failure OR when
* the cumulative entry/byte caps would be exceeded (the caller fails the
* transfer). */
bool dir_time_list_add(DirTimeList* list, const char* wire_path, const FileMetadata* metadata,
const FileXattrList* xattrs);
/* Apply every accumulated directory's metadata beneath `root_directory`,
* confined fd-relative: ownership through the negotiated identity policy,
* times (mtime, plus atime when -U captured one under -t), the mode (through
* --chmod when configured, under -p), and the captured xattrs/ACLs (under
* -X/-A). Best-effort per entry: an absent directory (an empty/pruned source
* dir that was deliberately not created) or a non-directory at the path is
* skipped QUIETLY, an unreachable one with a warning, and never fatal. */
void dir_metadata_list_apply(const DirTimeList* list, const char* root_directory,
const Config* config);
/* A received delete-manifest frame: the keep-set (`keeps`, destination-relative
paths the sender transferred/keeps) plus `protected`, destination-relative
@@ -80,11 +88,18 @@ typedef struct DeleteManifest {
ArrayList* keeps;
ArrayList* protected;
ArrayList* missing;
/* Destination-relative paths of the directories the sender synchronized for
this run. The extras walker only removes entries directly inside one of
these (the receive root is the "." sentinel); `--files-from` runs therefore
leave untransmitted directories and the unlisted parts of listed ones
alone, matching rsync's "delete only in synchronized directories". */
ArrayList* dirs;
} DeleteManifest;
void delete_manifest_free(DeleteManifest* manifest);
/* Read a delete-manifest frame: keep count + keeps, then protected count +
protected prefixes, then missing count + missing paths (self-delimiting; the
/* Read a delete-manifest frame (protocol 2.23.0): keep count + keeps, then
protected count + protected prefixes, then missing count + missing paths,
then synchronized-directory count + directory paths (self-delimiting; the
leading STATUS_MANIFEST code has been consumed). Returns an owned
DeleteManifest, or NULL after signalling STATUS_ERROR on a malformed frame. */
DeleteManifest* receive_manifest_entries(int fd);
@@ -103,11 +118,22 @@ bool manifest_delete_extras(const Config* config, DeleteManifest* manifest);
confinement or I/O error (the run then fails); tolerated per-path cases are
reported and skipped. */
bool manifest_delete_missing_args(const Config* config, DeleteManifest* manifest);
/* Outcome of committing a delete manifest. LIMIT_REACHED reports rsync's
partial --max-delete result: the budget allowed some deletions and the rest
were skipped (the run still stores all file data but the client exits 25). */
typedef enum {
DELETE_COMMIT_OK = 0,
DELETE_COMMIT_LIMIT_REACHED,
DELETE_COMMIT_ERROR
} DeleteCommitResult;
/* Run every deletion family the manifest carries: the --delete-missing-args
exact-path deletions first (user requests are not blocked by exclusion
protection), then the ordinary extras walk when --delete is active. Returns
true when nothing to do or everything committed. */
bool manifest_delete_all(const Config* config, DeleteManifest* manifest);
protection), then the ordinary extras walk when --delete is active. Both
share one --max-delete budget. Returns DELETE_COMMIT_OK when nothing was to
do or everything committed, DELETE_COMMIT_LIMIT_REACHED when the budget
stopped part of the work, or DELETE_COMMIT_ERROR on a genuine failure. */
DeleteCommitResult manifest_delete_all(const Config* config, DeleteManifest* manifest);
/* Outcome of a single file_save_to_disk operation. The receiver needs to
distinguish "written" from "skipped" so --remove-source-files can be told
+18 -10
View File
@@ -143,20 +143,28 @@ bool file_send_sendfile_with_skip(File* file, int file_descriptor, bool use_meta
}
off_t offset = 0;
/* A non-positive --timeout disables the deadline: poll blocks until the
* socket is writable (rsync's --timeout=0 default). */
int io_timeout_sec = protocol_get_io_timeout_sec();
struct timespec deadline;
clock_gettime(CLOCK_MONOTONIC, &deadline);
deadline.tv_sec += protocol_get_io_timeout_sec();
if (io_timeout_sec > 0) {
clock_gettime(CLOCK_MONOTONIC, &deadline);
deadline.tv_sec += io_timeout_sec;
}
while ((unsigned long long)offset < file_size) {
struct timespec now;
clock_gettime(CLOCK_MONOTONIC, &now);
long long remaining = (long long)(deadline.tv_sec - now.tv_sec) * 1000LL +
(deadline.tv_nsec - now.tv_nsec) / 1000000LL;
if (remaining <= 0) {
close(fd);
return false;
int timeout = -1;
if (io_timeout_sec > 0) {
struct timespec now;
clock_gettime(CLOCK_MONOTONIC, &now);
long long remaining = (long long)(deadline.tv_sec - now.tv_sec) * 1000LL +
(deadline.tv_nsec - now.tv_nsec) / 1000000LL;
if (remaining <= 0) {
close(fd);
return false;
}
timeout = remaining > INT_MAX ? INT_MAX : (int)remaining;
}
struct pollfd pfd = {.fd = file_descriptor, .events = POLLOUT};
int timeout = remaining > INT_MAX ? INT_MAX : (int)remaining;
int polled = poll(&pfd, 1, timeout);
if (polled <= 0 || (pfd.revents & (POLLERR | POLLHUP | POLLNVAL))) {
close(fd);
+7
View File
@@ -2,6 +2,7 @@
#define FILE_TYPES_H
#include "data.h"
#include "format.h"
#include "xattr.h"
#include <stdbool.h>
#include <sys/stat.h>
@@ -86,6 +87,12 @@ typedef struct {
* Receiver: parsed off the wire, attached here, and applied fd-relative on
* the written file. NULL/0 == the file carries no xattrs. */
FileXattrList* xattrs;
/* Sender-side output-parity state (never serialized): the receiver-reported
* pre-transfer destination snapshot for this entry, filled by the per-file
* STATUS_CHECK exchange when report_dest_info is set. `known` is false when
* no report was requested/received, in which case -i/--out-format treats the
* entry conservatively as newly created. */
OutputDestState dest_state;
} File;
/* The path that should be sent on the wire and used for the receiver-side
+103
View File
@@ -0,0 +1,103 @@
#include "format.h"
#include "protocol.h"
#include <stdio.h>
#include <string.h>
bool format_human_size_decimal(unsigned long long bytes, char* buffer, size_t buffer_size) {
if (!buffer || buffer_size == 0)
return false;
if (bytes < 1000ULL) {
int written = snprintf(buffer, buffer_size, "%llu", bytes);
return written >= 0 && (size_t)written < buffer_size;
}
static const char units[] = "KMGTPE";
double value = (double)bytes;
size_t divisions = 0;
while (value >= 1000.0 && divisions < sizeof(units) - 1) {
value /= 1000.0;
divisions++;
}
int written = snprintf(buffer, buffer_size, "%.2f%c", value, units[divisions - 1]);
return written >= 0 && (size_t)written < buffer_size;
}
bool format_big_num(unsigned long long value, bool human_readable, char* buffer,
size_t buffer_size) {
if (human_readable)
return format_human_size_decimal(value, buffer, buffer_size);
char digits[32];
int written = snprintf(digits, sizeof(digits), "%llu", value);
if (written < 0 || (size_t)written >= sizeof(digits))
return false;
size_t len = (size_t)written;
size_t separators = len > 1 ? (len - 1) / 3 : 0;
size_t total = len + separators;
if (total + 1 > buffer_size)
return false;
size_t out = total;
buffer[out] = '\0';
size_t digits_since_sep = 0;
for (size_t i = len; i > 0; i--) {
buffer[--out] = digits[i - 1];
digits_since_sep++;
if (digits_since_sep == 3 && i > 1) {
buffer[--out] = ',';
digits_since_sep = 0;
}
}
return true;
}
bool format_rsync_datetime(time_t when, bool dash, char* buffer, size_t buffer_size) {
if (!buffer || buffer_size == 0)
return false;
struct tm broken_down;
if (localtime_r(&when, &broken_down) == NULL)
return false;
const char* format = dash ? "%Y/%m/%d-%H:%M:%S" : "%Y/%m/%d %H:%M:%S";
return strftime(buffer, buffer_size, format, &broken_down) != 0;
}
bool format_dest_state_send(int fd, const OutputDestState* state) {
if (!state)
return false;
int32_t has_old = state->existed ? 1 : 0;
uint64_t size = (uint64_t)state->size;
int64_t mtime = (int64_t)state->mtime_sec;
int64_t mtime_nsec = state->mtime_nsec;
uint32_t mode = state->mode;
int32_t uid = state->uid;
int32_t gid = state->gid;
return send_n_data(fd, &has_old, sizeof(has_old)) && send_n_data(fd, &size, sizeof(size)) &&
send_n_data(fd, &mtime, sizeof(mtime)) &&
send_n_data(fd, &mtime_nsec, sizeof(mtime_nsec)) && send_n_data(fd, &mode, sizeof(mode)) &&
send_n_data(fd, &uid, sizeof(uid)) && send_n_data(fd, &gid, sizeof(gid));
}
bool format_dest_state_receive(int fd, OutputDestState* state) {
if (!state)
return false;
int32_t has_old = 0;
uint64_t size = 0;
int64_t mtime = 0;
int64_t mtime_nsec = 0;
uint32_t mode = 0;
int32_t uid = 0;
int32_t gid = 0;
if (!receive_n_data(fd, &has_old, sizeof(has_old)) || !receive_n_data(fd, &size, sizeof(size)) ||
!receive_n_data(fd, &mtime, sizeof(mtime)) ||
!receive_n_data(fd, &mtime_nsec, sizeof(mtime_nsec)) ||
!receive_n_data(fd, &mode, sizeof(mode)) || !receive_n_data(fd, &uid, sizeof(uid)) ||
!receive_n_data(fd, &gid, sizeof(gid)))
return false;
memset(state, 0, sizeof(*state));
state->known = true;
state->existed = has_old != 0;
state->size = size;
state->mtime_sec = mtime;
state->mtime_nsec = mtime_nsec;
state->mode = mode;
state->uid = uid;
state->gid = gid;
return true;
}
+59
View File
@@ -0,0 +1,59 @@
#ifndef FORMAT_H
#define FORMAT_H
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#include <time.h>
/* Low-level output-formatting primitives shared by the change-event model
* (change_list.c) and the transfer driver (client_send.c).
*
* The functions here are pure/string-level except for the STATUS_DEST_INFO
* codec, which lets the receiver report the pre-transfer destination entry so
* the sender can render rsync-accurate --itemize-changes / --out-format
* columns (see protocol.h). */
/* Pre-transfer destination snapshot, reported by the receiver when the wire
* config carries report_dest_info. `known` distinguishes "no report was
* requested/received" from "the destination did not exist" (`existed == false`
* with `known == true`). */
typedef struct {
bool known;
bool existed;
unsigned long long size;
long long mtime_sec;
long long mtime_nsec;
uint32_t mode;
int32_t uid;
int32_t gid;
} OutputDestState;
/* rsync's -h/--human-readable size (decimal, base 1000): integers below 1000
* print verbatim; larger values use the largest unit that keeps the value
* below 1000 (K/M/G/T/P/E) with exactly two decimals, so 1500000 -> "1.50M"
* and 999999 -> "1000.00K" (matching rsync's human_num). Returns false when
* the buffer is too small (nothing is written). */
bool format_human_size_decimal(unsigned long long bytes, char* buffer, size_t buffer_size);
/* rsync's general number formatting (big_num). When `human_readable` is true
* this is format_human_size_decimal; otherwise the integer is rendered with a
* ',' thousands separator every three digits (rsync's separator in the C
* locale). Returns false on an undersized buffer. */
bool format_big_num(unsigned long long value, bool human_readable, char* buffer,
size_t buffer_size);
/* rsync's %M/%t timestamp. When `dash` is true the separator between the date
* and the time is '-' (the %M form: "YYYY/MM/DD-HH:MM:SS"); otherwise it is a
* space (the %t form: "YYYY/MM/DD HH:MM:SS"). Local time. Returns false on a
* bad time or an undersized buffer. */
bool format_rsync_datetime(time_t when, bool dash, char* buffer, size_t buffer_size);
/* Fixed-width STATUS_DEST_INFO record codec (int32 has_old, uint64 size,
* int64 mtime, int64 mtime_nsec, uint32 mode, int32 uid, int32 gid). The
* status frame itself is sent/received by the caller. Returns false on I/O
* failure. */
bool format_dest_state_send(int fd, const OutputDestState* state);
bool format_dest_state_receive(int fd, OutputDestState* state);
#endif
+411 -115
View File
@@ -36,14 +36,34 @@ typedef struct {
bool copy_as_set;
int32_t copy_as_uid;
int32_t copy_as_gid;
/* -o/--owner and -g/--group: preserve the source owner/group through the
* normal name/identity resolution path. Split out of the former
* use_metadata bundle; unlike --numeric-ids/--chown/--usermap/--groupmap/-a
* these are a preserve-source request, not an arbitrary client-chosen owner,
* so they are tracked separately from the explicit ownership gate. */
bool preserve_owner;
bool preserve_group;
/* --fake-super: when active the receiver must only RECORD the (resolved)
* ownership in the reserved xattr, never perform a real chown. Snapshotted
* so the fd-relative ownership helpers can suppress the chown without a
* Config argument. */
bool fake_super;
bool set;
} IdentityActive;
static IdentityActive g_identity;
static void identity_active_reset(void) {
free(g_identity.usermap);
free(g_identity.groupmap);
if (g_identity.usermap) {
for (int i = 0; i < g_identity.usermap_count; i++)
free(g_identity.usermap[i].to_name);
free(g_identity.usermap);
}
if (g_identity.groupmap) {
for (int i = 0; i < g_identity.groupmap_count; i++)
free(g_identity.groupmap[i].to_name);
free(g_identity.groupmap);
}
g_identity.usermap = NULL;
g_identity.groupmap = NULL;
g_identity.usermap_count = 0;
@@ -57,6 +77,9 @@ static void identity_active_reset(void) {
g_identity.copy_as_set = false;
g_identity.copy_as_uid = 0;
g_identity.copy_as_gid = 0;
g_identity.preserve_owner = false;
g_identity.preserve_group = false;
g_identity.fake_super = false;
g_identity.set = false;
}
@@ -77,32 +100,57 @@ bool identity_set_active(const Config* config) {
g_identity.copy_as_set = config->copy_as_set;
g_identity.copy_as_uid = config->copy_as_uid;
g_identity.copy_as_gid = config->copy_as_gid;
g_identity.preserve_owner = config->preserve_owner;
g_identity.preserve_group = config->preserve_group;
g_identity.fake_super = config->fake_super;
if (config->usermap_count > 0) {
g_identity.usermap = calloc((size_t)config->usermap_count, sizeof(IdentityMap));
if (!g_identity.usermap)
goto alloc_failed;
memcpy(g_identity.usermap, config->usermap,
(size_t)config->usermap_count * sizeof(IdentityMap));
for (int i = 0; i < config->usermap_count; i++) {
g_identity.usermap[i] = config->usermap[i];
g_identity.usermap[i].to_name =
config->usermap[i].to_name ? str_dup(config->usermap[i].to_name) : NULL;
if (config->usermap[i].to_name && !g_identity.usermap[i].to_name) {
g_identity.usermap_count = i; /* free only the entries already duplicated */
goto alloc_failed;
}
}
g_identity.usermap_count = config->usermap_count;
}
if (config->groupmap_count > 0) {
g_identity.groupmap = calloc((size_t)config->groupmap_count, sizeof(IdentityMap));
if (!g_identity.groupmap)
goto alloc_failed;
memcpy(g_identity.groupmap, config->groupmap,
(size_t)config->groupmap_count * sizeof(IdentityMap));
for (int i = 0; i < config->groupmap_count; i++) {
g_identity.groupmap[i] = config->groupmap[i];
g_identity.groupmap[i].to_name =
config->groupmap[i].to_name ? str_dup(config->groupmap[i].to_name) : NULL;
if (config->groupmap[i].to_name && !g_identity.groupmap[i].to_name) {
g_identity.groupmap_count = i;
goto alloc_failed;
}
}
g_identity.groupmap_count = config->groupmap_count;
}
g_identity.set = true;
/* A root receiver would honor any client-supplied ownership request (a
--usermap/--groupmap/--chown/--copy-as, or raw ids under --numeric-ids).
Surface that prominently; a privileged daemon applying arbitrary client
ownership is a deliberate, opt-in choice the operator should be aware of. */
if (geteuid() == 0)
log_message(LOG_LEVEL_WARNING,
"identity mapping active and running as root: client-supplied "
"ownership (usermap/groupmap/chown/numeric-ids) will be honored; "
"run the daemon as an unprivileged user unless intended");
--usermap/--groupmap/--chown/--copy-as, or raw ids under --numeric-ids)
ONLY when super-user activities are permitted. --no-super (or a daemon
veto that forced SUPER_MODE_OFF) forbids the chown even for root, so do
not claim the ownership will be honored in that case. */
if (geteuid() == 0) {
if (privilege_super_mode_permitted(g_identity.super_mode))
log_message(LOG_LEVEL_WARNING,
"identity mapping active and running as root: client-supplied "
"ownership (usermap/groupmap/chown/numeric-ids) will be honored; "
"run the daemon as an unprivileged user unless intended");
else
log_message(LOG_LEVEL_WARNING,
"identity mapping active and running as root, but super-user activities are "
"disabled (--no-super): requested ownership will NOT be applied; run the "
"daemon as an unprivileged user unless intended");
}
/* --super explicitly requests super-user activities, but FastSync never
elevates privileges: when the receiver is not already root the kernel will
refuse those confined attempts and each is skipped per entry. Warn exactly
@@ -138,25 +186,55 @@ bool privilege_super_mode_permitted(SuperMode mode) {
}
bool identity_active_enabled(void) {
/* numeric_ids is included: this set only gates identity_apply_ownership,
which runs only when metadata is present (a -M/--preserve transfer). A
standalone --numeric-ids (no ownership-affecting flag) carries no
metadata, never reaches identity_apply_ownership, and therefore correctly
stays inert; combined with -M it activates raw-id application. --super /
--no-super does NOT enable ownership: it only permits or forbids the
already-requested super-user activities, so a --super with no explicit
identity flag must never silently apply client-chosen ownership. */
/* --numeric-ids is deliberately NOT included: it is a mapping MODIFIER (use
* the transmitted numeric id raw instead of a name lookup), not a request to
* change ownership. rsync's --numeric-ids on its own never chowns anything;
* it only changes how an already-requested -o/-g/map resolves. Ownership is
* activated only by an explicit request: --chown/--usermap/--groupmap/
* --copy-as or a preserve-source -o/--owner / -g/--group. --super/--no-super
* likewise does NOT enable ownership: it only permits or forbids the
* already-requested super-user activities. */
return g_identity.set &&
(g_identity.numeric_ids || g_identity.chown_uid_set || g_identity.chown_gid_set ||
g_identity.usermap_count > 0 || g_identity.groupmap_count > 0 || g_identity.copy_as_set);
(g_identity.chown_uid_set || g_identity.chown_gid_set || g_identity.usermap_count > 0 ||
g_identity.groupmap_count > 0 || g_identity.copy_as_set || g_identity.preserve_owner ||
g_identity.preserve_group);
}
bool identity_owner_requested(void) {
return g_identity.set && (g_identity.copy_as_set || g_identity.chown_uid_set ||
g_identity.preserve_owner || g_identity.usermap_count > 0);
}
bool identity_group_requested(void) {
return g_identity.set && (g_identity.copy_as_set || g_identity.chown_gid_set ||
g_identity.preserve_group || g_identity.groupmap_count > 0);
}
bool identity_ownership_requested(const Config* config) {
if (!config)
return false;
/* Every value that makes the receiver act on a client-chosen owner, plus an
* explicit --super (super-user device-node activities). Pure config, so the
* daemon gate can evaluate it before identity_set_active(). */
/* General-awareness predicate: every value that makes the receiver act on a
* client-chosen owner, plus an explicit --super (super-user device-node
* activities) and the preserve-source -o/-g requests. Pure config, so callers
* can evaluate it before identity_set_active(). The daemon module gate uses
* the narrower identity_explicit_ownership_requested() below, which treats a
* plain -o/-g/-a as a preserve-source request rather than arbitrary
* client-chosen ownership. */
return config->numeric_ids || config->chown_uid_set || config->chown_gid_set ||
config->usermap_count > 0 || config->groupmap_count > 0 || config->copy_as_set ||
config->preserve_owner || config->preserve_group || config->fake_super ||
config->super_mode == SUPER_MODE_ON;
}
bool identity_explicit_ownership_requested(const Config* config) {
if (!config)
return false;
/* The narrow set the daemon gate refuses for a non-opted module: a request
* that lets the CLIENT choose an arbitrary owner/group (rather than preserve
* the source's own). Deliberately EXCLUDES preserve_owner/preserve_group so a
* plain -a/-o/-g push is not refused; for those the gate instead forces
* super-user ownership activity off (no chown happens) unless the module has
* `client owner = yes`. */
return config->numeric_ids || config->chown_uid_set || config->chown_gid_set ||
config->usermap_count > 0 || config->groupmap_count > 0 || config->copy_as_set ||
config->fake_super || config->super_mode == SUPER_MODE_ON;
@@ -177,6 +255,29 @@ bool identity_copy_as_refused(const Config* config) {
return geteuid() != 0 || config->super_mode == SUPER_MODE_OFF;
}
/* Validate one received FROM:TO map rule. `from` is a single id, the LOW end
* of an inclusive range, IDENTITY_MATCH_ANY, or IDENTITY_MATCH_UNNAMED; a
* sentinel FROM must carry the same value in from_hi. `to` is a non-negative
* id, IDENTITY_CURRENT, or ignored when a bounded receiver-resolved `to_name`
* is present. */
static bool identity_wire_map_valid(const IdentityMap* map) {
if (!map)
return false;
if (map->from < IDENTITY_MATCH_UNNAMED)
return false;
if (map->from < 0) {
if (map->from_hi != map->from)
return false;
} else if (map->from_hi < map->from) {
return false;
}
if (map->to < IDENTITY_CURRENT)
return false;
if (map->to_name && strlen(map->to_name) > 255)
return false;
return true;
}
bool identity_wire_valid(const Config* config) {
if (!config)
return false;
@@ -188,11 +289,11 @@ bool identity_wire_valid(const Config* config) {
if (config->chown_gid_set && config->chown_gid < IDENTITY_MATCH_ANY)
return false;
for (int i = 0; i < config->usermap_count; i++) {
if (config->usermap[i].from < IDENTITY_MATCH_ANY || config->usermap[i].to < IDENTITY_CURRENT)
if (!identity_wire_map_valid(&config->usermap[i]))
return false;
}
for (int i = 0; i < config->groupmap_count; i++) {
if (config->groupmap[i].from < IDENTITY_MATCH_ANY || config->groupmap[i].to < IDENTITY_CURRENT)
if (!identity_wire_map_valid(&config->groupmap[i]))
return false;
}
/* Defense-in-depth: a --copy-as block must never carry a negative (sentinel)
@@ -250,15 +351,137 @@ static int identity_resolve_token(const char* token, bool is_group, int32_t* out
return 0;
}
static int identity_append_rule(IdentityMap** map, int* count, int32_t from, int32_t to) {
static bool identity_all_digits(const char* token) {
if (!token || *token == '\0')
return false;
for (const char* p = token; *p; p++)
if (*p < '0' || *p > '9')
return false;
return true;
}
static bool identity_token_has_glob(const char* token) {
return token && (strchr(token, '*') || strchr(token, '?') || strchr(token, '['));
}
/* Parse a --usermap/--groupmap FROM token into a matcher (from/from_hi). rsync
* accepts a name, a numeric id, an inclusive LOW-HIGH range, '*' (any id), or an
* empty token (ids with no name on the sender). Returns 0 on success, -1 on a
* malformed token or an unresolvable sender-side name. */
static int identity_parse_from(const char* token, bool is_group, int32_t* out_from,
int32_t* out_hi) {
if (token[0] == '\0') {
*out_from = IDENTITY_MATCH_UNNAMED;
*out_hi = IDENTITY_MATCH_UNNAMED;
return 0;
}
if (strcmp(token, "*") == 0) {
*out_from = IDENTITY_MATCH_ANY;
*out_hi = IDENTITY_MATCH_ANY;
return 0;
}
const char* num = token[0] == '@' ? token + 1 : token;
if (identity_all_digits(num)) {
int32_t id;
if (identity_resolve_token(token, is_group, &id) != 0)
return -1;
*out_from = id;
*out_hi = id;
return 0;
}
/* An inclusive LOW-HIGH numeric range. */
const char* dash = strchr(num, '-');
if (dash && dash != num && dash[1] != '\0' && strchr(dash + 1, '-') == NULL) {
size_t lo_len = (size_t)(dash - num);
size_t hi_len = strlen(dash + 1);
char low[16];
char high[16];
if (lo_len < sizeof(low) && hi_len < sizeof(high)) {
memcpy(low, num, lo_len);
low[lo_len] = '\0';
memcpy(high, dash + 1, hi_len);
high[hi_len] = '\0';
if (identity_all_digits(low) && identity_all_digits(high)) {
char* endptr = NULL;
errno = 0;
long lo = strtol(low, &endptr, 10);
if (errno != 0 || !endptr || *endptr != '\0')
return -1;
errno = 0;
long hi = strtol(high, &endptr, 10);
if (errno != 0 || !endptr || *endptr != '\0' || hi < lo || hi > INT32_MAX)
return -1;
*out_from = (int32_t)lo;
*out_hi = (int32_t)hi;
return 0;
}
}
/* Not a numeric LOW-HIGH range: fall through and treat as a name (a
* hyphenated account name like "wayne-smith" must still resolve). */
}
/* A sender-side name. A wildcard other than the bare '*' is matched by rsync
* against the sender's names; because FastSync transmits numeric ids only, the
* receiver cannot evaluate it, so reject rather than silently mis-match. */
if (identity_token_has_glob(token)) {
log_message(LOG_LEVEL_ERROR,
"%smap FROM '%s': name wildcards other than '*' are not supported "
"(FastSync transmits numeric ids, so sender names are unavailable on the "
"receiver)",
is_group ? "--group" : "--user", token);
return -1;
}
int32_t id;
if (identity_resolve_token(token, is_group, &id) != 0)
return -1;
*out_from = id;
*out_hi = id;
return 0;
}
/* Parse a --usermap/--groupmap TO token. '*', a bare numeric id, or an @N id is
* stored numerically; every other non-empty token is a NAME resolved on the
* RECEIVER at apply time (rsync resolves TO names against the receiving side).
* Returns 0 on success, -1 on an empty/malformed token. */
static int identity_parse_to(const char* token, bool is_group, int32_t* out_to, char** out_name) {
if (token[0] == '\0') {
log_message(LOG_LEVEL_ERROR, "%smap TO value is missing", is_group ? "--group" : "--user");
return -1;
}
if (strcmp(token, "*") == 0) {
*out_to = IDENTITY_CURRENT;
*out_name = NULL;
return 0;
}
const char* num = token[0] == '@' ? token + 1 : token;
if (identity_all_digits(num)) {
int32_t id;
if (identity_resolve_token(token, is_group, &id) != 0)
return -1;
*out_to = id;
*out_name = NULL;
return 0;
}
if (identity_token_has_glob(token)) {
log_message(LOG_LEVEL_ERROR, "%smap TO '%s' may not contain a wildcard",
is_group ? "--group" : "--user", token);
return -1;
}
char* name = str_dup(token);
if (!name)
return -1;
*out_to = 0;
*out_name = name;
return 0;
}
static int identity_append_rule(IdentityMap** map, int* count, const IdentityMap* rule) {
if (*count >= MAX_IDENTITY_MAP)
return -1;
IdentityMap* grown = realloc(*map, (size_t)(*count + 1) * sizeof(IdentityMap));
if (!grown)
return -1;
*map = grown;
(*map)[*count].from = from;
(*map)[*count].to = to;
(*map)[*count] = *rule;
(*count)++;
return 0;
}
@@ -275,7 +498,7 @@ int identity_parse_map(Config* config, const char* value, bool is_group) {
char* saveptr = NULL;
for (char* rule = strtok_r(list, ",", &saveptr); rule; rule = strtok_r(NULL, ",", &saveptr)) {
char* colon = strchr(rule, ':');
if (!colon || colon == rule) {
if (!colon) {
/* Log before freeing: `rule` points into the str_dup'd list. */
log_message(LOG_LEVEL_ERROR, "%s rules must be FROM:TO (got '%s')", optname, rule);
free(list);
@@ -284,25 +507,25 @@ int identity_parse_map(Config* config, const char* value, bool is_group) {
*colon = '\0';
char* from_token = rule;
char* to_token = colon + 1;
if (*to_token == '\0') {
IdentityMap parsed;
memset(&parsed, 0, sizeof(parsed));
if (identity_parse_from(from_token, is_group, &parsed.from, &parsed.from_hi) != 0) {
log_message(LOG_LEVEL_ERROR,
"%s could not resolve FROM '%s' in '%s' (a name must exist on the "
"source; use @N for a numeric id)",
optname, from_token, value);
free(list);
log_message(LOG_LEVEL_ERROR, "%s rule 'FROM:' is missing the TO value (got '%s')", optname,
value);
return -1;
}
int32_t from_id, to_id;
if (identity_resolve_token(from_token, is_group, &from_id) != 0 ||
identity_resolve_token(to_token, is_group, &to_id) != 0) {
if (identity_parse_to(to_token, is_group, &parsed.to, &parsed.to_name) != 0) {
log_message(LOG_LEVEL_ERROR, "%s could not parse TO '%s' in '%s'", optname, to_token, value);
free(list);
log_message(LOG_LEVEL_ERROR,
"%s could not resolve '%s' (name must exist on the source; use "
"@N for a numeric id)",
optname, value);
return -1;
}
if (identity_append_rule(is_group ? &config->groupmap : &config->usermap,
is_group ? &config->groupmap_count : &config->usermap_count, from_id,
to_id) != 0) {
is_group ? &config->groupmap_count : &config->usermap_count,
&parsed) != 0) {
free(parsed.to_name);
free(list);
log_message(LOG_LEVEL_ERROR, "%s has too many rules (max %d)", optname, MAX_IDENTITY_MAP);
return -1;
@@ -567,9 +790,6 @@ int identity_parse_copy_as(Config* config, const char* value) {
config->copy_as_set = true;
config->copy_as_uid = uid;
config->copy_as_gid = gid;
/* Ownership application needs the metadata path (the source uid/gid must be
* transmitted); imply it exactly like --chown/--usermap/--groupmap. */
config->use_metadata = true;
ret = 0;
done:
@@ -580,34 +800,120 @@ done:
/* ---- Receiver-side ownership application ---- */
static bool identity_map_lookup(const IdentityMap* map, int count, int32_t source_id,
/* True when a map rule's FROM matcher accepts `id`. A sentinel FROM never
* carries a range. IDENTITY_MATCH_UNNAMED mirrors rsync's empty FROM: it
* matches only ids that have no name in the account database (rsync matches the
* sender's names; FastSync transmits numeric ids only, so it approximates this
* with the receiver's database -- documented in RSYNC_COMPAT.md). */
static bool identity_map_from_matches(const IdentityMap* map, int32_t id, bool is_group) {
if (map->from == IDENTITY_MATCH_ANY)
return true;
if (map->from == IDENTITY_MATCH_UNNAMED)
return is_group ? (getgrgid((gid_t)id) == NULL) : (getpwuid((uid_t)id) == NULL);
return id >= map->from && id <= map->from_hi;
}
/* First matching rule wins. A rule whose TO is a receiver-side name resolves it
* against the receiver's account database here; an unresolvable TO name is
* skipped with a warning and the next rule is considered (rsync prints "Unknown
* --usermap name on receiver" and leaves the id unmapped rather than aborting). */
static bool identity_map_lookup(const IdentityMap* map, int count, int32_t source_id, bool is_group,
int32_t* out_to) {
for (int i = 0; i < count; i++) {
if (map[i].from == IDENTITY_MATCH_ANY || map[i].from == source_id) {
if (!identity_map_from_matches(&map[i], source_id, is_group))
continue;
if (map[i].to_name) {
if (is_group) {
struct group* gr = getgrnam(map[i].to_name);
if (!gr) {
log_message(LOG_LEVEL_WARNING, "Unknown --groupmap name on receiver: %s", map[i].to_name);
continue;
}
*out_to = (int32_t)gr->gr_gid;
} else {
struct passwd* pw = getpwnam(map[i].to_name);
if (!pw) {
log_message(LOG_LEVEL_WARNING, "Unknown --usermap name on receiver: %s", map[i].to_name);
continue;
}
*out_to = (int32_t)pw->pw_uid;
}
} else {
*out_to = map[i].to;
return true;
}
return true;
}
return false;
}
/* Resolve the owner side from the negotiated policy. Sets *out and returns
* true when an owner-affecting request is active (a usermap, --chown USER, or
* -o/--owner); returns false (leaving *out untouched) when the owner side is
* not requested, so callers can pass (uid_t)-1 to fchown and leave it as-is.
* --numeric-ids only changes the RESOLUTION (raw id instead of a name lookup);
* it never makes the side requested. */
static bool identity_resolve_owner(int32_t source_uid, uid_t* out) {
if (!(g_identity.chown_uid_set || g_identity.preserve_owner || g_identity.usermap_count > 0))
return false;
int32_t target;
if (identity_map_lookup(g_identity.usermap, g_identity.usermap_count, source_uid, false,
&target)) {
*out = target == IDENTITY_CURRENT ? geteuid() : (uid_t)target;
} else if (g_identity.chown_uid_set) {
*out = g_identity.chown_uid == IDENTITY_CURRENT ? geteuid() : (uid_t)g_identity.chown_uid;
} else if (g_identity.numeric_ids) {
*out = (uid_t)source_uid;
} else {
/* Best-effort name mapping against the receiver's own database. When the
* transmitted (numeric) id has no name here, fall back to the raw numeric id
* so -o still preserves the source owner. */
struct passwd* pw = getpwuid((uid_t)source_uid);
if (pw) {
const struct passwd* mapped = getpwnam(pw->pw_name);
*out = mapped ? mapped->pw_uid : (uid_t)source_uid;
} else {
*out = (uid_t)source_uid;
}
}
return true;
}
/* Group-side counterpart of identity_resolve_owner(). */
static bool identity_resolve_group(int32_t source_gid, gid_t* out) {
if (!(g_identity.chown_gid_set || g_identity.preserve_group || g_identity.groupmap_count > 0))
return false;
int32_t target;
if (identity_map_lookup(g_identity.groupmap, g_identity.groupmap_count, source_gid, true,
&target)) {
*out = target == IDENTITY_CURRENT ? getegid() : (gid_t)target;
} else if (g_identity.chown_gid_set) {
*out = g_identity.chown_gid == IDENTITY_CURRENT ? getegid() : (gid_t)g_identity.chown_gid;
} else if (g_identity.numeric_ids) {
*out = (gid_t)source_gid;
} else {
struct group* gr = getgrgid((gid_t)source_gid);
if (gr) {
const struct group* mapped = getgrnam(gr->gr_name);
*out = mapped ? mapped->gr_gid : (gid_t)source_gid;
} else {
*out = (gid_t)source_gid;
}
}
return true;
}
/* Resolve the target ownership from the negotiated policy against the entry's
* current stat. Shared by the fd (regular file) and no-follow (symlink) apply
* paths. Returns false when no side is to be changed. */
static bool identity_resolve_targets(const struct stat* st, int32_t source_uid, int32_t source_gid,
uid_t* out_uid, gid_t* out_gid) {
bool set_uid = false;
bool set_gid = false;
uid_t uid = 0;
gid_t gid = 0;
/* --copy-as (P7 Wave E) has the highest priority: it forces BOTH the owner
* and group of every written entry to the requested ids, beating usermap /
* groupmap / --chown / --numeric-ids and the best-effort name lookup. Only
* skip when the entry already carries exactly those ids. */
if (g_identity.copy_as_set) {
uid = (uid_t)g_identity.copy_as_uid;
gid = (gid_t)g_identity.copy_as_gid;
uid_t uid = (uid_t)g_identity.copy_as_uid;
gid_t gid = (gid_t)g_identity.copy_as_gid;
if (st->st_uid == uid && st->st_gid == gid)
return false;
*out_uid = uid;
@@ -615,67 +921,52 @@ static bool identity_resolve_targets(const struct stat* st, int32_t source_uid,
return true;
}
int32_t target;
if (identity_map_lookup(g_identity.usermap, g_identity.usermap_count, source_uid, &target)) {
uid = target == IDENTITY_CURRENT ? geteuid() : (uid_t)target;
set_uid = true;
} else if (g_identity.chown_uid_set) {
uid = g_identity.chown_uid == IDENTITY_CURRENT ? geteuid() : (uid_t)g_identity.chown_uid;
set_uid = true;
} else if (g_identity.numeric_ids) {
uid = (uid_t)source_uid;
set_uid = true;
} else {
/* Best-effort name mapping against the receiver's own database: if the
* transmitted (numeric) id resolves to a name present on this machine,
* re-resolve it. On a shared-account host this is the identity operation;
* when the id has no name here, the user side is left alone. */
struct passwd* pw = getpwuid((uid_t)source_uid);
if (pw) {
const struct passwd* mapped = getpwnam(pw->pw_name);
if (mapped) {
uid = mapped->pw_uid;
set_uid = true;
}
}
}
if (identity_map_lookup(g_identity.groupmap, g_identity.groupmap_count, source_gid, &target)) {
gid = target == IDENTITY_CURRENT ? getegid() : (gid_t)target;
set_gid = true;
} else if (g_identity.chown_gid_set) {
gid = g_identity.chown_gid == IDENTITY_CURRENT ? getegid() : (gid_t)g_identity.chown_gid;
set_gid = true;
} else if (g_identity.numeric_ids) {
gid = (gid_t)source_gid;
set_gid = true;
} else {
struct group* gr = getgrgid((gid_t)source_gid);
if (gr) {
const struct group* mapped = getgrnam(gr->gr_name);
if (mapped) {
gid = mapped->gr_gid;
set_gid = true;
}
}
}
if (!set_uid && !set_gid)
/* Each side is resolved independently: -o/-g and the explicit identity flags
* request the owner/group respectively, and a side that is NOT requested must
* be left exactly as it is (`-1` to fchown on that side). This is what lets
* plain -g change only the group, or -o only the owner. */
uid_t uid = (uid_t)-1;
gid_t gid = (gid_t)-1;
bool owner_requested = identity_resolve_owner(source_uid, &uid);
bool group_requested = identity_resolve_group(source_gid, &gid);
if (!owner_requested && !group_requested)
return false;
/* An unset side keeps the file's current id so the other side can change. */
if (!set_uid)
uid = st->st_uid;
if (!set_gid)
gid = st->st_gid;
/* Only change ownership when the target differs (avoid needless syscalls and
* any chance of clearing setuid/setgid on an already-correct entry). */
if (st->st_uid == uid && st->st_gid == gid)
/* Only change ownership when a requested side actually differs (avoid
* needless syscalls and any chance of clearing setuid/setgid on an
* already-correct entry). */
bool changed = (owner_requested && uid != st->st_uid) || (group_requested && gid != st->st_gid);
if (!changed)
return false;
*out_uid = uid;
*out_gid = gid;
return true;
}
/* --fake-super storage resolution: the receiver records the ownership it WOULD
* have applied. A requested side uses the resolved mapping (--copy-as /
* usermap / --chown / -o/-g, with --numeric-ids as the raw-id modifier); a side
* that was not requested keeps the source's own id, so a plain --fake-super run
* records the source owner untouched. */
void identity_resolve_storage_ids(int32_t source_uid, int32_t source_gid, uint32_t* out_uid,
uint32_t* out_gid) {
if (g_identity.copy_as_set) {
*out_uid = (uint32_t)g_identity.copy_as_uid;
*out_gid = (uint32_t)g_identity.copy_as_gid;
return;
}
uid_t uid = (uid_t)source_uid;
gid_t gid = (gid_t)source_gid;
uid_t resolved_uid;
gid_t resolved_gid;
if (identity_resolve_owner(source_uid, &resolved_uid))
uid = resolved_uid;
if (identity_resolve_group(source_gid, &resolved_gid))
gid = resolved_gid;
*out_uid = (uint32_t)uid;
*out_gid = (uint32_t)gid;
}
static void identity_log_chown_failure(const char* what, uid_t uid, gid_t gid) {
/* EPERM/EACCES are expected when the receiver is not privileged (e.g. the CI
* `nobody` user): warn and continue, never abort the transfer. Any other
@@ -710,8 +1001,12 @@ bool identity_apply_ownership(int fd, int32_t source_uid, int32_t source_gid) {
/* Ownership application is OFF unless the client requested an identity flag.
* This is the controlled gate: a default (or plain -M) transfer never changes
* ownership, byte-for-byte preserving FastSync's existing behavior. --no-super
* additionally forbids it even when the receiver is root. */
if (!identity_active_enabled() || !privilege_super_permitted() || fd < 0)
* additionally forbids it even when the receiver is root. --fake-super never
* performs a REAL chown: that would defeat the point of the flag (record the
* source ownership on an unprivileged receiver for a later privileged
* restore); the resolved ownership is stored in the reserved xattr instead by
* fake_super_store_fd(). */
if (!identity_active_enabled() || g_identity.fake_super || !privilege_super_permitted() || fd < 0)
return true;
struct stat st;
if (fstat(fd, &st) != 0)
@@ -731,7 +1026,8 @@ bool identity_apply_ownership(int fd, int32_t source_uid, int32_t source_gid) {
bool identity_apply_ownership_link(int parent_fd, const char* leaf, int32_t source_uid,
int32_t source_gid) {
if (!identity_active_enabled() || !privilege_super_permitted() || parent_fd < 0 || !leaf)
if (!identity_active_enabled() || g_identity.fake_super || !privilege_super_permitted() ||
parent_fd < 0 || !leaf)
return true;
struct stat st;
if (fstatat(parent_fd, leaf, &st, AT_SYMLINK_NOFOLLOW) != 0)
+35 -6
View File
@@ -80,16 +80,37 @@ void identity_clear_active(void);
* snapshot. Ownership stays OFF ("do not apply") for every transfer that
* requests none of them, preserving FastSync's existing behavior. --super /
* --no-super alone does NOT enable ownership; an explicit identity flag
* (--numeric-ids / --chown / --usermap / --groupmap / --copy-as) is required. */
* (--numeric-ids / --chown / --usermap / --groupmap / --copy-as) or a
* preserve-source -o/--owner / -g/--group request is required. */
bool identity_active_enabled(void);
/* Pure, config-only predicate: true when the client requested ANY
* client-chosen ownership or super-user activity (--numeric-ids, --chown,
* --usermap/--groupmap, --copy-as, --fake-super, or an explicit --super). Used
* by the daemon module gate to decide whether a module's per-module opt-in is
* required; it never reads the per-connection snapshot. */
/* Per-side predicates over the ACTIVE per-connection snapshot (call
* identity_set_active() first). They mirror the owner_requested /
* group_requested conditions inside identity_resolve_targets() exactly, so
* callers that must apply only one side (e.g. the --fake-super owner replay)
* can pass (uid_t)-1 / (gid_t)-1 for the side that was NOT requested and leave
* it untouched. The owner side is requested by --copy-as, --chown USER,
* --numeric-ids, -o/--owner, or a non-empty --usermap; the group side by
* --copy-as, --chown :GROUP, --numeric-ids, -g/--group, or a non-empty
* --groupmap. */
bool identity_owner_requested(void);
bool identity_group_requested(void);
/* Pure, config-only predicate: true when the client requested ANY client-chosen
* ownership or super-user activity (--numeric-ids, --chown, --usermap/--groupmap,
* --copy-as, --fake-super, an explicit --super, or a preserve-source -o/-g).
* General awareness only; the daemon module gate uses the narrower
* identity_explicit_ownership_requested() below. Never reads the snapshot. */
bool identity_ownership_requested(const Config* config);
/* Pure, config-only predicate for the narrow set that lets the CLIENT choose an
* arbitrary owner/group: --numeric-ids, --chown, --usermap/--groupmap,
* --copy-as, --fake-super, or an explicit --super. Deliberately EXCLUDES a
* plain -o/--owner / -g/--group (or -a) preserve-source request, which the
* daemon gate handles by forcing super-user ownership activity off rather than
* refusing the whole transfer. Never reads the snapshot. */
bool identity_explicit_ownership_requested(const Config* config);
/* Apply the negotiated ownership to an already-written file descriptor.
* source_uid/source_gid are the transmitted numeric ids. Resolution order:
* --copy-as (highest priority, forces both ids), then a matching
@@ -106,6 +127,14 @@ bool identity_ownership_requested(const Config* config);
* is active returns true. */
bool identity_apply_ownership(int fd, int32_t source_uid, int32_t source_gid);
/* Resolve the ownership that --fake-super should RECORD in the reserved xattr
* (rather than chown for real). A requested side (--copy-as / usermap /
* --chown / -o / -g, with --numeric-ids as the raw-id modifier) yields the
* resolved target; a side that was not requested keeps the transmitted source
* id. Must be called after identity_set_active(). */
void identity_resolve_storage_ids(int32_t source_uid, int32_t source_gid, uint32_t* out_uid,
uint32_t* out_gid);
/* P7 Wave D: the no-follow (symlink) counterpart. Resolves the same
* usermap/groupmap/chown/numeric-ids/copy-as policy but applies it with
* fchownat(..., AT_SYMLINK_NOFOLLOW) so a symlink's own ownership is changed
+109 -52
View File
@@ -209,22 +209,58 @@ FileMetadata* metadata_receive(int file_descriptor, int* ok) {
return m;
}
static mode_t metadata_mode(const FileMetadata* metadata, mode_t current_mode,
bool preserve_executability) {
bool metadata_mode_for_policy(mode_t source_mode, mode_t current_mode, FileAttrPolicy policy,
mode_t* out_mode) {
const mode_t execute_bits = S_IXUSR | S_IXGRP | S_IXOTH;
if (preserve_executability)
return (current_mode & 0777 & ~execute_bits) | (metadata->mode & execute_bits);
return metadata->mode & 0777 & ~(S_IWGRP | S_IWOTH);
if (policy.perms) {
/* rsync --perms copies the source's permission and special bits exactly,
* including group/other write and setuid/setgid/sticky. The kernel may
* still clear setgid when the receiver is not in the file's group; the
* caller logs a failed chmod rather than silently masking the bits here. */
*out_mode = source_mode & (mode_t)(S_ISUID | S_ISGID | S_ISVTX | 0777);
return true;
}
if (policy.executability) {
/* -E/--executability (rsync 3.4 rule): do NOT copy the source's execute
* bits per class. If the source is executable at all, derive the execute
* bits from the DESTINATION's own read bits (so a class that can read may
* execute); otherwise clear every execute bit. This runs on the
* destination-derived base (pre-existing dest mode, or source&~umask for a
* new file), and leaves the special bits untouched. --perms wins when both
* are set (handled above). */
mode_t base = current_mode & (mode_t)(S_ISUID | S_ISGID | S_ISVTX | 0777);
if (source_mode & 0111)
*out_mode = base | ((base & 0444) >> 2);
else
*out_mode = base & ~execute_bits;
return true;
}
/* Neither requested: no source mode is applied at all. */
return false;
}
void file_restore_metadata(const char* path, const FileMetadata* metadata,
bool preserve_executability) {
FileAttrPolicy file_attr_policy_from_config(const Config* config) {
FileAttrPolicy policy = {false, false, false, false};
if (config) {
policy.perms = config->preserve_perms;
policy.times = config->preserve_times;
policy.atimes = config->preserve_atimes;
policy.executability = config->use_executability;
}
return policy;
}
void file_restore_metadata(const char* path, const FileMetadata* metadata, FileAttrPolicy policy) {
if (metadata == NULL)
return;
struct stat current;
mode_t current_mode = stat(path, &current) == 0 ? current.st_mode : 0;
mode_t safe_mode = metadata_mode(metadata, current_mode, preserve_executability);
if (chmod(path, safe_mode) != 0) {
bool apply_mode = false;
mode_t safe_mode = 0;
if (policy.perms || policy.executability) {
struct stat current;
mode_t current_mode = stat(path, &current) == 0 ? current.st_mode : 0;
apply_mode = metadata_mode_for_policy(metadata->mode, current_mode, policy, &safe_mode);
}
if (apply_mode && chmod(path, safe_mode) != 0) {
char* escaped_path = output_escape(path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "Failed to chmod %s: %s",
escaped_path ? escaped_path : "<allocation failed>", strerror(errno));
@@ -232,14 +268,23 @@ void file_restore_metadata(const char* path, const FileMetadata* metadata,
}
/* Never apply client-supplied ownership. The descriptor API below is the
receiver write path; retain this legacy API only for compatibility. */
struct timespec times[2];
times[0].tv_sec = 0;
times[0].tv_nsec = UTIME_OMIT;
times[1].tv_sec = metadata->mtime_sec;
times[1].tv_nsec = metadata->mtime_nsec;
if (metadata->atime_valid) {
times[0].tv_sec = metadata->atime_sec;
times[0].tv_nsec = metadata->atime_nsec;
if (policy.times || (policy.atimes && metadata->atime_valid)) {
struct timespec times[2] = {{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
{.tv_sec = 0, .tv_nsec = UTIME_OMIT}};
if (policy.times) {
times[1].tv_sec = metadata->mtime_sec;
times[1].tv_nsec = metadata->mtime_nsec;
}
if (policy.atimes && metadata->atime_valid) {
times[0].tv_sec = metadata->atime_sec;
times[0].tv_nsec = metadata->atime_nsec;
}
if (utimensat(AT_FDCWD, path, times, 0) != 0) {
char* escaped_path = output_escape(path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "Failed to set timestamps on %s: %s",
escaped_path ? escaped_path : "<allocation failed>", strerror(errno));
free(escaped_path);
}
}
if (metadata->crtime_valid) {
log_message(LOG_LEVEL_DEBUG,
@@ -247,16 +292,10 @@ void file_restore_metadata(const char* path, const FileMetadata* metadata,
"setter exists",
(long long)metadata->crtime_sec, metadata->crtime_nsec, path);
}
if (utimensat(AT_FDCWD, path, times, 0) != 0) {
char* escaped_path = output_escape(path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "Failed to set timestamps on %s: %s",
escaped_path ? escaped_path : "<allocation failed>", strerror(errno));
free(escaped_path);
}
}
bool file_restore_symlink_metadata(const char* path, const FileMetadata* metadata,
bool omit_link_times) {
FileAttrPolicy policy, bool omit_link_times) {
if (path == NULL || metadata == NULL)
return !identity_copy_as_active();
char* leaf = NULL;
@@ -269,18 +308,25 @@ bool file_restore_symlink_metadata(const char* path, const FileMetadata* metadat
best-effort. */
bool owned = identity_apply_ownership_link(parent_fd, leaf, (int32_t)metadata->uid,
(int32_t)metadata->gid);
/* Symlink mode: not settable on Linux (fchmodat AT_SYMLINK_NOFOLLOW returns
EOPNOTSUPP/ENOTSUP); attempt it for platforms that support it and quietly
ignore the unsupported case so the transfer never fails over it. */
mode_t link_mode = metadata->mode & 0777;
if (fchmodat(parent_fd, leaf, link_mode, AT_SYMLINK_NOFOLLOW) != 0 && errno != EOPNOTSUPP &&
errno != ENOTSUP && errno != ENOSYS) {
log_message(LOG_LEVEL_DEBUG, "Could not set symlink mode on %s: %s", path, strerror(errno));
/* Symlink mode: only when -p is in effect. It is not settable on Linux
(fchmodat AT_SYMLINK_NOFOLLOW returns EOPNOTSUPP/ENOTSUP); attempt it for
platforms that support it and quietly ignore the unsupported case so the
transfer never fails over it. */
if (policy.perms) {
mode_t link_mode = metadata->mode & (mode_t)(S_ISUID | S_ISGID | S_ISVTX | 0777);
if (fchmodat(parent_fd, leaf, link_mode, AT_SYMLINK_NOFOLLOW) != 0 && errno != EOPNOTSUPP &&
errno != ENOTSUP && errno != ENOSYS) {
log_message(LOG_LEVEL_DEBUG, "Could not set symlink mode on %s: %s", path, strerror(errno));
}
}
if (!omit_link_times) {
if (!omit_link_times && (policy.times || (policy.atimes && metadata->atime_valid))) {
struct timespec times[2] = {{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
{.tv_sec = metadata->mtime_sec, .tv_nsec = metadata->mtime_nsec}};
if (metadata->atime_valid) {
{.tv_sec = 0, .tv_nsec = UTIME_OMIT}};
if (policy.times) {
times[1].tv_sec = metadata->mtime_sec;
times[1].tv_nsec = metadata->mtime_nsec;
}
if (policy.atimes && metadata->atime_valid) {
times[0].tv_sec = metadata->atime_sec;
times[0].tv_nsec = metadata->atime_nsec;
}
@@ -296,19 +342,13 @@ bool file_restore_symlink_metadata(const char* path, const FileMetadata* metadat
return owned;
}
bool file_restore_metadata_fd(int fd, const FileMetadata* metadata, bool preserve_executability) {
bool file_restore_metadata_fd(int fd, const FileMetadata* metadata, FileAttrPolicy policy) {
if (fd < 0 || metadata == NULL)
return metadata == NULL;
bool ok = true;
struct stat current;
if (fstat(fd, &current) != 0)
return false;
mode_t safe_mode = metadata_mode(metadata, current.st_mode, preserve_executability);
if (fchmod(fd, safe_mode) != 0)
ok = false;
/* Client uid/gid values are deliberately not authoritative UNLESS the client
explicitly opted in with an identity flag (--numeric-ids / --usermap /
--groupmap / --chown). identity_apply_ownership is the controlled,
--groupmap / --chown / -o/-g). identity_apply_ownership is the controlled,
privilege-gated path: it consults the negotiated policy, resolves the
target ids, and applies them via an fd-relative fchown() that is confined
to the just-written file (EPERM/EACCES are logged, never fatal) -- EXCEPT
@@ -316,14 +356,19 @@ bool file_restore_metadata_fd(int fd, const FileMetadata* metadata, bool preserv
marks this entry as failed instead of reporting a wrong-owner write as
success. With no identity flag set it is a no-op, so a default or plain -M
transfer keeps FastSync's existing behavior of never applying client
ownership. */
ownership. Ownership runs BEFORE the mode because a chown clears
setuid/setgid; rsync likewise chowns first and then restores the source
mode (including its special bits). */
if (!identity_apply_ownership(fd, (int32_t)metadata->uid, (int32_t)metadata->gid))
ok = false;
struct timespec times[2] = {{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
{.tv_sec = metadata->mtime_sec, .tv_nsec = metadata->mtime_nsec}};
if (metadata->atime_valid) {
times[0].tv_sec = metadata->atime_sec;
times[0].tv_nsec = metadata->atime_nsec;
if (policy.perms || policy.executability) {
struct stat current;
if (fstat(fd, &current) != 0)
return false;
mode_t safe_mode = 0;
bool apply_mode = metadata_mode_for_policy(metadata->mode, current.st_mode, policy, &safe_mode);
if (apply_mode && fchmod(fd, safe_mode) != 0)
ok = false;
}
/* --crtimes captures and transmits the source birth time, but there is no
* portable way to set a birth time (utimensat can only set atime/mtime), so
@@ -335,7 +380,19 @@ bool file_restore_metadata_fd(int fd, const FileMetadata* metadata, bool preserv
"crtime (birth time) %lld.%09ld transmitted but not applied: no portable setter",
(long long)metadata->crtime_sec, metadata->crtime_nsec);
}
if (futimens(fd, times) != 0)
ok = false;
if (policy.times || (policy.atimes && metadata->atime_valid)) {
struct timespec times[2] = {{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
{.tv_sec = 0, .tv_nsec = UTIME_OMIT}};
if (policy.times) {
times[1].tv_sec = metadata->mtime_sec;
times[1].tv_nsec = metadata->mtime_nsec;
}
if (policy.atimes && metadata->atime_valid) {
times[0].tv_sec = metadata->atime_sec;
times[0].tv_nsec = metadata->atime_nsec;
}
if (futimens(fd, times) != 0)
ok = false;
}
return ok;
}
+20 -7
View File
@@ -2,6 +2,7 @@
#define METADATA_H
#include "file.h"
#include "file_attr.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
@@ -49,19 +50,31 @@ void metadata_to_buf(char** buf, const FileMetadata* m);
FileMetadata* metadata_from_buf(const uint8_t* buf, size_t len);
bool metadata_send(int file_descriptor, const FileMetadata* m);
FileMetadata* metadata_receive(int file_descriptor, int* ok);
void file_restore_metadata(const char* path, const FileMetadata* metadata,
bool preserve_executability);
bool file_restore_metadata_fd(int fd, const FileMetadata* metadata, bool preserve_executability);
void file_restore_metadata(const char* path, const FileMetadata* metadata, FileAttrPolicy policy);
bool file_restore_metadata_fd(int fd, const FileMetadata* metadata, FileAttrPolicy policy);
/* Shared mode-policy helper: the single source of truth for the receiver's
* mode rule. Given a source mode and the destination's CURRENT mode, returns
* true and stores the exact mode to apply in *out_mode when `policy` requests
* a change, or false when it requests neither --perms nor --executability (the
* caller then leaves the destination mode alone). --perms wins over -E; the
* -E rule derives exec bits from the destination's read bits (rsync 3.4);
* group/other write is never granted from a client-supplied mode. Shared by
* file_restore_metadata_fd() and the --fake-super replay so the two cannot
* diverge. */
bool metadata_mode_for_policy(mode_t source_mode, mode_t current_mode, FileAttrPolicy policy,
mode_t* out_mode);
/* P7 Wave D: apply a SYMLINK's own metadata using no-follow primitives only
* (utimensat/lchown/fchmodat with AT_SYMLINK_NOFOLLOW), confined fd-relative
* under the authorized root. `omit_link_times` (-J/--omit-link-times)
* suppresses the timestamps; the link's mode/ownership are still attempted
* (ownership stays gated by the identity policy and by default is not applied).
* under the authorized root. The link's mode is applied only when policy.perms;
* policy.times (further suppressed by `omit_link_times` for -J) applies the
* mtime with policy.atimes controlling the atime slot; ownership stays gated by
* the identity policy and by default is not applied.
* A null metadata or an unfollowable parent is a harmless no-op. Returns false
* only when a REQUIRED --copy-as ownership application failed, so the caller can
* report the entry as failed instead of claiming a wrong-owner success. */
bool file_restore_symlink_metadata(const char* path, const FileMetadata* metadata,
bool omit_link_times);
FileAttrPolicy policy, bool omit_link_times);
/* Compare timestamps using rsync's whole-second modification window. */
bool metadata_mtime_matches(time_t left_sec, long left_nsec, time_t right_sec, long right_nsec,
+7
View File
@@ -30,6 +30,8 @@ PipelineContextSender* pipeline_context_sender_create(Config* config, Queue* que
context->max_queue_bytes = 0;
context->manifest = NULL;
context->excluded_paths = NULL;
context->size_skipped_paths = NULL;
context->synced_dirs = NULL;
context->missing_args = NULL;
context->scan_had_io_error = false;
context->remove_source_files = NULL;
@@ -44,6 +46,7 @@ PipelineContextSender* pipeline_context_sender_create(Config* config, Queue* que
protocol_session_set_max_alloc(&context->allocation_session, config->max_alloc);
context->dir_entries = NULL;
context->dir_entries_mutex_init = false;
context->delete_limit = false;
int init = 0;
if (config->use_metadata) {
context->dir_entries = array_list_create(file_destroy);
@@ -186,6 +189,10 @@ void pipeline_context_sender_destroy(PipelineContextSender* context) {
}
if (context->excluded_paths)
array_list_delete(context->excluded_paths);
if (context->size_skipped_paths)
array_list_delete(context->size_skipped_paths);
if (context->synced_dirs)
array_list_delete(context->synced_dirs);
if (context->missing_args)
array_list_delete(context->missing_args);
if (context->remove_source_files)
+16
View File
@@ -42,6 +42,18 @@ typedef struct {
scanner's exclusion sink) or, in the early modes, by the path-only pre-scan
on the calling thread before the pipeline starts. */
ArrayList* excluded_paths;
/* --max-size/--min-size pruned source paths. These are ALWAYS sent as
protected prefixes (even with --delete-excluded), so the destination
mirrors of size-skipped files survive --delete like rsync. Populated by
the scanner thread (workers append under mutex_scanner) or, in the early
modes, by the path-only pre-scan on the calling thread. */
ArrayList* size_skipped_paths;
/* Destination-relative paths of the directories the source scan synchronized
for this run (the receive root is the "." sentinel). Sent with the
manifest so the receiver confines its extras walk to them, matching rsync's
"delete only in synchronized directories" (notably for --files-from).
Populated by the scanner thread or the early pre-scan. */
ArrayList* synced_dirs;
/* --delete-missing-args: the destination-relative mirrors of the --files-from
entries that are missing under the source. Computed by the preflight on
the calling thread before the pipeline starts; the sender thread transmits
@@ -83,6 +95,10 @@ typedef struct {
ArrayList* dir_entries;
mtx_t dir_entries_mutex;
bool dir_entries_mutex_init;
/* Set by the sender thread when the receiver reported a --max-delete-capped
deletion (STATUS_DELETE_LIMIT): the transfer succeeded and the process must
exit 25 like rsync. Read by the caller after the sender thread is joined. */
bool delete_limit;
} PipelineContextSender;
/* `config` is borrowed and must outlive the context: destroy does NOT free it,
+49 -22
View File
@@ -12,8 +12,7 @@
#include <time.h>
#include <unistd.h>
#define RECEIVE_TIMEOUT_SEC 60 /* 60 second per-message timeout */
#define SEND_TIMEOUT_SEC 60
#define RECEIVE_TIMEOUT_SEC 60 /* built-in fallback for explicit -timed calls only */
static __thread int io_read_fd = -1;
static __thread int io_write_fd = -1;
@@ -99,8 +98,14 @@ void protocol_session_set_io_timeout(ProtocolSession* session, int sec) {
int protocol_get_io_timeout_sec(void) {
const ProtocolSession* session = bound_session ? bound_session : &legacy_io_session;
int sec = session->io_timeout_sec;
return sec > 0 ? sec : RECEIVE_TIMEOUT_SEC;
/* 0 (or negative) means the session timeout is disabled, matching rsync's
* --timeout=0 default. Callers must treat a non-positive result as "wait
* without a deadline" instead of substituting a built-in window. */
return session->io_timeout_sec > 0 ? session->io_timeout_sec : 0;
}
int protocol_server_io_timeout_sec(int client_timeout) {
return client_timeout > 0 ? client_timeout : SERVER_IO_TIMEOUT_SEC;
}
void protocol_session_set_max_alloc(ProtocolSession* session, unsigned long long max_alloc) {
@@ -110,7 +115,8 @@ void protocol_session_set_max_alloc(ProtocolSession* session, unsigned long long
}
static bool allocation_allowed(const ProtocolSession* session, size_t size) {
return (unsigned long long)size <= session->max_alloc;
/* max_alloc == 0 is rsync's --max-alloc=0 "no limit". */
return session->max_alloc == 0 || (unsigned long long)size <= session->max_alloc;
}
static void* protocol_alloc_for_session(const ProtocolSession* session, size_t size) {
@@ -280,11 +286,15 @@ bool protocol_send_n_data(ProtocolSession* session, const void* data, size_t dat
log_debug_message(LOG_DEBUG_IO, " Sending n Data: %zu", data_size);
if (!session)
return false;
int timeout_sec = session->io_timeout_sec > 0 ? session->io_timeout_sec : SEND_TIMEOUT_SEC;
/* A non-positive session timeout disables the deadline entirely (rsync's
* --timeout=0 default); poll then blocks until the socket becomes writable. */
int timeout_sec = session->io_timeout_sec > 0 ? session->io_timeout_sec : 0;
int fd = session->write_fd;
struct timespec deadline;
clock_gettime(CLOCK_MONOTONIC, &deadline);
deadline.tv_sec += timeout_sec;
if (timeout_sec > 0) {
clock_gettime(CLOCK_MONOTONIC, &deadline);
deadline.tv_sec += timeout_sec;
}
short wait_events = POLLOUT;
ssize_t total_bytes_send = 0;
while ((size_t)total_bytes_send < data_size) {
@@ -292,7 +302,7 @@ bool protocol_send_n_data(ProtocolSession* session, const void* data, size_t dat
if (session->bwlimit > 0 && chunk > 65536)
chunk = 65536;
struct pollfd pfd = {.fd = fd, .events = wait_events};
int poll_result = poll(&pfd, 1, deadline_remaining_ms(&deadline));
int poll_result = poll(&pfd, 1, timeout_sec > 0 ? deadline_remaining_ms(&deadline) : -1);
if (poll_result == 0 || (poll_result < 0 && errno != EINTR)) {
log_message(LOG_LEVEL_ERROR, "Send timeout or poll failure");
return false;
@@ -338,12 +348,21 @@ bool protocol_send_n_data(ProtocolSession* session, const void* data, size_t dat
bool protocol_receive_n_data_timed(ProtocolSession* session, void* data, size_t data_size,
int timeout_sec);
static bool protocol_receive_n_data_until(ProtocolSession* session, void* data, size_t data_size,
const struct timespec* deadline);
bool protocol_receive_n_data(ProtocolSession* session, void* data, size_t data_size) {
/* Honor the session's configured deadline; protocol_receive_n_data_timed
* re-applies the built-in 60 s default when the value is <= 0. */
int timeout_sec = session ? session->io_timeout_sec : 0;
return protocol_receive_n_data_timed(session, data, data_size, timeout_sec);
/* Honor the session's configured deadline. A non-positive value disables the
* deadline (rsync's --timeout=0 default): wait without a poll timeout. The
* explicit _timed variants keep their own 0 -> built-in-default contract. */
if (!session)
return false;
if (session->io_timeout_sec <= 0)
return protocol_receive_n_data_until(session, data, data_size, NULL);
struct timespec deadline;
clock_gettime(CLOCK_MONOTONIC, &deadline);
deadline.tv_sec += session->io_timeout_sec;
return protocol_receive_n_data_until(session, data, data_size, &deadline);
}
/* Read exactly `data_size` bytes from `session` before `deadline` elapses
@@ -353,7 +372,7 @@ bool protocol_receive_n_data(ProtocolSession* session, void* data, size_t data_s
static bool protocol_receive_n_data_until(ProtocolSession* session, void* data, size_t data_size,
const struct timespec* deadline) {
log_debug_message(LOG_DEBUG_IO, " Receiving n Data: %zu", data_size);
if (!session || !deadline)
if (!session)
return false;
int fd = session->read_fd;
@@ -362,7 +381,8 @@ static bool protocol_receive_n_data_until(ProtocolSession* session, void* data,
while (total_bytes_received < data_size) {
if (!session->ssl || SSL_pending(session->ssl) == 0) {
struct pollfd pfd = {.fd = fd, .events = wait_events};
int poll_result = poll(&pfd, 1, deadline_remaining_ms(deadline));
/* A NULL deadline means "wait indefinitely" (timeout disabled). */
int poll_result = poll(&pfd, 1, deadline ? deadline_remaining_ms(deadline) : -1);
if (poll_result == 0) {
log_message(LOG_LEVEL_ERROR, "Receive timeout");
return false;
@@ -479,6 +499,10 @@ static const char* status_to_string(Status status) {
return "ERROR_DETAIL";
case STATUS_DRY_RUN_TRANSFER:
return "DRY_RUN_TRANSFER";
case STATUS_DELETE_LIMIT:
return "DELETE_LIMIT";
case STATUS_DEST_INFO:
return "DEST_INFO";
default:
return "UNKNOWN";
}
@@ -702,13 +726,16 @@ static bool protocol_capture_error_detail(ProtocolSession* session, Status* stat
bool protocol_receive_status(ProtocolSession* session, Status* status) {
if (!session || !status)
return false;
int timeout_sec = session->io_timeout_sec > 0 ? session->io_timeout_sec : RECEIVE_TIMEOUT_SEC;
struct timespec deadline;
clock_gettime(CLOCK_MONOTONIC, &deadline);
deadline.tv_sec += timeout_sec;
if (!protocol_receive_n_data_until(session, status, sizeof(Status), &deadline))
const struct timespec* deadline_ptr = NULL;
if (session->io_timeout_sec > 0) {
clock_gettime(CLOCK_MONOTONIC, &deadline);
deadline.tv_sec += session->io_timeout_sec;
deadline_ptr = &deadline;
}
if (!protocol_receive_n_data_until(session, status, sizeof(Status), deadline_ptr))
return false;
if (!protocol_capture_error_detail(session, status, &deadline, NULL))
if (!protocol_capture_error_detail(session, status, deadline_ptr, NULL))
return false;
log_debug_message(LOG_DEBUG_PROTO, "Received Status: %s", status_to_string(*status));
return true;
@@ -747,8 +774,8 @@ static bool protocol_read_status_until(ProtocolSession* session, Status* status,
short wait_events = POLLIN;
while (got < sizeof(Status)) {
if (!session->ssl || SSL_pending(session->ssl) == 0) {
int remaining_ms = deadline_remaining_ms(deadline);
if (remaining_ms <= 0) {
int remaining_ms = deadline ? deadline_remaining_ms(deadline) : -1;
if (remaining_ms == 0) {
log_message(LOG_LEVEL_ERROR, "Receive timeout while reading status");
return false;
}
+43 -12
View File
@@ -34,6 +34,11 @@
#define DEFAULT_MAX_ALLOC (1ULL * 1024 * 1024 * 1024)
/* Server policy ceiling for a client-provided allocation limit. */
#define MAX_SERVER_ALLOC (256ULL * 1024 * 1024)
/* Server-owned floor for the per-message I/O deadline. A client --timeout=0
(rsync's default) disables the client's own deadlines, but a server session
must never be held open forever by a silent peer (slow-loris), so the server
floors the effective deadline at this value. */
#define SERVER_IO_TIMEOUT_SEC 60
/* Bounded cumulative per-connection receive budget. In-flight wire buffers,
decompression buffers and queued (not yet written) file payloads for a
connection must stay within this ceiling. */
@@ -59,10 +64,12 @@ typedef struct ProtocolSession {
bool eight_bit_output;
unsigned long long max_alloc;
/* Per-session deadline (seconds) applied to every protocol send/receive by
* protocol_send_n_data / protocol_receive_n_data. Defaults to the built-in
* 60 s window; a value <= 0 falls back to that default. Set from the
* negotiated Config->timeout so --timeout is honored by the poll()-driven
* protocol I/O, not just the socket SO_RCVTIMEO/SO_SNDTIMEO. */
* protocol_send_n_data / protocol_receive_n_data. The initialized default is
* the built-in 60 s window; a value <= 0 disables the deadline (rsync's
* --timeout=0). Set from the negotiated Config->timeout so --timeout is
* honored by the poll()-driven protocol I/O, not just the socket
* SO_RCVTIMEO/SO_SNDTIMEO. The server does not propagate a client 0 here: it
* installs protocol_server_io_timeout_sec() so its sessions keep a floor. */
int io_timeout_sec;
} ProtocolSession;
@@ -155,7 +162,26 @@ enum NET_STATUS {
* (the receiver reads none in dry-run). STATUS_OK keeps its meaning in this
* path ("already up to date / nothing to do"). Appended after
* STATUS_ERROR_DETAIL so no existing status is renumbered. */
STATUS_DRY_RUN_TRANSFER
STATUS_DRY_RUN_TRANSFER,
/* --max-delete budget exhausted (protocol 2.23.0). Sent by the receiver as
* the terminal success status INSTEAD of STATUS_OK when a --delete/
* --delete-missing-args commit removed up to the --max-delete bound but had
* to skip further extras. The transfer itself succeeded and all file data is
* stored; the sender maps this to rsync's exit code 25 ("the --max-delete
* limit stopped deletions"). Appended after STATUS_DRY_RUN_TRANSFER so no
* existing status is renumbered. */
STATUS_DELETE_LIMIT,
/* Destination-state report for output parity (protocol 2.23.0). When the
* wire config carries report_dest_info=true, the receiver answers every
* per-file STATUS_CHECK request with STATUS_DEST_INFO FIRST, followed by a
* fixed record describing the pre-transfer destination entry
* (int32 has_old; uint64 size; int64 mtime; int64 mtime_nsec; uint32 mode;
* int32 uid; int32 gid). The ordinary STATUS_OK/STATUS_NEXT/... verdict
* follows, so the sender can render rsync-accurate -i/--out-format columns
* (new vs modified, and which of size/time/perms/owner/group differ) without
* changing the transfer decision itself. Appended after
* STATUS_DELETE_LIMIT so no existing status is renumbered. */
STATUS_DEST_INFO
};
void io_set_fds(int read_fd, int write_fd);
@@ -170,15 +196,20 @@ void protocol_session_unbind(void);
void protocol_session_set_ssl(ProtocolSession* session, SSL* ssl);
void protocol_session_set_bwlimit(ProtocolSession* session, unsigned long long bytes_per_sec);
void protocol_session_set_max_alloc(ProtocolSession* session, unsigned long long max_alloc);
/* Override the per-message send/receive deadline for this session.
* `sec` <= 0 restores the built-in 60 s default (used for --timeout=0/unset).
* An explicit long deadline (e.g. the delete-ack wait) is applied per-call by
* protocol_receive_status_timed and is unaffected by this setter. */
/* Override the per-message send/receive deadline for this session. The value
* is stored verbatim: a positive value sets the deadline, `sec` <= 0 disables
* it (rsync's --timeout=0). An explicit long deadline (e.g. the delete-ack
* wait) is applied per-call by protocol_receive_status_timed and is unaffected
* by this setter. */
void protocol_session_set_io_timeout(ProtocolSession* session, int sec);
/* Effective per-message I/O deadline (seconds) for the currently-bound session,
* falling back to the built-in default. Used by the plaintext sendfile path
* which bypasses the protocol send primitive. */
/* Effective per-message I/O deadline (seconds) for the currently-bound session.
* Zero means the deadline is disabled (rsync's --timeout=0). Used by the
* plaintext sendfile path which bypasses the protocol send primitive. */
int protocol_get_io_timeout_sec(void);
/* The server-side effective deadline for a client-requested timeout: a positive
* client value is honored, otherwise the SERVER_IO_TIMEOUT_SEC floor applies so
* a silent peer can never hold a session open forever. */
int protocol_server_io_timeout_sec(int client_timeout);
void* protocol_alloc(size_t size);
void* protocol_realloc(void* ptr, size_t size);
void protocol_session_set_8_bit_output(ProtocolSession* session, bool enabled);
+181 -21
View File
@@ -44,6 +44,181 @@ static bool parse_two_digits(const char* s, int* out) {
return true;
}
/* True when the current character of the cursor is a decimal digit. */
static bool is_digit(const char* cp) {
return *cp >= '0' && *cp <= '9';
}
/* rsync 3.4.1's flexible --stop-at date parser (ported from
* options.c:parse_time). Returns a time_t, or (time_t)-1 on a malformed value.
* Accepted forms include Y-M-DTh:m, Y/M/DTh:m, Y-M-D, M-D, D, h:m, :m and
* "T h:m"; a 1- or 2-digit year and omitted fields are resolved to the next
* matching point in time in the local timezone. Seconds are NOT accepted
* (rsync rejects them too); FastSync keeps its own HH:MM:SS spelling as an
* extension handled by the caller. `now` is passed in so tests are
* deterministic; production passes time(NULL). */
static time_t parse_time_rsync(const char* value, time_t now) {
const char* cp;
time_t val;
struct tm today;
if (!localtime_r(&now, &today))
return (time_t)-1;
struct tm t;
int in_date, old_mday, n;
memset(&t, 0, sizeof t);
t.tm_year = t.tm_mon = t.tm_mday = -1;
t.tm_hour = t.tm_min = t.tm_isdst = -1;
cp = value;
if (*cp == 'T' || *cp == 't' || *cp == ':') {
in_date = *cp == ':' ? 0 : -1;
cp++;
} else
in_date = 1;
for (;; cp++) {
if (!is_digit(cp))
return (time_t)-1;
n = 0;
do {
n = n * 10 + *cp++ - '0';
} while (is_digit(cp));
if (*cp == ':')
in_date = 0;
if (in_date > 0) {
if (t.tm_year != -1)
return (time_t)-1;
t.tm_year = t.tm_mon;
t.tm_mon = t.tm_mday;
t.tm_mday = n;
if (!*cp)
break;
if (*cp == 'T' || *cp == 't') {
if (!cp[1])
break;
in_date = -1;
} else if (*cp != '-' && *cp != '/')
return (time_t)-1;
continue;
}
if (t.tm_hour != -1)
return (time_t)-1;
t.tm_hour = t.tm_min;
t.tm_min = n;
if (!*cp) {
if (in_date < 0)
return (time_t)-1;
break;
}
if (*cp != ':')
return (time_t)-1;
in_date = 0;
}
in_date = 0;
if (t.tm_year < 0) {
t.tm_year = today.tm_year;
in_date = 1;
} else if (t.tm_year < 100) {
while (t.tm_year < today.tm_year)
t.tm_year += 100;
} else
t.tm_year -= 1900;
if (t.tm_mon < 0) {
t.tm_mon = today.tm_mon;
in_date = 2;
} else
t.tm_mon--;
if (t.tm_mday < 0) {
t.tm_mday = today.tm_mday;
in_date = 3;
}
n = 0;
if (t.tm_min < 0) {
t.tm_hour = t.tm_min = 0;
} else if (t.tm_hour < 0) {
if (in_date != 3)
return (time_t)-1;
in_date = 0;
t.tm_hour = today.tm_hour;
n = 60 * 60;
}
/* mktime() may roll a too-large tm_mday into the following month; undo that
* in the "next match" loop below. */
old_mday = t.tm_mday;
if (t.tm_hour > 23 || t.tm_min > 59 || t.tm_mon < 0 || t.tm_mon >= 12 || t.tm_mday < 1 ||
t.tm_mday > 31 || (val = mktime(&t)) == (time_t)-1)
return (time_t)-1;
while (in_date && (val <= now || t.tm_mday < old_mday)) {
switch (in_date) {
case 3:
old_mday = ++t.tm_mday;
break;
case 2:
if (t.tm_mday < old_mday)
t.tm_mday = old_mday; /* the month already got bumped forward */
else if (++t.tm_mon == 12) {
t.tm_mon = 0;
t.tm_year++;
}
break;
case 1:
if (t.tm_mday < old_mday) {
/* mon==1 mday==29 got bumped to mon==2 */
if (t.tm_mon != 2 || old_mday != 29)
return (time_t)-1;
t.tm_mon = 1;
t.tm_mday = 29;
}
t.tm_year++;
break;
}
if ((val = mktime(&t)) == (time_t)-1) {
if (in_date != 3 || t.tm_mday <= 28)
return (time_t)-1;
t.tm_mday = old_mday = 1;
in_date = 2;
}
}
if (n) {
while (val <= now)
val += n;
}
return val;
}
/* FastSync's HH:MM or HH:MM:SS spelling on the current local day. rsync's own
* --stop-at accepts only HH:MM, so this is a strict superset extension. */
static bool parse_clock_time(const char* value, time_t now, time_t* out_deadline) {
size_t len = strlen(value);
if (len != 5 && len != 8)
return false;
if (value[2] != ':' || (len == 8 && value[5] != ':'))
return false;
int hh, mm, ss = 0;
if (!parse_two_digits(value, &hh) || !parse_two_digits(value + 3, &mm))
return false;
if (len == 8 && !parse_two_digits(value + 6, &ss))
return false;
if (hh > 23 || mm > 59 || ss > 59)
return false;
struct tm today;
if (!localtime_r(&now, &today))
return false;
today.tm_hour = hh;
today.tm_min = mm;
today.tm_sec = ss;
today.tm_isdst = -1;
time_t deadline = mktime(&today);
if (deadline == (time_t)-1)
return false;
*out_deadline = deadline;
return true;
}
bool stop_parse_at_time(const char* value, time_t now, time_t* out_deadline) {
if (!value || !out_deadline)
return false;
@@ -91,28 +266,13 @@ bool stop_parse_at_time(const char* value, time_t now, time_t* out_deadline) {
return true;
}
/* HH:MM or HH:MM:SS on the current local day. */
size_t len = strlen(value);
if (len != 5 && len != 8)
return false;
if (value[2] != ':' || (len == 8 && value[5] != ':'))
return false;
int hh, mm, ss = 0;
if (!parse_two_digits(value, &hh) || !parse_two_digits(value + 3, &mm))
return false;
if (len == 8 && !parse_two_digits(value + 6, &ss))
return false;
if (hh > 23 || mm > 59 || ss > 59)
return false;
/* HH:MM or HH:MM:SS on the current local day (FastSync extension). */
if (parse_clock_time(value, now, out_deadline))
return true;
struct tm today;
if (!localtime_r(&now, &today))
return false;
today.tm_hour = hh;
today.tm_min = mm;
today.tm_sec = ss;
today.tm_isdst = -1;
time_t deadline = mktime(&today);
/* rsync's full/partial date-and-time form (e.g. 2000-12-31T23:59, 12-31,
* 14:00, :59, 1, 1-30). */
time_t deadline = parse_time_rsync(value, now);
if (deadline == (time_t)-1)
return false;
*out_deadline = deadline;
+19 -11
View File
@@ -289,14 +289,14 @@ void server_accept_loop(Server* server, void (*child_fn)(int, void*), void* chil
accept_loop(server, child_fn, child_ctx, log_fmt);
}
static int g_timeout_sec = 30;
static int g_contimeout_sec = 10;
/* rsync defaults: --timeout=0 (disabled) and --contimeout=60. A non-positive
* value means "no timeout" rather than "leave the built-in value in place". */
static int g_timeout_sec = 0;
static int g_contimeout_sec = 60;
void tcp_set_timeouts(int timeout_sec, int contimeout_sec) {
if (timeout_sec > 0)
g_timeout_sec = timeout_sec;
if (contimeout_sec > 0)
g_contimeout_sec = contimeout_sec;
g_timeout_sec = timeout_sec > 0 ? timeout_sec : 0;
g_contimeout_sec = contimeout_sec > 0 ? contimeout_sec : 0;
}
int tcp_get_contimeout_sec(void) {
@@ -308,6 +308,10 @@ int tcp_get_timeout_sec(void) {
}
static void tcp_apply_socket_timeout(int fd) {
/* timeout 0 means no timeout: leave the socket in its default (blocking)
* mode instead of installing a zero SO_RCVTIMEO/SO_SNDTIMEO. */
if (g_timeout_sec <= 0)
return;
struct timeval tv;
tv.tv_sec = g_timeout_sec;
tv.tv_usec = 0;
@@ -483,11 +487,15 @@ bool tcp_connect_socket_ex(Client* client, const char* host, int port,
break;
}
struct timeval ct;
ct.tv_sec = g_contimeout_sec;
ct.tv_usec = 0;
setsockopt(client->file_descriptor, SOL_SOCKET, SO_RCVTIMEO, &ct, sizeof(ct));
setsockopt(client->file_descriptor, SOL_SOCKET, SO_SNDTIMEO, &ct, sizeof(ct));
/* --contimeout=0 disables the connect timeout: skip the pre-connect socket
* timeouts entirely. */
if (g_contimeout_sec > 0) {
struct timeval ct;
ct.tv_sec = g_contimeout_sec;
ct.tv_usec = 0;
setsockopt(client->file_descriptor, SOL_SOCKET, SO_RCVTIMEO, &ct, sizeof(ct));
setsockopt(client->file_descriptor, SOL_SOCKET, SO_SNDTIMEO, &ct, sizeof(ct));
}
if (bind_addr_family != 0) {
if (rp->ai_family != bind_addr_family) {
+109 -165
View File
@@ -584,22 +584,41 @@ bool path_under_skip_prefix(const char* child_rel, bool at_root, const DeleteSki
return false;
}
/* All-or-nothing max-delete needs to know BEFORE any unlink whether the run
would delete more than max_delete entries. This rehearsal pass walks the
destination with the same decisions as the delete pass but never touches the
filesystem: it counts every regular file the delete pass would unlink and
every directory it would rmdir (a directory is removed only once every entry
below it has been removed and nothing the walker leaves in place survives).
Entries the walker never removes (symlinks, manifest-listed files, protected
prefixes) mark the enclosing directory as surviving, exactly as they would
make a real rmdir fail with ENOTEMPTY. Stops early once *count reaches the
cap (sets *exceeds). Returns false on a traversal error. */
static bool count_extras_fd(int dirfd, const char* rel_path, const PathIndex* keep, size_t cap,
size_t* count, bool* exceeds, const DeleteSkipEntry* skips,
int skip_count, bool* survives) {
/* Per-run deletion budget and tallies. `max_delete` is the cap on the number
of entries the walker may remove (SIZE_MAX = unlimited); once it is reached
the remaining extras are counted in `skipped` and left in place, matching
rsync's partial --max-delete behavior. */
typedef struct {
size_t max_delete;
size_t deleted;
size_t skipped;
bool limit_hit;
} DeleteBudget;
/* True when direct children of the directory named by `rel` may be removed.
With no synchronization info (dirs == NULL) the whole tree is deletable; when
a dirs index is supplied only its exact entries are (the receive root is the
"." sentinel). */
static bool is_synced_dir(const PathIndex* dirs, const char* rel) {
if (!dirs)
return true;
return path_index_contains(dirs, rel[0] == '\0' ? "." : rel);
}
/* Remove the extras directly inside the directory open on `dirfd`, recursing
into every child directory so kept content below a synchronized prefix is
reached. `all_removed` reports whether every child entry was removed (so the
caller may rmdir this directory). A child directory is never removed when it
is itself a synchronized directory or holds kept content; with a dirs index
supplied, direct children of a non-synchronized directory are never extras at
all (they are left in place but still descended into). Symlinks are unlinked
like any other non-directory extra (never followed). */
static bool delete_extras_fd(int dirfd, const char* rel_path, const PathIndex* keep,
const PathIndex* dirs, DeleteBudget* budget,
const DeleteSkipEntry* skips, int skip_count, bool parent_deletable,
bool* all_removed) {
/* openat(dirfd, ".") opens an independent file description: a dup() would
share dirfd's file offset, and a prior rehearsal pass must not have drained
this directory's stream before the delete pass reads it again. */
share dirfd's file offset and a prior pass could leave the stream drained. */
int scanfd = openat(dirfd, ".", O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
if (scanfd < 0)
return false;
@@ -610,93 +629,10 @@ static bool count_extras_fd(int dirfd, const char* rel_path, const PathIndex* ke
}
bool operation_ok = true;
bool local_survives = false;
bool at_root = rel_path[0] == '\0';
const struct dirent* entry;
while ((entry = readdir(dir)) != NULL) {
if (strcmp(entry->d_name, ".") == 0 || strcmp(entry->d_name, "..") == 0)
continue;
if (*exceeds)
break;
char* child_rel = path_cat((char*)rel_path, entry->d_name);
if (!child_rel) {
operation_ok = false;
continue;
}
if (path_under_skip_prefix(child_rel, at_root, skips, skip_count)) {
local_survives = true;
free(child_rel);
continue;
}
struct stat st;
if (fstatat(dirfd, entry->d_name, &st, AT_SYMLINK_NOFOLLOW) != 0) {
if (errno != ENOENT)
operation_ok = false;
free(child_rel);
continue;
}
if (S_ISLNK(st.st_mode)) {
local_survives = true;
free(child_rel);
continue;
}
if (S_ISDIR(st.st_mode)) {
int childfd = openat(dirfd, entry->d_name, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
bool child_ok = true;
bool child_survives = true;
if (childfd >= 0) {
child_ok = count_extras_fd(childfd, child_rel, keep, cap, count, exceeds, skips, skip_count,
&child_survives);
close(childfd);
} else if (errno != ENOENT) {
operation_ok = false;
}
if (!child_ok)
operation_ok = false;
if (keep_is_dir(keep, child_rel)) {
/* A directory with kept content below it is never removed. */
local_survives = true;
} else if (child_survives) {
/* The directory still holds entries the walker leaves in place, so an
rmdir would fail with ENOTEMPTY; the delete pass leaves it behind
rather than reporting an error (matching rsync). */
local_survives = true;
} else {
if (*count >= cap) {
*exceeds = true;
} else {
(*count)++;
}
}
} else {
bool found = keep_is_file(keep, child_rel);
if (!found) {
if (*count >= cap) {
*exceeds = true;
} else {
(*count)++;
}
}
}
free(child_rel);
}
closedir(dir);
*survives = local_survives;
return operation_ok;
}
static bool delete_extras_fd(int dirfd, const char* rel_path, const PathIndex* keep,
size_t max_delete, size_t* deleted_count, const DeleteSkipEntry* skips,
int skip_count) {
/* Independent file description (see count_extras_fd). */
int scanfd = openat(dirfd, ".", O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
if (scanfd < 0)
return false;
DIR* dir = fdopendir(scanfd);
if (!dir) {
close(scanfd);
return false;
}
bool operation_ok = true;
/* A directory is deletable when it or ANY ancestor is synchronized; the
`parent_deletable` flag carries that down the recursion so dest-only
directories below a synchronized root are removed wholesale. */
bool deletable = parent_deletable || is_synced_dir(dirs, rel_path);
const struct dirent* entry;
while ((entry = readdir(dir)) != NULL) {
if (strcmp(entry->d_name, ".") == 0 || strcmp(entry->d_name, "..") == 0)
@@ -714,6 +650,7 @@ static bool delete_extras_fd(int dirfd, const char* rel_path, const PathIndex* k
destination directory that happens to be called .fastsync-stage is
ordinary content. */
if (path_under_skip_prefix(child_rel, rel_path[0] == '\0', skips, skip_count)) {
local_survives = true;
free(child_rel);
continue;
}
@@ -724,55 +661,58 @@ static bool delete_extras_fd(int dirfd, const char* rel_path, const PathIndex* k
free(child_rel);
continue;
}
// Skip symlinks to prevent following them outside the destination tree
if (S_ISLNK(st.st_mode)) {
free(child_rel);
continue;
}
if (S_ISDIR(st.st_mode)) {
int childfd = openat(dirfd, entry->d_name, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
bool child_removed = false;
bool child_all_removed = false;
if (childfd >= 0) {
child_removed = delete_extras_fd(childfd, child_rel, keep, max_delete, deleted_count, skips,
skip_count);
if (!child_removed)
if (!delete_extras_fd(childfd, child_rel, keep, dirs, budget, skips, skip_count, deletable,
&child_all_removed))
operation_ok = false;
close(childfd);
} else if (errno != ENOENT) {
operation_ok = false;
}
if (child_removed && !keep_is_dir(keep, child_rel)) {
if (*deleted_count >= max_delete) {
operation_ok = false;
bool child_synced = dirs && path_index_contains(dirs, child_rel);
if (child_synced || keep_is_dir(keep, child_rel)) {
/* A synchronized directory and a directory holding kept content are
never removed. */
local_survives = true;
} else if (child_all_removed && deletable) {
if (budget->deleted >= budget->max_delete) {
budget->limit_hit = true;
budget->skipped++;
local_survives = true;
} else if (unlinkat(dirfd, entry->d_name, AT_REMOVEDIR) != 0) {
/* ENOENT: already gone (fine). ENOTEMPTY/EEXIST: the directory
still holds entries the walker leaves in place (a protected
excluded prefix, a kept file the manifest protects, a symlink);
rsync leaves such a directory behind, so this is not an error.
Only genuine I/O failures abort the deletion. */
if (errno != ENOENT && errno != ENOTEMPTY && errno != EEXIST)
operation_ok = false;
local_survives = true;
} else {
if (unlinkat(dirfd, entry->d_name, AT_REMOVEDIR) != 0) {
/* ENOENT: already gone (fine). ENOTEMPTY/EEXIST: the directory
still holds entries the walker leaves in place (a protected
excluded prefix, a kept file the manifest protects, a symlink);
rsync leaves such a directory behind, so this is not an error.
Only genuine I/O failures abort the deletion. */
if (errno != ENOENT && errno != ENOTEMPTY && errno != EEXIST)
operation_ok = false;
} else {
(*deleted_count)++;
}
budget->deleted++;
}
} else {
local_survives = true;
}
} else {
// Check if relative path is in manifest
bool found = keep_is_file(keep, child_rel);
if (!found) {
if (*deleted_count >= max_delete) {
if (found || !deletable) {
/* Kept file, or a child of a directory that is not synchronized: never
an extra for this run. */
local_survives = true;
} else if (budget->deleted >= budget->max_delete) {
budget->limit_hit = true;
budget->skipped++;
local_survives = true;
} else if (unlinkat(dirfd, entry->d_name, 0) != 0) {
if (errno != ENOENT)
operation_ok = false;
free(child_rel);
continue;
}
if (unlinkat(dirfd, entry->d_name, 0) != 0) {
if (errno != ENOENT)
operation_ok = false;
} else {
(*deleted_count)++;
}
local_survives = true;
} else {
budget->deleted++;
char* escaped_path = output_escape(child_rel, log_get_8_bit_output());
fprintf(stderr, " Deleted: %s\n", escaped_path ? escaped_path : "<allocation failed>");
free(escaped_path);
@@ -781,21 +721,33 @@ static bool delete_extras_fd(int dirfd, const char* rel_path, const PathIndex* k
free(child_rel);
}
closedir(dir);
*all_removed = !local_survives;
return operation_ok;
}
DeleteWalkResult delete_extras_limited(const char* dest_root, const ArrayList* manifest,
size_t max_delete, const DeleteSkipEntry* skips,
int skip_count, size_t* deleted_out) {
const ArrayList* synced_dirs, size_t max_delete,
const DeleteSkipEntry* skips, int skip_count,
size_t* deleted_out, size_t* skipped_out) {
if (deleted_out)
*deleted_out = 0;
if (skipped_out)
*skipped_out = 0;
if (!manifest)
return DELETE_WALK_ERROR;
/* Index the keep-set once so both passes answer membership in O(path length)
instead of scanning every manifest entry for every destination entry. */
/* Index the keep-set (and the synchronized-dir set, when supplied) once so
membership is answered in O(path length) instead of scanning every entry
for every destination entry. */
PathIndex keep;
if (!build_keep_index(manifest, &keep))
return DELETE_WALK_ERROR;
PathIndex dirs;
bool have_dirs = synced_dirs != NULL;
if (have_dirs &&
!path_index_build(&dirs, (const char* const*)synced_dirs->items, (size_t)synced_dirs->size)) {
path_index_free(&keep);
return DELETE_WALK_ERROR;
}
int rootfd;
int root_fd = utils_get_authorized_root_fd();
if (root_fd >= 0) {
@@ -810,39 +762,31 @@ DeleteWalkResult delete_extras_limited(const char* dest_root, const ArrayList* m
}
if (rootfd < 0) {
path_index_free(&keep);
if (have_dirs)
path_index_free(&dirs);
return DELETE_WALK_ERROR;
}
if (max_delete != SIZE_MAX) {
/* Rehearse the deletion first so a run that would exceed the cap removes
nothing (rsync's all-or-nothing --max-delete contract). */
size_t count = 0;
bool exceeds = false;
bool survives = false;
bool counted_ok = count_extras_fd(rootfd, "", &keep, max_delete, &count, &exceeds, skips,
skip_count, &survives);
if (!counted_ok) {
close(rootfd);
path_index_free(&keep);
return DELETE_WALK_ERROR;
}
if (exceeds) {
close(rootfd);
path_index_free(&keep);
return DELETE_WALK_LIMIT_EXCEEDED;
}
}
size_t deleted_count = 0;
bool ok = delete_extras_fd(rootfd, "", &keep, max_delete, &deleted_count, skips, skip_count);
DeleteBudget budget = {.max_delete = max_delete, .deleted = 0, .skipped = 0, .limit_hit = false};
bool all_removed = false;
bool ok = delete_extras_fd(rootfd, "", &keep, have_dirs ? &dirs : NULL, &budget, skips,
skip_count, false, &all_removed);
if (close(rootfd) != 0)
ok = false;
path_index_free(&keep);
if (have_dirs)
path_index_free(&dirs);
if (deleted_out)
*deleted_out = deleted_count;
return ok ? DELETE_WALK_OK : DELETE_WALK_ERROR;
*deleted_out = budget.deleted;
if (skipped_out)
*skipped_out = budget.skipped;
if (!ok)
return DELETE_WALK_ERROR;
return budget.limit_hit ? DELETE_WALK_LIMIT_REACHED : DELETE_WALK_OK;
}
bool delete_extras(const char* dest_root, const ArrayList* manifest) {
return delete_extras_limited(dest_root, manifest, SIZE_MAX, NULL, 0, NULL) == DELETE_WALK_OK;
return delete_extras_limited(dest_root, manifest, NULL, SIZE_MAX, NULL, 0, NULL, NULL) ==
DELETE_WALK_OK;
}
bool has_path_traversal(const char* path) {
+19 -16
View File
@@ -98,10 +98,10 @@ bool glob_match(const char* pattern, const char* str);
typedef enum {
/* Every extra entry was removed (or there were none). */
DELETE_WALK_OK = 0,
/* The destination holds more extras than the numeric cap for this run. With
the all-or-nothing max-delete semantics NOTHING was removed (the walker
counts first and refuses to start when the run would exceed the limit). */
DELETE_WALK_LIMIT_EXCEEDED,
/* The numeric cap for this run was reached before every extra was removed.
The walker removed exactly the entries the cap allowed and skipped (without
removing) the rest, matching rsync's partial --max-delete behavior. */
DELETE_WALK_LIMIT_REACHED,
/* A traversal or unlink failure aborted the deletion (partial removal is
possible, mirroring the delete pass). */
DELETE_WALK_ERROR
@@ -122,19 +122,22 @@ typedef struct {
only DIRECT children of the destination root, i.e. child_rel has no '/'). */
bool path_under_skip_prefix(const char* child_rel, bool at_root, const DeleteSkipEntry* skips,
int skip_count);
/* Remove files/dirs under dest_root that are not listed in manifest without
ever descending into a protected prefix (see DeleteSkipEntry). When
max_delete is not SIZE_MAX the run is all-or-nothing: extras are counted
first and DELETE_WALK_LIMIT_EXCEEDED is returned (with nothing removed) when
the count would exceed the cap. `deleted_out` optionally receives the number
of entries actually removed. The all-or-nothing guarantee holds only while
the destination tree is not being concurrently modified: the rehearsal pass
and the delete pass are two separate walks, so a concurrent change between
them (another process adding/removing entries) can make the second pass
delete a different set than the first one counted. */
/* Remove files/dirs/symlinks under dest_root that are not listed in manifest
without ever descending into a protected prefix (see DeleteSkipEntry). When
`synced_dirs` is non-NULL, extras are only removed directly inside a directory
whose destination-relative path is an exact entry in that list (the receive
root is the "." sentinel); directories outside the synchronized set are still
descended into so kept content below a listed directory is preserved, but
nothing in them is removed. A NULL `synced_dirs` keeps the legacy behavior of
treating the whole destination tree as deletable. `max_delete` caps the
number of removed entries (SIZE_MAX = unlimited): the walker removes up to the
cap and returns DELETE_WALK_LIMIT_REACHED when more extras remained.
`deleted_out`/`skipped_out` optionally receive the number of entries removed
and the number skipped because of the cap. */
DeleteWalkResult delete_extras_limited(const char* dest_root, const ArrayList* manifest,
size_t max_delete, const DeleteSkipEntry* skips,
int skip_count, size_t* deleted_out);
const ArrayList* synced_dirs, size_t max_delete,
const DeleteSkipEntry* skips, int skip_count,
size_t* deleted_out, size_t* skipped_out);
bool delete_extras(const char* dest_root, const ArrayList* manifest);
bool utils_set_authorized_root(int fd, const char* canonical_path);
/* The fd-only compatibility form is fail-closed for path-based operations;
+56 -44
View File
@@ -2,6 +2,7 @@
#include "xattr.h"
#include "identity.h"
#include "log.h"
#include "metadata.h"
#include "protocol.h"
#include "utils.h"
#include "file_types.h"
@@ -37,6 +38,22 @@ void xattr_list_free(FileXattrList* list) {
free(list);
}
FileXattrList* xattr_list_clone(const FileXattrList* list) {
if (!list)
return NULL;
FileXattrList* clone = xattr_list_new();
if (!clone)
return NULL;
for (int i = 0; i < list->count; i++) {
if (!xattr_list_append(clone, list->items[i].name, list->items[i].value,
list->items[i].value_len)) {
xattr_list_free(clone);
return NULL;
}
}
return clone;
}
bool xattr_list_append(FileXattrList* list, const char* name, const void* value, size_t value_len) {
if (!list || !name || (!value && value_len != 0))
return false;
@@ -364,26 +381,12 @@ void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, int6
}
}
/* --fake-super replay: read the freshly-stored record and re-apply the source
* stat fd-relative. A privileged (root) run can actually change the owner;
* a non-root run silently skips the fchown on EPERM/EACCES (never fatal,
* mirroring the normal metadata identity path; other errors are logged) and
* still applies mode/mtime where permitted.
*
* The OWNER leg additionally honors three policies:
* - an explicit ownership identity policy must be active (numeric-ids /
* chown / usermap / groupmap / copy-as). --fake-super on its own only
* RECORDS the source owner; replaying that owner as a live chown without an
* explicit ownership opt-in would be an un-gated client-chosen-ownership
* primitive.
* - --no-super (privilege_super_permitted() false) suppresses it even for a
* root receiver, exactly like the normal metadata identity path.
* - an active --copy-as is AUTHORITATIVE: the identity path already forced the
* target owner, so replaying the recorded source owner here would silently
* override it. The xattr record is still stored/replayed for a later
* privileged restore; only the live chown is skipped. Mode/mtime remain
* applied either way so unprivileged --fake-super still works. */
bool fake_super_restore_fd(int fd) {
/* --fake-super replay: read the freshly-stored record and re-apply mode/mtime
* fd-relative. The recorded uid/gid are retained for a later privileged
* restore but are NEVER chowned here: --fake-super only RECORDS ownership, it
* must not real-chown the recorded (resolved) owner. Mode/mtime still apply so
* unprivileged --fake-super keeps working. */
bool fake_super_restore_fd(int fd, FileAttrPolicy policy) {
if (fd < 0)
return false;
char record[128];
@@ -398,28 +401,37 @@ bool fake_super_restore_fd(int fd) {
5)
return false; /* malformed record: skip, never fatal */
/* Owner is applied best-effort only: a non-root process cannot chown and
must not abort the transfer for that reason (FastSync identity philosophy).
EPERM/EACCES (expected for a non-root receiver) are skipped silently; a
genuine EINVAL (an impossible stored id) is logged so the corruption is
not hidden. --no-super suppresses the owner leg even for root, and an
active --copy-as is authoritative so its forced owner must not be
overwritten by the recorded source owner. */
if (identity_active_enabled() && privilege_super_permitted() && !identity_copy_as_active() &&
fchown(fd, (uid_t)ul_uid, (gid_t)ul_gid) != 0 && errno != EPERM && errno != EACCES)
log_message(LOG_LEVEL_WARNING, "--fake-super: could not restore owner on destination file: %s",
strerror(errno));
/* Mode is applied through the same sanitization the normal metadata path
uses (metadata_mode): group/other write bits are never granted, so a
recorded source mode of 0666 restores as 0644 — identical to a non-fake-
super --preserve run, never a privilege-granting regression. */
if (fchmod(fd, (mode_t)(ul_mode & 0777U & ~(S_IWGRP | S_IWOTH))) != 0)
log_message(LOG_LEVEL_WARNING, "--fake-super: could not restore mode on destination file: %s",
strerror(errno));
struct timespec times[2] = {{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
{.tv_sec = (time_t)mtime_sec, .tv_nsec = mtime_nsec}};
if (futimens(fd, times) != 0)
log_message(LOG_LEVEL_WARNING, "--fake-super: could not restore mtime on destination file: %s",
strerror(errno));
/* --fake-super NEVER performs a real chown: that would defeat the whole
point of the flag (record privileged ownership on an unprivileged receiver
for a later privileged restore). The uid/gid parsed above are retained in
the record for that later restore, but no ownership change happens here. */
(void)ul_uid;
(void)ul_gid;
/* Mode is applied only when the per-attribute policy asks for it, through the
SAME shared helper the normal metadata path uses (metadata_mode_for_policy):
group/other write bits are never granted, so a recorded source mode of 0666
restores as 0644 — identical to a non-fake-super --preserve run, never a
privilege-granting regression — and the -E rule derives exec bits from the
destination's read bits exactly like file_restore_metadata_fd. */
if (policy.perms || policy.executability) {
struct stat cur;
mode_t want = 0;
if (fstat(fd, &cur) != 0) {
log_message(LOG_LEVEL_WARNING, "--fake-super: could not read destination mode: %s",
strerror(errno));
} else if (metadata_mode_for_policy((mode_t)ul_mode, cur.st_mode, policy, &want)) {
if (fchmod(fd, want) != 0)
log_message(LOG_LEVEL_WARNING,
"--fake-super: could not restore mode on destination file: %s",
strerror(errno));
}
}
if (policy.times) {
struct timespec times[2] = {{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
{.tv_sec = (time_t)mtime_sec, .tv_nsec = mtime_nsec}};
if (futimens(fd, times) != 0)
log_message(LOG_LEVEL_WARNING,
"--fake-super: could not restore mtime on destination file: %s", strerror(errno));
}
return true;
}
}
+14 -10
View File
@@ -1,6 +1,7 @@
#ifndef XATTR_H
#define XATTR_H
#include "file_attr.h"
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
@@ -55,6 +56,8 @@ typedef struct {
FileXattrList* xattr_list_new(void);
void xattr_list_free(FileXattrList* list);
/* Deep-copy `list` (NULL in, NULL out). Returns NULL on allocation failure. */
FileXattrList* xattr_list_clone(const FileXattrList* list);
/* Append one entry (deep copy). Returns false on allocation failure. */
bool xattr_list_append(FileXattrList* list, const char* name, const void* value, size_t value_len);
@@ -95,15 +98,16 @@ void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, int6
int64_t mtime_nsec);
/* --fake-super replay: parse the FAKESUPER_XATTR record previously written on
* `fd` by fake_super_store_fd and re-apply uid/gid/mode/mtime fd-relative.
* Best-effort: absence of the xattr or a malformed record is a silent no-op
* that never fails the transfer. The OWNER leg is applied only when an explicit
* ownership identity policy is active (numeric-ids/chown/usermap/groupmap/
* copy-as), when super-user activities are permitted, and when --copy-as is not
* authoritative; a non-root EPERM/EACCES is skipped silently, matching
* FastSync's identity philosophy. The mode is sanitized exactly like the normal
* metadata path (group/other write bits never granted). Returns true when the
* xattr was present and parsed. */
bool fake_super_restore_fd(int fd);
* `fd` by fake_super_store_fd and re-apply mode/mtime fd-relative. The
* recorded uid/gid are deliberately NOT chowned for real: --fake-super only
* RECORDS ownership (the caller stores the resolved mapping via
* identity_resolve_storage_ids), it never performs a real chown. Best-effort:
* absence of the xattr or a malformed record is a silent no-op that never fails
* the transfer. The MODE leg is applied only when policy.perms||policy.
* executability and the MTIME leg only when policy.times, so the fake-super
* replay cannot bypass the per-attribute split; the mode is sanitized exactly
* like the normal metadata path (group/other write bits never granted).
* Returns true when the xattr was present and parsed. */
bool fake_super_restore_fd(int fd, FileAttrPolicy policy);
#endif
+2
View File
@@ -115,7 +115,9 @@ static void build_canonical_frame(void) {
if (cfg->usermap) {
cfg->usermap_count = 1;
cfg->usermap[0].from = MAP_FROM;
cfg->usermap[0].from_hi = MAP_FROM;
cfg->usermap[0].to = MAP_TO;
cfg->usermap[0].to_name = NULL;
}
if (!cfg->send_directory || !cfg->receive_root_directory || !cfg->usermap) {
config_delete(cfg);
+7
View File
@@ -254,6 +254,13 @@ def _wait_for_port(port, timeout=5):
def _wait_proc(proc, timeout=5):
"""Stop a long-lived subprocess promptly. The server installs a SIGTERM
handler, so signal first and only escalate to SIGKILL if it does not exit;
waiting without signalling would burn the full timeout on every stop."""
if proc.poll() is not None:
proc.wait()
return
proc.terminate()
try:
proc.wait(timeout=timeout)
except subprocess.TimeoutExpired:
+5 -1
View File
@@ -36,7 +36,7 @@ from common import ( # noqa: E402
verify_transfer,
)
PROTOCOL_VERSION = b"2.21.0"
PROTOCOL_VERSION = b"2.23.0"
STATUS_MANIFEST = 5
STATUS_OK = 0
@@ -59,6 +59,10 @@ def _seed_source():
if os.path.exists(SOURCE_DIR):
shutil.rmtree(SOURCE_DIR)
os.makedirs(os.path.join(SOURCE_DIR, "nested"))
# The receiver rejects a destination root that does not exist, and the
# capture fixture can run before any test that creates it, so create it
# here (test order/distribution must not matter).
os.makedirs(DEST_DIR, exist_ok=True)
with open(os.path.join(SOURCE_DIR, "hello.txt"), "wb") as fh:
fh.write(b"fault injection payload\n" * 64)
with open(os.path.join(SOURCE_DIR, "nested", "deep.bin"), "wb") as fh:
File diff suppressed because it is too large. Load diff
+282
View File
@@ -0,0 +1,282 @@
"""Output-parity tests (#291 selection/output, #292 output formatting).
These tests exercise rsync-style selection ordering and output formatting. The
differential tests run the SAME transfer with real ``rsync 3.4.1`` and with
fastsync and compare stdout, so they are skipped when rsync is unavailable.
"""
import os
import shutil
import subprocess
import sys
import pytest
sys.path.insert(0, os.path.dirname(__file__))
from common import TEST_DATA_DIR, run_client, clean_dir, get_dest_received_dir
RSYNC = shutil.which("rsync")
requires_rsync = pytest.mark.skipif(RSYNC is None, reason="rsync 3.4.1 not installed")
def _rsync(args):
env = dict(os.environ, LC_ALL="C")
return subprocess.run(
[RSYNC] + args, capture_output=True, text=True, env=env, timeout=120
)
def _make_selection_tree(root):
clean_dir(root)
os.makedirs(os.path.join(root, "sub"))
with open(os.path.join(root, "a.txt"), "wb") as fh:
fh.write(b"top text\n")
with open(os.path.join(root, "b.log"), "wb") as fh:
fh.write(b"log data\n")
with open(os.path.join(root, "sub", "c.txt"), "wb") as fh:
fh.write(b"nested text\n")
with open(os.path.join(root, "sub", "d.log"), "wb") as fh:
fh.write(b"nested log\n")
class TestSelectionOrdering:
"""#291: --include/--exclude compile into one ordered rule list."""
@pytest.mark.ci
def test_include_then_exclude_keeps_only_matching(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "out_inc_src")
dest = os.path.join(TEST_DATA_DIR, "out_inc_dst")
_make_selection_tree(source)
clean_dir(dest)
result, _ = run_client(
source, dest,
flags=["--preserve", "--include=*.txt", "--exclude=*"],
port=shared_server.port,
)
assert result.returncode == 0, f"include/exclude failed: {result.stderr[:300]}"
received = get_dest_received_dir(dest, source)
assert os.path.exists(os.path.join(received, "a.txt"))
# `*` also excludes the directory, so nothing below sub/ is sent.
assert not os.path.exists(os.path.join(received, "b.log"))
assert not os.path.exists(os.path.join(received, "sub", "c.txt"))
@pytest.mark.ci
def test_include_dirs_then_files_idiom(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "out_inc2_src")
dest = os.path.join(TEST_DATA_DIR, "out_inc2_dst")
_make_selection_tree(source)
clean_dir(dest)
result, _ = run_client(
source, dest,
flags=["--preserve", "--include=*/", "--include=*.txt", "--exclude=*"],
port=shared_server.port,
)
assert result.returncode == 0, f"include/exclude failed: {result.stderr[:300]}"
received = get_dest_received_dir(dest, source)
assert os.path.exists(os.path.join(received, "a.txt"))
assert os.path.exists(os.path.join(received, "sub", "c.txt"))
assert not os.path.exists(os.path.join(received, "b.log"))
assert not os.path.exists(os.path.join(received, "sub", "d.log"))
@requires_rsync
def test_include_idiom_matches_rsync_selection(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "out_inc3_src")
dest = os.path.join(TEST_DATA_DIR, "out_inc3_dst")
rdst = os.path.join(TEST_DATA_DIR, "out_inc3_rdst")
_make_selection_tree(source)
clean_dir(dest)
clean_dir(rdst)
flags = ["--include=*/", "--include=*.txt", "--exclude=*"]
rsync_result = _rsync(["-a"] + flags + [source + "/", rdst + "/"])
assert rsync_result.returncode == 0, rsync_result.stderr
result, _ = run_client(source, dest, flags=["--preserve"] + flags,
port=shared_server.port)
assert result.returncode == 0
received = get_dest_received_dir(dest, source)
assert os.path.exists(os.path.join(received, "a.txt"))
assert os.path.exists(os.path.join(received, "sub", "c.txt"))
assert not os.path.exists(os.path.join(received, "b.log"))
# rsync -a src/ dst/ writes directly into dst/
assert os.path.exists(os.path.join(rdst, "a.txt"))
assert os.path.exists(os.path.join(rdst, "sub", "c.txt"))
assert not os.path.exists(os.path.join(rdst, "b.log"))
class TestOneFileSystem:
"""#291: -x emits the mount-point directory but not its contents."""
def test_one_file_system_emits_mount_point_dir(self, shared_server):
local = os.stat(".")
shm = "/dev/shm"
try:
shm_stat = os.stat(shm)
except OSError:
pytest.skip("/dev/shm not available")
if shm_stat.st_dev == local.st_dev:
pytest.skip("no cross-device filesystem available")
source = os.path.join(TEST_DATA_DIR, "out_ofs_src")
dest = os.path.join(TEST_DATA_DIR, "out_ofs_dst")
clean_dir(source)
clean_dir(dest)
os.makedirs(os.path.join(source, "nested"))
os.makedirs(os.path.join(shm, "fastsync_ofs_probe"), exist_ok=True)
with open(os.path.join(source, "keep.txt"), "wb") as fh:
fh.write(b"keep\n")
with open(os.path.join(shm, "fastsync_ofs_probe", "inside.txt"), "wb") as fh:
fh.write(b"cross\n")
link = os.path.join(source, "nested", "link")
try:
os.symlink(os.path.join(shm, "fastsync_ofs_probe"), link)
except OSError:
pytest.skip("cannot create symlink")
try:
result, _ = run_client(
source, dest,
flags=["--preserve", "--copy-links", "-x"],
port=shared_server.port,
)
assert result.returncode == 0, f"-x failed: {result.stderr[:300]}"
received = get_dest_received_dir(dest, source)
assert os.path.exists(os.path.join(received, "keep.txt"))
# The mount-point directory entry is created but its contents are not.
assert os.path.isdir(os.path.join(received, "nested", "link"))
assert not os.path.exists(os.path.join(received, "nested", "link", "inside.txt"))
finally:
shutil.rmtree(os.path.join(shm, "fastsync_ofs_probe"), ignore_errors=True)
def _make_output_tree(root):
clean_dir(root)
os.makedirs(os.path.join(root, "sub"))
with open(os.path.join(root, "a.txt"), "wb") as fh:
fh.write(b"hello\n")
with open(os.path.join(root, "sub", "b.txt"), "wb") as fh:
fh.write("wörld\n".encode("utf-8"))
os.symlink("a.txt", os.path.join(root, "link"))
class TestItemizeParity:
"""#292: -i output matches rsync 3.4.1 for the cases fastsync can observe."""
@requires_rsync
@pytest.mark.ci
def test_itemize_first_transfer_matches_rsync(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "out_item_src")
dest = os.path.join(TEST_DATA_DIR, "out_item_dst")
rdst = os.path.join(TEST_DATA_DIR, "out_item_rdst")
_make_output_tree(source)
clean_dir(dest)
clean_dir(rdst)
rsync_result = _rsync(["-a", "-i", source + "/", rdst + "/"])
assert rsync_result.returncode == 0, rsync_result.stderr
rsync_lines = sorted(
line for line in rsync_result.stdout.splitlines()
if line.startswith(">f") or line.startswith("cL")
)
result, _ = run_client(source, dest, flags=["-a", "-i"],
port=shared_server.port)
assert result.returncode == 0, result.stderr[:300]
fast_lines = sorted(
line for line in result.stdout.splitlines()
if line.startswith(">f") or line.startswith("cL")
)
assert fast_lines == rsync_lines, f"rsync={rsync_lines} fastsync={fast_lines}"
@requires_rsync
@pytest.mark.ci
def test_itemize_modified_file_matches_rsync(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "out_item2_src")
dest = os.path.join(TEST_DATA_DIR, "out_item2_dst")
rdst = os.path.join(TEST_DATA_DIR, "out_item2_rdst")
_make_output_tree(source)
clean_dir(dest)
clean_dir(rdst)
seed = run_client(source, dest, flags=["-a"], port=shared_server.port)
assert seed[0].returncode == 0, seed[0].stderr[:300]
assert _rsync(["-a", source + "/", rdst + "/"]).returncode == 0
with open(os.path.join(source, "a.txt"), "wb") as fh:
fh.write(b"hello changed and longer\n")
# Pin the source mtime so rsync's `t` column is deterministic (a write
# that lands in the same whole second as the seed would not show `t`).
os.utime(os.path.join(source, "a.txt"), (1000000000, 1000000000))
rsync_result = _rsync(["-a", "-i", source + "/", rdst + "/"])
assert rsync_result.returncode == 0, rsync_result.stderr
rsync_lines = sorted(
line for line in rsync_result.stdout.splitlines() if line.startswith(">f")
)
result, _ = run_client(source, dest,
flags=["-a", "-i", "--incremental"],
port=shared_server.port)
assert result.returncode == 0, result.stderr[:300]
fast_lines = sorted(
line for line in result.stdout.splitlines() if line.startswith(">f")
)
assert fast_lines == rsync_lines, f"rsync={rsync_lines} fastsync={fast_lines}"
class TestOutFormatParity:
@requires_rsync
@pytest.mark.ci
def test_out_format_n_l_matches_rsync(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "out_fmt_src")
dest = os.path.join(TEST_DATA_DIR, "out_fmt_dst")
rdst = os.path.join(TEST_DATA_DIR, "out_fmt_rdst")
_make_output_tree(source)
clean_dir(dest)
clean_dir(rdst)
fmt = "%n %l"
rsync_result = _rsync(["-a", "--out-format=" + fmt, source + "/", rdst + "/"])
assert rsync_result.returncode == 0, rsync_result.stderr
rsync_lines = sorted(
line for line in rsync_result.stdout.splitlines()
if line and not line.split(" ", 1)[0].endswith("/")
)
result, _ = run_client(source, dest,
flags=["-a", "--out-format=" + fmt],
port=shared_server.port)
assert result.returncode == 0, result.stderr[:300]
fast_lines = sorted(
line for line in result.stdout.splitlines()
if line and not line.split(" ", 1)[0].endswith("/")
)
assert fast_lines == rsync_lines, f"rsync={rsync_lines} fastsync={fast_lines}"
@requires_rsync
@pytest.mark.ci
def test_out_format_M_datetime_shape(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "out_M_src")
dest = os.path.join(TEST_DATA_DIR, "out_M_dst")
_make_output_tree(source)
clean_dir(dest)
result, _ = run_client(source, dest,
flags=["-a", "--out-format=%M %f"],
port=shared_server.port)
assert result.returncode == 0, result.stderr[:300]
import re
pattern = re.compile(r"^\d{4}/\d{2}/\d{2}-\d{2}:\d{2}:\d{2} ")
for line in result.stdout.splitlines():
if line:
assert pattern.match(line), f"bad %M format: {line!r}"
class TestListOnlyParity:
@requires_rsync
@pytest.mark.ci
def test_list_only_matches_rsync(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "out_list_src")
dest = os.path.join(TEST_DATA_DIR, "out_list_dst")
_make_output_tree(source)
clean_dir(dest)
rsync_result = _rsync(["-r", "--list-only", source + "/"])
assert rsync_result.returncode == 0, rsync_result.stderr
rsync_lines = sorted(rsync_result.stdout.splitlines())
result, _ = run_client(source, dest, flags=["--list-only", "-l"],
port=shared_server.port)
assert result.returncode == 0, result.stderr[:300]
fast_lines = sorted(result.stdout.splitlines())
assert fast_lines == rsync_lines, (
f"rsync={rsync_lines}\nfastsync={fast_lines}"
)
+4 -3
View File
@@ -94,14 +94,14 @@ def _seed_protocol_source(source):
class TestProtocol:
@pytest.mark.ci
def test_protocol_current_version_accepted(self, shared_server):
"""--protocol=2.21.0 (the current PROTOCOL_VERSION) is accepted and the
"""--protocol=2.23.0 (the current PROTOCOL_VERSION) is accepted and the
transfer completes normally."""
source = os.path.join(TEST_DATA_DIR, "proto_ok_src")
dest = os.path.join(TEST_DATA_DIR, "proto_ok_dst")
shutil.rmtree(dest, ignore_errors=True)
os.makedirs(dest)
_seed_protocol_source(source)
result, _ = run_client(source, dest, flags=["--protocol=2.21.0"],
result, _ = run_client(source, dest, flags=["--protocol=2.23.0"],
port=shared_server.port)
assert result.returncode == 0, \
f"--protocol current run failed: {(result.stderr or result.stdout)[:400]}"
@@ -118,7 +118,8 @@ class TestProtocol:
shutil.rmtree(dest, ignore_errors=True)
os.makedirs(dest)
_seed_protocol_source(source)
for bad in ("2.20.0", "2.19.0", "2.18.0", "2.17.0", "2.15.0", "2.16.0", "216", "31"):
for bad in ("2.22.0", "2.21.0", "2.20.0", "2.19.0", "2.18.0", "2.17.0", "2.15.0", "2.16.0",
"216", "31"):
result, _ = run_client(source, dest, flags=[f"--protocol={bad}"],
port=shared_server.port)
assert result.returncode != 0, f"--protocol={bad} should be rejected"
+489
View File
@@ -0,0 +1,489 @@
"""Wave 2b: per-attribute preservation split (-p/-t/-o/-g and their negations).
The receiver applies each attribute independently (see src/shared/file_attr.h).
These tests cover the per-flag behavior end-to-end, the CLI negations, directory
modes, and the unprivileged best-effort / root-only ownership paths. They reuse
the established helpers from common.py.
The `-s` spelling is rsync's --secluded-args no-op in FastSync; chunk
serialization is the long-form --chunk-serialization, which is what the feature
matrix below exercises.
"""
import os
import stat
import sys
import time
import pytest
sys.path.insert(0, os.path.dirname(__file__))
from common import (
TEST_DATA_DIR,
run_client,
clean_dir,
get_dest_received_dir,
ServerManager,
)
DISTINCT_MTIME = 1_000_000_000 # 2001-09-09T01:46:40Z, a whole second
def _process_umask():
current = os.umask(0)
os.umask(current)
return current
def _seed_file(source, dest, name, content, mode, mtime=None):
"""Create a one-file source tree at an explicit mode (and mtime), and a
clean destination. Returns the source file path."""
clean_dir(source)
clean_dir(dest)
path = os.path.join(source, name)
with open(path, "wb") as fh:
fh.write(content)
os.chmod(path, mode)
if mtime is not None:
os.utime(path, (mtime, mtime))
return path
def _received(dest, source, name):
return os.path.join(get_dest_received_dir(dest, source), name)
class TestPreservePerms:
@pytest.mark.ci
def test_p_applies_source_mode(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "perms_p_src")
dest = os.path.join(TEST_DATA_DIR, "perms_p_dst")
_seed_file(source, dest, "f.txt", b"perms\n", 0o750)
result, _ = run_client(source, dest, flags=["-p"], port=shared_server.port)
assert result.returncode == 0, \
f"-p failed: {(result.stderr or result.stdout)[:300]}"
got = os.stat(_received(dest, source, "f.txt")).st_mode & 0o777
assert got == 0o750, f"-p must apply the source mode, got {oct(got)}"
@pytest.mark.ci
def test_without_p_preexisting_dest_keeps_mode(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "perms_nop_exist_src")
dest = os.path.join(TEST_DATA_DIR, "perms_nop_exist_dst")
src_file = _seed_file(source, dest, "f.txt", b"one\n", 0o750)
# Seed the destination.
result, _ = run_client(source, dest, flags=["-p"], port=shared_server.port)
assert result.returncode == 0, f"seed failed: {(result.stderr or '')[:200]}"
# Give the destination a distinguishable mode, then re-transfer without
# -p (but with -t so metadata still travels).
dst_file = _received(dest, source, "f.txt")
os.chmod(dst_file, 0o600)
with open(src_file, "wb") as fh:
fh.write(b"two, changed content\n")
result, _ = run_client(source, dest, flags=["-t"], port=shared_server.port)
assert result.returncode == 0, f"re-run failed: {(result.stderr or '')[:200]}"
got = os.stat(dst_file).st_mode & 0o777
assert got == 0o600, \
f"without -p a pre-existing destination must keep its mode, got {oct(got)}"
with open(dst_file, "rb") as fh:
assert fh.read() == b"two, changed content\n"
@pytest.mark.ci
def test_without_p_new_dest_gets_source_and_umask(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "perms_nop_new_src")
dest = os.path.join(TEST_DATA_DIR, "perms_nop_new_dst")
# 0664 has group/other bits that the umask strips, so the result is not
# just the source mode. Under strict rsync parity the source mode is
# masked only by the umask (group/other write is no longer force-cleared
# on top of it).
_seed_file(source, dest, "f.txt", b"new\n", 0o664)
result, _ = run_client(source, dest, flags=["-t"], port=shared_server.port)
assert result.returncode == 0, f"-t failed: {(result.stderr or result.stdout)[:300]}"
want = 0o664 & ~_process_umask()
got = os.stat(_received(dest, source, "f.txt")).st_mode & 0o777
assert got == want, \
f"new no--p destination mode: want {oct(want)}, got {oct(got)}"
class TestPreserveTimes:
@pytest.mark.ci
def test_t_applies_mtime(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "times_t_src")
dest = os.path.join(TEST_DATA_DIR, "times_t_dst")
_seed_file(source, dest, "f.txt", b"times\n", 0o644, mtime=DISTINCT_MTIME)
result, _ = run_client(source, dest, flags=["-t"], port=shared_server.port)
assert result.returncode == 0, f"-t failed: {(result.stderr or '')[:300]}"
dst_m = os.stat(_received(dest, source, "f.txt")).st_mtime
assert abs(dst_m - DISTINCT_MTIME) < 2, \
f"-t must apply the source mtime, got {dst_m}"
@pytest.mark.ci
def test_without_t_dest_mtime_differs(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "times_not_src")
dest = os.path.join(TEST_DATA_DIR, "times_not_dst")
_seed_file(source, dest, "f.txt", b"times\n", 0o644, mtime=DISTINCT_MTIME)
# -p transmits metadata but must not apply the source mtime.
result, _ = run_client(source, dest, flags=["-p"], port=shared_server.port)
assert result.returncode == 0, f"-p failed: {(result.stderr or '')[:300]}"
dst_m = os.stat(_received(dest, source, "f.txt")).st_mtime
assert abs(dst_m - DISTINCT_MTIME) > 24 * 3600, \
f"without -t the destination mtime must not be the source mtime ({dst_m})"
@pytest.mark.ci
def test_incremental_t_retransfers_after_no_t(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "times_incr_src")
dest = os.path.join(TEST_DATA_DIR, "times_incr_dst")
_seed_file(source, dest, "f.txt", b"retransfer\n", 0o644, mtime=DISTINCT_MTIME)
# First run without -t: the destination mtime becomes "now", differing
# from the pinned source mtime.
result, _ = run_client(source, dest, flags=["-p"], port=shared_server.port)
assert result.returncode == 0, f"seed failed: {(result.stderr or '')[:200]}"
dst_file = _received(dest, source, "f.txt")
assert abs(os.stat(dst_file).st_mtime - DISTINCT_MTIME) > 24 * 3600
# The incremental quick-check now sees a mtime mismatch, so the file is
# re-transferred and -t stamps the source time.
result, _ = run_client(source, dest, flags=["--incremental", "-t"],
port=shared_server.port)
assert result.returncode == 0, f"incremental -t failed: {(result.stderr or '')[:300]}"
dst_m = os.stat(dst_file).st_mtime
assert abs(dst_m - DISTINCT_MTIME) < 2, \
f"second --incremental -t run must re-transfer and stamp the mtime, got {dst_m}"
class TestPreserveNegations:
@pytest.mark.ci
def test_a_no_owner_no_group_keeps_perms_and_times(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "neg_owner_group_src")
dest = os.path.join(TEST_DATA_DIR, "neg_owner_group_dst")
_seed_file(source, dest, "f.txt", b"neg\n", 0o750, mtime=DISTINCT_MTIME)
result, _ = run_client(source, dest, flags=["-a", "--no-owner", "--no-group"],
port=shared_server.port)
assert result.returncode == 0, f"-a --no-owner --no-group: {(result.stderr or '')[:300]}"
st = os.stat(_received(dest, source, "f.txt"))
assert st.st_mode & 0o777 == 0o750, "perms must survive the owner/group negation"
assert abs(st.st_mtime - DISTINCT_MTIME) < 2, "times must survive the owner/group negation"
@pytest.mark.ci
def test_a_no_perms_keeps_times_and_dest_mode(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "neg_perms_src")
dest = os.path.join(TEST_DATA_DIR, "neg_perms_dst")
src_file = _seed_file(source, dest, "f.txt", b"one\n", 0o750, mtime=DISTINCT_MTIME)
result, _ = run_client(source, dest, flags=["-a"], port=shared_server.port)
assert result.returncode == 0, f"seed failed: {(result.stderr or '')[:200]}"
dst_file = _received(dest, source, "f.txt")
os.chmod(dst_file, 0o600)
with open(src_file, "wb") as fh:
fh.write(b"changed\n")
# Rewriting the source bumped its mtime; restore the pinned value so the
# --no-perms run still has a distinct source time to apply.
os.utime(src_file, (DISTINCT_MTIME, DISTINCT_MTIME))
result, _ = run_client(source, dest, flags=["-a", "--no-perms"],
port=shared_server.port)
assert result.returncode == 0, f"-a --no-perms: {(result.stderr or '')[:300]}"
st = os.stat(dst_file)
assert st.st_mode & 0o777 == 0o600, \
f"--no-perms must keep the destination mode, got {oct(st.st_mode & 0o777)}"
assert abs(st.st_mtime - DISTINCT_MTIME) < 2, "--no-perms must not disable times"
@pytest.mark.ci
def test_a_no_times_keeps_perms_but_not_mtime(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "neg_times_src")
dest = os.path.join(TEST_DATA_DIR, "neg_times_dst")
_seed_file(source, dest, "f.txt", b"neg times\n", 0o750, mtime=DISTINCT_MTIME)
result, _ = run_client(source, dest, flags=["-a", "--no-times"],
port=shared_server.port)
assert result.returncode == 0, f"-a --no-times: {(result.stderr or '')[:300]}"
st = os.stat(_received(dest, source, "f.txt"))
assert st.st_mode & 0o777 == 0o750, "--no-times must not disable perms"
assert abs(st.st_mtime - DISTINCT_MTIME) > 24 * 3600, \
"--no-times must not apply the source mtime"
@pytest.mark.ci
def test_preserve_no_preserve_clears_all(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "neg_bundle_src")
dest = os.path.join(TEST_DATA_DIR, "neg_bundle_dst")
_seed_file(source, dest, "f.txt", b"bundle\n", 0o750, mtime=DISTINCT_MTIME)
result, _ = run_client(source, dest, flags=["--preserve", "--no-preserve"],
port=shared_server.port)
assert result.returncode == 0, f"--preserve --no-preserve: {(result.stderr or '')[:300]}"
dst_file = _received(dest, source, "f.txt")
with open(dst_file, "rb") as fh:
assert fh.read() == b"bundle\n"
st = os.stat(dst_file)
# No metadata travels at all: a new file gets the fixed safe 0644 and
# the source mtime is not applied.
assert st.st_mode & 0o777 == 0o644, \
f"--no-preserve must not apply the source mode, got {oct(st.st_mode & 0o777)}"
assert abs(st.st_mtime - DISTINCT_MTIME) > 24 * 3600, \
"--no-preserve must not apply the source mtime"
class TestDirectoryModes:
def _tree(self, name, dir_mode, pin_mtime):
source = os.path.join(TEST_DATA_DIR, name + "_src")
dest = os.path.join(TEST_DATA_DIR, name + "_dst")
clean_dir(source)
clean_dir(dest)
sub = os.path.join(source, "sub")
os.makedirs(sub)
with open(os.path.join(sub, "file.txt"), "wb") as fh:
fh.write(b"dir mode content\n")
os.chmod(sub, dir_mode)
if pin_mtime:
os.utime(sub, (DISTINCT_MTIME, DISTINCT_MTIME))
return source, dest, sub
@pytest.mark.ci
def test_p_applies_directory_mode(self, shared_server):
source, dest, _ = self._tree("dirmode_p", 0o750, pin_mtime=False)
result, _ = run_client(source, dest, flags=["-p"], port=shared_server.port)
assert result.returncode == 0, f"-p failed: {(result.stderr or result.stdout)[:300]}"
got = os.stat(os.path.join(get_dest_received_dir(dest, source), "sub")).st_mode & 0o777
assert got == 0o750, f"-p must apply the source directory mode, got {oct(got)}"
@pytest.mark.ci
def test_p_preserves_directory_group_other_write(self, shared_server):
# Strict rsync parity: -p copies the source directory mode exactly,
# including group/other write (the old sanitization is gone).
source, dest, _ = self._tree("dirmode_go_write", 0o777, pin_mtime=False)
result, _ = run_client(source, dest, flags=["-p"], port=shared_server.port)
assert result.returncode == 0, f"-p failed: {(result.stderr or result.stdout)[:300]}"
mode = os.stat(os.path.join(get_dest_received_dir(dest, source), "sub")).st_mode & 0o777
assert mode == 0o777, \
f"-p must preserve the source directory mode exactly, got {oct(mode)}"
@pytest.mark.ci
def test_omit_dir_times_suppresses_times_not_modes(self, shared_server):
source, dest, _ = self._tree("dirmode_omit", 0o750, pin_mtime=True)
result, _ = run_client(source, dest, flags=["-a", "-O"], port=shared_server.port)
assert result.returncode == 0, f"-a -O failed: {(result.stderr or result.stdout)[:300]}"
st = os.stat(os.path.join(get_dest_received_dir(dest, source), "sub"))
assert st.st_mode & 0o777 == 0o750, \
f"-O must suppress only dir times, not dir modes (got {oct(st.st_mode & 0o777)})"
assert abs(st.st_mtime - DISTINCT_MTIME) > 5, \
f"-O must not apply the directory mtime (got {st.st_mtime})"
class TestOwnershipBestEffort:
"""-o/-g/-a must succeed with correct content even when the receiver cannot
chown (the unprivileged CI case). Ownership is deliberately not asserted."""
@pytest.mark.ci
@pytest.mark.parametrize("flags", [["-o"], ["-g"], ["-a"]])
def test_ownership_flags_succeed_unprivileged(self, shared_server, flags):
tag = flags[0].strip("-")
source = os.path.join(TEST_DATA_DIR, f"best_effort_{tag}_src")
dest = os.path.join(TEST_DATA_DIR, f"best_effort_{tag}_dst")
_seed_file(source, dest, "f.txt", b"best effort ownership\n", 0o640)
result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, \
f"{flags} exit {result.returncode}: {(result.stderr or result.stdout)[:300]}"
with open(_received(dest, source, "f.txt"), "rb") as fh:
assert fh.read() == b"best effort ownership\n"
@pytest.mark.skipif(os.geteuid() != 0, reason="only root can change ownership")
class TestOwnershipRoot:
"""Root-only per-attribute ownership application. Not marked ci: the PR
gate runs as an unprivileged user."""
def _seed_owned(self, tag, uid, gid):
source = os.path.join(TEST_DATA_DIR, f"root_owner_{tag}_src")
dest = os.path.join(TEST_DATA_DIR, f"root_owner_{tag}_dst")
path = _seed_file(source, dest, "f.txt", b"root ownership\n", 0o644)
os.chown(path, uid, gid)
return source, dest
def test_o_applies_owner_only(self, shared_server):
source, dest = self._seed_owned("o", 12345, 12346)
result, _ = run_client(source, dest, flags=["-o"], port=shared_server.port)
assert result.returncode == 0, f"-o failed: {(result.stderr or '')[:300]}"
st = os.stat(_received(dest, source, "f.txt"))
assert st.st_uid == 12345, f"-o must apply the owner, got uid={st.st_uid}"
assert st.st_gid != 12346, "-o must not change the group"
def test_g_applies_group_only(self, shared_server):
source, dest = self._seed_owned("g", 12345, 54321)
result, _ = run_client(source, dest, flags=["-g"], port=shared_server.port)
assert result.returncode == 0, f"-g failed: {(result.stderr or '')[:300]}"
st = os.stat(_received(dest, source, "f.txt"))
assert st.st_gid == 54321, f"-g must apply the group, got gid={st.st_gid}"
assert st.st_uid != 12345, "-g must not change the owner"
def test_a_applies_owner_and_group(self, shared_server):
source, dest = self._seed_owned("a", 12345, 54321)
result, _ = run_client(source, dest, flags=["-a"], port=shared_server.port)
assert result.returncode == 0, f"-a failed: {(result.stderr or '')[:300]}"
st = os.stat(_received(dest, source, "f.txt"))
assert (st.st_uid, st.st_gid) == (12345, 54321), \
f"-a must apply owner+group, got uid={st.st_uid} gid={st.st_gid}"
def test_chown_overrides_o(self, shared_server):
source, dest = self._seed_owned("chown", 11111, 22222)
result, _ = run_client(source, dest, flags=["-o", "--chown=@33333:@44444"],
port=shared_server.port)
assert result.returncode == 0, f"-o --chown failed: {(result.stderr or '')[:300]}"
st = os.stat(_received(dest, source, "f.txt"))
assert (st.st_uid, st.st_gid) == (33333, 44444), \
f"--chown must override -o, got uid={st.st_uid} gid={st.st_gid}"
def test_fake_super_o_does_not_real_chown(self, shared_server):
# #294: --fake-super only RECORDS ownership; it must never real-chown the
# recorded source owner (that defeats the point of the flag). With -o the
# resolved owner is parked in the reserved xattr and the on-disk owner is
# left as the receiver's.
source, dest = self._seed_owned("fake_o", 12345, 54321)
result, _ = run_client(source, dest, flags=["--fake-super", "-o"],
port=shared_server.port)
assert result.returncode == 0, f"--fake-super -o failed: {(result.stderr or '')[:300]}"
dst = _received(dest, source, "f.txt")
st = os.stat(dst)
assert st.st_uid != 12345, \
f"--fake-super -o must NOT real-chown the source owner, got uid={st.st_uid}"
record = os.getxattr(dst, "user.fastsync.stat").decode()
fields = record.split(":")
assert fields[0] == "12345", \
f"--fake-super must record the resolved owner, got {fields[0]}"
def test_o_applies_directory_owner(self, shared_server):
"""#286.2: -o must apply the source owner to DIRECTORIES too (the
deferred directory-metadata application now runs the identity path)."""
source = os.path.join(TEST_DATA_DIR, "root_dir_o_src")
dest = os.path.join(TEST_DATA_DIR, "root_dir_o_dst")
clean_dir(source)
clean_dir(dest)
os.makedirs(os.path.join(source, "sub", "deep"))
with open(os.path.join(source, "sub", "deep", "f.txt"), "wb") as fh:
fh.write(b"dir owner\n")
os.chown(os.path.join(source, "sub"), 12345, 12346)
os.chown(os.path.join(source, "sub", "deep"), 23456, 34567)
result, _ = run_client(source, dest, flags=["-o", "-t"], port=shared_server.port)
assert result.returncode == 0, f"-o dir failed: {(result.stderr or '')[:300]}"
received = get_dest_received_dir(dest, source)
sub = os.stat(os.path.join(received, "sub"))
deep = os.stat(os.path.join(received, "sub", "deep"))
assert sub.st_uid == 12345, f"dir 'sub' owner not applied: {sub.st_uid}"
assert deep.st_uid == 23456, f"dir 'sub/deep' owner not applied: {deep.st_uid}"
# -o alone must not change the group.
assert sub.st_gid != 12346
def test_a_applies_directory_owner_and_group(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "root_dir_a_src")
dest = os.path.join(TEST_DATA_DIR, "root_dir_a_dst")
clean_dir(source)
clean_dir(dest)
os.makedirs(os.path.join(source, "sub"))
with open(os.path.join(source, "sub", "f.txt"), "wb") as fh:
fh.write(b"dir owner group\n")
os.chown(os.path.join(source, "sub"), 12345, 54321)
result, _ = run_client(source, dest, flags=["-a"], port=shared_server.port)
assert result.returncode == 0, f"-a dir failed: {(result.stderr or '')[:300]}"
received = get_dest_received_dir(dest, source)
st = os.stat(os.path.join(received, "sub"))
assert (st.st_uid, st.st_gid) == (12345, 54321), \
f"-a must apply dir owner+group, got uid={st.st_uid} gid={st.st_gid}"
def test_numeric_ids_alone_does_not_chown(self, shared_server):
"""#286.1: --numeric-ids is a mapping modifier, not an ownership request.
`-t --numeric-ids` must leave the receiver's ownership untouched."""
source, dest = self._seed_owned("num_only", 12345, 54321)
result, _ = run_client(source, dest, flags=["-t", "--numeric-ids"],
port=shared_server.port)
assert result.returncode == 0, \
f"-t --numeric-ids failed: {(result.stderr or '')[:300]}"
st = os.stat(_received(dest, source, "f.txt"))
assert st.st_uid != 12345, \
f"--numeric-ids alone must not chown, got uid={st.st_uid}"
def test_numeric_ids_with_o_uses_raw_id(self, shared_server):
source, dest = self._seed_owned("num_o", 12345, 54321)
result, _ = run_client(source, dest, flags=["-o", "-t", "--numeric-ids"],
port=shared_server.port)
assert result.returncode == 0, \
f"-o --numeric-ids failed: {(result.stderr or '')[:300]}"
st = os.stat(_received(dest, source, "f.txt"))
assert st.st_uid == 12345, \
f"-o --numeric-ids must apply the raw id, got uid={st.st_uid}"
class TestPreserveFeatureMatrix:
"""A representative per-attribute check under the alternate transfer engines
(chunk serialization, --delay-updates, and the multithreaded scanner)."""
@pytest.mark.ci
@pytest.mark.parametrize("extra", ["--chunk-serialization", "--delay-updates", "--threads"])
def test_p_and_t_hold_under_engine(self, shared_server, extra):
tag = extra.strip("-").replace("-", "_")
source = os.path.join(TEST_DATA_DIR, f"matrix_{tag}_src")
dest = os.path.join(TEST_DATA_DIR, f"matrix_{tag}_dst")
_seed_file(source, dest, "f.txt", b"matrix\n", 0o750, mtime=DISTINCT_MTIME)
result, _ = run_client(source, dest, flags=["-p", "-t", extra],
port=shared_server.port)
assert result.returncode == 0, \
f"-p -t {extra} failed: {(result.stderr or result.stdout)[:300]}"
st = os.stat(_received(dest, source, "f.txt"))
assert st.st_mode & 0o777 == 0o750, f"mode lost under {extra}"
assert abs(st.st_mtime - DISTINCT_MTIME) < 2, f"mtime lost under {extra}"
class TestSpecialNodeModes:
"""Strict rsync parity: with -p the source FIFO mode is copied exactly,
including group/other write. Without -p the node follows the same
source & ~umask base as any other new entry. FIFOs are created
unprivileged via mkfifo."""
@pytest.mark.ci
def test_specials_p_preserves_fifo_mode(self):
source = os.path.join(TEST_DATA_DIR, "specialmode_src")
dest = os.path.join(TEST_DATA_DIR, "specialmode_dst")
clean_dir(source)
clean_dir(dest)
src_fifo = os.path.join(source, "world.fifo")
os.mkfifo(src_fifo)
os.chmod(src_fifo, 0o777)
assert os.stat(src_fifo).st_mode & 0o777 == 0o777
# Production daemonizes with umask(0) (server.c) so the source mode is
# what reaches mkfifo. The session server runs in the foreground and
# would inherit the runner's umask, which alone would strip the write
# bits and mask a regression. Start a dedicated foreground server under
# umask(0) to exercise the real path.
server = ServerManager()
saved_umask = os.umask(0)
try:
server.start(extra_args=["--allow-super"])
finally:
os.umask(saved_umask)
try:
result, _ = run_client(source, dest, flags=["--specials", "-p"],
port=server.port)
finally:
server.stop()
assert result.returncode == 0, \
f"--specials -p failed: {(result.stderr or result.stdout)[:300]}"
received = _received(dest, source, "world.fifo")
assert os.path.lexists(received), "source FIFO was not recreated on the destination"
st = os.lstat(received)
assert stat.S_ISFIFO(st.st_mode), f"received entry is not a FIFO: {oct(st.st_mode)}"
mode = st.st_mode & 0o777
assert mode == 0o777, \
f"-p must preserve the source FIFO mode exactly (want 0o777), got {oct(mode)}"
@@ -0,0 +1,262 @@
"""Guard the README against drifting from the real CLI.
This test parses README.md and checks it against the actual sources of truth
instead of against a hand-maintained copy:
* client ``--help`` output -> ``src/client/usage.c`` (``print_usage``)
* server ``--help`` output -> ``src/server/server.c`` (``print_server_usage``)
* ``FASTSYNC_*`` env vars -> ``getenv("...")`` call sites under ``src/``
It is deliberately offline and read-only: no server is started, no transfer is
performed. Each binary is invoked at most once per test session and the result
is cached.
"""
import functools
import os
import re
import subprocess
import sys
import pytest
sys.path.insert(0, os.path.dirname(__file__))
from common import BUILD_DIR, PROJECT_ROOT
pytestmark = pytest.mark.ci
README_PATH = os.path.join(PROJECT_ROOT, "README.md")
SRC_DIR = os.path.join(PROJECT_ROOT, "src")
# ---------------------------------------------------------------------------
# Markdown helpers
# ---------------------------------------------------------------------------
_HEADING_RE = re.compile(r"^(#+)\s+(.*?)\s*$")
_BACKTICK_RE = re.compile(r"`([^`]*)`")
# A documented option may carry an argument annotation that is not part of the
# option name itself: ``--out=FILE``, ``--exclude <pattern>``, ``--threads[=N]``,
# ``--copy-as=USER[:GROUP]``. Cut the name loose from the first such marker.
_OPTION_SUFFIX_RE = re.compile(r"[=\s<\[(].*$")
_OPTION_TOKEN_RE = re.compile(r"^--?[A-Za-z][A-Za-z0-9-]*$")
def _readme_lines():
with open(README_PATH, encoding="utf-8") as fh:
return fh.read().splitlines()
def _heading_level(line):
match = _HEADING_RE.match(line)
return len(match.group(1)) if match else 0
def _section(lines, heading):
"""Return ``(lineno, line)`` pairs under the first exact ``heading``.
The section runs until the next heading of the same or higher level, so a
``##`` section includes its ``###`` subsections. Line numbers are 1-based
to match what a reader sees in an editor.
"""
target_level = _heading_level(heading)
for index, line in enumerate(lines):
if line.strip() != heading:
continue
start = index + 1
for end in range(start, len(lines)):
level = _heading_level(lines[end])
if level and level <= target_level:
return [(n + 1, lines[n]) for n in range(start, end)]
return [(n + 1, lines[n]) for n in range(start, len(lines))]
raise AssertionError(
f"README heading not found (has the README been restructured?): {heading!r}"
)
def _first_column_spans(section_lines):
"""Backticked spans from the first column of every markdown table row."""
spans = []
for lineno, line in section_lines:
stripped = line.strip()
if not stripped.startswith("|"):
continue
cells = stripped.split("|")
if len(cells) < 2:
continue
first = cells[1]
if set(first.strip()) <= set("-: "):
continue # header separator row, e.g. |---|---|
for match in _BACKTICK_RE.finditer(first):
spans.append((match.group(1), lineno))
return spans
def _documented_option_tokens(section_lines):
"""``(token, lineno, raw_cell)`` for each CLI option in a section's tables."""
found = []
for raw, lineno in _first_column_spans(section_lines):
for piece in re.split(r"[,\s]+", raw):
name = _OPTION_SUFFIX_RE.sub("", piece).strip()
if _OPTION_TOKEN_RE.match(name):
found.append((name, lineno, raw))
return found
# ---------------------------------------------------------------------------
# Sources of truth
# ---------------------------------------------------------------------------
_GETENV_RE = re.compile(r'getenv\s*\(\s*"([^"]+)"\s*\)')
_FASTSYNC_ENV_RE = re.compile(r"FASTSYNC_[A-Z0-9_]+")
def _getenv_names():
"""Every string literal passed to ``getenv()`` anywhere under ``src/``."""
names = set()
for root, _dirs, files in os.walk(SRC_DIR):
for filename in files:
if not filename.endswith((".c", ".h")):
continue
path = os.path.join(root, filename)
with open(path, encoding="utf-8", errors="replace") as fh:
names.update(_GETENV_RE.findall(fh.read()))
return names
@functools.lru_cache(maxsize=None)
def _help_stdout(binary_name):
"""Cached ``<binary> --help`` stdout; skipped (not failed) if unbuilt."""
binary = os.path.join(BUILD_DIR, binary_name)
if not (os.path.isfile(binary) and os.access(binary, os.X_OK)):
pytest.skip(
f"{binary} is not built; run "
"`cmake -B build -S . && cmake --build build` first"
)
try:
result = subprocess.run(
[binary, "--help"], capture_output=True, text=True, timeout=30
)
except OSError as exc:
pytest.skip(f"could not execute {binary}: {exc}")
assert result.returncode == 0, (
f"{binary} --help exited {result.returncode}: "
f"{(result.stderr or result.stdout).strip()[:200]}"
)
return result.stdout
def _mentions_option(help_text, token):
"""True when ``token`` appears as a standalone option in ``help_text``.
A plain substring test would let a removed token hide behind a longer one
(``--del`` inside ``--delete``); requiring a non-word boundary on both sides
keeps every documented token individually accountable.
"""
return (
re.search(r"(?<![\w-])" + re.escape(token) + r"(?![\w-])", help_text)
is not None
)
# Options the help text intentionally expresses only as a family (for example
# the generic ``--no-OPTION`` entry) rather than by spelling every member out.
# Add an entry here only with a one-line justification; prefer fixing the
# extractor first. Currently empty: every option the README documents is
# printed verbatim by the matching ``--help`` (including ``--no-super`` and
# ``--no-detach``).
_FAMILY_FORM_ALLOWLIST = frozenset()
# ---------------------------------------------------------------------------
# Tests
# ---------------------------------------------------------------------------
def test_documented_env_vars_exist():
"""Every ``FASTSYNC_*`` in the README env table has a ``getenv()`` site.
Both directions are checked so the documented set and the source set stay
identical: a documented variable with no call site is a README defect, and a
new ``FASTSYNC_*`` call site without documentation is a README gap.
"""
documented = _documented_env_vars()
assert documented, "no FASTSYNC_* variables found in the README env table"
getenv_names = _getenv_names()
documented_names = {name for name, _lineno in documented}
missing_in_source = [
(name, lineno) for name, lineno in documented if name not in getenv_names
]
if missing_in_source:
details = "; ".join(
f"`{name}` (README.md line {lineno})"
for name, lineno in sorted(missing_in_source, key=lambda item: item[1])
)
pytest.fail(
"README documents environment variable(s) with no getenv() call "
f"site under src/: {details}"
)
fastsync_getenv = {name for name in getenv_names if name.startswith("FASTSYNC_")}
undocumented = sorted(fastsync_getenv - documented_names)
assert not undocumented, (
"src/ reads FASTSYNC_* environment variable(s) that the README does not "
f"document in '## Environment Variables': {', '.join(undocumented)}"
)
def test_documented_client_flags_exist_in_help():
"""Client options in README tables must appear in ``client --help``."""
_assert_documented_flags(
"client",
["### Client", "## Client Options", "## FastSync Extensions"],
)
def test_documented_server_flags_exist_in_help():
"""Server options in README tables must appear in ``server --help``."""
_assert_documented_flags("server", ["### Server", "## Server Options"])
# ---------------------------------------------------------------------------
# Implementation helpers for the tests above
# ---------------------------------------------------------------------------
def _documented_env_vars():
"""``(name, lineno)`` for each ``FASTSYNC_*`` token in the env table."""
lines = _readme_lines()
section = _section(lines, "## Environment Variables")
found = []
for raw, lineno in _first_column_spans(section):
for match in _FASTSYNC_ENV_RE.finditer(raw):
found.append((match.group(0), lineno))
return found
def _assert_documented_flags(binary_name, headings):
help_text = _help_stdout(binary_name)
readme_lines = _readme_lines()
failures = []
for heading in headings:
for token, lineno, raw in _documented_option_tokens(
_section(readme_lines, heading)
):
if token in _FAMILY_FORM_ALLOWLIST:
continue
if not _mentions_option(help_text, token):
failures.append((lineno, token, heading, raw))
if failures:
failures.sort()
shown = "\n".join(
f" {token} (README.md line {lineno}, section {heading!r}, "
f"table cell `{raw}`)"
for lineno, token, heading, raw in failures
)
pytest.fail(
f"README documents option(s) missing from `{binary_name} --help`:\n"
f"{shown}"
)
+25 -2
View File
@@ -150,13 +150,36 @@ class TestStopAt:
assert _received_files(received) == [], \
f"expected nothing transferred, got {_received_files(received)}"
@pytest.mark.ci
def test_stop_at_rsync_date_form(self, shared_server):
"""rsync's full date form (Y-M-DTh:m) is accepted; a deadline well in the
future lets the transfer complete normally."""
source, dest = _make("dateform")
_seed_source(source)
stamp = time.strftime("%Y-%m-%dT%H:%M", time.localtime(time.time() + 3600))
result, _ = run_client(source, dest, flags=[f"--stop-at={stamp}"],
port=shared_server.port)
assert result.returncode == 0, \
f"--stop-at={stamp} should be accepted: " \
f"{(result.stderr or result.stdout)[:400]}"
received = get_dest_received_dir(dest, source)
mismatches, missing = verify_transfer(source, received)
assert not mismatches and not missing
# The slash-separated date spelling is accepted too.
slash = time.strftime("%Y/%m/%dT%H:%M", time.localtime(time.time() + 3600))
result, _ = run_client(source, dest, flags=[f"--stop-at={slash}"],
port=shared_server.port)
assert result.returncode == 0, f"--stop-at={slash} should be accepted"
@pytest.mark.ci
def test_stop_rejects_garbage(self, shared_server):
"""Malformed --stop-at/--stop-after values are rejected up front."""
source, dest = _make("garbage")
_seed_source(source)
for flag in ("--stop-after=abc", "--stop-at=12:99", "--stop-at=12",
"--stop-at=now+5x", "--stop-at=now-5s"):
for flag in ("--stop-after=abc", "--stop-at=12:99", "--stop-at=1234",
"--stop-at=now+5x", "--stop-at=now-5s",
"--stop-at=2000-13-45", "--stop-at=2030-12-31T23:59:59"):
result, _ = run_client(source, dest, flags=[flag],
port=shared_server.port)
assert result.returncode != 0, f"{flag} should be rejected"
+2
View File
@@ -15,6 +15,7 @@
#include "test_file.h"
#include "test_file_list.h"
#include "test_file_sendfile.h"
#include "test_format.h"
#include "test_fuzz_smoke.h"
#include "test_glob.h"
#include "test_hardlink.h"
@@ -58,6 +59,7 @@ int main() {
RUN_TEST(test_chunk);
RUN_TEST(test_batch);
RUN_TEST(test_change_list);
RUN_TEST(test_format);
RUN_TEST(test_config);
RUN_TEST(test_credentials);
RUN_TEST(test_compression);
+106 -16
View File
@@ -1,5 +1,6 @@
#include "test_change_list.h"
#include "change_list.h"
#include "config.h"
#include "test_utils.h"
#include "utils.h"
#include <stdlib.h>
@@ -9,64 +10,150 @@
static ChangeEvent sample_event(void) {
ChangeEvent event;
memset(&event, 0, sizeof(event));
event.path = "/srv/root/sub/file.txt";
event.path = "src/sub/file.txt";
event.name = "sub/file.txt";
event.decision = CHANGE_SENT;
event.is_directory = false;
event.size = 12345;
event.bytes_sent = 999;
event.mtime_sec = 1700000000;
event.mtime_nsec = 0;
event.mode = 0100644;
event.uid = 1000;
event.gid = 1000;
return event;
}
/* Expected %M expansion computed independently with localtime_r. */
static void expected_mtime(time_t when, char out[32]) {
struct tm broken_down;
localtime_r(&when, &broken_down);
strftime(out, 32, "%Y/%m/%d-%H:%M:%S", &broken_down);
}
static void test_format_tokens() {
ChangeEvent event = sample_event();
char* line = change_render_format("%f %n %l %b %M %%", &event);
Config* config = config_create();
char when[32];
expected_mtime(event.mtime_sec, when);
char* line = change_render_format("%f %n %l %b %M %%", config, &event);
EXPECT_NOT_NULL(line);
EXPECT_EQ_STR(line, "/srv/root/sub/file.txt file.txt 12345 999 1700000000 %");
char expected[256];
snprintf(expected, sizeof(expected), "src/sub/file.txt sub/file.txt 12345 999 %s %%", when);
EXPECT_EQ_STR(line, expected);
free(line);
config_delete(config);
}
static void test_format_unknown_tokens_preserved() {
ChangeEvent event = sample_event();
char* line = change_render_format("x%q=%f%z", &event);
Config* config = config_create();
char* line = change_render_format("x%q=%f%z", config, &event);
EXPECT_NOT_NULL(line);
EXPECT_EQ_STR(line, "x%q=/srv/root/sub/file.txt%z");
EXPECT_EQ_STR(line, "x%q=src/sub/file.txt%z");
free(line);
config_delete(config);
}
static void test_format_leaf_name() {
static void test_format_directory_name_has_trailing_slash() {
ChangeEvent event = sample_event();
event.path = "bare.txt";
char* line = change_render_format("%n|%f", &event);
event.is_directory = true;
event.path = "src/sub";
event.name = "sub";
Config* config = config_create();
char* line = change_render_format("%n|%f", config, &event);
EXPECT_NOT_NULL(line);
EXPECT_EQ_STR(line, "bare.txt|bare.txt");
EXPECT_EQ_STR(line, "sub/|src/sub");
free(line);
config_delete(config);
}
static void test_render_itemize_sent_file() {
ChangeEvent event = sample_event();
char* line = change_render_itemize(&event);
Config* config = config_create();
char* line = change_render_itemize(config, &event);
EXPECT_NOT_NULL(line);
EXPECT_EQ_STR(line, ">f+++++++++ /srv/root/sub/file.txt");
EXPECT_EQ_STR(line, ">f+++++++++ sub/file.txt");
free(line);
config_delete(config);
}
static void test_render_itemize_directory() {
ChangeEvent event = sample_event();
event.is_directory = true;
event.path = "src/sub";
event.name = "sub";
Config* config = config_create();
char* line = change_render_itemize(config, &event);
EXPECT_NOT_NULL(line);
EXPECT_EQ_STR(line, "cd+++++++++ sub/");
free(line);
config_delete(config);
}
static void test_render_itemize_symlink() {
ChangeEvent event = sample_event();
event.is_symlink = true;
event.path = "src/link";
event.name = "link";
event.symlink_target = "a.txt";
Config* config = config_create();
char* line = change_render_itemize(config, &event);
EXPECT_NOT_NULL(line);
EXPECT_EQ_STR(line, "cL+++++++++ link -> a.txt");
free(line);
config_delete(config);
}
static void test_render_itemize_compares_destination() {
ChangeEvent event = sample_event();
Config* config = config_create();
config->preserve_perms = true;
config->preserve_owner = true;
config->preserve_group = true;
event.dest.known = true;
event.dest.existed = true;
event.dest.size = 1;
event.dest.mtime_sec = 1700000000;
event.dest.mtime_nsec = 0;
event.dest.mode = 0100600;
event.dest.uid = 1;
event.dest.gid = 2;
char* line = change_render_itemize(config, &event);
EXPECT_NOT_NULL(line);
/* size, perms, owner and group differ; time matches. */
EXPECT_EQ_STR(line, ">f.s.pog... sub/file.txt");
free(line);
config_delete(config);
}
static void test_render_itemize_up_to_date_is_empty() {
ChangeEvent event = sample_event();
Config* config = config_create();
event.decision = CHANGE_UP_TO_DATE;
char* line = change_render_itemize(&event);
char* line = change_render_itemize(config, &event);
EXPECT_NOT_NULL(line);
EXPECT_EQ_STR(line, "");
free(line);
config_delete(config);
}
static void test_render_list_line() {
char* line = change_render_list_line(0100644, 4096, 1700000000, "/srv/x.txt");
ChangeEvent event;
memset(&event, 0, sizeof(event));
Config* config = config_create();
event.name = "sub/x.txt";
event.path = "sub/x.txt";
event.mode = 0100644;
event.size = 4096;
event.mtime_sec = 1700000000;
char* line = change_render_list_line(config, &event);
EXPECT_NOT_NULL(line);
EXPECT_TRUE(strncmp(line, "-rw-r--r--", 10) == 0);
EXPECT_TRUE(strstr(line, "4096") != NULL);
EXPECT_TRUE(strstr(line, "/srv/x.txt") != NULL);
EXPECT_TRUE(strstr(line, "4,096") != NULL);
EXPECT_TRUE(strstr(line, "sub/x.txt") != NULL);
free(line);
config_delete(config);
}
static void test_change_list_enabled() {
@@ -93,8 +180,11 @@ static void test_change_list_enabled() {
void test_change_list() {
test_format_tokens();
test_format_unknown_tokens_preserved();
test_format_leaf_name();
test_format_directory_name_has_trailing_slash();
test_render_itemize_sent_file();
test_render_itemize_directory();
test_render_itemize_symlink();
test_render_itemize_compares_destination();
test_render_itemize_up_to_date_is_empty();
test_render_list_line();
test_change_list_enabled();
+31 -1
View File
@@ -98,18 +98,47 @@ static void test_checksum_algo_name_mapping() {
EXPECT_EQ_INT(checksum_algo_from_name("XXHASH"), (int)CHECKSUM_ALGO_XXH64);
EXPECT_EQ_INT(checksum_algo_from_name("md5"), (int)CHECKSUM_ALGO_MD5);
EXPECT_EQ_INT(checksum_algo_from_name("MD5"), (int)CHECKSUM_ALGO_MD5);
EXPECT_EQ_INT(checksum_algo_from_name("xxh3"), (int)CHECKSUM_ALGO_XXH3);
EXPECT_EQ_INT(checksum_algo_from_name("XXH3"), (int)CHECKSUM_ALGO_XXH3);
EXPECT_EQ_INT(checksum_algo_from_name("xxh128"), (int)CHECKSUM_ALGO_XXH128);
EXPECT_EQ_INT(checksum_algo_from_name("XXH128"), (int)CHECKSUM_ALGO_XXH128);
/* rsync choices FastSync does not implement are rejected by name. */
EXPECT_TRUE(checksum_algo_from_name("md4") < 0);
EXPECT_TRUE(checksum_algo_from_name("sha1") < 0);
EXPECT_TRUE(checksum_algo_from_name("sha256") < 0);
EXPECT_TRUE(checksum_algo_from_name("crc32") < 0);
EXPECT_TRUE(checksum_algo_from_name("none") < 0);
EXPECT_TRUE(checksum_algo_from_name("xxh3") < 0);
EXPECT_TRUE(checksum_algo_from_name("") < 0);
EXPECT_TRUE(checksum_algo_from_name(NULL) < 0);
EXPECT_TRUE(checksum_algo_valid((int)CHECKSUM_ALGO_XXH64));
EXPECT_TRUE(checksum_algo_valid((int)CHECKSUM_ALGO_MD5));
EXPECT_TRUE(checksum_algo_valid((int)CHECKSUM_ALGO_XXH3));
EXPECT_TRUE(checksum_algo_valid((int)CHECKSUM_ALGO_XXH128));
EXPECT_FALSE(checksum_algo_valid(99));
EXPECT_EQ_STR(checksum_algo_name(CHECKSUM_ALGO_XXH64), "xxh64");
EXPECT_EQ_STR(checksum_algo_name(CHECKSUM_ALGO_MD5), "md5");
EXPECT_EQ_STR(checksum_algo_name(CHECKSUM_ALGO_XXH3), "xxh3");
EXPECT_EQ_STR(checksum_algo_name(CHECKSUM_ALGO_XXH128), "xxh128");
}
/* xxh3 is 8 bytes and seed-aware; xxh128 is 16 bytes and differs from both
* xxh64 and md5 for the same input. */
static void test_checksum_xxh3_xxh128() {
EXPECT_EQ_INT((int)checksum_digest_len(CHECKSUM_ALGO_XXH3), 8);
EXPECT_EQ_INT((int)checksum_digest_len(CHECKSUM_ALGO_XXH128), 16);
uint8_t a[CHECKSUM_MAX_DIGEST_LEN], b[CHECKSUM_MAX_DIGEST_LEN];
size_t alen = 0, blen = 0;
EXPECT_TRUE(checksum_digest(CHECKSUM_ALGO_XXH3, 0, "payload", 7, a, sizeof(a), &alen));
EXPECT_TRUE(alen == (size_t)8);
EXPECT_TRUE(checksum_digest(CHECKSUM_ALGO_XXH3, 5, "payload", 7, b, sizeof(b), &blen));
EXPECT_TRUE(memcmp(a, b, alen) != 0);
EXPECT_TRUE(checksum_digest(CHECKSUM_ALGO_XXH128, 0, "payload", 7, a, sizeof(a), &alen));
EXPECT_TRUE(alen == (size_t)16);
EXPECT_TRUE(checksum_digest(CHECKSUM_ALGO_XXH128, 0, "payload", 7, b, sizeof(b), &blen));
EXPECT_TRUE(memcmp(a, b, blen) == 0);
}
static void test_checksum_truncated_buffer_rejected() {
@@ -142,6 +171,7 @@ void test_checksum(void) {
test_checksum_algo_lengths_distinct();
test_checksum_md5_seed_ignored();
test_checksum_algo_name_mapping();
test_checksum_xxh3_xxh128();
test_checksum_truncated_buffer_rejected();
test_checksum_null_empty_digest();
}
+910 -18
View File
File diff suppressed because it is too large. Load diff
+15
View File
@@ -64,6 +64,21 @@ static void test_skip_compress_suffix_matching() {
EXPECT_TRUE(compression_should_skip_with_suffixes("backup.TAR.GZ", suffixes, 2));
EXPECT_FALSE(compression_should_skip_with_suffixes("notes.txt", suffixes, 2));
EXPECT_FALSE(compression_should_skip_with_suffixes("archive.zip", suffixes, 0));
/* A user suffix may omit the leading dot (rsync's spelling). */
char* bare[] = {"zip", "gz"};
EXPECT_TRUE(compression_should_skip_with_suffixes("archive.zip", bare, 2));
EXPECT_TRUE(compression_should_skip_with_suffixes("x.GZ", bare, 2));
/* No user list (count < 0) selects rsync 3.4.1's built-in default list. */
EXPECT_TRUE(compression_should_skip_with_suffixes("movie.mp4", NULL, -1));
EXPECT_TRUE(compression_should_skip_with_suffixes("archive.TAR.GZ", NULL, -1));
EXPECT_TRUE(compression_should_skip_with_suffixes("photo.jpeg", NULL, -1));
EXPECT_TRUE(compression_should_skip_with_suffixes("disk.squashfs", NULL, -1));
EXPECT_TRUE(compression_should_skip_with_suffixes("data.7z", NULL, -1));
EXPECT_FALSE(compression_should_skip_with_suffixes("notes.txt", NULL, -1));
EXPECT_FALSE(compression_should_skip_with_suffixes("program", NULL, -1));
EXPECT_FALSE(compression_should_skip_with_suffixes("trailing.", NULL, -1));
}
static void test_data_compress_with_threads_roundtrip() {
+323 -17
View File
@@ -9,6 +9,7 @@
#include "test_utils.h"
#include "utils.h"
#include <signal.h>
#include <stddef.h>
#include <stdlib.h>
#include <sys/socket.h>
#include <string.h>
@@ -551,6 +552,83 @@ static void test_config_send_receive() {
}
}
/* #5: a received --max-alloc=0 (rsync's "no limit") is floored to the server
* ceiling on the receive path, so a client cannot disable it. */
static void test_config_receive_max_alloc_zero_floored() {
Config* send_cfg = config_create();
EXPECT_NOT_NULL(send_cfg);
send_cfg->send_directory = str_dup("/send/src");
send_cfg->receive_root_directory = str_dup("/send/dst");
send_cfg->max_alloc = 0;
int p[2];
EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0);
io_set_fds(p[0], p[1]);
io_set_bwlimit(0);
pid_t pid = fork();
if (pid == 0) {
close(p[1]);
io_set_fds(p[0], p[0]);
Config* recv_cfg = config_receive(p[0]);
bool ok = recv_cfg != NULL && recv_cfg->max_alloc == MAX_SERVER_ALLOC;
config_delete(recv_cfg);
close(p[0]);
close(p[1]);
_exit(ok ? 0 : 1);
} else {
close(p[0]);
io_set_fds(p[1], p[1]);
bool sent = config_send(p[1], send_cfg);
int status;
waitpid(pid, &status, 0);
close(p[0]);
close(p[1]);
config_delete(send_cfg);
EXPECT_TRUE(sent);
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
}
}
/* #4: a hostile/older client that still sends compress_choice=auto must be
* accepted (as zstd) rather than failing the whole transfer. */
static void test_config_receive_compress_choice_auto_canonicalized() {
Config* send_cfg = config_create();
EXPECT_NOT_NULL(send_cfg);
send_cfg->send_directory = str_dup("/send/src");
send_cfg->receive_root_directory = str_dup("/send/dst");
free(send_cfg->compress_choice);
send_cfg->compress_choice = str_dup("auto");
int p[2];
EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0);
io_set_fds(p[0], p[1]);
io_set_bwlimit(0);
pid_t pid = fork();
if (pid == 0) {
close(p[1]);
io_set_fds(p[0], p[0]);
Config* recv_cfg = config_receive(p[0]);
bool ok = recv_cfg != NULL && strcmp(recv_cfg->compress_choice, "zstd") == 0;
config_delete(recv_cfg);
close(p[0]);
close(p[1]);
_exit(ok ? 0 : 1);
} else {
close(p[0]);
io_set_fds(p[1], p[1]);
bool sent = config_send(p[1], send_cfg);
int status;
waitpid(pid, &status, 0);
close(p[0]);
close(p[1]);
config_delete(send_cfg);
EXPECT_TRUE(sent);
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
}
}
static void test_config_send_receive_version_mismatch() {
/* A peer using the previous wire format must be rejected. */
Config* cfg = config_create();
@@ -1292,6 +1370,9 @@ static void test_config_metadata_times_wire_roundtrip() {
send_cfg->preserve_crtimes = true;
send_cfg->omit_dir_times = true;
send_cfg->omit_link_times = true;
/* The preservation attributes now require the metadata frame to travel
* (config_invariants_error rejects them otherwise). */
send_cfg->use_metadata = true;
/* --open-noatime is client-only and must NOT cross the wire. */
send_cfg->open_noatime = true;
@@ -1341,13 +1422,19 @@ static void test_config_identity_wire_roundtrip() {
send_cfg->usermap_count = 2;
send_cfg->usermap = calloc(2, sizeof(IdentityMap));
send_cfg->usermap[0].from = IDENTITY_MATCH_ANY;
send_cfg->usermap[0].from_hi = IDENTITY_MATCH_ANY;
send_cfg->usermap[0].to = 65534;
send_cfg->usermap[0].to_name = NULL;
send_cfg->usermap[1].from = 1000;
send_cfg->usermap[1].from_hi = 1000;
send_cfg->usermap[1].to = 1000;
send_cfg->usermap[1].to_name = NULL;
send_cfg->groupmap_count = 1;
send_cfg->groupmap = calloc(1, sizeof(IdentityMap));
send_cfg->groupmap[0].from = 0;
send_cfg->groupmap[0].from_hi = 0;
send_cfg->groupmap[0].to = IDENTITY_CURRENT;
send_cfg->groupmap[0].to_name = str_dup("root");
int p[2];
EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0);
@@ -1363,9 +1450,12 @@ static void test_config_identity_wire_roundtrip() {
ok = recv->numeric_ids && recv->chown_uid_set && recv->chown_uid == 1001 &&
recv->chown_gid_set && recv->chown_gid == IDENTITY_CURRENT && recv->usermap_count == 2 &&
recv->groupmap_count == 1 && recv->usermap[0].from == IDENTITY_MATCH_ANY &&
recv->usermap[0].to == 65534 && recv->usermap[1].from == 1000 &&
recv->usermap[1].to == 1000 && recv->groupmap[0].from == 0 &&
recv->groupmap[0].to == IDENTITY_CURRENT;
recv->usermap[0].from_hi == IDENTITY_MATCH_ANY && recv->usermap[0].to == 65534 &&
recv->usermap[0].to_name == NULL && recv->usermap[1].from == 1000 &&
recv->usermap[1].from_hi == 1000 && recv->usermap[1].to == 1000 &&
recv->groupmap[0].from == 0 && recv->groupmap[0].from_hi == 0 &&
recv->groupmap[0].to == IDENTITY_CURRENT && recv->groupmap[0].to_name != NULL &&
strcmp(recv->groupmap[0].to_name, "root") == 0;
}
config_delete(recv);
close(p[0]);
@@ -1395,7 +1485,8 @@ static void test_config_receive_rejects_invalid_identity() {
c->receive_root_directory = str_dup("/dst");
c->usermap_count = 1;
c->usermap = calloc(1, sizeof(IdentityMap));
c->usermap[0].from = -2; /* below IDENTITY_MATCH_ANY */
c->usermap[0].from = -3; /* below IDENTITY_MATCH_UNNAMED */
c->usermap[0].from_hi = -3;
c->usermap[0].to = 0;
EXPECT_FALSE(roundtrip_config_ok(c));
config_delete(c);
@@ -2056,9 +2147,51 @@ static void test_identity_ownership_requested() {
config_delete(gm);
}
/* The narrow client-CHOSEN ownership predicate the daemon module gate refuses:
* a preserve-source -o/-g (or -a) must NOT be in it (it is handled by forcing
* super-user activity off instead), while every explicit identity flag is. */
static void test_identity_explicit_ownership_requested() {
EXPECT_FALSE(identity_explicit_ownership_requested(NULL));
Config* c = config_create();
EXPECT_NOT_NULL(c);
EXPECT_FALSE(identity_explicit_ownership_requested(c));
c->preserve_owner = true;
EXPECT_FALSE(identity_explicit_ownership_requested(c));
EXPECT_TRUE(identity_ownership_requested(c)); /* general awareness does see -o */
c->preserve_group = true;
EXPECT_FALSE(identity_explicit_ownership_requested(c));
c->preserve_owner = false;
c->preserve_group = false;
c->numeric_ids = true;
EXPECT_TRUE(identity_explicit_ownership_requested(c));
c->numeric_ids = false;
c->chown_uid_set = true;
EXPECT_TRUE(identity_explicit_ownership_requested(c));
c->chown_uid_set = false;
c->chown_gid_set = true;
EXPECT_TRUE(identity_explicit_ownership_requested(c));
c->chown_gid_set = false;
c->copy_as_set = true;
EXPECT_TRUE(identity_explicit_ownership_requested(c));
c->copy_as_set = false;
c->fake_super = true;
EXPECT_TRUE(identity_explicit_ownership_requested(c));
c->fake_super = false;
c->super_mode = SUPER_MODE_ON;
EXPECT_TRUE(identity_explicit_ownership_requested(c));
c->super_mode = SUPER_MODE_AUTO;
EXPECT_EQ_INT(identity_parse_map(c, "@1:@2", false), 0);
EXPECT_TRUE(identity_explicit_ownership_requested(c));
config_delete(c);
}
/* P7 Wave E hardening (A3): --super no longer implies raw numeric-id
preservation, so it must never enable ownership application on its own; an
explicit identity flag is required. */
preservation, so it must never enable ownership application on its own.
#286: --numeric-ids is a mapping MODIFIER only and is likewise inert on its
own; a real ownership request (-o/-g or an explicit identity flag) is
required to activate chown. */
static void test_super_does_not_imply_numeric() {
Config* c = config_create();
EXPECT_NOT_NULL(c);
@@ -2068,11 +2201,40 @@ static void test_super_does_not_imply_numeric() {
EXPECT_FALSE(identity_active_enabled());
c->numeric_ids = true;
EXPECT_TRUE(identity_set_active(c));
EXPECT_FALSE(identity_active_enabled()); /* mapping modifier only */
c->preserve_owner = true;
EXPECT_TRUE(identity_set_active(c));
EXPECT_TRUE(identity_active_enabled());
identity_clear_active();
config_delete(c);
}
/* The preserve-source -o/-g requests enable ownership application through the
* active snapshot (identity_active_enabled) even though they are deliberately
* absent from the narrow client-chosen identity_explicit_ownership_requested()
* gate. */
static void test_identity_active_enabled_includes_preserve_attrs() {
Config* c = config_create();
EXPECT_NOT_NULL(c);
c->use_metadata = true;
EXPECT_TRUE(identity_set_active(c));
EXPECT_FALSE(identity_active_enabled());
c->preserve_owner = true;
EXPECT_TRUE(identity_set_active(c));
EXPECT_TRUE(identity_active_enabled());
EXPECT_FALSE(identity_explicit_ownership_requested(c));
c->preserve_owner = false;
c->preserve_group = true;
EXPECT_TRUE(identity_set_active(c));
EXPECT_TRUE(identity_active_enabled());
EXPECT_FALSE(identity_explicit_ownership_requested(c));
identity_clear_active();
config_delete(c);
}
/* The single shared predicate must reject every cross-field combination the
client/server enforce and accept a plain valid config. Because both
validate_config() (client) and validate_received_config() (server) call it,
@@ -2193,6 +2355,95 @@ static void test_config_invariants_error_all_combinations() {
config_delete(c);
}
/* Every per-attribute preservation flag requires the metadata frame to travel:
* the invariant rejects any of them while use_metadata is false, and setting
* use_metadata clears the violation. */
static void test_config_preservation_requires_metadata() {
static const size_t attrs[] = {
offsetof(Config, preserve_perms), offsetof(Config, preserve_times),
offsetof(Config, preserve_owner), offsetof(Config, preserve_group),
offsetof(Config, preserve_atimes), offsetof(Config, preserve_crtimes),
offsetof(Config, use_executability),
};
Config* c = config_create();
EXPECT_NOT_NULL(c);
EXPECT_NULL(config_invariants_error(c));
for (size_t i = 0; i < sizeof(attrs) / sizeof(attrs[0]); i++) {
bool* field = (bool*)((char*)c + attrs[i]);
*field = true;
EXPECT_NOT_NULL(config_invariants_error(c));
c->use_metadata = true;
EXPECT_NULL(config_invariants_error(c));
c->use_metadata = false;
*field = false;
}
config_delete(c);
}
/* config_derived_use_metadata is the single source of truth for the derived
* transport bit: each representative flag turns it on, and it stays off for a
* bare config (numeric_ids alone, omit flags, whole-file, ...). */
static void test_config_derived_use_metadata() {
static const size_t true_flags[] = {
offsetof(Config, preserve_perms), offsetof(Config, preserve_times),
offsetof(Config, preserve_owner), offsetof(Config, preserve_group),
offsetof(Config, preserve_atimes), offsetof(Config, preserve_crtimes),
offsetof(Config, use_executability), offsetof(Config, preserve_xattrs),
offsetof(Config, preserve_acls), offsetof(Config, fake_super),
offsetof(Config, preserve_devices), offsetof(Config, preserve_specials),
offsetof(Config, copy_devices), offsetof(Config, write_devices),
offsetof(Config, copy_as_set), offsetof(Config, chown_uid_set),
offsetof(Config, chown_gid_set), offsetof(Config, update),
};
Config* c = config_create();
EXPECT_NOT_NULL(c);
EXPECT_FALSE(config_derived_use_metadata(c));
EXPECT_FALSE(config_derived_use_metadata(NULL));
for (size_t i = 0; i < sizeof(true_flags) / sizeof(true_flags[0]); i++) {
bool* field = (bool*)((char*)c + true_flags[i]);
*field = true;
EXPECT_TRUE(config_derived_use_metadata(c));
*field = false;
}
/* A non-empty --chmod spec. */
c->chmod_spec = str_dup("u=rw");
EXPECT_TRUE(config_derived_use_metadata(c));
free(c->chmod_spec);
c->chmod_spec = NULL;
/* Identity-map counts. */
c->usermap_count = 1;
EXPECT_TRUE(config_derived_use_metadata(c));
c->usermap_count = 0;
c->groupmap_count = 1;
EXPECT_TRUE(config_derived_use_metadata(c));
c->groupmap_count = 0;
/* Incremental/delta imply metadata unless --no-preserve disabled it. */
c->use_incremental = true;
EXPECT_TRUE(config_derived_use_metadata(c));
c->metadata_explicitly_disabled = true;
EXPECT_FALSE(config_derived_use_metadata(c));
c->metadata_explicitly_disabled = false;
c->use_incremental = false;
c->use_delta = true;
EXPECT_TRUE(config_derived_use_metadata(c));
c->metadata_explicitly_disabled = true;
EXPECT_FALSE(config_derived_use_metadata(c));
c->metadata_explicitly_disabled = false;
c->use_delta = false;
/* Flags that must NOT imply metadata on their own. */
c->numeric_ids = true;
c->omit_dir_times = true;
c->omit_link_times = true;
c->whole_file = true;
c->ignore_times = true;
EXPECT_FALSE(config_derived_use_metadata(c));
config_delete(c);
}
/* The receiver previously missed several of these; a forged frame that sets
the offending serialized fields must now be refused at the config
handshake. (whole_file is client-only, so its rules cannot appear here.) */
@@ -2355,6 +2606,34 @@ static void test_config_wire_roundtrip_all_fields() {
config_delete(populated);
}
/* Each of the four split-out preservation bools must survive a frame
* round-trip on its own. The all-fields golden sets an alternating
* true/false pattern precisely because a run of identical adjacent booleans
* would let a same-KIND field swap produce the same bytes; isolating one true
* bit at a time pins each new field's position and width independently. */
static void test_config_preserve_attribute_wire_roundtrip() {
if (is_running_under_valgrind())
return;
static const size_t attrs[] = {
offsetof(Config, preserve_perms),
offsetof(Config, preserve_times),
offsetof(Config, preserve_owner),
offsetof(Config, preserve_group),
};
for (size_t i = 0; i < sizeof(attrs) / sizeof(attrs[0]); i++) {
Config* c = config_create();
EXPECT_NOT_NULL(c);
c->send_directory = str_dup("/src");
c->receive_root_directory = str_dup("/dst");
/* The preservation invariant requires the metadata frame to travel, so set
* the transport bit; otherwise config_receive() legitimately refuses. */
c->use_metadata = true;
*(bool*)((char*)c + attrs[i]) = true;
EXPECT_TRUE(roundtrip_and_compare(c));
config_delete(c);
}
}
/* Populate every serialized field with a non-default value so the wire frame
* exercises each table entry. Boolean runs deliberately alternate true/false:
* a run of identical booleans would make an adjacent swap (same KIND) produce
@@ -2427,7 +2706,7 @@ static void golden_config_populate(Config* c) {
c->modify_window = 3;
c->compress_choice = str_dup("zstd");
/* "u=rwx,go=rx" is the same 11 bytes as the original "u=rwX,go=rX" (so the
* frame stays 637 bytes) but X is not in FastSync's chmod grammar, and the
* frame stays 653 bytes) but X is not in FastSync's chmod grammar, and the
* receive-side golden validates the frame. */
c->chmod_spec = str_dup("u=rwx,go=rx");
c->skip_compress_set = true;
@@ -2448,17 +2727,29 @@ static void golden_config_populate(Config* c) {
c->usermap_count = 2;
c->usermap = calloc(2, sizeof(IdentityMap));
c->usermap[0].from = IDENTITY_MATCH_ANY;
c->usermap[0].from_hi = IDENTITY_MATCH_ANY;
c->usermap[0].to = 1000;
c->usermap[0].to_name = NULL;
c->usermap[1].from = 5;
c->usermap[1].from_hi = 9;
c->usermap[1].to = 6;
c->usermap[1].to_name = NULL;
c->groupmap_count = 1;
c->groupmap = calloc(1, sizeof(IdentityMap));
c->groupmap[0].from = 7;
c->groupmap[0].from_hi = 7;
c->groupmap[0].to = 8;
c->groupmap[0].to_name = str_dup("root");
c->preserve_atimes = true;
c->preserve_crtimes = false;
c->omit_dir_times = true;
c->omit_link_times = false;
/* Mixed true/false so a field reorder or a dropped attribute changes the
* pinned hash rather than passing silently. */
c->preserve_perms = true;
c->preserve_times = false;
c->preserve_owner = true;
c->preserve_group = false;
c->munge_links = true;
c->keep_dirlinks = false;
c->fake_super = true;
@@ -2472,13 +2763,14 @@ static void golden_config_populate(Config* c) {
c->copy_as_gid = 222;
}
/* The pinned golden frame (protocol 2.21.0). The values below are the only
/* The pinned golden frame (protocol 2.23.0). The values below are the only
* thing that ties the generated table to the historical wire format; update
* them ONLY with a PROTOCOL_VERSION bump and a documented reason. The combined
* 2.21.0 wave appends the serialized dry_run bool to CONFIG_WIRE_CORE_FIELDS
* and keeps the protocol version string at 2.21.0. */
#define GOLDEN_WIRE_LEN 637
#define GOLDEN_WIRE_HASH 13228626061067899189ULL
* them ONLY with a PROTOCOL_VERSION bump and a documented reason. The 2.23.0
* rsync-parity wave changes the config-frame layout (map-entry range + TO name,
* one report_dest_info bool, and other wire changes landing in this version);
* the byte-exact values are recomputed for the merged layout. */
#define GOLDEN_WIRE_LEN 697
#define GOLDEN_WIRE_HASH 7835017034643051109ULL
static unsigned long long fnv1a_64(const unsigned char* buf, size_t len) {
unsigned long long h = 1469598103934665603ULL;
@@ -2560,7 +2852,7 @@ static unsigned long long capture_wire_hash(const Config* cfg, size_t* out_len)
return h;
}
/* Byte-for-byte wire compatibility guard (protocol 2.21.0). The expected hash
/* Byte-for-byte wire compatibility guard (protocol 2.23.0). The expected hash
* pins the pre-X-macro byte stream; the refactor MUST NOT change it. */
static void test_config_wire_golden() {
if (is_running_under_valgrind())
@@ -2613,12 +2905,18 @@ static void test_config_wire_golden_receive() {
!recv->use_multithreading;
ok = ok && recv->compression_level == 7 && recv->chunk_size == 65536;
ok = ok && recv->use_delta && !recv->whole_file && recv->use_xattrs;
/* Per-attribute preservation split decoded from the pinned bytes. */
ok = ok && recv->preserve_perms && !recv->preserve_times && recv->preserve_owner &&
!recv->preserve_group;
/* Bounded/validated KINDs decoded from the pinned bytes. */
ok = ok && recv->checksum_algo == CHECKSUM_ALGO_MD5;
ok = ok && recv->super_mode == SUPER_MODE_ON;
ok = ok && recv->chown_uid == 1234 && recv->chown_gid == 5678;
ok = ok && recv->usermap_count == 2 && recv->usermap[0].from == IDENTITY_MATCH_ANY &&
recv->usermap[0].to == 1000 && recv->usermap[1].from == 5 && recv->usermap[1].to == 6;
recv->usermap[0].from_hi == IDENTITY_MATCH_ANY && recv->usermap[0].to == 1000 &&
recv->usermap[1].from == 5 && recv->usermap[1].from_hi == 9 && recv->usermap[1].to == 6;
ok = ok && recv->groupmap_count == 1 && recv->groupmap[0].from == 7 &&
recv->groupmap[0].to_name != NULL && strcmp(recv->groupmap[0].to_name, "root") == 0;
ok = ok && recv->basis_count == 2 && recv->basis_dirs[0].type == BASIS_DEST_COMPARE &&
recv->basis_dirs[1].type == BASIS_DEST_LINK;
ok = ok && recv->module != NULL && strcmp(recv->module, "goldenmod") == 0;
@@ -2702,13 +3000,14 @@ static void test_config_wire_receive_bounds() {
/* BOOL: only 0/1 is a legal wire value. */
EXPECT_TRUE(receive_hand_built_frame_rejected(write_frame_with_invalid_bool));
/* RAW_MAXALLOC: zero is rejected before it can become the session ceiling. */
/* RAW_MAXALLOC: zero is rsync's --max-alloc=0 "no limit" and round-trips;
* only the over-ceiling clamp is applied server-side. */
Config* c = config_create();
EXPECT_NOT_NULL(c);
c->send_directory = str_dup("/src");
c->receive_root_directory = str_dup("/dst");
c->max_alloc = 0;
EXPECT_TRUE(roundtrip_config_rejected(c));
EXPECT_FALSE(roundtrip_config_rejected(c));
config_delete(c);
/* STR_MODULE: a name outside [A-Za-z0-9._-] is refused. */
@@ -2829,6 +3128,8 @@ void test_config() {
test_pipeline_receiver_lifecycle();
if (!is_running_under_valgrind()) {
test_config_send_receive();
test_config_receive_max_alloc_zero_floored();
test_config_receive_compress_choice_auto_canonicalized();
test_config_local_only_fields_not_serialized();
test_config_send_receive_version_mismatch();
test_config_receive_truncated();
@@ -2867,15 +3168,20 @@ void test_config() {
test_config_receive_rejects_oversized_string_budget();
test_config_receive_with_validate_rejects();
test_config_invariants_error_all_combinations();
test_config_preservation_requires_metadata();
test_config_derived_use_metadata();
test_config_receive_rejects_unified_invariants();
test_config_wire_golden();
test_config_wire_golden_receive();
test_config_wire_receive_bounds();
test_config_receive_rejects_overcap_counts();
test_config_wire_roundtrip_all_fields();
test_config_preserve_attribute_wire_roundtrip();
}
test_identity_copy_as_refused();
test_identity_ownership_requested();
test_identity_explicit_ownership_requested();
test_identity_active_enabled_includes_preserve_attrs();
test_super_does_not_imply_numeric();
test_privilege_super_permitted_modes();
test_config_delete_timing_early_helper();
+402 -42
View File
@@ -28,6 +28,10 @@ static void test_file_create() {
EXPECT_NULL(f->data->data);
EXPECT_EQ_INT((int)f->data->size, 0);
EXPECT_NULL(f->metadata);
/* An unset destination snapshot must read as known == false, never
indeterminate bytes (-i/--out-format without --incremental). */
EXPECT_FALSE(f->dest_state.known);
EXPECT_FALSE(f->dest_state.existed);
file_destroy(f);
}
@@ -326,6 +330,52 @@ static void test_file_save_to_disk_partial_install() {
rmdir(root);
}
/* --temp-dir is a client-controlled wire value that must be confined below the
* receive root: an absolute or `..`-escaping value is rejected (a client must
* never make the receiver write scratch files in an arbitrary directory), while
* a relative one resolves under the root and is used for the atomic install. */
static void test_file_save_to_disk_temp_dir_confined() {
const char* root = "test_temp_confine_tmp";
const char* dest_file = "test_temp_confine_tmp/file.txt";
char outside[PATH_MAX];
snprintf(outside, sizeof(outside), "/tmp/fastsync_temp_outside_%d", (int)getpid());
unlink(dest_file);
rmdir("test_temp_confine_tmp/scratch");
rmdir(root);
mkdir(root, 0755);
mkdir("test_temp_confine_tmp/scratch", 0755);
mkdir(outside, 0755);
File* f = file_create("file.txt");
EXPECT_NOT_NULL(f);
const char* content = "confined temp dir";
f->data->data = malloc(strlen(content));
EXPECT_NOT_NULL(f->data->data);
memcpy(f->data->data, content, strlen(content));
f->data->size = strlen(content);
Config* config = config_create();
EXPECT_NOT_NULL(config);
config->temp_dir = str_dup(outside);
EXPECT_EQ_INT(file_save_to_disk_full(root, f, config), FILE_SAVE_ERROR);
EXPECT_EQ_INT(access(dest_file, F_OK), -1);
free(config->temp_dir);
config->temp_dir = str_dup("../escape");
EXPECT_EQ_INT(file_save_to_disk_full(root, f, config), FILE_SAVE_ERROR);
EXPECT_EQ_INT(access(dest_file, F_OK), -1);
free(config->temp_dir);
config->temp_dir = str_dup("scratch");
EXPECT_EQ_INT(file_save_to_disk_full(root, f, config), FILE_SAVE_WRITTEN);
EXPECT_EQ_INT(access(dest_file, F_OK), 0);
file_destroy(f);
config_delete(config);
unlink(dest_file);
rmdir("test_temp_confine_tmp/scratch");
rmdir(root);
rmdir(outside);
}
/* Issue #251: file_save_to_disk_full must distinguish receiver-side skips
(--existing/--ignore-existing/--update) from real writes so the sender can
decide whether --remove-source-files may unlink its source. */
@@ -410,7 +460,7 @@ static void test_file_write_to_disk_with_fsync() {
const char* path = "test_file_write_to_disk_fsync.txt";
const char* content = "fsync file content";
EXPECT_TRUE(file_to_disk_secure_with_fsync(path, content, strlen(content), false, false, false,
NULL, false, true, NULL));
NULL, (FileAttrPolicy){0}, true, NULL));
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
EXPECT_EQ_INT((int)st.st_size, (int)strlen(content));
@@ -420,8 +470,8 @@ static void test_file_write_to_disk_with_fsync() {
static void test_file_write_to_disk_preallocate_atomic() {
const char* path = "test_file_write_prealloc_atomic.txt";
const char* content = "prealloc atomic content";
EXPECT_TRUE(
file_to_disk_secure(path, content, strlen(content), false, false, true, NULL, false, NULL));
EXPECT_TRUE(file_to_disk_secure(path, content, strlen(content), false, false, true, NULL,
(FileAttrPolicy){0}, NULL));
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
EXPECT_EQ_INT((int)st.st_size, (int)strlen(content));
@@ -438,8 +488,8 @@ static void test_file_write_to_disk_preallocate_atomic() {
static void test_file_write_to_disk_preallocate_inplace() {
const char* path = "test_file_write_prealloc_inplace.txt";
const char* content = "prealloc inplace content";
EXPECT_TRUE(
file_to_disk_secure(path, content, strlen(content), true, false, true, NULL, false, NULL));
EXPECT_TRUE(file_to_disk_secure(path, content, strlen(content), true, false, true, NULL,
(FileAttrPolicy){0}, NULL));
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
EXPECT_EQ_INT((int)st.st_size, (int)strlen(content));
@@ -518,6 +568,20 @@ static void test_file_symlink_helpers() {
EXPECT_FALSE(file_symlink_target_contained("../escape"));
EXPECT_FALSE(file_symlink_target_contained("a/../b"));
EXPECT_FALSE(file_symlink_target_contained(""));
/* rsync 3.4.1 unsafe_symlink(): absolute/empty are unsafe; ".." is measured
against the symlink's own transfer-relative directory depth. */
EXPECT_TRUE(file_symlink_unsafe("/etc/passwd", "link"));
EXPECT_TRUE(file_symlink_unsafe("", "link"));
EXPECT_FALSE(file_symlink_unsafe("a.txt", "link"));
EXPECT_FALSE(file_symlink_unsafe("./a.txt", "link"));
EXPECT_FALSE(file_symlink_unsafe("../real.txt", "a/up1"));
EXPECT_FALSE(file_symlink_unsafe("../../real.txt", "a/b/up3"));
EXPECT_TRUE(file_symlink_unsafe("../../../outside", "a/b/esc"));
EXPECT_TRUE(file_symlink_unsafe("../outside", "esc"));
/* Internal /../ and a trailing /.. are rejected by rsync 3.4.1. */
EXPECT_TRUE(file_symlink_unsafe("a/b/../real.txt", "norm"));
EXPECT_TRUE(file_symlink_unsafe("dir/..", "link"));
}
static void test_file_symlink_at_secure() {
@@ -943,7 +1007,8 @@ static void test_inplace_overwrite_metadata_strips_special_bits() {
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
/* Metadata-derived mode is applied and never includes setuid/setgid/sticky. */
/* No -p: the pre-existing destination mode (without its special bits) is
* restored; the source mode is not applied. */
EXPECT_EQ_INT((int)(st.st_mode & (S_ISUID | S_ISGID | S_ISVTX)), 0);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0755);
@@ -951,6 +1016,212 @@ static void test_inplace_overwrite_metadata_strips_special_bits() {
rmdir(root);
}
/* The per-attribute split: with no -p/-E the atomic (inode-replacing) write
* must restore the PRE-EXISTING destination mode instead of the source mode; a
* brand-new file keeps the historical 0644 default; -p applies the source. */
static void test_atomic_no_perms_preserves_destination_mode() {
const char* path = "test_attr_split_mode.txt";
unlink(path);
int fd = open(path, O_WRONLY | O_CREAT | O_TRUNC | O_CLOEXEC, 0640);
EXPECT_TRUE(fd >= 0);
/* cppcheck-suppress knownConditionTrueFalse */
if (fd < 0)
return;
EXPECT_EQ_INT(fchmod(fd, 0640), 0);
EXPECT_EQ_INT(close(fd), 0);
FileMetadata m;
memset(&m, 0, sizeof(m));
m.mode = 0755;
m.uid = geteuid();
m.gid = getegid();
m.mtime_sec = 1700000000;
/* No -p/-E: the pre-existing 0640 survives the atomic overwrite. */
bool ok = file_to_disk_secure_attrs(path, "data", 4, false, false, false, &m,
(FileAttrPolicy){false, false, false, false}, false, false,
false, NULL, false, false, NULL);
EXPECT_TRUE(ok);
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0640);
/* -p: the source mode wins. */
ok = file_to_disk_secure_attrs(path, "data2", 5, false, false, false, &m,
(FileAttrPolicy){true, true, false, false}, false, false, false,
NULL, false, false, NULL);
EXPECT_TRUE(ok);
EXPECT_EQ_INT(stat(path, &st), 0);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0755);
/* -E only (rsync rule): an executable source derives exec from the
pre-existing destination's read bits. Dest 0640 (owner+group read) with a
source 0755 gives 0750, not 0751 and not the scratch 0711. */
EXPECT_EQ_INT(chmod(path, 0640), 0);
ok = file_to_disk_secure_attrs(path, "data3", 6, false, false, false, &m,
(FileAttrPolicy){false, false, false, true}, false, false, false,
NULL, false, false, NULL);
EXPECT_TRUE(ok);
EXPECT_EQ_INT(stat(path, &st), 0);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0750);
unlink(path);
/* A brand-new file with no -p uses rsync's source&~umask base when metadata
is available (m.mode is 0755 here). */
const char* fresh = "test_attr_split_fresh.txt";
unlink(fresh);
ok = file_to_disk_secure_attrs(fresh, "data", 4, false, false, false, &m,
(FileAttrPolicy){false, false, false, false}, false, false, false,
NULL, false, false, NULL);
EXPECT_TRUE(ok);
EXPECT_EQ_INT(stat(fresh, &st), 0);
EXPECT_EQ_INT((int)(st.st_mode & 0777), (int)(m.mode & 0777 & ~(mode_t)file_process_umask()));
unlink(fresh);
/* Without any metadata the historical fixed 0644 default still applies. */
unlink(fresh);
ok = file_to_disk_secure_attrs(fresh, "data", 4, false, false, false, NULL,
(FileAttrPolicy){false, false, false, false}, false, false, false,
NULL, false, false, NULL);
EXPECT_TRUE(ok);
EXPECT_EQ_INT(stat(fresh, &st), 0);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0644);
unlink(fresh);
}
/* Strict rsync parity: a brand-new destination file with no -p follows
* rsync's source_mode & ~umask base, so group/other write in the source mode is
* honored exactly as the umask allows (it is no longer force-cleared). */
static void test_new_file_mode_honors_source_and_umask() {
const char* path = "test_new_file_mode.bin";
unlink(path);
FileMetadata m;
memset(&m, 0, sizeof(m));
m.mode = 0666; /* maximal group/other write in the source mode */
m.uid = geteuid();
m.gid = getegid();
bool ok = file_to_disk_secure_attrs(path, "x", 1, false, false, false, &m,
(FileAttrPolicy){false, false, false, false}, false, false,
false, NULL, false, false, NULL);
EXPECT_TRUE(ok);
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
EXPECT_EQ_INT((int)(st.st_mode & 0777), (int)(0666 & ~(mode_t)file_process_umask()));
unlink(path);
}
/* Strict rsync parity for recreated special nodes: with -p the source mode is
* copied exactly (0777 -> 0777), and without -p the same source & ~umask base
* as any other new entry applies. The process umask is cleared so the source
* bits are what reaches mkfifo. */
static void test_special_fifo_mode_honors_source_and_umask_impl() {
const char* root = "test_special_mode_tmp";
const char* with_p = "test_special_mode_tmp/with_p.fifo";
const char* no_p = "test_special_mode_tmp/no_p.fifo";
unlink(with_p);
unlink(no_p);
rmdir(root);
EXPECT_EQ_INT(mkdir(root, 0700), 0);
Config* cfg = config_create();
EXPECT_NOT_NULL(cfg);
FileMetadata meta;
memset(&meta, 0, sizeof(meta));
meta.mode = S_IFIFO | 0777;
meta.uid = geteuid();
meta.gid = getegid();
meta.mtime_sec = 1000000000;
/* -p: the source mode (including group/other write) is copied exactly. */
File* f = file_create("with_p.fifo");
EXPECT_NOT_NULL(f);
f->is_special = true;
f->metadata = &meta;
cfg->preserve_specials = true;
cfg->preserve_perms = true;
cfg->use_metadata = true;
EXPECT_EQ_INT(file_save_to_disk_full(root, f, cfg), FILE_SAVE_WRITTEN);
struct stat st;
EXPECT_EQ_INT(lstat(with_p, &st), 0);
EXPECT_TRUE(S_ISFIFO(st.st_mode));
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0777);
f->metadata = NULL;
file_destroy(f);
/* No -p: source & ~umask (umask is cleared, so 0777). */
f = file_create("no_p.fifo");
EXPECT_NOT_NULL(f);
f->is_special = true;
f->metadata = &meta;
cfg->preserve_perms = false;
EXPECT_EQ_INT(file_save_to_disk_full(root, f, cfg), FILE_SAVE_WRITTEN);
EXPECT_EQ_INT(lstat(no_p, &st), 0);
EXPECT_TRUE(S_ISFIFO(st.st_mode));
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0777);
f->metadata = NULL;
file_destroy(f);
config_delete(cfg);
unlink(with_p);
unlink(no_p);
rmdir(root);
}
/* The receiver daemon runs umask(0), so the source mode reaches mkfifo
* unmasked. Run the body with umask(0) and refresh the cached process umask so
* file_process_umask() agrees, then restore both. */
static void test_special_fifo_mode_honors_source_and_umask() {
mode_t saved_umask = umask(0);
file_umask_capture();
test_special_fifo_mode_honors_source_and_umask_impl();
umask(saved_umask);
file_umask_capture();
}
/* --specials recreates a unix-domain socket via mknod(S_IFSOCK), which Linux
* permits unprivileged. Without --specials the entry is skipped. */
static void test_special_socket_recreated() {
const char* root = "test_special_sock_tmp";
const char* sock = "test_special_sock_tmp/source.sock";
unlink(sock);
rmdir(root);
EXPECT_EQ_INT(mkdir(root, 0700), 0);
Config* cfg = config_create();
EXPECT_NOT_NULL(cfg);
FileMetadata meta;
memset(&meta, 0, sizeof(meta));
meta.mode = S_IFSOCK | 0600;
meta.uid = geteuid();
meta.gid = getegid();
File* f = file_create("source.sock");
EXPECT_NOT_NULL(f);
f->is_special = true;
f->metadata = &meta;
cfg->preserve_specials = true;
cfg->use_metadata = true;
EXPECT_EQ_INT(file_save_to_disk_full(root, f, cfg), FILE_SAVE_WRITTEN);
struct stat st;
EXPECT_EQ_INT(lstat(sock, &st), 0);
EXPECT_TRUE(S_ISSOCK(st.st_mode));
/* Without --specials the same entry is skipped, never a regular file. */
unlink(sock);
cfg->preserve_specials = false;
EXPECT_EQ_INT(file_save_to_disk_full(root, f, cfg), FILE_SAVE_SKIPPED);
EXPECT_EQ_INT(lstat(sock, &st), -1);
f->metadata = NULL;
file_destroy(f);
config_delete(cfg);
unlink(sock);
rmdir(root);
}
static void test_inplace_overwrite_truncates_shorter_payload() {
const char* root = "test_inplace_trunc_tmp";
const char* path = "test_inplace_trunc_tmp/big.txt";
@@ -1125,34 +1396,27 @@ static void test_dir_entry_save_to_disk() {
* receiver enables it from its own process (the standalone server's --trust-
* sender CLI switch, which a client forwards as --remote-option=--trust-sender),
* so these tests force file_set_trust_sender(true) directly. Trust must RELAX
* only the redundant list-level re-validation (an escaping symlink TARGET is
* copied verbatim, rsync -l parity) and must NEVER disable the low-level
* fd-relative confinement floor: file_open_secure_parent's ".." rejection, the
* O_NOFOLLOW parent walk, leaf/destination confinement, and the ungated
* has_path_traversal on the link's own placement path in file_symlink_at_secure
* stay hard. A hostile sender therefore still cannot place a file, directory
* or symlink outside the receive root even with trust on. */
* only the redundant list-level re-validation and must NEVER disable the
* low-level fd-relative confinement floor: file_open_secure_parent's ".."
* rejection, the O_NOFOLLOW parent walk, leaf/destination confinement, and the
* ungated has_path_traversal on the link's own placement path in
* file_symlink_at_secure stay hard. A hostile sender therefore still cannot
* place a file, directory or symlink outside the receive root. */
static void test_trust_sender_relaxes_symlink_target() {
const char* root = "test_trust_sender_root";
const char* link = "test_trust_sender_root/escape_link";
static void test_symlink_target_verbatim() {
const char* root = "test_symlink_verbatim_root";
const char* link = "test_symlink_verbatim_root/escape_link";
unlink(link);
rmdir(root);
EXPECT_EQ_INT(mkdir(root, 0755), 0);
/* Control: without trust an absolute (escaping) target is refused and the
link is never placed. */
/* rsync -l parity: a symlink target is stored verbatim, absolute or not; the
scanner's --safe-links/--copy-unsafe-links is what filters links. */
file_set_trust_sender(false);
EXPECT_FALSE(file_symlink_at_secure(link, "/etc/passwd"));
struct stat st;
EXPECT_EQ_INT(lstat(link, &st), -1);
/* Trust ON: the escaping target is copied verbatim (rsync -l parity) ... */
file_set_trust_sender(true);
EXPECT_TRUE(file_symlink_at_secure(link, "/etc/passwd"));
struct stat st;
EXPECT_EQ_INT(lstat(link, &st), 0);
EXPECT_TRUE(S_ISLNK(st.st_mode));
/* ...but the link itself still lands beneath the receive root. */
char target[128];
ssize_t target_len = readlink(link, target, sizeof(target) - 1);
EXPECT_TRUE(target_len > 0);
@@ -1163,10 +1427,10 @@ static void test_trust_sender_relaxes_symlink_target() {
}
unlink(link);
/* Same relaxation through the real save funnel (file_save_to_disk_full). */
/* The same through the real save funnel: verbatim by default. */
Config* config = config_create();
EXPECT_NOT_NULL(config);
const char* save_link = "test_trust_sender_root/save_link";
const char* save_link = "test_symlink_verbatim_root/save_link";
unlink(save_link);
File* sym = file_create("save_link");
@@ -1176,10 +1440,6 @@ static void test_trust_sender_relaxes_symlink_target() {
EXPECT_NOT_NULL(sym->symlink_target);
file_set_trust_sender(false);
EXPECT_EQ_INT(file_save_to_disk_full(root, sym, config), FILE_SAVE_SKIPPED);
EXPECT_EQ_INT(lstat(save_link, &st), -1);
file_set_trust_sender(true);
EXPECT_EQ_INT(file_save_to_disk_full(root, sym, config), FILE_SAVE_WRITTEN);
EXPECT_EQ_INT(lstat(save_link, &st), 0);
EXPECT_TRUE(S_ISLNK(st.st_mode));
@@ -1305,7 +1565,7 @@ void test_trust_sender() {
helper), so a later group never inherits a stray trust/authorized-root
policy. */
file_set_trust_sender(false);
test_trust_sender_relaxes_symlink_target();
test_symlink_target_verbatim();
test_trust_sender_confines_hostile_paths();
test_trust_sender_authorized_root_confinement();
file_set_trust_sender(false);
@@ -1335,7 +1595,8 @@ static void test_file_write_to_disk_sparse_preserves_holes() {
buf[size - 1 - i] = (unsigned char)((i * 7) % 253);
}
EXPECT_TRUE(file_to_disk_secure(path, buf, size, false, true, false, NULL, false, NULL));
EXPECT_TRUE(
file_to_disk_secure(path, buf, size, false, true, false, NULL, (FileAttrPolicy){0}, NULL));
/* Logical size must equal data_size exactly. */
struct stat st;
@@ -1400,8 +1661,9 @@ static void test_file_write_to_disk_partial_retention() {
m.mtime_nsec = 2000000000; /* invalid: forces futimens EINVAL after the write */
m.atime_valid = false;
m.crtime_valid = false;
bool ok = file_to_disk_secure_attrs(path, content, strlen(content), false, false, true, &m, false,
false, false, false, NULL, false, true, NULL);
bool ok = file_to_disk_secure_attrs(path, content, strlen(content), false, false, true, &m,
(FileAttrPolicy){true, true, false, false}, false, false,
false, NULL, false, true, NULL);
EXPECT_FALSE(ok); /* the write itself succeeded, but metadata restore failed */
/* Retained: the already-written temp now sits at the destination path. */
int fd = open(path, O_RDONLY);
@@ -1419,8 +1681,9 @@ static void test_file_write_to_disk_partial_retention() {
unlink(path);
/* Same failure with keep_partial=false: temp is unlinked, nothing retained. */
ok = file_to_disk_secure_attrs(path, content, strlen(content), false, false, true, &m, false,
false, false, false, NULL, false, false, NULL);
ok = file_to_disk_secure_attrs(path, content, strlen(content), false, false, true, &m,
(FileAttrPolicy){true, true, false, false}, false, false, false,
NULL, false, false, NULL);
EXPECT_FALSE(ok);
EXPECT_TRUE(access(path, F_OK) == -1);
}
@@ -1440,19 +1703,35 @@ static void test_dir_time_list() {
dir_time_list_init(&list);
EXPECT_EQ_INT((int)list.count, 0);
FileMetadata metadata = {.mtime_sec = 1000000000, .mtime_nsec = 0};
EXPECT_TRUE(dir_time_list_add(&list, "sub", &metadata));
EXPECT_TRUE(dir_time_list_add(&list, "sub", &metadata));
EXPECT_TRUE(dir_time_list_add(&list, "sub", &metadata, NULL));
/* A captured xattr block is deep-copied into the list. */
FileXattrList* xl = xattr_list_new();
EXPECT_NOT_NULL(xl);
EXPECT_TRUE(xattr_list_append(xl, "user.dir", "v", 1));
EXPECT_TRUE(dir_time_list_add(&list, "sub", &metadata, xl));
xattr_list_free(xl); /* the list owns its own copy now */
EXPECT_EQ_INT((int)list.count, 2);
EXPECT_NOT_NULL(list.xattrs);
EXPECT_NOT_NULL(list.xattrs[1]);
EXPECT_EQ_INT(list.xattrs[1]->count, 1);
EXPECT_EQ_STR(list.xattrs[1]->items[0].name, "user.dir");
EXPECT_NULL(list.xattrs[0]);
dir_time_list_apply(&list, root);
Config* cfg = config_create();
EXPECT_NOT_NULL(cfg);
cfg->use_metadata = true;
cfg->preserve_times = true;
dir_metadata_list_apply(&list, root, cfg);
struct stat st;
EXPECT_EQ_INT(stat(sub, &st), 0);
EXPECT_EQ_INT((int)st.st_mtime, 1000000000);
config_delete(cfg);
dir_time_list_free(&list);
EXPECT_EQ_INT((int)list.count, 0);
EXPECT_NULL(list.paths);
EXPECT_NULL(list.entries);
EXPECT_NULL(list.xattrs);
rmdir(sub);
rmdir(root);
@@ -1477,14 +1756,14 @@ static void test_dir_time_list_cap() {
for (size_t i = 0; i < MAX_DIR_TIME_ENTRIES + 1 && !rejected; i++) {
size_t before_count = list.count;
size_t before_bytes = list.bytes;
if (!dir_time_list_add(&list, path, &metadata)) {
if (!dir_time_list_add(&list, path, &metadata, NULL)) {
rejected = true;
/* The rejected add must not have partially mutated the list. */
EXPECT_TRUE(list.count == before_count);
EXPECT_TRUE(list.bytes == before_bytes);
} else {
EXPECT_TRUE(list.count == before_count + 1);
EXPECT_TRUE(list.bytes == before_bytes + path_len + sizeof(FileMetadata) + sizeof(char*));
EXPECT_TRUE(list.bytes == before_bytes + path_len + sizeof(FileMetadata) + 2 * sizeof(char*));
}
}
EXPECT_TRUE(rejected);
@@ -1648,6 +1927,81 @@ static void test_keep_dirlinks_secure_open() {
file_set_keep_dirlinks(false);
}
/* Build an ArrayList of str_dup'd strings (NULL on allocation failure). */
static ArrayList* make_manifest_string_list(const char* const* entries, int count) {
ArrayList* list = array_list_create(free);
if (!list)
return NULL;
for (int i = 0; i < count; i++) {
char* dup = str_dup(entries[i]);
if (!dup || !array_list_add(list, dup)) {
free(dup);
array_list_delete(list);
return NULL;
}
}
return list;
}
/* Regression (#3): a non-empty --delete-missing-args directory charges each
* removed entry exactly once. The directory itself must not be counted twice;
* if it were, `deleted` would exceed --max-delete and the extras walk would
* underflow its remaining budget and delete past the user's cap. */
static void test_manifest_delete_missing_dir_budget_double_count() {
char root[PATH_MAX];
snprintf(root, sizeof(root), "/tmp/fastsync_mgdir_%d", (int)getpid());
char* gone = path_cat(root, "gone");
char* gone_file = path_cat(gone, "f0");
char* extra = path_cat(root, "extra.txt");
EXPECT_NOT_NULL(gone);
EXPECT_NOT_NULL(gone_file);
EXPECT_NOT_NULL(extra);
mkdir(root, 0755);
mkdir(gone, 0755);
EXPECT_EQ_INT(access(extra, F_OK), -1);
EXPECT_TRUE(file_write_to_disk(extra, "extra", 5, false, false));
/* The missing-arg directory holds N-1 == 2 entries; with the directory itself
that is exactly --max-delete=3. */
EXPECT_TRUE(file_write_to_disk(gone_file, "x", 1, false, false));
char* gone_file2 = path_cat(gone, "f1");
EXPECT_TRUE(gone_file2 != NULL && file_write_to_disk(gone_file2, "x", 1, false, false));
Config* cfg = config_create();
EXPECT_NOT_NULL(cfg);
cfg->receive_root_directory = str_dup(root);
cfg->use_delete = true;
cfg->delete_missing_args = true;
cfg->max_delete = 3;
const char* missing_names[] = {"gone"};
const char* synced[] = {"."};
DeleteManifest manifest = {0};
manifest.keeps = make_manifest_string_list(NULL, 0);
manifest.missing = make_manifest_string_list(missing_names, 1);
manifest.dirs = make_manifest_string_list(synced, 1);
EXPECT_NOT_NULL(manifest.keeps);
EXPECT_NOT_NULL(manifest.missing);
EXPECT_NOT_NULL(manifest.dirs);
DeleteCommitResult result = manifest_delete_all(cfg, &manifest);
EXPECT_EQ_INT((int)result, (int)DELETE_COMMIT_LIMIT_REACHED);
/* The whole missing-arg directory is gone (dir + its 2 entries == 3). */
EXPECT_EQ_INT(access(gone, F_OK), -1);
/* The saturated budget must leave the in-scope extra untouched. */
EXPECT_EQ_INT(access(extra, F_OK), 0);
array_list_delete(manifest.keeps);
array_list_delete(manifest.missing);
array_list_delete(manifest.dirs);
config_delete(cfg);
unlink(extra);
free(gone);
free(gone_file);
free(gone_file2);
free(extra);
rmdir(root);
}
void test_file() {
test_file_create();
test_file_special_rdev_valid();
@@ -1661,6 +2015,7 @@ void test_file() {
test_file_save_to_disk_ignore_existing();
test_file_save_to_disk_ignore_existing_entry_types();
test_file_save_to_disk_partial_install();
test_file_save_to_disk_temp_dir_confined();
test_file_save_to_disk_reports_skips();
test_file_write_to_disk_sparse_preserves_holes();
test_file_write_to_disk_partial_retention();
@@ -1694,7 +2049,12 @@ void test_file() {
test_keep_dirlinks_secure_open();
test_inplace_overwrite_clears_special_mode_bits();
test_inplace_overwrite_metadata_strips_special_bits();
test_atomic_no_perms_preserves_destination_mode();
test_new_file_mode_honors_source_and_umask();
test_special_fifo_mode_honors_source_and_umask();
test_special_socket_recreated();
test_inplace_overwrite_truncates_shorter_payload();
test_inplace_refuses_fifo_destination();
test_inplace_refuses_device_destination();
test_manifest_delete_missing_dir_budget_double_count();
}
+45
View File
@@ -148,6 +148,50 @@ static void test_ancestor_and_descendant_queries() {
remove(path);
}
/* The delete-walker's synchronized-directory predicate: a directory is in scope
only when it is a listed directory or lies below one, NOT when it is merely an
implied parent of a listed file. */
static void test_dir_in_scope() {
char err[160];
/* NULL set / empty list semantics. */
EXPECT_TRUE(file_list_dir_in_scope(NULL, "anything"));
const char* path = "test_file_list_dirscope.txt";
write_list(path, "d1/leaf.txt\n");
FileListSet* set = file_list_load(path, false, err, sizeof(err));
EXPECT_NOT_NULL(set);
/* d1 is only an implied parent of a listed FILE: not synchronized. */
EXPECT_FALSE(file_list_dir_in_scope(set, "d1"));
EXPECT_FALSE(file_list_dir_in_scope(set, "d1/sub"));
EXPECT_FALSE(file_list_dir_in_scope(set, "other"));
file_list_destroy(set);
remove(path);
/* A listed DIRECTORY synchronizes itself and its whole subtree. */
write_list(path, "d1/\nother\n");
set = file_list_load(path, false, err, sizeof(err));
EXPECT_NOT_NULL(set);
EXPECT_TRUE(file_list_dir_in_scope(set, "d1"));
EXPECT_TRUE(file_list_dir_in_scope(set, "d1/sub/deep"));
EXPECT_TRUE(file_list_dir_in_scope(set, "other"));
EXPECT_TRUE(file_list_dir_in_scope(set, "other/x"));
EXPECT_FALSE(file_list_dir_in_scope(set, "d2"));
EXPECT_FALSE(file_list_dir_in_scope(set, "d1x")); /* component boundary */
EXPECT_FALSE(file_list_dir_in_scope(set, ""));
file_list_destroy(set);
remove(path);
/* "." lists the whole tree. */
write_list(path, ".\n");
set = file_list_load(path, false, err, sizeof(err));
EXPECT_NOT_NULL(set);
EXPECT_TRUE(file_list_dir_in_scope(set, ""));
EXPECT_TRUE(file_list_dir_in_scope(set, "anything/at/all"));
file_list_destroy(set);
remove(path);
}
/* Regression for the remote OOM: an adversarial --files-from entry made of a
very deep chain of repeated components must be indexed with memory
proportional to the entry count. The old implementation stored one copied
@@ -219,6 +263,7 @@ static void test_oversized_entry_rejected() {
void test_file_list() {
test_membership_matches_reference();
test_ancestor_and_descendant_queries();
test_dir_in_scope();
test_deep_paths_are_bounded();
test_oversized_entry_rejected();
}
+94
View File
@@ -0,0 +1,94 @@
#include "test_format.h"
#include "format.h"
#include "test_utils.h"
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <time.h>
#include <unistd.h>
static void expect_big_num(unsigned long long value, bool human, const char* expected) {
char buffer[64];
EXPECT_TRUE(format_big_num(value, human, buffer, sizeof(buffer)));
EXPECT_EQ_STR(buffer, expected);
}
static void test_human_size_decimal() {
/* Values below 1000 print verbatim; larger values use the largest unit that
* keeps the value below 1000 and exactly two decimals (rsync human_num). */
expect_big_num(0, true, "0");
expect_big_num(999, true, "999");
expect_big_num(1000, true, "1.00K");
expect_big_num(1500, true, "1.50K");
expect_big_num(9999, true, "10.00K");
expect_big_num(999999, true, "1000.00K");
expect_big_num(1000000, true, "1.00M");
expect_big_num(1500000, true, "1.50M");
}
static void test_big_num_grouping() {
/* Non-human numbers are comma-grouped every three digits (rsync big_num). */
expect_big_num(0, false, "0");
expect_big_num(1, false, "1");
expect_big_num(999, false, "999");
expect_big_num(1000, false, "1,000");
expect_big_num(4096, false, "4,096");
expect_big_num(1234567, false, "1,234,567");
expect_big_num(1000000000ULL, false, "1,000,000,000");
}
static void test_datetime_format() {
char buffer[32];
time_t when = 1700000000;
EXPECT_TRUE(format_rsync_datetime(when, true, buffer, sizeof(buffer)));
/* %M shape: YYYY/MM/DD-HH:MM:SS */
EXPECT_EQ_INT(strlen(buffer), 19);
EXPECT_EQ_INT(buffer[4], '/');
EXPECT_EQ_INT(buffer[7], '/');
EXPECT_EQ_INT(buffer[10], '-');
EXPECT_EQ_INT(buffer[13], ':');
EXPECT_EQ_INT(buffer[16], ':');
char space_form[32];
EXPECT_TRUE(format_rsync_datetime(when, false, space_form, sizeof(space_form)));
EXPECT_EQ_INT(space_form[10], ' ');
}
static void test_dest_state_roundtrip() {
/* The wire codec is exercised over a socketpair so the real send/receive
* primitives run. */
int fds[2];
if (socketpair(AF_UNIX, SOCK_STREAM, 0, fds) != 0)
return;
OutputDestState out;
memset(&out, 0, sizeof(out));
out.known = true;
out.existed = true;
out.size = 123456789ULL;
out.mtime_sec = 1700000000;
out.mtime_nsec = 123456789;
out.mode = 0100644;
out.uid = 1000;
out.gid = 1000;
OutputDestState in;
memset(&in, 0, sizeof(in));
EXPECT_TRUE(format_dest_state_send(fds[0], &out));
EXPECT_TRUE(format_dest_state_receive(fds[1], &in));
EXPECT_TRUE(in.known);
EXPECT_TRUE(in.existed);
EXPECT_TRUE(in.size == out.size);
EXPECT_TRUE(in.mtime_sec == out.mtime_sec);
EXPECT_TRUE(in.mtime_nsec == out.mtime_nsec);
EXPECT_TRUE(in.mode == out.mode);
EXPECT_TRUE(in.uid == out.uid);
EXPECT_TRUE(in.gid == out.gid);
close(fds[0]);
close(fds[1]);
}
void test_format(void) {
test_human_size_decimal();
test_big_num_grouping();
test_datetime_format();
test_dest_state_roundtrip();
}
+6
View File
@@ -0,0 +1,6 @@
#ifndef TEST_FORMAT_H
#define TEST_FORMAT_H
void test_format(void);
#endif
+21 -13
View File
@@ -18,6 +18,9 @@
/* P8 config-frame tail: super_mode (4) + copy-as presence (4) + uid (4) + gid (4). */
#define P8_TAIL_BYTES 16
/* Protocol 2.23.0 appends one trailing bool (report_dest_info) AFTER the P8
* tail, so the P8 fields sit this many bytes before the end of the frame. */
#define OUTPUT_TAIL_BYTES 4
/* Smoke test for chunk_deserialize fuzz target */
static void test_fuzz_chunk_deserialize() {
@@ -334,31 +337,31 @@ static void test_fuzz_config_receive_p8_tail() {
/* super_mode outside the 0..2 tri-state is refused. */
memcpy(mut, frame, len);
put_i32(mut, len - P8_TAIL_BYTES, 99);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES, 99);
EXPECT_FALSE(receive_config_frame(mut, len));
put_i32(mut, len - P8_TAIL_BYTES, -1);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES, -1);
EXPECT_FALSE(receive_config_frame(mut, len));
/* A negative (sentinel) and an extreme copy-as uid/gid are refused. */
memcpy(mut, frame, len);
put_i32(mut, len - P8_TAIL_BYTES, SUPER_MODE_AUTO);
put_i32(mut, len - P8_TAIL_BYTES + 4, 1);
put_i32(mut, len - P8_TAIL_BYTES + 8, -1);
put_i32(mut, len - P8_TAIL_BYTES + 12, 0);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES, SUPER_MODE_AUTO);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES + 4, 1);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES + 8, -1);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES + 12, 0);
EXPECT_FALSE(receive_config_frame(mut, len));
put_i32(mut, len - P8_TAIL_BYTES + 8, 0);
put_i32(mut, len - P8_TAIL_BYTES + 12, INT32_MIN);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES + 8, 0);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES + 12, INT32_MIN);
EXPECT_FALSE(receive_config_frame(mut, len));
/* A presence int that is not a wire bool is refused. */
memcpy(mut, frame, len);
put_i32(mut, len - P8_TAIL_BYTES, SUPER_MODE_AUTO);
put_i32(mut, len - P8_TAIL_BYTES + 4, 2);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES, SUPER_MODE_AUTO);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES + 4, 2);
EXPECT_FALSE(receive_config_frame(mut, len));
/* Truncating anywhere inside the P8 tail is refused. */
EXPECT_FALSE(receive_config_frame(frame, len - 2));
EXPECT_FALSE(receive_config_frame(frame, len - P8_TAIL_BYTES));
EXPECT_FALSE(receive_config_frame(frame, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES));
free(mut);
free(frame);
@@ -379,7 +382,9 @@ static void test_fuzz_config_receive_huge_map_count() {
}
c->usermap_count = 1;
c->usermap[0].from = sentinel_from;
c->usermap[0].from_hi = sentinel_from;
c->usermap[0].to = sentinel_to;
c->usermap[0].to_name = NULL;
unsigned char* frame = NULL;
size_t len = 0;
@@ -390,9 +395,12 @@ static void test_fuzz_config_receive_huge_map_count() {
return;
}
unsigned char pattern[8];
/* One wire entry is [from][from_hi][to][to_name]; search the fixed-width
prefix (the to_name length-prefixed string follows). */
unsigned char pattern[12];
memcpy(pattern, &sentinel_from, sizeof(sentinel_from));
memcpy(pattern + sizeof(sentinel_from), &sentinel_to, sizeof(sentinel_to));
memcpy(pattern + sizeof(sentinel_from), &sentinel_from, sizeof(sentinel_from));
memcpy(pattern + 2 * sizeof(sentinel_from), &sentinel_to, sizeof(sentinel_to));
size_t entry_off = find_bytes(frame, len, pattern, sizeof(pattern));
if (entry_off == SIZE_MAX || entry_off < sizeof(int32_t)) {
free(frame);
+303 -8
View File
@@ -1,8 +1,10 @@
#include "test_metadata.h"
#include "chmod.h"
#include "identity.h"
#include "metadata.h"
#include "protocol.h"
#include "test_utils.h"
#include <fcntl.h>
#include <stdlib.h>
#include <string.h>
#include <sys/ioctl.h>
@@ -337,7 +339,7 @@ static void test_file_restore_metadata_applies_atime() {
m.crtime_sec = 0;
m.crtime_nsec = 0;
file_restore_metadata(path, &m, false);
file_restore_metadata(path, &m, (FileAttrPolicy){true, true, true, false});
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
@@ -376,7 +378,7 @@ static void test_file_restore_metadata() {
.atime_valid = false,
.crtime_valid = false};
file_restore_metadata(path, &m, false);
file_restore_metadata(path, &m, (FileAttrPolicy){true, true, false, false});
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
@@ -393,7 +395,7 @@ static void test_file_restore_executability_only() {
FileMetadata m = {
.mode = 0751, .uid = getuid(), .gid = getgid(), .mtime_sec = 0, .mtime_nsec = 0};
file_restore_metadata(path, &m, true);
file_restore_metadata(path, &m, (FileAttrPolicy){false, false, false, true});
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
@@ -407,14 +409,190 @@ static void test_directory_restore_executability_only() {
FileMetadata m = {
.mode = 0755, .uid = getuid(), .gid = getgid(), .mtime_sec = 0, .mtime_nsec = 0};
file_restore_metadata(path, &m, true);
file_restore_metadata(path, &m, (FileAttrPolicy){false, false, false, true});
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
EXPECT_EQ_INT(st.st_mode & 0777, 0711);
/* rsync -E: an executable source derives exec from the DESTINATION's read
* bits. A 0700 directory has read only for the owner, so only the owner
* gains exec -- the result stays 0700 (not 0711, the old per-class copy). */
EXPECT_EQ_INT(st.st_mode & 0777, 0700);
rmdir(path);
}
/* rsync 3.4 -E truth table (preserve_perms off), verified against rsync 3.4.1:
* (src,dest) -> result. A non-executable source clears every execute bit; an
* executable source sets a class's execute bit iff that class can read. */
static void test_file_restore_executability_rsync_rule() {
static const struct {
mode_t src;
mode_t dest;
mode_t want;
} cases[] = {
{0755, 0644, 0755}, {0755, 0600, 0700}, {0755, 0640, 0750}, {0755, 0666, 0777},
{0700, 0640, 0750}, {0111, 0644, 0755}, {0644, 0755, 0644}, {0644, 0600, 0600},
};
const char* path = "temp_exec_rsync_rule.txt";
for (size_t i = 0; i < sizeof(cases) / sizeof(cases[0]); i++) {
EXPECT_TRUE(file_write_to_disk(path, "x", 1, false, false));
EXPECT_EQ_INT(chmod(path, cases[i].dest), 0);
FileMetadata m = {.mode = cases[i].src, .uid = getuid(), .gid = getgid()};
file_restore_metadata(path, &m, (FileAttrPolicy){false, false, false, true});
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
EXPECT_EQ_INT(st.st_mode & 0777, cases[i].want);
unlink(path);
}
}
/* The shared metadata_mode_for_policy() helper is the single source of truth
* used by both the normal metadata path and the --fake-super replay. It must
* reproduce the per-attribute split: no mode change when neither -p nor -E is
* set; -p applies the source mode exactly (including group/other write and the
* setuid/setgid/sticky bits) regardless of the destination; -E derives exec
* bits from the destination and --perms wins when both are set. */
static void test_metadata_mode_for_policy() {
mode_t out = 0xdead;
EXPECT_FALSE(
metadata_mode_for_policy(0777, 0644, (FileAttrPolicy){false, false, false, false}, &out));
EXPECT_EQ_INT((int)out, 0xdead); /* untouched when no change is requested */
EXPECT_TRUE(
metadata_mode_for_policy(0777, 0644, (FileAttrPolicy){true, false, false, false}, &out));
EXPECT_EQ_INT((int)(out & 0777), 0777); /* group/other write is preserved */
mode_t specials = (mode_t)(S_ISUID | S_ISGID | S_ISVTX | 0672);
EXPECT_TRUE(
metadata_mode_for_policy(specials, 0644, (FileAttrPolicy){true, false, false, false}, &out));
EXPECT_EQ_INT((int)(out & (S_ISUID | S_ISGID | S_ISVTX | 0777)),
(int)(S_ISUID | S_ISGID | S_ISVTX | 0672));
/* -E: exec bits derive from the DESTINATION's read bits. */
EXPECT_TRUE(
metadata_mode_for_policy(0755, 0644, (FileAttrPolicy){false, false, false, true}, &out));
EXPECT_EQ_INT((int)(out & 0777), 0755);
EXPECT_TRUE(
metadata_mode_for_policy(0644, 0755, (FileAttrPolicy){false, false, false, true}, &out));
EXPECT_EQ_INT((int)(out & 0777), 0644);
EXPECT_TRUE(
metadata_mode_for_policy(0755, 0600, (FileAttrPolicy){false, false, false, true}, &out));
EXPECT_EQ_INT((int)(out & 0777), 0700);
/* --perms wins over -E when both are set. */
EXPECT_TRUE(
metadata_mode_for_policy(0700, 0644, (FileAttrPolicy){true, false, false, true}, &out));
EXPECT_EQ_INT((int)(out & 0777), 0700);
}
/* A brand-new file with -p off is created like rsync: source_mode & 0777 &
* ~umask (when metadata is available). The -E rule is then layered on top. */
static void test_new_file_mode_from_source_and_umask() {
const char* path = "temp_new_file_base.txt";
unlink(path);
FileMetadata m = {.mode = 0751, .uid = getuid(), .gid = getgid()};
mode_t want = (mode_t)(0751 & 0777 & ~(mode_t)file_process_umask());
bool ok = file_to_disk_secure_attrs(path, "x", 1, false, false, false, &m,
(FileAttrPolicy){false, false, false, false}, false, false,
false, NULL, false, false, NULL);
EXPECT_TRUE(ok);
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
EXPECT_EQ_INT(st.st_mode & 0777, want);
unlink(path);
/* -E on top of the source&~umask base (src 0751, umask 022 -> 0751). */
ok = file_to_disk_secure_attrs(path, "x", 1, false, false, false, &m,
(FileAttrPolicy){false, false, false, true}, false, false, false,
NULL, false, false, NULL);
EXPECT_TRUE(ok);
EXPECT_EQ_INT(stat(path, &st), 0);
mode_t want_e =
(want & 0444) ? (mode_t)(want | ((want & 0444) >> 2)) : (mode_t)(want & ~(mode_t)0111);
EXPECT_EQ_INT(st.st_mode & 0777, want_e);
unlink(path);
}
/* The per-attribute split: -t/-U apply times without touching the mode; an
* all-off policy applies neither mode nor times. */
static void test_file_restore_attribute_split() {
const char* path = "temp_meta_split_test.txt";
EXPECT_TRUE(file_write_to_disk(path, "x", 1, false, false));
EXPECT_EQ_INT(chmod(path, 0640), 0);
FileMetadata m = {.mode = 0755,
.uid = getuid(),
.gid = getgid(),
.mtime_sec = 1234567890,
.mtime_nsec = 0,
.atime_valid = false,
.crtime_valid = false};
/* times only: mtime changes, mode stays 0640. */
file_restore_metadata(path, &m, (FileAttrPolicy){false, true, false, false});
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
EXPECT_EQ_INT(st.st_mode & 0777, 0640);
EXPECT_EQ_INT((int)st.st_mtime, 1234567890);
/* no attributes: neither mode nor mtime changes. */
EXPECT_EQ_INT(chmod(path, 0640), 0);
struct timespec ts[2] = {{.tv_sec = 1000000000, .tv_nsec = 0},
{.tv_sec = 1000000000, .tv_nsec = 0}};
EXPECT_EQ_INT(utimensat(AT_FDCWD, path, ts, 0), 0);
FileMetadata m2 = m;
m2.mode = 0700;
m2.mtime_sec = 1600000000;
file_restore_metadata(path, &m2, (FileAttrPolicy){false, false, false, false});
EXPECT_EQ_INT(stat(path, &st), 0);
EXPECT_EQ_INT(st.st_mode & 0777, 0640);
EXPECT_EQ_INT((int)st.st_mtime, 1000000000);
unlink(path);
}
static void test_file_attr_policy_from_config() {
FileAttrPolicy none = file_attr_policy_from_config(NULL);
EXPECT_FALSE(none.perms);
EXPECT_FALSE(none.times);
EXPECT_FALSE(none.atimes);
EXPECT_FALSE(none.executability);
Config* c = config_create();
EXPECT_NOT_NULL(c);
c->preserve_perms = true;
c->preserve_times = true;
c->preserve_atimes = true;
c->use_executability = true;
FileAttrPolicy p = file_attr_policy_from_config(c);
EXPECT_TRUE(p.perms);
EXPECT_TRUE(p.times);
EXPECT_TRUE(p.atimes);
EXPECT_TRUE(p.executability);
config_delete(c);
}
/* Strict rsync parity: -p copies the source's setuid/setgid/sticky bits (they
* are attempted, not masked away). On Linux these are settable on a file the
* receiving user owns; a mount that denies them would log a chmod failure. */
static void test_perms_preserves_special_bits() {
const char* path = "temp_special_bits.txt";
unlink(path);
FileMetadata m = {
.mode = (mode_t)(S_ISUID | S_ISGID | S_ISVTX | 0755), .uid = getuid(), .gid = getgid()};
bool ok = file_to_disk_secure_attrs(path, "x", 1, false, false, false, &m,
(FileAttrPolicy){true, false, false, false}, false, false,
false, NULL, false, false, NULL);
EXPECT_TRUE(ok);
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0755);
EXPECT_EQ_INT((int)(st.st_mode & (S_ISUID | S_ISGID | S_ISVTX)),
(int)(S_ISUID | S_ISGID | S_ISVTX));
unlink(path);
}
static void test_chmod_changes() {
mode_t result;
EXPECT_TRUE(chmod_apply(0777, "u=rw,go=r", &result));
@@ -432,8 +610,45 @@ static void test_chmod_changes() {
EXPECT_EQ_INT(result, 0755);
EXPECT_FALSE(chmod_apply(0777, "888", &result));
EXPECT_FALSE(chmod_apply(0777, "10000", &result));
EXPECT_FALSE(chmod_apply(0777, "a+X", &result));
EXPECT_FALSE(chmod_apply(0777, "a+r,", &result));
/* go+w is honored (rsync gives 0666 from a 0644 file). */
EXPECT_TRUE(chmod_apply(0644, "go+w", &result));
EXPECT_EQ_INT(result, 0666);
/* X only sets execute on directories or already-executable files. */
EXPECT_TRUE(chmod_apply(0644, "a+X", &result));
EXPECT_EQ_INT(result, 0644);
EXPECT_TRUE(chmod_apply(0755, "a+X", &result));
EXPECT_EQ_INT(result, 0755);
EXPECT_TRUE(chmod_apply((mode_t)(S_IFDIR | 0644), "a+X", &result));
EXPECT_EQ_INT((int)(result & 0777), 0755);
EXPECT_TRUE(S_ISDIR(result));
/* D/F selectors restrict a clause to directories/files. */
EXPECT_TRUE(chmod_apply((mode_t)(S_IFDIR | 0700), "Dg+s", &result));
EXPECT_EQ_INT((int)(result & 07777), 02700);
EXPECT_TRUE(chmod_apply((mode_t)(S_IFREG | 0644), "Dg+s", &result));
EXPECT_EQ_INT((int)(result & 07777), 0644);
EXPECT_TRUE(chmod_apply((mode_t)(S_IFREG | 0644), "Fo-w", &result));
EXPECT_EQ_INT((int)(result & 07777), 0644);
EXPECT_TRUE(chmod_apply((mode_t)(S_IFREG | 0666), "Fo-w", &result));
EXPECT_EQ_INT((int)(result & 07777), 0664);
EXPECT_TRUE(chmod_apply((mode_t)(S_IFDIR | 0666), "Fo-w", &result));
EXPECT_EQ_INT((int)(result & 07777), 0666);
EXPECT_FALSE(chmod_apply(0644, "DFu+w", &result));
/* Special bits: s/t map to setuid/setgid/sticky like rsync. */
EXPECT_TRUE(chmod_apply(0755, "u+s", &result));
EXPECT_EQ_INT((int)(result & 07777), 04755);
EXPECT_TRUE(chmod_apply(0755, "g+s", &result));
EXPECT_EQ_INT((int)(result & 07777), 02755);
EXPECT_TRUE(chmod_apply(0755, "a+t", &result));
EXPECT_EQ_INT((int)(result & 07777), 01755);
/* Comma-separated clauses accumulate (the CLI joins repeated options). */
EXPECT_TRUE(chmod_apply(0644, "g+w,u+x", &result));
EXPECT_EQ_INT((int)(result & 07777), 0764);
}
/* P7 Wave D: symlink metadata is applied with no-follow primitives, and -J
@@ -455,7 +670,7 @@ static void test_file_restore_symlink_metadata() {
/* Positive path: a non-omitted apply stamps the link's own mtime. */
FileMetadata applied = {.mtime_sec = 1000000000, .mtime_nsec = 0};
file_restore_symlink_metadata(link, &applied, false);
file_restore_symlink_metadata(link, &applied, (FileAttrPolicy){false, true, false, false}, false);
struct stat st;
EXPECT_EQ_INT(lstat(link, &st), 0);
EXPECT_TRUE(S_ISLNK(st.st_mode));
@@ -464,7 +679,7 @@ static void test_file_restore_symlink_metadata() {
/* -J: a different time must be left untouched. */
FileMetadata newer = {.mtime_sec = 1234567890, .mtime_nsec = 0};
file_restore_symlink_metadata(link, &newer, true);
file_restore_symlink_metadata(link, &newer, (FileAttrPolicy){false, true, false, false}, true);
EXPECT_EQ_INT(lstat(link, &st), 0);
EXPECT_EQ_INT((int)st.st_mtime, (int)t1);
if (symlink_times_supported)
@@ -475,6 +690,79 @@ static void test_file_restore_symlink_metadata() {
rmdir(dir);
}
/* Per-attribute gating of the descriptor restore path: -p alone applies the
* mode, -t alone the mtime, -U alone the atime, and an all-off policy leaves
* the destination's mode and times exactly as they are. This pins the fd API
* the receiver actually uses (the path-based file_restore_metadata has its own
* split test). */
static void test_file_restore_metadata_fd_attribute_split() {
identity_clear_active(); /* no ownership policy leaking from a previous test */
const char* path = "temp_meta_fd_split_test.txt";
EXPECT_TRUE(file_write_to_disk(path, "x", 1, false, false));
EXPECT_EQ_INT(chmod(path, 0640), 0);
int fd = open(path, O_RDWR);
EXPECT_TRUE(fd >= 0);
/* cppcheck-suppress knownConditionTrueFalse -- EXPECT_TRUE above asserts,
but cppcheck cannot see through the macro; the guard is defensive. */
if (fd < 0) {
unlink(path);
return;
}
FileMetadata m = {.mode = 0755,
.uid = getuid(),
.gid = getgid(),
.mtime_sec = 1234567890,
.mtime_nsec = 0,
.atime_valid = true,
.atime_sec = 999999999,
.atime_nsec = 0,
.crtime_valid = false};
struct stat st;
struct stat before;
/* perms-only: mode applied, mtime untouched. */
EXPECT_EQ_INT(fstat(fd, &before), 0);
EXPECT_TRUE(file_restore_metadata_fd(fd, &m, (FileAttrPolicy){true, false, false, false}));
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT(st.st_mode & 0777, 0755);
EXPECT_EQ_INT((int)st.st_mtime, (int)before.st_mtime);
/* times-only: mtime applied, mode untouched. */
EXPECT_EQ_INT(chmod(path, 0600), 0);
EXPECT_TRUE(file_restore_metadata_fd(fd, &m, (FileAttrPolicy){false, true, false, false}));
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT(st.st_mode & 0777, 0600);
EXPECT_EQ_INT((int)st.st_mtime, 1234567890);
/* atime-only: atime applied, mtime and mode untouched. */
struct timespec reset[2] = {{.tv_sec = 1000000000, .tv_nsec = 0},
{.tv_sec = 1000000000, .tv_nsec = 0}};
EXPECT_EQ_INT(futimens(fd, reset), 0);
EXPECT_EQ_INT(fstat(fd, &before), 0);
EXPECT_TRUE(file_restore_metadata_fd(fd, &m, (FileAttrPolicy){false, false, true, false}));
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)st.st_atime, 999999999);
EXPECT_EQ_INT((int)st.st_mtime, (int)before.st_mtime);
EXPECT_EQ_INT(st.st_mode & 0777, 0600);
/* all-off: neither mode nor either time is touched. */
EXPECT_EQ_INT(chmod(path, 0640), 0);
EXPECT_EQ_INT(futimens(fd, reset), 0);
FileMetadata m2 = m;
m2.mode = 0700;
m2.mtime_sec = 1600000000;
m2.atime_sec = 1700000000;
EXPECT_TRUE(file_restore_metadata_fd(fd, &m2, (FileAttrPolicy){false, false, false, false}));
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT(st.st_mode & 0777, 0640);
EXPECT_EQ_INT((int)st.st_mtime, 1000000000);
EXPECT_EQ_INT((int)st.st_atime, 1000000000);
close(fd);
unlink(path);
}
void test_metadata() {
test_metadata_to_from_buf_roundtrip();
test_metadata_to_buf_null();
@@ -491,6 +779,13 @@ void test_metadata() {
test_file_restore_metadata_applies_atime();
test_file_restore_executability_only();
test_directory_restore_executability_only();
test_file_restore_executability_rsync_rule();
test_metadata_mode_for_policy();
test_new_file_mode_from_source_and_umask();
test_file_restore_attribute_split();
test_file_restore_metadata_fd_attribute_split();
test_file_attr_policy_from_config();
test_file_restore_symlink_metadata();
test_perms_preserves_special_bits();
test_chmod_changes();
}
+44 -3
View File
@@ -291,6 +291,35 @@ static void test_max_alloc_allows_configured_buffer() {
protocol_session_unbind();
}
/* max_alloc == 0 is rsync's --max-alloc=0 "no limit": allocations of any size
* are permitted. */
static void test_max_alloc_zero_means_unlimited() {
ProtocolSession session;
protocol_session_init(&session, -1, -1);
protocol_session_set_max_alloc(&session, 0);
protocol_session_bind(&session);
void* first = protocol_alloc(1024 * 1024);
void* second = protocol_alloc(8 * 1024 * 1024);
EXPECT_NOT_NULL(first);
EXPECT_NOT_NULL(second);
free(first);
free(second);
protocol_session_unbind();
}
/* A non-positive session io timeout disables the deadline: the getter reports 0
* (not the built-in 60 s fallback) so callers know to wait indefinitely. */
static void test_protocol_get_io_timeout_zero_disables() {
ProtocolSession session;
protocol_session_init(&session, -1, -1);
protocol_session_bind(&session);
protocol_session_set_io_timeout(&session, 0);
EXPECT_EQ_INT(protocol_get_io_timeout_sec(), 0);
protocol_session_set_io_timeout(&session, 45);
EXPECT_EQ_INT(protocol_get_io_timeout_sec(), 45);
protocol_session_unbind();
}
static void test_max_alloc_is_bound_in_worker_threads() {
enum { WORKER_COUNT = 4 };
ProtocolSession sessions[WORKER_COUNT];
@@ -493,10 +522,19 @@ static void test_data_create_starts_uncharged_and_unowned() {
data_destroy(reserved);
}
/* The server floors a client --timeout=0 at SERVER_IO_TIMEOUT_SEC so a silent
* peer can never hold a session slot forever (slow-loris). */
static void test_protocol_server_io_timeout_floor() {
EXPECT_EQ_INT(protocol_server_io_timeout_sec(0), SERVER_IO_TIMEOUT_SEC);
EXPECT_EQ_INT(protocol_server_io_timeout_sec(-7), SERVER_IO_TIMEOUT_SEC);
EXPECT_EQ_INT(protocol_server_io_timeout_sec(30), 30);
EXPECT_TRUE(SERVER_IO_TIMEOUT_SEC > 0);
}
static void test_protocol_session_io_timeout() {
/* Default is the built-in 60 s window; the setter stores exactly what it is
* given (<= 0 means "fall back to the default") so callers can propagate
* --timeout without special-casing 0. */
/* The default is the built-in 60 s window; the setter stores exactly what it
* is given (<= 0 disables the deadline, matching rsync's --timeout=0) so
* callers can propagate --timeout without special-casing 0. */
ProtocolSession session;
protocol_session_init(&session, -1, -1);
EXPECT_EQ_INT(session.io_timeout_sec, 60);
@@ -641,6 +679,7 @@ void test_protocol() {
test_send_receive_int();
test_send_receive_status();
test_protocol_session_io_timeout();
test_protocol_server_io_timeout_floor();
test_send_receive_status_timed();
test_receive_status_keepalive_skips_reply();
test_receive_status_keepalive_aborts();
@@ -650,6 +689,8 @@ void test_protocol() {
test_max_alloc_rejects_single_buffer();
test_explicit_session_max_alloc_cannot_be_bypassed();
test_max_alloc_allows_configured_buffer();
test_max_alloc_zero_means_unlimited();
test_protocol_get_io_timeout_zero_disables();
test_max_alloc_is_bound_in_worker_threads();
test_protocol_accounting_is_released_in_worker_threads();
test_protocol_accounting_reservation_is_atomic();
+34 -3
View File
@@ -644,17 +644,19 @@ static void test_scanner_one_file_system_cross_device() {
EXPECT_EQ_INT(seq_off_rc, 0);
EXPECT_TRUE(seq_off_found);
EXPECT_EQ_INT(seq_off_total, 2);
/* Sequential: with -x the cross-device subtree is dropped, keep.txt remains. */
/* Sequential: with -x the cross-device subtree is not descended into, but
* rsync-compatible behavior still emits the mount-point directory entry as an
* empty directory File, so keep.txt plus that entry are present. */
EXPECT_EQ_INT(seq_on_rc, 0);
EXPECT_FALSE(seq_on_found);
EXPECT_EQ_INT(seq_on_total, 1);
EXPECT_EQ_INT(seq_on_total, 2);
/* Parallel: same behavior, worker path (depth > 1). */
EXPECT_EQ_INT(par_off_rc, 0);
EXPECT_TRUE(par_off_found);
EXPECT_EQ_INT(par_off_total, 2);
EXPECT_EQ_INT(par_on_rc, 0);
EXPECT_FALSE(par_on_found);
EXPECT_EQ_INT(par_on_total, 1);
EXPECT_EQ_INT(par_on_total, 2);
}
/* Collect emitted file paths (relative to `root`) from a sequential scan.
@@ -884,6 +886,35 @@ static void test_filter_rules(bool parallel) {
free_paths(paths, count);
filter_rule_list_free(base);
/* The common include idiom (the exact rule order the CLI compiles from
* --include='*.txt' --exclude='*'): only .txt files survive. */
const char* idiom[] = {"+ *.txt", "- *"};
base = filter_base_build(idiom, 2, false, err, sizeof(err));
EXPECT_NOT_NULL(base);
options.base_filters = base;
rc = parallel ? collect_files_parallel(root, &options, &paths, &count)
: collect_files(root, &options, &paths, &count);
EXPECT_EQ_INT(rc, 0);
EXPECT_EQ_INT(count, 2);
EXPECT_TRUE(has_path(paths, count, "a.txt"));
EXPECT_TRUE(has_path(paths, count, "c.txt"));
EXPECT_FALSE(has_path(paths, count, "b.tmp"));
free_paths(paths, count);
filter_rule_list_free(base);
/* An include rule alone is NOT a mandatory whitelist (rsync semantics): only
* the matching file is affected, everything else is still transferred. */
const char* include_alone[] = {"+ *.txt"};
base = filter_base_build(include_alone, 1, false, err, sizeof(err));
EXPECT_NOT_NULL(base);
options.base_filters = base;
rc = parallel ? collect_files_parallel(root, &options, &paths, &count)
: collect_files(root, &options, &paths, &count);
EXPECT_EQ_INT(rc, 0);
EXPECT_EQ_INT(count, 3);
free_paths(paths, count);
filter_rule_list_free(base);
unlink("test_scan_filter/a.txt");
unlink("test_scan_filter/b.tmp");
unlink("test_scan_filter/c.txt");
+37 -6
View File
@@ -582,6 +582,7 @@ static void test_late_manifest_abort_frees_keepset() {
EXPECT_TRUE(send_str(p[1], "keep.txt"));
EXPECT_TRUE(send_int(p[1], 0)); /* protected-prefix section is empty */
EXPECT_TRUE(send_int(p[1], 0)); /* missing-args section is empty */
EXPECT_TRUE(send_int(p[1], 0)); /* synchronized-directories section is empty */
EXPECT_TRUE(send_status(p[1], STATUS_ABORT));
DeleteManifest* pending = NULL;
@@ -606,6 +607,7 @@ static void test_late_manifest_eof_frees_keepset() {
EXPECT_TRUE(send_str(p[1], "keep.txt"));
EXPECT_TRUE(send_int(p[1], 0)); /* protected-prefix section is empty */
EXPECT_TRUE(send_int(p[1], 0)); /* missing-args section is empty */
EXPECT_TRUE(send_int(p[1], 0)); /* synchronized-directories section is empty */
shutdown(p[1], SHUT_WR);
DeleteManifest* pending = NULL;
@@ -630,11 +632,13 @@ static void test_late_second_manifest_frees_both() {
EXPECT_TRUE(send_str(p[1], "first.txt"));
EXPECT_TRUE(send_int(p[1], 0)); /* protected-prefix section is empty */
EXPECT_TRUE(send_int(p[1], 0)); /* missing-args section is empty */
EXPECT_TRUE(send_int(p[1], 0)); /* synchronized-directories section is empty */
EXPECT_TRUE(send_status(p[1], STATUS_MANIFEST));
EXPECT_TRUE(send_int(p[1], 1));
EXPECT_TRUE(send_str(p[1], "second.txt"));
EXPECT_TRUE(send_int(p[1], 0)); /* protected-prefix section is empty */
EXPECT_TRUE(send_int(p[1], 0)); /* missing-args section is empty */
EXPECT_TRUE(send_int(p[1], 0)); /* synchronized-directories section is empty */
DeleteManifest* pending = NULL;
EXPECT_EQ_INT(run_pending_receiver(cfg, p[0], &pending), -1);
@@ -645,9 +649,10 @@ static void test_late_second_manifest_frees_both() {
config_delete(cfg);
}
/* A delete-manifest frame with a third (missing-args) section round-trips: the
receiver keeps all three sections and the missing paths are confined exactly
like the keep-set (a traversal entry in the missing section is rejected).
/* A delete-manifest frame with all four sections round-trips: the receiver
keeps the keep-set, protected prefixes, missing-args paths and synchronized
directories, and every section is confined exactly like the keep-set (a
traversal entry in the missing section is rejected).
receive_manifest_entries() reads the counts directly (the leading
STATUS_MANIFEST code is consumed by the caller, so these frames do not send
it). */
@@ -666,6 +671,9 @@ static void test_receive_manifest_three_sections() {
EXPECT_TRUE(send_int(p[1], 2));
EXPECT_TRUE(send_str(p[1], "gone.txt"));
EXPECT_TRUE(send_str(p[1], "dir/gone.bin"));
EXPECT_TRUE(send_int(p[1], 2));
EXPECT_TRUE(send_str(p[1], "."));
EXPECT_TRUE(send_str(p[1], "dir"));
DeleteManifest* manifest = receive_manifest_entries(p[0]);
EXPECT_NOT_NULL(manifest);
@@ -676,9 +684,12 @@ static void test_receive_manifest_three_sections() {
EXPECT_EQ_INT(manifest->missing->size, 2);
EXPECT_EQ_STR((char*)manifest->missing->items[0], "gone.txt");
EXPECT_EQ_STR((char*)manifest->missing->items[1], "dir/gone.bin");
EXPECT_EQ_INT(manifest->dirs->size, 2);
EXPECT_EQ_STR((char*)manifest->dirs->items[0], ".");
EXPECT_EQ_STR((char*)manifest->dirs->items[1], "dir");
delete_manifest_free(manifest);
/* A traversal entry in the third section is rejected like every other. */
/* A traversal entry in the missing section is rejected like every other. */
EXPECT_TRUE(send_int(p[1], 0));
EXPECT_TRUE(send_int(p[1], 0));
EXPECT_TRUE(send_int(p[1], 1));
@@ -780,6 +791,7 @@ static void test_receiver_pending_commits_missing_args() {
EXPECT_TRUE(send_int(p[1], 2));
EXPECT_TRUE(send_str(p[1], "gone.txt"));
EXPECT_TRUE(send_str(p[1], "never_here.txt"));
EXPECT_TRUE(send_int(p[1], 0)); /* no synchronized directories */
EXPECT_TRUE(send_status(p[1], STATUS_FINISHED));
/* NULL pending: the single-threaded commit path deletes at FINISHED. The
@@ -818,10 +830,24 @@ static void test_special_socket_path_log_escaped() {
set_log_level(LOG_LEVEL_WARNING);
log_set_8_bit_output(false);
const char* root = "test_special_sock_escape_root";
const char* existing = "test_special_sock_escape_root/evil\npath";
unlink(existing);
rmdir(root);
EXPECT_EQ_INT(mkdir(root, 0700), 0);
FILE* planted = fopen(existing, "wb");
EXPECT_NOT_NULL(planted);
fclose(planted);
FILE* capture = tmpfile();
EXPECT_NOT_NULL(capture);
log_set_file(capture);
Config* cfg = config_create();
EXPECT_NOT_NULL(cfg);
cfg->preserve_specials = true;
cfg->use_metadata = true;
File* file = file_create("evil\npath");
EXPECT_NOT_NULL(file);
file->is_special = true;
@@ -829,7 +855,9 @@ static void test_special_socket_path_log_escaped() {
EXPECT_NOT_NULL(file->metadata);
file->metadata->mode = S_IFSOCK | 0644;
FileSaveResult result = file_save_to_disk_full("/tmp/dst", file, NULL);
/* A non-matching entry already occupies the path: the socket creation is
refused and the warning must escape the path's control byte. */
FileSaveResult result = file_save_to_disk_full(root, file, cfg);
EXPECT_EQ_INT(result, FILE_SAVE_SKIPPED);
fflush(capture);
@@ -841,8 +869,11 @@ static void test_special_socket_path_log_escaped() {
log_set_file(NULL);
fclose(capture);
file_destroy(file);
config_delete(cfg);
unlink(existing);
rmdir(root);
EXPECT_NOT_NULL(strstr(output, "socket not recreated: evil\\#012path"));
EXPECT_NOT_NULL(strstr(output, "refusing to replace existing entry with socket: evil\\#012path"));
}
/* B1: a client-planted FIFO at the destination must not block the receiver's
+26
View File
@@ -234,6 +234,31 @@ static void test_server_cli_password_requires_daemon() {
server_cli_options_free(&opts);
}
/* --port is the rsync-style alias for -p, in both the separate and =value
* spellings; an invalid value is still validated. */
static void test_server_cli_port_alias() {
const char* a1[] = {"fastsync-server", "--port", "9000"};
ServerCliOptions opts;
EXPECT_EQ_INT(parse_ok(a1, 3, &opts), 0);
EXPECT_EQ_INT(opts.port, 9000);
EXPECT_TRUE(opts.port_set);
server_cli_options_free(&opts);
const char* a2[] = {"fastsync-server", "--port=9001"};
ServerCliOptions opts2;
EXPECT_EQ_INT(parse_ok(a2, 2, &opts2), 0);
EXPECT_EQ_INT(opts2.port, 9001);
EXPECT_TRUE(opts2.port_set);
server_cli_options_free(&opts2);
char err[128];
ServerCliOptions opts3;
const char* a3[] = {"fastsync-server", "--port", "notaport"};
EXPECT_EQ_INT(server_cli_parse(3, (char**)a3, &opts3, err, sizeof(err)), -1);
EXPECT_TRUE(strstr(err, "invalid port") != NULL);
server_cli_options_free(&opts3);
}
static void test_server_cli_help() {
char err[256];
const char* a1[] = {"s", "--help"};
@@ -254,5 +279,6 @@ void test_server_cli() {
test_server_cli_password_requires_daemon();
test_server_cli_no_super();
test_server_cli_allow_super();
test_server_cli_port_alias();
test_server_cli_help();
}
+91 -59
View File
@@ -136,7 +136,8 @@ static void test_walker_removes_extras_keeps_manifest_and_protected() {
EXPECT_NOT_NULL(manifest);
DeleteSkipEntry skip = {"prot", false};
size_t deleted = 0;
DeleteWalkResult result = delete_extras_limited(root, manifest, 100000, &skip, 1, &deleted);
DeleteWalkResult result =
delete_extras_limited(root, manifest, NULL, 100000, &skip, 1, &deleted, NULL);
EXPECT_EQ_INT((int)result, (int)DELETE_WALK_OK);
EXPECT_FALSE(file_exists(root, "a.txt"));
EXPECT_TRUE(file_exists(root, "keep.txt"));
@@ -170,7 +171,8 @@ static void test_walker_keeps_nested_manifest_dirs() {
ArrayList* manifest = make_manifest_strings(keeps, 3);
EXPECT_NOT_NULL(manifest);
size_t deleted = 0;
DeleteWalkResult result = delete_extras_limited(root, manifest, 100000, NULL, 0, &deleted);
DeleteWalkResult result =
delete_extras_limited(root, manifest, NULL, 100000, NULL, 0, &deleted, NULL);
EXPECT_EQ_INT((int)result, (int)DELETE_WALK_OK);
EXPECT_FALSE(file_exists(root, "extra.txt"));
EXPECT_TRUE(file_exists(root, "keepdir/deep/keep.txt"));
@@ -187,7 +189,9 @@ static void test_walker_keeps_nested_manifest_dirs() {
free(root);
}
static void test_walker_max_delete_exceeded_deletes_nothing() {
/* --max-delete is a partial cap (rsync parity): delete up to the limit, skip
the rest, and report DELETE_WALK_LIMIT_REACHED. */
static void test_walker_max_delete_partial_deletes_up_to_cap() {
char* root = make_walk_root("maxdel");
EXPECT_NOT_NULL(root);
EXPECT_TRUE(write_file_at(root, "a.txt", "extra"));
@@ -197,12 +201,15 @@ static void test_walker_max_delete_exceeded_deletes_nothing() {
ArrayList* manifest = make_manifest_strings(keeps, 0);
EXPECT_NOT_NULL(manifest);
size_t deleted = 999;
DeleteWalkResult result = delete_extras_limited(root, manifest, 2, NULL, 0, &deleted);
EXPECT_EQ_INT((int)result, (int)DELETE_WALK_LIMIT_EXCEEDED);
EXPECT_EQ_INT((int)deleted, 0);
EXPECT_TRUE(file_exists(root, "a.txt"));
EXPECT_TRUE(file_exists(root, "b.txt"));
EXPECT_TRUE(file_exists(root, "c.txt"));
size_t skipped = 0;
DeleteWalkResult result =
delete_extras_limited(root, manifest, NULL, 2, NULL, 0, &deleted, &skipped);
EXPECT_EQ_INT((int)result, (int)DELETE_WALK_LIMIT_REACHED);
EXPECT_EQ_INT((int)deleted, 2);
EXPECT_EQ_INT((int)skipped, 1);
int remaining = (file_exists(root, "a.txt") ? 1 : 0) + (file_exists(root, "b.txt") ? 1 : 0) +
(file_exists(root, "c.txt") ? 1 : 0);
EXPECT_EQ_INT(remaining, 1);
array_list_delete(manifest);
remove_walk_tree(root);
free(root);
@@ -217,7 +224,7 @@ static void test_walker_max_delete_exact_bound_deletes() {
ArrayList* manifest = make_manifest_strings(keeps, 0);
EXPECT_NOT_NULL(manifest);
size_t deleted = 0;
DeleteWalkResult result = delete_extras_limited(root, manifest, 2, NULL, 0, &deleted);
DeleteWalkResult result = delete_extras_limited(root, manifest, NULL, 2, NULL, 0, &deleted, NULL);
EXPECT_EQ_INT((int)result, (int)DELETE_WALK_OK);
EXPECT_EQ_INT((int)deleted, 2);
EXPECT_FALSE(file_exists(root, "a.txt"));
@@ -227,6 +234,77 @@ static void test_walker_max_delete_exact_bound_deletes() {
free(root);
}
/* Extraneous destination symlinks (including one pointing at a directory) must
be unlinked, never followed, so their targets survive. */
static void test_walker_removes_extraneous_symlinks() {
char* root = make_walk_root("symlink");
char* outside = make_walk_root("symlink_out");
EXPECT_NOT_NULL(root);
EXPECT_NOT_NULL(outside);
EXPECT_TRUE(write_file_at(outside, "secret.txt", "keep"));
EXPECT_TRUE(write_file_at(root, "keep.txt", "kept"));
char* link_file = path_cat(root, "link_file");
char* link_dir = path_cat(root, "link_dir");
char* link_broken = path_cat(root, "link_broken");
EXPECT_NOT_NULL(link_file);
EXPECT_NOT_NULL(link_dir);
EXPECT_NOT_NULL(link_broken);
EXPECT_EQ_INT(symlink("keep.txt", link_file), 0);
EXPECT_EQ_INT(symlink(outside, link_dir), 0);
EXPECT_EQ_INT(symlink("/nonexistent-target", link_broken), 0);
const char* keeps[] = {"keep.txt"};
ArrayList* manifest = make_manifest_strings(keeps, 1);
EXPECT_NOT_NULL(manifest);
size_t deleted = 0;
DeleteWalkResult result =
delete_extras_limited(root, manifest, NULL, 100000, NULL, 0, &deleted, NULL);
EXPECT_EQ_INT((int)result, (int)DELETE_WALK_OK);
EXPECT_FALSE(file_exists(root, "link_file"));
EXPECT_FALSE(file_exists(root, "link_dir"));
EXPECT_FALSE(file_exists(root, "link_broken"));
EXPECT_TRUE(file_exists(root, "keep.txt"));
EXPECT_TRUE(file_exists(outside, "secret.txt"));
free(link_file);
free(link_dir);
free(link_broken);
array_list_delete(manifest);
remove_walk_tree(root);
remove_walk_tree(outside);
free(root);
free(outside);
}
/* With a synchronized-dir set, extras outside it survive while extras directly
inside a listed directory are removed; the receive root is the "." sentinel. */
static void test_walker_confines_deletion_to_synced_dirs() {
char* root = make_walk_root("synced");
EXPECT_NOT_NULL(root);
EXPECT_TRUE(write_file_at(root, "rootextra.txt", "keep"));
EXPECT_EQ_INT(make_subdir(root, "inscope"), 0);
EXPECT_TRUE(write_file_at(root, "inscope/extra.txt", "delete"));
EXPECT_TRUE(write_file_at(root, "inscope/keep.txt", "kept"));
EXPECT_EQ_INT(make_subdir(root, "outscope"), 0);
EXPECT_TRUE(write_file_at(root, "outscope/extra.txt", "keep"));
const char* keeps[] = {"inscope/keep.txt"};
ArrayList* manifest = make_manifest_strings(keeps, 1);
ArrayList* dirs = array_list_create(free);
EXPECT_NOT_NULL(manifest);
EXPECT_NOT_NULL(dirs);
EXPECT_TRUE(array_list_add(dirs, str_dup("inscope")));
size_t deleted = 0;
DeleteWalkResult result =
delete_extras_limited(root, manifest, dirs, 100000, NULL, 0, &deleted, NULL);
EXPECT_EQ_INT((int)result, (int)DELETE_WALK_OK);
EXPECT_TRUE(file_exists(root, "rootextra.txt"));
EXPECT_FALSE(file_exists(root, "inscope/extra.txt"));
EXPECT_TRUE(file_exists(root, "inscope/keep.txt"));
EXPECT_TRUE(file_exists(root, "outscope/extra.txt"));
array_list_delete(manifest);
array_list_delete(dirs);
remove_walk_tree(root);
free(root);
}
static void test_walker_unlimited_deletes_all() {
char* root = make_walk_root("unlim");
EXPECT_NOT_NULL(root);
@@ -245,53 +323,6 @@ static void test_walker_unlimited_deletes_all() {
free(root);
}
/* The 100000-entry server hard bound (MAX_SERVER_DELETE_COUNT, which this test
exercises through a literal to avoid reaching into file_receive.c) is also
all-or-nothing: a destination holding more extras than the bound must be left
completely untouched. Skipped under valgrind: 100k file creations would be
far too slow under instrumentation. */
static void test_walker_hard_bound_all_or_nothing() {
if (is_running_under_valgrind())
return;
enum { HARD_BOUND = 100000 };
char* root = make_walk_root("hardbound");
EXPECT_NOT_NULL(root);
int rootfd = open(root, O_RDONLY | O_DIRECTORY | O_CLOEXEC);
EXPECT_TRUE(rootfd >= 0);
bool created = true;
for (int i = 0; created && i < HARD_BOUND + 1; i++) {
char name[32];
snprintf(name, sizeof(name), "f%d", i);
int fd = openat(rootfd, name, O_WRONLY | O_CREAT | O_TRUNC, 0644);
if (fd < 0)
created = false;
else
close(fd);
}
EXPECT_TRUE(created);
const char* keeps[1] = {NULL};
ArrayList* manifest = make_manifest_strings(keeps, 0);
EXPECT_NOT_NULL(manifest);
size_t deleted = 999;
DeleteWalkResult result = delete_extras_limited(root, manifest, HARD_BOUND, NULL, 0, &deleted);
EXPECT_EQ_INT((int)result, (int)DELETE_WALK_LIMIT_EXCEEDED);
EXPECT_EQ_INT((int)deleted, 0);
EXPECT_TRUE(file_exists(root, "f0"));
EXPECT_TRUE(file_exists(root, "f100000"));
array_list_delete(manifest);
/* Fast cleanup: unlink every created name through the still-open root fd. */
if (rootfd >= 0) {
for (int i = 0; i < HARD_BOUND + 1; i++) {
char name[32];
snprintf(name, sizeof(name), "f%d", i);
(void)unlinkat(rootfd, name, 0);
}
close(rootfd);
}
rmdir(root);
free(root);
}
typedef struct {
bool eight_bit_output;
const char* expected;
@@ -553,10 +584,11 @@ void test_shared_utils() {
test_getdelim_bounded();
test_walker_removes_extras_keeps_manifest_and_protected();
test_walker_keeps_nested_manifest_dirs();
test_walker_max_delete_exceeded_deletes_nothing();
test_walker_max_delete_partial_deletes_up_to_cap();
test_walker_max_delete_exact_bound_deletes();
test_walker_removes_extraneous_symlinks();
test_walker_confines_deletion_to_synced_dirs();
test_walker_unlimited_deletes_all();
test_walker_hard_bound_all_or_nothing();
test_loopback_helpers();
test_fd_peer_ip();
+49 -2
View File
@@ -74,8 +74,6 @@ static void test_stop_at_parse_now_plus() {
static void test_stop_at_parse_invalid() {
time_t now = 1700000000;
time_t deadline = 0;
EXPECT_FALSE(stop_parse_at_time("12", now, &deadline));
EXPECT_FALSE(stop_parse_at_time("12:3", now, &deadline));
EXPECT_FALSE(stop_parse_at_time("1234", now, &deadline));
EXPECT_FALSE(stop_parse_at_time("12:30:5", now, &deadline));
EXPECT_FALSE(stop_parse_at_time("12:30:5x", now, &deadline));
@@ -100,6 +98,54 @@ static void test_stop_at_parse_invalid() {
EXPECT_FALSE(stop_parse_at_time(NULL, now, &deadline));
}
/* rsync's flexible date form for --stop-at (y-m-dTh:m, with / separators and
* abbreviable fields). */
static void test_stop_at_parse_date_forms() {
time_t now = 1700000000;
time_t deadline = 0;
struct tm t;
EXPECT_TRUE(stop_parse_at_time("2030-12-31T23:59", now, &deadline));
EXPECT_NOT_NULL(localtime_r(&deadline, &t));
EXPECT_EQ_INT(t.tm_year + 1900, 2030);
EXPECT_EQ_INT(t.tm_mon + 1, 12);
EXPECT_EQ_INT(t.tm_mday, 31);
EXPECT_EQ_INT(t.tm_hour, 23);
EXPECT_EQ_INT(t.tm_min, 59);
EXPECT_TRUE(stop_parse_at_time("2030/12/31T23:59", now, &deadline));
EXPECT_NOT_NULL(localtime_r(&deadline, &t));
EXPECT_EQ_INT(t.tm_year + 1900, 2030);
EXPECT_EQ_INT(t.tm_mon + 1, 12);
EXPECT_EQ_INT(t.tm_mday, 31);
EXPECT_TRUE(stop_parse_at_time("2030-12-31", now, &deadline));
EXPECT_NOT_NULL(localtime_r(&deadline, &t));
EXPECT_EQ_INT(t.tm_year + 1900, 2030);
EXPECT_EQ_INT(t.tm_hour, 0);
EXPECT_EQ_INT(t.tm_min, 0);
/* Partial forms resolve to the next matching point in the future. */
EXPECT_TRUE(stop_parse_at_time(":59", now, &deadline));
EXPECT_TRUE(deadline > now);
EXPECT_NOT_NULL(localtime_r(&deadline, &t));
EXPECT_EQ_INT(t.tm_min, 59);
EXPECT_TRUE(stop_parse_at_time("1-30", now, &deadline));
EXPECT_TRUE(deadline > now);
EXPECT_NOT_NULL(localtime_r(&deadline, &t));
EXPECT_EQ_INT(t.tm_mon + 1, 1);
EXPECT_EQ_INT(t.tm_mday, 30);
EXPECT_TRUE(stop_parse_at_time("1", now, &deadline));
EXPECT_TRUE(deadline > now);
EXPECT_NOT_NULL(localtime_r(&deadline, &t));
EXPECT_EQ_INT(t.tm_mday, 1);
/* Seconds are not part of rsync's date form. */
EXPECT_FALSE(stop_parse_at_time("2030-12-31T23:59:59", now, &deadline));
}
static void test_stop_deadline_latency() {
struct timespec now;
EXPECT_EQ_INT(clock_gettime(CLOCK_MONOTONIC, &now), 0);
@@ -145,6 +191,7 @@ void test_stop(void) {
test_stop_after_parse_invalid();
test_stop_at_parse_hhmm();
test_stop_at_parse_now_plus();
test_stop_at_parse_date_forms();
test_stop_at_parse_invalid();
test_stop_deadline_latency();
}
+14 -7
View File
@@ -144,14 +144,21 @@ static void test_client_delete_null() {
client_delete(c);
}
/* Test tcp_set_timeouts with valid values */
/* Test tcp_set_timeouts: a non-positive value disables the timeout (rsync's
* --timeout=0 / --contimeout=0), it is not a "leave unchanged" sentinel. */
static void test_tcp_set_timeouts() {
/* Just verify the function doesn't crash with edge cases */
tcp_set_timeouts(0, 0); /* zero means "don't change" */
tcp_set_timeouts(60, 20); /* normal values */
tcp_set_timeouts(-1, -1); /* negative means "don't change" */
/* If we got here without crashing, the test passes */
EXPECT_TRUE(true);
tcp_set_timeouts(0, 0);
EXPECT_EQ_INT(tcp_get_timeout_sec(), 0);
EXPECT_EQ_INT(tcp_get_contimeout_sec(), 0);
tcp_set_timeouts(60, 20);
EXPECT_EQ_INT(tcp_get_timeout_sec(), 60);
EXPECT_EQ_INT(tcp_get_contimeout_sec(), 20);
tcp_set_timeouts(-1, -1);
EXPECT_EQ_INT(tcp_get_timeout_sec(), 0);
EXPECT_EQ_INT(tcp_get_contimeout_sec(), 0);
/* Restore finite defaults so later tests that rely on a bounded connect/IO
* timeout (e.g. connecting to a non-routable address) cannot block forever. */
tcp_set_timeouts(30, 10);
}
/* Test client_connect with an invalid host (should fail gracefully) */
+128 -64
View File
@@ -246,7 +246,8 @@ static void test_link_copy_fallback_preserves_xattrs() {
m.atime_valid = false;
m.crtime_valid = false;
bool ok = file_to_disk_secure_link_attrs(dest, basis_dir, "payload", 7, false, &m, false, false,
bool ok = file_to_disk_secure_link_attrs(dest, basis_dir, "payload", 7, false, &m,
(FileAttrPolicy){true, true, false, false}, false,
xattrs, true, NULL);
xattr_list_free(xattrs);
EXPECT_TRUE(ok);
@@ -367,21 +368,21 @@ static void test_fake_super_restore() {
}
/* No xattr present yet: restore is a silent no-op (returns false, no crash). */
EXPECT_FALSE(fake_super_restore_fd(fd));
FileAttrPolicy policy = {true, true, false, false};
EXPECT_FALSE(fake_super_restore_fd(fd, policy));
fake_super_store_fd(fd, 1001, 1002, 0751, 1700000000, 123456789);
EXPECT_TRUE(fake_super_restore_fd(fd));
EXPECT_TRUE(fake_super_restore_fd(fd, policy));
struct stat st;
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)(st.st_mode & 07777), 0751);
/* Mode sanitization: the normal metadata path never grants group/other write
bits, and fake-super replay must not re-add them (a recorded 0666 restores
as 0644, never as world-writable). */
/* Strict rsync parity: -p restores the recorded mode exactly, including
group/other write (a recorded 0666 restores as 0666). */
fake_super_store_fd(fd, 1001, 1002, 0666, 1700000000, 0);
EXPECT_TRUE(fake_super_restore_fd(fd));
EXPECT_TRUE(fake_super_restore_fd(fd, policy));
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0644);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0666);
/* Restore with a malformed record must skip without failing. */
time_t before = st.st_mtime;
@@ -390,7 +391,7 @@ static void test_fake_super_restore() {
EXPECT_EQ_INT((int)fsetxattr(wfd, FAKESUPER_XATTR, "not-a-valid-record", 19, 0), 0);
close(wfd);
}
EXPECT_FALSE(fake_super_restore_fd(fd));
EXPECT_FALSE(fake_super_restore_fd(fd, policy));
fstat(fd, &st);
EXPECT_EQ_INT((int)st.st_mtime, (int)before);
@@ -398,14 +399,12 @@ static void test_fake_super_restore() {
unlink(path);
}
/* --fake-super owner replay must honor the super gate and copy-as authority:
--no-super suppresses the recorded-source-owner chown even for root, and an
active --copy-as keeps its forced owner (the recorded source owner must never
override it). Root-gated: only root can observe a chown actually landing. */
static void test_fake_super_owner_gate() {
if (geteuid() != 0)
return; /* non-root cannot observe ownership changes; skip silently */
const char* path = "test_fake_super_owner_gate.txt";
/* --fake-super must NEVER perform a real chown: fake_super_restore_fd applies
* only mode/mtime and leaves the entry's uid/gid exactly as they were, even
* when an explicit ownership policy is active and super_mode permits it. This
* is observable unprivileged (the file's owner is simply unchanged). */
static void test_fake_super_no_real_chown() {
const char* path = "test_fake_super_nochown.txt";
unlink(path);
int fd = open(path, O_WRONLY | O_CREAT | O_TRUNC, 0600);
if (fd < 0)
@@ -414,62 +413,34 @@ static void test_fake_super_owner_gate() {
if (has_xattr)
removexattr(path, "user.fastsync.xprobe");
if (!has_xattr) {
close(fd);
unlink(path);
return; /* filesystem without xattr support */
}
if (fchown(fd, 0, 0) != 0) {
close(fd);
unlink(path);
return;
}
struct stat before;
EXPECT_EQ_INT(fstat(fd, &before), 0);
fake_super_store_fd(fd, 12345, 12346, 0755, 1700000000, 0);
Config* c = config_create();
FileAttrPolicy policy = {true, true, false, false};
EXPECT_NOT_NULL(c);
/* An explicit ownership policy is required before fake-super replay may
chown; --fake-super alone only records the source owner (A2). */
c->numeric_ids = true;
/* --no-super: the owner leg is skipped even as root. */
c->super_mode = SUPER_MODE_OFF;
EXPECT_TRUE(identity_set_active(c));
EXPECT_TRUE(fake_super_restore_fd(fd));
struct stat st;
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)st.st_uid, 0);
EXPECT_EQ_INT((int)st.st_gid, 0);
/* AUTO with an identity policy: the recorded source owner is applied. */
c->super_mode = SUPER_MODE_AUTO;
EXPECT_TRUE(identity_set_active(c));
EXPECT_TRUE(fake_super_restore_fd(fd));
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)st.st_uid, 12345);
EXPECT_EQ_INT((int)st.st_gid, 12346);
/* --super / --fake-super with NO explicit identity flag must NOT apply a
client-chosen owner: super_mode alone never enables ownership. */
EXPECT_EQ_INT(fchown(fd, 0, 0), 0);
c->numeric_ids = false;
/* The strongest ownership request available plus permitted super mode. */
c->preserve_owner = true;
c->preserve_group = true;
c->chown_uid_set = true;
c->chown_uid = 12345;
c->chown_gid_set = true;
c->chown_gid = 12346;
c->super_mode = SUPER_MODE_ON;
c->fake_super = true;
EXPECT_TRUE(identity_set_active(c));
EXPECT_TRUE(fake_super_restore_fd(fd));
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)st.st_uid, 0);
EXPECT_EQ_INT((int)st.st_gid, 0);
/* Active --copy-as is authoritative: the recorded source owner must not
override it, even with AUTO/ON. */
c->copy_as_set = true;
c->copy_as_uid = 777;
c->copy_as_gid = 778;
EXPECT_TRUE(identity_set_active(c));
EXPECT_TRUE(fake_super_restore_fd(fd));
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)st.st_uid, 0);
EXPECT_EQ_INT((int)st.st_gid, 0);
EXPECT_TRUE(fake_super_restore_fd(fd, policy));
struct stat after;
EXPECT_EQ_INT(fstat(fd, &after), 0);
EXPECT_EQ_INT((int)after.st_uid, (int)before.st_uid);
EXPECT_EQ_INT((int)after.st_gid, (int)before.st_gid);
/* Mode is still replayed (policy-gated). */
EXPECT_EQ_INT((int)(after.st_mode & 0777), 0755);
identity_clear_active();
config_delete(c);
@@ -477,7 +448,99 @@ static void test_fake_super_owner_gate() {
unlink(path);
}
/* identity_resolve_storage_ids() is what --fake-super RECORDS: the resolved
* mapping for a requested side, and the source's own id for a side never
* requested. Also pins the #286 rule that --numeric-ids alone never activates
* ownership (it is only a mapping modifier). */
static void test_fake_super_storage_resolution() {
uint32_t uid = 0, gid = 0;
/* --numeric-ids alone is INERT: no ownership request, storage unchanged. */
Config* c = config_create();
EXPECT_NOT_NULL(c);
c->numeric_ids = true;
EXPECT_TRUE(identity_set_active(c));
EXPECT_FALSE(identity_active_enabled());
EXPECT_FALSE(identity_owner_requested());
EXPECT_FALSE(identity_group_requested());
identity_resolve_storage_ids(12345, 6789, &uid, &gid);
EXPECT_EQ_INT((int)uid, 12345);
EXPECT_EQ_INT((int)gid, 6789);
config_delete(c);
/* --fake-super with no ownership request records the raw source ids. */
c = config_create();
EXPECT_NOT_NULL(c);
c->fake_super = true;
EXPECT_TRUE(identity_set_active(c));
identity_resolve_storage_ids(12345, 6789, &uid, &gid);
EXPECT_EQ_INT((int)uid, 12345);
EXPECT_EQ_INT((int)gid, 6789);
/* -o + --numeric-ids: raw owner, un-requested group stays the source gid. */
c->preserve_owner = true;
c->numeric_ids = true;
EXPECT_TRUE(identity_set_active(c));
identity_resolve_storage_ids(12345, 6789, &uid, &gid);
EXPECT_EQ_INT((int)uid, 12345);
EXPECT_EQ_INT((int)gid, 6789);
/* --chown overrides both sides. */
c->chown_uid_set = true;
c->chown_uid = 777;
c->chown_gid_set = true;
c->chown_gid = 778;
EXPECT_TRUE(identity_set_active(c));
identity_resolve_storage_ids(12345, 6789, &uid, &gid);
EXPECT_EQ_INT((int)uid, 777);
EXPECT_EQ_INT((int)gid, 778);
/* A usermap match beats --chown on the owner side only. */
c->usermap_count = 1;
c->usermap = calloc(1, sizeof(IdentityMap));
EXPECT_NOT_NULL(c->usermap);
c->usermap[0].from = IDENTITY_MATCH_ANY;
c->usermap[0].to = 999;
EXPECT_TRUE(identity_set_active(c));
identity_resolve_storage_ids(12345, 6789, &uid, &gid);
EXPECT_EQ_INT((int)uid, 999);
EXPECT_EQ_INT((int)gid, 778);
/* --copy-as is authoritative for both sides. */
c->copy_as_set = true;
c->copy_as_uid = 111;
c->copy_as_gid = 222;
EXPECT_TRUE(identity_set_active(c));
identity_resolve_storage_ids(12345, 6789, &uid, &gid);
EXPECT_EQ_INT((int)uid, 111);
EXPECT_EQ_INT((int)gid, 222);
identity_clear_active();
config_delete(c);
}
/* xattr_list_clone deep-copies names/values (used by the deferred directory
* metadata accumulator), so the clone stays valid after the original is freed. */
static void test_xattr_list_clone() {
EXPECT_NULL(xattr_list_clone(NULL));
FileXattrList* list = xattr_list_new();
EXPECT_NOT_NULL(list);
EXPECT_TRUE(xattr_list_append(list, "user.a", "1", 1));
EXPECT_TRUE(xattr_list_append(list, "user.b", "22", 2));
FileXattrList* clone = xattr_list_clone(list);
EXPECT_NOT_NULL(clone);
EXPECT_EQ_INT(clone->count, 2);
EXPECT_EQ_STR(clone->items[0].name, "user.a");
EXPECT_EQ_INT((int)clone->items[1].value_len, 2);
EXPECT_TRUE(memcmp(clone->items[1].value, "22", 2) == 0);
EXPECT_TRUE(clone->items[0].name != list->items[0].name);
xattr_list_free(list);
EXPECT_EQ_STR(clone->items[0].name, "user.a");
xattr_list_free(clone);
}
void test_xattr() {
test_xattr_list_clone();
test_xattr_wire_roundtrip();
test_xattr_reject_privileged_namespace();
test_xattr_reject_oversized_value();
@@ -487,5 +550,6 @@ void test_xattr() {
test_xattr_receive_drops_acl_without_preserve_acls();
test_link_copy_fallback_preserves_xattrs();
test_fake_super_restore();
test_fake_super_owner_gate();
test_fake_super_no_real_chown();
test_fake_super_storage_resolution();
}