14 Commits
Author SHA1 Message Date
TapTap 1116da9f64 docs: correct rsync-parity claims and stale facts (#297)
CI / lint (pull_request) Successful in 1m47s
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
CI / build-and-test (pull_request) Successful in 40s
Reclassify every rsync-compatibility row as parity / caveat / divergent
(replacing the misleading 143-OK / 0-divergence summary), and document the
protocol 2.23.0 behavior:

- Split the conflated `-M, --preserve` row: `-M` is `--remote-option`,
  `--preserve` is the FastSync `-p`+`-t` alias.
- Fix `MAX_CONNECTION_MEMORY` (256 MiB, not 1 GB), `--rsync-path`
  (client-only, never crosses the wire), and the `-p` mode behavior
  (strict rsync parity; no masking).
- `--specials` now recreates sockets, so `-D` is real parity; fake-super
  records the resolved owner and replays mode/time (never real-chowns).
- Document short options/clustering, checksum/compression choices, seed
  randomization, timeout/max-alloc defaults, temp-dir confinement + EXDEV,
  identity/map parity, verbatim symlinks, delete scoping, `--max-delete`
  partial + exit 25, `--chmod`, output caveats, and server `--port`.
- Bump version refs to 2.23.0 and add the 2.23.0 CHANGELOG entry.

Docs-only; no source changes.
2026-09-16 01:48:13 +02:00
TapTap 684153350a Merge branch 'fix/parity-chmod' into feat/rsync-parity 2026-09-16 01:24:05 +02:00
TapTap ec206b02d0 chmod: match rsync 3.4.1 --chmod and remove mode masking (#293)
- --chmod no longer implies --preserve-perms; repeated --chmod options
  accumulate, and D/F/X selectors plus s/t special bits are supported with
  rsync's exact parse_chmod/tweak_mode semantics.
- Stop masking group/other write and setuid/setgid/sticky: -p copies the
  source mode exactly, no-p new entries use source&~umask, directories keep
  setgid/sticky, and special nodes follow the same rules.
- Apply ownership before mode on the fd path so a chown cannot clear the
  setuid/setgid bits -p just restored (rsync order).
- Update unit and integration tests, including differential checks against
  rsync 3.4.1.
2026-09-16 01:23:45 +02:00
TapTap 88bdfeeb58 fix(parity): receiver temp-dir confinement, server I/O floor, delete budget
Address review findings on feat/rsync-parity:
- confine --temp-dir below the receive root (reject absolute/.. like
  backup-dir/partial-dir); keep EXDEV non-atomic fallback
- floor server session I/O deadlines at SERVER_IO_TIMEOUT_SEC (60s) and
  install it on the socket layer at startup (slow-loris)
- charge each --delete-missing-args directory removal once and clamp the
  extras-walk remaining budget so it can never underflow past --max-delete
- normalize --compress-choice=auto to zstd client-side and accept it on
  receive so auto transfers no longer fail
- map received --max-alloc=0 to MAX_SERVER_ALLOC (receive path only)
- zero File.dest_state; include log-file-format in report_dest_info;
  add STATUS_DELETE_LIMIT name; recognize --skip-compress as a
  separate-value option; OOM-guard send_list_only root entry; drop the
  dead -M= branch; record the bare relative protected prefix for -R
  size-prunes in both scanners; refresh delete-manifest comment
- pin the rsync tarball sha256 and bump integrator image to v11

Tests: temp-dir rejection/relative/cross-device, server timeout floor,
delete-missing dir budget regression, compress-choice=auto e2e,
max-alloc=0 receive mapping, dest_state, report_dest_info modes,
skip-compress dash value, -M short forms, -R root size-prune mirror
protection (rsync 3.4.1 confirmed).
2026-09-16 01:11:59 +02:00
TapTap 3f5b0250f4 fix: ASan out-of-bounds argv in cli test, cppcheck uninit rate buffer 2026-09-15 23:53:49 +02:00
TapTap c41bfb2cdb test: align trust-sender tests with rsync-parity symlink storage 2026-09-15 23:44:14 +02:00
TapTap 1b2632f968 test: fix merged parity branches (4-section manifest fixtures, timeout-teardown) 2026-09-15 23:38:15 +02:00
TapTap 7dbca70a4b Merge branch 'feat/parity-output' into feat/rsync-parity
# Conflicts:
#	src/shared/config.h
#	src/shared/protocol.h
#	tests/test_config.c
2026-09-15 23:25:34 +02:00
TapTap 1a053f06e5 Merge branch 'feat/parity-ownership' into feat/rsync-parity
# Conflicts:
#	src/shared/config.h
#	tests/test_config.c
2026-09-15 23:24:58 +02:00
TapTap 58b3a33e82 Merge branch 'feat/parity-delete' into feat/rsync-parity 2026-09-15 23:24:24 +02:00
TapTap 376e6500ab fix(delete): count recursive missing-arg removals per entry (#290)
A non-empty --delete-missing-args directory removed under --force/--delete
now has its contents deleted entry-by-entry through the budgeted walker, so
every deleted file/dir counts toward --max-delete exactly like rsync (a
capped run leaves the remaining entries and exits 25).
2026-09-15 23:23:52 +02:00
TapTap 82a1d5e240 fix(delete): match rsync deletion semantics (#290)
- Scope the --delete extras walk to directories synchronized by the
  transfer: add a synchronized-directory section to the delete manifest
  (protocol 2.23.0) so --files-from subsets no longer delete untransmitted
  paths outside listed directory subtrees (data-loss fix).
- Separate --max-size/--min-size prune protection from --delete-excluded so
  size-pruned source mirrors survive (rsync parity).
- Unlink extraneous destination symlinks instead of skipping them.
- Make --max-delete partial (delete up to N, skip the rest) and exit 25;
  accept negative values as unlimited.
- Draw --delete-missing-args deletions from the shared --max-delete budget.
- Honor --force during --delay-updates publication.

Add unit and integration regression tests; update the pinned config wire
golden and version strings for the 2.23.0 manifest/status additions.
2026-09-15 23:12:03 +02:00
TapTap 6144c7fc7f fix(identity): rsync ownership parity for numeric-ids, dirs, maps, fake-super (#286, #294)
- #286: --numeric-ids is a mapping modifier only; it no longer activates
  chown by itself (identity_active_enabled/owner/group predicates), and
  --fake-super stores the resolved mapping instead of real-chowning.
- #286: apply owner/group to directories via the deferred directory
  metadata path; capture+transmit+apply directory xattrs/ACLs (-aX/-aA),
  including default ACLs, in STATUS_MKDIR/STATUS_DIR_TIMES.
- #294: --usermap/--groupmap support inclusive ranges, '*', empty FROM
  (unnamed ids), and receiver-side TO name resolution; --chown mixing with
  a same-side map is rejected like rsync.
- Protocol 2.22.0 -> 2.23.0 (map wire entry gains from_hi + to_name;
  dir frames gain a bounded xattr block).
2026-09-15 22:10:02 +02:00
TapTap 84827ca617 feat(output): rsync 3.4.1 selection and output parity (#291, #292)
#291:
- Compile --exclude/--include/--exclude-from/--include-from into the SAME
  ordered rule list as --filter/-f (first match wins), so the common
  `--include='*.txt' --exclude='*'` idiom and include-alone semantics match
  rsync. The legacy per-kind scanner arrays are no longer applied.
- -x/--one-file-system emits the cross-device mount-point directory entry
  (empty) instead of dropping it, in both the sequential and parallel scanners.
- Stop passing the legacy arrays to the scanner; document -f is --filter.

#292:
- New src/shared/format.c/.h: rsync "big_num" (comma-grouped integers) and
  decimal -h human sizes, %M/%t timestamp, and the STATUS_DEST_INFO codec.
- Receiver answers each STATUS_CHECK with a pre-transfer destination snapshot
  (new report_dest_info wire field + STATUS_DEST_INFO, PROTOCOL_VERSION
  2.23.0) so the sender can render true itemize columns.
- Itemize now emits rsync-correct update/type chars and c/s/t/p/o/g columns
  for files, dirs, symlinks and hard links, comparing size/time/perms/owner/
  group against the reported destination.
- --out-format gains %i %n %f %l %b %M %t %o %p %B %U %G %L; %f is the
  relative display path, %M the YYYY/MM/DD-HH:MM:SS form, %b the literal
  bytes sent.
- --list-only prints transfer-relative names, directory entries and ls-style
  grouped sizes.
- --stats prints rsync's multi-line block on stdout; -h uses decimal units.

Tests: unit tests for the filter ordering, format primitives, itemize
columns; integration + differential tests against real rsync 3.4.1 for
itemize/out-format/list-only/selection and -x. Golden wire len/hash and
protocol version strings updated for 2.23.0.
2026-09-15 21:57:39 +02:00
68 changed files with 5508 additions and 1711 deletions

No files matched your search

+1 -1
View File
@@ -116,7 +116,7 @@ The project uses Gitea Actions. Key jobs:
jobs:
new-job:
runs-on: ubuntu-latest
container: gitea.tap-tap.win/taptap/fastsync-ci:v10
container: gitea.tap-tap.win/taptap/fastsync-ci:v11
steps:
- uses: actions/checkout@v4
- name: Configure
+1 -1
View File
@@ -16,7 +16,7 @@ Ask the user or determine from context:
- **Minor** (x.Y.0) — new features, backward compatible
- **Patch** (x.y.Z) — bug fixes, no protocol changes
Current version: `PROTOCOL_VERSION "2.22.0"` in `src/shared/config.h`
Current version: `PROTOCOL_VERSION "2.23.0"` in `src/shared/config.h`
### Step 2: Check Protocol Version
+74
View File
@@ -4,6 +4,80 @@ All notable changes to FastSync are documented here. Versions match
`PROTOCOL_VERSION` (printed by `fastsync --version`); the client and server must
run the same version because the handshake is strict.
## [2.23.0] - 2026-09-16
### Added
- **Rsync-parity wave.** Closed the remaining CLI, filesystem, ownership,
deletion, and output gaps against rsync 3.4.1.
- Short options `-r` (`--recursive`), `-b` (`--backup`), `-L`
(`--copy-links`), and `-B` (`--block-size`/`--delta-block`); rsync
short-option clustering (`-av`, `-aAX`, `-rlpt`) and attached/inline values
(`--opt=value`, `-B1000`, `-essh`, `-MOPT`). A value that starts with `-`
is not mistaken for a cluster.
- `-c`/`--checksum` now implies the incremental checksum quick-check (and,
like rsync, does not imply `-t`).
- `--checksum-choice`/`--cc` accepts `xxh64`/`xxhash`/`xxh3`/`xxh128`/`md5`/
`auto` and rejects `md4`/`sha1`/`none` and the two-name form by name;
`--checksum-seed=0` (the default) is randomized per transfer and the chosen
seed is sent to the receiver.
- `--compress-choice`/`--zc` accepts `zstd`/`none`/`auto` and rejects
`lz4`/`zlib`/`zlibx` by name; `--skip-compress` defaults to rsync 3.4.1's
built-in suffix list; `--no-whole-file` is accepted.
- `--timeout` defaults to 0 (disabled) and `--contimeout` to 60 s (both `0`
disables), matching rsync; `--max-alloc=0` means no local limit.
- `--temp-dir` is confined to the receive root (absolute/`..` rejected by the
receiver) and an `EXDEV` install falls back to a non-atomic copy.
- `--numeric-ids` is documented as a mapping modifier only;
`--usermap`/`--groupmap` support inclusive `LOW-HIGH` ranges, `*`,
empty-`FROM` (unnamed ids), and receiver-resolved `TO` names; `--chown`
conflicts with a map on the same side are rejected.
- `--fake-super` records the *resolved* owner (never a real chown) and replays
mode/time; directory ownership and directory xattrs/ACLs are preserved.
- `-l`/`--links` stores symlink targets verbatim (absolute and `..`-bearing
included), matching rsync; `--safe-links`/`--copy-unsafe-links` are applied
sender-side and `--munge-links` uses rsync's `/rsyncd-munged/` marker;
`--trust-sender` no longer affects symlink targets.
- `--specials` recreates unix sockets with `mknod(S_IFSOCK)` (so `-D` covers
the full rsync node set).
- Deletion: the manifest carries a synchronized-directory section so
`--files-from` subsets no longer delete untransmitted paths;
`--delete-excluded` leaves size-pruned mirrors protected; extraneous
destination symlinks are unlinked (never followed); `--max-delete=N` is
partial (delete up to N, skip the rest, exit 25) and `--delete-missing-args`
removals draw from the same budget; `--force` is honored during
`--delay-updates` publication.
- `-x`/`--one-file-system` emits the mount-point directory entry; the
`--include`/`--exclude` layers are an ordered first-match rule list.
- `--chmod` is a faithful port of rsync 3.4.1 (numeric/symbolic, `D`/`F`/`X`,
`s`/`t`, append semantics, no `-p` implication, no sanitization).
### Changed
- `PROTOCOL_VERSION` bumped `2.22.0 → 2.23.0`: the delete manifest gains a
synchronized-directory section and the terminal status gains
`STATUS_DELETE_LIMIT` (client exit 25 on a `--max-delete`-capped commit).
- **The 2.22.0 mode-masking divergence is removed.** Under `-p` the source mode
is copied exactly, including `S_IWGRP`/`S_IWOTH` and setuid/setgid/sticky;
`--chmod` no longer implies `-p`. New files without `-p` still use
`source_mode & ~umask` when metadata is present (else `0644`), and new
directories without `-p` still use the `0755` creation default.
- `--protocol=NUM` accepts only the current `2.23.0` version string.
### Notes
- The rsync-compatibility matrix (`RSYNC_COMPAT.md`) now classifies every row
as **parity**, **caveat** (works with a documented divergence), or
**divergent** (not supported/no-op/impossible), replacing the previous
misleading "N implemented / 0 divergence" summary. Durable documented
divergences remain: receiver-side symlink target containment is not enforced
by default (verbatim storage is rsync parity; use `--safe-links`),
`--temp-dir` rejects absolute/foreign-filesystem paths, `--copy-devices`
reads a bounded `st_size`, a broken referent under `--copy-links` exits 0,
new directories without `-p` use `0755`, `--stats` receiver-only counters are
0, and `--password-file`/`--early-input`/`--hash-credentials`/`--iterations`
and the batch format are FastSync-native.
## [2.22.0] - 2026-09-15
### Added
+3 -1
View File
@@ -1,6 +1,6 @@
cmake_minimum_required(VERSION 3.22)
project(FastFileTransfer VERSION 2.22.0)
project(FastFileTransfer VERSION 2.23.0)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
set(CMAKE_C_STANDARD 11)
@@ -96,6 +96,7 @@ set(SHARED_SRCS
src/shared/file_send.c
src/shared/file_store.c
src/shared/filter.c
src/shared/format.c
src/shared/hardlink.c
src/shared/identity.c
src/shared/log.c
@@ -213,6 +214,7 @@ set(TEST_SRCS
tests/test_file.c
tests/test_file_list.c
tests/test_file_sendfile.c
tests/test_format.c
tests/test_fuzz_smoke.c
tests/test_glob.c
tests/test_hardlink.c
+2
View File
@@ -12,7 +12,9 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
# rsync is used as the reference implementation for drop-in parity tests.
# Ubuntu 24.04 ships 3.2.7, so build the pinned 3.4.1 reference from source.
ARG RSYNC_VERSION=3.4.1
ARG RSYNC_SHA256=2924bcb3a1ed8b551fc101f740b9f0fe0a202b115027647cf69850d65fd88c52
RUN curl -fsSL "https://download.samba.org/pub/rsync/src/rsync-${RSYNC_VERSION}.tar.gz" -o /tmp/rsync.tar.gz && \
echo "${RSYNC_SHA256} /tmp/rsync.tar.gz" | sha256sum -c - && \
tar -xzf /tmp/rsync.tar.gz -C /tmp && \
cd "/tmp/rsync-${RSYNC_VERSION}" && \
./configure --enable-zstd --enable-xxhash --enable-lz4 && \
+3 -2
View File
@@ -8,8 +8,8 @@
- **Release PR #284 (`dev` -> `main`)** open, CI green (run 553).
`main` is protected: it needs review/approval to merge.
https://gitea.tap-tap.win/TapTap/FastSync/pulls/284
- **`PROTOCOL_VERSION` = `"2.22.0"`** (`src/shared/config.h`); CMake
`project(FastFileTransfer VERSION 2.22.0)`.
- **`PROTOCOL_VERSION` = `"2.23.0"`** (`src/shared/config.h`); CMake
`project(FastFileTransfer VERSION 2.23.0)`.
- Working tree clean; no wave worktrees remain.
## What landed this session
@@ -38,6 +38,7 @@
`build-bench/`, `--warm` mode); `shell.nix` full toolchain and no build-on-entry;
docs state push-only / remote-source unsupported.
5. **Preserve-attribute split (protocol 2.22.0)** landed on `feat/preserve-attr-split`: per-attribute `-p/-t/-o/-g` + `--no-*` negations, `-a` = `-rlptgoD`, and the 2.21.0 → 2.22.0 wire bump.
6. **Rsync-parity wave (protocol 2.23.0)** on `feat/rsync-parity`: rsync short options/clustering/attached values (`-r`/`-b`/`-L`/`-B`, `-av`, `-aAX`, `-B1000`, `-essh`, `-MOPT`), `-c` checksum quick-check, `--checksum-choice`/`--compress-choice` validation and seed randomization, rsync timeout/max-alloc defaults, temp-dir confinement + `EXDEV` fallback, ownership/mapping parity (numeric-ids modifier, map ranges/`*`/empty-FROM, `--chown`+map conflicts, fake-super resolved-owner record), verbatim symlink storage with rsync `--safe-links`/`--munge-links`, socket recreation under `--specials`, `--chmod` 3.4.1 semantics, and delete scoping + `--max-delete` partial/exit-25. Wire: appended delete-manifest synchronized-directory section and `STATUS_DELETE_LIMIT`.
## Next steps
1. **Merge PR #284** (`dev` -> `main`) once reviewed (protected branch).
+106 -62
View File
@@ -61,20 +61,29 @@ replacement for every rsync feature or protocol mode.
- Temporary-file writes with atomic rename by default.
- Path traversal checks and destination-root confinement.
### Not yet equivalent to rsync
### Boundaries and documented divergences
The items below summarize FastSync's rsync compatibility status — recently
closed gaps and the remaining known divergences. Each row of the detailed
matrix is classified as parity, caveat, or divergent in
[`RSYNC_COMPAT.md`](RSYNC_COMPAT.md).
- The FastSync wire protocol is not the rsync wire protocol.
- SSH mode requires `fastsync-server` on the remote host.
- Archive mode covers rsync's `-rlptgoD` behavior — links, permissions, times,
owner, group, devices, and special files — and does not imply compression or
multithreading (see [Client](#client)). Ownership application is still
privilege-gated: a receiver that cannot `chown` logs a warning and skips it,
and a client-supplied mode can never grant group/other write (see
privilege-gated: a receiver that cannot `chown` logs a warning and skips it.
Under `-p` the source mode is copied exactly, including setuid/setgid/sticky
and group/other-write bits (strict rsync parity; see
[`RSYNC_COMPAT.md`](RSYNC_COMPAT.md)).
- Symlink transfer recreates only relative, `..`-free link targets
(`-l`/`--links`); an absolute target or any target containing a `..` component
is dropped rather than created, even if it would resolve within the receive
root. This containment check is skipped under `--trust-sender`.
- Symlink transfer stores targets **verbatim** (`-l`/`--links`), including
absolute and `..`-bearing targets, matching rsync. The receiver does not
enforce a containment predicate by default; `--safe-links` drops unsafe
targets on the sender, and `--munge-links` rewrites them with rsync's
`/rsyncd-munged/` marker. `--trust-sender` does not affect symlink targets.
A destination later consumed by a link-following tool can therefore follow a
link outside the receive root — use `--safe-links` for untrusted sources.
- Hard links (`-H`/`--hard-links`), extended attributes (`-X`/`--xattrs`), and
POSIX ACLs (`-A`/`--acls`) are preserved; owner/group is applied through
`-o`/`-g` (or an `-a`/`--archive` transfer), through the opt-in identity flags
@@ -84,8 +93,8 @@ replacement for every rsync feature or protocol mode.
divergences.
- Device and special-file preservation is implemented with documented
divergences: recreated device nodes require `CAP_MKNOD` on the receiver (a
non-root receiver skips the entry), and sockets cannot be recreated (FIFOs
are).
non-root receiver skips the entry), while FIFOs **and unix sockets** are
recreated (`--specials`).
- Sparse-file hole preservation (`-S`, `--sparse`) is implemented receiver-side:
long all-zero runs are written as holes (no wire change; the full file image
is already in memory).
@@ -98,11 +107,18 @@ replacement for every rsync feature or protocol mode.
- Short-option names are now rsync-parity (Phase 7 Wave A): FastSync's former
collisions were renamed (`-j`/`--threads`, `--preserve`, `--sendfile`,
`--chunk-serialization`, `--timeout`, `--ssh-port`), so `-m`, `-M`, `-f`,
`-s`, `-T`, `-p`, `-c`, `-a`, and `-z` follow rsync. See `RSYNC_COMPAT.md`.
`-s`, `-T`, `-p`, `-c`, `-a`, and `-z` follow rsync.
- Short-option clustering (`-av`, `-aAX`, `-rlpt`) and attached values
(`-B1000`, `-essh`, `-MOPT`, `--opt=value`) are accepted, matching rsync.
- `-r`, `-b`, `-L`, and `-B` are parsed with the rsync short names.
- `--stats` prints the counters FastSync can observe locally; receiver-only
counters (matched data, file-list bytes, deleted count) are reported as 0, and
`--progress` is an aggregate line rather than a per-file block.
The detailed flag matrix is maintained in
[`RSYNC_COMPAT.md`](RSYNC_COMPAT.md). It distinguishes implemented,
partial, alternate, and planned behavior.
[`RSYNC_COMPAT.md`](RSYNC_COMPAT.md). It reports each row as **parity**,
**caveat** (works with a documented divergence), or **divergent** (not
supported), rather than treating "parsed" as parity.
## Quick Start
@@ -120,11 +136,15 @@ This produces `./build/client` and `./build/server`. `compile_commands.json` is
| Argument | Description |
|----------|-------------|
| Positional | `<source> <dest>` — automatic SSH detection if dest contains `:` |
| `-c, --checksum` | Verify content by checksum instead of size+mtime |
| `-c, --checksum` | Verify content by checksum instead of size+mtime (implies the incremental checksum quick-check) |
| `--checksum-choice <alg>` | Whole-file checksum algorithm: `xxh64`/`xxhash` (default), `xxh3`, `xxh128`, `md5`, or `auto`; `md4`/`sha1`/`none` are rejected by name |
| `-z, --compress [level]` | Enable streaming zstd compression (level 1–22, default 5) |
| `--compress-choice <alg>` | Compression algorithm: `zstd` (default), `none`, or `auto`; `lz4`/`zlib`/`zlibx` are rejected by name |
| `--skip-compress <list>` | Skip compression for suffixes (`/`- or `,`-separated); defaults to rsync 3.4.1's built-in suffix list |
| `-a, --archive` | rsync archive mode (`-rlptgoD`): links, perms, times, owner, group, devices and specials; ownership application stays privilege-gated (not compression/multithreading) |
| `-j, --threads[=N]` | Multithreading mode; `N` (1–256) sets the parallel scanner worker count, bare `-j`/`--threads` uses the default |
| `-m` | rsync `--prune-empty-dirs` (short form now rsync-parity) |
| `-r, --recursive` | Recurse into directories (FastSync is always recursive; accepted for rsync compatibility) |
| `-d, --dirs` | Transfer the named directory entries without recursing into their contents; aliases `--old-dirs`/`--old-d` |
| `-R, --relative` | With `--files-from`, preserve each listed entry's relative path below the destination root |
| `--chunk-serialization` | Chunk serialization (batch all files per chunk; long form only) |
@@ -133,13 +153,15 @@ This produces `./build/client` and `./build/server`. `compile_commands.json` is
| `--preallocate` | Allocate destination file space up front (fail-fast on a full disk) |
| `--append` | Resume a shorter destination by appending only its tail (prefix not verified; requires `--incremental`) |
| `--append-verify` | Like `--append`, but verifies the retained prefix checksum first (falls back to a full transfer on mismatch) |
| `-W, --whole-file` | Transfer changed files without delta processing |
| `-W, --whole-file` | Transfer changed files without delta processing; `--no-whole-file` clears it |
| `-B <n>, --block-size <n>` | Delta block size in bytes (alias `--delta-block`) |
| `--checksum-seed <n>` | Seed for the whole-file xxHash digest; an unset/`0` seed is randomized per transfer, matching rsync |
| `-I, --ignore-times` | Transfer files even when size and mtime match |
| `--size-only` | Skip incremental files matching in size, ignoring mtime |
| `--preserve` | Preserve mode and mtime (`-p` + `-t`; add `-o`/`-g` for owner/group or `-U`/`--atimes` for atime; `-N`/`--crtimes` captures birth time but cannot apply it) |
| `-U, --atimes` | Preserve access times. Captured with the metadata payload; does not enable ownership. |
| `-N, --crtimes` | Capture birth time; cannot be applied (documented divergence) |
| `-p, --perms` | Preserve permission bits (a client mode never grants group/other write) |
| `-p, --perms` | Preserve permission bits. Strict rsync parity: the source mode is copied exactly, including setuid/setgid/sticky and group/other-write bits |
| `-t, --times` | Preserve modification times |
| `-o, --owner` | Preserve the source owner (privilege-gated; mapped by name on the receiver with a numeric fallback) |
| `-g, --group` | Preserve the source group (privilege-gated; mapped by name on the receiver with a numeric fallback) |
@@ -153,11 +175,11 @@ This produces `./build/client` and `./build/server`. `compile_commands.json` is
| `--groupmap=MAP` | Map group names when applying ownership |
| `--numeric-ids` | Apply source numeric uid/gid directly instead of mapping by name |
| `--copy-as=USER[:GROUP]` | Force every written entry to USER[:GROUP] (requires a privileged receiver) |
| `--fake-super` | Record/replay effective metadata via a reserved `user.fastsync.stat` xattr |
| `--fake-super` | Record the resolved owner plus mode/time in a reserved `user.fastsync.stat` xattr and replay mode/time; never performs a real chown |
| `--super` | Permit the receiver to attempt confined super-user activities (device nodes) |
| `-D` | Preserve device and special files (implies `--devices --specials`) |
| `--devices` | Recreate device nodes on the destination (privileged; skipped without `CAP_MKNOD`) |
| `--specials` | Recreate special files (FIFOs); sockets cannot be recreated |
| `--specials` | Recreate special files: FIFOs and unix sockets |
| `--remove-source-files` | Remove regular source files after a successful transfer |
| `--exclude <pattern>` | Exclude files matching glob pattern (repeatable) |
| `--exclude-from <file>` | Read exclude patterns from a file (one per line) |
@@ -166,30 +188,34 @@ This produces `./build/client` and `./build/server`. `compile_commands.json` is
| `--files-from <file>` | Read the source file list from FILE (paths relative to the source root) |
| `--max-size <n>` | Skip files larger than n bytes |
| `--min-size <n>` | Skip files smaller than n bytes |
| `--max-alloc <SIZE>` | Maximum single allocation (binary units: B, K, M, G, T, P, E; default 1G) |
| `-x, --one-file-system` | Do not cross filesystem boundaries; the mount-point directory entry is emitted (empty at the destination) without descending |
| `--max-alloc <SIZE>` | Maximum single allocation (binary units: B, K, M, G, T, P, E; default 1G; `0` = no local limit, matching rsync) |
| `-u, --update` | Skip files newer than the source on the receiver |
| `--incremental` | Skip files unchanged since last transfer (size + mtime). Auto-enables `--preserve`. Incompatible with `--chunk-serialization`. |
| `--existing` | Skip files not already present at the destination; update existing files normally. |
| `--compare-dest <dir>` | Extra comparison basis: unchanged files are not transferred (requires/implies `--incremental`) |
| `--copy-dest <dir>` | Like `--compare-dest`, but copies the unchanged file from DIR into the destination |
| `--link-dest <dir>` | Like `--copy-dest`, but hard-links the unchanged file from DIR (repeatable; earlier DIRs win) |
| `--delete` | Delete files on receiver not present in source (default timing: delete-after, i.e. only after the whole transfer succeeded) |
| `--delete` | Delete files on receiver not present in source (default timing: delete-after, i.e. only after the whole transfer succeeded). Scoped to the synchronized directories, so `--files-from` subsets are safe |
| `--delete-before` | Delete extras before the transfer starts (implies `--delete`) |
| `--delete-during`, `--del` | Delete extras once the keep-set is known, before data is applied (implies `--delete`) |
| `--delete-delay` | Delete extras only after a successful transfer (implies `--delete`) |
| `--delete-after` | Explicit delete-after timing (implies `--delete`) |
| `--delay-updates` | Put updated files into place only at the end of the transfer |
| `--delete-excluded` | Also delete filter-excluded destination mirrors (size-pruned mirrors stay protected) |
| `--max-delete <n>` | Delete at most n destination entries; the rest are skipped and the run exits 25 (partial), matching rsync |
| `--delay-updates` | Put updated files into place only at the end of the transfer (`--force` is honored at publication) |
| `-T, --temp-dir <dir>` | Scratch directory for temp files before the atomic install; confined to the receive root (relative only), with an `EXDEV` non-atomic copy fallback |
| `-n, --dry-run` | Report what would be transferred without mutating the destination. Since protocol 2.21.0 a server-routed target contacts the receiver and reports would-transfer based on receiver state; a plain local destination keeps the client-side scan. Never mutates or deletes. |
| `-v, --verbose` | Enable debug logging |
| `-q, --quiet` | Suppress non-error output |
| `--progress` | Show real-time transfer speed |
| `--progress` | Show a periodic aggregate transfer line (bytes sent, current rate); not rsync's per-file progress block |
| `-P` | Enables partial-transfer mode + progress output; interrupted writes retain the already-written temp for resumption |
| `--stats` | Print transfer statistics at end (bytes, files, timing) |
| `--stats` | Print transfer statistics at end (bytes, files, timing). Receiver-only counters (matched data, file-list bytes, deleted count) are reported as 0 |
| `-i, --itemize-changes` | Print an rsync-style per-file change line |
| `--out-format=FORMAT` | Output format for changed files (`%f %n %l %b %M %%`) |
| `--list-only` | List source files instead of transferring |
| `--fsync` | Fsync every written file before publication |
| `-h, --human-readable` | Format transfer byte sizes with binary units |
| `-h, --human-readable` | Format transfer byte/rate counts with rsync's decimal (base-1000) units |
| `--max-depth <n>` | Maximum directory depth to recurse (0 = unlimited, default: 0) |
| `--log-file <path>` | Write log messages to file instead of stderr |
| `--write-batch=FILE` | Run the normal live transfer and also emit a self-contained batch file of the source tree |
@@ -209,11 +235,11 @@ This produces `./build/client` and `./build/server`. `compile_commands.json` is
| `--sockopts=OPTS` | Comma-separated OPT=VAL socket options applied before connect (`TCP_NODELAY`, `SO_KEEPALIVE`, `SO_RCVBUF`, `SO_SNDBUF`, `SO_REUSEADDR`) |
| `--bwlimit <KB/s>` | Bandwidth limit in kilobytes per second |
| `--chunk-size <n>` | Chunk size in bytes (default: 10485760) |
| `--timeout <sec>` | Positive I/O timeout in seconds, applied to both the socket (`SO_RCVTIMEO`/`SO_SNDTIMEO`, built-in default 30 s) and the per-message protocol poll deadline (built-in default 60 s). Omit the option to keep both built-ins; `0` is rejected. The server side keeps the built-in 60 s protocol window (the value is not sent on the wire). |
| `--contimeout <sec>` | Connection timeout in seconds (default: 10) |
| `--timeout <sec>` | I/O timeout in seconds, applied to both the socket (`SO_RCVTIMEO`/`SO_SNDTIMEO`) and the per-message protocol poll deadline. Default `0` = disabled (matching rsync); `0` disables it. `--no-timeout` is the negation. The value is not sent on the wire; the server side keeps its own safe floor. |
| `--contimeout <sec>` | Connection timeout in seconds (default: 60, matching rsync); `0` disables it (`--no-contimeout` is the negation) |
| `--stop-after=MINS` | Stop the transfer after MINS minutes (a positive integer); whatever was already transferred is kept |
| `--stop-at=TIME` | Stop at an absolute time (`HH:MM`, `HH:MM:SS`, or `now+N[smhd]`); an early stop skips the late `--delete` keep-set |
| `--backup` | Backup existing destination files before overwriting |
| `-b, --backup` | Backup existing destination files before overwriting |
| `--backup-dir <dir>` | Target directory for backups (requires `--backup`) |
| `--tls` | Enable TLS encryption |
| `--cert <path>` | TLS certificate file (PEM) |
@@ -474,9 +500,9 @@ features without changing the meaning of ordinary compatibility options.
| `-j`, `--threads[=N]` | Enable the multithreaded scanner/loader/sender pipeline. `N` (1–256) sets the parallel scanner worker count; bare `-j`/`--threads` uses the default. |
| `-z [level]`, `--compress [level]` | Enable streaming zstd compression, levels 1-22. |
| `--compress-level <n>` | Set the zstd compression level. |
| `--zc <alg>` | Alias for `--compress-choice`. FastSync supports `zstd` and `none`. |
| `--zc <alg>` | Alias for `--compress-choice`. FastSync supports `zstd`, `none`, and `auto`; `lz4`/`zlib`/`zlibx` are rejected by name. |
| `--zl <n>` | Alias for `--compress-level`. |
| `--skip-compress <list>` | Skip compression for comma-separated suffixes; incompatible with `--chunk-serialization`. |
| `--skip-compress <list>` | Skip compression for `/`- or `,`-separated suffixes; defaults to rsync 3.4.1's built-in list. Incompatible with `--chunk-serialization`. |
| `--compress-threads <n>` | Use `n` zstd compression workers. Requires compression and a zstd build with threaded support; the setting affects sender CPU work only. |
| `--chunk-size <bytes>` | Set the transfer chunk size. |
| `--chunk-serialization` | Enable FastSync chunk serialization (long form only; `-s` is rsync's `--secluded-args`). |
@@ -488,10 +514,10 @@ features without changing the meaning of ordinary compatibility options.
| `--server-port <port>` | Select the TCP server port (`--port <port>` and `--port=<port>` are rsync-friendly aliases). |
| `--tls` | Enable TLS for TCP transport. |
| `--bwlimit <KB/s>` | Apply token-bucket bandwidth limiting. |
| `--progress` | Show transfer progress and throughput. |
| `--stats` | Print transfer statistics. |
| `--timeout <seconds>` | Set the socket **and** per-message protocol I/O timeout (positive seconds). Omit to keep the built-in 30 s socket / 60 s protocol defaults. |
| `--contimeout <seconds>` | Set connection timeout. |
| `--progress` | Show a periodic aggregate transfer line (throughput; not a per-file block). |
| `--stats` | Print transfer statistics (receiver-only counters are 0). |
| `--timeout <seconds>` | Set the socket **and** per-message protocol I/O timeout. Default `0` = disabled (matching rsync); `0` disables it. |
| `--contimeout <seconds>` | Connection timeout (default 60, matching rsync); `0` disables it. |
Short-option conflicts with rsync have been resolved for the CLI namespace
(Phase 7): `-c` is now rsync's `--checksum`, `-m` is `--prune-empty-dirs`, `-M`
@@ -517,11 +543,14 @@ remote SSH argv is already built injection-safe.
| `-n`, `--dry-run` | Report what would be transferred without mutating the destination. Since protocol 2.21.0 a server-routed target contacts the receiver and reports would-transfer based on receiver state; a plain local destination keeps the client-side scan. Never mutates or deletes. |
| `--remove-source-files` | Remove regular source files after a successful transfer. |
| `--incremental` | Skip files matching destination size and mtime. Auto-enables `--preserve`. Incompatible with `--chunk-serialization`. |
| `--checksum` | Include xxHash64 content checks in incremental comparisons. |
| `-c, --checksum` | Verify content by checksum (implies the incremental quick-check). Algorithm selectable with `--checksum-choice`. |
| `--checksum-choice <alg>` | Whole-file checksum algorithm: `xxh64`/`xxhash` (default), `xxh3`, `xxh128`, `md5`, or `auto`. |
| `--checksum-seed <n>` | Seed for the whole-file xxHash digest; an unset/`0` seed is randomized per transfer, matching rsync. |
| `--size-only` | Skip incremental files matching in size, ignoring mtime. |
| `-I, --ignore-times` | Transfer files even when size and mtime match. |
| `-u, --update` | Skip files newer than the source on the receiver. |
| `-W, --whole-file` | Transfer changed files without delta processing. |
| `-W, --whole-file` | Transfer changed files without delta processing (`--no-whole-file` clears it). |
| `-B <n>, --block-size <n>` | Delta block size in bytes (alias `--delta-block`). |
| `-d, --dirs` | Transfer the named directory entries without recursing into their contents (aliases `--old-dirs`/`--old-d`). |
| `-R, --relative` | With `--files-from`, preserve each listed entry's relative path below the destination root. |
| `--files-from <file>` | Read the source file list from FILE (paths relative to the source root). |
@@ -532,20 +561,24 @@ remote SSH argv is already built injection-safe.
| `--preallocate` | Allocate destination file space up front (fail-fast on a full disk). |
| `--append` | Resume a shorter destination by appending only its tail (prefix not verified; requires `--incremental`). |
| `--append-verify` | Like `--append`, but verifies the retained prefix checksum first (falls back to a full transfer on mismatch). |
| `--delete` | Request removal of destination entries absent from the source. The server must allow deletion. Default timing is delete-after: extras are removed only after the whole transfer succeeded. |
| `--delete` | Request removal of destination entries absent from the source. The server must allow deletion. Default timing is delete-after: extras are removed only after the whole transfer succeeded. Scoped to the synchronized directories, so `--files-from` subsets are safe. |
| `--delete-before` | Delete extras before the transfer starts (implies `--delete`). |
| `--delete-during`, `--del` | Delete extras once the keep-set manifest is known, before data is applied (implies `--delete`; early mode, same engine behaviour as `--delete-before`). |
| `--delete-delay` | Delete extras only after a successful transfer (implies `--delete`; commit mode, same behaviour as `--delete-after`). |
| `--delete-after` | Explicit delete-after timing: delete only after the transfer succeeded (implies `--delete`). |
| `--delete-excluded` | Also delete filter-excluded destination mirrors (size-pruned mirrors stay protected). |
| `--max-delete <n>` | Delete at most n destination entries; the rest are skipped and the run exits 25 (partial), matching rsync. |
| `--force` | Allow an incoming file/symlink to replace a destination directory (also during `--delay-updates` publication). |
| `--exclude <pattern>` | Exclude matching paths. Repeatable. |
| `--include <pattern>` | Include matching paths. Repeatable. |
| `--exclude-from <file>` | Read exclude patterns from a file. |
| `--include-from <file>` | Read include patterns from a file. |
| `--max-size <bytes>` | Skip files larger than the limit. |
| `--min-size <bytes>` | Skip files smaller than the limit. |
| `--max-alloc <SIZE>` | Maximum single allocation (binary units; default 1G). |
| `--max-alloc <SIZE>` | Maximum single allocation (binary units; default 1G; `0` = no local limit). |
| `--max-depth <n>` | Limit recursive scanning depth; zero means unlimited. |
| `--backup` | Back up overwritten files. |
| `-b, --backup` | Back up overwritten files. |
| `-T, --temp-dir <dir>` | Scratch directory for temp files before the atomic install (confined to the receive root; `EXDEV` falls back to a non-atomic copy). |
| `--backup-dir <dir>` | Store backups under a separate directory (requires `--backup`). |
| `--suffix <suffix>` | Set the backup filename suffix (default: `~`). |
| `--partial` | Select partial-transfer handling. On failed/interrupted writes the already-written temp file is retained (best-effort) for resumption. With `--partial --partial-dir <dir>`, completed files are written under the partial directory and installed atomically. |
@@ -565,7 +598,7 @@ remote SSH argv is already built injection-safe.
| `--preserve` | Preserve mode and mtime (long form only; equivalent to `-p` + `-t`). Add `-o`/`-g` for owner/group, `-U`/`--atimes` for atime, or an identity flag (`--chown`/`--usermap`/`--groupmap`/`--numeric-ids`/`--copy-as`) for mapped ownership. |
| `-U`, `--atimes` | Preserve access times. Captured with the metadata payload; does not enable ownership. |
| `-N`, `--crtimes` | Capture birth time and transmit it; it cannot be applied because no portable filesystem call can set a birth time (documented divergence). |
| `-p`, `--perms` | Preserve permission bits. One of the four per-attribute preserve flags (with `-t`/`-o`/`-g`); a client-supplied mode never grants group/other write. |
| `-p`, `--perms` | Preserve permission bits. One of the four per-attribute preserve flags (with `-t`/`-o`/`-g`); under `-p` the source mode is copied exactly (setuid/setgid/sticky and group/other-write included), matching rsync. |
| `-t`, `--times` | Preserve modification times. Independent of the other attributes; `-O`/`--omit-dir-times` suppresses directories only. |
| `-o`, `--owner` | Preserve the source owner (uid). Mapped by name on the receiver with a raw-numeric fallback (only numeric ids cross the wire); application is privilege-gated. |
| `-g`, `--group` | Preserve the source group (gid). Same name-mapping/numeric-fallback and privilege gating as `-o`. |
@@ -573,23 +606,26 @@ remote SSH argv is already built injection-safe.
| `-E`, `--executability` | Preserve executable permission bits. |
| `-X`, `--xattrs` | Preserve user `user.*` extended attributes. |
| `-A`, `--acls` | Preserve POSIX ACLs. |
| `--chmod <changes>` | Modify transferred permissions (rsync syntax). |
| `--chown=USER:GROUP` | Override the ownership of transferred files (`USER:GROUP`, `USER`, or `:GROUP`). |
| `--usermap=MAP` | Map usernames when applying ownership (comma-separated `FROM:TO` rules). |
| `--chmod <changes>` | Modify transferred permissions (rsync syntax, including `D`/`F`/`X` selectors and `s`/`t`); does not imply `-p`. |
| `--chown=USER:GROUP` | Override the ownership of transferred files (`USER:GROUP`, `USER`, or `:GROUP`); conflicts with `--usermap`/`--groupmap` on the same side. |
| `--usermap=MAP` | Map usernames when applying ownership (`FROM:TO` rules; names, ids, `LOW-HIGH` ranges, `*`, empty-`FROM`). |
| `--groupmap=MAP` | Map group names when applying ownership (same syntax as `--usermap`). |
| `--numeric-ids` | Apply the source numeric uid/gid directly instead of mapping by name. |
| `--numeric-ids` | Mapping modifier: apply the source numeric uid/gid directly instead of mapping by name (combine with `-o`/`-g`, `-a`, or a map). |
| `--copy-as=USER[:GROUP]` | Force every written entry to USER[:GROUP]; requires a privileged receiver. |
| `--fake-super` | Record/replay effective metadata via a reserved `user.fastsync.stat` xattr. |
| `--fake-super` | Record the resolved owner plus mode/time in a reserved `user.fastsync.stat` xattr and replay mode/time; never performs a real chown. |
| `--super` | Permit the receiver to attempt confined super-user activities (device nodes). |
| `--no-super` | Forbid those super-user activities even when the receiver is root. |
| `-l`, `--links` | Copy symlinks as symlinks; the target is transmitted and recreated under the receive root. |
| `--copy-links` | Copy symlink referents. |
| `--safe-links` | Skip symlinks that point outside the transfer tree. |
| `-l`, `--links` | Copy symlinks as symlinks; the target is stored verbatim (absolute and `..`-bearing targets included), matching rsync. |
| `-L`, `--copy-links` | Copy symlink referents (a broken referent exits 0). |
| `--safe-links` | Skip symlinks whose target points outside the transfer tree (applied on the sender). |
| `--copy-unsafe-links` | Copy unsafe symlink referents. |
| `--munge-links` | Rewrite stored symlink targets with rsync's `/rsyncd-munged/` marker. |
| `-k`, `--copy-dirlinks` | Treat a symlink to a directory as a real directory on the sender. |
| `-K`, `--keep-dirlinks` | Follow an existing destination symlink-to-directory (confined to the receive root). |
| `-H`, `--hard-links` | Preserve hard-link relationships across the transfer. |
| `-D` | Preserve device and special files (implies `--devices --specials`). |
| `--devices` | Recreate device nodes on the destination (privileged; skipped without `CAP_MKNOD`). |
| `--specials` | Recreate special files (FIFOs); sockets cannot be recreated. |
| `--specials` | Recreate special files: FIFOs and unix sockets. |
| `-S`, `--sparse` | Sparse-file handling: receiver preserves holes (zero runs are written as holes; no wire change). |
### Output and logging
@@ -598,8 +634,8 @@ remote SSH argv is already built injection-safe.
|---|---|
| `-v`, `--verbose` | Enable debug logging. |
| `-q`, `--quiet` | Suppress non-error output. |
| `--progress` | Show live transfer progress. |
| `--stats` | Print transfer statistics. |
| `--progress` | Show a periodic aggregate transfer line (not a per-file block). |
| `--stats` | Print transfer statistics (receiver-only counters are reported as 0). |
| `-i`, `--itemize-changes` | Print an rsync-style per-file change line. |
| `--out-format=FORMAT` | Output format for changed files (`%f %n %l %b %M %%`). |
| `--list-only` | List source files instead of transferring. |
@@ -613,8 +649,12 @@ remote SSH argv is already built injection-safe.
|---|---|
| `--ssh-port <port>` | SSH port for the SSH transport (default: 22). Note the short `-p` is now rsync's `--perms`. |
| `-e`, `--rsh <command>` | Remote shell to launch for the SSH transport (default: `ssh`; may include arguments). |
| `--fastsync-server-path <path>` | Remote FastSync server path for SSH mode. |
| `-M`, `--remote-option=OPT` | Append OPT to the remote server invocation over SSH (repeatable). |
| `--fastsync-server-path <path>` | Remote FastSync server path for SSH mode (client-only; never crosses the wire). |
| `--rsync-path <path>` | Alias for `--fastsync-server-path`. |
| `-M`, `--remote-option=OPT` | Append OPT to the remote server invocation over SSH (repeatable; rejected for daemon/TCP destinations). |
| `--trust-sender` | Receiver-local: trust the remote sender's file list and skip path re-validation (does not affect symlink targets). |
| `--timeout <sec>` | Socket + per-message I/O timeout; default `0` = disabled. |
| `--contimeout <sec>` | Connection timeout; default 60; `0` disables. |
| `--source-dir <path>` | Set the source directory explicitly. |
| `--dest-dir <path>` | Set the destination directory explicitly. |
| `--save-to-disk` | Enable server-side disk persistence. |
@@ -638,7 +678,7 @@ remote SSH argv is already built injection-safe.
| `--config=FILE` | Daemon config file (default: `~/.config/fastsync/fastsyncd.conf`, else `/etc/fastsyncd.conf`). Requires `--daemon`. |
| `--dparam=KEY=VALUE` | Override one global config key on the command line. Requires `--daemon`. |
| `--no-detach` | Stay in the foreground (default detaches to the background when running `--daemon`). |
| `-p <port>` | TCP listen port (default: 8080, range: 1–65535). |
| `-p, --port <port>` | TCP listen port (default: 8080, range: 1–65535). |
| `--tls` | Enable TLS. |
| `--cert <path>` | TLS certificate file (PEM). |
| `--key <path>` | TLS private key file (PEM). |
@@ -650,7 +690,7 @@ remote SSH argv is already built injection-safe.
| `-6`, `--ipv6` | Bind an IPv6 socket. |
| `--allow-delete` | Permit client delete manifests. Deletion is refused by default. This also gates `--force` (which can recursively replace/remove a destination directory tree). |
| `--allow-super` | Standalone TCP listener only: keep super-user activities enabled for a **root** receiver. Without it a root standalone server forces `SUPER_MODE_OFF`, so client `--devices`/`--write-devices`/`--super` and client-chosen ownership requests are skipped/refused. Rejected with `--stdio` (the SSH remote argv is client-composed; use a forced command if the default must hold). No effect when not root. Daemon modules opt in per module with `client owner = yes`. |
| `--trust-sender` | Trust the remote sender's file list: skip the receiver's up-front path-traversal and escaping-symlink-target containment re-validation (fewer checks, faster, potentially unsafe; off by default). |
| `--trust-sender` | Trust the remote sender's file list: skip the receiver's up-front path-traversal re-validation (fewer checks, faster, potentially unsafe; off by default). It does not affect symlink targets, which are stored verbatim either way. |
| `--no-super` | Operator veto: never attempt super-user activities (ownership, device nodes) even as root, and refuse any client `--copy-as`/`--super` request. |
| `--allow-unauthenticated` | Permit plaintext/anonymous network clients; an auth-required module still accepts only opted-in loopback plaintext. |
| `--iconv=LOCAL[,REMOTE]` | Declare this server's LOCAL charset for file-name conversion. |
@@ -742,7 +782,7 @@ before the module list, before authentication, and the connecting peer address
## Protocol and Security
FastSync protocol version `2.22.0` is shared by the client and server. The
FastSync protocol version `2.23.0` is shared by the client and server. The
current protocol is sender-driven and includes configuration negotiation,
including the maximum allocation limit, incremental checks, checksums,
manifests, keep-alives, abort handling, per-file remove-source results, and
@@ -807,20 +847,24 @@ operations require the server's explicit `--allow-delete` policy.
The project will reach the drop-in replacement goal in stages:
1. Correct rsync option meanings, including short options, combined options,
and `--option=value` syntax.
and `--option=value` syntax — **done** in the rsync-parity wave: `-r`/`-b`/
`-L`/`-B`, short-option clustering (`-av`, `-aAX`, `-rlpt`), and attached
values (`-B1000`, `-essh`, `-MOPT`) all parse.
2. Add differential tests that compare FastSync and rsync contents, metadata,
links, deletes, filters, dry runs, and exit codes.
3. `-a` now implements the expected recursive, links, permissions, times,
owner/group (`-o`/`-g`), and supported device/special-file behavior (full
rsync `-rlptgoD`); ownership application stays privilege-gated and remaining
work is the documented device/special-file divergences.
4. Symlink, sparse-file, metadata, delete-policy, and resumable-write semantics
are implemented; remaining work is the documented edge cases.
3. `-a` implements full rsync `-rlptgoD`; under `-p` the source mode is copied
exactly (no masking). Ownership application stays privilege-gated, as in
rsync.
4. Symlink (verbatim storage), sparse-file, metadata, delete-policy (including
`--max-delete` partial + exit 25), and resumable-write semantics are
implemented; remaining work is the documented edge cases, which the
**Rsync-Parity Wave** section of `RSYNC_COMPAT.md` enumerates honestly.
5. Add rsync remote-shell and daemon protocol interoperability.
6. Keep FastSync performance options as negotiated, optional extensions.
The exhaustive implementation matrix and compatibility notes are in
[`RSYNC_COMPAT.md`](RSYNC_COMPAT.md).
[`RSYNC_COMPAT.md`](RSYNC_COMPAT.md); each row is classified as parity, caveat,
or divergent.
## Testing
+430 -262
View File
@@ -6,13 +6,34 @@ This document maps rsync's full feature set to FastSync's current implementation
| Status | Count | Description |
|--------|-------|-------------|
| ✅ Implemented | 143 | Feature works end-to-end |
| 🔀 Alt Arg | 0 | Functionality exists but under different flag/semantics |
| ⛔ Impossible/Divergence | 4 | Flag is a documented divergence or cannot be implemented on any portable filesystem call |
| ⚠️ Partial | 0 | Flag parsed/stored but behavior incomplete |
| 🔄 Compatibility No-op | 0 | Flag is accepted for CLI compatibility but has no effect |
| ❌ Not Implemented | 0 | Flag not recognized or no behavior |
| **Total** | **147** | |
| ✅ Parity | 83 | Reproduces rsync's semantics for this option's scope |
| ⚠️ Caveat | 63 | Fully wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
| ❌ Divergent | 4 | Rejected, an accepted no-op, or impossible on any portable filesystem call |
| **Total** | **150** | One row per rsync option/feature group; a row may name several spellings |
This matrix reports honest rsync parity, not "implemented" as a synonym for
"parsed". A ✅ row matches rsync for the option's scope. A ⚠️ row is real and
tested but diverges in at least one documented way — FastSync's push-only model,
its own wire protocol, the delete timings that approximate rsync's engine modes,
the safe-subset privilege model (`--super`/`--copy-as`), the stricter
xattr/ACL and temp-dir policies, and the output counters that rsync computes on
the generator side. An ❌ row is either rejected (`--stderr=client`, `--protocol`
with any value but the current one), an accepted no-op (`-s`/`--secluded-args`),
or impossible (`-N`/`--crtimes`). The counts are derived from the rows below;
update them together with the table.
**Recently closed parity gaps (protocol 2.23.0).** The rsync-parity wave wired up
the short options `-r`, `-b`, `-L`, `-B`; rsync short-option clustering
(`-av`, `-aAX`, `-rlpt`) and attached/inline values (`--opt=value`, `-B1000`,
`-essh`, `-MOPT`); `-c` now implies the checksum quick-check; `--checksum-choice`
accepts `xxh64`/`xxhash`/`xxh3`/`xxh128`/`md5`/`auto` and rejects `md4`/`sha1`/
`none` by name; `--compress-choice` accepts `zstd`/`none`/`auto`; `--checksum-seed=0`
is randomized per transfer; `--skip-compress` uses rsync's default suffix list;
`--timeout`/`--contimeout` match rsync's defaults; deletion gained
`--max-delete` partial semantics with exit 25; symlinks are stored verbatim; and
`--specials` recreates sockets. Every one of those still has an entry below with
its remaining caveats. See the **Rsync-Parity Wave (protocol 2.23.0)** section
near the end for the full list and the known limitations.
---
@@ -20,100 +41,100 @@ This document maps rsync's full feature set to FastSync's current implementation
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-a`, `--archive` | Archive mode is -rlptgoD (rsync includes owner/group) | ✅ Implemented | Phase 7 Wave A: real rsync archive. `-a`/`--archive` now implies `--links` + the four per-attribute preserve flags (perms/times/owner/group) + `--devices` + `--specials`, i.e. **`-rlptgoD`**. Owner/group **are** implied, but their application stays privilege-gated exactly like rsync: a receiver that cannot `chown` logs a warning and skips it (see the preserve-attribute split note below). FastSync is always recursive, so no `-r` is needed. It no longer implies compression or multithreading (those moved to `-z`/`-j`). The short-option namespace is now rsync-parity (see the Phase 7 note) |
| `-v`, `--verbose` | Increase verbosity | ✅ Implemented | Sets `log_level=DEBUG` |
| `-q`, `--quiet` | Suppress non-error messages | ✅ Implemented | Suppresses client output while preserving errors |
| `--help` | Show help | ✅ Implemented | Prints usage and exits; `-h` is not accepted |
| `-V`, `--version` | Print version | ✅ Implemented | |
| `--info=FLAGS` | Fine-grained info verbosity | ✅ Implemented | Supports `copy`, `misc`, `skip`, `stats`, `all`, and `none`; explicit flags override `--verbose`, and `none` suppresses info output; unsupported names are rejected |
| `--debug=FLAGS` | Fine-grained debug verbosity | ✅ Implemented | `io`, `proto`, `pack`, and `util` are supported; `--debug=help` lists flags; other rsync categories are rejected |
| `--stderr=MODE` | Change stderr output mode | ⛔ Impossible/Divergence | `errors` (default) and `all` are supported; `client` is rejected with a clear error (`--stderr=client is not supported`) because FastSync has no rsync client-message channel — the rejection itself is the documented behavior (Phase 7 Wave B decision). The modes that exist work; the missing rsync channel cannot be emulated without a wire change |
| `--no-motd` | Suppress daemon MOTD | ✅ Implemented | Client-only display switch (Wave C): the daemon still sends the configured `motd file` on a `host::module/path` connection; the client reads and discards the frame without showing it. Without the flag the MOTD is printed to stdout after the config/auth handshake and escaped so control bytes cannot inject terminal sequences |
| `--exclude=PATTERN` | Exclude files matching pattern | ✅ Implemented | Glob matching in scanner |
| `--include=PATTERN` | Include files matching pattern | ✅ Implemented | Glob matching in scanner |
| `-C`, `--cvs-exclude` | Auto-ignore CVS files | ✅ Implemented | Applies the well-known rsync default exclude set as exclude rules during scanning (RCS SCCS CVS CVS.adm RCSLOG cvslog.* tags TAGS .make.state .nse_depinfo *~ #* .#* ,* _$* *$ *.old *.bak *.BAK *.orig *.rej .del-* *.a *.olb *.o *.obj *.so *.exe *.Z *.elc *.ln core .svn/ .git/ .hg/ .bzr/); `.git/`-style repo dirs are pruned without descending |
| `-a`, `--archive` | Archive mode is -rlptgoD (rsync includes owner/group) | ✅ Parity | Phase 7 Wave A: real rsync archive. `-a`/`--archive` now implies `--links` + the four per-attribute preserve flags (perms/times/owner/group) + `--devices` + `--specials`, i.e. **`-rlptgoD`**. Owner/group **are** implied, but their application stays privilege-gated exactly like rsync: a receiver that cannot `chown` logs a warning and skips it (see the preserve-attribute split note below). FastSync is always recursive, so no `-r` is needed. It no longer implies compression or multithreading (those moved to `-z`/`-j`). The short-option namespace is now rsync-parity (see the Phase 7 note) |
| `-v`, `--verbose` | Increase verbosity | ✅ Parity | Sets `log_level=DEBUG` |
| `-q`, `--quiet` | Suppress non-error messages | ✅ Parity | Suppresses client output while preserving errors |
| `--help` | Show help | ✅ Parity | Prints usage and exits; `-h` is not accepted |
| `-V`, `--version` | Print version | ✅ Parity | |
| `--info=FLAGS` | Fine-grained info verbosity | ⚠️ Caveat | Supports `copy`, `misc`, `skip`, `stats`, `all`, and `none`; explicit flags override `--verbose`, and `none` suppresses info output; unsupported names are rejected |
| `--debug=FLAGS` | Fine-grained debug verbosity | ⚠️ Caveat | `io`, `proto`, `pack`, and `util` are supported; `--debug=help` lists flags; other rsync categories are rejected |
| `--stderr=MODE` | Change stderr output mode | ❌ Divergent | `errors` (default) and `all` are supported; `client` is rejected with a clear error (`--stderr=client is not supported`) because FastSync has no rsync client-message channel — the rejection itself is the documented behavior (Phase 7 Wave B decision). The modes that exist work; the missing rsync channel cannot be emulated without a wire change |
| `--no-motd` | Suppress daemon MOTD | ✅ Parity | Client-only display switch (Wave C): the daemon still sends the configured `motd file` on a `host::module/path` connection; the client reads and discards the frame without showing it. Without the flag the MOTD is printed to stdout after the config/auth handshake and escaped so control bytes cannot inject terminal sequences |
| `--exclude=PATTERN` | Exclude files matching pattern | ✅ Parity | Glob matching in scanner |
| `--include=PATTERN` | Include files matching pattern | ✅ Parity | Glob matching in scanner |
| `-C`, `--cvs-exclude` | Auto-ignore CVS files | ✅ Parity | Applies the well-known rsync default exclude set as exclude rules during scanning (RCS SCCS CVS CVS.adm RCSLOG cvslog.* tags TAGS .make.state .nse_depinfo *~ #* .#* ,* _$* *$ *.old *.bak *.BAK *.orig *.rej .del-* *.a *.olb *.o *.obj *.so *.exe *.Z *.elc *.ln core .svn/ .git/ .hg/ .bzr/); `.git/`-style repo dirs are pruned without descending |
## 2. Modifying Output
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `--stats` | Give transfer stats | ✅ Implemented | Prints file/byte counts |
| `-h`, `--human-readable` | Human-readable numbers | ✅ Implemented | Formats transfer byte sizes using binary units |
| `-i`, `--itemize-changes` | Per-file change summary | ✅ Implemented | Prints rsync-style `>f+++++++++` lines to stdout only for files actually sent (also under `-j`/`--threads`); unchanged files print nothing, matching single-`-i` behavior |
| `--progress` | Show progress | ✅ Implemented | Progress callback in sender |
| `-P` | Same as --partial --progress | ✅ Implemented | Phase 7 Wave B: `-P` parses to `--partial` + `--progress`. On a failed/interrupted write the receiver now retains the already-written temp file at the destination path (best-effort rename instead of unlink when configured), so a later `--append`/`--append-verify` run can resume it; `--partial-dir` still stages completed files under the confined partial dir and installs them atomically. The retention never runs when `--partial` is off, when no data was actually written, or under `--ignore-existing`/`--existing` (the destination is not ours to overwrite), and it only ever renames the already-written temp (never a corrupt blend; a failed rename falls back to the normal unlink). See the `-S`/`--sparse` interplay note (a retained sparse temp has full logical size) |
| `--out-format=FORMAT` | Custom output format | ✅ Implemented | Per-transfer template on stdout; tokens `%f` `%n` `%l` `%b` `%M` `%%` (`%b` is the source length, always `== %l`; post-compression/delta wire bytes are not counted); unknown escapes preserved |
| `--log-file=FILE` | Log to file | ✅ Implemented | `log_file` config field |
| `--log-file-format=FMT` | Log format | ✅ Implemented | Requires `--log-file`; writes one template line per transferred file using the same token set as `--out-format` (including `%b` `==` source length) |
| `--8-bit-output`, `-8` | Leave high-bit chars unescaped | ✅ Implemented | Applies to displayed paths and protocol debug output |
| `--list-only` | List files instead of copying | ✅ Implemented | `ls -l`-style listing of files that would be transferred; scans the source only, contacts no server, writes nothing; also works with `-n` |
| `--stats` | Give transfer stats | ⚠️ Caveat | Prints file/byte counts. **Divergence:** the receiver-only counters rsync derives during its generator pass (matched/unchanged data, file-list bytes, deleted-entry count) are reported as **0** by FastSync, and the byte total counts source bytes actually sent rather than the post-delta/post-compression wire volume. Counts that FastSync can observe locally (files, bytes, timing) are accurate |
| `-h`, `--human-readable` | Human-readable numbers | ✅ Parity | Formats transfer byte and rate counts using rsync's **decimal** (base-1000) units, matching rsync `-h` (e.g. `1.23M`), not binary units |
| `-i`, `--itemize-changes` | Per-file change summary | ✅ Parity | Prints rsync-style `>f+++++++++` lines to stdout only for files actually sent (also under `-j`/`--threads`); unchanged files print nothing, matching single-`-i` behavior |
| `--progress` | Show progress | ⚠️ Caveat | Prints a periodic **aggregate** transfer line (bytes sent and current rate), not rsync's per-file progress block. With `-P` the partial-file retention behavior is fully implemented; only the progress presentation differs |
| `-P` | Same as --partial --progress | ⚠️ Caveat | Phase 7 Wave B: `-P` parses to `--partial` + `--progress`. On a failed/interrupted write the receiver now retains the already-written temp file at the destination path (best-effort rename instead of unlink when configured), so a later `--append`/`--append-verify` run can resume it; `--partial-dir` still stages completed files under the confined partial dir and installs them atomically. The retention never runs when `--partial` is off, when no data was actually written, or under `--ignore-existing`/`--existing` (the destination is not ours to overwrite), and it only ever renames the already-written temp (never a corrupt blend; a failed rename falls back to the normal unlink). See the `-S`/`--sparse` interplay note (a retained sparse temp has full logical size) |
| `--out-format=FORMAT` | Custom output format | ⚠️ Caveat | Per-transfer template on stdout; tokens `%f` `%n` `%l` `%b` `%M` `%%` (`%b` is the source length, always `== %l`; post-compression/delta wire bytes are not counted); unknown escapes preserved |
| `--log-file=FILE` | Log to file | ✅ Parity | `log_file` config field |
| `--log-file-format=FMT` | Log format | ✅ Parity | Requires `--log-file`; writes one template line per transferred file using the same token set as `--out-format` (including `%b` `==` source length) |
| `--8-bit-output`, `-8` | Leave high-bit chars unescaped | ✅ Parity | Applies to displayed paths and protocol debug output |
| `--list-only` | List files instead of copying | ✅ Parity | `ls -l`-style listing of files that would be transferred; scans the source only, contacts no server, writes nothing; also works with `-n` |
## 3. File Selection
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `--exclude-from=FILE` | Read exclude patterns from file | ✅ Implemented | Reads patterns from file |
| `--include-from=FILE` | Read include patterns from file | ✅ Implemented | Reads patterns from file |
| `--filter=RULE` | Add file-filtering rule | ✅ Implemented | Long option only: rsync's short `-f` conflicts with FastSync sendfile (see FastSync-specific list), so `-f` is not reassigned. Supported subset: `+`/`-` include/exclude, implicit-exclude patterns, `include`/`exclude` word forms, a leading `/` anchor (to the transfer root, or to a `.rsync-filter` file's directory), and a trailing `/` for dir-only rules; first match wins with a default of include inside the filter layer. Filters are an independent layer from `--exclude`/`--include` (an entry must pass both). Rejected with a clear error (no silent no-ops): `merge`/`dir-merge`/`hide`/`show`/`protect`/`risk`/`clear` words, rules that begin with `:`/`.`/`!` (merge/dir-merge/list-clear shorthands), and include/exclude modifiers other than `/` (`! C s r p x`) |
| `--files-from=FILE` | Read source file list from file | ✅ Implemented | Entries are paths relative to the source root (leading `./` stripped, `..`/absolute entries rejected at parse time, blank lines ignored; NUL-delimited with `-0`). A listed regular file is transferred; a listed directory transfers its whole subtree (FastSync recursion is always on, unlike rsync's non-recursive default). Non-listed paths and their subtrees are pruned by the scanner. A listed entry that does not exist under the source (and an empty list) is a hard error reported before any transfer, unless `--ignore-missing-args` / `--delete-missing-args` is given (see the Safety & Security rows): those flags downgrade the listed-but-missing case to a skip and, for `--delete-missing-args`, a destination deletion; an empty list stays a hard error in every mode. Listing `.` (whole tree) and empty listed directories are fine. Scalability note: `file_list_affects` is O(list size) per scanned entry, so a very large `--files-from` list against a huge tree is quadratic; lists are typically small enough that this is acceptable, but it is the documented bound. The delete manifest still derives from what was actually sent, so `--delete` stays consistent with the subset |
| `-0`, `--from0` | Delimit *-from files with NULs | ✅ Implemented | `--files-from` entries become NUL-delimited; the flag may appear before or after `--files-from` on the command line. NUL mode preserves entry bytes exactly (trailing CR/LF are part of the name; only newline mode trims them) |
| `--max-size=SIZE` | Skip files larger than SIZE | ✅ Implemented | `max_size` in scanner |
| `--min-size=SIZE` | Skip files smaller than SIZE | ✅ Implemented | `min_size` in scanner |
| `-I`, `--ignore-times` | Don't skip files matching size+time | ✅ Implemented | `ignore_times` config field (crosses the wire). Disables the size+mtime quick-check in the `--incremental` per-file handshake and the basis-dir quick-match, forcing the file to be transferred rather than skipped as unchanged. Receiver-side policy: `match_by_metadata` (file_receive.c) is bypassed, so the receiver never replies `STATUS_OK` for a matching size+mtime. Requires `--incremental` to have the handshake to act on (rsync does its quick check by default; FastSync's `-I`/`--size-only`/`--modify-window` only take effect under `--incremental`, exactly like they take effect through the basis check) |
| `--size-only` | Skip based on size only | ✅ Implemented | With `--incremental`, ignores mtime |
| `-@`, `--modify-window=NUM` | Mod-time comparison accuracy | ✅ Implemented | Whole-second tolerance with nanosecond-aware comparisons |
| `--existing` | Skip creating new files on receiver | ✅ Implemented | Existing destination files continue through normal update handling |
| `--ignore-existing` | Skip updating existing files | ✅ Implemented | `ignore_existing` config field (crosses the wire; receiver-side policy). For a destination entry that already exists, the receiver skips the write: in the regular-file path, existing/delay-updates-staged, hardlink-sibling, and special/device handlers all return `FILE_SAVE_SKIPPED` without overwriting (passed as `no_replace` to the write engine), and `--backup` is disabled for skipped files. Note: it is applied at write time, so an existing dest whose size+mtime differ still has its data (or delta) transmitted before the write is discarded — functionally correct, bandwidth-suboptimal vs rsync, which short-circuits earlier. Like rsync, it does not apply to directories/symlinks (those return before the block). Combines with `-j`/`--threads` and `--delay-updates`. See Phase-4/— notes below |
| `--remove-source-files` | Sender removes regular files after confirmed transfer | ✅ Implemented | |
| `-x`, `--one-file-system` | Do not cross filesystem boundaries | ✅ Implemented | Sender scanner captures the root device and skips descending into mount-point crossings (`st_dev` differs); cross-filesystem mount-point subdirectories are dropped entirely, matching rsync |
| `-F` | Add the default `.rsync-filter` rules | ✅ Implemented | Reads one filter rule per line from each directory's `.rsync-filter` file during traversal and applies it to that directory's subtree; the current directory's rules are evaluated before its ancestors', so deeper files override shallower ones and per-directory files override the command-line `--filter`/`-C` base by default (matching rsync's first-match-wins precedence); `.rsync-filter` files are never transferred. The rsync `-FF` behavior (also `.cvsignore`) is out of scope; unsupported rule types inside the file abort with a clear error |
| `--exclude-from=FILE` | Read exclude patterns from file | ✅ Parity | Reads patterns from file |
| `--include-from=FILE` | Read include patterns from file | ✅ Parity | Reads patterns from file |
| `--filter=RULE` | Add file-filtering rule | ⚠️ Caveat | Long option only: rsync's short `-f` conflicts with FastSync sendfile (see FastSync-specific list), so `-f` is not reassigned. Supported subset: `+`/`-` include/exclude, implicit-exclude patterns, `include`/`exclude` word forms, a leading `/` anchor (to the transfer root, or to a `.rsync-filter` file's directory), and a trailing `/` for dir-only rules; first match wins with a default of include inside the filter layer. Filters are an independent layer from `--exclude`/`--include` (an entry must pass both). Rejected with a clear error (no silent no-ops): `merge`/`dir-merge`/`hide`/`show`/`protect`/`risk`/`clear` words, rules that begin with `:`/`.`/`!` (merge/dir-merge/list-clear shorthands), and include/exclude modifiers other than `/` (`! C s r p x`) |
| `--files-from=FILE` | Read source file list from file | ⚠️ Caveat | Entries are paths relative to the source root (leading `./` stripped, `..`/absolute entries rejected at parse time, blank lines ignored; NUL-delimited with `-0`). A listed regular file is transferred; a listed directory transfers its whole subtree (FastSync recursion is always on, unlike rsync's non-recursive default). Non-listed paths and their subtrees are pruned by the scanner. A listed entry that does not exist under the source (and an empty list) is a hard error reported before any transfer, unless `--ignore-missing-args` / `--delete-missing-args` is given (see the Safety & Security rows): those flags downgrade the listed-but-missing case to a skip and, for `--delete-missing-args`, a destination deletion; an empty list stays a hard error in every mode. Listing `.` (whole tree) and empty listed directories are fine. Scalability note: `file_list_affects` is O(list size) per scanned entry, so a very large `--files-from` list against a huge tree is quadratic; lists are typically small enough that this is acceptable, but it is the documented bound. Delete scoping (protocol 2.23.0): the manifest carries the set of synchronized directories, and the extras walk only visits those subtrees, so `--delete` with a `--files-from` subset no longer removes destination paths outside the listed directory subtrees (a data-loss fix matching rsync) |
| `-0`, `--from0` | Delimit *-from files with NULs | ✅ Parity | `--files-from` entries become NUL-delimited; the flag may appear before or after `--files-from` on the command line. NUL mode preserves entry bytes exactly (trailing CR/LF are part of the name; only newline mode trims them) |
| `--max-size=SIZE` | Skip files larger than SIZE | ✅ Parity | `max_size` in scanner |
| `--min-size=SIZE` | Skip files smaller than SIZE | ✅ Parity | `min_size` in scanner |
| `-I`, `--ignore-times` | Don't skip files matching size+time | ✅ Parity | `ignore_times` config field (crosses the wire). Disables the size+mtime quick-check in the `--incremental` per-file handshake and the basis-dir quick-match, forcing the file to be transferred rather than skipped as unchanged. Receiver-side policy: `match_by_metadata` (file_receive.c) is bypassed, so the receiver never replies `STATUS_OK` for a matching size+mtime. Requires `--incremental` to have the handshake to act on (rsync does its quick check by default; FastSync's `-I`/`--size-only`/`--modify-window` only take effect under `--incremental`, exactly like they take effect through the basis check) |
| `--size-only` | Skip based on size only | ✅ Parity | With `--incremental`, ignores mtime |
| `-@`, `--modify-window=NUM` | Mod-time comparison accuracy | ✅ Parity | Whole-second tolerance with nanosecond-aware comparisons |
| `--existing` | Skip creating new files on receiver | ✅ Parity | Existing destination files continue through normal update handling |
| `--ignore-existing` | Skip updating existing files | ⚠️ Caveat | `ignore_existing` config field (crosses the wire; receiver-side policy). For a destination entry that already exists, the receiver skips the write: in the regular-file path, existing/delay-updates-staged, hardlink-sibling, and special/device handlers all return `FILE_SAVE_SKIPPED` without overwriting (passed as `no_replace` to the write engine), and `--backup` is disabled for skipped files. Note: it is applied at write time, so an existing dest whose size+mtime differ still has its data (or delta) transmitted before the write is discarded — functionally correct, bandwidth-suboptimal vs rsync, which short-circuits earlier. Like rsync, it does not apply to directories/symlinks (those return before the block). Combines with `-j`/`--threads` and `--delay-updates`. See Phase-4/— notes below |
| `--remove-source-files` | Sender removes regular files after confirmed transfer | ✅ Parity | |
| `-x`, `--one-file-system` | Do not cross filesystem boundaries | ✅ Parity | Sender scanner captures the root device and does not descend into mount-point crossings (`st_dev` differs). **Protocol 2.23.0 matches rsync's entry emission:** the mount-point directory itself is emitted as a payload-less directory entry (so the destination gets an empty directory) while its contents are skipped; previously the crossing subdirectory was dropped entirely |
| `-F` | Add the default `.rsync-filter` rules | ⚠️ Caveat | Reads one filter rule per line from each directory's `.rsync-filter` file during traversal and applies it to that directory's subtree; the current directory's rules are evaluated before its ancestors', so deeper files override shallower ones and per-directory files override the command-line `--filter`/`-C` base by default (matching rsync's first-match-wins precedence); `.rsync-filter` files are never transferred. The rsync `-FF` behavior (also `.cvsignore`) is out of scope; unsupported rule types inside the file abort with a clear error |
## 4. Directory Options
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-r`, `--recursive` | Recurse into directories | ✅ Implemented | Default behavior |
| `-R`, `--relative` | Use relative path names | ✅ Implemented | Meaningful together with `--files-from` (FastSync's default full-tree scan always mirrors the full source argument path below the destination root, so -R does not change it). With `-R` + `--files-from` each listed entry is transmitted under its bare relative destination path: an entry `sub/x.txt` lands at `<dest>/sub/x.txt` (its leading components preserved) instead of under the `<dest>/<full source path>` mirror. Only the path sent on the wire changes; the client still reads the absolute source path, and the delete manifest derives from the sent (relative) paths so `--delete` and `--remove-source-files` stay consistent in both layouts. Works single-threaded and under `-j`/`--threads` (including chunk serialization) |
| `--no-implied-dirs` | Don't send implied dirs with -R | ✅ Implemented | Client-side, meaningful only with `-R` + `--files-from`. rsync would normally create the ancestor directories implied by a listed file so it can be written; with `--no-implied-dirs` a listed file whose parent directory is not itself (or via an ancestor) explicitly listed cannot be placed, and FastSync fails the whole run up front with a clear error (`--no-implied-dirs: cannot place file '...': parent directory '...' is not explicitly listed`). Listing the directory (or an ancestor of it, or the whole tree `.`) permits the file. In every other mode the option has no effect. FastSync has no per-entry skip channel, so the rsync "omit the file" case is surfaced as a hard pre-transfer error |
| `-d`, `--dirs`, `--old-dirs`, `--old-d` | Transfer dirs without recursing | ✅ Implemented | `-d <dir>` transmits an explicit directory entry for the source-root directory, so the destination mirror is created empty and nothing is descended into. With `--files-from` exactly the listed items are transferred: a listed directory is created empty (no descent) and a listed file is transferred with its content; the dest layout follows the same -R rules as plain files. A new wire frame (`STATUS_MKDIR`) carries each directory entry — the path and, when `--preserve`/`-a` (metadata mode) is negotiated, the directory's metadata; the receiver creates it with the same confined mkdir-parent semantics as regular writes, in single-threaded and `-j`/`--threads` receivers (chunk serialization carries a per-entry type marker). Directory entries appear in the delete manifest so `--delete` prunes correctly. Directory TIMES are transmitted (the `STATUS_DIR_TIMES` frame carries every traversed source directory's captured times, including `--dirs` entries) and applied by the receiver at the END of the transfer, after all children and the delete/publication phases, so a later child write cannot clobber a directory's mtime (`-O`/`--omit-dir-times` skips this application). FastSync divergences: directory modes/ownership are still not applied (only times are), and empty directories are still never created (a `STATUS_DIR_TIMES` entry is record-only), filter/`--exclude` rules are not re-applied to the listed dirs mode (there is no descent during which they would apply), and `-d` never creates the intermediate directories between the destination root and a listed file beyond the usual on-demand parent creation. Under `--delay-updates` only regular files are staged: directory entries are created immediately, so a delayed run that fails part way can leave the already-created empty directories behind (matching rsync, which also creates directories as it processes the file list and only delays regular-file data) |
| `--mkpath` | Create missing path components | ✅ Implemented | Wire option (client → server). At connection start the server creates the client's destination root directory (and any missing leading components below its own authorized root) when `--mkpath` is set, failing the connection cleanly if it cannot. Without `--mkpath` a destination root that does not exist yet is rejected up front (rsync semantics), so the flag is the only way to transfer into a not-yet-created destination directory. Creation is confined by the same secure mkdir walk as file writes (`O_NOFOLLOW`, no `..`) |
| `-r`, `--recursive` | Recurse into directories | ✅ Parity | Default behavior |
| `-R`, `--relative` | Use relative path names | ⚠️ Caveat | Meaningful together with `--files-from` (FastSync's default full-tree scan always mirrors the full source argument path below the destination root, so -R does not change it). With `-R` + `--files-from` each listed entry is transmitted under its bare relative destination path: an entry `sub/x.txt` lands at `<dest>/sub/x.txt` (its leading components preserved) instead of under the `<dest>/<full source path>` mirror. Only the path sent on the wire changes; the client still reads the absolute source path, and the delete manifest derives from the sent (relative) paths so `--delete` and `--remove-source-files` stay consistent in both layouts. Works single-threaded and under `-j`/`--threads` (including chunk serialization) |
| `--no-implied-dirs` | Don't send implied dirs with -R | ⚠️ Caveat | Client-side, meaningful only with `-R` + `--files-from`. rsync would normally create the ancestor directories implied by a listed file so it can be written; with `--no-implied-dirs` a listed file whose parent directory is not itself (or via an ancestor) explicitly listed cannot be placed, and FastSync fails the whole run up front with a clear error (`--no-implied-dirs: cannot place file '...': parent directory '...' is not explicitly listed`). Listing the directory (or an ancestor of it, or the whole tree `.`) permits the file. In every other mode the option has no effect. FastSync has no per-entry skip channel, so the rsync "omit the file" case is surfaced as a hard pre-transfer error |
| `-d`, `--dirs`, `--old-dirs`, `--old-d` | Transfer dirs without recursing | ⚠️ Caveat | `-d <dir>` transmits an explicit directory entry for the source-root directory, so the destination mirror is created empty and nothing is descended into. With `--files-from` exactly the listed items are transferred: a listed directory is created empty (no descent) and a listed file is transferred with its content; the dest layout follows the same -R rules as plain files. A new wire frame (`STATUS_MKDIR`) carries each directory entry — the path and, when `--preserve`/`-a` (metadata mode) is negotiated, the directory's metadata; the receiver creates it with the same confined mkdir-parent semantics as regular writes, in single-threaded and `-j`/`--threads` receivers (chunk serialization carries a per-entry type marker). Directory entries appear in the delete manifest so `--delete` prunes correctly. Directory TIMES are transmitted (the `STATUS_DIR_TIMES` frame carries every traversed source directory's captured times, including `--dirs` entries) and applied by the receiver at the END of the transfer, after all children and the delete/publication phases, so a later child write cannot clobber a directory's mtime (`-O`/`--omit-dir-times` skips this application). FastSync divergences: directory modes/ownership are still not applied (only times are), and empty directories are still never created (a `STATUS_DIR_TIMES` entry is record-only), filter/`--exclude` rules are not re-applied to the listed dirs mode (there is no descent during which they would apply), and `-d` never creates the intermediate directories between the destination root and a listed file beyond the usual on-demand parent creation. Under `--delay-updates` only regular files are staged: directory entries are created immediately, so a delayed run that fails part way can leave the already-created empty directories behind (matching rsync, which also creates directories as it processes the file list and only delays regular-file data) |
| `--mkpath` | Create missing path components | ✅ Parity | Wire option (client → server). At connection start the server creates the client's destination root directory (and any missing leading components below its own authorized root) when `--mkpath` is set, failing the connection cleanly if it cannot. Without `--mkpath` a destination root that does not exist yet is rejected up front (rsync semantics), so the flag is the only way to transfer into a not-yet-created destination directory. Creation is confined by the same secure mkdir walk as file writes (`O_NOFOLLOW`, no `..`) |
## 5. Transfer Modifications
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-u`, `--update` | Skip files newer on receiver | ✅ Implemented | `update` config field (crosses the wire; receiver-side policy, implies `-M` metadata). Before writing a regular file, the receiver checks `file_destination_is_newer_secure()` (via `stat_is_newer`, second-then-nanosecond strict `>` on the existing destination) and skips the write when the destination is newer than the source (`FILE_SAVE_SKIPPED`); equal-or-older destination (or a newer source) is transferred normally. Applied at write time on the regular-file, delay-updates-staged, hardlink-sibling, and special/device paths. Only regular destinations can be guarded (the newer-check requires `S_ISREG`), and like the other write-time policies it does not short-circuit the data transfer for a differing-size dest. `--remove-source-files` correctly respects the receiver's skip outcome so a skipped source is not removed |
| `--inplace` | Update files in-place | ✅ Implemented | Direct write mode |
| `--append` | Append data to shorter files | ✅ Implemented | Tail-only resume. When an existing destination file is SHORTER than the source, the receiver negotiates a resume offset with the sender and only the tail is transferred; the receiver rebuilds the full file (retained prefix + tail) and installs it through the normal atomic store path, so the result is byte-identical to the source whenever the retained prefix matches. Plain `--append` does NOT content-verify that prefix (rsync parity): a destination whose prefix differs from the source is resumed anyway, so the result (wrong prefix + correct tail) is NOT byte-identical and the file is effectively left corrupt — the documented rsync-parity risk (use `--append-verify` when the prefix cannot be trusted). Non-content attributes (permissions/ownership/mtime, via `-M`) are still applied. Requires the per-file `STATUS_CHECK` handshake, so it implies `--incremental`; it takes precedence over block delta for a growing file and falls back to delta/full when the destination is not shorter. Incompatible with `-s` (chunk serialization) and `--whole-file` (both rejected up front so the mode never silently degrades to a full transfer). Combines with `--inplace`, `--partial`/`--partial-dir`, and `--delay-updates` (the reconstructed full file flows through those paths unchanged). Divergence: rsync appends in place; FastSync reconstructs and atomically installs, so an interrupted or failed resume never leaves a half-written file at the destination (no corruption window), and `--append` is thus safe to use with the normal atomic path — not only with in-place writes |
| `--append-verify` | Append with old-data checksum | ✅ Implemented | Like `--append`, but the retained prefix IS verified before resuming: the sender transmits the source prefix checksum and the receiver compares it to the xxHash64 of the retained destination prefix; on a match only the tail is transferred, on a MISMATCH the run falls back to a clean full transfer so the result is always a byte-identical source copy (never a corrupt prefix+tail blend). Wire/protocol: the append handshake adds `STATUS_APPEND` / `STATUS_APPEND_SIG` / `STATUS_APPEND_OK` / `STATUS_APPEND_DATA` frames and `PROTOCOL_VERSION` was bumped **2.9.0 → 2.10.0** (peers must match, and both must be 2.10.0 or the run fails the version check). Same implications/incompatibilities as `--append`; when both spellings are given `--append-verify` wins (the safer semantics). See the Phase-3 append notes below |
| `-W`, `--whole-file` | Copy whole file (no delta) | ✅ Implemented | `whole_file` config field. Forces a full (whole-file) copy, disabling the block-level delta machinery: the sender only sends `STATUS_NEXT` + full data (client_send.c) and the receiver never requests a delta signature/reconstruction — the receiver's `try_delta = use_delta && !whole_file && ...` short-circuits. `whole_file` crosses the wire folded into `use_delta` (the wire carries `use_delta && !whole_file`), so no separate field/bump is needed. Delta is opt-in (`--delta` needs `--incremental`); `-W` additionally makes `--fuzzy` inert (no similar-file delta basis). `--append`/`--append-verify` are incompatible with `-W` and rejected up front (both sides). See the delta/append notes below |
| `--block-size=SIZE` | Force checksum block-size | ✅ Implemented | Phase 7 Wave B: `--block-size` is an alias for `--delta-block`; both set `config->delta_block_size` (default `DELTA_BLOCK_SIZE_DEFAULT`, bounds `DELTA_BLOCK_SIZE_MIN..MAX`, out-of-range values are rejected with the default kept). The value is genuinely honored by the delta engine end-to-end: `delta_signature_create_seeded(old, size, config->delta_block_size, seed)` on the sender and receiver, `delta_apply(old, ...)` with the same size, so a non-default block size changes the block count of every signature the harnesses exchange (verified by unit + integration tests) |
| `-u`, `--update` | Skip files newer on receiver | ✅ Parity | `update` config field (crosses the wire; receiver-side policy, implies metadata transmission). Before writing a regular file, the receiver checks `file_destination_is_newer_secure()` (via `stat_is_newer`, second-then-nanosecond strict `>` on the existing destination) and skips the write when the destination is newer than the source (`FILE_SAVE_SKIPPED`); equal-or-older destination (or a newer source) is transferred normally. Applied at write time on the regular-file, delay-updates-staged, hardlink-sibling, and special/device paths. Only regular destinations can be guarded (the newer-check requires `S_ISREG`), and like the other write-time policies it does not short-circuit the data transfer for a differing-size dest. `--remove-source-files` correctly respects the receiver's skip outcome so a skipped source is not removed |
| `--inplace` | Update files in-place | ✅ Parity | Direct write mode |
| `--append` | Append data to shorter files | ⚠️ Caveat | Tail-only resume. When an existing destination file is SHORTER than the source, the receiver negotiates a resume offset with the sender and only the tail is transferred; the receiver rebuilds the full file (retained prefix + tail) and installs it through the normal atomic store path, so the result is byte-identical to the source whenever the retained prefix matches. Plain `--append` does NOT content-verify that prefix (rsync parity): a destination whose prefix differs from the source is resumed anyway, so the result (wrong prefix + correct tail) is NOT byte-identical and the file is effectively left corrupt — the documented rsync-parity risk (use `--append-verify` when the prefix cannot be trusted). Non-content attributes (permissions/ownership/mtime, via `-M`) are still applied. Requires the per-file `STATUS_CHECK` handshake, so it implies `--incremental`; it takes precedence over block delta for a growing file and falls back to delta/full when the destination is not shorter. Incompatible with `-s` (chunk serialization) and `--whole-file` (both rejected up front so the mode never silently degrades to a full transfer). Combines with `--inplace`, `--partial`/`--partial-dir`, and `--delay-updates` (the reconstructed full file flows through those paths unchanged). Divergence: rsync appends in place; FastSync reconstructs and atomically installs, so an interrupted or failed resume never leaves a half-written file at the destination (no corruption window), and `--append` is thus safe to use with the normal atomic path — not only with in-place writes |
| `--append-verify` | Append with old-data checksum | ⚠️ Caveat | Like `--append`, but the retained prefix IS verified before resuming: the sender transmits the source prefix checksum and the receiver compares it to the xxHash64 of the retained destination prefix; on a match only the tail is transferred, on a MISMATCH the run falls back to a clean full transfer so the result is always a byte-identical source copy (never a corrupt prefix+tail blend). Wire/protocol: the append handshake adds `STATUS_APPEND` / `STATUS_APPEND_SIG` / `STATUS_APPEND_OK` / `STATUS_APPEND_DATA` frames and `PROTOCOL_VERSION` was bumped **2.9.0 → 2.10.0** (peers must match, and both must be 2.10.0 or the run fails the version check). Same implications/incompatibilities as `--append`; when both spellings are given `--append-verify` wins (the safer semantics). See the Phase-3 append notes below |
| `-W`, `--whole-file` | Copy whole file (no delta) | ✅ Parity | `whole_file` config field. Forces a full (whole-file) copy, disabling the block-level delta machinery: the sender only sends `STATUS_NEXT` + full data (client_send.c) and the receiver never requests a delta signature/reconstruction — the receiver's `try_delta = use_delta && !whole_file && ...` short-circuits. `whole_file` crosses the wire folded into `use_delta` (the wire carries `use_delta && !whole_file`), so no separate field/bump is needed. Delta is opt-in (`--delta` needs `--incremental`); `-W` additionally makes `--fuzzy` inert (no similar-file delta basis). `--append`/`--append-verify` are incompatible with `-W` and rejected up front (both sides). See the delta/append notes below |
| `--block-size=SIZE` | Force checksum block-size | ✅ Parity | Phase 7 Wave B: `--block-size` is an alias for `--delta-block`; both set `config->delta_block_size` (default `DELTA_BLOCK_SIZE_DEFAULT`, bounds `DELTA_BLOCK_SIZE_MIN..MAX`, out-of-range values are rejected with the default kept). The value is genuinely honored by the delta engine end-to-end: `delta_signature_create_seeded(old, size, config->delta_block_size, seed)` on the sender and receiver, `delta_apply(old, ...)` with the same size, so a non-default block size changes the block count of every signature the harnesses exchange (verified by unit + integration tests) |
## 6. Destination Handling
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-n`, `--dry-run` | Trial run with no changes | ✅ Implemented | Server-contacting since protocol 2.21.0. The final routing predicate is `dry_run_targets_server()` in `src/client/client_send.c`: any target a real run would reach over the wire selects the server-contacting path — an SSH transport, a daemon `host::module` destination, an explicit `--server-host` or `--server-port`/`--port`, TLS, or a source-bind `--address` — and the client handshakes with the receiver, which runs the normal read-only per-file check and answers `STATUS_DRY_RUN_TRANSFER`/`STATUS_OK` without mutating anything. A plain local destination (none of those) keeps the original client-side manifest that never dials the default `127.0.0.1:8080`. Would-delete reporting for `--delete*` is deferred (dry-run never deletes). |
| `-b`, `--backup` | Make backups of overwritten files | ✅ Implemented | Backup before overwrite |
| `--backup-dir=DIR` | Backup directory hierarchy | ✅ Implemented | `backup_dir` config field |
| `--suffix=SUFFIX` | Backup suffix (default ~) | ✅ Implemented | `suffix` config field |
| `--delay-updates` | Put updated files in place at end | ✅ Implemented | Successfully received files are staged under a private 0700 `.fastsync-stage` dir inside the receive root and atomically renamed into their final destinations only after the whole transfer (manifest/delete handling included) succeeds, just before the success/outcome frame is sent. The delete walker deliberately skips the staging dir at the receive root, so `--delete` removes genuine extras but never the staged files (deletion runs before publication; rsync's delete-after ordering is not implemented). `--existing`/`--ignore-existing`/`--update` decide against the final destination path at stage time; `--backup` moves the old file aside at publication. Incompatible with `--inplace` and with `--backup-dir=.fastsync-stage` (the internal staging name is reserved; both are rejected). The staging dir name is fixed, so two simultaneous delayed transfers to the same destination root are serialized with an exclusive advisory lock held for the whole transfer: the second session fails cleanly instead of corrupting the first. Aborting or failing before publication installs nothing and removes the staging tree; a crash between stage and publish leaves staged leftovers that the next delayed run wipes at start (process death releases the lock). A stage→publish failure aborts the transfer (best-effort cleanup of the not-yet-published staged files; already-published files are not rolled back). Works in single-threaded and `-j`/`--threads` modes |
| `-T`, `--temp-dir=DIR` | Create temporary files in DIR | ✅ Implemented | `--temp-dir` with the rsync short `-T` (Phase 7 Wave A; the timeout alias moved to long-only `--timeout`). Scratch dir is resolved under the receive root; temp copies use a unique name there and are atomically renamed into place. If the scratch dir and destination are on different filesystems the atomic rename fails with EXDEV and the file save fails, which aborts the whole transfer (FastSync has no per-file skip/resume on a save error; rsync's non-atomic copy fallback is deliberately not used). `--inplace` and `--partial-dir` writes bypass the scratch dir |
| `-n`, `--dry-run` | Trial run with no changes | ⚠️ Caveat | Server-contacting since protocol 2.21.0. The final routing predicate is `dry_run_targets_server()` in `src/client/client_send.c`: any target a real run would reach over the wire selects the server-contacting path — an SSH transport, a daemon `host::module` destination, an explicit `--server-host` or `--server-port`/`--port`, TLS, or a source-bind `--address` — and the client handshakes with the receiver, which runs the normal read-only per-file check and answers `STATUS_DRY_RUN_TRANSFER`/`STATUS_OK` without mutating anything. A plain local destination (none of those) keeps the original client-side manifest that never dials the default `127.0.0.1:8080`. Would-delete reporting for `--delete*` is deferred (dry-run never deletes). |
| `-b`, `--backup` | Make backups of overwritten files | ✅ Parity | Backup before overwrite |
| `--backup-dir=DIR` | Backup directory hierarchy | ✅ Parity | `backup_dir` config field |
| `--suffix=SUFFIX` | Backup suffix (default ~) | ✅ Parity | `suffix` config field |
| `--delay-updates` | Put updated files in place at end | ⚠️ Caveat | Successfully received files are staged under a private 0700 `.fastsync-stage` dir inside the receive root and atomically renamed into their final destinations only after the whole transfer (manifest/delete handling included) succeeds, just before the success/outcome frame is sent. The delete walker deliberately skips the staging dir at the receive root, so `--delete` removes genuine extras but never the staged files (deletion runs before publication; rsync's delete-after ordering is not implemented). `--existing`/`--ignore-existing`/`--update` decide against the final destination path at stage time; `--backup` moves the old file aside at publication, and **`--force` is honored at publication** (protocol 2.23.0): a staged regular file or symlink may replace a destination directory that blocks it. Incompatible with `--inplace` and with `--backup-dir=.fastsync-stage` (the internal staging name is reserved; both are rejected). The staging dir name is fixed, so two simultaneous delayed transfers to the same destination root are serialized with an exclusive advisory lock held for the whole transfer: the second session fails cleanly instead of corrupting the first. Aborting or failing before publication installs nothing and removes the staging tree; a crash between stage and publish leaves staged leftovers that the next delayed run wipes at start (process death releases the lock). A stage→publish failure aborts the transfer (best-effort cleanup of the not-yet-published staged files; already-published files are not rolled back). Works in single-threaded and `-j`/`--threads` modes |
| `-T`, `--temp-dir=DIR` | Create temporary files in DIR | ⚠️ Caveat | `--temp-dir` with the rsync short `-T` (the timeout alias moved to long-only `--timeout`). **Protocol 2.23.0 receiver policy: the scratch dir is confined to the receive root — a relative dir is resolved below it, and an absolute path or one containing `..` is rejected by the receiver** (an absolute/foreign-filesystem scratch dir was the divergence; rsync's standalone mode would follow an absolute `--temp-dir`, while its daemon also confines). Temp copies use a unique name there and are atomically renamed into place. **On `EXDEV` (scratch dir and destination on different filesystems) the receiver falls back to a non-atomic copy instead of aborting the transfer**, matching rsync. `--inplace` and `--partial-dir` writes bypass the scratch dir |
## 7. Deletion
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `--delete` | Delete extraneous files from dest | ✅ Implemented | `use_delete` config field. Deletion is always derived from the transmitted keep-set manifest of the paths the sender sent/keeps (never from unchecked input), runs through the symlink-safe walker bounded by `MAX_SERVER_DELETE_COUNT`, and skips the `.fastsync-stage` staging dir under `--delay-updates`. FastSync's default timing when no timing flag is given is **delete-after** (extras are removed only once the whole transfer succeeded) — intentionally NOT rsync's `--del`/delete-during default, to preserve FastSync's commit-style safety. By default the destination mirror of a path the source scan pruned (filter/exclude/size rules) is **protected** from deletion — matching rsync, which does not delete excluded files under `--delete`; `--delete-excluded` opts back into deleting them (see below). The bounded deletion is **all-or-nothing**: if the destination holds more extras than the effective bound (a client `--max-delete=NUM` or the 100000-entry server bound) nothing is deleted and the run fails with a distinct error instead of silently truncating |
| `--delete-before` | Delete before transfer | ✅ Implemented | Implies `--delete`. The sender runs a full source pre-scan (paths only) and transmits the keep-set manifest BEFORE any file data; the receiver validates it, removes every destination entry not listed (all-or-nothing bounded walk, staging-dir skip, protected prefixes honored), then acks `STATUS_OK`. The sender only starts streaming after the deletion committed, or aborts if the receiver reported a deletion error. By definition the deletions already happened when a later transfer phase fails — rsync's delete-before is destructive the same way; a subsequent failure does not restore the removed files. Divergence: the keep-set is the pre-scan snapshot, so a file that appears on the source between the pre-scan and the data pass is still transferred but was not protected from deletion |
| `--del`, `--delete-during` | Delete during transfer | ✅ Implemented | Both spellings accepted; imply `--delete`. FastSync streams the source in a single directory scan and has no per-directory generator pass, so deletions cannot be interleaved per-directory the way rsync's delete-during does. `--delete-during` therefore selects the same early engine mode as `--delete-before` (manifest transmitted before any data, extras removed and acknowledged before data is applied); observable success/failure behaviour equals `--delete-before`. That is the documented divergence from rsync, where `--del` is the default meaning of `--delete` |
| `--delete-delay` | Find deletions during, delete after | ✅ Implemented | Implies `--delete`. Commit-mode timing: extras are removed only after the whole transfer succeeded. rsync's delete-delay records the deletion list during its scan and applies it at the end; FastSync never snapshots the destination while data flows (the keep-set is the transmitted manifest and the destination is listed only at deletion time), so `--delete-delay` is implemented as the same end-of-transfer commit as `--delete-after` with identical safety. That is the documented divergence |
| `--delete-after` | Delete after transfer | ✅ Implemented | Implies `--delete`. The delete-after timing is also what plain `--delete` does: the keep-set manifest closes the data stream and the receiver commits the bounded deletion only after the terminal `STATUS_FINISHED` proves the whole transfer (every data frame received and stored) succeeded. A failed or aborted transfer removes nothing |
| `--delete-excluded` | Also delete excluded files | ✅ Implemented | `delete_excluded` config field. Under `--delete` FastSync now protects (rsync's default) the destination mirror of paths the sender's source scan pruned by user-selection rules — the `--filter`/`-F`/`-C` layer, the legacy `--exclude`/`--include` layer, and `--max-size`/`--min-size`. The sender transmits those concrete pruned paths as **protected prefixes** in the delete-manifest frame (see the Phase-3 notes below); the walker never descends into or removes them. `--delete-excluded` opts back in: the sender sends an empty protected list, so the excluded destination mirrors become ordinary extras and are removed. Divergences (documented): protection is derived only from what the source scan actually pruned — a stray destination-only file that happens to match an exclude rule is not protected (FastSync never re-applies rules to the destination, keeping deletion sender-derived), and `--files-from` subset pruning stays keep-set-only (an unlisted source path is treated as absent and its mirror is deletable, matching the `--files-from` delete note below). The two are orthogonal: `--delete-excluded` removes filter-excluded mirrors; it does not make `--files-from` prune things |
| `--max-delete=NUM` | Max files to delete | ✅ Implemented | `max_delete` config field (default -1 = no client limit; 0 = delete nothing). NUM bounds a `--delete` run with rsync's all-or-nothing semantics: the receiver rehearses the deletion first and, if the destination holds more than NUM extras, deletes NOTHING and fails the transfer with a distinct `--max-delete` error. A run at or below NUM deletes exactly the extras. NUM only applies together with `--delete` (it is inert otherwise, matching rsync). The hard server bound `MAX_SERVER_DELETE_COUNT` (100000) still caps the walk; a NUM above it never raises that cap, and exceeding the server bound is its own all-or-nothing error. Directories count toward the limit (each removed empty directory is one deletion), like rsync |
| `--ignore-errors` | Delete even with I/O errors | ✅ Implemented | Sender-side, client-only config field. rsync suppresses `--delete` when the transfer had I/O errors; FastSync's equivalent is a source-scan I/O error (an unreadable directory, e.g. EACCES): by default the scan aborts the run so no deletion happens. With `--ignore-errors` the scan continues past the unreadable directory, the readable tree is transferred and the deletion still runs (the mirror of the unreadable directory is treated as an extra). The run still exits non-zero (the error is reported, matching rsync's error status). Divergence: without the flag FastSync aborts the whole run on the scan error, whereas rsync transfers the rest of the tree and merely skips the deletion; both leave the deletion undone |
| `--force` | Force deletion of non-empty dirs | ✅ Implemented | `force_delete` receiver config field (crosses the wire). rsync's `--force` lets an incoming non-directory replace a destination directory; FastSync implements exactly that: when a regular file is written to a path that is currently a (possibly non-empty) destination directory, `--force` removes that directory tree first — confined to the receive root and symlink-safe (O_NOFOLLOW fd walk, symlinks removed by name, never followed) — so the atomic install can place the file. Without `--force` such a write fails and the run aborts. Divergence: `--force` acts on the immediate-install path only; under `--delay-updates` a blocking directory is not cleared (publication renames over regular files) |
| `-m`, `--prune-empty-dirs` | Prune empty dir chains | ✅ Implemented | `-m`/`--prune-empty-dirs` (Phase 7 Wave A freed the rsync short `-m`; FastSync multithreading is now `-j`/`--threads`). FastSync's recursive transfer records directory times but never CREATES an empty directory (a `STATUS_DIR_TIMES` entry is record-only, and `--dirs` empty entries are pruned by this flag), so empty directories are inherently never transferred (which is rsync's `-m` behavior) and truly-empty destination directory chains are removed by `--delete` regardless of this flag. The flag's additional real effect is on the `--dirs` explicit directory-entry generator: a plain `-d <empty-dir>` run omits the empty source directory's entry, so nothing is created at the destination (no `STATUS_MKDIR`, no `-i`/`--out-format` change line, and an existing empty mirror becomes an extra that `--delete` prunes). Explicitly `--files-from`-listed directories always pass through (documented `--files-from` behavior). A directory that still holds an excluded-but-protected file survives, matching the `--delete-excluded` default |
| `--delete` | Delete extraneous files from dest | ⚠️ Caveat | `use_delete` config field. Deletion is always derived from the transmitted keep-set manifest of the paths the sender sent/keeps (never from unchecked input), runs through the symlink-safe walker bounded by `MAX_SERVER_DELETE_COUNT`, and skips the `.fastsync-stage` staging dir under `--delay-updates`. FastSync's default timing when no timing flag is given is **delete-after** (extras are removed only once the whole transfer succeeded) — intentionally NOT rsync's `--del`/delete-during default, to preserve FastSync's commit-style safety. By default the destination mirror of a path the source scan pruned (filter/exclude/size rules) is **protected** from deletion — matching rsync, which does not delete excluded files under `--delete`; `--delete-excluded` opts back into deleting them (see below). Deletion is scoped to the **synchronized directories** sent in the manifest (protocol 2.23.0), so a `--files-from` subset no longer deletes untransmitted paths outside the listed directory subtrees. The walk is bounded: a client `--max-delete=NUM` (or the 100000-entry server bound) makes it **partial** — entries up to the bound are removed, the rest are skipped, and the client exits **25** (`RERR_PARTIAL`), matching rsync, rather than failing the transfer. Extraneous destination symlinks are unlinked by name (never followed); a directory still holding a kept/protected entry is left behind rather than failing |
| `--delete-before` | Delete before transfer | ⚠️ Caveat | Implies `--delete`. The sender runs a full source pre-scan (paths only) and transmits the keep-set manifest BEFORE any file data; the receiver validates it, removes every destination entry not listed (bounded walk, staging-dir skip, protected prefixes honored), then acks `STATUS_OK`. The sender only starts streaming after the deletion committed, or aborts if the receiver reported a deletion error. By definition the deletions already happened when a later transfer phase fails — rsync's delete-before is destructive the same way; a subsequent failure does not restore the removed files. Divergence: the keep-set is the pre-scan snapshot, so a file that appears on the source between the pre-scan and the data pass is still transferred but was not protected from deletion |
| `--del`, `--delete-during` | Delete during transfer | ⚠️ Caveat | Both spellings accepted; imply `--delete`. FastSync streams the source in a single directory scan and has no per-directory generator pass, so deletions cannot be interleaved per-directory the way rsync's delete-during does. `--delete-during` therefore selects the same early engine mode as `--delete-before` (manifest transmitted before any data, extras removed and acknowledged before data is applied); observable success/failure behaviour equals `--delete-before`. That is the documented divergence from rsync, where `--del` is the default meaning of `--delete` |
| `--delete-delay` | Find deletions during, delete after | ⚠️ Caveat | Implies `--delete`. Commit-mode timing: extras are removed only after the whole transfer succeeded. rsync's delete-delay records the deletion list during its scan and applies it at the end; FastSync never snapshots the destination while data flows (the keep-set is the transmitted manifest and the destination is listed only at deletion time), so `--delete-delay` is implemented as the same end-of-transfer commit as `--delete-after` with identical safety. That is the documented divergence |
| `--delete-after` | Delete after transfer | ✅ Parity | Implies `--delete`. The delete-after timing is also what plain `--delete` does: the keep-set manifest closes the data stream and the receiver commits the bounded deletion only after the terminal `STATUS_FINISHED` proves the whole transfer (every data frame received and stored) succeeded. A failed or aborted transfer removes nothing |
| `--delete-excluded` | Also delete excluded files | ⚠️ Caveat | `delete_excluded` config field. Under `--delete` FastSync protects (rsync's default) the destination mirror of paths the sender's source scan pruned by the user-selection rules — the `--filter`/`-F`/`-C` layer and the legacy `--exclude`/`--include` layer. The sender transmits those concrete pruned paths as **protected prefixes** in the delete-manifest frame (see the Phase-3 notes below); the walker never descends into or removes them. `--delete-excluded` opts back in: the sender sends an empty protected list, so the excluded destination mirrors become ordinary extras and are removed. **`--max-size`/`--min-size` pruned mirrors are a separate, always-on protection** (protocol 2.23.0, rsync parity): size-pruned source mirrors survive `--delete` even with `--delete-excluded`. Divergences (documented): protection is derived only from what the source scan actually pruned — a stray destination-only file that happens to match an exclude rule is not protected (FastSync never re-applies rules to the destination, keeping deletion sender-derived) |
| `--max-delete=NUM` | Max files to delete | ✅ Parity | `max_delete` config field (default -1 = no client limit; 0 = delete nothing). **Protocol 2.23.0 matches rsync's partial semantics:** the receiver deletes up to NUM entries (regular files, symlinks and empty directories; each directory removal counts as one) and then **stops deleting, skips the rest, and reports the run as partial**. The client prints a "deletions stopped due to `--max-delete` limit" message and exits **25** (rsync's `RERR_PARTIAL`), not a hard failure — the transfer itself succeeded. NUM only applies together with `--delete` (it is inert otherwise, matching rsync). A client NUM below the server hard bound `MAX_SERVER_DELETE_COUNT` (100000) replaces it; a NUM above it never raises that cap. Deleting an entire destination with no limit is still bounded by the server's 100000-entry ceiling. `--delete-missing-args` exact-path deletions and the ordinary extras walk draw from the same budget, matching rsync |
| `--ignore-errors` | Delete even with I/O errors | ⚠️ Caveat | Sender-side, client-only config field. rsync suppresses `--delete` when the transfer had I/O errors; FastSync's equivalent is a source-scan I/O error (an unreadable directory, e.g. EACCES): by default the scan aborts the run so no deletion happens. With `--ignore-errors` the scan continues past the unreadable directory, the readable tree is transferred and the deletion still runs (the mirror of the unreadable directory is treated as an extra). The run still exits non-zero (the error is reported, matching rsync's error status). Divergence: without the flag FastSync aborts the whole run on the scan error, whereas rsync transfers the rest of the tree and merely skips the deletion; both leave the deletion undone |
| `--force` | Force deletion of non-empty dirs | ⚠️ Caveat | `force_delete` receiver config field (crosses the wire). rsync's `--force` lets an incoming non-directory replace a destination directory; FastSync implements exactly that: when a regular file (or symlink) is written to a path that is currently a (possibly non-empty) destination directory, `--force` removes that directory tree first — confined to the receive root and symlink-safe (O_NOFOLLOW fd walk, symlinks removed by name, never followed) — so the install can place the file. **Protocol 2.23.0 honors `--force` on the `--delay-updates` publication path too**, not only the immediate-install path. Without `--force` such a write fails and the run aborts. Gated by the server `--allow-delete` policy (a client cannot use `--force` to remove a destination tree on a server that forbids deletion) |
| `-m`, `--prune-empty-dirs` | Prune empty dir chains | ✅ Parity | `-m`/`--prune-empty-dirs` (Phase 7 Wave A freed the rsync short `-m`; FastSync multithreading is now `-j`/`--threads`). FastSync's recursive transfer records directory times but never CREATES an empty directory (a `STATUS_DIR_TIMES` entry is record-only, and `--dirs` empty entries are pruned by this flag), so empty directories are inherently never transferred (which is rsync's `-m` behavior) and truly-empty destination directory chains are removed by `--delete` regardless of this flag. The flag's additional real effect is on the `--dirs` explicit directory-entry generator: a plain `-d <empty-dir>` run omits the empty source directory's entry, so nothing is created at the destination (no `STATUS_MKDIR`, no `-i`/`--out-format` change line, and an existing empty mirror becomes an extra that `--delete` prunes). Explicitly `--files-from`-listed directories always pass through (documented `--files-from` behavior). A directory that still holds an excluded-but-protected file survives, matching the `--delete-excluded` default |
**Deletion-timing implementation notes (Phase 3):** the delete flags above are
real. Two new config booleans (`delete_during`, `delete_delay`) join the already
@@ -133,64 +154,66 @@ noted in the rows above.
**Deletion-policy notes (Phase 3, delete-policy wave):** this wave made the
deletion family real — `--delete-excluded`, `--max-delete`, `--ignore-errors`,
`--force`, `--prune-empty-dirs` — and, to support them, the `STATUS_MANIFEST`
frame now carries **two sections**: the keep-set paths followed by a list of
**protected prefixes** (destination-relative paths the source scan pruned by
user-selection rules, which the walker must never delete unless
`--delete-excluded` opted out). Two config booleans were added for the wave:
`force_delete` (crosses the wire; the receiver clears a directory that blocks an
incoming file) and `ignore_errors` (client-only; the sender's scan continues
past an unreadable directory). `max_delete`'s default became -1 ("no client
limit"). These wire/layout changes bumped `PROTOCOL_VERSION` **2.8.0 → 2.9.0**
(peers must match). All four wire additions — `force_delete`,
`delete_excluded`, `prune_empty_dirs`, `max_delete` — round-trip unchanged and
are validated on receive.
frame carries the keep-set paths followed by a list of **protected prefixes**
(destination-relative paths the source scan pruned by user-selection rules,
which the walker must never delete unless `--delete-excluded` opted out).
Two config booleans were added for the wave: `force_delete` (crosses the wire;
the receiver clears a directory that blocks an incoming file) and
`ignore_errors` (client-only; the sender's scan continues past an unreadable
directory). `max_delete`'s default became -1 ("no client limit"). These
wire/layout changes bumped `PROTOCOL_VERSION` **2.8.0 → 2.9.0** (peers must
match). All four wire additions — `force_delete`, `delete_excluded`,
`prune_empty_dirs`, `max_delete` — round-trip unchanged and are validated on
receive.
**Missing-args note (Phase 3, missing-args wave):** `--ignore-missing-args` and
`--delete-missing-args` are implemented as described in the Safety & Security
rows. Wire impact: the `STATUS_MANIFEST` frame now carries a **third section** —
**Missing-args note (Phase 3, missing-args wave; extended in 2.23.0):**
`--ignore-missing-args` and `--delete-missing-args` are implemented as described
in the Safety & Security rows. Wire impact: the `STATUS_MANIFEST` frame carries
a list of destination-relative **exact-delete paths** (the missing entries'
mirrors) — and the config frame gained a `delete_missing_args` boolean
mirrors), and the config frame gained a `delete_missing_args` boolean
(`ignore_missing_args` stays client-only, exactly like `ignore_errors`). These
wire/layout changes bumped `PROTOCOL_VERSION` **2.9.0 → 2.10.0** (peers must
match). The receiver validates the third section identically to the keep-set
(non-empty, relative, traversal-free; `MAX_MANIFEST_ENTRIES` per section, a
single `MAX_MANIFEST_BYTES` budget shared across all three). On commit the
receiver runs the exact-path deletions FIRST (`manifest_delete_missing_args`:
confined per-path unlink/rmdir, deep removal only under `--force`/`--delete`,
staging/basis protected, never blocked by the protected-prefix list) and then
the ordinary extras walk when `--delete` is active (`manifest_delete_all`). A
client may request the exact-path deletions without `--delete`; the server's
`--allow-delete` policy gates them exactly like `--delete`, so an unauthorized
server ignores the request while the missing entries are still skipped.
match). The receiver validates the section identically to the keep-set (non-empty,
relative, traversal-free; the shared `MAX_MANIFEST_ENTRIES`/`MAX_MANIFEST_BYTES`
budget spans every section). On commit the receiver runs the exact-path deletions
FIRST (`manifest_delete_missing_args`: confined per-path unlink/rmdir, deep
removal only under `--force`/`--delete`, staging/basis protected, never blocked
by the protected-prefix list) and then the ordinary extras walk when `--delete`
is active (`manifest_delete_all`). A client may request the exact-path deletions
without `--delete`; the server's `--allow-delete` policy gates them exactly like
`--delete`, so an unauthorized server ignores the request while the missing
entries are still skipped.
The deletion walker is now **all-or-nothing**: before any unlink it rehearses
the deletion (an fd-relative walk identical to the delete pass, counting every
regular file it would unlink and every directory it would remove) and refuses to
start when the extras exceed the effective bound — a client `--max-delete=NUM`
below the hard bound, or the hard `MAX_SERVER_DELETE_COUNT` (100000) bound
itself. Previously the walker removed up to `MAX_SERVER_DELETE_COUNT` extras and
then reported an error (a truncated deletion); it now removes nothing and fails
with an error naming the bound. Directories count toward the bound. A directory
that still holds entries the walker leaves in place (a protected excluded file,
a kept manifest entry, a symlink) is left behind rather than failing the run —
matching rsync's "cannot delete non-empty directory" behaviour. The
all-or-nothing guarantee holds only while the destination is not concurrently
modified: rehearsal and delete are two separate walks, so a concurrent change
between them (another process adding or removing destination entries) can make
the actual deletion diverge from the counted set.
**Delete scoping and partial limits (protocol 2.23.0).** The `STATUS_MANIFEST`
frame now carries **four sections** — keep-set, protected prefixes, exact-delete
(missing-args) paths, and the set of **synchronized directories**. The extras
walk is scoped to the synchronized directories, so a `--files-from` subset no
longer deletes untransmitted destination paths outside the listed directory
subtrees (a data-loss fix, matching rsync). `--max-size`/`--min-size` pruned
source mirrors are protected independently of `--delete-excluded`. Extraneous
destination symlinks are unlinked by name (never followed). The `--max-delete`
budget is **partial**: the walker deletes up to the effective bound (a client
`--max-delete=NUM` below the hard bound, else the hard
`MAX_SERVER_DELETE_COUNT` = 100000) and then stops, skips the rest, and reports
the run as partial so the client exits **25** (`RERR_PARTIAL`) exactly like
rsync — it is a successful transfer with an incomplete deletion, not a hard
failure. The exact-path missing-args removals and the extras walk share that one
budget. A directory that still holds entries the walker leaves in place (a
protected excluded file, a kept manifest entry, a symlink) is left behind rather
than failing the run — matching rsync's "cannot delete non-empty directory"
behaviour.
Manifest size: the sender's keep-set and protected-prefix collections (streaming
or early pre-scan) are unbounded, but the receiver rejects a manifest beyond
`MAX_MANIFEST_ENTRIES` (1 048 576 entries, applied to EACH section — a frame can
therefore total up to 2 097 152 entries) / `MAX_MANIFEST_BYTES` (16 MB of paths,
counted across BOTH sections) as a hard protocol error. A heavily filtered
source whose exclusion list grows large thus fails the run cleanly on the
receiver (STATUS_ERROR) instead of being silently truncated. In the commit
modes this only means the deletion is refused after the data already arrived; in
the early modes (`--delete-before`/`--delete-during`) the manifest is the first
frame, so an oversized keep-set or protected list aborts the whole transfer
BEFORE any data is sent. Keep the source tree small enough for the receiver's
manifest caps when using the early timing.
Manifest size: the sender's collections (streaming or early pre-scan) are
unbounded, but the receiver rejects a manifest whose **aggregate** count exceeds
`MAX_MANIFEST_ENTRIES` (1 048 576 entries across ALL sections) or whose aggregate
path bytes exceed `MAX_MANIFEST_BYTES` (16 MB across all sections) as a hard
protocol error. A heavily filtered source whose exclusion list grows large thus
fails the run cleanly on the receiver (STATUS_ERROR) instead of being silently
truncated. In the commit modes this only means the deletion is refused after the
data already arrived; in the early modes (`--delete-before`/`--delete-during`)
the manifest is the first frame, so an oversized manifest aborts the whole
transfer BEFORE any data is sent. Keep the source tree small enough for the
receiver's manifest caps when using the early timing.
Early-delete ACK wait: after committing a large deletion (up to
`MAX_SERVER_DELETE_COUNT` removals) the receiver's `STATUS_OK`/`STATUS_ERROR`
@@ -238,33 +261,33 @@ why plain `--append` works on the normal atomic path, not only with `--inplace`.
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-M`, `--preserve` | Preserve file metadata | ✅ Implemented | `--preserve` means `-p` + `-t` (mode + mtime); the wire metadata also carries uid/gid for `-o`/`-g`/`-a`, and ownership is applied via `-o`/`-g`, `-a`, or an explicit identity flag (`--numeric-ids`/`--usermap`/`--groupmap`/`--chown`/`--copy-as`) |
| `-p`, `--perms` | Preserve permissions | ✅ Implemented | Real per-attribute flag (protocol 2.22.0): `preserve_perms` applies the source mode independently of times/owner/group. A client-supplied mode never grants group/other write — `S_IWGRP|S_IWOTH` are always stripped (rsync's `-p` preserves them exactly). `--chmod` and `-A/--acls` also imply `-p`; `-X/--xattrs` does not. The SSH port moved to `--ssh-port`. rsync-parity short form |
| `-o`, `--owner` | Preserve owner | ✅ Implemented | Real per-attribute flag (`preserve_owner`): preserve the source uid, resolved on the receiver by name against its own user database with a raw-numeric fallback (only numeric ids cross the wire). `--usermap`/`--chown=USER` imply it. Application follows the `--super`/`--no-super` policy; a non-opted daemon module applies no ownership (see the Daemon Mode notes) |
| `-g`, `--group` | Preserve group | ✅ Implemented | Real per-attribute flag (`preserve_group`): preserve the source gid, resolved by name on the receiver with a raw-numeric fallback. `--groupmap`/`--chown=:GROUP` imply it. Same privilege/super-policy gating as `-o` |
| `-t`, `--times` | Preserve modification times | ✅ Implemented | Real per-attribute flag (`preserve_times`): apply the source mtime independently of the other attributes. `-O/--omit-dir-times` suppresses directories only and `-J/--omit-link-times` suppresses symlinks only; `-U`/`-N` do not imply it. `--preserve`/`-a` imply it, and `--incremental`/`--delta` auto-enable it unless `--no-times`/`--no-preserve` |
| `-E`, `--executability` | Preserve executability | ✅ Implemented | Preserves executable permission bits (implies metadata preservation) |
| `--chmod=CHMOD` | Affect file permissions | ✅ Implemented | Supports numeric and symbolic `ugo` `rwx` changes; retains receiver safety masking |
| `-A`, `--acls` | Preserve ACLs | ✅ Implemented | Implemented on Linux via the POSIX-ACL xattr representation: the sender captures the `system.posix_acl_access` / `system.posix_acl_default` xattrs into the same bounded whitelisted set as `-X`, transmits them per-file, and the receiver re-applies them fd-relative. Setting an ACL the receiver is not permitted to set (non-root on a file it does not own, unsupported filesystem) is logged and skipped, never fatal. libacl is **not** required. Only the `system.posix_acl_*` namespaces plus `user.*` are ever applied; privileged namespaces are never applied (see the Phase-4 xattr/ACL notes below). Implies metadata transmission |
| `-X`, `--xattrs` | Preserve extended attributes | ✅ Implemented | Preserves unprivileged `user.*` extended attributes (Linux `listxattr`/`getxattr` on capture, `fsetxattr` on the written destination fd). Both capture (sender) and application (receiver) are restricted to the `user.*` namespace and the two POSIX ACL xattrs, so a client can **never** force a `security.*`/`trusted.*`/privileged attribute onto the destination; the receiver independently re-validates every incoming name against this whitelist and rejects anything else. Payloads are bounded (per-name ≤255B, per-value ≤1MiB, per-file count ≤256 total bytes ≤4MiB) on both ends, and an oversized/malformed frame is a clean protocol rejection (no OOM). Applied fd-relative to the exact written file. Implies metadata transmission. Incompatible with `-s` (chunk serialization), rejected up front (see the notes); a `--link-dest`/`-H` hard-link copy fallback re-applies the attributes so they are not dropped when a link is refused |
| `-H`, `--hard-links` | Preserve hard links | ✅ Implemented | Files on the source that share an inode (`st_dev`+`st_ino`, e.g. a `cp -al` tree) are re-created as hard links to one another on the destination, so duplicate links stay deduplicated and only the first member's data is sent (later members are transmitted as payload-less `STATUS_HARDLINK` frames). The receiver links each sibling to the first member's installed file with an atomic link + rename; on `link()` failure it falls back to a byte-identical local copy of the first member, never a partial/corrupt file. Requires the sequential scan for ordering (the first member is always emitted and installed before any sibling is linked). Works single-threaded and under `-j`/`--threads`, `--inplace`, `--delay-updates` (links staged and published by rename) and `--partial`. Crosses the wire (`preserve_hard_links` bool; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0**, peers must match). Incompatible with `-s` (chunk serialization) and `--append`/`--append-verify`, rejected up front with a distinct error. See the Phase-4 hard-links notes below |
| `-D` | Same as --devices --specials | ✅ Implemented | Implies `--devices --specials`. `-D` was unassigned in FastSync (verified: no collision), so it is free to imply both device-node and special-file preservation. See the `--devices`/`--specials` rows and the Phase-4 devices notes below |
| `--devices` | Preserve device files | ✅ Implemented | Recreates char/block device nodes on the destination via `mknod` instead of transferring content. Type + rdev are validated strictly (S_IFMT from the transmitted mode; major/minor range-checked, non-negative), and creation is **privilege-gated**: `mknod` needs `CAP_MKNOD`, so a non-root receiver (CI runs via setpriv as non-root) logs a warning and **skips the device entry safely** — the whole transfer never aborts just because the node could not be made. The node is created fd-relative below the receive root (`mknodat` on the confined secure parent), so it can never be placed outside the authorized root, never follows a symlink, and never replaces an existing directory. Only a char/block mode is honored. Crosses the wire (a new `STATUS_SPECIAL` frame carries the path + metadata mode + rdev; `PROTOCOL_VERSION` bumped **2.12.0 → 2.13.0**). Divergence: per-entry skip (not a hard error) when the receiver lacks `CAP_MKNOD`, documented in the Phase-4 devices notes |
| `--specials` | Preserve special files | ⛔ Impossible/Divergence | **FIFO recreation works**: FIFOs are recreated on the destination via `mkfifo` (unprivileged, so this is a real, assertable behavior under CI). **Only socket recreation is impossible**: a socket entry can be created only by `bind(2)` on a live socket, not by any filesystem call, so a source socket is skipped with an explicit note. That one unsupported node kind is why the flag is classified Impossible/Divergence even though FIFO recreation itself works; its normal path is otherwise complete. FIFO creation is privileged-gated only in the sense of graceful skip on any permission failure. Node creation is confined below the receive root (`mkfifoat` on the secure fd-relative parent; no `..`, no symlink follow). Crosses the wire like `--devices` (the `STATUS_SPECIAL` frame; `PROTOCOL_VERSION` bumped **2.12.0 → 2.13.0**). See the Phase-4 devices notes |
| `--copy-devices` | Copy device contents as file | ✅ Implemented | Copy a device's CONTENT into an ordinary regular file on the destination instead of recreating the node — non-privileged and safe. FastSync scans a device/FIFO as a regular file: its reported size (`st_size`, typically 0 for char devices and FIFOs) is copied, so a FIFO or a non-readable device becomes an empty (or size-bounded) regular file. The default data path is size-bounded and never blocks (it sends exactly `st_size` bytes, never an unbounded pseudo-device stream); with `--sendfile`, a non-regular source (FIFO/device) is detected from its `stat` mode and falls back to that same buffered read, so `--copy-devices --sendfile` cannot hang either. The run always succeeds and never crashes on such input. **Deliberate, safe divergence from rsync's dd-like unbounded device read.** See the Phase-4 devices notes |
| `--write-devices` | Write to devices as files | ✅ Implemented | Write the received data directly into an **existing** device node on the destination instead of creating a regular file. Restricted and best-effort: the destination must already exist and be a char/block device (opened only under the confined receive root, with `O_NOFOLLOW` + `O_NONBLOCK`); a missing, symlinked, FIFO-with-no-reader (`ENXIO`), non-device destination, or any write failure is **skipped with a warning** rather than allowed, so a run can never clobber the system, never blocks on a special-file target, and never aborts on an unusable target. See the Phase-4 devices notes |
| `-U`, `--atimes` | Preserve access times | ✅ Implemented | Captures the source access time (from the scanner's pre-read stat, so it is not clobbered by reading the file for transfer) and transmits it over the wire; the receiver restores it together with the mtime via `futimens`/`utimensat`. Implies metadata transmission (the times travel inside the `-M` metadata payload), but does not enable ownership application (that stays opt-in via the identity flags). Wire: new `atime` fields on the metadata frame + a `preserve_atimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** |
| `-N`, `--crtimes` | Preserve create times | ⛔ Impossible/Divergence | Birth-times cannot be set by any portable filesystem call (`utimensat`/`futimens` only set atime/mtime), so this row is an explicit **Impossible/Divergence** (Phase 7 Wave B). Capture + transmit stays: `statx(STATX_BTIME)` on Linux records the source birth time as a wire field; the receiver logs a debug note that it cannot be applied and continues — never failing the transfer and never pretending it worked. On platforms without `statx` it parses as a documented no-op (flag accepted; nothing is captured). Implies metadata transmission. Wire: new `crtime` fields + a `preserve_crtimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** (see the Phase-4 metadata-time notes) |
| `-O`, `--omit-dir-times` | Omit dirs from --times | ✅ Implemented | Real modifier now that FastSync preserves directory times. With metadata on, the scanner captures every traversed source directory's mtime (and atime under `-U`) and the sender transmits them in trailing `STATUS_DIR_TIMES` frame(s) **after all file data and the optional delete manifest** (chunked at the receiver's `MAX_MANIFEST_ENTRIES` per-frame cap); a dir-time entry only RECORDS metadata and never creates the directory, so empty source directories stay untransferred. The receiver defers applying them until its delete / `--delay-updates` publication phases have committed, so writing or removing a child never clobbers a parent directory's mtime (rsync applies directory times at the end for exactly this reason). When `-O` is set (the boolean crosses the wire) the receiver does not apply any of them; without `-O` an `-a`/`--preserve` transfer now restores directory times (reversing the old "never preserves dir times" divergence). Wire change: the terminal `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** |
| `-J`, `--omit-link-times` | Omit symlinks from --times | ✅ Implemented | Real modifier now that FastSync preserves symlink times. Symlink entries already carried their metadata on `STATUS_SYMLINK`; the receiver now applies it with **no-follow primitives only** (`utimensat(..., AT_SYMLINK_NOFOLLOW)`, plus best-effort `fchmodat(..., AT_SYMLINK_NOFOLLOW)` and policy-gated `fchownat(..., AT_SYMLINK_NOFOLLOW)`), so the link itself is stamped without ever dereferencing it, confined fd-relative below the authorized receive root. A symlink has no children, so the times are applied immediately at creation. When `-J` is set (the boolean crosses the wire) the receiver skips the timestamps (mode/ownership are unaffected); without `-J` an `-a`/`-l` transfer restores symlink mtimes. Wire change alongside `-O`: the shared `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** |
| `--super` | Receiver attempts super-user activities | ✅ Implemented | Phase 7 Wave E: receiver-side **safe-subset + clear-refusal** privilege model, tri-state `super_mode` (auto/on/off). `--super` **permits** the receiver to attempt super-user activities — ownership application and char/block device-node creation — that are already confined fd-relative below the authorized receive root; `--no-super` **forbids** them even when the receiver is root; the default (`auto`) preserves the pre-existing **best-effort** behavior of *attempting* them (not only when already root: an unprivileged attempt is refused by the kernel and skipped per entry, matching FastSync's history). The server additionally accepts an operator-level `--no-super` veto that forces `OFF` for every connection it accepts (so it also refuses any client `--copy-as`/`--super`); a **privileged (root) standalone TCP listener now also defaults to `OFF`** unless the operator opts in with the new server-only `--allow-super` flag (the flag is **rejected with `--stdio`**, whose remote argv is composed by the client and must never defeat the secure default; operators exposing `fastsync-server --stdio` over SSH need a forced command if the default must hold. An unprivileged receiver is unchanged, since the kernel refuses the confined attempts anyway; the `--daemon` path keeps its per-module `client owner = yes` opt-in); the `--fake-super` owner replay and the `--write-devices` write path are gated by the same policy. **FastSync never elevates**: no `setuid`/`seteuid`/`setgid` is ever called, and `--super` never bypasses the confinement floor (`file_open_secure_parent`, `O_NOFOLLOW`, root checks) — it only permits an attempt that is already confined. `--super` does **not** imply `--numeric-ids` and never enables client-chosen ownership on its own: ownership is applied only when an explicit identity policy (`--usermap`/`--groupmap`/`--chown`/`--numeric-ids`/`--copy-as`) or a preserve-source request (`-o`/`-g`, or `-a`/`--archive`) is also given. A non-root receiver given `--super` logs exactly one warning at activation and each confined attempt is then refused by the kernel and skipped per entry (never aborts); `--no-super` suppresses ownership, char/block `mknod`, `--write-devices` and the fake-super owner replay, while unprivileged FIFO creation is unaffected. Wire: one trailing `super_mode` int on the config frame (validated 0..2), sent **before** the `--copy-as` block (fixed order: super int, then copy-as presence int + ids); `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Documented divergence from rsync:** rsync's `--super` runs the receiver with elevated privilege; FastSync only permits a confined attempt and never elevates |
| `--fake-super` | Store/recover privileged attrs via xattrs | ✅ Implemented | Phase 7 Wave B: full record **and replay**. The receiver writes the source `uid:gid:mode:mtime_sec:mtime_nsec` into a reserved `user.fastsync.stat` xattr on each written file (best-effort, fd-relative, format unchanged), then immediately re-applies it via `fake_super_restore_fd`: `fchown` (only where privileged — a non-root EPERM/EACCES is skipped silently, matching FastSync's identity philosophy), `fchmod`, and `futimens`. The OWNER leg is additionally skipped unless an explicit ownership identity policy (`--numeric-ids`/`--usermap`/`--groupmap`/`--chown`/`--copy-as`) or a preserve-source request (`-o`/`-g`, or `-a`/`--archive`) is active — `--fake-super` on its own only *records* the source owner and must not act as an un-gated chown primitive — when `--no-super` forbids super-user activities (even for root), or when an active `--copy-as` is authoritative, so the recorded source owner can never override a forced `--copy-as` owner; the xattr record is still stored/replayed for a later privileged restore and mode/mtime still apply, so unprivileged `--fake-super` keeps working. The restored mode goes through the same sanitization as the normal metadata path (group/other write bits are never granted, so a recorded 0666 restores as 0644), so fake-super replay can never grant group/other-write that plain `--preserve` would refuse. Absence or a malformed record is a silent no-op, never fatal. The recording format diverges from rsync's `user.rsync.%stat%`; no cross-tool conversion is attempted. Implies metadata transmission so the source uid/gid/mode/mtime are available. Both it and `-X`/`-A` are incompatible with `-s` (chunk serialization), rejected up front |
| `--open-noatime` | Avoid changing access time when opening files | ✅ Implemented | Sender-side policy: the sender opens source files with `O_NOATIME` (Linux) when reading them for transfer, so the open/read does NOT bump the source's on-disk access time. Degrades safely when `O_NOATIME` is unavailable (not defined) or refused (`EPERM`, since it needs `CAP_FOWNER` or file ownership): the code falls back to a normal open, so the data always transfers — only the atime-bump is skipped. It does not itself capture/preserve atime; it only avoids modifying it. **Client-only, never crosses the wire.** Exposed as `file_open_for_read()` and applied to both the buffered data path and the sendfile path |
| `--numeric-ids` | Do not map uid/gid by name | ✅ Implemented | Ownership is applied through FastSync's opt-in identity path (see the Phase-4 identity notes below). `--numeric-ids` is a mapping-policy modifier: when applying ownership it uses the transmitted numeric uid/gid directly, skipping the name lookup. Without an ownership-affecting option it is inert (FastSync only applies ownership when the user opts in). It does not need `-M` to be parsed, but ownership is only applied when metadata (hence the source uid/gid) is actually transmitted (see the notes) |
| `--usermap=STRING` | Map usernames | ✅ Implemented | Opt-in ownership application. rsync subset implemented: comma-separated `FROM:TO` rules evaluated in order, first match wins; `FROM`/`TO` are group/user names (resolved on the SOURCE machine at parse time), `*` (FROM matches any id / TO = the receiving process's current euid), and an `@N` or bare `N` numeric id. Rules are carried over the wire as resolved numeric id pairs; the receiver applies a matching rule (else falls back to `--chown`, `--numeric-ids`, then a best-effort name lookup) via an fd-relative `fchown`. Malformed/unresolvable specs are rejected with a clear error, never a silent no-op. Implies metadata preservation so the source uid/gid travel. Only effective when the receiver can actually change ownership (root or membership); otherwise it warns and continues |
| `--groupmap=STRING` | Map group names | ✅ Implemented | Same rsync subset and semantics as `--usermap` but for the group (gid) side and the group databases. See the Phase-4 identity notes |
| `--chown=USER:GROUP` | Map owner and group | ✅ Implemented | Opt-in ownership override applied receiver-side. Forms: `USER:GROUP`, `USER` (owner only), `:GROUP` (group only); a `*` for USER/GROUP means the current/root user or group as appropriate; an `@N`/bare `N` numeric id is accepted. A `:` inside a name may be escaped as `\:`. Equivalent to a trailing `*:*` usermap+groupmap rule (so an explicit `--usermap`/`--groupmap` match wins). Malformed or unresolvable specs are clear parse errors. Implies metadata preservation. Only effective when the receiver has permission to chown; otherwise it warns and continues (rsync parity) |
| `--copy-as=USER[:GROUP]` | Perform the copy as another user/group | ✅ Implemented | Safe-subset implementation, an explicit divergence from rsync's **real identity switching**. rsync makes the receiving process actually assume USER/GROUP (setuid/setgid); FastSync's receiver is multithreaded, so a real credential drop would be unsafe and is never attempted — FastSync never calls `setuid`/`seteuid`/`setgid`. Instead the receiver FORCES the ownership of every entry it writes to `copy_as_uid`/`copy_as_gid` through the existing confined, fd-relative identity path (the same `fchown`/`fchownat` mechanism as `--chown`/`--usermap`/`--groupmap`; symlinks use `fchownat(..., AT_SYMLINK_NOFOLLOW)`, and directories — including intermediate parents created implicitly while writing a nested file — and char/block/FIFO nodes are owned no-follow too, so a directory never keeps the receiver's owner while its children get the target owner), with `--copy-as` at the **highest priority** — it beats usermap/groupmap/`--chown`/`--numeric-ids` and the best-effort name lookup. This REQUIRES a privileged (root) receiver: an unprivileged receiver REFUSES the whole transfer up front at the config handshake (`server_module_gate`, running inside `config_receive_with_validate` before the `STATUS_OK` ack) with a clear error and no file data exchanged — never a silent wrong-ownership result. A server running with an operator `--no-super` veto also refuses it; a privileged (root) standalone TCP listener refuses it by default too and only honors it after the operator passes `--allow-super` (the flag is rejected with `--stdio`, where the client-composed remote argv could otherwise defeat the default; a forced command is required if the default must hold), and a **daemon** refuses `--copy-as`, like every other client-chosen-ownership request (`--numeric-ids`/`--chown`/`--usermap`/`--groupmap`/`--fake-super`/explicit `--super`), unless the selected module opts in with `client owner = yes`; without that per-module opt-in a daemon must not honor an arbitrary client-selected owner (a root standalone listener honors these for its single operator-authorized root only when started with `--allow-super`). `--fake-super` interaction: `--copy-as` is authoritative, so the recorded source owner is never replayed over the forced target owner. If the ownership apply still fails with EPERM/EACCES (capability-restricted root, root-squash, read-only mount) the failure is logged at ERROR and the **entry is reported as failed** rather than written with the wrong owner, which fails the transfer (fail-fast) so overall success is never reported with the wrong owner. USER is resolved on the client against the user database (a name, an `@N`/bare `N` numeric id, or `*` meaning the client's current euid); when `:GROUP` is present it is resolved against the group database (`*` meaning the client's egid). **Group-default rule:** when the group is omitted FastSync uses the user's primary gid (`getpwuid(uid)->pw_gid`); a numeric id with no local passwd entry has no primary gid to look up, so `gid` falls back to `uid` (documented divergence). Malformed/empty/unresolvable specs are clear parse errors, never a silent no-op. Never elevates privileges and never bypasses the confined receive root. Implies metadata preservation (the source uid/gid must be transmitted). Wire: a new trailing config-frame block **sent after** the `--super` int (presence int, then the two int32 ids, both validated `>= 0` on receive; the ids are also rejected if they do not fit int32 at CLI parse time); `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0** |
| `--preserve` | (FastSync alias, not an rsync flag) | ✅ Parity | **FastSync-only alias** for `-p` + `-t` (mode + mtime), long-form only. It is not rsync's `--preserve` (rsync has no such option); the short `-M` that used to spell it is now rsync's `--remote-option`. The wire metadata also carries uid/gid for `-o`/`-g`/`-a`, and ownership is applied via `-o`/`-g`, `-a`, or an explicit identity flag (`--numeric-ids`/`--usermap`/`--groupmap`/`--chown`/`--copy-as`) |
| `-p`, `--perms` | Preserve permissions | ✅ Parity | Real per-attribute flag (protocol 2.22.0): `preserve_perms` applies the source mode independently of times/owner/group. **Strict rsync parity (protocol 2.23.0): the source mode is copied exactly, including setuid/setgid/sticky and group/other-write bits — there is no masking.** Without `-p`, a new file gets `source_mode & ~umask` when metadata is present (else the historical fixed `0644`); new directories without `-p` still use FastSync's `0755` creation default, because directory metadata is only applied when a directory attribute is requested. `-A/--acls` implies `-p`; `--chmod` does **not** imply `-p` (rsync parity) and applies its own unsanitized changes to the new mode. `-X/--xattrs` does not imply `-p`. The SSH port moved to `--ssh-port`. rsync-parity short form |
| `-o`, `--owner` | Preserve owner | ✅ Parity | Real per-attribute flag (`preserve_owner`): preserve the source uid, resolved on the receiver by name against its own user database with a raw-numeric fallback (only numeric ids cross the wire). `--usermap`/`--chown=USER` imply it. Application follows the `--super`/`--no-super` policy; a non-opted daemon module applies no ownership (see the Daemon Mode notes) |
| `-g`, `--group` | Preserve group | ✅ Parity | Real per-attribute flag (`preserve_group`): preserve the source gid, resolved by name on the receiver with a raw-numeric fallback. `--groupmap`/`--chown=:GROUP` imply it. Same privilege/super-policy gating as `-o` |
| `-t`, `--times` | Preserve modification times | ✅ Parity | Real per-attribute flag (`preserve_times`): apply the source mtime independently of the other attributes. `-O/--omit-dir-times` suppresses directories only and `-J/--omit-link-times` suppresses symlinks only; `-U`/`-N` do not imply it. `--preserve`/`-a` imply it, and `--incremental`/`--delta` auto-enable it unless `--no-times`/`--no-preserve` |
| `-E`, `--executability` | Preserve executability | ✅ Parity | Preserves executable permission bits (implies metadata preservation) |
| `--chmod=CHMOD` | Affect file permissions | ✅ Parity | Faithful port of rsync 3.4.1's `parse_chmod`/`tweak_mode`: numeric octal and symbolic `ugo`/`rwx` changes, `D`/`F` directory/file selectors, `X` (execute only on directories or already-executable files), `s`/`t` setuid/setgid/sticky, and append semantics — repeated clauses and repeated `--chmod` options accumulate in order (joined with commas). The changes are applied to the new mode **without sanitization** (matching rsync) and `--chmod` does **not** imply `-p` (rsync parity). Applied to files and directories on the receiver |
| `-A`, `--acls` | Preserve ACLs | ⚠️ Caveat | Implemented on Linux via the POSIX-ACL xattr representation: the sender captures the `system.posix_acl_access` / `system.posix_acl_default` xattrs into the same bounded whitelisted set as `-X`, transmits them per-file, and the receiver re-applies them fd-relative. Setting an ACL the receiver is not permitted to set (non-root on a file it does not own, unsupported filesystem) is logged and skipped, never fatal. libacl is **not** required. Only the `system.posix_acl_*` namespaces plus `user.*` are ever applied; privileged namespaces are never applied (see the Phase-4 xattr/ACL notes below). Implies metadata transmission |
| `-X`, `--xattrs` | Preserve extended attributes | ⚠️ Caveat | Preserves unprivileged `user.*` extended attributes (Linux `listxattr`/`getxattr` on capture, `fsetxattr` on the written destination fd). Both capture (sender) and application (receiver) are restricted to the `user.*` namespace and the two POSIX ACL xattrs, so a client can **never** force a `security.*`/`trusted.*`/privileged attribute onto the destination; the receiver independently re-validates every incoming name against this whitelist and rejects anything else. Payloads are bounded (per-name ≤255B, per-value ≤1MiB, per-file count ≤256 total bytes ≤4MiB) on both ends, and an oversized/malformed frame is a clean protocol rejection (no OOM). Applied fd-relative to the exact written file. Implies metadata transmission. Incompatible with `-s` (chunk serialization), rejected up front (see the notes); a `--link-dest`/`-H` hard-link copy fallback re-applies the attributes so they are not dropped when a link is refused |
| `-H`, `--hard-links` | Preserve hard links | ✅ Parity | Files on the source that share an inode (`st_dev`+`st_ino`, e.g. a `cp -al` tree) are re-created as hard links to one another on the destination, so duplicate links stay deduplicated and only the first member's data is sent (later members are transmitted as payload-less `STATUS_HARDLINK` frames). The receiver links each sibling to the first member's installed file with an atomic link + rename; on `link()` failure it falls back to a byte-identical local copy of the first member, never a partial/corrupt file. Requires the sequential scan for ordering (the first member is always emitted and installed before any sibling is linked). Works single-threaded and under `-j`/`--threads`, `--inplace`, `--delay-updates` (links staged and published by rename) and `--partial`. Crosses the wire (`preserve_hard_links` bool; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0**, peers must match). Incompatible with `-s` (chunk serialization) and `--append`/`--append-verify`, rejected up front with a distinct error. See the Phase-4 hard-links notes below |
| `-D` | Same as --devices --specials | ✅ Parity | Implies `--devices --specials`. `-D` was unassigned in FastSync (verified: no collision), so it is free to imply both device-node and special-file preservation. As of protocol 2.23.0 `--specials` genuinely covers **both FIFOs and unix sockets**, so `-D` covers the full rsync set. See the `--devices`/`--specials` rows and the Phase-4 devices notes below |
| `--devices` | Preserve device files | ⚠️ Caveat | Recreates char/block device nodes on the destination via `mknod` instead of transferring content. Type + rdev are validated strictly (S_IFMT from the transmitted mode; major/minor range-checked, non-negative), and creation is **privilege-gated**: `mknod` needs `CAP_MKNOD`, so a non-root receiver (CI runs via setpriv as non-root) logs a warning and **skips the device entry safely** — the whole transfer never aborts just because the node could not be made. The node is created fd-relative below the receive root (`mknodat` on the confined secure parent), so it can never be placed outside the authorized root, never follows a symlink, and never replaces an existing directory. Only a char/block mode is honored. Crosses the wire (a `STATUS_SPECIAL` frame carries the path + metadata mode + rdev). Divergence: per-entry skip (not a hard error) when the receiver lacks `CAP_MKNOD`, documented in the Phase-4 devices notes |
| `--specials` | Preserve special files | ✅ Parity | **FIFO and unix-socket recreation work** (protocol 2.23.0): FIFOs are recreated with `mkfifoat`, and sockets with `mknodat(..., S_IFSOCK)` — the latter is unprivileged on Linux because it materializes the socket *node*, not a live bound socket, so it is a real, assertable behavior under CI (it matches rsync, which also recreates a socket by `mknod`). Node creation is confined below the receive root (fd-relative parent; no `..`, no symlink follow) and type/rdev are validated strictly; a matching existing node is left in place and an unrelated entry is never replaced. Crosses the wire like `--devices` (the `STATUS_SPECIAL` frame). See the Phase-4 devices notes |
| `--copy-devices` | Copy device contents as file | ⚠️ Caveat | Copy a device's CONTENT into an ordinary regular file on the destination instead of recreating the node — non-privileged and safe. FastSync scans a device/FIFO as a regular file: its reported size (`st_size`, typically 0 for char devices and FIFOs) is copied, so a FIFO or a non-readable device becomes an empty (or size-bounded) regular file. The default data path is size-bounded and never blocks (it sends exactly `st_size` bytes, never an unbounded pseudo-device stream); with `--sendfile`, a non-regular source (FIFO/device) is detected from its `stat` mode and falls back to that same buffered read, so `--copy-devices --sendfile` cannot hang either. The run always succeeds and never crashes on such input. **Deliberate, safe divergence from rsync's dd-like unbounded device read.** See the Phase-4 devices notes |
| `--write-devices` | Write to devices as files | ⚠️ Caveat | Write the received data directly into an **existing** device node on the destination instead of creating a regular file. Restricted and best-effort: the destination must already exist and be a char/block device (opened only under the confined receive root, with `O_NOFOLLOW` + `O_NONBLOCK`); a missing, symlinked, FIFO-with-no-reader (`ENXIO`), non-device destination, or any write failure is **skipped with a warning** rather than allowed, so a run can never clobber the system, never blocks on a special-file target, and never aborts on an unusable target. See the Phase-4 devices notes |
| `-U`, `--atimes` | Preserve access times | ✅ Parity | Captures the source access time (from the scanner's pre-read stat, so it is not clobbered by reading the file for transfer) and transmits it over the wire; the receiver restores it together with the mtime via `futimens`/`utimensat`. Implies metadata transmission (the times travel inside the shared metadata payload), but does not enable ownership application (that stays opt-in via the identity flags). Wire: `atime` fields on the metadata frame + a `preserve_atimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** |
| `-N`, `--crtimes` | Preserve create times | ❌ Divergent | Birth-times cannot be set by any portable filesystem call (`utimensat`/`futimens` only set atime/mtime), so this row is an explicit **Divergent** entry (Phase 7 Wave B). Capture + transmit stays: `statx(STATX_BTIME)` on Linux records the source birth time as a wire field; the receiver logs a debug note that it cannot be applied and continues — never failing the transfer and never pretending it worked. On platforms without `statx` it parses as a documented no-op (flag accepted; nothing is captured). Implies metadata transmission. Wire: new `crtime` fields + a `preserve_crtimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** (see the Phase-4 metadata-time notes) |
| `-O`, `--omit-dir-times` | Omit dirs from --times | ✅ Parity | Real modifier now that FastSync preserves directory times. With metadata on, the scanner captures every traversed source directory's mtime (and atime under `-U`) and the sender transmits them in trailing `STATUS_DIR_TIMES` frame(s) **after all file data and the optional delete manifest** (chunked at the receiver's `MAX_MANIFEST_ENTRIES` per-frame cap); a dir-time entry only RECORDS metadata and never creates the directory, so empty source directories stay untransferred. The receiver defers applying them until its delete / `--delay-updates` publication phases have committed, so writing or removing a child never clobbers a parent directory's mtime (rsync applies directory times at the end for exactly this reason). When `-O` is set (the boolean crosses the wire) the receiver does not apply any of them; without `-O` an `-a`/`--preserve` transfer now restores directory times (reversing the old "never preserves dir times" divergence). Wire change: the terminal `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** |
| `-J`, `--omit-link-times` | Omit symlinks from --times | ✅ Parity | Real modifier now that FastSync preserves symlink times. Symlink entries already carried their metadata on `STATUS_SYMLINK`; the receiver now applies it with **no-follow primitives only** (`utimensat(..., AT_SYMLINK_NOFOLLOW)`, plus best-effort `fchmodat(..., AT_SYMLINK_NOFOLLOW)` and policy-gated `fchownat(..., AT_SYMLINK_NOFOLLOW)`), so the link itself is stamped without ever dereferencing it, confined fd-relative below the authorized receive root. A symlink has no children, so the times are applied immediately at creation. When `-J` is set (the boolean crosses the wire) the receiver skips the timestamps (mode/ownership are unaffected); without `-J` an `-a`/`-l` transfer restores symlink mtimes. Wire change alongside `-O`: the shared `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** |
| `--super` | Receiver attempts super-user activities | ⚠️ Caveat | Phase 7 Wave E: receiver-side **safe-subset + clear-refusal** privilege model, tri-state `super_mode` (auto/on/off). `--super` **permits** the receiver to attempt super-user activities — ownership application and char/block device-node creation — that are already confined fd-relative below the authorized receive root; `--no-super` **forbids** them even when the receiver is root; the default (`auto`) preserves the pre-existing **best-effort** behavior of *attempting* them (not only when already root: an unprivileged attempt is refused by the kernel and skipped per entry, matching FastSync's history). The server additionally accepts an operator-level `--no-super` veto that forces `OFF` for every connection it accepts (so it also refuses any client `--copy-as`/`--super`); a **privileged (root) standalone TCP listener now also defaults to `OFF`** unless the operator opts in with the new server-only `--allow-super` flag (the flag is **rejected with `--stdio`**, whose remote argv is composed by the client and must never defeat the secure default; operators exposing `fastsync-server --stdio` over SSH need a forced command if the default must hold. An unprivileged receiver is unchanged, since the kernel refuses the confined attempts anyway; the `--daemon` path keeps its per-module `client owner = yes` opt-in); the `--fake-super` owner replay and the `--write-devices` write path are gated by the same policy. **FastSync never elevates**: no `setuid`/`seteuid`/`setgid` is ever called, and `--super` never bypasses the confinement floor (`file_open_secure_parent`, `O_NOFOLLOW`, root checks) — it only permits an attempt that is already confined. `--super` does **not** imply `--numeric-ids` and never enables client-chosen ownership on its own: ownership is applied only when an explicit identity policy (`--usermap`/`--groupmap`/`--chown`/`--numeric-ids`/`--copy-as`) or a preserve-source request (`-o`/`-g`, or `-a`/`--archive`) is also given. A non-root receiver given `--super` logs exactly one warning at activation and each confined attempt is then refused by the kernel and skipped per entry (never aborts); `--no-super` suppresses ownership, char/block `mknod`, `--write-devices` and the fake-super owner replay, while unprivileged FIFO creation is unaffected. Wire: one trailing `super_mode` int on the config frame (validated 0..2), sent **before** the `--copy-as` block (fixed order: super int, then copy-as presence int + ids); `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Documented divergence from rsync:** rsync's `--super` runs the receiver with elevated privilege; FastSync only permits a confined attempt and never elevates |
| `--fake-super` | Store/recover privileged attrs via xattrs | ⚠️ Caveat | Full record **and replay** (protocol 2.23.0 parity update). The receiver writes the resolved `uid:gid:mode:mtime_sec:mtime_nsec` into a reserved `user.fastsync.stat` xattr on each written file (best-effort, fd-relative), then immediately re-applies the mode and times via `fake_super_restore_fd` (`fchmod` + `futimens`; absent/malformed records are a silent no-op, never fatal). **`--fake-super` never performs a real `chown`**: when an explicit ownership mapping (`--chown`/`--usermap`/`--groupmap`/`--copy-as`) is active the receiver records the *resolved* id, otherwise the source's own id, but the owner leg is always suppressed so recording can never defeat the flag; the record is retained for a later privileged restore. The replayed mode goes through the shared `metadata_mode_for_policy` helper, so under `-p` it is copied exactly (including group/other-write and special bits — strict rsync parity, no masking) and under `-E` it follows the rsync executability rule. Directory ownership and directory xattrs/ACLs are preserved alongside file entries (mode/owner are applied to directories under the same per-attribute policy and `-A`/`-X` carry the directory ACL/xattr block). Implies metadata transmission so the source uid/gid/mode/mtime are available. The recording format diverges from rsync's `user.rsync.%stat%`; no cross-tool conversion is attempted. Both it and `-X`/`-A` are incompatible with `-s` (chunk serialization), rejected up front |
| `--open-noatime` | Avoid changing access time when opening files | ✅ Parity | Sender-side policy: the sender opens source files with `O_NOATIME` (Linux) when reading them for transfer, so the open/read does NOT bump the source's on-disk access time. Degrades safely when `O_NOATIME` is unavailable (not defined) or refused (`EPERM`, since it needs `CAP_FOWNER` or file ownership): the code falls back to a normal open, so the data always transfers — only the atime-bump is skipped. It does not itself capture/preserve atime; it only avoids modifying it. **Client-only, never crosses the wire.** Exposed as `file_open_for_read()` and applied to both the buffered data path and the sendfile path |
| `--numeric-ids` | Do not map uid/gid by name | ✅ Parity | **A mapping modifier only:** when ownership is being applied it uses the transmitted numeric uid/gid directly, skipping the name lookup. It does **not** request ownership application on its own — combine it with `-o`/`-g`, `-a`, or an explicit map (`--chown`/`--usermap`/`--groupmap`) — and it does not need any metadata flag merely to parse. Ownership is only applied when metadata (hence the source uid/gid) is actually transmitted (see the Phase-4 identity notes) |
| `--usermap=STRING` | Map usernames | ⚠️ Caveat | Opt-in ownership application. rsync subset implemented (protocol 2.23.0): comma-separated `FROM:TO` rules evaluated in order, first match wins. `FROM` accepts a user name (resolved on the SOURCE machine at parse time), an `@N`/bare `N` numeric id, an inclusive `LOW-HIGH` **id range**, `*` (matches any id), or an **empty** field (matches ids with no name on the source). `TO` accepts a name (resolved on the **receiver**), an `@N`/bare `N` id, or `*` (the receiving process's current euid). Rules are carried over the wire as resolved numeric id pairs; the receiver applies a matching rule (else falls back to `--chown`, `--numeric-ids`, then a best-effort name lookup) via an fd-relative `fchown`, including directory entries. Malformed/unresolvable specs are rejected with a clear error, never a silent no-op. Implies metadata preservation so the source uid/gid travel. Only effective when the receiver can actually change ownership (root or membership); otherwise it warns and continues |
| `--groupmap=STRING` | Map group names | ⚠️ Caveat | Same rsync subset and semantics as `--usermap` (names, `@N`/bare `N`, inclusive ranges, `*`, empty-FROM for unnamed ids, receiver-resolved `TO` names) but for the group (gid) side and the group databases. See the Phase-4 identity notes |
| `--chown=USER:GROUP` | Map owner and group | ⚠️ Caveat | Opt-in ownership override applied receiver-side. Forms: `USER:GROUP`, `USER` (owner only), `:GROUP` (group only); a `*` for USER/GROUP means the current/root user or group as appropriate; an `@N`/bare `N` numeric id is accepted. A `:` inside a name may be escaped as `\:`. Equivalent to a trailing `*:*` usermap+groupmap rule (so an explicit `--usermap`/`--groupmap` match wins). **Protocol 2.23.0 makes `--chown` and `--usermap`/`--groupmap` mutually exclusive on the same side: combining them (in either order) is a clear configuration error** (`--usermap conflicts with prior --chown`), matching rsync and never an order-dependent silent winner. Malformed or unresolvable specs are clear parse errors. Implies metadata preservation. Only effective when the receiver has permission to chown; otherwise it warns and continues (rsync parity) |
| `--copy-as=USER[:GROUP]` | Perform the copy as another user/group | ⚠️ Caveat | Safe-subset implementation, an explicit divergence from rsync's **real identity switching**. rsync makes the receiving process actually assume USER/GROUP (setuid/setgid); FastSync's receiver is multithreaded, so a real credential drop would be unsafe and is never attempted — FastSync never calls `setuid`/`seteuid`/`setgid`. Instead the receiver FORCES the ownership of every entry it writes to `copy_as_uid`/`copy_as_gid` through the existing confined, fd-relative identity path (the same `fchown`/`fchownat` mechanism as `--chown`/`--usermap`/`--groupmap`; symlinks use `fchownat(..., AT_SYMLINK_NOFOLLOW)`, and directories — including intermediate parents created implicitly while writing a nested file — and char/block/FIFO nodes are owned no-follow too, so a directory never keeps the receiver's owner while its children get the target owner), with `--copy-as` at the **highest priority** — it beats usermap/groupmap/`--chown`/`--numeric-ids` and the best-effort name lookup. This REQUIRES a privileged (root) receiver: an unprivileged receiver REFUSES the whole transfer up front at the config handshake (`server_module_gate`, running inside `config_receive_with_validate` before the `STATUS_OK` ack) with a clear error and no file data exchanged — never a silent wrong-ownership result. A server running with an operator `--no-super` veto also refuses it; a privileged (root) standalone TCP listener refuses it by default too and only honors it after the operator passes `--allow-super` (the flag is rejected with `--stdio`, where the client-composed remote argv could otherwise defeat the default; a forced command is required if the default must hold), and a **daemon** refuses `--copy-as`, like every other client-chosen-ownership request (`--numeric-ids`/`--chown`/`--usermap`/`--groupmap`/`--fake-super`/explicit `--super`), unless the selected module opts in with `client owner = yes`; without that per-module opt-in a daemon must not honor an arbitrary client-selected owner (a root standalone listener honors these for its single operator-authorized root only when started with `--allow-super`). `--fake-super` interaction: `--copy-as` is authoritative, so the recorded source owner is never replayed over the forced target owner. If the ownership apply still fails with EPERM/EACCES (capability-restricted root, root-squash, read-only mount) the failure is logged at ERROR and the **entry is reported as failed** rather than written with the wrong owner, which fails the transfer (fail-fast) so overall success is never reported with the wrong owner. USER is resolved on the client against the user database (a name, an `@N`/bare `N` numeric id, or `*` meaning the client's current euid); when `:GROUP` is present it is resolved against the group database (`*` meaning the client's egid). **Group-default rule:** when the group is omitted FastSync uses the user's primary gid (`getpwuid(uid)->pw_gid`); a numeric id with no local passwd entry has no primary gid to look up, so `gid` falls back to `uid` (documented divergence). Malformed/empty/unresolvable specs are clear parse errors, never a silent no-op. Never elevates privileges and never bypasses the confined receive root. Implies metadata preservation (the source uid/gid must be transmitted). Wire: a new trailing config-frame block **sent after** the `--super` int (presence int, then the two int32 ids, both validated `>= 0` on receive; the ids are also rejected if they do not fit int32 at CLI parse time); `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0** |
**Phase-4 metadata-time notes:** `-U/--atimes`, `-N/--crtimes`,
`-O/--omit-dir-times`, `-J/--omit-link-times`, and `--open-noatime` are new.
@@ -326,8 +349,13 @@ match, exactly as prior phases did).
fatal.
- **`--fake-super`**: see the row above; the reserved key is `user.fastsync.stat`
with the documented `uid:gid:mode:mtime_sec:mtime_nsec` (mode octal) format.
It is honest but partial — there is no replay, and it does not interoperate
with rsync's `user.rsync.%stat%`.
**Replay exists**: after each stored record the receiver immediately re-applies
the recorded mode and times fd-relative (`fake_super_restore_fd`), but it
deliberately never performs a real `chown` — `--fake-super` only *records*
the resolved owner (the active `--chown`/`--usermap`/`--groupmap`/`--copy-as`
mapping when one is in effect, otherwise the source's own id) for a later
privileged restore. The recording format diverges from rsync's
`user.rsync.%stat%`; no cross-tool conversion is attempted.
- **Chunk serialization (`-s`) incompatibility:** the per-file xattr block rides
the streaming per-file frame, which `-s` replaces with a fixed buffer format,
so `-X` / `-A` combined with `-s` is rejected up front on both ends (mirroring
@@ -359,7 +387,7 @@ symlink timestamps (ownership/mode application is unaffected and stays governed
by the identity opt-in). Both config booleans already crossed the wire. See the
`-O`/`-J` rows and the Wave D note below.
**-U/-N and -M interaction:** because FastSync carries all metadata (mode, uid,
**-U/-N and metadata-bundle interaction:** because FastSync carries all metadata (mode, uid,
gid, mtime, and now atime/crtime) in one bounded payload that is only sent when
metadata transmission is on, `-U` and `-N` imply metadata transmission (the
times travel inside that payload). They do **not** enable ownership application,
@@ -458,19 +486,22 @@ CI runs the integration suite as a NON-ROOT user (via setpriv), so `mknod` fails
with `EPERM`. The receiver treats this as a graceful, logged *skip of the entry*
returned as a success/skip outcome — the whole transfer NEVER aborts just because
the environment cannot create the node. `mkfifo` (FIFOs) is unprivileged, so
`--specials` FIFO creation is a real, assertable behavior under CI; sockets cannot
be recreated by any standard filesystem call and are skipped with an explicit
note. The "device actually created" integration assertions are guarded to run
only as root. User-facing expectation: point `--devices` at devices and a
non-root receiver will faithfully skip them while transferring everything else.
`--specials` FIFO creation is a real, assertable behavior under CI. **Sockets are
recreated too** (protocol 2.23.0) with `mknodat(..., S_IFSOCK)`: Linux allows an
unprivileged `mknod` of a socket node because no live bound socket is created,
so a source socket materializes as a socket-type filesystem entry exactly as
rsync does. The "device actually created" integration assertions are guarded to
run only as root. User-facing expectation: point `--devices` at devices and a
non-root receiver will faithfully skip them while transferring everything else;
`--specials` recreates FIFOs and socket nodes for any receiver.
**Confinement & validation:** a special/device node is created with
`mknodat`/`mkfifoat` on the parent directory opened fd-relative below the receive
root (`file_open_secure_parent`: `O_NOFOLLOW`, no `..` components, root-checked),
so a node can never be created outside the authorized destination root and never
through a symlinked parent. The transmitted type is derived ONLY from the
validated S_IFMT bits of the metadata mode (char/block/FIFO honored, socket
skipped, regular/dir rejected as an invalid special), and the transmitted rdev is
validated S_IFMT bits of the metadata mode (char/block/FIFO and socket honored;
regular/dir rejected as an invalid special), and the transmitted rdev is
validated both on the wire (`file_receive_special`, `chunk_deserialize`) and at
the creation site (`file_special_rdev_valid`): a negative, oversize, or
non-device-carrying rdev is rejected outright (receiver aborts the frame), and a
@@ -496,13 +527,13 @@ warning + skip, never a system-clobbering write or an abort.
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-l`, `--links` | Copy symlinks as symlinks | ✅ Implemented | A symlink is transmitted as a real symlink: its target string crosses the wire (a new `STATUS_SYMLINK` frame / chunk entry type) and the receiver creates it with `symlinkat` beneath the receive root. This makes the previously-`-l`-included-but-targetless symlink handling complete. See the Phase-4 symlink-trust notes |
| `-L`, `--copy-links` | Transform symlink to referent | ✅ Implemented | `copy_links` config field |
| `--copy-unsafe-links` | Transform unsafe symlinks | ✅ Implemented | `copy_unsafe_links` config field |
| `--safe-links` | Ignore symlinks outside tree | ✅ Implemented | `safe_links` config field |
| `--munge-links` | Munge symlinks for safety | ✅ Implemented | Sender rewrites each transmitted symlink target with a `#SYMLINK/` marker; a target that could escape the receive root (absolute or containing `..`) is never transmitted (contained/skipped); the receiver strips the marker to restore the real target. See the Phase-4 symlink-trust notes |
| `-k`, `--copy-dirlinks` | Transform symlink to dir | ✅ Implemented | A symlink whose referent is a directory is dereferenced and recursed as a real directory; a symlink to a regular file stays a symlink. Sender-side only. See the Phase-4 symlink-trust notes |
| `-K`, `--keep-dirlinks` | Treat symlinked dir as dir | ✅ Implemented | On the receiver, an existing destination symlink-to-a-directory is used as that directory (followed) instead of being replaced; it is followed only when it resolves to a directory that stays beneath the receive root. See the Phase-4 symlink-trust notes |
| `-l`, `--links` | Copy symlinks as symlinks | ⚠️ Caveat | A symlink is transmitted as a real symlink: its target string crosses the wire (`STATUS_SYMLINK` / chunk entry type) and the receiver creates it with `symlinkat` beneath the receive root, never following the target. **Targets are stored verbatim (protocol 2.23.0), matching rsync `-l`: an absolute target or one containing `..` is copied exactly, and the receiver no longer enforces a containment predicate by default.** `--safe-links` is the sender-side opt-in that drops unsafe targets before transmission; `--trust-sender` does **not** affect symlink targets (it only relaxes the receiver's path-list re-validation). The *placement* path is still hard-confined (`has_path_traversal`, O_NOFOLLOW fd walk), and the link's own mode/times are applied with no-follow primitives. See the Phase-4 symlink-trust notes and the residual-risk note below |
| `-L`, `--copy-links` | Transform symlink to referent | ⚠️ Caveat | Sender-side: every symlink is replaced by its referent's content (`copy_links` config field). A referent that cannot be read, including a broken symlink, is treated as a non-error and the run exits 0 — where rsync exits 23 (`RERR_PARTIAL`). This is the documented status-code divergence |
| `--copy-unsafe-links` | Transform unsafe symlinks | ⚠️ Caveat | Sender-side: only symlinks whose target is unsafe (absolute or escaping via `..`, matching rsync's `unsafe_symlink()` semantics) are dereferenced into their referent; safe links stay symlinks. Same broken-referent exit-0 caveat as `-L` (`copy_unsafe_links` config field) |
| `--safe-links` | Ignore symlinks outside tree | ✅ Parity | Sender-side: a symlink whose target is unsafe is not transmitted at all (skipped), matching rsync's `--safe-links`. Because FastSync applies this while scanning the source, the receiver does not need to repeat it (`safe_links` config field) |
| `--munge-links` | Munge symlinks for safety | ✅ Parity | Sender rewrites each transmitted symlink target with rsync's `/rsyncd-munged/` prefix; the receiver strips the marker (only when the negotiated `munge_links` policy is on, so a source link that genuinely begins with the marker round-trips verbatim) and restores the exact real target. Unlike rsync, FastSync prefixes on the *sender* and un-munges on the receiver, but the wire result and the stored marker match rsync. See the Phase-4 symlink-trust notes |
| `-k`, `--copy-dirlinks` | Transform symlink to dir | ✅ Parity | A symlink whose referent is a directory is dereferenced and recursed as a real directory; a symlink to a regular file stays a symlink. Sender-side only. See the Phase-4 symlink-trust notes |
| `-K`, `--keep-dirlinks` | Treat symlinked dir as dir | ✅ Parity | On the receiver, an existing destination symlink-to-a-directory is used as that directory (followed) instead of being replaced; it is followed only when it resolves to a directory that stays beneath the receive root. See the Phase-4 symlink-trust notes |
**Phase-4 symlink-trust notes:** `-l/--links`, `-k/--copy-dirlinks`,
`-K/--keep-dirlinks`, and `--munge-links` form the "symlink trust boundaries"
@@ -518,17 +549,20 @@ was bumped **2.12.0 → 2.13.0** (peers must match, exactly as prior phases did)
**Per-flag semantics and divergences.**
- **`-l/--links`** copies a symlink as a symlink: the scanner `readlink`s the
target, the sender transmits it, and the receiver `symlinkat`s it. FastSync
`-l` never preserved symlink targets before (the flag was documented partial
and, in fact, tried to read the referent as file data); it now does, matching
rsync. Divergences: because the receiver enforces the symlink containment
predicate unconditionally, a plain `-l` sync **refuses to round-trip a
legitimate absolute symlink target** (it is dropped, never created pointing
outside the root — see the `--munge-links` note for the symmetric trust
boundary); a relative in-root target is copied as-is. As of P7 Wave D FastSync
also applies the symlink's own metadata with no-follow primitives
target, the sender transmits it, and the receiver `symlinkat`s it. **Targets
are stored verbatim (protocol 2.23.0), matching rsync `-l`:** an absolute
target or one containing `..` is copied exactly as-is. The receiver no longer
enforces the strict containment predicate on the link *value*; target policy
belongs to the sender (`--safe-links`/`--copy-unsafe-links`) exactly as in
rsync. The link's *placement* path is still hard-confined
(`has_path_traversal`, O_NOFOLLOW fd walk), and the link's own metadata is
applied with no-follow primitives
(`utimensat`/`fchownat`/`fchmodat` with `AT_SYMLINK_NOFOLLOW`), so `-J` is a
real omit switch rather than a no-op.
real omit switch rather than a no-op. **Residual risk:** because `-l` stores
targets verbatim and does not enforce containment, a destination later
consumed by a link-following tool can follow a link outside the receive root.
Use `--safe-links` when the source is not trusted; a destination that only
ever uses `openat`-style no-follow access is unaffected.
- **`-k/--copy-dirlinks`** (sender): a symlink whose referent is a directory is
dereferenced and recursed into as a real directory; a symlink to a regular
file (or any non-directory) is kept as a symlink. This is rsync's `-k`. When
@@ -547,40 +581,35 @@ was bumped **2.12.0 → 2.13.0** (peers must match, exactly as prior phases did)
divergence for `--delete` over an existing symlinked dir). Without `-K` the
destination symlink is not followed (the O_NOFOLLOW walk fails the write),
which is the safe default.
- **`--munge-links`** (sender security rewrite; crosses the wire so the receiver
unmunges): every transmitted symlink target is prefixed with the marker
`#SYMLINK/`; the receiver strips the marker (only when the negotiated
`munge_links` policy is on — a plain `-l` run never strips the prefix, so a
source symlink that genuinely begins with `#SYMLINK/` round-trips verbatim)
and restores the exact real target. The trust boundary is **symmetric and
enforced receiver-side**, independent of the sender: `file_symlink_at_secure`
refuses any target that `file_symlink_target_contained` rejects (absolute
`/...` or relative with a `..` component), and `file_save_to_disk_full`
contains such an entry (skipped) rather than materializing it. A deliberate confinement trade-off: because the receiver
enforces containment unconditionally, a plain `-l` (no `--munge-links`) sync
*refuses to round-trip a legitimate absolute symlink target* — such target is
dropped, never created pointing outside the root. This is a stricter subset of
rsync: rsync stores munged targets on the RECEIVING side and depends on both
ends running `--munge-links`; FastSync additionally enforces the containment
predicate at the receiver regardless of what the sender transmitted. When no
symlink is being transmitted (`-l`/`-k`/`-a` off) `--munge-links` has nothing
to rewrite and is inert. -*K/`--keep-dirlinks` policy is installed per
connection at config-accept (stable for the whole transfer, never racy under
`-j`/`--threads`), and only ever follows an in-root symlink-to-directory.*
- **`--munge-links`** (sender rewrite; crosses the wire so the receiver
unmunges): every transmitted symlink target is prefixed with rsync's marker
`SYMLINK_MUNGE_PREFIX` = `/rsyncd-munged/`; the receiver strips the marker
(only when the negotiated `munge_links` policy is on — a plain `-l` run never
strips the prefix, so a source symlink that genuinely begins with
`/rsyncd-munged/` round-trips verbatim) and restores the exact real target.
This matches rsync's stored marker and its both-ends-negotiated model, with the
prefix applied on the sender rather than the receiver. The link *value* is
otherwise stored verbatim; the *placement* path still goes through
`file_symlink_at_secure`'s confined fd walk (`has_path_traversal` on the
destination path, no symlink follow). When no symlink is being transmitted
(`-l`/`-k`/`-a` off) `--munge-links` has nothing to rewrite and is inert.
`-K`/`--keep-dirlinks` policy is installed per connection at config-accept
(stable for the whole transfer, never racy under `-j`/`--threads`), and only
ever follows an in-root symlink-to-directory.
**Compatibility (byte-identical when all three are absent):** `-k`, `-K` and
`--munge-links` are opt-in. Without them the scanner's link handling, the wire
frames, and the receiver's writes are unchanged for every other option set, so a
run that previously worked continues to behave identically. `-l/--links` itself
now transmits targets (the prior behavior was broken/partial); its status moved
`⚠️ Partial → ✅ Implemented`.
**Compatibility:** `-k`, `-K` and `--munge-links` are opt-in. Without them the
scanner's link handling, the wire frames, and the receiver's writes are unchanged
for every other option set. `--safe-links`/`--copy-unsafe-links` are applied
sender-side; `--trust-sender` no longer changes how symlink targets are stored
(it only skips the receiver's path-list re-validation). `-l/--links` stores
targets verbatim, matching rsync.
## 10. Sparse & Device
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-S`, `--sparse` | Sparse block handling | ✅ Implemented | Phase 7 Wave B: real hole preservation with no wire change. The receiver's sparse-aware writer (`write_all_sparse`, next to `write_all` in `src/shared/file.c` and `src/shared/file_store.c`) walks the in-memory file image and emits any all-zero run ≥ 4096 bytes as a hole via `lseek(SEEK_CUR)` (the pre-size `ftruncate` guarantees the offset bookkeeping and logical size), `ftruncate(size)` after the last run pins the final size even with a hole tail. Wired into both the atomic temp+rename store and `--inplace` when `sparse` is set; the non-sparse path is byte-identical to before. **Sparse wins over `--preallocate`** (posix_fallocate is skipped when sparse is set, so the holes are not re-allocated). Interplay note: under `--partial` a retained sparse temp already has the full logical size (trailing content is holes), so `--append`'s "shorter destination" resume does not re-run; the retained file is still valid and a normal re-transfer (or `-W`/delta) repairs it — documented so the combination is never surprising |
| `--preallocate` | Allocate dest files before writing | ✅ Implemented | The receiver preallocates the destination file's full expected space before any data is written, so a transfer that would overflow disk fails fast at allocation time (a clean error, not a half-written file) and the file is laid out contiguously, avoiding fragmentation. Crosses the wire (the config frame carries a `preallocate` boolean; `PROTOCOL_VERSION` bumped **2.10.0 → 2.11.0**, peers must match) so the sender knows the receiver will preallocate and the receiver performs it. **Allocation approach:** `posix_fallocate()` is preferred because it reserves *real* disk blocks (true fail-fast on ENOSPC), falling back to plain `ftruncate()` only when the filesystem reports the allocation is unsupported (`EOPNOTSUPP`/`ENOSYS`); `ftruncate` still extends the logical size so the intent degrades gracefully. **Fallback/error semantics:** `EOPNOTSUPP`/`ENOSYS` → clean fallback to `ftruncate` (best-effort, preallocates the logical size and never fails a transfer on filesystems that lack `posix_fallocate`); a genuine allocation failure (`ENOSPC`/`EDQUOT`/`EFBIG`/…) aborts the file/receive with a distinct `preallocate failed ... transfer aborted` error — it does **not** fall back to a normal non-preallocated write, preserving the fail-fast purpose. **Size-known requirement:** preallocation only runs when the final size is already known up front (the normal regular-file case); unknown-length data is skipped (never failed). **Orthogonality:** applies uniformly across the atomic temp+rename store path, `--inplace`, `--partial`/`--partial-dir`, `--delay-updates` (the staged temp file is preallocated before data flows) and the `--link-dest` copy fallback; it neither implies nor conflicts with `-s`, `--append`, or delta. rsync-divergence: rsync signals that `--preallocate` is ignored with `--sparse`; FastSync gives **sparse precedence** — when both are set, `posix_fallocate` is skipped so the holes the sparse writer creates are not re-allocated (the `ftruncate` presize sizing stays), matching the intent of "sparse wins". See the Phase-4 preallocate notes below |
| `-S`, `--sparse` | Sparse block handling | ✅ Parity | Phase 7 Wave B: real hole preservation with no wire change. The receiver's sparse-aware writer (`write_all_sparse`, next to `write_all` in `src/shared/file.c` and `src/shared/file_store.c`) walks the in-memory file image and emits any all-zero run ≥ 4096 bytes as a hole via `lseek(SEEK_CUR)` (the pre-size `ftruncate` guarantees the offset bookkeeping and logical size), `ftruncate(size)` after the last run pins the final size even with a hole tail. Wired into both the atomic temp+rename store and `--inplace` when `sparse` is set; the non-sparse path is byte-identical to before. **Sparse wins over `--preallocate`** (posix_fallocate is skipped when sparse is set, so the holes are not re-allocated). Interplay note: under `--partial` a retained sparse temp already has the full logical size (trailing content is holes), so `--append`'s "shorter destination" resume does not re-run; the retained file is still valid and a normal re-transfer (or `-W`/delta) repairs it — documented so the combination is never surprising |
| `--preallocate` | Allocate dest files before writing | ⚠️ Caveat | The receiver preallocates the destination file's full expected space before any data is written, so a transfer that would overflow disk fails fast at allocation time (a clean error, not a half-written file) and the file is laid out contiguously, avoiding fragmentation. Crosses the wire (the config frame carries a `preallocate` boolean; `PROTOCOL_VERSION` bumped **2.10.0 → 2.11.0**, peers must match) so the sender knows the receiver will preallocate and the receiver performs it. **Allocation approach:** `posix_fallocate()` is preferred because it reserves *real* disk blocks (true fail-fast on ENOSPC), falling back to plain `ftruncate()` only when the filesystem reports the allocation is unsupported (`EOPNOTSUPP`/`ENOSYS`); `ftruncate` still extends the logical size so the intent degrades gracefully. **Fallback/error semantics:** `EOPNOTSUPP`/`ENOSYS` → clean fallback to `ftruncate` (best-effort, preallocates the logical size and never fails a transfer on filesystems that lack `posix_fallocate`); a genuine allocation failure (`ENOSPC`/`EDQUOT`/`EFBIG`/…) aborts the file/receive with a distinct `preallocate failed ... transfer aborted` error — it does **not** fall back to a normal non-preallocated write, preserving the fail-fast purpose. **Size-known requirement:** preallocation only runs when the final size is already known up front (the normal regular-file case); unknown-length data is skipped (never failed). **Orthogonality:** applies uniformly across the atomic temp+rename store path, `--inplace`, `--partial`/`--partial-dir`, `--delay-updates` (the staged temp file is preallocated before data flows) and the `--link-dest` copy fallback; it neither implies nor conflicts with `-s`, `--append`, or delta. rsync-divergence: rsync signals that `--preallocate` is ignored with `--sparse`; FastSync gives **sparse precedence** — when both are set, `posix_fallocate` is skipped so the holes the sparse writer creates are not re-allocated (the `ftruncate` presize sizing stays), matching the intent of "sparse wins". See the Phase-4 preallocate notes below |
**Preallocate notes (Phase 4, preallocate wave):** `--preallocate` is implemented as a real receiver-side allocation of the destination file's space before data is written. It is a plain boolean config flag that crosses the wire (serialized in the config frame's selection-options block, mirroring `--inplace`/`--append`/`--force`), so the run requires matching ends: `PROTOCOL_VERSION` was bumped **2.10.0 → 2.11.0** (peers must match or the version check fails). The allocation is performed on the exact destination fd, immediately after it is opened, before any bytes are streamed; `posix_fallocate` (and the `ftruncate` fallback) leave the fd's file offset untouched, so the subsequent data write at offset 0 is unaffected and complete. Because FastSync writes each file's byte payload in one in-memory batch, the "full expected size" is exactly the known `data_size`, which is what gets preallocated. Unknown-length/streamed payloads are skipped rather than failed. A failed allocation logs a distinct `preallocate failed` error and aborts the file (the atomic temp is unlinked, the inplace target is left untrimmed) so the run fails cleanly and never silently degrades to a non-preallocated write — preserving rsync's fail-fast intent on a full disk.
@@ -589,49 +618,50 @@ now transmits targets (the prior behavior was broken/partial); its status moved
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `--checksum` | Skip based on checksum | ✅ Implemented | With `--incremental`, compares per-file whole-file content digests to skip unchanged files. The digest algorithm is `xxh64` with seed 0 by default and is selectable via `--checksum-choice`/`--cc` (xxh64/xxhash or md5) and `--checksum-seed=NUM` (see those rows); `-c` remains compression |
| `--checksum-choice=STR`, `--cc=STR` | Choose checksum algorithm | ✅ Implemented | Real algorithm selection for the per-file whole-file digest used by the `--incremental`/`--checksum` handshake and by the basis-dir content verification. FastSync genuinely supports `xxh64` (the default, exact xxHash64, seeded by `--checksum-seed`) and `md5` (via OpenSSL EVP); `xxhash` is accepted as rsync's spelling of xxHash64. Any other name (md4/sha1/sha256/crc32/none/…) is rejected with a clear error at parse time — never a silent no-op. `--cc` is the alias (`--cc=ALG` and space forms both parse). The algorithm id and seed cross the wire with the config frame, so the receiver hashes its on-disk old file with the SAME algorithm+seed the sender used and both agree on a match; the sender's digest and the receiver's comparison live in the per-file `STATUS_CHECK` handshake, which now carries a length-prefixed, bounded (1..16 byte) digest instead of a fixed 64-bit value, and the receiver pins the received length to the negotiated algorithm's digest length (defense-in-depth: a mismatched/malicious length only forces a safe re-transfer). Note: `md5` is a FIPS-non-approved algorithm, so under an OpenSSL build with FIPS mode enabled `--checksum-choice=md5` fails loudly rather than silently falling back. Protocol/layout: `PROTOCOL_VERSION` bumped **2.9.0 → 2.10.0** (peers must match). Defaults preserve the pre-existing behavior byte-for-byte (xxh64, seed 0). Like rsync, the choice only takes effect where a whole-file digest is actually computed (`--checksum` on, or a basis-dir flag); it does not itself enable `--checksum`. Closely-related divergence: the delta BLOCK strong checksum (§11 delta) stays xxHash32 — `--checksum-choice` selects only the whole-file digest, matching rsync where the per-block checksum is independent of the whole-file checksum choice |
| `--compare-dest=DIR` | Compare dest files relative to DIR | ✅ Implemented | DIR is a receiver-side basis relative to the destination root (confined below it; absolute/`..`/`.` rejected, `//` collapsed and trailing `/` dropped). On the receiver's per-file check (implies `--incremental`) an exact match = same size + mtime (unless `--size-only`; `-I` disables matching) **and** equal xxHash64 of the sender's file; a match suppresses the data transfer. compare-dest never copies: it only skips a file the destination does **not** already hold (sparse destination, rsync parity), and is consulted before the normal delta/full paths. Repeatable; searched in command-line order, first match wins. Divergences: when the destination already holds a *different* version rsync deletes it but FastSync instead transfers the data (keeps the mirror complete; never deletes without `--delete`); attribute-only differences on a match are not re-applied (data is skipped so the sender never sends metadata); content is verified by xxHash64, stricter than rsync's default quick check. Sizing: FastSync's whole-file payload limit is 256 MiB on **every** transfer path (not basis-specific); rsync applies basis dirs to arbitrary sizes, so FastSync refuses a basis run whose source contains a larger file up front with a clear error before any transfer. Wire: a basis-count field is always present on the config frame (protocol 2.9.0, so clients and servers must both be 2.9.0) |
| `--copy-dest=DIR` | Include copies of unchanged files | ✅ Implemented | Same basis rules as `--compare-dest`, but an exact match materializes a **local copy** of the DIR file into the destination (via the normal atomic temp+rename store path, so `--existing`/`--ignore-existing`/`--update`/`--backup`/`--delay-updates` all still apply) instead of transferring data. Repeatable; command-line order = priority. Content is xxHash64-verified before the copy. Divergences: a basis-hit destination keeps the basis file's own mode/uid/gid and mtime (the sender sends no metadata on a skip), so with `--size-only` its mtime can differ from the source and attribute-only differences are copied with the basis attributes rather than rsync's "copy + fix attributes". Requires `--incremental` (implied); incompatible with `-s`. Wire: protocol 2.9.0 |
| `--link-dest=DIR` | Hardlink to files when unchanged | ✅ Implemented | Same basis rules as `--copy-dest`, but an exact match installs an atomic **hard link** to the DIR file (temp hard link + rename) so no data or disk space is used; where the link is impossible (basis on another filesystem, filesystem refuses links) it falls back cleanly to a byte-identical local copy, never a corrupt/partial file. `--delay-updates` stages the link and publishes by rename, so the final entry stays a real hard link. Repeatable (searched in command-line order, first match wins). Content is xxHash64-verified before linking. Divergences and caveats: an already up-to-date destination file is not re-linked to a basis file (only files that would otherwise be written are linked); a link keeps the basis inode's own mode/uid/gid and mtime — metadata is never written through the shared inode (that would mutate the basis file), so a later `--inplace` run that rewrites such a destination path **will mutate the basis snapshot** through the shared inode (use `--copy-dest` when the destination must stay independently writable); with `--size-only` the linked mtime can differ from the source; a `--remove-source-files` source satisfied by a basis dir is treated as skipped and therefore **retained** (never removed); basis dirs are excluded from `--delete`. Requires `--incremental` (implied); incompatible with `-s`. Wire: protocol 2.9.0 |
| `-y`, `--fuzzy`, `--no-fuzzy` | Find similar file for basis | ✅ Implemented | `-y/--fuzzy` is a pure bandwidth optimization on the existing receiver-driven delta path: when a file must be transferred and the destination holds no usable content at the exact path (file absent, or the destination file is outside the delta engine's size bounds), the receiver searches the SAME destination directory for an existing regular file whose basename is similar to the incoming name and uses it as the delta basis, so the sender transmits only the differences instead of the whole file. The output is always byte-exact regardless of which (or whether any) basis is chosen. Decision location: the receiver performs the candidate search inside `receive_incremental_check` and sends the normal `STATUS_DELTA_SIGNATURE`; the sender never learns the basis was a different file, so no new frame type or sender logic was needed — only the config frame grew a `fuzzy` boolean, so `PROTOCOL_VERSION` was bumped **2.8.0 → 2.9.0** (peers must match). Similarity heuristic (deterministic, simpler than rsync's deliberately-fuzzy matching, and documented precisely): candidates are the target's sibling entries in its destination directory, opened `O_NOFOLLOW`/`AT_SYMLINK_NOFOLLOW` under the confined root (symlinks never followed; nothing outside the destination root is ever read or hashed); dotfiles, directories, the target's own name, and the `.fastsync-stage`/temp scratch names are excluded; like the ordinary delta path, the block signature the receiver transmits is derived from on-disk content it may not otherwise send, so a negotiated `--fuzzy` run exposes the destination's sibling files (at block granularity) to the sender as a known-plaintext oracle — the same information class as the normal delta handshake over the file being replaced; the size gate is the delta engine's own bounds (both files ≥ 16 KiB, ≤ `--delta-max`, ratio ≤ 10×) rather than rsync's ~1.5× size window; the name gate is a Levenshtein edit distance between the basenames accepted only when ≤ half the length of the longer basename; the single best candidate (smallest distance, tie-break size closest to the incoming file then lexicographically smaller basename) is read; the directory scan is capped at 4096 entries so a pathological directory cannot stall a transfer. When fuzzy applies: only to files the receiver would otherwise send whole — the destination's own file is always preferred as the delta basis when it exists and fits the delta size bounds, so fuzzy does NOT replace an existing-but-different destination basis; FastSync's 10× delta size-ratio bound means an existing destination file that is too far away in size still lets the fuzzy search run. When no similar candidate exists the transfer falls back to the normal whole-file transfer. rsync-divergence note: rsync's own matching uses a fuzzy name/size rule set; FastSync implements the closest safe deterministic approximation above. Because FastSync's delta machinery is off by default (rsync's is on), `--fuzzy` implies `--incremental` + `--delta` (unless `--whole-file`/`-W` or an explicit `--no-delta` switched delta off, in which case fuzzy is inert — matching rsync where `--whole-file` makes fuzzy irrelevant). Unlike the basis-dir options, `--fuzzy` honors an explicit `--no-incremental` (it does not force the handshake back on); an explicit `--no-incremental` also suppresses the delta implication so no invalid `--delta requires --incremental` config results. `--no-fuzzy` negates it. All surrounding semantics are untouched: a fuzzy-reconstructed file is stored as a normal file, so `--remove-source-files`, itemize/`-i`, `--stats`, `--backup`, `--delay-updates`, `--existing`/`--ignore-existing`/`--update` behave exactly as for a whole-file transfer (the fuzzy delta does not skip the file) |
| `--checksum` | Skip based on checksum | ✅ Parity | `-c`/`--checksum` compares per-file whole-file content digests to skip unchanged files. **As of protocol 2.23.0 the short `-c` implies the checksum quick-check**, so a plain `-c` run verifies content rather than only affecting the `--incremental` handshake. The digest algorithm is `xxh64` by default and is selectable via `--checksum-choice`/`--cc` (`xxh64`/`xxhash`/`xxh3`/`xxh128`/`md5`/`auto`) and `--checksum-seed=NUM` (see those rows) |
| `--checksum-choice=STR`, `--cc=STR` | Choose checksum algorithm | ⚠️ Caveat | Real algorithm selection for the per-file whole-file digest used by the `--incremental`/`--checksum` handshake and by the basis-dir content verification. **Protocol 2.23.0 accepts `xxh64` (the default), `xxhash` (rsync's spelling of xxHash64), `xxh3`, `xxh128`, `md5`, and `auto` (which selects FastSync's default).** rsync choices FastSync does not implement — `md4`, `sha1`, `none`, and the two-name `transfer,pre-transfer` form — are **rejected by name** with a clear error at parse time, never a silent no-op. `--cc` is the alias (`--cc=ALG` and space forms both parse). The algorithm id and seed cross the wire with the config frame, so the receiver hashes its on-disk old file with the SAME algorithm+seed the sender used and both agree on a match; the sender's digest and the receiver's comparison live in the per-file `STATUS_CHECK` handshake, which carries a length-prefixed, bounded (1..16 byte) digest, and the receiver pins the received length to the negotiated algorithm's digest length (defense-in-depth: a mismatched/malicious length only forces a safe re-transfer). Digest lengths: `xxh64`/`xxh3` = 8 bytes, `xxh128`/`md5` = 16. Note: `md5` is a FIPS-non-approved algorithm, so under an OpenSSL build with FIPS mode enabled `--checksum-choice=md5` fails loudly rather than silently falling back. `PROTOCOL_VERSION` has moved well past the original 2.10.0 digest-frame bump. Like rsync, the choice only takes effect where a whole-file digest is actually computed (`--checksum` on, or a basis-dir flag). Closely-related divergence: the delta BLOCK strong checksum stays xxHash32 — `--checksum-choice` selects only the whole-file digest, matching rsync where the per-block checksum is independent of the whole-file choice |
| `--compare-dest=DIR` | Compare dest files relative to DIR | ⚠️ Caveat | DIR is a receiver-side basis relative to the destination root (confined below it; absolute/`..`/`.` rejected, `//` collapsed and trailing `/` dropped). On the receiver's per-file check (implies `--incremental`) an exact match = same size + mtime (unless `--size-only`; `-I` disables matching) **and** equal xxHash64 of the sender's file; a match suppresses the data transfer. compare-dest never copies: it only skips a file the destination does **not** already hold (sparse destination, rsync parity), and is consulted before the normal delta/full paths. Repeatable; searched in command-line order, first match wins. Divergences: when the destination already holds a *different* version rsync deletes it but FastSync instead transfers the data (keeps the mirror complete; never deletes without `--delete`); attribute-only differences on a match are not re-applied (data is skipped so the sender never sends metadata); content is verified by xxHash64, stricter than rsync's default quick check. Sizing: FastSync's whole-file payload limit is 256 MiB on **every** transfer path (not basis-specific); rsync applies basis dirs to arbitrary sizes, so FastSync refuses a basis run whose source contains a larger file up front with a clear error before any transfer. Wire: a basis-count field is always present on the config frame (protocol 2.9.0, so clients and servers must both be 2.9.0) |
| `--copy-dest=DIR` | Include copies of unchanged files | ⚠️ Caveat | Same basis rules as `--compare-dest`, but an exact match materializes a **local copy** of the DIR file into the destination (via the normal atomic temp+rename store path, so `--existing`/`--ignore-existing`/`--update`/`--backup`/`--delay-updates` all still apply) instead of transferring data. Repeatable; command-line order = priority. Content is xxHash64-verified before the copy. Divergences: a basis-hit destination keeps the basis file's own mode/uid/gid and mtime (the sender sends no metadata on a skip), so with `--size-only` its mtime can differ from the source and attribute-only differences are copied with the basis attributes rather than rsync's "copy + fix attributes". Requires `--incremental` (implied); incompatible with `-s`. Wire: protocol 2.9.0 |
| `--link-dest=DIR` | Hardlink to files when unchanged | ⚠️ Caveat | Same basis rules as `--copy-dest`, but an exact match installs an atomic **hard link** to the DIR file (temp hard link + rename) so no data or disk space is used; where the link is impossible (basis on another filesystem, filesystem refuses links) it falls back cleanly to a byte-identical local copy, never a corrupt/partial file. `--delay-updates` stages the link and publishes by rename, so the final entry stays a real hard link. Repeatable (searched in command-line order, first match wins). Content is xxHash64-verified before linking. Divergences and caveats: an already up-to-date destination file is not re-linked to a basis file (only files that would otherwise be written are linked); a link keeps the basis inode's own mode/uid/gid and mtime — metadata is never written through the shared inode (that would mutate the basis file), so a later `--inplace` run that rewrites such a destination path **will mutate the basis snapshot** through the shared inode (use `--copy-dest` when the destination must stay independently writable); with `--size-only` the linked mtime can differ from the source; a `--remove-source-files` source satisfied by a basis dir is treated as skipped and therefore **retained** (never removed); basis dirs are excluded from `--delete`. Requires `--incremental` (implied); incompatible with `-s`. Wire: protocol 2.9.0 |
| `-y`, `--fuzzy`, `--no-fuzzy` | Find similar file for basis | ⚠️ Caveat | `-y/--fuzzy` is a pure bandwidth optimization on the existing receiver-driven delta path: when a file must be transferred and the destination holds no usable content at the exact path (file absent, or the destination file is outside the delta engine's size bounds), the receiver searches the SAME destination directory for an existing regular file whose basename is similar to the incoming name and uses it as the delta basis, so the sender transmits only the differences instead of the whole file. The output is always byte-exact regardless of which (or whether any) basis is chosen. Decision location: the receiver performs the candidate search inside `receive_incremental_check` and sends the normal `STATUS_DELTA_SIGNATURE`; the sender never learns the basis was a different file, so no new frame type or sender logic was needed — only the config frame grew a `fuzzy` boolean, so `PROTOCOL_VERSION` was bumped **2.8.0 → 2.9.0** (peers must match). Similarity heuristic (deterministic, simpler than rsync's deliberately-fuzzy matching, and documented precisely): candidates are the target's sibling entries in its destination directory, opened `O_NOFOLLOW`/`AT_SYMLINK_NOFOLLOW` under the confined root (symlinks never followed; nothing outside the destination root is ever read or hashed); dotfiles, directories, the target's own name, and the `.fastsync-stage`/temp scratch names are excluded; like the ordinary delta path, the block signature the receiver transmits is derived from on-disk content it may not otherwise send, so a negotiated `--fuzzy` run exposes the destination's sibling files (at block granularity) to the sender as a known-plaintext oracle — the same information class as the normal delta handshake over the file being replaced; the size gate is the delta engine's own bounds (both files ≥ 16 KiB, ≤ `--delta-max`, ratio ≤ 10×) rather than rsync's ~1.5× size window; the name gate is a Levenshtein edit distance between the basenames accepted only when ≤ half the length of the longer basename; the single best candidate (smallest distance, tie-break size closest to the incoming file then lexicographically smaller basename) is read; the directory scan is capped at 4096 entries so a pathological directory cannot stall a transfer. When fuzzy applies: only to files the receiver would otherwise send whole — the destination's own file is always preferred as the delta basis when it exists and fits the delta size bounds, so fuzzy does NOT replace an existing-but-different destination basis; FastSync's 10× delta size-ratio bound means an existing destination file that is too far away in size still lets the fuzzy search run. When no similar candidate exists the transfer falls back to the normal whole-file transfer. rsync-divergence note: rsync's own matching uses a fuzzy name/size rule set; FastSync implements the closest safe deterministic approximation above. Because FastSync's delta machinery is off by default (rsync's is on), `--fuzzy` implies `--incremental` + `--delta` (unless `--whole-file`/`-W` or an explicit `--no-delta` switched delta off, in which case fuzzy is inert — matching rsync where `--whole-file` makes fuzzy irrelevant). Unlike the basis-dir options, `--fuzzy` honors an explicit `--no-incremental` (it does not force the handshake back on); an explicit `--no-incremental` also suppresses the delta implication so no invalid `--delta requires --incremental` config results. `--no-fuzzy` negates it. All surrounding semantics are untouched: a fuzzy-reconstructed file is stored as a normal file, so `--remove-source-files`, itemize/`-i`, `--stats`, `--backup`, `--delay-updates`, `--existing`/`--ignore-existing`/`--update` behave exactly as for a whole-file transfer (the fuzzy delta does not skip the file) |
## 12. Compression
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-z`, `--compress` | Compress file data | ✅ Implemented | Always uses zstd (rsync supports multiple algorithms — a documented divergence, selectable via `--compress-choice`). Phase 7 Wave A: `-z` is now the compression short form; `-c` is rsync's `--checksum` |
| `--compress-choice=STR`, `--zc=STR` | Choose compression algorithm | ✅ Implemented | FastSync supports `zstd` and `none` |
| `--compress-level=NUM`, `--zl=NUM` | Set compression level | ✅ Implemented | 1-22, default 5 |
| `--compress-threads=NUM` | Set compression threads | ✅ Implemented | `compression_threads` config field (client-only; does not cross the wire). Sets the number of worker threads used by the zstd compression pool to NUM (1..64; 0/garbage/oversized rejected up front). Accepted in both `--compress-threads=NUM` and two-argument `--compress-threads NUM` forms. Composes with `-z`/compression; under the `-j`/`--threads` multithreaded pipeline it parallelizes compressed chunk encoding. See test_tcp.py `-z --compress-threads=2` and test_client_cli.c |
| `--skip-compress=LIST` | Skip compress for suffixes | ✅ Implemented | Comma-separated, case-insensitive suffix list; empty list skips none; incompatible with FastSync chunk serialization (`-s`) |
| `-z`, `--compress` | Compress file data | ⚠️ Caveat | Streaming zstd (rsync supports multiple algorithms — a documented divergence, selectable via `--compress-choice`). `-z` is the compression short form; `-c` is rsync's `--checksum`. `--skip-compress` applies rsync 3.4.1's default suffix list when no list is given |
| `--compress-choice=STR`, `--zc=STR` | Choose compression algorithm | ⚠️ Caveat | FastSync supports `zstd` (default), `none`, and `auto`. rsync's other compiled-in choices (`lz4`, `zlib`, `zlibx`) are **rejected by name** at parse time with a clear error, never silently ignored. `--zc` is the alias |
| `--compress-level=NUM`, `--zl=NUM` | Set compression level | ✅ Parity | 1-22, default 5 |
| `--compress-threads=NUM` | Set compression threads | ✅ Parity | `compression_threads` config field (client-only; does not cross the wire). Sets the number of worker threads used by the zstd compression pool to NUM (1..64; 0/garbage/oversized rejected up front). Accepted in both `--compress-threads=NUM` and two-argument `--compress-threads NUM` forms. Composes with `-z`/compression; under the `-j`/`--threads` multithreaded pipeline it parallelizes compressed chunk encoding. See test_tcp.py `-z --compress-threads=2` and test_client_cli.c |
| `--skip-compress=LIST` | Skip compress for suffixes | ⚠️ Caveat | Comma-separated (or `/`-separated, as in rsync) case-insensitive suffix list; a leading dot is optional; an empty list skips none. **When the option is omitted, rsync 3.4.1's built-in default suffix list applies** (`3g2 3gp 7z aac … zip zst`); an explicit list replaces that default entirely, matching rsync. A user-supplied list is a client-side compression choice; incompatible with FastSync chunk serialization (`-s`) |
## 13. Connectivity
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-e`, `--rsh=COMMAND` | Remote shell to use | ✅ Implemented | `-e`/`--rsh` (and `--rsh=COMMAND`) select the remote-shell program used to build the SSH child argv, overriding the default `ssh`. The command is whitespace-split into the leading argv words so rsync's `-e "ssh -p 2222"` works; the standard `-o` family, an optional `-p` port, `user@host` and the quoted remote command (`fastsync-server --stdio`) follow. Stored in the `rsh_command` config field. **Client-only, never crosses the wire** (it is a launch concern, not a handshake property) |
| `--rsync-path=PROGRAM` | rsync binary on remote | ✅ Implemented | Alias for `--fastsync-server-path`: both write the `fastsync_server_path` config field used as the remote-side server program (always quoted as one remote-shell word), which CROSSES the wire as before. Kept separate from `--rsh`, which names the local connecting program |
| `--port=PORT`, `--port PORT` | Alternate daemon port | ✅ Implemented | rsync's daemon-port flag is an alias for `--server-port`: both spellings (and `--server-port=PORT`) map to the client-side `server_port` config field. The client connects to a TCP/TLS server (incl. `host::module/path` daemon destinations) on that port, and the `fastsync-server --daemon` listener's port is taken from its config's `port` key (default 873) or overridden by `--dparam port=` / `-p` |
| `--sockopts=OPTIONS` | Custom TCP options | ✅ Implemented | Comma-separated allowlist of `OPT=VAL` applied via `setsockopt` after `socket()` before `connect()`/`bind()`. Only `TCP_NODELAY`, `SO_KEEPALIVE`, `SO_REUSEADDR` (0/1) and `SO_RCVBUF`/`SO_SNDBUF` (byte count) are accepted; an unknown option name or a bad value is rejected up front, never silently ignored. A value is required for every option (`OPT=VAL`; a bare name is an error). Applied to the outgoing TCP and TLS client socket; absent by default. `SockOptEntry`/`sockopts` config fields. Local socket concern: never crosses the wire |
| `--blocking-io` | Use blocking I/O for remote shell | ✅ Implemented | With `--blocking-io` the SSH-transport socketpair socket is left without `SO_RCVTIMEO`/`SO_SNDTIMEO`, so the transfer blocks naturally; by default it gets the same read/write timeout as the TCP transport (see `--timeout`). `blocking_io` config bool. **Client-only, never crosses the wire** |
| `--outbuf=N\|L\|B` | Set output buffering | ✅ Implemented | `N` (none/unbuffered) → `_IONBF`, `L` (line) → `_IOLBF`, `B` (block, the default) → `_IOFBF` via `setvbuf` on stdout and stderr. Garbage values are rejected. `outbuf` config field (`OutbufMode`). **Client-only, never crosses the wire** |
| `--address=ADDRESS` | Bind address for outgoing socket | ✅ Implemented | Binds the outgoing client socket to a local source address before `connect()` (resolved with the same `-4`/`-6` family hints as the destination). Local socket concern: never crosses the wire |
| `-4`, `--ipv4` | Prefer IPv4 | ✅ Implemented | Forces `AF_INET` in the `getaddrinfo` hints for client destination/source resolution and the server bind (see the Phase 5, Wave B note). Mutually exclusive with `-6` |
| `-6`, `--ipv6` | Prefer IPv6 | ✅ Implemented | Forces `AF_INET6` in the `getaddrinfo` hints for client destination/source resolution and the server bind. Mutually exclusive with `-4` |
| `--remote-option=OPT`, `-M` | Send an option only to the remote side | ✅ Implemented | Each value is appended to the remote server invocation over SSH as an individually single-quote-escaped shell word in `ssh_build_remote_command()`. Values are validated (non-empty, no control characters) and shell metacharacters cannot break out of the quoting (`;`, `&`, `|`, <code>`</code>, `$`, `(`, `)`, quotes are neutralized), so a value cannot inject an arbitrary remote command and a subsequent `--` on the client line cannot be turned into one. The options never cross the binary config frame. Phase 7 Wave A: the short `-M` form is now available (as `-M OPT` and `-M=OPT`), matching rsync; metadata mode moved to long-only `--preserve` |
| `-e`, `--rsh=COMMAND` | Remote shell to use | ✅ Parity | `-e`/`--rsh` (and `--rsh=COMMAND`) select the remote-shell program used to build the SSH child argv, overriding the default `ssh`. The command is whitespace-split into the leading argv words so rsync's `-e "ssh -p 2222"` works; the standard `-o` family, an optional `-p` port, `user@host` and the quoted remote command (`fastsync-server --stdio`) follow. Stored in the `rsh_command` config field. **Client-only, never crosses the wire** (it is a launch concern, not a handshake property) |
| `--rsync-path=PROGRAM` | rsync binary on remote | ✅ Parity | Alias for `--fastsync-server-path`: both write the `fastsync_server_path` config field used as the remote-side server program. The path is always quoted as one remote-shell word in the SSH argv. **Client-only: `fastsync_server_path` never crosses the wire** (it is a launch concern, not a handshake property), matching rsync, where `--rsync-path` likewise names the remote program locally. Kept separate from `--rsh`, which names the local connecting program |
| `--port=PORT`, `--port PORT` | Alternate daemon port | ✅ Parity | rsync's daemon-port flag is an alias for `--server-port`: both spellings (and `--server-port=PORT`) map to the client-side `server_port` config field. The client connects to a TCP/TLS server (incl. `host::module/path` daemon destinations) on that port, and the `fastsync-server --daemon` listener's port is taken from its config's `port` key (default 873) or overridden by `--dparam port=` / `-p` |
| `--sockopts=OPTIONS` | Custom TCP options | ✅ Parity | Comma-separated allowlist of `OPT=VAL` applied via `setsockopt` after `socket()` before `connect()`/`bind()`. Only `TCP_NODELAY`, `SO_KEEPALIVE`, `SO_REUSEADDR` (0/1) and `SO_RCVBUF`/`SO_SNDBUF` (byte count) are accepted; an unknown option name or a bad value is rejected up front, never silently ignored. A value is required for every option (`OPT=VAL`; a bare name is an error). Applied to the outgoing TCP and TLS client socket; absent by default. `SockOptEntry`/`sockopts` config fields. Local socket concern: never crosses the wire |
| `--blocking-io` | Use blocking I/O for remote shell | ✅ Parity | With `--blocking-io` the SSH-transport socketpair socket is left without `SO_RCVTIMEO`/`SO_SNDTIMEO`, so the transfer blocks naturally; by default it gets the same read/write timeout as the TCP transport (see `--timeout`). `blocking_io` config bool. **Client-only, never crosses the wire** |
| `--timeout=SEC`, `--contimeout=SEC` | Set I/O / connect timeouts | ✅ Parity | Protocol 2.23.0 matches rsync's defaults: **`--timeout` defaults to 0 (I/O deadlines disabled) and `--contimeout` to 60 s; `0` disables either.** A positive `--timeout` bounds both the socket (`SO_RCVTIMEO`/`SO_SNDTIMEO`) and the per-message protocol poll deadline on the client; the server floors its session deadline so a client `0` can never hold a session open forever. `--no-timeout`/`--no-contimeout` are the negations. Both are client-side deadlines and are not sent on the wire |
| `--outbuf=N\|L\|B` | Set output buffering | ✅ Parity | `N` (none/unbuffered) → `_IONBF`, `L` (line) → `_IOLBF`, `B` (block, the default) → `_IOFBF` via `setvbuf` on stdout and stderr. Garbage values are rejected. `outbuf` config field (`OutbufMode`). **Client-only, never crosses the wire** |
| `--address=ADDRESS` | Bind address for outgoing socket | ✅ Parity | Binds the outgoing client socket to a local source address before `connect()` (resolved with the same `-4`/`-6` family hints as the destination). Local socket concern: never crosses the wire |
| `-4`, `--ipv4` | Prefer IPv4 | ✅ Parity | Forces `AF_INET` in the `getaddrinfo` hints for client destination/source resolution and the server bind (see the Phase 5, Wave B note). Mutually exclusive with `-6` |
| `-6`, `--ipv6` | Prefer IPv6 | ✅ Parity | Forces `AF_INET6` in the `getaddrinfo` hints for client destination/source resolution and the server bind. Mutually exclusive with `-4` |
| `--remote-option=OPT`, `-M` | Send an option only to the remote side | ⚠️ Caveat | Each value is appended to the remote server invocation over SSH as an individually single-quote-escaped shell word in `ssh_build_remote_command()`. Values are validated (non-empty, no control characters) and shell metacharacters cannot break out of the quoting (`;`, `&`, `\|`, <code>`</code>, `$`, `(`, `)`, quotes are neutralized), so a value cannot inject an arbitrary remote command and a subsequent `--` on the client line cannot be turned into one. The short `-M` form (`-M OPT`, `-M=OPT`, and rsync-style attached `-MOPT`) is available, matching rsync; metadata mode moved to long-only `--preserve`. **Divergence:** `-M` is only meaningful for the SSH transport (`user@host:path`); a daemon (`host::module/path`) or local TCP destination **rejects** it (there is no remote command line to append to), whereas rsync applies it to its own remote process on every transport. The options never cross the binary config frame |
## 14. Daemon Mode
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `--daemon` | Run as rsync daemon | ✅ Implemented | Wave A: a real persistent listener. `fastsync-server --daemon --config FILE` (plus `--no-detach` to stay foreground; without it the listener detaches to the background after binding) reads a FastSync-native module config file and serves each connection confined to the requested module's `path` root (never a client-chosen root; every client-chosen-ownership/super-user request (`--numeric-ids`/`--chown`/`--usermap`/`--groupmap`/`--fake-super`/`--copy-as`/explicit `--super`) is refused unless the module opts in with `client owner = yes`, and the operator `--no-super` veto is honored). TCP/TLS via the existing `--tls` stack; plaintext still requires `--allow-unauthenticated` (same secure default as the standalone server). Client destinations use rsync's `host::module/path` form. Wire/protocol: the config frame gained a trailing daemon-module string and `PROTOCOL_VERSION` was bumped **2.14.0 → 2.15.0** (see the Daemon Mode notes below). Daemon mode is built in FastSync's own protocol/config grammar, not rsync's SMB/daemon option encoding |
| `--config=FILE` | Alternate rsyncd.conf file | ✅ Implemented | Wave A: selects the daemon config file. Default when omitted (in `--daemon` mode): `~/.config/fastsync/fastsyncd.conf` if it exists, else `/etc/fastsyncd.conf`. The grammar is FastSync-native (documented in the Daemon Mode notes below) and strictly rejects unknown keys so a typo can never silently change what a module serves; requires `--daemon` |
| `--dparam=OVERRIDE` | Override global daemon config | ✅ Implemented | Wave A: overrides one global scalar from the command line (`--dparam port=8734` and `--dparam=KEY=VALUE` both work). Limited to the global keys the grammar defines (`port`, `motd file`, `address`, `max connections`, `max connections per host`, `auth failure delay`, `auth lockout threshold`, `auth lockout duration`, `hosts allow`, `hosts deny`); keys are case-insensitive and unknown keys/invalid values are rejected. Requires `--daemon` |
| `--no-detach` | Don't detach from parent | ✅ Implemented | Wave A: with `--daemon`, keeps the listener in the foreground (what integration tests use). Without it the daemonizes (fork/setsid, stdio redirected to /dev/null) after the listening socket is bound. Requires `--daemon` |
| `--password-file=FILE` | Read daemon password from file | ✅ Implemented | A7 daemon auth. Client: `--password-file` supplies `user:password` for a `host::module/path` destination (the username is taken from this file, so `user@host::module` stays rejected); the literal password is held client-side only for the SCRAM handshake and wiped at teardown. Server (`fastsync-server --daemon --password-file FILE`): the salted-PBKDF2 verifier store that modules with `auth users` are verified against. **Neither the password nor any replayable bearer value crosses the wire or is stored server-side** — the store holds a per-user salt plus derived keys, and the daemon proves the secret with a per-connection nonce challenge. The file must be private to its owner: both the client and server verify the exact inode they read (open-then-`fstat`, so the check cannot be raced) and refuse a `--password-file`/`--early-input` that is not owned by the current user or grants any group/other permission bit (mode 0600), mirroring the TLS private-key check. A process-substitution pipe (`--early-input <(vault ...)`) is still accepted when it satisfies those checks. See the Daemon Mode notes below for the file formats and the plaintext/TLS caveat |
| `--early-input=FILE` | Use FILE for daemon early exec | ✅ Implemented | Server-only (requires `--daemon`): a second credential-store file, same new-format grammar as `--password-file`, read before the listener accepts connections (a secrets-manager / process-substitution source). Its entries layer over `--password-file`: byte-identical verifiers dedupe, a conflicting verifier for the same user is a startup error. A daemon whose modules declare `auth users` must be given at least one of the two, or it refuses to start (fail closed) |
| `--hash-credentials=FILE`, `--iterations N` | Hash a plaintext credential file | ✅ Implemented | Server-only offline tool (A7): reads the `user:password` lines of FILE (same owner-only 0600 check) and prints one new-format store line per entry to stdout, then exits. `--iterations` sets the PBKDF2 work factor (default 600000, range 100000–10000000). Dependency-free and does not run a listener. Use its output as `--password-file` for `--daemon`. There is no auto-upgrade: a legacy store line is hard-rejected by the loader and must be regenerated |
| `--daemon` | Run as rsync daemon | ⚠️ Caveat | Wave A: a real persistent listener. `fastsync-server --daemon --config FILE` (plus `--no-detach` to stay foreground; without it the listener detaches to the background after binding) reads a FastSync-native module config file and serves each connection confined to the requested module's `path` root (never a client-chosen root; every client-chosen-ownership/super-user request (`--numeric-ids`/`--chown`/`--usermap`/`--groupmap`/`--fake-super`/`--copy-as`/explicit `--super`) is refused unless the module opts in with `client owner = yes`, and the operator `--no-super` veto is honored). TCP/TLS via the existing `--tls` stack; plaintext still requires `--allow-unauthenticated` (same secure default as the standalone server). Client destinations use rsync's `host::module/path` form. Wire/protocol: the config frame gained a trailing daemon-module string and `PROTOCOL_VERSION` was bumped **2.14.0 → 2.15.0** (see the Daemon Mode notes below). Daemon mode is built in FastSync's own protocol/config grammar, not rsync's SMB/daemon option encoding |
| `--config=FILE` | Alternate rsyncd.conf file | ⚠️ Caveat | Wave A: selects the daemon config file. Default when omitted (in `--daemon` mode): `~/.config/fastsync/fastsyncd.conf` if it exists, else `/etc/fastsyncd.conf`. The grammar is FastSync-native (documented in the Daemon Mode notes below) and strictly rejects unknown keys so a typo can never silently change what a module serves; requires `--daemon` |
| `--dparam=OVERRIDE` | Override global daemon config | ⚠️ Caveat | Wave A: overrides one global scalar from the command line (`--dparam port=8734` and `--dparam=KEY=VALUE` both work). Limited to the global keys the grammar defines (`port`, `motd file`, `address`, `max connections`, `max connections per host`, `auth failure delay`, `auth lockout threshold`, `auth lockout duration`, `hosts allow`, `hosts deny`); keys are case-insensitive and unknown keys/invalid values are rejected. Requires `--daemon` |
| `--no-detach` | Don't detach from parent | ✅ Parity | Wave A: with `--daemon`, keeps the listener in the foreground (what integration tests use). Without it the daemonizes (fork/setsid, stdio redirected to /dev/null) after the listening socket is bound. Requires `--daemon` |
| `--password-file=FILE` | Read daemon password from file | ⚠️ Caveat | A7 daemon auth. Client: `--password-file` supplies `user:password` for a `host::module/path` destination (the username is taken from this file, so `user@host::module` stays rejected); the literal password is held client-side only for the SCRAM handshake and wiped at teardown. Server (`fastsync-server --daemon --password-file FILE`): the salted-PBKDF2 verifier store that modules with `auth users` are verified against. **Neither the password nor any replayable bearer value crosses the wire or is stored server-side** — the store holds a per-user salt plus derived keys, and the daemon proves the secret with a per-connection nonce challenge. The file must be private to its owner: both the client and server verify the exact inode they read (open-then-`fstat`, so the check cannot be raced) and refuse a `--password-file`/`--early-input` that is not owned by the current user or grants any group/other permission bit (mode 0600), mirroring the TLS private-key check. A process-substitution pipe (`--early-input <(vault ...)`) is still accepted when it satisfies those checks. See the Daemon Mode notes below for the file formats and the plaintext/TLS caveat |
| `--early-input=FILE` | Use FILE for daemon early exec | ⚠️ Caveat | Server-only (requires `--daemon`): a second credential-store file, same new-format grammar as `--password-file`, read before the listener accepts connections (a secrets-manager / process-substitution source). Its entries layer over `--password-file`: byte-identical verifiers dedupe, a conflicting verifier for the same user is a startup error. A daemon whose modules declare `auth users` must be given at least one of the two, or it refuses to start (fail closed) |
| `--hash-credentials=FILE`, `--iterations N` | Hash a plaintext credential file | ⚠️ Caveat | Server-only offline tool (A7): reads the `user:password` lines of FILE (same owner-only 0600 check) and prints one new-format store line per entry to stdout, then exits. `--iterations` sets the PBKDF2 work factor (default 600000, range 100000–10000000). Dependency-free and does not run a listener. Use its output as `--password-file` for `--daemon`. There is no auto-upgrade: a legacy store line is hard-rejected by the loader and must be regenerated |
**Daemon Mode notes (Wave A protocol 2.15.0; A7 auth protocol 2.19.0; MOTD no bump):** FastSync daemon mode is supported in FastSync's own protocol/config grammar, not rsync's SMB/daemon option encoding.
@@ -654,37 +684,37 @@ now transmits targets (the prior behavior was broken/partial); its status moved
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| Path escape detection | Ensure files stay within root | ✅ Implemented | `has_path_traversal()` + realpath |
| Symlink-safe delete | Skip symlinks in delete walk | ✅ Implemented | `delete_extras_walk()` |
| Protocol version check | Verify compatible versions | ✅ Implemented | `config_receive()` |
| Max data/string/chunk sizes | Prevent OOM attacks | ✅ Implemented | Per-message limits |
| Per-connection memory limit | 1GB per connection | ✅ Implemented | `MAX_CONNECTION_MEMORY` |
| `--max-alloc=SIZE` | Limit a single memory allocation | ✅ Implemented | Caps the largest single allocation; binary units, default 1G |
| `--trust-sender` | Trust remote sender's file list | ✅ Implemented | Long-form-only, receiver-local policy that never crosses the wire. The receiver skips its redundant up-front re-validation of the incoming file list (empty/`..` path rejection and the escaping-symlink-target containment), trusting the sender instead of double-checking (fewer checks, faster, potentially unsafe, matching rsync). Off by default. The low-level fd-relative confinement primitives (`file_open_secure_parent`, the O_NOFOLLOW parent walk, leaf/destination confinement) are deliberately KEPT even under `--trust-sender`, so a hostile sender still cannot write or link outside the authorized root (see Phase-5 notes below) |
| `--old-args` | Disable modern arg protection | ✅ Implemented | SSH-only; accepted for CLI compatibility but is now a **documented no-op**: FastSync always single-quote-escapes the remote server path and each `--remote-option` value (`ssh_build_remote_command`), so a metacharacter-bearing `--rsync-path` can never be interpreted by the remote shell. The flag no longer disables that quoting (the old raw-construction behavior was an injection foot-gun and is removed); the safety-relevant behavior is identical either way |
| `--ignore-missing-args` | Ignore missing source args | ✅ Implemented | FastSync has a single source-root argument (which always exists), so the "explicitly requested source arguments" are the `--files-from` entries and the flags only ever apply there (inert without `--files-from`, like `-R`). Without the flag a listed-but-missing entry stays a hard pre-transfer error (nothing is transferred). With it each missing entry is skipped: nothing is sent for it, it never enters the keep-set, and the run succeeds for the rest — an all-missing non-empty list succeeds transferring nothing, matching rsync. `--dirs` + `--files-from` missing entries are skipped the same way. Every skipped entry is logged and a per-run warning names the count, so the handling is never a silent no-op. Divergences: an EMPTY `--files-from` file stays a hard error in every mode (no argument was requested at all; rsync likewise reports "no source files specified"); missing-arg skipping only applies to the pre-transfer list validation, so an entry that is present at preflight and vanishes mid-transfer still fails (matching rsync, whose flag "does not affect subsequent vanished-file errors"); `--no-ignore-missing-args` is not a supported negation |
| `--delete-missing-args` | Delete missing source args | ✅ Implemented | Implies `--ignore-missing-args` (order-independent) and additionally removes each missing entry's destination mirror receiver-side. The mirror is computed exactly like a present sibling's wire path: the bare relative entry under `-R`, otherwise the full source-mirror path below the destination root. rsync parity, verified against the man page: it does **not** imply `--delete` generally and is "independent of any other type of delete processing" — unrelated destination extras are untouched unless `--delete` is also present. Composition with `--delete` + timing: the exact-path deletions commit with the manifest, early for `--delete-before`/`--delete-during`, else only after a fully-successful transfer (delete-after/commit). A non-empty directory mirror is removed only when `--force` or `--delete` is in effect (otherwise it is left with a warning and the run continues, like rsync); an absent mirror is a no-op. `--force` is deletion authority and is therefore gated by the server `--allow-delete` policy exactly like `--delete`/`--delete-missing-args`: without it the receiver clears the flag, so a client cannot use `--force` to recursively replace or remove a destination directory tree. An explicitly listed missing arg is a user request, not an excluded file: its deletion is never blocked by the filter-exclusion protection of excluded destination mirrors (a mirror sitting inside a filter-excluded directory is still removed). Safety/policy: gated by the server `--allow-delete` policy like `--delete`; the request paths cross the wire only in the delete-manifest frame and are confined by the same receiver validation as the keep-set (non-empty, relative, traversal-free, bounded by the per-section/per-frame manifest caps); the `--delay-updates` staging directory and basis snapshots are protected exactly as in the extras walker. Divergence: the missing-args deletions are not counted toward `--max-delete` (they are explicit per-path requests, not discovered extras). See the Phase-3 wire note below for the `PROTOCOL_VERSION` bump |
| Path escape detection | Ensure files stay within root | ✅ Parity | `has_path_traversal()` + realpath |
| Symlink-safe delete | Skip symlinks in delete walk | ✅ Parity | `delete_extras_walk()` |
| Protocol version check | Verify compatible versions | ✅ Parity | `config_receive()` |
| Max data/string/chunk sizes | Prevent OOM attacks | ✅ Parity | Per-message limits |
| Per-connection memory limit | Cap memory per connection | ✅ Parity | `MAX_CONNECTION_MEMORY` is **256 MiB per connection** (256 * 1024 * 1024 bytes), charged across protocol reservations and decompression/chunk allocations. This is a FastSync-internal bound with no direct rsync analogue |
| `--max-alloc=SIZE` | Limit a single memory allocation | ✅ Parity | Caps the largest single allocation; binary units, default 1G |
| `--trust-sender` | Trust remote sender's file list | ⚠️ Caveat | Long-form-only, receiver-local policy that never crosses the wire. The receiver skips its redundant up-front re-validation of the incoming file list (empty/`..` path rejection), trusting the sender instead of double-checking (fewer checks, faster, potentially unsafe, matching rsync). Off by default. **It no longer affects symlink targets** (protocol 2.23.0): targets are stored verbatim under `-l` regardless of `--trust-sender`; the flag only relaxes the receiver's path-list checks. The low-level fd-relative confinement primitives (`file_open_secure_parent`, the O_NOFOLLOW parent walk, leaf/destination confinement) are deliberately KEPT even under `--trust-sender`, so a hostile sender still cannot write or link outside the authorized root (see Phase-5 notes below) |
| `--old-args` | Disable modern arg protection | ⚠️ Caveat | SSH-only; accepted for CLI compatibility but is now a **documented no-op**: FastSync always single-quote-escapes the remote server path and each `--remote-option` value (`ssh_build_remote_command`), so a metacharacter-bearing `--rsync-path` can never be interpreted by the remote shell. The flag no longer disables that quoting (the old raw-construction behavior was an injection foot-gun and is removed); the safety-relevant behavior is identical either way |
| `--ignore-missing-args` | Ignore missing source args | ⚠️ Caveat | FastSync has a single source-root argument (which always exists), so the "explicitly requested source arguments" are the `--files-from` entries and the flags only ever apply there (inert without `--files-from`, like `-R`). Without the flag a listed-but-missing entry stays a hard pre-transfer error (nothing is transferred). With it each missing entry is skipped: nothing is sent for it, it never enters the keep-set, and the run succeeds for the rest — an all-missing non-empty list succeeds transferring nothing, matching rsync. `--dirs` + `--files-from` missing entries are skipped the same way. Every skipped entry is logged and a per-run warning names the count, so the handling is never a silent no-op. Divergences: an EMPTY `--files-from` file stays a hard error in every mode (no argument was requested at all; rsync likewise reports "no source files specified"); missing-arg skipping only applies to the pre-transfer list validation, so an entry that is present at preflight and vanishes mid-transfer still fails (matching rsync, whose flag "does not affect subsequent vanished-file errors"); `--no-ignore-missing-args` is not a supported negation |
| `--delete-missing-args` | Delete missing source args | ✅ Parity | Implies `--ignore-missing-args` (order-independent) and additionally removes each missing entry's destination mirror receiver-side. The mirror is computed exactly like a present sibling's wire path: the bare relative entry under `-R`, otherwise the full source-mirror path below the destination root. rsync parity, verified against the man page: it does **not** imply `--delete` generally and is "independent of any other type of delete processing" — unrelated destination extras are untouched unless `--delete` is also present. Composition with `--delete` + timing: the exact-path deletions commit with the manifest, early for `--delete-before`/`--delete-during`, else only after a fully-successful transfer (delete-after/commit). A non-empty directory mirror is removed only when `--force` or `--delete` is in effect (otherwise it is left with a warning and the run continues, like rsync); an absent mirror is a no-op. `--force` is deletion authority and is therefore gated by the server `--allow-delete` policy exactly like `--delete`/`--delete-missing-args`: without it the receiver clears the flag, so a client cannot use `--force` to recursively replace or remove a destination directory tree. An explicitly listed missing arg is a user request, not an excluded file: its deletion is never blocked by the filter-exclusion protection of excluded destination mirrors (a mirror sitting inside a filter-excluded directory is still removed). Safety/policy: gated by the server `--allow-delete` policy like `--delete`; the request paths cross the wire only in the delete-manifest frame and are confined by the same receiver validation as the keep-set (non-empty, relative, traversal-free, bounded by the per-section/per-frame manifest caps); the `--delay-updates` staging directory and basis snapshots are protected exactly as in the extras walker. Protocol 2.23.0 parity: the missing-args exact-path removals and the ordinary extras walk **draw from one shared `--max-delete` budget**, so a capped run stops part-way and exits 25 exactly like rsync. See the Phase-3 wire note below for the `PROTOCOL_VERSION` bump |
## 16. Batch Operations
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `--write-batch=FILE` | Write batched update to file | ✅ Implemented | Phase-6 residual-batch (client-only): runs the normal live transfer AND additionally emits a self-contained single-file batch of the whole source tree. The batch is a magic/format-version header followed by length-prefixed `chunk_serialize` blobs (full file images), replayable byte-identically by `--read-batch` on another machine with no source/server. `--write-batch` drives the single-threaded transfer path (the multithreaded path consumes the config before the separate batch scan pass). See the Phase-6 batch note below |
| `--only-write-batch=FILE` | Write batch without updating dest | ✅ Implemented | Phase-6 residual-batch: emits the self-contained batch FILE only — NO destination update, NO server connection. Requires a source (scans it and serializes the full tree to FILE). Same single-file format as `--write-batch`, so the file is re-appliable via `--read-batch=FILE DEST`. See the Phase-6 batch note below |
| `--read-batch=FILE` | Read batched update from file | ✅ Implemented | Phase-6 residual-batch: applies a previously written batch FILE locally to the destination. NO source and NO server — positional args are the destination only. Reads the magic/version header, then length-prefixed records, `chunk_deserialize`, and applies each via the confined `file_save_to_disk_full` path (same O_NOFOLLOW / `..`-rejection / root-confinement as the network receiver, so an attacker-controlled batch cannot escape the destination root). Malformed/truncated/oversized/traversal records are rejected cleanly. See the Phase-6 batch note below |
| `--write-batch=FILE` | Write batched update to file | ⚠️ Caveat | Phase-6 residual-batch (client-only): runs the normal live transfer AND additionally emits a self-contained single-file batch of the whole source tree. The batch is a magic/format-version header followed by length-prefixed `chunk_serialize` blobs (full file images), replayable byte-identically by `--read-batch` on another machine with no source/server. `--write-batch` drives the single-threaded transfer path (the multithreaded path consumes the config before the separate batch scan pass). See the Phase-6 batch note below |
| `--only-write-batch=FILE` | Write batch without updating dest | ⚠️ Caveat | Phase-6 residual-batch: emits the self-contained batch FILE only — NO destination update, NO server connection. Requires a source (scans it and serializes the full tree to FILE). Same single-file format as `--write-batch`, so the file is re-appliable via `--read-batch=FILE DEST`. See the Phase-6 batch note below |
| `--read-batch=FILE` | Read batched update from file | ⚠️ Caveat | Phase-6 residual-batch: applies a previously written batch FILE locally to the destination. NO source and NO server — positional args are the destination only. Reads the magic/version header, then length-prefixed records, `chunk_deserialize`, and applies each via the confined `file_save_to_disk_full` path (same O_NOFOLLOW / `..`-rejection / root-confinement as the network receiver, so an attacker-controlled batch cannot escape the destination root). Malformed/truncated/oversized/traversal records are rejected cleanly. See the Phase-6 batch note below |
## 17. Advanced
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `--stop-after=MINS` | Stop after N minutes | ✅ Implemented | Client-only sender stop deadline (Phase 6): computing `--stop-after=MINS` (a positive minute count; 0/negative/garbage rejected) and `--stop-at=TIME` (`HH:MM`, `HH:MM:SS`, or `now+N[smhd]`; a past time stops immediately). The transfer stops ELEGANTLY at the next chunk boundary: everything already fully sent is kept and applied, the run returns 0, and --delete (late/delete-after timing) does NOT wipe the destination — when the scan is cut short the partial keep-set manifest is suppressed with a warning (the delete walk is skipped rather than acting on an incomplete keep-set, so unscanned source mirrors survive). `--delete-before`/`--delete-during` still run their complete pre-scan (which ignores the deadline). Local client-only fields: never serialized into the wire config frame, so no PROTOCOL_VERSION bump. `--stop-after` uses CLOCK_MONOTONIC; `--stop-at` uses the wall clock. Works single-threaded and under `-j`/`--threads` (multithreaded). Divergence: rsync computes `--stop-after` from the run start; FastSync likewise. When both are given, the earlier of the two deadlines wins (checked per iteration). See the Phase-6 stop notes below |
| `--stop-at=TIME` | Stop at specified time | ✅ Implemented | Same feature as `--stop-after` (deadline transfer stop), absolute wall-clock form (`HH:MM[:SS]` or `now+N[smhd]`). See the row above and the Phase-6 stop notes |
| `--fsync` | Fsync every written file before publication | ✅ Implemented | |
| `--protocol=NUM` | Force older protocol version | ✅ Implemented | Forces the wire protocol version for this transfer. FastSync has exactly ONE wire format (`PROTOCOL_VERSION`, currently 2.22.0) with no downgrade/backward-compat code paths, so `--protocol=2.22.0` is accepted (it sets the version claim the client sends, which the server already requires to match exactly) and **every other value is rejected up front** with a clear error before any connection — it does not and cannot speak an older or virtual wire format. Divergence from rsync (which negotiates a range and downgrades to an integer 0..31): FastSync's honest contract is force-to-the-one-supported-value; a genuine downgrade would require a per-version compatibility layer that does not exist. Client-only; the server-side exact-match check is unchanged. `--protocol=2.21.0`/`2.20.0`/`2.19.0`/`2.18.0`/`2.18`/`2.17.0`/`2.16.0`/`2.15.0`/`216`/`31`/garbage are all rejected. See the Phase-6 protocol note below |
| `--iconv=CONVERT_SPEC` | Charset conversion | ✅ Implemented | Charset conversion of FILE NAMES (not content) at the protocol boundary via iconv(3): `--iconv=LOCAL[,REMOTE]` — the sender converts each local filename LOCAL→REMOTE before transmitting, and the receiver converts each wire filename REMOTE→LOCAL before creating/writing. The full CONVERT_SPEC is serialized into the config frame as a new trailing string field so the peer knows the wire charset; **PROTOCOL_VERSION bumped 2.15.0 → 2.16.0**. `LOCAL[,REMOTE]` parse: single charset ⇒ LOCAL==REMOTE (identity both ways); garbage rejected up front. Validation probes BOTH directions (a spec that only opens one way is refused, as is a NUL-emitting target charset like utf-16/utf-32/ucs-2, since filenames cannot contain NUL). An unrepresentable name (EILSEQ/EINVAL) fails that path cleanly with a logged `--iconv: cannot convert file name ...` and is never written mangled/truncated. Conversion is applied at EVERY wire-path site (regular/MKDIR/hardlink path+target/symlink path+target/SPECIAL, the delete manifest, the incremental-check path, and the `-s`/`chunk_serialize` embedded blob path), on both client and server (`--iconv` is also a server/daemon option). Zero overhead when unset. See the Phase-6 iconv notes below |
| `--checksum-seed=NUM` | Set checksum seed | ✅ Implemented | Sets the seed for FastSync's whole-file xxHash64 digest (full 64-bit seed) and for the delta path's per-block xxHash32 strong checksum (low 32 bits of the seed). An explicit seed deterministically changes every computed digest on BOTH endpoints (sender and receiver share the seed via the config frame, protocol 2.10.0), so identical runs with the same seed skip the same files and a changed seed changes the digests — the explicit-seed path that makes xxHash comparisons deterministic. `--checksum-choice=md5` has no seed and ignores it (documented). The value is a strict decimal 0..2⁶⁴-1 (blank, signed, or non-numeric values are rejected). Like rsync, a seed only matters where a digest is actually computed (`--checksum` or a basis-dir run, or a delta transfer); it does not by itself enable `--checksum`/`--delta`. Divergence from rsync: the default is seed 0, and FastSync never randomizes the seed (rsync uses a random per-transfer seed when `--checksum-seed` is unset); FastSync's unset default therefore reproduces its historical byte-for-byte behavior |
| `--secluded-args`, `-s` | Use protocol to send args | ⛔ Impossible/Divergence | Accepted for CLI compatibility (including the rsync short `-s`, Phase 7 Wave A) but a documented **no-op / divergence**. rsync's `-s` protects arguments from shell expansion by shipping them over the protocol; FastSync never passes remote arguments through a shell expansion boundary in the first place — its SSH transport builds the remote argv as **single-quote-escaped shell words** (`ssh_build_remote_command`), so the injection/leak that `-s` guards against does not exist and there is nothing to "seclude". Implementing a true arg-send protocol would mean replacing the argv-based SSH launch with an in-band argument channel, a large redesign of the transport that buys no security here. Chunk serialization remains the long-only `--chunk-serialization`. |
| `--no-OPTION` | Turn off implied option | ✅ Supported | Supported boolean FastSync options and archive-implied options; unsafe or value-taking options are rejected. |
| `--stop-after=MINS` | Stop after N minutes | ✅ Parity | Client-only sender stop deadline (Phase 6): computing `--stop-after=MINS` (a positive minute count; 0/negative/garbage rejected) and `--stop-at=TIME` (`HH:MM`, `HH:MM:SS`, or `now+N[smhd]`; a past time stops immediately). The transfer stops ELEGANTLY at the next chunk boundary: everything already fully sent is kept and applied, the run returns 0, and --delete (late/delete-after timing) does NOT wipe the destination — when the scan is cut short the partial keep-set manifest is suppressed with a warning (the delete walk is skipped rather than acting on an incomplete keep-set, so unscanned source mirrors survive). `--delete-before`/`--delete-during` still run their complete pre-scan (which ignores the deadline). Local client-only fields: never serialized into the wire config frame, so no PROTOCOL_VERSION bump. `--stop-after` uses CLOCK_MONOTONIC; `--stop-at` uses the wall clock. Works single-threaded and under `-j`/`--threads` (multithreaded). Divergence: rsync computes `--stop-after` from the run start; FastSync likewise. When both are given, the earlier of the two deadlines wins (checked per iteration). See the Phase-6 stop notes below |
| `--stop-at=TIME` | Stop at specified time | ⚠️ Caveat | Same feature as `--stop-after` (deadline transfer stop), absolute wall-clock form (`HH:MM[:SS]` or `now+N[smhd]`). See the row above and the Phase-6 stop notes |
| `--fsync` | Fsync every written file before publication | ✅ Parity | |
| `--protocol=NUM` | Force older protocol version | ❌ Divergent | Forces the wire protocol version for this transfer. FastSync has exactly ONE wire format (`PROTOCOL_VERSION`, currently 2.23.0) with no downgrade/backward-compat code paths, so `--protocol=2.23.0` is accepted (it sets the version claim the client sends, which the server already requires to match exactly) and **every other value is rejected up front** with a clear error before any connection — it does not and cannot speak an older or virtual wire format. Divergence from rsync (which negotiates a range and downgrades to an integer 0..31): FastSync's honest contract is force-to-the-one-supported-value; a genuine downgrade would require a per-version compatibility layer that does not exist. Client-only; the server-side exact-match check is unchanged. `--protocol=2.21.0`/`2.20.0`/`2.19.0`/`2.18.0`/`2.18`/`2.17.0`/`2.16.0`/`2.15.0`/`216`/`31`/garbage are all rejected. See the Phase-6 protocol note below |
| `--iconv=CONVERT_SPEC` | Charset conversion | ⚠️ Caveat | Charset conversion of FILE NAMES (not content) at the protocol boundary via iconv(3): `--iconv=LOCAL[,REMOTE]` — the sender converts each local filename LOCAL→REMOTE before transmitting, and the receiver converts each wire filename REMOTE→LOCAL before creating/writing. The full CONVERT_SPEC is serialized into the config frame as a new trailing string field so the peer knows the wire charset; **PROTOCOL_VERSION bumped 2.15.0 → 2.16.0**. `LOCAL[,REMOTE]` parse: single charset ⇒ LOCAL==REMOTE (identity both ways); garbage rejected up front. Validation probes BOTH directions (a spec that only opens one way is refused, as is a NUL-emitting target charset like utf-16/utf-32/ucs-2, since filenames cannot contain NUL). An unrepresentable name (EILSEQ/EINVAL) fails that path cleanly with a logged `--iconv: cannot convert file name ...` and is never written mangled/truncated. Conversion is applied at EVERY wire-path site (regular/MKDIR/hardlink path+target/symlink path+target/SPECIAL, the delete manifest, the incremental-check path, and the `-s`/`chunk_serialize` embedded blob path), on both client and server (`--iconv` is also a server/daemon option). Zero overhead when unset. See the Phase-6 iconv notes below |
| `--checksum-seed=NUM` | Set checksum seed | ✅ Parity | Sets the seed for FastSync's whole-file xxHash digest (full 64-bit seed) and for the delta path's per-block xxHash32 strong checksum (low 32 bits of the seed). **As of protocol 2.23.0 a seed of `0` — the default when the flag is unset — is randomized per transfer and the chosen seed is sent to the receiver**, exactly like rsync, so two runs against different content do not share a predictable seed; an explicit non-zero seed is used verbatim, so an explicit seed deterministically reproduces every computed digest on BOTH endpoints (the seed crosses in the config frame). `--checksum-choice=md5` has no seed and ignores it (documented). The value is a strict decimal 0..2⁶⁴-1 (blank, signed, or non-numeric values are rejected). Like rsync, a seed only matters where a digest is actually computed (`--checksum` or a basis-dir run, or a delta transfer); it does not by itself enable `--checksum`/`--delta` |
| `--secluded-args`, `-s` | Use protocol to send args | ❌ Divergent | Accepted for CLI compatibility (including the rsync short `-s`, Phase 7 Wave A) but a documented **no-op / divergence**. rsync's `-s` protects arguments from shell expansion by shipping them over the protocol; FastSync never passes remote arguments through a shell expansion boundary in the first place — its SSH transport builds the remote argv as **single-quote-escaped shell words** (`ssh_build_remote_command`), so the injection/leak that `-s` guards against does not exist and there is nothing to "seclude". Implementing a true arg-send protocol would mean replacing the argv-based SSH launch with an in-band argument channel, a large redesign of the transport that buys no security here. Chunk serialization remains the long-only `--chunk-serialization`. |
| `--no-OPTION` | Turn off implied option | ✅ Parity | Supported boolean FastSync options and archive-implied options; unsafe or value-taking options are rejected. |
---
@@ -692,7 +722,7 @@ now transmits targets (the prior behavior was broken/partial); its status moved
**Phase 5 notes (remote-option wave):** `--remote-option=OPT` (long form only) and `--trust-sender` landed here.
- `--remote-option` is CLIENT-only and never serialized into the binary config frame. On the SSH transport the client forwards each value to the remote server by appending it to the remote command line in `ssh_build_remote_command()`, after ` --stdio`, as an individually single-quoted shell word (`'...'` with `'\''` for embedded quotes). Values are validated at CLI parse time (non-empty; no ASCII control characters) and rejected otherwise, and a non-conforming value is refused again in the command builder, so shell metacharacters (`;`, `&`, `|`, backticks, `$()`, quotes) can never break out of the quoting to inject an unrelated remote command — including after a client-side `--` separator, whose arguments are never forwarded anyway. Because the remote options affect the *remote server invocation*, not the transmitted config, the wire frame layout is unchanged, but `PROTOCOL_VERSION` was bumped **2.13.0 → 2.14.0** as the Phase-5 lockstep release marker (a 2.14 client against a 2.13 server fails the version check cleanly rather than the old server rejecting an unfamiliar forwarded argv later). Divergence: rsync's short `-M` form of `--remote-option` was intentionally NOT implemented at that time because `-M` was FastSync metadata mode; **Phase 7 Wave A later freed `-M` for `--remote-option` and moved metadata to long-only `--preserve`** (see the Sending Options table).
- `--trust-sender` is a receiver-local policy: it never crosses the wire (the sender's value is never serialized, so a wire peer can never enable it). On the receiving process it skips the up-front re-validation of the incoming file list (empty/`..` path rejection and the escaping-symlink-target containment), trusting the sender's list instead of double-checking — fewer checks, faster, and potentially unsafe, matching rsync. It is OFF by default (`config.trust_sender`). As a deliberate safety floor, the low-level fd-relative confinement primitives are NOT disabled: `file_open_secure_parent()` (O_NOFOLLOW walk, `..` rejection, root containment) and leaf/destination confinement still hold, so even under `--trust-sender` a hostile sender cannot write or create a symlink outside the authorized root — the relaxation only removes the redundant list-layer double-checks, never the root-confinement guarantees.
- `--trust-sender` is a receiver-local policy: it never crosses the wire (the sender's value is never serialized, so a wire peer can never enable it). On the receiving process it skips the up-front re-validation of the incoming file list (empty/`..` path rejection), trusting the sender's list instead of double-checking — fewer checks, faster, and potentially unsafe, matching rsync. It is OFF by default (`config.trust_sender`). Since protocol 2.23.0 it does **not** gate symlink-target handling: `-l` stores targets verbatim either way. As a deliberate safety floor, the low-level fd-relative confinement primitives are NOT disabled: `file_open_secure_parent()` (O_NOFOLLOW walk, `..` rejection, root containment) and leaf/destination confinement still hold, so even under `--trust-sender` a hostile sender cannot write or place a *path* outside the authorized root — the relaxation only removes the redundant list-layer double-checks, never the root-confinement guarantees for paths and placements.
The estimates below cover the currently unimplemented features in this document. They assume one engineer familiar with the codebase, include implementation and focused tests, and exclude production rollout time. A feature should not be marked implemented until its behavior is tested in both local and SSH/TCP paths where applicable.
@@ -796,7 +826,7 @@ These are the hardest compatibility items because they require durable formats o
**Phase 6, Wave B (iconv) shipping note (PROTOCOL 2.15.0 → 2.16.0):** `--iconv=LOCAL[,REMOTE]` converts file NAMES at the wire boundary (never content). The full CONVERT_SPEC is serialized into the config frame as a new trailing string field (empty→NULL canonicalized), so both ends share the same wire charset interpretation; this required the PROTOCOL bump because the frame is a strict ordered sequence and a peer that does not parse the new trailing field would desynchronize. Each end derives LOCAL (its own charset) and REMOTE (the wire charset): the sender opens LOCAL→REMOTE and converts every transmitted filename; the receiver opens REMOTE→LOCAL and converts every received filename before creating/writing. Conversion is applied at every wire-path site (regular/MKDIR/hardlink path+target/symlink path+target/SPECIAL, the delete manifest keep/protected/missing entries, the incremental-check path, and the embedded `-s`/chunk-blob path). A name it cannot convert (EILSEQ/EINVAL) is failed cleanly with a logged `--iconv: cannot convert file name ...` and is never written truncated/mangled. Validation probes both directions up front (both the sender local→remote and the receiver remote→local, and, for a server/daemon with its own `--iconv`, the client-REMOTE→server-LOCAL pair) so an unusable spec is rejected before the connection rather than mid-transfer, and NUL-emitting target charsets (utf-16/utf-32/ucs-2) are refused because filenames cannot contain NUL. Divergence documented upstream: the receiver does NOT half-swap; the wire charset always comes from the sender's REMOTE half, so a server whose local charset differs from the client's LOCAL must declare it with its own `--iconv`. Conversion is process-global and runs on a single thread per process (sender thread / receiver-loop thread), initialized before worker threads start and freed after they join.
**Phase 6, Wave C (protocol-version) shipping note (no PROTOCOL_VERSION change):** `--protocol=NUM` lets the client force the wire protocol version for a transfer. FastSync's protocol is a single lockstep format: the config frame is a strict ordered sequence and the server requires the client's version string to equal `PROTOCOL_VERSION` exactly (`config_receive_with_validate`, src/shared/config.c) — there are no older-format code paths and no downgrade/negotiation machinery, so a lower/higher/virtual version can never be spoken. The honest contract is therefore: `--protocol=2.22.0` (the current `PROTOCOL_VERSION`, as of the preserve-attribute split wave) is accepted and stored into the client's `version` claim (which `config_send` already transmits), and every other value — `2.21.0`, `2.20.0`, `2.19.0`, `2.18.0`, `2.18`, `2.17.0`, `2.16.0`, `2.15.0`, `3.0.0`, rsync-integer spellings like `216`/`31`, garbage, empty — is rejected up front in `validate_config()` before any connection, with a clear error that FastSync supports only its current wire protocol and cannot speak an older or virtual one. Implementation is client-only: a server-side `--protocol` is intentionally not added because the server has no negotiation (it only enforces exact match), and it could only ever be the current version. This preserves (and slightly tightens) existing validation: the client now also refuses to launch with a version it cannot actually speak, rather than only the server rejecting it later. A genuine downgrade would require a per-version compatibility layer for every frame/feature added since (append 2.10, preallocate 2.11, hardlinks 2.12, devices/specials/symlink-trust/xattr 2.13, remote-option 2.14, daemon module/auth 2.15, iconv 2.16, dir/symlink times 2.17, privilege flags --super/--copy-as 2.18, SCRAM daemon auth 2.19, packed metadata 2.20, error-detail/dry-run 2.21, preserve-attribute split 2.22) and is intentionally out of scope — documented divergences from rsync's integer-negotiated downgrade remain.
**Phase 6, Wave C (protocol-version) shipping note (no PROTOCOL_VERSION change):** `--protocol=NUM` lets the client force the wire protocol version for a transfer. FastSync's protocol is a single lockstep format: the config frame is a strict ordered sequence and the server requires the client's version string to equal `PROTOCOL_VERSION` exactly (`config_receive_with_validate`, src/shared/config.c) — there are no older-format code paths and no downgrade/negotiation machinery, so a lower/higher/virtual version can never be spoken. The honest contract is therefore: `--protocol=2.23.0` (the current `PROTOCOL_VERSION`, as of the rsync-parity wave) is accepted and stored into the client's `version` claim (which `config_send` already transmits), and every other value — `2.22.0`, `2.21.0`, `2.20.0`, `2.19.0`, `2.18.0`, `2.18`, `2.17.0`, `2.16.0`, `2.15.0`, `3.0.0`, rsync-integer spellings like `216`/`31`, garbage, empty — is rejected up front in `validate_config()` before any connection, with a clear error that FastSync supports only its current wire protocol and cannot speak an older or virtual one. Implementation is client-only: a server-side `--protocol` is intentionally not added because the server has no negotiation (it only enforces exact match), and it could only ever be the current version. This preserves (and slightly tightens) existing validation: the client now also refuses to launch with a version it cannot actually speak, rather than only the server rejecting it later. A genuine downgrade would require a per-version compatibility layer for every frame/feature added since (append 2.10, preallocate 2.11, hardlinks 2.12, devices/specials/symlink-trust/xattr 2.13, remote-option 2.14, daemon module/auth 2.15, iconv 2.16, dir/symlink times 2.17, privilege flags --super/--copy-as 2.18, SCRAM daemon auth 2.19, packed metadata 2.20, error-detail/dry-run 2.21, preserve-attribute split 2.22, rsync-parity wave 2.23) and is intentionally out of scope — documented divergences from rsync's integer-negotiated downgrade remain.
**Phase-1/2 selection-and-update status correction (docs):** `-I/--ignore-times`, `--size-only`, `-@/--modify-window`, `--existing`, `--ignore-existing`, `-u/--update`, `-W/--whole-file`, and `--compress-threads` were previously listed as not-implemented in this document but are in fact fully implemented and tested on `dev`. This pass corrects the matrix to match the code. The realistic model of these is that FastSync is a *sender-driven* whole-tree copy, so the size+mtime quick-check and all three receiver-policy skips (`--existing`, `--ignore-existing`, `-u`) are evaluated against the **destination** on the receiver side, and their booleans cross the wire in the config frame. `-I`/`--size-only`/`--modify-window` modify the `--incremental` per-file `STATUS_CHECK` handshake's match predicate (`-I` disables the mtime leg and forces transfer; `--size-only` drops only the mtime leg; `--modify-window` adds tolerance to `metadata_mtime_matches`); they require `--incremental` (or a basis dir) to have a handshake to affect, mirroring how they only matter where a quick-check exists in rsync. `--existing`/`--ignore-existing`/`-u` are receiver write-time policies (skipping the write / newer-destination guard) applied across the regular-file, `--delay-updates`-staged, hardlink-sibling, and special/device paths; `-u` implies `-M` metadata and uses a second-then-nanosecond strict `>` newer check; both correctly influence `--remove-source-files` (a skipped source is not removed). `-W/--whole-file` disables block-level delta (opt-in via `--delta`), folded into the wire `use_delta` so no protocol bump was needed, and makes `--fuzzy` inert; `--append`/`--append-verify` are rejected with `-W`. `--compress-threads=NUM` (1..64, client-only, never crosses the wire) sizes the zstd compression worker pool. No code was changed by this correction; the implementation had landed in earlier merge waves (feat/ignore-times, feat/ignore-existing via the newer `file_to_disk_secure_no_replace`/`linkat EEXIST` path, feat/size-only, feat/modify-window, feat/whole-file, feat/update, compression-threads).
@@ -819,17 +849,17 @@ These are the last compatibility items and the closing phase toward rsync flag p
| `-T` / `--timeout` | `-T` = `--temp-dir` | → `--timeout` (long-only) |
| `-a` / `--archive` (= `-c -m -M`) | `-a` = `-rlptD` | → becomes **real rsync `-a`** after the renames |
**Wave B — Output & filesystem completion (✅ implemented).** `-S`/`--sparse` (`⚠️→✅`): real hole preservation — a sparse-aware writer (`write_all_sparse`) skips all-zero runs ≥ 4096 bytes with `lseek(SEEK_CUR)` and `ftruncate`s the final size, wired into both the atomic temp+rename store and `--inplace` receiver-side with **no wire change** (the full file image is already in memory; the ftruncate presize is kept). `-P` (`⚠️→✅`): interrupted-write retention — on a save failure after data reached the temp fd, `--partial` now renames the already-written temp to the destination path (best-effort; falls through to the normal unlink on failure, never retains when `--partial` is off) so a later `--append`/`--append-verify` run can resume. `--block-size=SIZE` (`⚠️→✅`): promoted after verification — `--block-size` is now an alias for `--delta-block`, both set `config->delta_block_size`, which the delta engine already honored end-to-end (`delta_signature_create_seeded` + `delta_apply`); out-of-range values keep the default. `--fake-super` (`⚠️→✅`): added `fake_super_restore_fd` to parse and re-apply the recorded `user.fastsync.stat` record fd-relative (fchown best-effort/non-root skipped, fchmod, futimens); a save under `--fake-super` now re-applies the recorded attrs instead of only recording them, with the recording format unchanged. `--stderr=client` (`⚠️→⛔ Impossible/Divergence`): FastSync has no rsync client-message channel, and `client` is rejected at CLI parse — the rejection is the documented behavior (unit-tested). `-N`/`--crtimes` (`⚠️→⛔ Impossible/Divergence`): birth-times cannot be set by any portable fs call (`utimensat` sets only atime/mtime); capture/transmit stays, setting is impossible, the flag is accepted and safely inert. Review-hardening (post-eval): fake-super replay applies the mode through the same sanitization as the normal metadata path (group/other write bits are never granted); `--sparse` takes precedence over `--preallocate` (posix_fallocate skipped so holes survive); `--partial` retention is disabled for `--no_replace` (ignore/existing) and only marks a write-attempt after the actual write begins; `--block-size=SIZE`/`--delta-block=SIZE` inline forms are accepted.
**Wave B — Output & filesystem completion (✅ implemented).** `-S`/`--sparse` (`⚠️→✅`): real hole preservation — a sparse-aware writer (`write_all_sparse`) skips all-zero runs ≥ 4096 bytes with `lseek(SEEK_CUR)` and `ftruncate`s the final size, wired into both the atomic temp+rename store and `--inplace` receiver-side with **no wire change** (the full file image is already in memory; the ftruncate presize is kept). `-P` (`⚠️→✅`): interrupted-write retention — on a save failure after data reached the temp fd, `--partial` now renames the already-written temp to the destination path (best-effort; falls through to the normal unlink on failure, never retains when `--partial` is off) so a later `--append`/`--append-verify` run can resume. `--block-size=SIZE` (`⚠️→✅`): promoted after verification — `--block-size` is now an alias for `--delta-block`, both set `config->delta_block_size`, which the delta engine already honored end-to-end (`delta_signature_create_seeded` + `delta_apply`); out-of-range values keep the default. `--fake-super` (`⚠️→✅`): added `fake_super_restore_fd` to parse and re-apply the recorded `user.fastsync.stat` record fd-relative (mode/time only — protocol 2.23.0: **never a real chown**; the resolved owner is recorded for a later privileged restore); a save under `--fake-super` now re-applies the recorded attrs instead of only recording them, with the recording format unchanged. `--stderr=client` (`⚠️→❌ Divergent`): FastSync has no rsync client-message channel, and `client` is rejected at CLI parse — the rejection is the documented behavior (unit-tested). `-N`/`--crtimes` (`⚠️→❌ Divergent`): birth-times cannot be set by any portable fs call (`utimensat` sets only atime/mtime); capture/transmit stays, setting is impossible, the flag is accepted and safely inert. Review-hardening (post-eval): fake-super replay applies the mode through the shared `metadata_mode_for_policy` helper (protocol 2.23.0: exactly the source mode under `-p`, with no masking); `--sparse` takes precedence over `--preallocate` (posix_fallocate skipped so holes survive); `--partial` retention is disabled for `--no_replace` (ignore/existing) and only marks a write-attempt after the actual write begins; `--block-size=SIZE`/`--delta-block=SIZE` inline forms are accepted.
**Wave C — Devices & special files (finalize statuses + tests) (✅ implemented).** The four special-file rows are finalized with coverage tests. `--devices`, `--copy-devices`, and `--write-devices` are **✅ Implemented**, each with a documented, safety-driven divergence: device-node creation is privilege-gated, so a receiver without `CAP_MKNOD` skips that entry with a warning (a per-entry skip, never a transfer failure); `--copy-devices` copies a device/FIFO's reported size into an ordinary regular file (a size-bounded safe divergence from rsync's unbounded dd-like read); `--write-devices` writes only into an existing char/block node under the confined receive root and skips every unusable target rather than clobbering or aborting. `--specials` is classified **⛔ Impossible/Divergence** for one reason only: **FIFO recreation works** (unprivileged `mkfifo`, asserted under CI), but **sockets cannot be recreated by any standard filesystem call**, so a source socket is skipped with an explicit note. Tests assert FIFO recreation, the safe socket skip, the regular-file result of `--copy-devices`, the skipped/missing and non-device `--write-devices` targets, and (root-gated) real device-node creation; a root runner additionally drops the receiver to an unprivileged user to assert the `CAP_MKNOD` skip is graceful.
**Wave C — Devices & special files (finalize statuses + tests) (✅ implemented).** The four special-file rows are finalized with coverage tests. `--devices`, `--copy-devices`, and `--write-devices` are **✅ Implemented**, each with a documented, safety-driven divergence: device-node creation is privilege-gated, so a receiver without `CAP_MKNOD` skips that entry with a warning (a per-entry skip, never a transfer failure); `--copy-devices` copies a device/FIFO's reported size into an ordinary regular file (a size-bounded safe divergence from rsync's unbounded dd-like read); `--write-devices` writes only into an existing char/block node under the confined receive root and skips every unusable target rather than clobbering or aborting. `--specials` reclassified from **⛔ Impossible/Divergence** to **✅ Parity** in protocol 2.23.0: **FIFO recreation works** (unprivileged `mkfifo`) **and unix sockets are recreated** with `mknod(S_IFSOCK)`, which Linux permits unprivileged (the flag previously assumed sockets were impossible — see the `--specials` row). Tests assert FIFO recreation, socket recreation, the regular-file result of `--copy-devices`, the skipped/missing and non-device `--write-devices` targets, and (root-gated) real device-node creation; a root runner additionally drops the receiver to an unprivileged user to assert the `CAP_MKNOD` skip is graceful.
**Wave D — Times superstructure & arg-protection no-ops (✅ implemented, `--secluded-args` ⛔).** `-O`/`--omit-dir-times` and `-J`/`--omit-link-times` are now **real modifiers** (both `🔄 → ✅ Implemented`), reversing the old "never preserves directory/symlink times" divergence:
**Wave D — Times superstructure & arg-protection no-ops (✅ implemented, `--secluded-args` ❌).** `-O`/`--omit-dir-times` and `-J`/`--omit-link-times` are now **real modifiers** (both `🔄 → ✅ Implemented`), reversing the old "never preserves directory/symlink times" divergence:
- **Directory times.** The recursive scanner captures every traversed source directory's metadata (mtime, plus atime under `-U`) into a per-transfer list — two paths are covered: the sequential `DirectoryScanner` captures each opened directory (including the transfer root), and the parallel scanner captures both the root in `parallel_scanner_create_with_options` and each worker's subdirectories in `open_next_directory` (appends are guarded by a mutex shared with the sender's pipeline context). The sender transmits them in trailing `STATUS_DIR_TIMES` frames (each: int count + count × (wire path, metadata) pairs) sent **after all file data and after the optional delete manifest**, just before `STATUS_FINISHED`. A tree larger than `MAX_MANIFEST_ENTRIES` (1 048 576) directories is chunked into repeated frames, each within the receiver's per-frame bound. A dir-time entry is RECORD-ONLY (`file->dir_time_only`): `file_save_to_disk_full` returns `FILE_SAVE_SKIPPED` without creating anything, so a source directory that was empty (or pruned by `-m/--prune-empty-dirs`) is never resurrected. The receiver accumulates received directory metadata in a `DirTimeList` and applies it only at the very end — after the entire stream, after the commit-style `--delete` deletion, and after `--delay-updates` publication — because creating or removing a child bumps the parent's mtime. Application is fd-relative/walk-confined (`file_open_secure_parent` + `utimensat(..., AT_SYMLINK_NOFOLLOW)`) and best-effort per entry: an absent path (an intentionally uncreated empty dir) is skipped QUIETLY and only a real existing directory is stamped. `-O` (config boolean, already on the wire) makes the receiver skip the whole set. The single-threaded sink applies in `receiver_send_success_frame`; the `-j`/`--threads` sink accumulates in `write_thread` and server.c applies after both threads join and the deletion commits.
- **Symlink times/owner/mode.** `STATUS_SYMLINK` already carried metadata; the receiver now applies it with no-follow primitives only: `utimensat(..., AT_SYMLINK_NOFOLLOW)`, best-effort `fchmodat(..., AT_SYMLINK_NOFOLLOW)` (honest no-op where unsupported, e.g. Linux), and policy-gated `fchownat(..., AT_SYMLINK_NOFOLLOW)` via a new `identity_apply_ownership_link` that shares the identity resolver with the fd path. `-J` suppresses only the timestamps; ownership stays governed by the identity opt-in (`--numeric-ids`/`--usermap`/`--groupmap`/`--chown`) exactly like regular files. A symlink has no children, so this is applied immediately at creation.
- **Wire:** the shared `STATUS_DIR_TIMES` frame (and metadata on `STATUS_MKDIR` for `--dirs` entries) is a frame-sequence change, so `PROTOCOL_VERSION` was bumped **2.16.0 → 2.17.0**; every version-sensitive test (`--protocol` accepted/rejected values) was updated. The config-frame layout itself is unchanged (the omit booleans already crossed). Non-metadata and `--no-preserve` transfers send no `STATUS_DIR_TIMES` frame and no directory metadata, keeping them byte-identical.
`--secluded-args` (`🔄 → ⛔ Impossible/Divergence`): a true arg-send protocol would replace the argv-based SSH launch with an in-band channel, and FastSync already builds the remote SSH argv injection-safe (single-quote-escaped shell words), so there is no argument-leak to close; the already-safe behavior is documented in the row and no transport change is made.
`--secluded-args` (`🔄 → ❌ Divergent`): a true arg-send protocol would replace the argv-based SSH launch with an in-band channel, and FastSync already builds the remote SSH argv injection-safe (single-quote-escaped shell words), so there is no argument-leak to close; the already-safe behavior is documented in the row and no transport change is made.
**Wave E (LAST) — Privilege: `--super`/`--no-super` and `--copy-as=USER[:GROUP]` (✅ implemented).** FastSync adopts a **safe-subset + clear-refusal** privilege model: it never blind-elevates and never calls `setuid`/`seteuid`/`setgid`. All privileged operations remain fd-relative and confined below the authorized receive root.
@@ -839,9 +869,147 @@ These are the last compatibility items and the closing phase toward rsync flag p
**Wire:** two trailing config-frame blocks after the `--iconv` spec, in fixed order — `send_privilege_options`/`receive_privilege_options` (one `super_mode` int, validated `0..2`), then `send_copy_as_options`/`receive_copy_as_options` (presence int + two int32 ids, validated `>= 0`, with `copy_as_set ⇒ use_metadata`). `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Divergences from rsync:** rsync's `--super` elevates the receiver and `--copy-as` actually switches its credentials; FastSync never elevates and only permits/forwards confined attempts, and `--copy-as` forces ownership rather than switching identity.
**Post-Phase-7 Summary (after Waves A–E).** ✅143 / 🔀0 / ⛔4 / ⚠️0 / 🔄0 / ❌0 = 147. The 3 `🔀 Alt Arg` rows (`-a`, `-p`, `-z`) are ✅ (Wave A). All 10 prior `⚠️ Partial` rows are resolved to ✅ (`-S`, `-P`, `--block-size`, `--fake-super`, `--devices`, `--copy-devices`, `--write-devices`) or ⛔ (`--stderr=client`, `-N/--crtimes`, `--specials` for the impossible socket case). The 3 `🔄 Compatibility No-op` rows are resolved: `-O`/`-J` are now real ✅ (Wave D), `--secluded-args` is ⛔. The **Impossible/Divergence** bucket holds the 4 physically-impossible/divergent flags: `--stderr=client`, `-N/--crtimes`, `--specials` (sockets), `--secluded-args`. The last two `❌ Not Implemented` rows — `--super` and `--copy-as=USER[:GROUP]` — are now ✅ (Wave E). **No `❌ Not Implemented` rows remain.**
**Current honest status (protocol 2.23.0).** ✅ Parity 83 / ⚠️ Caveat 63 / ❌ Divergent 4 = 150 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. The reclassification makes the differences explicit and the rsync-parity wave closed the genuine gaps (short options, clustering, checksum/compression choices, seed randomization, timeout defaults, delete scoping and partial limits, verbatim symlink storage, socket recreation, `--chmod`, and more — see the next section). The four ❌ rows are `--stderr=client` (no rsync client-message channel), `-N/--crtimes` (no portable setter), `--protocol=NUM` (only the current wire version is accepted), and `-s/--secluded-args` (accepted no-op). `--specials` is now ✅ because sockets are recreated with `mknod(S_IFSOCK)`. **No `❌ Not Implemented` rows remain.**
**Preserve-attribute split (protocol 2.21.0 → 2.22.0) — ✅ implemented.** FastSync splits the former single metadata bundle into four independent, rsync-compatible per-attribute flags — `-p/--perms`, `-t/--times`, `-o/--owner`, `-g/--group` — each with a negation (`--no-perms`/`--no-times`/`--no-owner`/`--no-group`, short `--no-p`/`--no-t`/`--no-o`/`--no-g`), plus `--no-preserve` clearing all four. `-a/--archive` is now full rsync `-rlptgoD` (owner and group included, though their application stays privilege-gated), `-A/--acls` and `--chmod` imply `-p`, `-X/--xattrs` does not, `-E/--executability` sets only executability, and `-U`/`-N` do not imply `-t`. `--incremental`/`--delta` still auto-preserve perms+times unless the user explicitly negated them. Wire: the binary config frame gains four appended booleans (`preserve_perms`/`preserve_times`/`preserve_owner`/`preserve_group`) after `omit_link_times`, so `PROTOCOL_VERSION` is bumped **2.21.0 → 2.22.0**; the fixed-width `FileMetadata` layout is unchanged and the receiver gates the metadata frame on a derived `use_metadata`. Receiver behavior: each attribute is applied independently, directory modes are applied under `-p` (at the end of the transfer, alongside dir times), symlink mode under `-p`, and `-O/--omit-dir-times` suppresses directory times only. Documented divergences: (a) a client-supplied mode never grants group/other write — `S_IWGRP|S_IWOTH` are stripped for files, directories, symlinks, and specials (rsync's `-p` preserves them exactly); (b) a brand-new file without `-p` gets `source_mode & ~umask` (sanitized) when metadata is present, else the historical fixed `0644`; (c) `--chmod` implies `-p` (rsync does not); (d) `-o`/`-g` map by name on the receiver with a raw-numeric fallback (only numeric ids cross the wire); (e) a daemon module without `client owner = yes` does not refuse a plain `-a`/`-o`/`-g` — it forces super off, applies no ownership, and logs a warning, while explicit `--chown`/`--usermap`/`--groupmap`/`--numeric-ids`/`--copy-as`/`--super` are still refused.
**Preserve-attribute split (protocol 2.21.0 → 2.22.0) — ✅ implemented.** FastSync splits the former single metadata bundle into four independent, rsync-compatible per-attribute flags — `-p/--perms`, `-t/--times`, `-o/--owner`, `-g/--group` — each with a negation (`--no-perms`/`--no-times`/`--no-owner`/`--no-group`, short `--no-p`/`--no-t`/`--no-o`/`--no-g`), plus `--no-preserve` clearing all four. `-a/--archive` is now full rsync `-rlptgoD` (owner and group included, though their application stays privilege-gated), `-A/--acls` implies `-p`, `-X/--xattrs` does not, `-E/--executability` sets only executability, and `-U`/`-N` do not imply `-t`. `--incremental`/`--delta` still auto-preserve perms+times unless the user explicitly negated them. Wire: the binary config frame gains four appended booleans (`preserve_perms`/`preserve_times`/`preserve_owner`/`preserve_group`) after `omit_link_times`, so `PROTOCOL_VERSION` is bumped **2.21.0 → 2.22.0**; the fixed-width `FileMetadata` layout is unchanged and the receiver gates the metadata frame on a derived `use_metadata`. Receiver behavior: each attribute is applied independently, directory modes are applied under `-p` (at the end of the transfer, alongside dir times), symlink mode under `-p`, and `-O/--omit-dir-times` suppresses directory times only. Documented divergences as of 2.22.0, **all but (d)/(e) removed by the rsync-parity wave (protocol 2.23.0)**: (a) the mode-masking divergence is **gone** — under `-p` the source mode is now copied exactly, including `S_IWGRP`/`S_IWOTH` and setuid/setgid/sticky; (b) a brand-new file without `-p` still gets `source_mode & ~umask` when metadata is present (else the historical fixed `0644`), and a new *directory* without `-p` still uses FastSync's `0755` default; (c) the `--chmod`-implies-`-p` divergence is **gone** — `--chmod` no longer implies `-p` (rsync parity); (d) `-o`/`-g` map by name on the receiver with a raw-numeric fallback (only numeric ids cross the wire); (e) a daemon module without `client owner = yes` does not refuse a plain `-a`/`-o`/`-g` — it forces super off, applies no ownership, and logs a warning, while explicit `--chown`/`--usermap`/`--groupmap`/`--numeric-ids`/`--copy-as`/`--super` are still refused.
## Rsync-Parity Wave (protocol 2.23.0)
This wave closed the remaining CLI, filesystem, ownership, deletion, and output
gaps against rsync 3.4.1. It is a wire change: `PROTOCOL_VERSION` moved
**2.22.0 → 2.23.0** because the delete manifest gained a synchronized-directory
section and the terminal status gained `STATUS_DELETE_LIMIT` (see the deletion
notes above). Everything below is implemented and covered by unit and
integration tests unless it is explicitly listed as a limitation.
### CLI parsing
- **Short options now parsed:** `-r` (`--recursive`), `-b` (`--backup`),
`-L` (`--copy-links`), and `-B` (`--block-size`/`--delta-block`) are accepted
as rsync spells them.
- **rsync short-option clustering:** a token is expanded before parsing, so
`-av` → `-a -v`, `-aAX` → `-a -A -X`, `-rlpt` → `-r -l -p -t`, and so on.
A value-taking short option consumes the remainder of its token
(`-B1000` → `-B 1000`, `-essh` → `-e ssh`, `-MOPT` → `-M OPT`), with an
optional leading `=` dropped (`-B=1000`); a value-taking option written alone
takes the next argv entry, which is copied verbatim so a value that happens to
start with `-` (e.g. `--filter "- *.tmp"`) is not mistaken for a cluster.
- **Inline/attached long values:** `--opt=value` is accepted uniformly, and each
expanded token is mapped back to its original argv index so positional
arguments stay correct.
- **`-c` implies the checksum quick-check.** `-c`/`--checksum` sets the
incremental checksum comparison rather than doing nothing on its own; like
rsync, `-c` does not imply `-t`.
### Checksums and compression
- **`--checksum-choice`/`--cc`** accepts `xxh64` (default), `xxhash`, `xxh3`,
`xxh128`, `md5`, and `auto`; `md4`, `sha1`, `none`, and the two-name
`transfer,pre-transfer` form are **rejected by name**.
- **`--checksum-seed=0` is randomized per transfer** (the chosen seed is sent to
the receiver), matching rsync; an explicit non-zero seed is used verbatim.
- **`--compress-choice`/`--zc`** accepts `zstd` (default), `none`, and `auto`;
rsync's `lz4`/`zlib`/`zlibx` are **rejected by name**.
- **`--skip-compress`** uses rsync 3.4.1's built-in default suffix list when no
list is supplied; an explicit list replaces it.
- **`--no-whole-file`** is accepted as the rsync spelling that clears
`-W`/`--whole-file`.
### Timeouts and limits
- **`--timeout` defaults to 0 (disabled) and `--contimeout` to 60 s; `0`
disables either**, matching rsync.
- **`--max-alloc=0` means "no local allocation limit"** (rsync semantics). A
standalone server still keeps its own ceiling for the peer it serves.
### Filesystem and deletion semantics
- **`--temp-dir` is confined to the receive root on the receiver:** a relative
dir resolves below it; an absolute path or one containing `..` is rejected.
An `EXDEV` install falls back to a non-atomic copy instead of aborting.
- **Deletion scoping:** the manifest carries the synchronized directories, so
the extras walk only visits their subtrees; `--files-from` subsets no longer
delete untransmitted paths outside the listed directories.
- **`--delete-excluded`** removes filter-excluded mirrors but never
`--max-size`/`--min-size`-pruned mirrors (separate, always-on protection).
- **Destination symlinks** are unlinked by name, never followed; a directory
still holding one survives.
- **`--max-delete=N` is partial:** delete up to N, skip the rest, exit **25**.
`--delete-missing-args` removals draw from the same budget.
- **`--force` is honored during `--delay-updates` publication.**
- **`-x`/`--one-file-system` emits the mount-point directory entry** (an empty
directory at the destination) without descending into it.
- **`--include`/`--exclude` are an ordered first-match rule list**, evaluated
like `--filter`/`-F`/`-C` (first match wins), so an earlier rule can override a
later one.
### Ownership and metadata
- **`--numeric-ids` is a mapping modifier only** — it changes *how* ids map, not
*whether* ownership is applied; combine it with `-o`/`-g`, `-a`, or an
explicit map.
- **`--usermap`/`--groupmap`** support names, `@N`/bare `N` ids, inclusive
`LOW-HIGH` ranges, `*`, empty-`FROM` (unnamed ids), and receiver-resolved `TO`
names.
- **`--chown` conflicts with `--usermap`/`--groupmap` on the same side** and is a
clear configuration error (matching rsync) instead of an order-dependent
winner.
- **`--fake-super` never real-chowns.** It records the *resolved* owner (the
active mapping, else the source id) in `user.fastsync.stat` for a later
privileged restore and replays only mode/times. Directory ownership and
directory xattrs/ACLs are preserved alongside file entries.
- **`--chmod`** implements rsync's `D`/`F`/`X` selectors, `s`/`t`, append
semantics, does not imply `-p`, and applies its changes without sanitization.
### Symlinks and special files
- **`-l`/`--links` stores symlink targets verbatim** (absolute and `..`-bearing
targets included), matching rsync. `--safe-links`, `--copy-unsafe-links`, and
`--munge-links` (which now uses rsync's `/rsyncd-munged/` marker) match rsync
and are applied sender-side.
- **`--specials` recreates unix sockets** with `mknodat(..., S_IFSOCK)`, so
`-D`/`--devices --specials` now covers the full rsync node set.
- **`--copy-devices`** is implemented (see its caveat below).
### Output
- **`-i`/`--out-format`** print rsync-style change lines; **`--list-only`**
scans the source only and contacts no server; **`-h`** uses rsync's decimal
units; **`--progress`** is an aggregate line; **`--stats`** prints the counters
FastSync can observe locally (receiver-only counters are 0).
- **Server `--port`** is an alias of the `-p <port>` TCP listen port
(`--dparam port=` overrides the daemon config).
### Known intentional divergences and limitations
These remain after the wave; they are the reasons a row above is ⚠️.
- **Symlink target containment is not enforced receiver-side by default.**
Verbatim storage is rsync parity, but a destination later consumed by a
link-following tool can follow a link outside the receive root. Use
`--safe-links` when the source is untrusted. `--trust-sender` does **not**
affect symlink targets.
- **`--temp-dir` absolute/foreign-filesystem paths are rejected by the
receiver** (rsync's daemon also confines; standalone rsync differs).
- **`--copy-devices` reads a bounded `st_size`** rather than rsync's unbounded
device read.
- **A broken symlink referent under `--copy-links`/`--copy-unsafe-links` exits 0**
where rsync exits 23.
- **New directories without `-p` still use FastSync's `0755` creation default**
rather than `source & ~umask`; directory metadata is only applied when a
directory attribute is requested.
- **`--stats` receiver-only counters** (matched data, file-list bytes, deleted
count) are reported as 0; `--progress` is an aggregate line, not per-file.
- **`--password-file`/`--early-input`/`--hash-credentials`/`--iterations` are
FastSync-native** (SCRAM/PBKDF2), not rsync semantics; the batch format is not
rsync-interoperable.
- **xattr/ACL namespace policy** permits only `user.*` and
`system.posix_acl_*` when `-A` is negotiated (stricter than rsync).
- **`--stop-at` remains a FastSync-flexible parser** (client-only, not
serialized); `--stop-after` matches rsync.
- **Push-only model and a non-rsync wire protocol** remain by design;
`--protocol` accepts only the current version and `-s`/`--secluded-args` is an
accepted no-op.
## Packed Metadata Frame (protocol 2.20.0)
+348 -155
View File
@@ -7,21 +7,7 @@
#include <string.h>
#include <sys/stat.h>
#include <time.h>
/* Itemize code emitted for a transferred regular file.
*
* Layout (rsync-compatible 11-char item): `>f` marks a regular file that was
* transferred to the remote host; the trailing nine markers are, in order,
* c(hecksum) s(ize) t(ime) p(erms) o(wner) g(roup) u(ser/acl) a(ttrs) x(attrs).
* Every marker is `+` (FastSync does not compare each attribute on the
* receiving side, so a sent file is reported as fully updated). Files that
* are already up to date print no line at all, matching rsync's single -i
* which only itemizes changes.
*
* Because the scanner only yields regular-file transfer candidates, `>d`
* (directory) lines are never produced; directories are not transferred as
* items by FastSync. */
#define ITEMIZE_SENT_FILE ">f+++++++++"
#include <unistd.h>
typedef struct {
char* data;
@@ -80,103 +66,14 @@ static bool strbuf_append(StrBuf* buf, const char* text) {
return true;
}
static bool strbuf_append_ull(StrBuf* buf, unsigned long long value) {
char digits[32];
int written = snprintf(digits, sizeof(digits), "%llu", value);
if (written < 0 || (size_t)written >= sizeof(digits))
return false;
return strbuf_append(buf, digits);
}
static bool strbuf_append_longlong(StrBuf* buf, long long value) {
char digits[32];
int written = snprintf(digits, sizeof(digits), "%lld", value);
if (written < 0 || (size_t)written >= sizeof(digits))
return false;
return strbuf_append(buf, digits);
}
bool change_list_enabled(const Config* config) {
return config != NULL && (config->itemize_changes || config->out_format != NULL ||
(config->log_file != NULL && config->log_file_format != NULL));
}
char* change_render_itemize(const ChangeEvent* event) {
if (event == NULL || event->decision != CHANGE_SENT)
return str_dup("");
const char* code = event->is_directory ? ">d+++++++++" : ITEMIZE_SENT_FILE;
StrBuf line = {0};
bool ok = strbuf_append(&line, code) && strbuf_append(&line, " ") &&
strbuf_append(&line, event->path != NULL ? event->path : "");
if (!ok) {
strbuf_free(&line);
return NULL;
}
return line.data;
}
/* ---- Itemize code ---- */
static const char* leaf_name(const char* path) {
if (path == NULL)
return "";
const char* slash = strrchr(path, '/');
return slash != NULL && slash[1] != '\0' ? slash + 1 : path;
}
char* change_render_format(const char* format, const ChangeEvent* event) {
if (format == NULL)
return NULL;
StrBuf line = {0};
bool ok = true;
for (const char* p = format; *p != '\0' && ok;) {
if (*p != '%') {
ok = strbuf_append_char(&line, *p);
p++;
continue;
}
char token = p[1];
if (token == '\0') {
ok = strbuf_append_char(&line, '%');
break;
}
switch (token) {
case '%':
ok = strbuf_append_char(&line, '%');
break;
case 'f':
ok = strbuf_append(&line, event->path != NULL ? event->path : "");
break;
case 'n':
ok = strbuf_append(&line, leaf_name(event->path));
break;
case 'l':
ok = strbuf_append_ull(&line, event->size);
break;
case 'b':
ok = strbuf_append_ull(&line, event->bytes_sent);
break;
case 'M':
ok = strbuf_append_longlong(&line, (long long)event->mtime_sec);
break;
default:
/* Unknown escape sequences are preserved verbatim. */
ok = strbuf_append_char(&line, '%') && strbuf_append_char(&line, token);
break;
}
p += 2;
}
if (!ok) {
strbuf_free(&line);
return NULL;
}
if (line.data == NULL) {
line.data = str_dup("");
if (!line.data)
return NULL;
}
return line.data;
}
/* Format a mode as an `ls -l` permission string, e.g. `-rw-r--r--`. */
/* Format the permission bits as an `ls -l` string, e.g. `-rw-r--r--`. */
static void mode_to_ls_string(mode_t mode, char out[11]) {
out[0] = S_ISDIR(mode) ? 'd'
: S_ISLNK(mode) ? 'l'
@@ -198,29 +95,102 @@ static void mode_to_ls_string(mode_t mode, char out[11]) {
out[10] = '\0';
}
char* change_render_list_line(mode_t mode, unsigned long long size, time_t mtime,
const char* path) {
char permission[11];
mode_to_ls_string(mode, permission);
char date[32];
struct tm broken_down;
if (localtime_r(&mtime, &broken_down) != NULL) {
if (strftime(date, sizeof(date), "%Y/%m/%d %H:%M:%S", &broken_down) == 0)
snprintf(date, sizeof(date), "?");
} else {
snprintf(date, sizeof(date), "?");
static char itemize_type_char(const ChangeEvent* event) {
if (event->is_directory)
return 'd';
if (event->is_symlink)
return 'L';
if (event->is_special) {
if (S_ISCHR(event->mode) || S_ISBLK(event->mode))
return 'D';
return 'S';
}
return 'f';
}
static bool times_match(const Config* config, const ChangeEvent* event) {
if (!event->dest.known || !event->dest.existed)
return false;
if (event->mtime_sec == event->dest.mtime_sec)
return event->mtime_nsec == event->dest.mtime_nsec;
long long delta = (long long)event->mtime_sec - (long long)event->dest.mtime_sec;
if (delta < 0)
delta = -delta;
return delta <= (long long)config->modify_window;
}
/* Fill the 11-character itemize code (10 chars + NUL). `created` means the
* destination entry did not exist, so every attribute marker is `+`. */
static void itemize_code(const Config* config, const ChangeEvent* event, char code[12]) {
bool known = event->dest.known;
bool created = !known || !event->dest.existed;
char update;
if (event->is_hardlink)
update = 'h';
else if (created)
update = (event->is_directory || event->is_symlink || event->is_special) ? 'c' : '>';
else
update = '>';
code[0] = update;
code[1] = itemize_type_char(event);
if (created) {
for (int i = 0; i < 9; i++)
code[2 + i] = '+';
code[11] = '\0';
return;
}
bool size_diff = event->size != event->dest.size;
bool time_diff = !times_match(config, event);
bool perms_diff = (event->mode & 07777) != (event->dest.mode & 07777);
bool owner_diff = event->uid != (uid_t)event->dest.uid;
bool group_diff = event->gid != (gid_t)event->dest.gid;
code[2] = '.'; /* checksum: no destination digest available */
code[3] = size_diff ? 's' : '.';
code[4] = time_diff ? 't' : '.';
code[5] = (config->preserve_perms && perms_diff) ? 'p' : '.';
code[6] = (config->preserve_owner && owner_diff) ? 'o' : '.';
code[7] = (config->preserve_group && group_diff) ? 'g' : '.';
code[8] = '.'; /* reserved */
code[9] = '.'; /* acl: not compared */
code[10] = '.';
code[11] = '\0';
}
char* change_render_itemize_code(const Config* config, const ChangeEvent* event) {
if (event == NULL || event->decision != CHANGE_SENT)
return str_dup("");
char code[12];
itemize_code(config, event, code);
return str_dup(code);
}
/* rsync %n: the transfer-relative name, with a trailing slash for directories. */
static bool append_name(StrBuf* buf, const ChangeEvent* event) {
if (!strbuf_append(buf, event->name != NULL ? event->name : ""))
return false;
if (event->is_directory && (event->name == NULL || event->name[0] == '\0' ||
event->name[strlen(event->name) - 1] != '/'))
return strbuf_append_char(buf, '/');
return true;
}
/* rsync %L: " -> target" for a symlink, " => target" for a hard link, else "". */
static bool append_link_suffix(StrBuf* buf, const ChangeEvent* event) {
if (event->is_symlink && event->symlink_target != NULL)
return strbuf_append(buf, " -> ") && strbuf_append(buf, event->symlink_target);
if (event->is_hardlink && event->hardlink_target != NULL)
return strbuf_append(buf, " => ") && strbuf_append(buf, event->hardlink_target);
return true;
}
char* change_render_itemize(const Config* config, const ChangeEvent* event) {
if (event == NULL || event->decision != CHANGE_SENT)
return str_dup("");
char code[12];
itemize_code(config, event, code);
StrBuf line = {0};
char size_field[32];
int written = snprintf(size_field, sizeof(size_field), "%llu", size);
if (written < 0 || (size_t)written >= sizeof(size_field)) {
strbuf_free(&line);
return NULL;
}
bool ok = strbuf_append(&line, permission) && strbuf_append_char(&line, ' ') &&
strbuf_append(&line, size_field) && strbuf_append_char(&line, ' ') &&
strbuf_append(&line, date) && strbuf_append_char(&line, ' ') &&
strbuf_append(&line, path != NULL ? path : "");
bool ok = strbuf_append(&line, code) && strbuf_append_char(&line, ' ') &&
append_name(&line, event) && append_link_suffix(&line, event);
if (!ok) {
strbuf_free(&line);
return NULL;
@@ -228,6 +198,141 @@ char* change_render_list_line(mode_t mode, unsigned long long size, time_t mtime
return line.data;
}
/* ---- --out-format / --log-file-format ---- */
char* change_render_format(const char* format, const Config* config, const ChangeEvent* event) {
if (format == NULL || event == NULL)
return NULL;
StrBuf line = {0};
bool ok = true;
for (const char* p = format; *p != '\0' && ok;) {
if (*p != '%') {
ok = strbuf_append_char(&line, *p);
p++;
continue;
}
char token = p[1];
if (token == '\0') {
ok = strbuf_append_char(&line, '%');
break;
}
switch (token) {
case '%':
ok = strbuf_append_char(&line, '%');
break;
case 'i': {
char code[12];
itemize_code(config, event, code);
ok = strbuf_append(&line, code);
break;
}
case 'f':
ok = strbuf_append(&line, event->path != NULL ? event->path : "");
break;
case 'n':
ok = append_name(&line, event);
break;
case 'L':
ok = append_link_suffix(&line, event);
break;
case 'l': {
char digits[32];
int written = snprintf(digits, sizeof(digits), "%llu", event->size);
ok = written >= 0 && (size_t)written < sizeof(digits) && strbuf_append(&line, digits);
} break;
case 'b': {
char digits[32];
int written = snprintf(digits, sizeof(digits), "%llu", event->bytes_sent);
ok = written >= 0 && (size_t)written < sizeof(digits) && strbuf_append(&line, digits);
} break;
case 'M': {
char when[32];
if (format_rsync_datetime(event->mtime_sec, true, when, sizeof(when)))
ok = strbuf_append(&line, when);
} break;
case 't': {
char when[32];
if (format_rsync_datetime(time(NULL), false, when, sizeof(when)))
ok = strbuf_append(&line, when);
} break;
case 'o':
ok = strbuf_append(&line, "send");
break;
case 'p': {
char digits[32];
int written = snprintf(digits, sizeof(digits), "%ld", (long)getpid());
ok = written >= 0 && (size_t)written < sizeof(digits) && strbuf_append(&line, digits);
} break;
case 'B': {
char permission[11];
mode_to_ls_string(event->mode, permission);
ok = strbuf_append(&line, permission + 1);
} break;
case 'U': {
char digits[32];
int written = snprintf(digits, sizeof(digits), "%u", (unsigned)event->uid);
ok = written >= 0 && (size_t)written < sizeof(digits) && strbuf_append(&line, digits);
} break;
case 'G': {
char digits[32];
int written = snprintf(digits, sizeof(digits), "%u", (unsigned)event->gid);
ok = written >= 0 && (size_t)written < sizeof(digits) && strbuf_append(&line, digits);
} break;
default:
/* Unknown escape sequences are preserved verbatim. */
ok = strbuf_append_char(&line, '%') && strbuf_append_char(&line, token);
break;
}
p += 2;
}
if (!ok) {
strbuf_free(&line);
return NULL;
}
if (line.data == NULL) {
line.data = str_dup("");
if (!line.data)
return NULL;
}
return line.data;
}
/* ---- --list-only ---- */
char* change_render_list_line(const Config* config, const ChangeEvent* event) {
(void)config;
if (event == NULL)
return NULL;
char permission[11];
mode_to_ls_string(event->mode, permission);
char date[32];
if (!format_rsync_datetime(event->mtime_sec, false, date, sizeof(date)))
snprintf(date, sizeof(date), "?");
StrBuf line = {0};
char size_field[40];
char grouped[32];
if (!format_big_num(event->size, false, grouped, sizeof(grouped))) {
strbuf_free(&line);
return NULL;
}
int written = snprintf(size_field, sizeof(size_field), "%15s", grouped);
if (written < 0 || (size_t)written >= sizeof(size_field)) {
strbuf_free(&line);
return NULL;
}
const char* name = event->name != NULL && event->name[0] != '\0' ? event->name : ".";
bool ok = strbuf_append(&line, permission) && strbuf_append(&line, size_field) &&
strbuf_append_char(&line, ' ') && strbuf_append(&line, date) &&
strbuf_append_char(&line, ' ') && strbuf_append(&line, name);
if (!ok) {
strbuf_free(&line);
return NULL;
}
return line.data;
}
/* ---- Event emission ---- */
static void print_escaped_line(FILE* stream, const char* line, bool eight_bit_output) {
char* escaped = output_escape(line, eight_bit_output);
if (escaped != NULL) {
@@ -247,15 +352,16 @@ void change_emit(const Config* config, const ChangeEvent* event) {
bool to_stdout = config->itemize_changes || config->out_format != NULL;
bool to_log = config->log_file != NULL && config->log_file_format != NULL;
if (to_stdout) {
char* line = config->out_format != NULL ? change_render_format(config->out_format, event)
: change_render_itemize(event);
char* line = config->out_format != NULL
? change_render_format(config->out_format, config, event)
: change_render_itemize(config, event);
if (line != NULL) {
print_escaped_line(stdout, line, config->eight_bit_output);
free(line);
}
}
if (to_log) {
char* line = change_render_format(config->log_file_format, event);
char* line = change_render_format(config->log_file_format, config, event);
if (line != NULL) {
print_escaped_line(config->log_file, line, config->eight_bit_output);
free(line);
@@ -266,9 +372,6 @@ void change_emit(const Config* config, const ChangeEvent* event) {
static bool format_uses_mtime(const char* format) {
if (format == NULL)
return false;
/* Mirror change_render_format's tokenizer: "%%" is a literal percent (so
* "%%M" does NOT expand %M) and unknown "%X" escapes consume both chars.
* This keeps the optional stat() fallback below in step with the renderer. */
for (const char* p = format; *p != '\0';) {
if (*p != '%') {
p++;
@@ -284,46 +387,136 @@ static bool format_uses_mtime(const char* format) {
return false;
}
/* Relative path of an entry below the transfer root (no leading slash). Uses
* the sender-side send_path override when present (bare-relative -R layout). */
static char* relative_name(const Config* config, const File* file) {
const char* full = file_wire_path(file);
if (file->send_path != NULL)
return str_dup(full != NULL ? full : "");
const char* root = config->send_directory;
if (root == NULL || full == NULL)
return str_dup(full != NULL ? full : "");
size_t root_len = strlen(root);
while (root_len > 1 && root[root_len - 1] == '/')
root_len--;
if (strncmp(root, full, root_len) == 0) {
if (full[root_len] == '\0')
return str_dup("");
if (full[root_len] == '/')
return str_dup(full + root_len + 1);
}
return str_dup(full);
}
/* rsync %f long form: the source argument as typed (leading '/' removed,
* trailing '/' removed, leading "./" removed) joined to the relative name. */
static char* display_name(const Config* config, const char* name) {
const char* root = config->send_directory;
if (root == NULL)
return str_dup(name != NULL ? name : "");
const char* p = root;
while (*p == '/')
p++;
if (p[0] == '.' && p[1] == '/')
p += 2;
size_t root_len = strlen(p);
while (root_len > 0 && p[root_len - 1] == '/')
root_len--;
size_t name_len = name != NULL ? strlen(name) : 0;
if (root_len == 0 && name_len == 0)
return str_dup("");
char* out = malloc(root_len + (root_len > 0 && name_len > 0 ? 1 : 0) + name_len + 1);
if (!out)
return NULL;
size_t offset = 0;
if (root_len > 0) {
memcpy(out, p, root_len);
offset = root_len;
}
if (root_len > 0 && name_len > 0)
out[offset++] = '/';
if (name_len > 0)
memcpy(out + offset, name, name_len);
out[offset + name_len] = '\0';
return out;
}
static void fill_event_from_file(const Config* config, const File* file, ChangeEvent* event,
char** name_out, char** path_out) {
char* name = relative_name(config, file);
char* path = display_name(config, name);
event->name = name;
event->path = path;
*name_out = name;
*path_out = path;
if (file->metadata != NULL) {
event->mtime_sec = file->metadata->mtime_sec;
event->mtime_nsec = file->metadata->mtime_nsec;
event->mode = file->metadata->mode;
event->uid = file->metadata->uid;
event->gid = file->metadata->gid;
} else if (format_uses_mtime(config->out_format) || format_uses_mtime(config->log_file_format)) {
struct stat st;
if (file->path != NULL && stat(file->path, &st) == 0) {
event->mtime_sec = st.st_mtime;
event->mtime_nsec = st.st_mtim.tv_nsec;
}
}
}
void change_emit_file_sent(const Config* config, const File* file) {
if (file == NULL || !change_list_enabled(config))
return;
ChangeEvent event;
memset(&event, 0, sizeof(event));
/* The displayed path is the one transmitted (with -R + --files-from this is
the bare relative destination path); the metadata fallback below still
stats the local absolute path. */
event.path = file_wire_path(file);
event.decision = CHANGE_SENT;
event.is_directory = false;
event.is_symlink = false;
event.is_special = false;
event.is_hardlink = false;
event.size = file->data != NULL ? file->data->size : 0;
/* FastSync has no wire-byte counter yet, so %b reports the source length
* that had to be delivered (always equal to %l); the actual bytes written
* to the socket (compressed/delta) are not measured. */
event.bytes_sent = event.size;
if (file->metadata != NULL) {
event.mtime_sec = file->metadata->mtime_sec;
} else if (format_uses_mtime(config->out_format) || format_uses_mtime(config->log_file_format)) {
/* Best-effort fallback for %M when no metadata was captured (no -M): the
* path is stat()ed just to fill the field, and any failure leaves 0. */
struct stat st;
if (file->path != NULL && stat(file->path, &st) == 0)
event.mtime_sec = st.st_mtime;
event.dest = file->dest_state;
if (file->is_symlink) {
event.is_symlink = true;
event.symlink_target = file->symlink_target;
event.size = file->symlink_target != NULL ? strlen(file->symlink_target) : 0;
event.bytes_sent = 0;
} else if (file->is_special) {
event.is_special = true;
event.bytes_sent = 0;
} else if (file->link_group != 0 && !file->link_first) {
event.is_hardlink = true;
event.hardlink_target = file->hardlink_target;
event.bytes_sent = 0;
} else {
/* Literal payload bytes delivered; compressed/delta wire bytes are not
* separately counted. */
event.bytes_sent = event.size;
}
change_emit(config, &event);
char* name = NULL;
char* path = NULL;
fill_event_from_file(config, file, &event, &name, &path);
if (name != NULL && path != NULL)
change_emit(config, &event);
free(name);
free(path);
}
/* Build and emit a CHANGE_SENT event for an explicit directory entry (-d). */
void change_emit_dir_sent(const Config* config, const File* file) {
if (file == NULL || !change_list_enabled(config))
return;
ChangeEvent event;
memset(&event, 0, sizeof(event));
event.path = file_wire_path(file);
event.decision = CHANGE_SENT;
event.is_directory = true;
event.size = 0;
event.bytes_sent = 0;
if (file->metadata != NULL)
event.mtime_sec = file->metadata->mtime_sec;
change_emit(config, &event);
event.dest = file->dest_state;
char* name = NULL;
char* path = NULL;
fill_event_from_file(config, file, &event, &name, &path);
if (name != NULL && path != NULL)
change_emit(config, &event);
free(name);
free(path);
}
+35 -24
View File
@@ -3,6 +3,7 @@
#include "config.h"
#include "file_types.h"
#include "format.h"
#include <stdbool.h>
#include <sys/stat.h>
#include <time.h>
@@ -26,42 +27,52 @@ typedef enum {
} ChangeDecision;
typedef struct {
const char* path; /* full source path */
const char* path; /* long-form display path (rsync %f) */
const char* name; /* transfer-relative path (rsync %n), no trailing slash */
ChangeDecision decision;
bool is_directory;
unsigned long long size; /* source file length in bytes */
/* The number of bytes reported for a sent file. FastSync has no wire-byte
* counter, so this is always the source length (== size / %l); actual
* post-compression/delta bytes on the wire are not counted. */
unsigned long long bytes_sent;
time_t mtime_sec; /* 0 when unknown */
bool is_symlink;
bool is_special;
bool is_hardlink; /* a hard-link sibling (linked, no data sent) */
const char* symlink_target;
const char* hardlink_target;
unsigned long long size; /* source file length in bytes */
unsigned long long bytes_sent; /* literal data bytes actually transferred */
time_t mtime_sec;
long mtime_nsec;
mode_t mode;
uid_t uid;
gid_t gid;
/* Receiver-reported pre-transfer destination state (OutputDestState.known is
* false when no report was requested/received). */
OutputDestState dest;
} ChangeEvent;
/* True when any output mode is active and per-file events matter. */
bool change_list_enabled(const Config* config);
/* Render the rsync-style itemize line for a transferred file:
* `>f+++++++++ <path>`
* The 11-char code is `>f` (regular file transferred to the remote host)
* followed by c/s/t/p/o/g/u/a/x markers that are all `+` (value will be set
* / differs) because FastSync does not separately compare checksums, size,
* mtime, perms, owner, group, uid, acl, or xattr on the receiving side, so a
* sent file is reported as fully updated. Up-to-date files print no line
* (rsync single `-i` only shows changes). Caller frees the result. */
char* change_render_itemize(const ChangeEvent* event);
/* Render the rsync-style itemize line for a transferred item
* (`%i %n%L`): `>f+++++++++ sub/b.txt`. Caller frees the result. */
char* change_render_itemize(const Config* config, const ChangeEvent* event);
/* Expand an --out-format/--log-file-format template. Tokens:
* %f full source path %b "bytes sent" == the source length (%l);
* %n leaf (base) name actual post-compression/delta wire bytes
* %l file length in bytes are not counted
* %M mtime in whole seconds %% a literal percent sign
/* Render only the 11-character itemize code (rsync %i). Caller frees. */
char* change_render_itemize_code(const Config* config, const ChangeEvent* event);
/* Expand an --out-format/--log-file-format template. Supported tokens:
* %i itemize code %n transfer-relative name (dir: trailing /)
* %f long display path %l file length in bytes
* %b bytes actually sent %M mtime (YYYY/MM/DD-HH:MM:SS)
* %t current time %o operation ("send"/"del.")
* %p pid %B permission bits without the type char
* %U uid %G gid
* %L " -> target" / " => target" %% a literal percent sign
* Unknown %X sequences are preserved verbatim. Caller frees the result. */
char* change_render_format(const char* format, const ChangeEvent* event);
char* change_render_format(const char* format, const Config* config, const ChangeEvent* event);
/* Render one --list-only long-listing entry:
* `-rw-r--r-- 12 2026/09/06 10:00:00 <path>`
* `-rw-r--r-- 12 2026/09/06 10:00:00 sub/b.txt`
* (ls -l style columns; mtime in the local time zone). Caller frees it. */
char* change_render_list_line(mode_t mode, unsigned long long size, time_t mtime, const char* path);
char* change_render_list_line(const Config* config, const ChangeEvent* event);
/* Emit an event to every active destination:
* stdout: --itemize-changes line, or the --out-format expansion when set;
+188 -26
View File
@@ -117,6 +117,27 @@ static int set_string_option(char** dest, const char* value, const char* option_
return 0;
}
/* Append a --chmod clause list to the accumulated spec with a comma. rsync
* 3.2.4+ makes repeated --chmod options cumulative, so they must not replace
* the previous ones. Returns 0 on success, -1 on failure. */
static int append_chmod_spec(char** dest, const char* value) {
if (!*dest)
return set_string_option(dest, value, "--chmod");
size_t old_len = strlen(*dest);
size_t add_len = strlen(value);
char* merged = malloc(old_len + add_len + 2);
if (!merged) {
log_message(LOG_LEVEL_ERROR, "memory allocation failed for --chmod");
return -1;
}
memcpy(merged, *dest, old_len);
merged[old_len] = ',';
memcpy(merged + old_len + 1, value, add_len + 1);
free(*dest);
*dest = merged;
return 0;
}
/* Parse a string as a positive integer into *dest. Returns 0 on success, -1 on error. */
static int set_positive_int_option(int* dest, const char* value, const char* option_name) {
if (!parse_positive_int(value, dest)) {
@@ -132,16 +153,20 @@ static int set_positive_int_option(int* dest, const char* value, const char* opt
* zstd choice; any other rsync choice is rejected by name instead of being
* silently accepted and ignored. */
static int set_compression_choice(Config* config, const char* value) {
if (strcmp(value, "zstd") != 0 && strcmp(value, "none") != 0 && strcmp(value, "auto") != 0) {
/* rsync's "auto" is normalized to the canonical "zstd" at parse time (like
--checksum-choice=auto), so the value that crosses the wire is always one
the receiver accepts. */
const char* canonical = strcmp(value, "auto") == 0 ? "zstd" : value;
if (strcmp(canonical, "zstd") != 0 && strcmp(canonical, "none") != 0) {
log_message(LOG_LEVEL_ERROR,
"--compress-choice '%s' is not implemented; FastSync supports zstd, none or auto "
"(rsync's lz4/zlib/zlibx are rejected, never silently ignored)",
value);
return -1;
}
if (set_string_option(&config->compress_choice, value, "--compress-choice") != 0)
if (set_string_option(&config->compress_choice, canonical, "--compress-choice") != 0)
return -1;
config->use_compression = strcmp(value, "none") != 0;
config->use_compression = strcmp(canonical, "none") != 0;
return 0;
}
@@ -248,6 +273,25 @@ static int set_nonneg_int_option(int* dest, const char* value, const char* optio
return 0;
}
/* Parse a signed integer, clamping every negative value to -1. rsync's
--max-delete treats a negative argument (the deprecated -1 spelling) as "no
client limit", so -2/-5 must behave identically rather than being rejected. */
static int set_signed_clamped_int_option(int* dest, const char* value, const char* option_name) {
if (!value || *value == '\0') {
log_message(LOG_LEVEL_ERROR, "%s must be an integer", option_name);
return -1;
}
char* endptr;
errno = 0;
long parsed = strtol(value, &endptr, 10);
if (errno != 0 || *endptr != '\0' || parsed < INT_MIN || parsed > INT_MAX) {
log_message(LOG_LEVEL_ERROR, "%s must be an integer", option_name);
return -1;
}
*dest = parsed < 0 ? -1 : (int)parsed;
return 0;
}
/* Forward decl: config_add_pattern is defined below, but the --remote-option
* helper above needs it. */
static int config_add_pattern(char*** patterns, int* count, const char* value, const char* optname);
@@ -348,7 +392,8 @@ static void apply_output_buffering(const Config* config) {
}
#endif
static int read_patterns_from_file(const char* filepath, char*** patterns, int* count);
static int read_patterns_from_file(const char* filepath, char*** patterns, int* count,
Config* config, char sign, const char* optname);
static int parse_debug_flags(const char* value, Config* config) {
if (!value || value[0] == '\0' || value[0] == ',' || value[strlen(value) - 1] == ',' ||
@@ -578,6 +623,27 @@ static int config_add_filter(Config* config, const char* rule) {
return 0;
}
/* Compile one --exclude/--include pattern into the SAME ordered filter rule
* list used by --filter/-f: `--exclude P` becomes the rule "- P" and
* `--include P` becomes "+ P", appended in command-line order. This is what
* makes rsync's first-match-wins semantics hold across a mixed sequence such as
* `--include='*.txt' --exclude='*'`. Returns 0 on success, -1 on error. */
static int config_add_selection_rule(Config* config, char sign, const char* pattern,
const char* optname) {
size_t len = strlen(pattern);
char* rule = malloc(len + 3);
if (!rule) {
log_message(LOG_LEVEL_ERROR, "memory allocation failed for %s", optname);
return -1;
}
rule[0] = sign;
rule[1] = ' ';
memcpy(rule + 2, pattern, len + 1);
int rc = config_add_filter(config, rule);
free(rule);
return rc;
}
static int parse_skip_compress(Config* config, const char* value) {
char* list = str_dup(value);
if (!list)
@@ -616,6 +682,9 @@ typedef enum {
OPT_POS_INT,
OPT_NONNEG_INT,
OPT_ULL,
/* A signed integer whose negative values are clamped to -1 (rsync's
"no limit" spelling for --max-delete). */
OPT_SIGNED_INT,
} OptKind;
typedef struct {
@@ -730,7 +799,7 @@ static const OptionEntry OPTION_TABLE[] = {
{"--delete-delay", NULL, OPT_FLAG, offsetof(Config, delete_delay)},
{"--delete-after", NULL, OPT_FLAG, offsetof(Config, delete_after)},
{"--delete-excluded", NULL, OPT_FLAG, offsetof(Config, delete_excluded)},
{"--max-delete", NULL, OPT_NONNEG_INT, offsetof(Config, max_delete)},
{"--max-delete", NULL, OPT_SIGNED_INT, offsetof(Config, max_delete)},
{"--ignore-errors", NULL, OPT_FLAG, offsetof(Config, ignore_errors)},
{"--force", NULL, OPT_FLAG, offsetof(Config, force_delete)},
{"--prune-empty-dirs", "-m", OPT_FLAG, offsetof(Config, prune_empty_dirs)},
@@ -854,7 +923,8 @@ static const OptionEntry* find_table_option_with_equals(const char* arg, const c
(entry->alias && strlen(entry->alias) == name_len &&
strncmp(arg, entry->alias, name_len) == 0)) {
if (entry->kind == OPT_STRING || entry->kind == OPT_POS_INT ||
entry->kind == OPT_NONNEG_INT || entry->kind == OPT_ULL) {
entry->kind == OPT_NONNEG_INT || entry->kind == OPT_ULL ||
entry->kind == OPT_SIGNED_INT) {
*value = equals + 1;
return entry;
}
@@ -917,11 +987,15 @@ static int apply_table_option(Config* config, const OptionEntry* entry, const ch
case OPT_NOOP:
return 0;
case OPT_STRING:
if (entry->offset == offsetof(Config, chmod_spec))
return append_chmod_spec((char**)field, value);
return set_string_option((char**)field, value, entry->name);
case OPT_POS_INT:
return set_positive_int_option((int*)field, value, entry->name);
case OPT_NONNEG_INT:
return set_nonneg_int_option((int*)field, value, entry->name);
case OPT_SIGNED_INT:
return set_signed_clamped_int_option((int*)field, value, entry->name);
case OPT_ULL: {
unsigned long long v;
/* Size-limit options accept rsync-style suffixes (e.g. --max-size=2G); a
@@ -1173,7 +1247,6 @@ static bool cli_handle_table_option(CliParseCtx* ctx) {
ctx->exit_code = -1;
return true;
}
config->preserve_perms = true;
}
/* Remember that --server-host was explicitly given (the field itself
defaults to 127.0.0.1, so a value check cannot distinguish it). Used
@@ -1220,7 +1293,7 @@ static bool cli_handle_inline_chmod(CliParseCtx* ctx) {
const char* arg = ctx->argv[ctx->i];
if (strncmp(arg, "--chmod=", 8) != 0)
return false;
if (set_string_option(&config->chmod_spec, arg + 8, "--chmod") != 0) {
if (append_chmod_spec(&config->chmod_spec, arg + 8) != 0) {
ctx->exit_code = -1;
return true;
}
@@ -1230,7 +1303,6 @@ static bool cli_handle_inline_chmod(CliParseCtx* ctx) {
ctx->exit_code = -1;
return true;
}
config->preserve_perms = true;
return true;
}
@@ -1353,7 +1425,8 @@ static bool cli_handle_ssh_and_pattern_options(CliParseCtx* ctx) {
}
if (strncmp(arg, "--exclude=", 10) == 0) {
if (config_add_pattern(&config->exclude_patterns, &config->exclude_count, arg + 10,
"--exclude") != 0)
"--exclude") != 0 ||
config_add_selection_rule(config, '-', arg + 10, "--exclude") != 0)
ctx->exit_code = -1;
return true;
}
@@ -1364,13 +1437,15 @@ static bool cli_handle_ssh_and_pattern_options(CliParseCtx* ctx) {
return true;
}
if (config_add_pattern(&config->exclude_patterns, &config->exclude_count, ctx->argv[++ctx->i],
"--exclude") != 0)
"--exclude") != 0 ||
config_add_selection_rule(config, '-', ctx->argv[ctx->i], "--exclude") != 0)
ctx->exit_code = -1;
return true;
}
if (strncmp(arg, "--include=", 10) == 0) {
if (config_add_pattern(&config->include_patterns, &config->include_count, arg + 10,
"--include") != 0)
"--include") != 0 ||
config_add_selection_rule(config, '+', arg + 10, "--include") != 0)
ctx->exit_code = -1;
return true;
}
@@ -1381,7 +1456,8 @@ static bool cli_handle_ssh_and_pattern_options(CliParseCtx* ctx) {
return true;
}
if (config_add_pattern(&config->include_patterns, &config->include_count, ctx->argv[++ctx->i],
"--include") != 0)
"--include") != 0 ||
config_add_selection_rule(config, '+', ctx->argv[ctx->i], "--include") != 0)
ctx->exit_code = -1;
return true;
}
@@ -1664,7 +1740,8 @@ static bool cli_handle_filter_options(CliParseCtx* ctx) {
Config* config = ctx->config;
const char* arg = ctx->argv[ctx->i];
if (strncmp(arg, "--exclude-from=", 15) == 0) {
if (read_patterns_from_file(arg + 15, &config->exclude_patterns, &config->exclude_count) != 0)
if (read_patterns_from_file(arg + 15, &config->exclude_patterns, &config->exclude_count, config,
'-', "--exclude-from") != 0)
ctx->exit_code = -1;
return true;
}
@@ -1675,12 +1752,13 @@ static bool cli_handle_filter_options(CliParseCtx* ctx) {
return true;
}
if (read_patterns_from_file(ctx->argv[++ctx->i], &config->exclude_patterns,
&config->exclude_count) != 0)
&config->exclude_count, config, '-', "--exclude-from") != 0)
ctx->exit_code = -1;
return true;
}
if (strncmp(arg, "--include-from=", 15) == 0) {
if (read_patterns_from_file(arg + 15, &config->include_patterns, &config->include_count) != 0)
if (read_patterns_from_file(arg + 15, &config->include_patterns, &config->include_count, config,
'+', "--include-from") != 0)
ctx->exit_code = -1;
return true;
}
@@ -1691,7 +1769,7 @@ static bool cli_handle_filter_options(CliParseCtx* ctx) {
return true;
}
if (read_patterns_from_file(ctx->argv[++ctx->i], &config->include_patterns,
&config->include_count) != 0)
&config->include_count, config, '+', "--include-from") != 0)
ctx->exit_code = -1;
return true;
}
@@ -1860,6 +1938,75 @@ static bool cli_handle_checksum_options(CliParseCtx* ctx) {
return false;
}
/* Determine whether a --chown spec sets the owner and/or group side, honoring
* the same escape-aware splitting as identity_parse_chown(): a `\:` is a literal
* colon, not a field separator. */
static void chown_spec_sides(const char* value, bool* has_owner, bool* has_group) {
*has_owner = false;
*has_group = false;
if (!value)
return;
bool split = false;
for (const char* p = value; *p; p++) {
if (*p == '\\' && p[1] == ':') {
p++;
continue;
}
if (*p == ':') {
split = true;
continue;
}
if (split)
*has_group = true;
else
*has_owner = true;
}
}
/* rsync refuses to mix --chown with --usermap/--groupmap on the SAME side
* ("--usermap conflicts with prior --chown"). `chown_value` is non-NULL only
* for the --chown option itself. Returns true and records a parse error when
* the new option conflicts with one already seen. */
static bool mapping_option_conflicts(CliParseCtx* ctx, const char* optname, bool is_group,
const char* chown_value) {
const Config* config = ctx->config;
if (chown_value) {
bool has_owner;
bool has_group;
chown_spec_sides(chown_value, &has_owner, &has_group);
if (has_owner && config->usermap_count > 0) {
log_message(LOG_LEVEL_ERROR, "%s conflicts with prior --usermap", optname);
ctx->exit_code = -1;
return true;
}
if (has_group && config->groupmap_count > 0) {
log_message(LOG_LEVEL_ERROR, "%s conflicts with prior --groupmap", optname);
ctx->exit_code = -1;
return true;
}
return false;
}
if (!is_group && config->chown_uid_set) {
log_message(LOG_LEVEL_ERROR, "%s conflicts with prior --chown", optname);
ctx->exit_code = -1;
return true;
}
if (is_group && config->chown_gid_set) {
log_message(LOG_LEVEL_ERROR, "%s conflicts with prior --chown", optname);
ctx->exit_code = -1;
return true;
}
return false;
}
static bool usermap_conflicts_with_chown(CliParseCtx* ctx, const char* optname, bool is_group) {
return mapping_option_conflicts(ctx, optname, is_group, NULL);
}
static bool chown_conflicts_with_map(CliParseCtx* ctx, const char* optname, const char* value) {
return mapping_option_conflicts(ctx, optname, false, value);
}
/* Remote-option, basis-directory and identity-mapping options. Returns true
* when the argument was consumed. */
static bool cli_handle_remote_basis_options(CliParseCtx* ctx) {
@@ -1870,11 +2017,6 @@ static bool cli_handle_remote_basis_options(CliParseCtx* ctx) {
ctx->exit_code = -1;
return true;
}
if (strncmp(arg, "-M=", 3) == 0) {
if (config_add_remote_option(config, arg + 3, "-M") != 0)
ctx->exit_code = -1;
return true;
}
if (opt_is(arg, "--remote-option", "-M")) {
if (ctx->i + 1 >= ctx->argc) {
log_message(LOG_LEVEL_ERROR, "missing argument for --remote-option");
@@ -1932,6 +2074,8 @@ static bool cli_handle_remote_basis_options(CliParseCtx* ctx) {
return true;
}
if (strncmp(arg, "--usermap=", 10) == 0) {
if (usermap_conflicts_with_chown(ctx, "--usermap", false))
return true;
if (identity_parse_map(config, arg + 10, false) != 0) {
ctx->exit_code = -1;
return true;
@@ -1945,6 +2089,8 @@ static bool cli_handle_remote_basis_options(CliParseCtx* ctx) {
ctx->exit_code = -1;
return true;
}
if (usermap_conflicts_with_chown(ctx, "--usermap", false))
return true;
if (identity_parse_map(config, ctx->argv[++ctx->i], false) != 0) {
ctx->exit_code = -1;
return true;
@@ -1953,6 +2099,8 @@ static bool cli_handle_remote_basis_options(CliParseCtx* ctx) {
return true;
}
if (strncmp(arg, "--groupmap=", 11) == 0) {
if (usermap_conflicts_with_chown(ctx, "--groupmap", true))
return true;
if (identity_parse_map(config, arg + 11, true) != 0) {
ctx->exit_code = -1;
return true;
@@ -1966,6 +2114,8 @@ static bool cli_handle_remote_basis_options(CliParseCtx* ctx) {
ctx->exit_code = -1;
return true;
}
if (usermap_conflicts_with_chown(ctx, "--groupmap", true))
return true;
if (identity_parse_map(config, ctx->argv[++ctx->i], true) != 0) {
ctx->exit_code = -1;
return true;
@@ -1974,6 +2124,8 @@ static bool cli_handle_remote_basis_options(CliParseCtx* ctx) {
return true;
}
if (strncmp(arg, "--chown=", 8) == 0) {
if (chown_conflicts_with_map(ctx, "--chown", arg + 8))
return true;
if (identity_parse_chown(config, arg + 8) != 0) {
ctx->exit_code = -1;
return true;
@@ -1990,6 +2142,8 @@ static bool cli_handle_remote_basis_options(CliParseCtx* ctx) {
ctx->exit_code = -1;
return true;
}
if (chown_conflicts_with_map(ctx, "--chown", ctx->argv[ctx->i + 1]))
return true;
if (identity_parse_chown(config, ctx->argv[++ctx->i]) != 0) {
ctx->exit_code = -1;
return true;
@@ -2137,6 +2291,12 @@ static int cli_finalize_config(Config* config, bool verbose, bool no_delta, bool
* --no-xattrs/--no-acls negation) so the sender's wire gate always matches
* the flags the receiver will recompute from the received config. */
config->use_xattrs = config->preserve_acls || config->preserve_xattrs;
/* Output parity: -i/--itemize-changes, --out-format and --log-file-format
* need the pre-transfer destination snapshot (new vs modified and which
* attributes differ), so ask the receiver to report it on every per-file
* check. This is a wire field. */
config->report_dest_info = config->itemize_changes || config->out_format != NULL ||
(config->log_file != NULL && config->log_file_format != NULL);
return 0;
}
@@ -2158,7 +2318,7 @@ static bool cli_long_takes_separate_value(const char* arg) {
const OptionEntry* entry = find_table_option(arg);
if (entry)
return entry->kind == OPT_STRING || entry->kind == OPT_POS_INT ||
entry->kind == OPT_NONNEG_INT || entry->kind == OPT_ULL;
entry->kind == OPT_NONNEG_INT || entry->kind == OPT_ULL || entry->kind == OPT_SIGNED_INT;
static const char* const extra[] = {
"--ssh-port", "--exclude", "--include", "--exclude-from",
"--include-from", "--files-from", "--filter", "--delta-block",
@@ -2168,7 +2328,7 @@ static bool cli_long_takes_separate_value(const char* arg) {
"--checksum-choice", "--cc", "--checksum-seed", "--sockopts",
"--remote-option", "--compare-dest", "--copy-dest", "--link-dest",
"--usermap", "--groupmap", "--chown", "--copy-as",
"--outbuf", "--debug", "--info",
"--outbuf", "--debug", "--info", "--skip-compress",
};
for (size_t i = 0; i < sizeof(extra) / sizeof(extra[0]); i++)
if (strcmp(arg, extra[i]) == 0)
@@ -2333,7 +2493,8 @@ done:
return result;
}
static int read_patterns_from_file(const char* filepath, char*** patterns, int* count) {
static int read_patterns_from_file(const char* filepath, char*** patterns, int* count,
Config* config, char sign, const char* optname) {
FILE* fp = fopen(filepath, "r");
if (!fp) {
char* escaped = output_escape(filepath, false);
@@ -2375,7 +2536,8 @@ static int read_patterns_from_file(const char* filepath, char*** patterns, int*
p[--len] = '\0';
if (len == 0)
continue;
if (config_add_pattern(patterns, count, p, "pattern file") != 0) {
if (config_add_pattern(patterns, count, p, "pattern file") != 0 ||
config_add_selection_rule(config, sign, p, optname) != 0) {
free(line);
fclose(fp);
return -1;
+324 -83
View File
@@ -11,6 +11,7 @@
#include "file.h"
#include "file_list.h"
#include "filter.h"
#include "format.h"
#include "hardlink.h"
#include "metadata.h"
#include "motd.h"
@@ -69,32 +70,61 @@ static int progress_thread_fn(void* arg);
static const char* display_bytes(unsigned long long bytes, bool human_readable, char* buffer,
size_t buffer_size) {
if (human_readable && format_human_bytes(bytes, buffer, buffer_size))
if (human_readable && format_human_size_decimal(bytes, buffer, buffer_size))
return buffer;
snprintf(buffer, buffer_size, "%.1f MB", (double)bytes / (double)BYTES_PER_MIB);
return buffer;
}
/* Print the canonical `--stats` line. Shared by the single-threaded and
multithreaded send paths so both honor --stats, --human-readable and --quiet
identically; `start` marks the beginning of the transfer for the rate. */
/* rsync byte count: human-readable decimal when -h was given, otherwise a
* comma-grouped integer (rsync's big_num in the C locale). */
static const char* stats_bytes(const Config* config, unsigned long long bytes, char* buffer,
size_t buffer_size) {
if (!format_big_num(bytes, config->human_readable, buffer, buffer_size))
snprintf(buffer, buffer_size, "%llu", bytes);
return buffer;
}
/* Print the rsync `--stats` block on stdout. FastSync is a push sender, so a
few receiver-only counters (matched data, file-list bytes, deletion count)
are not observable and are reported as 0; the labels and layout match rsync
3.4.1. Shared by the single-threaded and multithreaded send paths. */
static void report_transfer_stats(const Config* config, int total_files,
unsigned long long total_bytes, time_t start) {
if (!config->stats || config->quiet)
return;
double elapsed = difftime(time(NULL), start);
double rate = elapsed > 0.0 ? (double)total_bytes / ((double)BYTES_PER_MIB * elapsed) : 0.0;
double rate = elapsed > 0.0 ? (double)total_bytes / elapsed : 0.0;
char total_buffer[32];
char rate_buffer[32] = {0};
char human_rate[32] = {0};
const char* total = stats_bytes(config, total_bytes, total_buffer, sizeof(total_buffer));
const char* rate_str = rate_buffer;
if (config->human_readable) {
char total_buffer[32];
char rate_buffer[32];
fprintf(stderr, "Stats: %d files, %s, %s/s\n", total_files,
display_bytes(total_bytes, true, total_buffer, sizeof(total_buffer)),
display_bytes((unsigned long long)(rate * (double)BYTES_PER_MIB), true, rate_buffer,
sizeof(rate_buffer)));
if (!format_human_size_decimal((unsigned long long)rate, human_rate, sizeof(human_rate)))
snprintf(human_rate, sizeof(human_rate), "0");
rate_str = human_rate;
} else {
fprintf(stderr, "Stats: %d files, %.1f MB, %.1f MB/s\n", total_files,
(double)total_bytes / (double)BYTES_PER_MIB, rate);
snprintf(rate_buffer, sizeof(rate_buffer), "%.2f", rate);
}
printf("\n");
printf("Number of files: %d\n", total_files);
printf("Number of created files: %d\n", total_files);
printf("Number of deleted files: 0\n");
printf("Number of regular files transferred: %d\n", total_files);
printf("Total file size: %s bytes\n", total);
printf("Total transferred file size: %s bytes\n", total);
printf("Literal data: %s bytes\n", total);
printf("Matched data: 0 bytes\n");
printf("File list size: 0\n");
printf("File list generation time: 0.000 seconds\n");
printf("File list transfer time: 0.000 seconds\n");
printf("Total bytes sent: %s\n", total);
printf("Total bytes received: 0\n");
printf("\n");
printf("sent %s bytes received 0 bytes %s bytes/sec\n", total, rate_str);
printf("total size is %s speedup is %.2f\n", total, 1.0);
fflush(stdout);
}
/* Compiled scanner inputs that are shared read-only across scanner instances
@@ -146,10 +176,16 @@ static bool prepare_scanner(const Config* config, int num_threads, PreparedScann
options->preserve_xattrs = config->preserve_xattrs;
options->preserve_acls = config->preserve_acls;
options->chunk_size = config->chunk_size;
options->exclude_patterns = config->exclude_patterns;
options->exclude_count = config->exclude_count;
options->include_patterns = config->include_patterns;
options->include_count = config->include_count;
/* --exclude/--include are compiled, in command-line order, into the SAME
* ordered filter rule list as --filter/-f (see config_add_selection_rule), so
* the legacy per-kind arrays are deliberately NOT passed to the scanner:
* doing so would re-apply them with the old "excludes first, then includes as
* a mandatory whitelist" precedence and defeat rsync's first-match-wins
* ordering. The arrays remain populated purely for the Config API surface. */
options->exclude_patterns = NULL;
options->exclude_count = 0;
options->include_patterns = NULL;
options->include_count = 0;
options->max_size = config->max_size;
options->min_size = config->min_size;
options->max_depth = config->max_depth;
@@ -175,6 +211,8 @@ static bool prepare_scanner(const Config* config, int num_threads, PreparedScann
options->ignore_missing_args = config->ignore_missing_args || config->delete_missing_args;
options->excluded_paths = NULL;
options->excluded_mutex = NULL;
options->size_skipped_paths = NULL;
options->synced_dirs = NULL;
options->hardlinks = NULL;
/* P7 Wave D: capture source directory metadata when a directory attribute is
requested (-p for modes, -t for times unless -O omits them). Whether they
@@ -467,7 +505,7 @@ static void receive_daemon_motd(Client* client, const Config* config) {
static Client* connect_transfer_client(const Config* config) {
if (config->transport == TRANSPORT_SSH) {
if (config->use_sendfile) {
log_message(LOG_LEVEL_ERROR, "-f/--sendfile is not supported with SSH transport");
log_message(LOG_LEVEL_ERROR, "--sendfile is not supported with SSH transport");
return NULL;
}
return client_connect_ssh(config->ssh_destination, config->ssh_port,
@@ -654,7 +692,10 @@ static void mark_sender_done(PipelineContextSender* context) {
file it processed, in send order: STATUS_NEXT means the file was written,
STATUS_OK means the file was skipped/unchanged. Skipped sources are marked
so the later removal pass keeps them. */
static bool finalize_transfer(Client* client, const Config* config, ArrayList* remove_sources) {
static bool finalize_transfer(Client* client, const Config* config, ArrayList* remove_sources,
bool* delete_limit_out) {
if (delete_limit_out)
*delete_limit_out = false;
if (!send_status(client->file_descriptor, STATUS_FINISHED))
return false;
if (config->remove_source_files && remove_sources) {
@@ -677,6 +718,15 @@ static bool finalize_transfer(Client* client, const Config* config, ArrayList* r
Status status;
if (!receive_status(client->file_descriptor, &status))
return false;
/* A capped --max-delete commit is a successful transfer that the client must
report with rsync's exit code 25 (not an error). */
if (status == STATUS_DELETE_LIMIT) {
log_message(LOG_LEVEL_ERROR,
"Deletions stopped due to --max-delete limit; some deletions were skipped");
if (delete_limit_out)
*delete_limit_out = true;
return true;
}
if (status != STATUS_OK) {
log_server_rejection("Receiver reported transfer failure");
return false;
@@ -782,30 +832,53 @@ static int send_dry_run_manifest(const Config* config) {
}
typedef struct {
char* path;
char* name; /* transfer-relative name ("" == the source root) */
mode_t mode;
unsigned long long size;
time_t mtime;
long mtime_nsec;
bool is_dir;
bool is_symlink;
char* link_target;
} ListEntry;
static void list_entries_destroy(ListEntry* entries, size_t count) {
if (entries == NULL)
return;
for (size_t i = 0; i < count; i++)
free(entries[i].path);
for (size_t i = 0; i < count; i++) {
free(entries[i].name);
free(entries[i].link_target);
}
free(entries);
}
static int compare_list_entries(const void* left, const void* right) {
const ListEntry* a = (const ListEntry*)left;
const ListEntry* b = (const ListEntry*)right;
return strcmp(a->path, b->path);
return strcmp(a->name, b->name);
}
/* --list-only: print an ls-style listing of the files that WOULD be
/* Relative path of an entry below `root` ("" for the root itself). Mirrors
* change_list's relative_name for list-only rendering. */
static char* list_relative_name(const char* root, const char* full) {
if (root == NULL || full == NULL)
return str_dup(full != NULL ? full : "");
size_t root_len = strlen(root);
while (root_len > 1 && root[root_len - 1] == '/')
root_len--;
if (strncmp(root, full, root_len) == 0) {
if (full[root_len] == '\0')
return str_dup("");
if (full[root_len] == '/')
return str_dup(full + root_len + 1);
}
return str_dup(full);
}
/* --list-only: print an ls-style listing of the entries that WOULD be
* transferred and exit without contacting the server or writing anything.
* Directory lines are not printed because the scanner only yields regular
* transfer candidates. Returns 0 on success, 1 on error. */
* Names are transfer-relative (rsync prints `a.txt`, `sub/b.txt`, `.`) and
* directory entries are included. Returns 0 on success, 1 on error. */
static int send_list_only(const Config* config) {
int skipped = 0;
if (!files_from_list_check(config, NULL, &skipped))
@@ -814,6 +887,7 @@ static int send_list_only(const Config* config) {
if (!prepare_scanner(config, 0, &prepared))
return 1;
prepared.options.use_metadata = true; /* capture mode + mtime for the listing */
prepared.options.list_dirs = true;
DirectoryScanner* scanner =
directory_scanner_create_with_options(config->send_directory, &prepared.options);
if (!scanner) {
@@ -823,9 +897,33 @@ static int send_list_only(const Config* config) {
ListEntry* entries = NULL;
size_t count = 0;
size_t capacity = 0;
Chunk* chunk;
bool oom = false;
while ((chunk = directory_scanner_next(scanner)) != NULL) {
/* rsync lists the source root itself (as "."). Only when the source is a
* directory and no --files-from subset is in effect. */
if (config->files_from_set == NULL && config->send_directory != NULL) {
struct stat st;
if (stat(config->send_directory, &st) == 0 && S_ISDIR(st.st_mode)) {
capacity = 64;
entries = calloc(capacity, sizeof(ListEntry));
if (entries == NULL) {
oom = true;
} else if ((entries[0].name = str_dup("")) == NULL) {
/* A NULL name would be dereferenced by qsort/render: fail the listing. */
oom = true;
} else {
entries[0].mode = st.st_mode;
entries[0].mtime = st.st_mtime;
entries[0].mtime_nsec = st.st_mtim.tv_nsec;
entries[0].size = (unsigned long long)st.st_size;
entries[0].is_dir = true;
count = 1;
}
}
}
Chunk* chunk;
while (!oom && (chunk = directory_scanner_next(scanner)) != NULL) {
for (int i = 0; i < chunk->element_count; i++) {
File* f = chunk->items[i];
if (f == NULL)
@@ -842,34 +940,47 @@ static int send_list_only(const Config* config) {
break;
}
entries = grown;
memset(entries + capacity, 0, (new_capacity - capacity) * sizeof(ListEntry));
capacity = new_capacity;
}
char* path = str_dup(file_wire_path(f));
if (!path) {
char* name = list_relative_name(config->send_directory, file_wire_path(f));
if (!name) {
oom = true;
break;
}
mode_t mode = 0;
time_t mtime = 0;
long mtime_nsec = 0;
if (f->metadata != NULL) {
mode = f->metadata->mode;
mtime = f->metadata->mtime_sec;
mtime_nsec = f->metadata->mtime_nsec;
} else {
struct stat st;
if (stat(f->path, &st) == 0) {
if (lstat(f->path, &st) == 0) {
mode = st.st_mode;
mtime = st.st_mtime;
mtime_nsec = st.st_mtim.tv_nsec;
}
}
entries[count].path = path;
entries[count].name = name;
entries[count].mode = mode;
entries[count].mtime = mtime;
entries[count].size = f->data != NULL ? f->data->size : 0;
entries[count].mtime_nsec = mtime_nsec;
if (f->is_symlink)
entries[count].size = f->symlink_target != NULL ? strlen(f->symlink_target) : 0;
else if (f->is_dir) {
struct stat dir_st;
entries[count].size = stat(f->path, &dir_st) == 0 ? (unsigned long long)dir_st.st_size : 0;
} else
entries[count].size = f->data != NULL ? f->data->size : 0;
entries[count].is_dir = f->is_dir;
entries[count].is_symlink = f->is_symlink;
entries[count].link_target =
f->is_symlink && f->symlink_target ? str_dup(f->symlink_target) : NULL;
count++;
}
chunk_destroy(chunk);
if (oom)
break;
}
bool failed = oom || directory_scanner_failed(scanner) || directory_scanner_had_io_error(scanner);
directory_scanner_destroy(scanner);
@@ -883,8 +994,18 @@ static int send_list_only(const Config* config) {
if (count > 1)
qsort(entries, count, sizeof(ListEntry), compare_list_entries);
for (size_t i = 0; i < count; i++) {
char* line = change_render_list_line(entries[i].mode, entries[i].size, entries[i].mtime,
entries[i].path);
ChangeEvent event;
memset(&event, 0, sizeof(event));
event.name = entries[i].name;
event.path = entries[i].name;
event.mode = entries[i].mode;
event.size = entries[i].size;
event.mtime_sec = entries[i].mtime;
event.mtime_nsec = entries[i].mtime_nsec;
event.is_directory = entries[i].is_dir;
event.is_symlink = entries[i].is_symlink;
event.symlink_target = entries[i].link_target;
char* line = change_render_list_line(config, &event);
if (line != NULL) {
char* escaped = output_escape(line, config->eight_bit_output);
printf("%s\n", escaped != NULL ? escaped : line);
@@ -896,21 +1017,25 @@ static int send_list_only(const Config* config) {
return 0;
}
/* Send the delete manifest (keep-set paths plus the protected excluded
prefixes and the --delete-missing-args exact-delete paths) to the server.
Returns 0 on success, -1 on failure. When --delete-excluded is given
`protected` is empty: excluded destination mirrors are then ordinary extras
and are removed. When --delete-missing-args is active `missing_args` holds
the destination mirrors of missing --files-from entries: each is an explicit
receiver-side deletion request, independent of the extras walk. A NULL
keep-set / protected / missing list transmits an empty section. All three
sections are unbounded on the sender; the receiver enforces
/* Send the delete manifest to the server. Returns 0 on success, -1 on
failure. It carries FOUR sections: the keep-set paths, the protected
excluded prefixes, the --delete-missing-args exact-delete paths, and the
destination-relative directories the sender synchronized this run.
When --delete-excluded is given `protected` is empty: excluded destination
mirrors are then ordinary extras and are removed. When
--delete-missing-args is active `missing_args` holds the destination mirrors
of missing --files-from entries: each is an explicit receiver-side deletion
request, independent of the extras walk. `synced_dirs` confines the extras
walk to entries directly inside a synchronized directory. A NULL
keep-set / protected / missing / dirs list transmits an empty section. All
four sections are unbounded on the sender; the receiver enforces
MAX_MANIFEST_ENTRIES per section and a single MAX_MANIFEST_BYTES budget
shared across the sections, rejecting (with STATUS_ERROR) an over-budget
frame. A heavily filtered source whose exclusion list is large therefore
fails the run cleanly on the receiver rather than being truncated. */
static int send_delete_manifest(int fd, ArrayList* manifest, ArrayList* protected_prefixes,
ArrayList* missing_args) {
ArrayList* size_skipped, ArrayList* missing_args,
ArrayList* synced_dirs) {
if (!send_status(fd, STATUS_MANIFEST))
return -1;
int keep_count = manifest ? manifest->size : 0;
@@ -920,12 +1045,24 @@ static int send_delete_manifest(int fd, ArrayList* manifest, ArrayList* protecte
if (!send_wire_str(fd, (char*)manifest->items[i]))
return -1;
}
int protected_count = protected_prefixes ? protected_prefixes->size : 0;
/* The receiver has ONE protected-prefix section; filter-excluded prefixes
(dropped under --delete-excluded) and size-pruned prefixes (always
protected) are concatenated into it. */
int protected_count =
(protected_prefixes ? protected_prefixes->size : 0) + (size_skipped ? size_skipped->size : 0);
if (!send_int(fd, protected_count))
return -1;
for (int i = 0; i < protected_count; i++) {
if (!send_wire_str(fd, (char*)protected_prefixes->items[i]))
return -1;
if (protected_prefixes) {
for (int i = 0; i < protected_prefixes->size; i++) {
if (!send_wire_str(fd, (char*)protected_prefixes->items[i]))
return -1;
}
}
if (size_skipped) {
for (int i = 0; i < size_skipped->size; i++) {
if (!send_wire_str(fd, (char*)size_skipped->items[i]))
return -1;
}
}
int missing_count = missing_args ? missing_args->size : 0;
if (!send_int(fd, missing_count))
@@ -934,6 +1071,13 @@ static int send_delete_manifest(int fd, ArrayList* manifest, ArrayList* protecte
if (!send_wire_str(fd, (char*)missing_args->items[i]))
return -1;
}
int dirs_count = synced_dirs ? synced_dirs->size : 0;
if (!send_int(fd, dirs_count))
return -1;
for (int i = 0; i < dirs_count; i++) {
if (!send_wire_str(fd, (char*)synced_dirs->items[i]))
return -1;
}
return 0;
}
@@ -953,11 +1097,12 @@ static int send_delete_manifest(int fd, ArrayList* manifest, ArrayList* protecte
#define DELETE_ACK_KEEPALIVE_SEC 10
static bool send_delete_manifest_early(Client* client, ArrayList* manifest,
ArrayList* protected_prefixes, ArrayList* missing_args) {
ArrayList* protected_prefixes, ArrayList* size_skipped,
ArrayList* missing_args, ArrayList* synced_dirs) {
if (!client || !manifest)
return false;
if (send_delete_manifest(client->file_descriptor, manifest, protected_prefixes, missing_args) !=
0)
if (send_delete_manifest(client->file_descriptor, manifest, protected_prefixes, size_skipped,
missing_args, synced_dirs) != 0)
return false;
Status ack;
/* The wait is long (up to an hour) and runs inline on this thread: a helper
@@ -1052,6 +1197,19 @@ static int incremental_check(Client* client, File* file, const Config* config,
Status s;
if (!receive_status(client->file_descriptor, &s))
return -1;
/* Output parity: when dest-info reporting is negotiated the receiver sends
* the pre-transfer destination snapshot BEFORE its ordinary verdict. Consume
* it here so the following status read stays in sync. */
if (config->report_dest_info) {
if (s != STATUS_DEST_INFO ||
!format_dest_state_receive(client->file_descriptor, &file->dest_state)) {
log_message(LOG_LEVEL_ERROR, "Unexpected reply to the destination-state report");
send_status(client->file_descriptor, STATUS_ERROR);
return -1;
}
if (!receive_status(client->file_descriptor, &s))
return -1;
}
if (s == STATUS_ERROR) {
log_server_rejection("Server reported error for file");
return -1;
@@ -1429,7 +1587,11 @@ static bool send_directory_entry(const Client* client, File* file, const Config*
if (!send_status(client->file_descriptor, STATUS_MKDIR) ||
!send_wire_str(client->file_descriptor, file_wire_path(file)))
return false;
return !config->use_metadata || metadata_send(client->file_descriptor, file->metadata);
if (config->use_metadata && !metadata_send(client->file_descriptor, file->metadata))
return false;
/* Directory xattrs/ACLs (-X/-A) ride the same trailing block as regular files
when the xattr transport was negotiated. */
return !config->use_xattrs || xattr_send(client->file_descriptor, file->xattrs);
}
/* P7 Wave D: transmit every captured source directory's metadata in terminal
@@ -1461,6 +1623,9 @@ static bool send_dir_times(const Client* client, const Config* config, ArrayList
return false;
if (!send_wire_str(fd, file_wire_path(file)) || !metadata_send(fd, file->metadata))
return false;
/* Directory xattrs/ACLs travel with the deferred directory metadata. */
if (config->use_xattrs && !xattr_send(fd, file->xattrs))
return false;
}
index += chunk;
}
@@ -1761,7 +1926,8 @@ static int send_chunks_multithreaded(void* pipeline_context) {
/* The keep-set manifest was prebuilt by a path-only pre-scan. Transmit it
and wait for the receiver to delete extras before streaming any data. */
if (!send_delete_manifest_early(client, context->manifest, context->excluded_paths,
context->missing_args)) {
context->size_skipped_paths, context->missing_args,
context->synced_dirs)) {
pipeline_cancel(context);
disconnect_transfer_client(client);
mark_sender_done(context);
@@ -1868,13 +2034,15 @@ static int send_chunks_multithreaded(void* pipeline_context) {
goto send_fail;
}
if (send_delete_manifest(client->file_descriptor, context->manifest, context->excluded_paths,
context->missing_args) != 0)
context->size_skipped_paths, context->missing_args,
context->synced_dirs) != 0)
goto send_fail;
} else if (context->config->delete_missing_args && !context->early_delete) {
/* --delete-missing-args without --delete: no keep-set is built, but the
exact-delete paths still ride the same manifest frame (commit once the
transfer succeeded). */
if (send_delete_manifest(client->file_descriptor, NULL, NULL, context->missing_args) != 0)
if (send_delete_manifest(client->file_descriptor, NULL, NULL, NULL, context->missing_args,
NULL) != 0)
goto send_fail;
}
/* P7 Wave D: transmit the captured directory times last. The scanner thread
@@ -1884,11 +2052,12 @@ static int send_chunks_multithreaded(void* pipeline_context) {
if (!context->scan_stopped_early &&
!send_dir_times(client, context->config, context->dir_entries))
goto send_fail;
bool ok = finalize_transfer(client, context->config, context->remove_source_files);
bool delete_limit = false;
bool ok = finalize_transfer(client, context->config, context->remove_source_files, &delete_limit);
context->delete_limit = delete_limit;
if (!ok && context->config->use_delete)
log_message(LOG_LEVEL_ERROR,
"server reported a deletion failure (--delete); see the server log for the "
"reason (a --max-delete limit that the run would exceed deletes nothing)");
"server reported a deletion failure (--delete); see the server log for the reason");
if (ok)
remove_transferred_sources(context->config, context->remove_source_files);
mtx_lock(&context->mutex_progress);
@@ -1931,11 +2100,18 @@ static int scan_directory_multithreaded(void* pipeline_context) {
prepared.options.dir_entries = context->dir_entries;
prepared.options.dir_entries_mutex = &context->dir_entries_mutex;
/* The keep-set manifest for the late modes is built from this data pass, so
the parallel scanner records the protected excluded prefixes here. The
early modes already transmitted the pre-scan keep-set and its protected
list, so the data pass must not append to it again. */
if (!context->early_delete)
the parallel scanner records the protected excluded prefixes and the
synchronized directories here (the size-prune protection is collected in
every mode). The early modes already transmitted the pre-scan keep-set and
its protected lists, so the data pass must not append to them again. */
if (!context->early_delete) {
prepared.options.excluded_paths = context->excluded_paths;
/* The root marker for a full recursive transfer is already in the list; do
not let the scanner append every directory to it. */
if (context->config->files_from_set != NULL)
prepared.options.synced_dirs = context->synced_dirs;
}
prepared.options.size_skipped_paths = context->size_skipped_paths;
bool dirs_mode = prepared.options.dirs;
/* -H also selects the sequential scanner (see the comment at the branch),
* so the loop below must choose the scanner by which object exists, not by
@@ -2242,6 +2418,9 @@ int send_files(Config* config) {
ArrayList* dir_entries = NULL;
/* Protected excluded prefixes (delete-excluded default protection). */
ArrayList* excluded = NULL;
/* Size-pruned prefixes (always protected) and synchronized directories. */
ArrayList* size_skipped = NULL;
ArrayList* synced_dirs = NULL;
bool delete_early = config->use_delete && config_delete_timing_early(config);
bool send_failed = false;
bool had_scan_io = false;
@@ -2265,11 +2444,31 @@ int send_files(Config* config) {
by user-selection rules so the receiver protects their destination mirrors
from --delete (rsync's default). Only scans that build the keep-set get the
sink attached (prescan for early timing, the streaming data pass otherwise). */
if (config->use_delete && !config->delete_excluded) {
excluded = array_list_create(free);
if (!excluded)
if (config->use_delete) {
if (!config->delete_excluded) {
excluded = array_list_create(free);
if (!excluded)
goto send_fail;
prepared.options.excluded_paths = excluded;
}
size_skipped = array_list_create(free);
synced_dirs = array_list_create(free);
if (!size_skipped || !synced_dirs)
goto send_fail;
prepared.options.excluded_paths = excluded;
prepared.options.size_skipped_paths = size_skipped;
/* Only a --files-from subset confines the extras walk to the directories
the scan synchronized; a full recursive transfer deletes throughout the
receive root, so mark the root itself (the "." sentinel) and let the
scanner record nothing extra. */
if (config->files_from_set == NULL) {
char* root_marker = str_dup(".");
if (!root_marker || !array_list_add(synced_dirs, root_marker)) {
free(root_marker);
goto send_fail;
}
} else {
prepared.options.synced_dirs = synced_dirs;
}
}
/* The late-timing modes (plain --delete / --delete-after / --delete-delay)
build the manifest while streaming and send it after the last data frame.
@@ -2296,13 +2495,16 @@ int send_files(Config* config) {
"with an empty keep-set (--delete)");
prescan_ok = false;
} else {
early_ok = send_delete_manifest_early(client, early_manifest, excluded, missing_args);
early_ok = send_delete_manifest_early(client, early_manifest, excluded, size_skipped,
missing_args, synced_dirs);
}
}
array_list_delete(early_manifest);
/* The keep-set (and its protected prefixes) are already on the wire; the
data pass must not append to the exclusion list again. */
/* The keep-set (and its protected prefixes and synchronized directories) are
already on the wire; the data pass must not append to those lists again. */
prepared.options.excluded_paths = NULL;
prepared.options.size_skipped_paths = NULL;
prepared.options.synced_dirs = NULL;
if (!prescan_ok || !early_ok)
goto send_fail;
} else if (config->use_delete) {
@@ -2446,7 +2648,8 @@ int send_files(Config* config) {
--delete-missing-args exact-path deletions only after the transfer
succeeds. In the early modes (--delete-before/--delete-during) the
manifest already went out up front, so nothing is re-sent here. */
if (send_delete_manifest(client->file_descriptor, manifest, excluded, missing_args) != 0) {
if (send_delete_manifest(client->file_descriptor, manifest, excluded, size_skipped,
missing_args, synced_dirs) != 0) {
if (manifest) {
array_list_delete(manifest);
manifest = NULL;
@@ -2464,11 +2667,11 @@ int send_files(Config* config) {
applying them until after its own deletion/publication phase. */
if (!send_dir_times(client, config, dir_entries))
goto send_fail;
bool ok = finalize_transfer(client, config, remove_sources);
bool delete_limit = false;
bool ok = finalize_transfer(client, config, remove_sources, &delete_limit);
if (!ok && config->use_delete)
log_message(LOG_LEVEL_ERROR,
"server reported a deletion failure (--delete); see the server log for the "
"reason (a --max-delete limit that the run would exceed deletes nothing)");
"server reported a deletion failure (--delete); see the server log for the reason");
if (ok)
remove_transferred_sources(config, remove_sources);
if (config->show_progress && !config->quiet)
@@ -2477,8 +2680,13 @@ int send_files(Config* config) {
log_info_message(LOG_INFO_STATS, "Transfer summary: %d files, %.1f MB", total_files,
(double)total_bytes / (double)BYTES_PER_MIB);
/* --ignore-errors: an unreadable source directory was skipped but the run
still completed (and deleted); report the run as errored like rsync does. */
ret = (ok && !had_scan_io) ? 0 : 1;
still completed (and deleted); report the run as errored like rsync does.
A --max-delete-capped commit is a successful transfer that rsync reports
with exit code 25. */
if (!ok || had_scan_io)
ret = 1;
else
ret = delete_limit ? 25 : 0;
send_fail:
/* Single cleanup path for all exits. The manifest is intentionally deleted
@@ -2487,6 +2695,10 @@ send_fail:
array_list_delete(manifest);
if (excluded)
array_list_delete(excluded);
if (size_skipped)
array_list_delete(size_skipped);
if (synced_dirs)
array_list_delete(synced_dirs);
if (missing_args)
array_list_delete(missing_args);
if (remove_sources)
@@ -2592,16 +2804,41 @@ int send_files_multithreaded(Config** config_ptr) {
return 1;
}
}
/* Size-pruned mirrors stay protected under every mode (even
--delete-excluded); synchronized directories confine the walk. A full
recursive transfer marks the receive root itself (".") so the walk is not
confined; only a --files-from subset records concrete directories. */
context->size_skipped_paths = array_list_create(free);
context->synced_dirs = array_list_create(free);
if (!context->size_skipped_paths || !context->synced_dirs) {
pipeline_context_sender_destroy(context);
return 1;
}
if (config->files_from_set == NULL) {
char* root_marker = str_dup(".");
if (!root_marker || !array_list_add(context->synced_dirs, root_marker)) {
free(root_marker);
pipeline_context_sender_destroy(context);
return 1;
}
}
if (config_delete_timing_early(config)) {
/* --delete-before/--delete-during: build the complete keep-set manifest
(paths only, nothing loaded or sent) up front so the sender thread can
transmit it before the first data byte. The path-only pre-scan also
fills the protected excluded prefixes. */
fills the protected excluded prefixes and synchronized directories. */
PreparedScanner prepared;
memset(&prepared, 0, sizeof(prepared));
bool prepared_ok = prepare_scanner(config, config->scanner_threads, &prepared);
if (prepared_ok && context->excluded_paths)
prepared.options.excluded_paths = context->excluded_paths;
if (prepared_ok) {
if (context->excluded_paths)
prepared.options.excluded_paths = context->excluded_paths;
prepared.options.size_skipped_paths = context->size_skipped_paths;
/* The root marker for a full recursive transfer is already in the list;
only a --files-from subset needs the scanner to record directories. */
if (config->files_from_set != NULL)
prepared.options.synced_dirs = context->synced_dirs;
}
bool prebuilt = prepared_ok && scan_paths_only(config, &prepared.options, context->manifest,
&context->scan_had_io_error);
prepared_scanner_destroy(&prepared);
@@ -2683,9 +2920,13 @@ int send_files_multithreaded(Config** config_ptr) {
scan_io = context->scan_had_io_error;
mtx_unlock(&context->mutex_scanner);
bool sender_ok = sender_result == thrd_success;
bool delete_limit = context->delete_limit;
/* --ignore-errors: the run completed (and deleted) past an unreadable source
directory; report it as errored like rsync does. */
directory; report it as errored like rsync does. A --max-delete-capped
commit is a successful transfer that rsync reports with exit code 25. */
pipeline_context_sender_destroy(context);
client_set_abort_armed(false);
return sender_ok && !scan_io ? 0 : 1;
if (!sender_ok || scan_io)
return 1;
return delete_limit ? 25 : 0;
}
+180 -34
View File
@@ -156,9 +156,13 @@ typedef struct {
bool is_symlink;
char* link_target;
/* True when the entry was pruned by a user selection rule (--filter/-C/per-dir
rules, the --exclude/--include layer, or --max-size/--min-size) rather than
skipped for another reason (unreadable, symlink policy, not applicable). */
rules or the --exclude/--include layer) rather than skipped for another
reason (unreadable, symlink policy, not applicable). */
bool excluded;
/* True when the entry was skipped specifically by --max-size/--min-size.
Size pruning protects the destination mirror even under --delete-excluded,
so it is recorded into a separate sink from `excluded`. */
bool size_excluded;
} ScannerEntry;
/* --one-file-system (-x) decision. Only directories can carry a different
@@ -168,6 +172,26 @@ bool scanner_same_filesystem(bool one_file_system, dev_t root_device, dev_t entr
return !one_file_system || entry_device == root_device;
}
/* Build a payload-less directory File carrying the captured metadata (when
* requested). Used by -x mount-point emission and --list-only directory
* entries. Returns NULL on allocation failure. */
static File* scanner_build_dir_file(const char* path, const struct stat* stats,
const ScannerOptions* options) {
File* dir = file_create(path);
if (dir == NULL)
return NULL;
dir->is_dir = true;
if (options->use_metadata) {
dir->metadata =
file_metadata_create(dir->path, stats, options->preserve_atimes, options->preserve_crtimes);
if (!dir->metadata) {
file_destroy(dir);
return NULL;
}
}
return dir;
}
/* Relative path of an on-disk path below `root`. The transfer root may be
* given with a trailing slash; the returned rel path never has one and is ""
* for the root itself. A root of "/" is handled (its children start at "/").
@@ -302,19 +326,49 @@ static bool excluded_sink_append(ArrayList* list, mtx_t* mtx, const char* rel) {
return ok;
}
/* Record one pruned-by-user-selection filesystem path in the scanner's
exclusion sink (see ScannerOptions.excluded_paths). The stored form is the
entry's wire/destination-relative path (a single leading '/' removed, exactly
how manifest keep entries are stored), so the receiver's walker prefixes
match the destination layout. An allocation failure is a fatal scan error. */
static void scanner_record_excluded(DirectoryScanner* scanner, const char* fs_path) {
if (!scanner->options.excluded_paths || !fs_path)
/* Record one pruned filesystem path in a delete-protection sink. The stored
form is the entry's wire/destination-relative path (a single leading '/'
removed, exactly how manifest keep entries are stored), so the receiver's
walker prefixes match the destination layout. An allocation failure is a
fatal scan error. */
static void scanner_record_protected(DirectoryScanner* scanner, const char* fs_path,
ArrayList* sink) {
if (!sink || !fs_path)
return;
const char* rel = *fs_path == '/' ? fs_path + 1 : fs_path;
if (!excluded_sink_append(scanner->options.excluded_paths, scanner->options.excluded_mutex, rel))
if (!excluded_sink_append(sink, scanner->options.excluded_mutex, rel))
scanner->failed = true;
}
/* A user-selection exclusion (--filter/-C/per-dir or --exclude/--include). */
static void scanner_record_excluded(DirectoryScanner* scanner, const char* fs_path) {
scanner_record_protected(scanner, fs_path, scanner->options.excluded_paths);
}
/* A --max-size/--min-size prune (always protected, even under --delete-excluded). */
static void scanner_record_size_skipped(DirectoryScanner* scanner, const char* fs_path) {
scanner_record_protected(scanner, fs_path, scanner->options.size_skipped_paths);
}
/* Record a directory the scan synchronized. `fs_path` is its absolute path and
`rel` its path relative to the transfer root ("" for the root); the stored
form matches the wire layout (the bare relative path in -R+--files-from, else
the source path with a leading '/' removed, with "." for the receive root).
Returns false on allocation failure. */
static bool scanner_record_synced_dir(const ScannerOptions* options, const char* fs_path,
const char* rel, bool relative_mode) {
if (!options->synced_dirs)
return true;
if (!file_list_dir_in_scope(options->file_list, rel))
return true;
const char* dest = relative_mode ? rel : fs_path;
if (dest[0] == '/')
dest++;
if (dest[0] == '\0')
dest = ".";
return excluded_sink_append(options->synced_dirs, options->excluded_mutex, dest);
}
/* Merge the open directory's own .rsync-filter rules into the inherited
* context, returning the context used for this directory's entries. On a parse
* error the scanner is marked failed. Returns 0 on success, -1 on failure. */
@@ -357,6 +411,7 @@ static int open_directory_filter_context(DirectoryScanner* scanner, const Filter
static int scanner_inspect_entry(const ScannerOptions* options, const char* containing_dir,
const char* link_rel, const char* name, ScannerEntry* entry) {
entry->excluded = false;
entry->size_excluded = false;
entry->is_symlink = false;
entry->link_target = NULL;
entry->path = path_cat(containing_dir, name);
@@ -443,6 +498,7 @@ apply_filters:
if ((options->max_size > 0 && (unsigned long long)entry->stats.st_size > options->max_size) ||
(options->min_size > 0 && (unsigned long long)entry->stats.st_size < options->min_size)) {
entry->excluded = true;
entry->size_excluded = true;
goto skip;
}
return 1;
@@ -609,7 +665,8 @@ static Chunk* chunk_data_to_chunk(ArrayList* chunk_data) {
* allocation failure is fatal and reported to the caller. */
static bool scanner_capture_dir_time(ArrayList* dir_entries, mtx_t* mutex, const char* root_path,
const char* fs_path, bool relative_mode, bool preserve_atimes,
bool preserve_crtimes) {
bool preserve_crtimes, bool preserve_xattrs,
bool preserve_acls) {
if (!dir_entries || !root_path || !fs_path)
return true;
struct stat st;
@@ -636,6 +693,11 @@ static bool scanner_capture_dir_time(ArrayList* dir_entries, mtx_t* mutex, const
file_destroy(file);
return false;
}
/* Directory xattrs/ACLs (-X/-A): captured here so the deferred
STATUS_DIR_TIMES frame can carry them and the receiver can re-apply them
fd-relative (a regular file's per-file block never covered directories). */
if (preserve_xattrs || preserve_acls)
file->xattrs = xattr_capture_path(fs_path, preserve_acls);
if (relative_mode) {
file->send_path = rel;
rel = NULL;
@@ -722,11 +784,24 @@ static int open_next_directory(DirectoryScanner* scanner) {
scanner->current_path = NULL;
return -1;
}
/* A successfully opened directory is synchronized for --delete: record it
so the receiver confines its extras walk to these (and the root sentinel
".") instead of the whole receive root. */
if (!scanner_record_synced_dir(&scanner->options, scanner->current_path, scanner->current_rel,
scanner->relative_mode)) {
closedir(scanner->current_dir);
scanner->current_dir = NULL;
free(scanner->current_path);
scanner->current_path = NULL;
scanner->failed = true;
return -1;
}
if (scanner->options.capture_dir_times &&
!scanner_capture_dir_time(scanner->options.dir_entries, scanner->options.dir_entries_mutex,
scanner->root_path, scanner->current_path, scanner->relative_mode,
scanner->options.preserve_atimes,
scanner->options.preserve_crtimes)) {
!scanner_capture_dir_time(
scanner->options.dir_entries, scanner->options.dir_entries_mutex, scanner->root_path,
scanner->current_path, scanner->relative_mode, scanner->options.preserve_atimes,
scanner->options.preserve_crtimes, scanner->options.preserve_xattrs,
scanner->options.preserve_acls)) {
closedir(scanner->current_dir);
scanner->current_dir = NULL;
free(scanner->current_path);
@@ -776,6 +851,7 @@ static File* dirs_root_dir_file(DirectoryScanner* scanner) {
return NULL;
}
}
scanner_capture_xattrs(scanner, file);
return file;
}
@@ -889,6 +965,7 @@ static File* dirs_file_for_entry(DirectoryScanner* scanner, const char* entry) {
return NULL;
}
}
scanner_capture_xattrs(scanner, file);
return file;
}
@@ -1041,17 +1118,25 @@ Chunk* directory_scanner_next(DirectoryScanner* scanner) {
break;
}
if (inspection == 0) {
/* The entry was pruned by a user selection rule (exclude/include/size) or
skipped for another reason; only the user-selection prunes protect the
corresponding destination mirror from --delete. */
/* A user-selection exclude protects its destination mirror from --delete
unless --delete-excluded; a size prune is always protected. Other
skips (unreadable, symlink policy) protect nothing. Under -R +
--files-from the protected prefix must be the entry's bare relative
wire path, not its source path (which would not match the destination
layout and would leave the mirror deletable). */
if (inspected.excluded) {
char* abs_path = path_cat(scanner->current_path, entry->d_name);
if (!abs_path) {
char* protected_path = scanner->relative_mode
? child_rel_path(scanner->current_rel, entry->d_name)
: path_cat(scanner->current_path, entry->d_name);
if (!protected_path) {
scanner->failed = true;
break;
}
scanner_record_excluded(scanner, abs_path);
free(abs_path);
if (inspected.size_excluded)
scanner_record_size_skipped(scanner, protected_path);
else
scanner_record_excluded(scanner, protected_path);
free(protected_path);
}
continue;
}
@@ -1100,9 +1185,31 @@ Chunk* directory_scanner_next(DirectoryScanner* scanner) {
free(rel_copy);
if (!scanner_same_filesystem(scanner->options.one_file_system, scanner->root_dev,
stats.st_dev)) {
/* rsync's -x/--one-file-system emits the mount-point directory entry
itself (so the destination gets an empty directory) but does NOT
descend into it. Build a payload-less directory File and hand it to
the caller; never enqueue it for traversal. */
File* mount = scanner_build_dir_file(cur_path, &stats, &scanner->options);
if (mount == NULL || !array_list_add(chunk_data, mount)) {
file_destroy(mount);
free(cur_path);
scanner->failed = true;
break;
}
free(cur_path);
continue;
}
/* --list-only: list directory entries too (rsync prints them), even
though a real transfer never sends them explicitly. */
if (scanner->options.list_dirs) {
File* dir = scanner_build_dir_file(cur_path, &stats, &scanner->options);
if (dir == NULL || !array_list_add(chunk_data, dir)) {
file_destroy(dir);
free(cur_path);
scanner->failed = true;
break;
}
}
int next_depth = scanner->current_depth + 1;
if (scanner->options.max_depth <= 0 || next_depth < scanner->options.max_depth) {
DirEntry* de = dir_entry_create(cur_path, next_depth, scanner->current_node);
@@ -1404,18 +1511,27 @@ static void scan_root_entry(const ScannerOptions* options, const FilterNode* roo
return;
}
if (inspection == 0) {
if (inspected.excluded && options->excluded_paths) {
/* A root-level user-selection prune protects the destination mirror of
the same-named wire path (at the root the bare name is the wire path in
every layout). */
char* abs_path = path_cat(root_directory, entry->d_name);
if (!abs_path) {
ps->failed = true;
} else {
const char* rel = *abs_path == '/' ? abs_path + 1 : abs_path;
if (!excluded_sink_append(options->excluded_paths, options->excluded_mutex, rel))
ArrayList* sink = NULL;
if (inspected.excluded)
sink = inspected.size_excluded ? options->size_skipped_paths : options->excluded_paths;
if (sink) {
/* A root-level prune protects the destination mirror of the entry's wire
path: under -R + --files-from that is the bare relative name, otherwise
it is the full source path with a leading '/' removed (matching the
send_path/file_wire_path the scanner hands the sender). */
if (options->relative && options->file_list != NULL) {
if (!excluded_sink_append(sink, options->excluded_mutex, entry->d_name))
ps->failed = true;
free(abs_path);
} else {
char* abs_path = path_cat(root_directory, entry->d_name);
if (!abs_path) {
ps->failed = true;
} else {
const char* rel = *abs_path == '/' ? abs_path + 1 : abs_path;
if (!excluded_sink_append(sink, options->excluded_mutex, rel))
ps->failed = true;
free(abs_path);
}
}
}
return;
@@ -1449,7 +1565,28 @@ static void scan_root_entry(const ScannerOptions* options, const FilterNode* roo
if (is_dir) {
free(rel);
if (!scanner_same_filesystem(options->one_file_system, root_dev, st.st_dev)) {
/* -x/--one-file-system: emit the mount-point directory entry (empty) but
do not descend into it (see the sequential scanner for the same rule). */
File* mount = file_create(cur_path);
free(cur_path);
if (mount == NULL) {
ps->failed = true;
return;
}
mount->is_dir = true;
if (options->use_metadata) {
mount->metadata = file_metadata_create(mount->path, &st, options->preserve_atimes,
options->preserve_crtimes);
if (!mount->metadata) {
file_destroy(mount);
ps->failed = true;
return;
}
}
if (!array_list_add(root_files, mount)) {
file_destroy(mount);
ps->failed = true;
}
return;
}
if (!array_list_add(subdirs, cur_path)) {
@@ -1534,6 +1671,14 @@ static bool scan_root_directory(ParallelScanner* ps, const char* root_directory,
log_perror("Could not open root directory for parallel scan");
return false;
}
/* The parallel scanner opens the transfer root directly (not through
open_next_directory), so record it as synchronized here. */
if (!scanner_record_synced_dir(options, root_directory, "",
options->relative && options->file_list != NULL)) {
closedir(dir);
ps->failed = true;
return false;
}
const struct dirent* entry;
while ((entry = readdir(dir)) != NULL) {
if (strcmp(entry->d_name, ".") == 0 || strcmp(entry->d_name, "..") == 0)
@@ -1698,7 +1843,8 @@ ParallelScanner* parallel_scanner_create_with_options(const char* root_directory
if (options->capture_dir_times &&
!scanner_capture_dir_time(options->dir_entries, options->dir_entries_mutex, root_directory,
root_directory, options->relative && options->file_list != NULL,
options->preserve_atimes, options->preserve_crtimes)) {
options->preserve_atimes, options->preserve_crtimes,
options->preserve_xattrs, options->preserve_acls)) {
array_list_delete(root_files);
array_list_delete(subdirs);
parallel_scanner_destroy(ps);
+28 -9
View File
@@ -65,6 +65,10 @@ typedef struct {
bool per_dir_filters; /* -F: read .rsync-filter per directory */
bool dirs; /* -d/--dirs: transfer dir entries, no recursion */
bool relative; /* -R/--relative (dest rel paths, with --files-from) */
/* --list-only: emit an is_dir File for every traversed directory (the listing
* includes directory entries, matching rsync). Client-only; never set on a
* real transfer, which relies on implicit parent creation. */
bool list_dirs;
/* --prune-empty-dirs (long only): in --dirs mode an empty source directory's
explicit entry is omitted from the transfer file list (so nothing is
created at the destination and it can be pruned by --delete); explicitly
@@ -73,17 +77,32 @@ typedef struct {
bool prune_empty_dirs;
/* Delete-excluded protection sink (optional): when non-NULL the scanner
* appends the destination-relative path of every entry it prunes because a
* USER SELECTION rule excluded it (--filter/-C/per-dir rules, the legacy
* --exclude/--include layer, and --max-size/--min-size). The sender turns
* this list into the manifest's protected prefixes so `--delete` leaves the
* destination mirror of excluded source paths alone (rsync's default), and
* empties it when --delete-excluded opts back into deleting them. NOT
* recorded for --files-from subset pruning (whose delete semantics stay
* keep-set-only) or for -R/--files-from relative wire paths. When
* `excluded_mutex` is non-NULL it is taken around every append (the parallel
* scanner shares one list across its worker threads). */
* USER SELECTION rule excluded it (--filter/-C/per-dir rules and the legacy
* --exclude/--include layer). The sender turns this list into the manifest's
* protected prefixes so `--delete` leaves the destination mirror of excluded
* source paths alone (rsync's default), and drops it when --delete-excluded
* opts back into deleting them. NOT recorded for --files-from subset pruning
* (whose delete semantics derive from the synchronized-directory set) or for
* -R/--files-from relative wire paths. When `excluded_mutex` is non-NULL it
* is taken around every append (the parallel scanner shares one list across
* its worker threads). */
ArrayList* excluded_paths;
mtx_t* excluded_mutex;
/* Size-prune protection sink (optional): when non-NULL the scanner appends
* the destination-relative path of every entry it skipped because of
* --max-size/--min-size. rsync never deletes a size-skipped source mirror,
* even under --delete-excluded, so the sender always transmits this list as
* protected prefixes (unlike excluded_paths, which --delete-excluded drops).
* Guarded by `excluded_mutex` like excluded_paths. */
ArrayList* size_skipped_paths;
/* Synchronized-directory sink (optional): when non-NULL the scanner appends
* the destination-relative path of every directory it is about to traverse
* that lies inside a --files-from listed directory (or of every traversed
* directory when there is no list). The sender sends this set with the delete
* manifest so the receiver confines its extras walk to synchronized
* directories, exactly like rsync; the receive root is the "." sentinel.
* Guarded by `excluded_mutex`. */
ArrayList* synced_dirs;
/* --ignore-errors: an unreadable directory during the scan is recorded as an
* I/O error and skipped instead of aborting the scan. Client-only. */
bool ignore_io_errors;
+13 -9
View File
@@ -164,7 +164,8 @@ void print_usage(void) {
printf(" --chunk-serialization Enable chunk serialization (long form only)\n");
printf(" -s, --secluded-args Protect-args compatibility option (no effect; remote\n");
printf(" SSH argv is already built injection-safe)\n");
printf(" --sendfile Enable sendfile zero-copy (TCP only; long form only)\n");
printf(" --sendfile Enable sendfile zero-copy (TCP only; long form only;\n");
printf(" -f is bound to --filter, not --sendfile)\n");
printf(" --compress-choice <alg> Compression algorithm (default: zstd)\n");
printf(" --zc <alg> Alias for --compress-choice\n");
printf(" -v, --verbose Enable debug logging\n");
@@ -197,17 +198,20 @@ void print_usage(void) {
printf(" within the confined receive root. Never elevates\n");
printf(" privileges and never bypasses confinement; ownership\n");
printf(" is still applied only with -o/--owner, -g/--group, or an\n");
printf(" explicit identity flag (--numeric-ids/--chown/--usermap/\n");
printf(" --groupmap/--copy-as)\n");
printf(" explicit identity flag (--chown/--usermap/--groupmap/\n");
printf(" --copy-as); --numeric-ids only changes how ids map\n");
printf(" --no-super Forbid those super-user activities even when the\n");
printf(" receiver is running as root\n");
printf(" --chmod <changes> Modify transferred permissions (rsync syntax)\n");
printf(" --numeric-ids Do not map uid/gid by name: use the source numeric\n");
printf(" ids directly when applying ownership\n");
printf(
" --chmod <changes> Modify new/transferred permissions (rsync syntax; implies no -p)\n");
printf(" --numeric-ids Map uid/gid by id instead of by name (a modifier, not\n");
printf(" an ownership request: combine with -o/-g or a map)\n");
printf(" --usermap=MAP Map usernames when applying ownership: comma-separated\n");
printf(" FROM:TO rules, first match wins. FROM/TO are names\n");
printf(" (resolved on the source machine), * (match any /\n");
printf(" current user), or @N numeric ids. e.g. *:nobody\n");
printf(" FROM:TO rules, first match wins. FROM is a name (from\n");
printf(" the source), an id, an inclusive LOW-HIGH range, *\n");
printf(" (any id), or empty (ids with no name). TO is an id, *\n");
printf(" (current user), or a name resolved on the receiver.\n");
printf(" e.g. 0-99:nobody,*:normal (cannot mix with --chown)\n");
printf(" --groupmap=MAP Map group names when applying ownership (same syntax)\n");
printf(" --chown=USER:GROUP Override the ownership of transferred files. Forms:\n");
printf(" USER:GROUP, USER (owner only), :GROUP (group only); a\n");
+33 -15
View File
@@ -43,17 +43,19 @@ void receiver_outcomes_destroy(ReceiverOutcomes* outcomes) {
/* End-of-transfer success frame. When --remove-source-files was negotiated
each processed data file is acknowledged first (STATUS_NEXT = written,
STATUS_OK = skipped) so the sender never removes a source the receiver did
not actually store. The frame always ends with a plain STATUS_OK. */
bool receiver_send_final_success(int fd, const Config* config, const ReceiverOutcomes* outcomes) {
not actually store. The frame ends with `final_status` (STATUS_OK, or
STATUS_DELETE_LIMIT when a --max-delete commit was capped). */
bool receiver_send_final_success(int fd, const Config* config, const ReceiverOutcomes* outcomes,
Status final_status) {
if (!config->remove_source_files)
return send_status(fd, STATUS_OK);
return send_status(fd, final_status);
size_t count = outcomes ? outcomes->count : 0;
for (size_t i = 0; i < count; i++) {
Status per_file = outcomes->entries[i] == FILE_SAVE_WRITTEN ? STATUS_NEXT : STATUS_OK;
if (!send_status(fd, per_file))
return false;
}
return send_status(fd, STATUS_OK);
return send_status(fd, final_status);
}
static bool receiver_process_chunk(Chunk* chunk, const ReceiverSink* sink) {
@@ -346,15 +348,19 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver
moment it arrives, before any file data. Delete now and acknowledge
so the sender only starts streaming once the deletion committed (or
failed). This is the rsync delete-before/delete-during window: a
later transfer failure does not restore these deletions. */
bool deletion_ok = (config->use_delete || config->delete_missing_args)
? manifest_delete_all(config, manifest)
: true;
later transfer failure does not restore these deletions. A
--max-delete-capped commit still succeeds and the transfer proceeds;
the terminal success frame reports the cap. */
DeleteCommitResult deletion = (config->use_delete || config->delete_missing_args)
? manifest_delete_all(config, manifest)
: DELETE_COMMIT_OK;
delete_manifest_free(manifest);
if (!deletion_ok) {
if (deletion == DELETE_COMMIT_ERROR) {
send_status(file_descriptor, STATUS_ERROR);
goto fail;
}
if (deletion == DELETE_COMMIT_LIMIT_REACHED && sink->note_delete_limit)
sink->note_delete_limit(sink->context);
if (!send_status(file_descriptor, STATUS_OK))
goto fail;
} else if (config->use_delete || config->delete_missing_args) {
@@ -407,13 +413,15 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver
*pending_manifest = deferred_manifest;
deferred_manifest = NULL;
} else {
bool deletion_ok = manifest_delete_all(config, deferred_manifest);
DeleteCommitResult deletion = manifest_delete_all(config, deferred_manifest);
delete_manifest_free(deferred_manifest);
deferred_manifest = NULL;
if (!deletion_ok) {
if (deletion == DELETE_COMMIT_ERROR) {
send_status(file_descriptor, STATUS_ERROR);
goto fail;
}
if (deletion == DELETE_COMMIT_LIMIT_REACHED && sink->note_delete_limit)
sink->note_delete_limit(sink->context);
}
}
if (sink->send_success) {
@@ -454,6 +462,9 @@ typedef struct {
after the whole transfer (and its delete/publication phases) has run so a
child write never clobbers a directory mtime. */
DirTimeList dir_times;
/* Set when a --max-delete commit was capped; the terminal frame then carries
STATUS_DELETE_LIMIT so the sender exits 25 like rsync. */
bool delete_limit_reached;
} ReceiverSaveContext;
static bool receiver_save_file(File* file, void* context_pointer) {
@@ -476,7 +487,7 @@ static bool receiver_save_file(File* file, void* context_pointer) {
receiver_send_success_frame). */
if (result != FILE_SAVE_ERROR && file->is_dir && file->metadata &&
dir_metadata_should_capture(context->config) &&
!dir_time_list_add(&context->dir_times, file->path, file->metadata)) {
!dir_time_list_add(&context->dir_times, file->path, file->metadata, file->xattrs)) {
file_destroy(file);
return false;
}
@@ -494,12 +505,18 @@ static bool receiver_save_file(File* file, void* context_pointer) {
return result != FILE_SAVE_ERROR;
}
static void receiver_note_delete_limit(void* context_pointer) {
ReceiverSaveContext* context = context_pointer;
context->delete_limit_reached = true;
}
static bool receiver_send_success_frame(int fd, void* context_pointer) {
ReceiverSaveContext* context = context_pointer;
Status final_status = context->delete_limit_reached ? STATUS_DELETE_LIMIT : STATUS_OK;
/* Server-contacting --dry-run: nothing was staged or written, so there is
nothing to publish and no directory times to stamp. */
if (context->config->dry_run)
return receiver_send_final_success(fd, context->config, &context->outcomes);
return receiver_send_final_success(fd, context->config, &context->outcomes, final_status);
/* --delay-updates: the whole protocol stream (including manifest/delete
handling, which ran inside receiver_process) has succeeded and every
staged file was fully written. Publish them atomically now, before the
@@ -517,13 +534,14 @@ static bool receiver_send_success_frame(int fd, void* context_pointer) {
before calling this success frame. */
dir_metadata_list_apply(&context->dir_times, context->config->receive_root_directory,
context->config);
return receiver_send_final_success(fd, context->config, &context->outcomes);
return receiver_send_final_success(fd, context->config, &context->outcomes, final_status);
}
int receiver_receive_files(Config* config, int file_descriptor) {
ReceiverSaveContext context = {.config = config, .outcomes = {0}};
dir_time_list_init(&context.dir_times);
ReceiverSink sink = {receiver_save_file, &context, true, true, receiver_send_success_frame};
ReceiverSink sink = {receiver_save_file, &context, true, true, receiver_send_success_frame,
receiver_note_delete_limit};
int ret = receiver_process(config, file_descriptor, &sink);
if (ret != 0 && config->delay_updates && config->delay_context)
delay_updates_cleanup(config->delay_context);
+14 -2
View File
@@ -4,6 +4,7 @@
#include "config.h"
#include "file.h"
#include "file_receive.h"
#include "protocol.h"
#include <stdbool.h>
#include <time.h>
@@ -21,6 +22,12 @@ typedef struct {
typedef bool (*ReceiverSuccessFrame)(int fd, void* context);
/* Records that a --max-delete commit stopped with extras left over, so the
caller's terminal success frame can carry STATUS_DELETE_LIMIT instead of
STATUS_OK. The commit runs on the receiver thread, so the flag is stored in
the sink's own context rather than in a shared global. */
typedef void (*ReceiverNoteDeleteLimit)(void* context);
typedef struct {
ReceiverFileSink store_file;
void* context;
@@ -28,13 +35,18 @@ typedef struct {
bool send_success;
/* Emits the end-of-transfer success frame. When the sender requested
--remove-source-files this includes one per-file status per processed
data file followed by the final STATUS_OK; otherwise just STATUS_OK. */
data file followed by the final status; otherwise just the final status. */
ReceiverSuccessFrame send_success_frame;
/* Optional; may be NULL when the sink has no --max-delete handling. */
ReceiverNoteDeleteLimit note_delete_limit;
} ReceiverSink;
bool receiver_outcomes_append(ReceiverOutcomes* outcomes, unsigned char code);
void receiver_outcomes_destroy(ReceiverOutcomes* outcomes);
bool receiver_send_final_success(int fd, const Config* config, const ReceiverOutcomes* outcomes);
/* Send the terminal success frame. `final_status` is usually STATUS_OK, or
STATUS_DELETE_LIMIT when a --max-delete commit was capped. */
bool receiver_send_final_success(int fd, const Config* config, const ReceiverOutcomes* outcomes,
Status final_status);
int receiver_process(Config* config, int file_descriptor, const ReceiverSink* sink);
/* receiver_process with an escape hatch for the commit-style (late) deletion:
+13 -2
View File
@@ -27,6 +27,7 @@ PipelineContextReceiver* pipeline_context_receiver_create(Config* config, Queue*
context->queued_bytes = 0;
context->max_queue_bytes = 0;
context->deferred_manifest = NULL;
context->delete_limit_reached = false;
atomic_init(&context->cancelled, false);
int init = 0;
if (mtx_init(&context->mutex, mtx_plain) != thrd_success)
@@ -135,6 +136,15 @@ static bool receiver_enqueue_file(File* file, void* context_pointer) {
return pipeline_context_receiver_enqueue_file(context, file);
}
/* Early delete modes (--delete-before/--delete-during) commit the manifest
inside receiver_process_pending on this thread; record a capped commit so
server.c's terminal frame can report STATUS_DELETE_LIMIT. The plain bool is
safe: receive_thread writes it before the main thread joins the thread. */
static void receiver_pipeline_note_delete_limit(void* context_pointer) {
PipelineContextReceiver* context = (PipelineContextReceiver*)context_pointer;
context->delete_limit_reached = true;
}
static void receiver_thread_fail(PipelineContextReceiver* context) {
mtx_lock(&context->mutex);
atomic_store(&context->cancelled, true);
@@ -152,7 +162,8 @@ int receive_thread(void* pipeline_context) {
const Config* config = context->config;
mtx_unlock(&context->mutex);
ReceiverSink sink = {receiver_enqueue_file, context, false, false, NULL};
ReceiverSink sink = {
receiver_enqueue_file, context, false, false, NULL, receiver_pipeline_note_delete_limit};
if (receiver_process_pending((Config*)config, file_descriptor, &sink,
&context->deferred_manifest) != 0) {
receiver_thread_fail(context);
@@ -221,7 +232,7 @@ int write_thread(void* pipeline_context) {
caller apply it once every writer has drained. */
if (!dry_run && result != FILE_SAVE_ERROR && file->is_dir && file->metadata &&
dir_metadata_should_capture(context->config) &&
!dir_time_list_add(&context->dir_times, file->path, file->metadata)) {
!dir_time_list_add(&context->dir_times, file->path, file->metadata, file->xattrs)) {
file_destroy(file);
pipeline_context_receiver_note_bytes_released(context, file_bytes);
mtx_lock(&context->mutex);
+4
View File
@@ -41,6 +41,10 @@ typedef struct PipelineContextReceiver {
transfer truly succeeded. NULL in the early delete modes (which delete at
the manifest). */
DeleteManifest* deferred_manifest;
/* Set by server.c when the deferred delete commit hit the --max-delete
budget; the terminal success frame then carries STATUS_DELETE_LIMIT
(rsync exit 25) while the transfer itself still succeeds. */
bool delete_limit_reached;
/* P7 Wave D: directory metadata collected by write_thread from received
directory entries. Only write_thread mutates it (before it joins); the
caller (server.c) applies it after the delete/delay-updates phase. */
+19 -7
View File
@@ -752,10 +752,11 @@ void handler(int file_descriptor) {
protocol_set_8_bit_output(config->eight_bit_output);
/* Server-side per-message protocol deadline for every frame from here on.
* `timeout` is not serialized, so this is the server's own config (the server
* has no --timeout CLI and defaults it to 0): the built-in 60 s window stays
* in effect. A client's --timeout tightens only that client's own protocol
* I/O and the server's socket read/write timeout is the transport default. */
protocol_session_set_io_timeout(&session, config->timeout);
* has no --timeout CLI and defaults it to 0). A client's --timeout tightens
* only that client's own protocol I/O; the server floors its own deadline at
* SERVER_IO_TIMEOUT_SEC so a silent peer can never hold a session slot
* forever (the socket layer gets the same floor at startup). */
protocol_session_set_io_timeout(&session, protocol_server_io_timeout_sec(config->timeout));
const char* authorized_root = utils_get_authorized_root_path();
if (!authorized_root) {
log_message(LOG_LEVEL_ERROR, "No server-side destination root configured");
@@ -904,7 +905,8 @@ void handler(int file_descriptor) {
goto done;
}
protocol_session_set_max_alloc(&context->session, config->max_alloc);
protocol_session_set_io_timeout(&context->session, config->timeout);
protocol_session_set_io_timeout(&context->session,
protocol_server_io_timeout_sec(config->timeout));
atomic_store(&context->session.total_allocated_bytes,
atomic_load(&session.total_allocated_bytes));
pipeline_context_receiver_set_queue_byte_limit(context, RECEIVER_QUEUE_MAX_BYTES);
@@ -949,8 +951,13 @@ void handler(int file_descriptor) {
--delay-updates run; the walker skips the staging directory. A
server-contacting --dry-run deletes nothing (no manifest is sent). */
if (context->deferred_manifest) {
if (!manifest_delete_all(config, context->deferred_manifest)) {
DeleteCommitResult deletion = manifest_delete_all(config, context->deferred_manifest);
if (deletion == DELETE_COMMIT_ERROR) {
transfer_ok = false;
} else if (deletion == DELETE_COMMIT_LIMIT_REACHED) {
/* The transfer still succeeds; the terminal frame reports the capped
deletion so the sender exits 25 like rsync. */
context->delete_limit_reached = true;
}
delete_manifest_free(context->deferred_manifest);
context->deferred_manifest = NULL;
@@ -974,7 +981,8 @@ void handler(int file_descriptor) {
dir_metadata_list_apply(&context->dir_times, config->receive_root_directory, config);
}
if (transfer_ok) {
if (!receiver_send_final_success(file_descriptor, config, &context->outcomes))
Status final_status = context->delete_limit_reached ? STATUS_DELETE_LIMIT : STATUS_OK;
if (!receiver_send_final_success(file_descriptor, config, &context->outcomes, final_status))
transfer_ok = false;
} else {
send_error_detail(file_descriptor, "transfer failed on receiver");
@@ -1214,6 +1222,10 @@ int main(int argc, char* argv[]) {
server_iconv_spec = opts.iconv_spec;
signal(SIGINT, cleanup);
signal(SIGTERM, cleanup);
/* Server-owned socket deadline floor: the client default --timeout=0 would
* otherwise leave accepted sockets without SO_RCVTIMEO/SO_SNDTIMEO and let a
* silent peer hold a connection (and its process slot) forever. */
tcp_set_timeouts(SERVER_IO_TIMEOUT_SEC, SERVER_IO_TIMEOUT_SEC);
if (opts.stdio_mode) {
/* SSH authenticates the stdio transport outside of FastSync. */
+1 -1
View File
@@ -172,7 +172,7 @@ int batch_read_apply(int fd, const Config* config, const char* dest_root) {
* destroyed; applied once the whole stream has been consumed. */
if (save != FILE_SAVE_ERROR && file->is_dir && file->metadata &&
dir_metadata_should_capture(config) &&
!dir_time_list_add(&dir_times, file->path, file->metadata)) {
!dir_time_list_add(&dir_times, file->path, file->metadata, file->xattrs)) {
file_destroy(file);
chunk_destroy(chunk);
goto done;
+170 -77
View File
@@ -1,90 +1,183 @@
#include "chmod.h"
#include "file.h"
#include <stddef.h>
#include <string.h>
static bool parse_clause(mode_t* mode, const char* begin, const char* end) {
const char* p = begin;
unsigned who = 0;
while (p < end && strchr("ugoa", *p)) {
if (*p == 'a')
who = 7;
else
who |= *p == 'u' ? 1U : (*p == 'g' ? 2U : 4U);
p++;
}
if (who == 0)
who = 7;
if (p == end || (*p != '+' && *p != '-' && *p != '='))
return false;
char operation = *p++;
mode_t bits = 0;
while (p < end) {
mode_t bit;
switch (*p++) {
case 'r':
bit = 4;
break;
case 'w':
bit = 2;
break;
case 'x':
bit = 1;
break;
default:
return false;
}
bits |= bit;
}
for (unsigned class_index = 0; class_index < 3; class_index++) {
unsigned class_bit = 1U << class_index;
if (!(who & class_bit))
continue;
mode_t shift = (mode_t)((2U - class_index) * 3U);
mode_t mask = (mode_t)(7U << shift);
mode_t class_bits = (mode_t)(bits << shift);
if (operation == '+')
*mode |= class_bits;
else if (operation == '-')
*mode &= ~class_bits;
else
*mode = (*mode & ~mask) | class_bits;
}
return true;
}
/* rsync's --chmod parser (parse_chmod + tweak_mode). A single clause is
* applied as it is completed, so repeated clauses and repeated --chmod options
* (joined with commas by the CLI) accumulate exactly like rsync. The D/F
* selectors restrict a clause to directories/files; X adds execute only to
* directories or files that were already executable. */
#define CHMOD_BITS 07777
#define CHMOD_FLAG_X_KEEP (1U << 0)
#define CHMOD_FLAG_DIRS_ONLY (1U << 1)
#define CHMOD_FLAG_FILES_ONLY (1U << 2)
enum chmod_op { CHMOD_OP_ADD = 1, CHMOD_OP_SUB, CHMOD_OP_EQ, CHMOD_OP_SET };
enum chmod_state {
CHMOD_STATE_ERROR,
CHMOD_STATE_1ST_HALF,
CHMOD_STATE_2ND_HALF,
CHMOD_STATE_OCTAL
};
bool chmod_apply(mode_t mode, const char* spec, mode_t* result) {
if (!spec || !*spec || !result)
return false;
bool numeric = true;
size_t length = strlen(spec);
if (length > 4)
numeric = false;
for (size_t i = 0; i < length && numeric; i++)
numeric = spec[i] >= '0' && spec[i] <= '7';
if (numeric) {
if (length == 0 || length > 4)
return false;
mode_t parsed = 0;
for (size_t i = 0; i < length; i++)
parsed = (mode_t)((parsed << 3) | (spec[i] - '0'));
*result = parsed;
return true;
}
const mode_t nonperm = mode & ~(mode_t)CHMOD_BITS;
const bool initially_executable = (mode & 0111) != 0;
mode_t changed = mode;
const char* begin = spec;
while (*begin) {
const char* end = strchr(begin, ',');
if (!end)
end = begin + strlen(begin);
if (!parse_clause(&changed, begin, end))
return false;
if (*end == '\0')
int state = CHMOD_STATE_1ST_HALF;
unsigned where = 0;
int what = 0, op = 0, topbits = 0, topoct = 0, flags = 0;
const char* p = spec;
while (state != CHMOD_STATE_ERROR) {
if (*p == '\0' || *p == ',') {
int bits;
if (!op) {
state = CHMOD_STATE_ERROR;
break;
}
if (where)
bits = (int)(where * (unsigned)what);
else {
where = 0111;
bits = (int)((where * (unsigned)what) & ~(unsigned)file_process_umask());
}
int mode_and, mode_or;
switch (op) {
case CHMOD_OP_ADD:
mode_and = CHMOD_BITS;
mode_or = bits + topoct;
break;
case CHMOD_OP_SUB:
mode_and = CHMOD_BITS - bits - topoct;
mode_or = 0;
break;
case CHMOD_OP_EQ:
mode_and = CHMOD_BITS - (int)(where * 7U) - (topoct ? topbits : 0);
mode_or = bits + topoct;
break;
default:
mode_and = 0;
mode_or = bits;
break;
}
bool is_dir = S_ISDIR(nonperm);
if (!((flags & CHMOD_FLAG_DIRS_ONLY) && !is_dir) &&
!((flags & CHMOD_FLAG_FILES_ONLY) && is_dir)) {
changed &= (mode_t)mode_and;
if ((flags & CHMOD_FLAG_X_KEEP) && !initially_executable && !is_dir)
changed |= (mode_t)(mode_or & ~0111);
else
changed |= (mode_t)mode_or;
}
if (*p == '\0')
break;
p++;
state = CHMOD_STATE_1ST_HALF;
where = 0;
what = op = topoct = topbits = flags = 0;
continue;
}
switch (state) {
case CHMOD_STATE_1ST_HALF:
switch (*p) {
case 'D':
if (flags & CHMOD_FLAG_FILES_ONLY) {
state = CHMOD_STATE_ERROR;
break;
}
flags |= CHMOD_FLAG_DIRS_ONLY;
break;
case 'F':
if (flags & CHMOD_FLAG_DIRS_ONLY) {
state = CHMOD_STATE_ERROR;
break;
}
flags |= CHMOD_FLAG_FILES_ONLY;
break;
case 'u':
where |= 0100;
topbits |= 04000;
break;
case 'g':
where |= 0010;
topbits |= 02000;
break;
case 'o':
where |= 0001;
break;
case 'a':
where |= 0111;
break;
case '+':
op = CHMOD_OP_ADD;
state = CHMOD_STATE_2ND_HALF;
break;
case '-':
op = CHMOD_OP_SUB;
state = CHMOD_STATE_2ND_HALF;
break;
case '=':
op = CHMOD_OP_EQ;
state = CHMOD_STATE_2ND_HALF;
break;
default:
if (*p >= '0' && *p <= '7' && !where) {
op = CHMOD_OP_SET;
state = CHMOD_STATE_OCTAL;
where = 1;
what = *p - '0';
} else {
state = CHMOD_STATE_ERROR;
}
break;
}
break;
begin = end + 1;
if (!*begin)
return false;
case CHMOD_STATE_2ND_HALF:
switch (*p) {
case 'r':
what |= 4;
break;
case 'w':
what |= 2;
break;
case 'X':
flags |= CHMOD_FLAG_X_KEEP;
/* fall through */
case 'x':
what |= 1;
break;
case 's':
if (topbits)
topoct |= topbits;
else
topoct = 04000;
break;
case 't':
topoct |= 01000;
break;
default:
state = CHMOD_STATE_ERROR;
break;
}
break;
default:
if (*p >= '0' && *p <= '7') {
what = what * 8 + (*p - '0');
if (what > CHMOD_BITS)
state = CHMOD_STATE_ERROR;
} else {
state = CHMOD_STATE_ERROR;
}
break;
}
p++;
}
*result = changed;
if (state == CHMOD_STATE_ERROR)
return false;
*result = (changed & (mode_t)CHMOD_BITS) | nonperm;
return true;
}
+4 -1
View File
@@ -4,7 +4,10 @@
#include <stdbool.h>
#include <sys/stat.h>
/* Apply the supported rsync --chmod syntax to a permission mode. */
/* Apply rsync's --chmod syntax to a permission mode, including the D/F/X
* selectors and the s/t special bits. `mode` should carry the file type bits
* (S_IFDIR/S_IFREG) so D/F/X can be evaluated; the type bits are preserved in
* `result`. A spec may contain comma-separated clauses, which accumulate. */
bool chmod_apply(mode_t mode, const char* spec, mode_t* result);
#endif
+56 -17
View File
@@ -46,9 +46,11 @@ static void config_set_defaults(Config* config) {
config->server_port_set = false;
config->server_host_set = false;
/* rsync defaults: --timeout=0 (I/O timeouts disabled) and --contimeout=60.
* A value of 0 disables the deadline on both the socket layer
* A value of 0 disables the client's own deadline on both the socket layer
* (tcp_set_timeouts) and the protocol layer
* (protocol_session_set_io_timeout); a positive value sets it. */
* (protocol_session_set_io_timeout); a positive value sets it. A server
* session floors the deadline at SERVER_IO_TIMEOUT_SEC so 0 can never hold a
* connection open forever. */
config->timeout = 0;
config->contimeout = 60;
config->quiet = false;
@@ -207,7 +209,8 @@ static bool validate_received_config(const Config* config) {
config->delta_block_size >= DELTA_BLOCK_SIZE_MIN &&
config->delta_block_size <= DELTA_BLOCK_SIZE_MAX &&
config->delta_max_file_size <= DELTA_MAX_FILE_SIZE && config->modify_window >= 0 &&
config->max_delete >= -1 && config->skip_compress_count >= 0 &&
config->max_delete >= -1 && config->max_alloc <= MAX_SERVER_ALLOC &&
config->skip_compress_count >= 0 &&
config->skip_compress_count <= MAX_SKIP_COMPRESS_SUFFIXES &&
(!config->chmod_spec || !*config->chmod_spec ||
chmod_apply(0, config->chmod_spec, &(mode_t){0})) &&
@@ -749,10 +752,18 @@ void config_delete(Config* config) {
free(config->skip_compress_suffixes[i]);
free(config->skip_compress_suffixes);
}
free(config->usermap);
if (config->usermap) {
for (int i = 0; i < config->usermap_count; i++)
free(config->usermap[i].to_name);
free(config->usermap);
}
config->usermap = NULL;
config->usermap_count = 0;
free(config->groupmap);
if (config->groupmap) {
for (int i = 0; i < config->groupmap_count; i++)
free(config->groupmap[i].to_name);
free(config->groupmap);
}
config->groupmap = NULL;
config->groupmap_count = 0;
if (config->filters) {
@@ -780,13 +791,14 @@ void config_delete(Config* config) {
* ------------------------------------------------------------------------- */
/* --max-alloc: raw 64-bit value, clamped server-side and installed as the
* session allocation ceiling. Zero means "no alloc limit" (rsync's
* --max-alloc=0) and is passed through; a non-zero value is clamped to the
* server's own ceiling. */
* session allocation ceiling. A received 0 is rsync's "no alloc limit"; on the
* receive path it is mapped to the server ceiling so a client can never disable
* it (client-side 0 remains unlimited). Any value above the ceiling is clamped
* to it. */
static bool config_receive_max_alloc(int fd, unsigned long long* value) {
if (!receive_n_data(fd, value, sizeof(*value)))
return false;
if (*value > MAX_SERVER_ALLOC)
if (*value == 0 || *value > MAX_SERVER_ALLOC)
*value = MAX_SERVER_ALLOC;
protocol_session_set_max_alloc(NULL, *value);
return true;
@@ -986,7 +998,8 @@ static bool receive_basis_entries(int fd, Config* c, ConfigStringBudget* budget)
static bool send_identity_entries(int fd, const IdentityMap* map, int count) {
for (int i = 0; i < count; i++) {
if (!send_int(fd, map[i].from) || !send_int(fd, map[i].to))
if (!send_int(fd, map[i].from) || !send_int(fd, map[i].from_hi) || !send_int(fd, map[i].to) ||
!send_str(fd, map[i].to_name ? map[i].to_name : ""))
return false;
}
return true;
@@ -994,20 +1007,32 @@ static bool send_identity_entries(int fd, const IdentityMap* map, int count) {
static bool receive_identity_entries(int fd, ConfigStringBudget* budget, int count,
IdentityMap** out) {
(void)budget;
if (count <= 0)
return true;
IdentityMap* map = calloc((size_t)count, sizeof(IdentityMap));
if (!map)
return false;
for (int i = 0; i < count; i++) {
if (!receive_int(fd, &map[i].from) || !receive_int(fd, &map[i].to)) {
free(map);
return false;
if (!receive_int(fd, &map[i].from) || !receive_int(fd, &map[i].from_hi) ||
!receive_int(fd, &map[i].to))
goto fail;
char* name = config_receive_str(fd, budget);
if (!name)
goto fail;
if (name[0] == '\0') {
free(name);
map[i].to_name = NULL;
} else {
map[i].to_name = name;
}
}
*out = map;
return true;
fail:
for (int i = 0; i < count; i++)
free(map[i].to_name);
free(map);
return false;
}
/* ---------------------------------------------------------------------------
@@ -1130,6 +1155,7 @@ CONFIG_DEFINE_SEND(send_daemon_auth, CONFIG_WIRE_DAEMON_AUTH_FIELDS)
CONFIG_DEFINE_SEND(send_iconv_spec, CONFIG_WIRE_ICONV_FIELDS)
CONFIG_DEFINE_SEND(send_privilege_options, CONFIG_WIRE_PRIVILEGE_FIELDS)
CONFIG_DEFINE_SEND(send_copy_as_options, CONFIG_WIRE_COPY_AS_FIELDS)
CONFIG_DEFINE_SEND(send_output_options, CONFIG_WIRE_OUTPUT_FIELDS)
CONFIG_DEFINE_RECV(receive_core_fields, CONFIG_WIRE_CORE_FIELDS)
CONFIG_DEFINE_RECV(receive_delta_fields, CONFIG_WIRE_DELTA_FIELDS)
@@ -1148,6 +1174,7 @@ CONFIG_DEFINE_RECV(receive_daemon_auth, CONFIG_WIRE_DAEMON_AUTH_FIELDS)
CONFIG_DEFINE_RECV(receive_iconv_spec, CONFIG_WIRE_ICONV_FIELDS)
CONFIG_DEFINE_RECV(receive_privilege_options, CONFIG_WIRE_PRIVILEGE_FIELDS)
CONFIG_DEFINE_RECV(receive_copy_as_options, CONFIG_WIRE_COPY_AS_FIELDS)
CONFIG_DEFINE_RECV(receive_output_options, CONFIG_WIRE_OUTPUT_FIELDS)
#undef XSEND
#undef XRECV
@@ -1264,7 +1291,8 @@ bool config_send_wire_block(int file_descriptor, const Config* config) {
send_daemon_module(file_descriptor, config) && send_daemon_auth(file_descriptor, config) &&
send_iconv_spec(file_descriptor, config) &&
send_privilege_options(file_descriptor, config) &&
send_copy_as_options(file_descriptor, config);
send_copy_as_options(file_descriptor, config) &&
send_output_options(file_descriptor, config);
}
bool config_send(int file_descriptor, const Config* config) {
@@ -1334,10 +1362,12 @@ Config* config_receive_with_validate(int file_descriptor, ConfigValidateFunc val
!receive_daemon_auth(file_descriptor, config, &budget) ||
!receive_iconv_spec(file_descriptor, config, &budget) ||
!receive_privilege_options(file_descriptor, config, &budget) ||
!receive_copy_as_options(file_descriptor, config, &budget))
!receive_copy_as_options(file_descriptor, config, &budget) ||
!receive_output_options(file_descriptor, config, &budget))
goto error;
if (config->compress_choice[0] != '\0' && strcmp(config->compress_choice, "zstd") != 0 &&
strcmp(config->compress_choice, "none") != 0) {
strcmp(config->compress_choice, "none") != 0 &&
strcmp(config->compress_choice, "auto") != 0) {
char* escaped_choice = output_escape(config->compress_choice, config->eight_bit_output);
log_message(LOG_LEVEL_ERROR, "Unsupported compression choice: %s",
escaped_choice ? escaped_choice : "<allocation failed>");
@@ -1348,6 +1378,15 @@ Config* config_receive_with_validate(int file_descriptor, ConfigValidateFunc val
free(escaped_choice);
goto error;
}
/* Defensive: an older/hostile client may still send "auto"; canonicalize it
to zstd (its effective choice) so the stored value is always concrete. */
if (strcmp(config->compress_choice, "auto") == 0) {
char* canonical = str_dup("zstd");
if (!canonical)
goto error;
free(config->compress_choice);
config->compress_choice = canonical;
}
if (!validate_received_config(config)) {
log_message(LOG_LEVEL_ERROR, "Invalid configuration received from client");
send_error_detail(file_descriptor, "invalid configuration received from client");
+82 -19
View File
@@ -39,15 +39,20 @@ typedef struct BasisDest {
char* path; /* relative to the destination root (receiver-confined) */
} BasisDest;
/* One resolved FROM:TO identity-mapping rule (--usermap / --groupmap). Both
* fields are numeric ids. IDENTITY_MATCH_ANY (-1) in `from` is rsync's '*'
* wildcard (matches any transmitted id); IDENTITY_CURRENT (-1) in `to` makes
* the receiver resolve the receiving process's own current euid/egid at apply
* time. Names are resolved to numbers at parse time on the client (see
* identity.h for the exact subset). */
/* One FROM:TO identity-mapping rule (--usermap / --groupmap). `from`/`from_hi`
* describe the sender-side FROM matcher (a single id when from_hi == from, an
* inclusive LOW-HIGH range, IDENTITY_MATCH_ANY for rsync's '*', or
* IDENTITY_MATCH_UNNAMED for rsync's empty FROM). `to` is the receiver-side TO
* numeric id (IDENTITY_CURRENT = the receiving process's own euid/egid) UNLESS
* `to_name` is non-NULL, in which case the receiver resolves the name against
* its own account database at apply time (rsync resolves TO names on the
* receiver) and `to` is ignored. FROM names/ranges/globs are resolved on the
* client (the sender) exactly as rsync matches them against sender names. */
typedef struct {
int32_t from;
int32_t from_hi;
int32_t to;
char* to_name;
} IdentityMap;
/* --sockopts=OPTIONS allowlist. Only these option names are accepted; anything
@@ -76,7 +81,7 @@ typedef struct {
typedef enum SuperMode { SUPER_MODE_AUTO = 0, SUPER_MODE_ON = 1, SUPER_MODE_OFF = 2 } SuperMode;
/* ===========================================================================
* Config wire-field table (single source of truth for protocol 2.22.0).
* Config wire-field table (single source of truth for protocol 2.23.0).
*
* Every field below crosses the wire. The table is the ONLY place a
* serialized field is named: config.h expands CONFIG_WIRE_FIELDS() to declare
@@ -241,6 +246,13 @@ typedef enum SuperMode { SUPER_MODE_AUTO = 0, SUPER_MODE_ON = 1, SUPER_MODE_OFF
X(copy_as_uid, int32_t, 0, COPY_AS_ID) \
X(copy_as_gid, int32_t, 0, COPY_AS_ID)
/* Output-parity wave (protocol 2.23.0). report_dest_info tells the receiver to
* answer every per-file STATUS_CHECK with a STATUS_DEST_INFO snapshot of the
* pre-transfer destination entry (see protocol.h). It is set by the client
* only when -i/--itemize-changes or --out-format asks for per-file change
* output; the transfer decision itself is unchanged. */
#define CONFIG_WIRE_OUTPUT_FIELDS(X) X(report_dest_info, bool, false, BOOL)
/* All serialized fields, in exact wire order. Concatenating the per-segment
* lists here is what keeps the declaration order = the wire order. */
#define CONFIG_WIRE_FIELDS(X) \
@@ -261,7 +273,8 @@ typedef enum SuperMode { SUPER_MODE_AUTO = 0, SUPER_MODE_ON = 1, SUPER_MODE_OFF
CONFIG_WIRE_DAEMON_AUTH_FIELDS(X) \
CONFIG_WIRE_ICONV_FIELDS(X) \
CONFIG_WIRE_PRIVILEGE_FIELDS(X) \
CONFIG_WIRE_COPY_AS_FIELDS(X)
CONFIG_WIRE_COPY_AS_FIELDS(X) \
CONFIG_WIRE_OUTPUT_FIELDS(X)
typedef struct Config {
/* -j/--threads=N: number of parallel scanner worker threads for the -m
@@ -315,9 +328,10 @@ typedef struct Config {
char* tls_key;
char* tls_ca;
/* --timeout: per-message I/O deadline in seconds. 0 (rsync's default)
* disables the deadline entirely on both the socket layer and the protocol
* layer; a positive value sets it. See protocol_session_set_io_timeout and
* tcp_set_timeouts. */
* disables the deadline entirely on the client's own socket and protocol
* layers; a positive value sets it. A server session never inherits the
* disabled value: it applies the SERVER_IO_TIMEOUT_SEC floor (see
* protocol_server_io_timeout_sec and tcp_set_timeouts). */
int timeout;
/* --contimeout: connect()/accept timeout in seconds (rsync's default 60);
* 0 disables it. Transport layer only. */
@@ -577,13 +591,17 @@ typedef struct Config {
* targets and, with -K, follows an in-root destination symlink-to-directory);
* -k/--copy-dirlinks is sender-only and is never serialized. */
/* numeric_ids */
/* --numeric-ids: no name lookup, use the transmitted numeric ids raw. */
/* --numeric-ids: a mapping MODIFIER only -- no name lookup, use the
* transmitted numeric ids raw. It does NOT by itself request ownership. */
/* chown_uid_set */
/* --chown USER (owner) override; IDENTITY_CURRENT = the receiver's euid. */
/* chown_gid_set */
/* --chown :GROUP (group) override; IDENTITY_CURRENT = the receiver's egid. */
/* usermap */
/* --usermap / --groupmap entries, in order (first match wins). */
/* --usermap / --groupmap entries, in order (first match wins). Each entry's
* from/from_hi are a single id, an inclusive range, IDENTITY_MATCH_ANY ('*'),
* or IDENTITY_MATCH_UNNAMED (empty FROM); to_name carries a receiver-resolved
* TO name (rsync resolves TO names on the receiving side). */
/* preserve_atimes */
/* -U/--atimes: preserve source access times on the destination. */
/* preserve_crtimes */
@@ -611,8 +629,11 @@ typedef struct Config {
* --copy-as) imply it. */
/* fake_super */
/* --fake-super: receiver-only. When set, each written file additionally gets
* a reserved user.fastsync.stat xattr recording the source uid/gid/mode/mtime
* so a later privileged restore could re-apply them. Crosses the wire. */
* a reserved user.fastsync.stat xattr recording the RESOLVED uid/gid (the
* source's own when no ownership request is active, else the --chown/--usermap
* result) plus mode/mtime so a later privileged restore could re-apply them.
* It NEVER real-chowns: the point is to record the source ownership on an
* unprivileged receiver. Crosses the wire. */
/* module */
/* Daemon module selection (Wave A, protocol 2.15.0). Client-composed from a
* host::module/path destination; NULL or "" means "no module" (the ordinary
@@ -850,8 +871,48 @@ typedef struct Config {
* version before parsing anything else) is what keeps a 2.22 client and a 2.21
* server from ever reaching that state. The fixed-width FileMetadata layout is
* UNCHANGED: the receiver still gates attribute application on use_metadata,
* which is now DERIVED from these attributes by config_derived_use_metadata(). */
#define PROTOCOL_VERSION "2.22.0"
* which is now DERIVED from these attributes by config_derived_use_metadata().
*
* Rsync-Parity Wave: 2.22.0 -> 2.23.0.
*
* WHY the bump, grounded in the wire. Several independent changes land in this
* protocol version:
*
* (1) Ownership parity (#286/#294): each --usermap/--groupmap wire entry grows
* from two int32s to [from][from_hi][to][to_name]; `from_hi` carries an
* inclusive LOW-HIGH range (== from for a single/any/unnamed matcher) and the
* trailing string carries a TO NAME for the receiver to resolve (rsync resolves
* TO names on the receiving side). The STATUS_MKDIR and STATUS_DIR_TIMES frames
* also gain a bounded per-entry xattr block when -X/-A is negotiated, so
* directory xattrs/ACLs (including default ACLs) are preserved like regular-file
* xattrs.
*
* (2) Delete semantics (#290): the delete-manifest frame gains a fourth trailing
* section -- a synchronized-directory count followed by that many
* destination-relative directory paths (the receive root is "."). The receiver
* confines its extras walk to these directories, so `--files-from` with
* `--delete` only removes inside listed directory subtrees (rsync parity)
* instead of deleting every untransmitted path under the receive root. The
* frame stream also gains STATUS_DELETE_LIMIT, the terminal success status sent
* instead of STATUS_OK when a --max-delete commit removes up to the bound and
* skips the rest (the sender then exits 25 like rsync).
*
* Any config-frame layout or frame-sequence change must bump the protocol
* version: a 2.22 peer would desynchronize on the new entry bytes, the extra
* trailing section or the unknown status, and the strict same-version handshake
* (config_receive rejects a mismatched version before parsing anything else) is
* what keeps a 2.23 client and a 2.22 server from ever reaching that state.
*
* (3) Output parity (#291/#292): -i/--itemize-changes and --out-format must
* compare the source against the PRE-TRANSFER destination entry (new vs
* modified, and which of size/time/perms/owner/group differ), but FastSync's
* push sender never sees the destination. The receiver therefore answers a
* per-file STATUS_CHECK with a new STATUS_DEST_INFO frame (a fixed-width
* snapshot of the old entry) before its ordinary verdict when the config frame
* carries the new report_dest_info bool appended after the --copy-as block.
* This is both a config-frame layout change (one trailing bool) and a frame
* sequence change (the new status). */
#define PROTOCOL_VERSION "2.23.0"
#define DEFAULT_CHUNK_SIZE (10 * 1024 * 1024)
/* Upper bound on total basis-dir entries (rsync caps --link-dest at 20). */
#define MAX_BASIS_DIRS 64
@@ -876,9 +937,11 @@ typedef struct Config {
/* Identity-mapping sentinels and bounds (see identity.h for semantics).
* IDENTITY_MATCH_ANY is a usermap/groupmap FROM '*' (matches any id);
* IDENTITY_CURRENT is a chown / map TO '*' (resolve to the receiver's current
* euid/egid at apply time). */
* IDENTITY_MATCH_UNNAMED is a FROM with an empty token (rsync's "ids with no
* name on the sender"); IDENTITY_CURRENT is a chown / map TO '*' (resolve to
* the receiver's current euid/egid at apply time). */
#define IDENTITY_MATCH_ANY (-1)
#define IDENTITY_MATCH_UNNAMED (-2)
#define IDENTITY_CURRENT (-1)
#define MAX_IDENTITY_MAP 128
+14
View File
@@ -264,6 +264,20 @@ static bool delay_publish_entry(DelayUpdatesContext* context, const Config* conf
const StagedFileEntry* entry) {
if (!delay_publish_backup(context, config, entry))
return false;
/* --force: an incoming regular file/symlink may replace a destination
DIRECTORY (possibly non-empty). The immediate-install path handles this in
file_receive; a --delay-updates run stages elsewhere and only discovers the
blocking directory here, so clear it before the rename (rsync's
"could not make way for new regular file" without --force). */
if (config && config->force_delete && file_directory_exists_secure(entry->final_path)) {
if (!file_remove_tree_secure(entry->final_path)) {
char* escaped = output_escape(entry->final_path, false);
log_message(LOG_LEVEL_ERROR, "could not remove destination directory blocking '%s': %s",
escaped ? escaped : "<allocation failed>", strerror(errno));
free(escaped);
return false;
}
}
if (!file_rename_secure(entry->staged_path, entry->final_path)) {
if (errno == EXDEV) {
char* escaped = output_escape(entry->final_path, false);
+18 -17
View File
@@ -112,21 +112,16 @@ unsigned file_process_umask(void) {
/* Base mode applied when the policy does not take the source mode wholesale
* (i.e. --perms is off). A pre-existing destination keeps its own mode; a
* brand-new file is created like rsync: source_mode & 0777 & ~umask, with
* S_IWGRP|S_IWOTH always cleared so a client mode can never grant group/other
* write (the daemon runs with umask(0)). Only when no metadata is available at
* all does the historical fixed 0644 default apply. The -E rule (and no-op for
* a plain -t) is layered on top of this base. */
* brand-new file is created like rsync: source_mode & 0777 & ~umask (special
* bits are not part of a mode-preserving transfer without -p). Only when no
* metadata is available at all does the historical fixed 0644 default apply.
* The -E rule (and no-op for a plain -t) is layered on top of this base. */
static mode_t file_mode_base(const FileMetadata* metadata, bool existing_known,
mode_t existing_mode) {
if (existing_known)
return existing_mode;
if (metadata)
/* A brand-new file follows rsync's source_mode & ~umask base, but a
* client-supplied source mode must never grant group/other write (the
* daemon runs with umask(0), so an unmasked 0666 would otherwise create a
* world-writable file). S_IWGRP|S_IWOTH are always cleared. */
return metadata->mode & 0777 & ~(mode_t)file_process_umask() & ~(S_IWGRP | S_IWOTH);
return metadata->mode & 0777 & ~(mode_t)file_process_umask();
return S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH;
}
@@ -182,6 +177,7 @@ File* file_create(const char* path) {
file->rdev_major = 0;
file->rdev_minor = 0;
file->xattrs = NULL;
file->dest_state = (OutputDestState){0};
return file;
}
@@ -986,13 +982,18 @@ static void restore_extra_fd(int fd, const FileMetadata* metadata, const FileXat
bool fake_super, FileAttrPolicy policy) {
xattr_apply_fd(fd, xattrs);
if (fake_super && metadata) {
fake_super_store_fd(fd, (uint32_t)metadata->uid, (uint32_t)metadata->gid,
(uint32_t)metadata->mode, metadata->mtime_sec, metadata->mtime_nsec);
/* Replay: re-apply the recorded uid/gid/mode/mtime fd-relative so a save
under --fake-super restores the attrs (when privileged) instead of only
recording them. Best-effort; fake_super_restore_fd silently skips a
non-root fchown EPERM/EACCES and never fatal. The replayed mode/mtime
honor the per-attribute policy so fake-super cannot bypass the split. */
/* Record the ownership that WOULD have been applied: when an explicit
ownership request (--chown/--usermap/--groupmap/--copy-as or -o/-g) is
active, the resolved mapping; otherwise the source's own id. The real
chown is suppressed (identity_apply_ownership early-returns under
--fake-super) so recording never defeats the flag. Mode/mtime are still
replayed (policy-gated) so unprivileged --fake-super keeps working. */
uint32_t store_uid;
uint32_t store_gid;
identity_resolve_storage_ids((int32_t)metadata->uid, (int32_t)metadata->gid, &store_uid,
&store_gid);
fake_super_store_fd(fd, store_uid, store_gid, (uint32_t)metadata->mode, metadata->mtime_sec,
metadata->mtime_nsec);
fake_super_restore_fd(fd, policy);
}
}
+26
View File
@@ -240,3 +240,29 @@ bool file_list_affects(const FileListSet* set, const char* rel) {
entry (binary search for the first entry at or after `rel` + '/'). */
return path_index_has_descendant(&set->index, rel);
}
bool file_list_dir_in_scope(const FileListSet* set, const char* rel) {
if (!set || set->whole_tree)
return true;
if (!rel || rel[0] == '\0')
return false;
/* `rel` itself is listed, or one of its ancestor prefixes is an exact listed
directory (a listed prefix of a directory path is necessarily a
directory). */
size_t len = strlen(rel);
while (len > 0) {
const char* slash = NULL;
for (size_t i = len; i-- > 0;) {
if (rel[i] == '/') {
slash = rel + i;
break;
}
}
if (!slash)
break;
len = (size_t)(slash - rel);
if (path_index_contains_n(&set->index, rel, len))
return true;
}
return path_index_contains(&set->index, rel);
}
+10
View File
@@ -40,4 +40,14 @@ void file_list_destroy(FileListSet* set);
* this returns true, files are transferred only when it returns true. */
bool file_list_affects(const FileListSet* set, const char* rel);
/* True when the DIRECTORY `rel` (path relative to the source root) is inside a
* listed directory subtree: `rel` itself is a listed entry, or one of `rel`'s
* ancestor directory prefixes is an exact listed entry. Unlike
* file_list_affects this does NOT treat an ancestor of a listed entry as
* affected, so an implied parent directory of a listed file is not synchronized
* (rsync deletes nothing in it). With no set or a whole-tree set every
* directory is in scope. This is the delete-walker's "synchronized directory"
* predicate. */
bool file_list_dir_in_scope(const FileListSet* set, const char* rel);
#endif
+282 -109
View File
@@ -19,6 +19,7 @@
#include "delay_updates.h"
#include "delta.h"
#include "file.h"
#include "format.h"
#include "identity.h"
#include "log.h"
#include "metadata.h"
@@ -294,12 +295,17 @@ static FileSaveResult file_save_hardlink_sibling(const char* root_directory, con
free(destination_path);
return absent_result;
}
/* Resolve a relative --temp-dir against the destination root, exactly as the
* primary save path does; an absolute one is used verbatim. */
/* Resolve a relative --temp-dir under the destination root, exactly as the
* primary save path does; an absolute or `..`-escaping value is rejected. */
char* resolved_temp = NULL;
if (cfg->temp_dir) {
resolved_temp =
cfg->temp_dir[0] == '/' ? str_dup(cfg->temp_dir) : path_cat(root_directory, cfg->temp_dir);
if (cfg->temp_dir[0] == '/' || has_path_traversal(cfg->temp_dir)) {
free(content);
free(first_disk);
free(destination_path);
return FILE_SAVE_ERROR;
}
resolved_temp = path_cat(root_directory, cfg->temp_dir);
if (!resolved_temp) {
free(content);
free(first_disk);
@@ -447,10 +453,12 @@ static FileSaveResult file_save_special_to_disk(const char* root_directory, cons
create_mode = S_IFIFO;
}
const char* node_kind = (is_char || is_blk) ? "device" : (is_fifo ? "FIFO" : "socket");
/* The creation permission bits come from the source only under -p/--perms;
* otherwise a safe default (0644, group/other write never granted) keeps an
* unprivileged no--p run from materializing a world-writable node. */
mode_t perms = config->preserve_perms ? (mode & 0777 & ~(S_IWGRP | S_IWOTH)) : 0644;
/* Under -p/--perms rsync copies the source's permission and special bits; a
* kernel that denies setuid/setgid/sticky reports the failure rather than
* having them masked here. Without -p the node is created like any other new
* entry: source_mode & 0777 & ~umask. */
mode_t perms = config->preserve_perms ? (mode & (mode_t)(S_ISUID | S_ISGID | S_ISVTX | 0777))
: (mode & 0777 & ~(mode_t)file_process_umask());
int rc = is_fifo ? mkfifoat(parent_fd, leaf, perms)
: mknodat(parent_fd, leaf, create_mode | perms, rdev);
@@ -771,15 +779,16 @@ FileSaveResult file_save_to_disk_full(const char* root_directory, const File* fi
return file_save_hardlink_sibling(root_directory, file, config);
}
/* These options arrive from the client. --backup-dir and --partial-dir are
names below the server root, never independent filesystem roots: an
absolute or `..`-escaping value is rejected outright. --temp-dir is
deliberately NOT confined: rsync accepts any temp dir (absolute, or
relative to the destination root), including one outside the destination
tree or on another filesystem, and falls back to a non-atomic copy when
the install rename hits EXDEV. */
/* These options arrive from the client. --backup-dir, --partial-dir and
--temp-dir are names below the server root, never independent filesystem
roots: an absolute or `..`-escaping value is rejected outright (rsync's
daemon confines temp-dir to the module the same way). A relative temp dir
is resolved under the receive root below; if that resolution still lands on
a different filesystem than the destination the install falls back to a
non-atomic copy (see file_to_disk_secure_impl), never an abort. */
if ((backup_dir && (backup_dir[0] == '/' || has_path_traversal(backup_dir))) ||
(partial_dir && (partial_dir[0] == '/' || has_path_traversal(partial_dir))))
(partial_dir && (partial_dir[0] == '/' || has_path_traversal(partial_dir))) ||
(temp_dir && (temp_dir[0] == '/' || has_path_traversal(temp_dir))))
return FILE_SAVE_ERROR;
if (backup_dir && !(confined_backup = path_cat(root_directory, backup_dir)))
return FILE_SAVE_ERROR;
@@ -905,20 +914,16 @@ FileSaveResult file_save_to_disk_full(const char* root_directory, const File* fi
/* A configured --temp-dir sends the temporary working copy to a scratch
directory; the engine then atomically renames the completed file into the
final destination directory. rsync resolves a relative temp dir against
the destination directory and uses an absolute one verbatim, requiring
that it already exist; the engine falls back to a non-atomic copy on
EXDEV. The partial-dir flow already keeps its working copy in a separate
directory and --inplace writes directly, so neither diverts through the
scratch dir (matching rsync, where --inplace/--partial-dir supersede
--temp-dir). */
final destination directory. A relative temp dir is resolved under the
receive root and must already exist (an absolute or `..`-escaping value was
rejected above); the engine falls back to a non-atomic copy on EXDEV. The
partial-dir flow already keeps its working copy in a separate directory and
--inplace writes directly, so neither diverts through the scratch dir
(matching rsync, where --inplace/--partial-dir supersede --temp-dir). */
char* confined_temp = NULL;
bool use_temp_dir = temp_dir != NULL && !inplace && !use_partial_root;
if (use_temp_dir) {
if (temp_dir[0] == '/')
confined_temp = str_dup(temp_dir);
else
confined_temp = path_cat(root_directory, temp_dir);
confined_temp = path_cat(root_directory, temp_dir);
if (!confined_temp)
goto fail;
/* A user-supplied trailing slash would leave the scratch path ending in
@@ -1871,6 +1876,33 @@ static IncrementalCheckOutcome incremental_check_open_destination(IncrementalChe
return INCREMENTAL_CONTINUE;
}
/* Output parity (protocol 2.23.0): when the wire config asked for it, report a
snapshot of the pre-transfer destination entry BEFORE the ordinary verdict so
the sender can render rsync-accurate -i/--out-format columns. A missing
destination is reported explicitly (existed=false) rather than omitted, so
the sender can distinguish "new" from "unknown". */
static IncrementalCheckOutcome incremental_check_report_dest_info(IncrementalCheckState* state) {
if (!state->config->report_dest_info)
return INCREMENTAL_CONTINUE;
OutputDestState info;
memset(&info, 0, sizeof(info));
info.known = true;
info.existed = state->has_old_file;
if (state->has_old_file) {
info.size = (unsigned long long)state->old_st.st_size;
info.mtime_sec = (long long)state->old_st.st_mtime;
#ifdef __linux__
info.mtime_nsec = state->old_st.st_mtim.tv_nsec;
#endif
info.mode = (uint32_t)state->old_st.st_mode;
info.uid = (int32_t)state->old_st.st_uid;
info.gid = (int32_t)state->old_st.st_gid;
}
if (!send_status(state->fd, STATUS_DEST_INFO) || !format_dest_state_send(state->fd, &info))
return INCREMENTAL_ERROR;
return INCREMENTAL_CONTINUE;
}
/* Metadata-only (and, when --checksum forces it, content) up-to-date decision.
Loads the old contents only when a checksum comparison or delta needs them. */
static IncrementalCheckOutcome incremental_check_quick_skip(IncrementalCheckState* state,
@@ -2315,6 +2347,10 @@ File* receive_incremental_check_ex(int fd, const Config* config, bool* skipped,
if (outcome == INCREMENTAL_ERROR)
goto done;
outcome = incremental_check_report_dest_info(&state);
if (outcome == INCREMENTAL_ERROR)
goto done;
outcome = incremental_check_quick_skip(&state, &try_delta);
if (outcome == INCREMENTAL_ERROR)
goto done;
@@ -2434,11 +2470,14 @@ File* file_receive(const Config* config, int file_descriptor) {
bool dir_metadata_should_capture(const Config* config) {
/* Directory metadata is captured when a directory attribute is actually
* requested: -p/--perms (directory modes) or -t/--times (directory mtimes,
* unless -O/--omit-dir-times suppresses them). --atimes/-U alone does not
* pull directory metadata (matching the original dir-time bundle). */
* requested: -p/--perms (directory modes), -t/--times (directory mtimes,
* unless -O/--omit-dir-times suppresses them), -o/-g (directory ownership),
* or -X/-A (directory xattrs/ACLs). --atimes/-U alone does not pull
* directory metadata (matching the original dir-time bundle). */
return config && config->use_metadata &&
(config->preserve_perms || (config->preserve_times && !config->omit_dir_times));
(config->preserve_perms || (config->preserve_times && !config->omit_dir_times) ||
config->preserve_owner || config->preserve_group || config->preserve_xattrs ||
config->preserve_acls);
}
void dir_time_list_init(DirTimeList* list) {
@@ -2446,6 +2485,7 @@ void dir_time_list_init(DirTimeList* list) {
return;
list->paths = NULL;
list->entries = NULL;
list->xattrs = NULL;
list->count = 0;
list->capacity = 0;
list->bytes = 0;
@@ -2454,18 +2494,23 @@ void dir_time_list_init(DirTimeList* list) {
void dir_time_list_free(DirTimeList* list) {
if (!list)
return;
for (size_t i = 0; i < list->count; i++)
for (size_t i = 0; i < list->count; i++) {
free(list->paths[i]);
xattr_list_free(list->xattrs ? list->xattrs[i] : NULL);
}
free(list->paths);
free(list->entries);
free(list->xattrs);
list->paths = NULL;
list->entries = NULL;
list->xattrs = NULL;
list->count = 0;
list->capacity = 0;
list->bytes = 0;
}
bool dir_time_list_add(DirTimeList* list, const char* wire_path, const FileMetadata* metadata) {
bool dir_time_list_add(DirTimeList* list, const char* wire_path, const FileMetadata* metadata,
const FileXattrList* xattrs) {
if (!list || !wire_path || !metadata)
return true; /* nothing to remember; never a hard error */
/* Cumulative, not per-frame: the sender may stream a tree across unbounded
@@ -2474,9 +2519,14 @@ bool dir_time_list_add(DirTimeList* list, const char* wire_path, const FileMetad
transfer, which becomes a clean protocol error). */
size_t path_len = strlen(wire_path);
/* Charge the whole per-entry cost (path copy + pointer slot + metadata
struct), not just the path, so the array growth is bounded by the same
cumulative budget. */
size_t entry_cost = path_len + sizeof(FileMetadata) + sizeof(char*);
struct + captured xattrs), not just the path, so the array growth is
bounded by the same cumulative budget. */
size_t xattr_cost = 0;
if (xattrs) {
for (int i = 0; i < xattrs->count; i++)
xattr_cost += strlen(xattrs->items[i].name) + xattrs->items[i].value_len + sizeof(FileXattr);
}
size_t entry_cost = path_len + sizeof(FileMetadata) + 2 * sizeof(char*) + xattr_cost;
if (list->count >= MAX_DIR_TIME_ENTRIES || entry_cost > MAX_DIR_TIME_BYTES - list->bytes)
return false;
if (list->count == list->capacity) {
@@ -2485,10 +2535,9 @@ bool dir_time_list_add(DirTimeList* list, const char* wire_path, const FileMetad
return false;
/* Assign each grown array as soon as its realloc succeeds: the old block is
already freed by then, so discarding the pointer would dangle. capacity
is advanced only after BOTH reallocs succeed, so a partial failure leaves
capacity no larger than the entries allocation (the paths array may be
over-allocated, which is harmless) -- never a mismatched list the next
add could write past. */
is advanced only after ALL reallocs succeed, so a partial failure leaves
capacity no larger than the smallest allocation -- never a mismatched
list the next add could write past. */
char** grown_paths = realloc(list->paths, new_capacity * sizeof(char*));
if (!grown_paths)
return false;
@@ -2497,13 +2546,23 @@ bool dir_time_list_add(DirTimeList* list, const char* wire_path, const FileMetad
if (!grown_entries)
return false;
list->entries = grown_entries;
FileXattrList** grown_xattrs = realloc(list->xattrs, new_capacity * sizeof(FileXattrList*));
if (!grown_xattrs)
return false;
list->xattrs = grown_xattrs;
list->capacity = new_capacity;
}
char* copy = str_dup(wire_path);
if (!copy)
return false;
FileXattrList* xattr_copy = xattr_list_clone(xattrs);
if (xattrs && !xattr_copy) {
free(copy);
return false;
}
list->paths[list->count] = copy;
list->entries[list->count] = *metadata;
list->xattrs[list->count] = xattr_copy;
list->count++;
list->bytes += entry_cost;
return true;
@@ -2515,7 +2574,12 @@ void dir_metadata_list_apply(const DirTimeList* list, const char* root_directory
return;
bool apply_times = config->preserve_times && !config->omit_dir_times;
bool apply_mode = config->preserve_perms;
if (!apply_times && !apply_mode)
bool apply_xattrs = config->use_xattrs;
/* Ownership is applied through the active identity snapshot (which no-ops
* unless an ownership request is active), and xattrs only when -X/-A was
* negotiated. Times/mode keep their own per-attribute gates. */
bool have_any = apply_times || apply_mode || apply_xattrs || identity_active_enabled();
if (!have_any)
return;
for (size_t i = 0; i < list->count; i++) {
char* dir_path = path_cat(root_directory, list->paths[i]);
@@ -2544,6 +2608,13 @@ void dir_metadata_list_apply(const DirTimeList* list, const char* root_directory
free(dir_path);
continue;
}
/* One O_DIRECTORY|O_NOFOLLOW fd drives ownership/mode/xattr application so
none of them can follow a same-named symlink planted after the fstatat. */
int dir_fd = openat(parent_fd, leaf, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
/* Ownership first: a chown clears setuid/setgid, so it must precede mode. */
if (dir_fd >= 0)
identity_apply_ownership(dir_fd, (int32_t)list->entries[i].uid,
(int32_t)list->entries[i].gid);
if (apply_times) {
struct timespec times[2] = {
{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
@@ -2571,30 +2642,28 @@ void dir_metadata_list_apply(const DirTimeList* list, const char* root_directory
mode_ready = false;
}
if (mode_ready) {
/* Route the directory mode through the SAME sanitization as the
* regular-file policy: a client-supplied mode never grants group/other
* write. Open the directory with O_DIRECTORY|O_NOFOLLOW (never
* following a same-named symlink) and fchmod the fd, avoiding the
* fchmodat(..., 0) TOCTOU/symlink-follow hole. */
mode_t safe_mode =
(dir_mode & 0777 & ~(S_IWGRP | S_IWOTH)) | (dir_mode & (S_ISGID | S_ISVTX));
int dir_fd = openat(parent_fd, leaf, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
/* rsync -p copies the source directory mode exactly, including
* group/other write and the setgid/sticky bits. */
mode_t safe_mode = dir_mode & (mode_t)(S_ISUID | S_ISGID | S_ISVTX | 0777);
if (dir_fd < 0) {
char* escaped_path = output_escape(dir_path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "Failed to open directory %s to set its mode: %s",
escaped_path ? escaped_path : "<allocation failed>", strerror(errno));
free(escaped_path);
} else {
if (fchmod(dir_fd, safe_mode) != 0) {
char* escaped_path = output_escape(dir_path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "Failed to set directory mode on %s: %s",
escaped_path ? escaped_path : "<allocation failed>", strerror(errno));
free(escaped_path);
}
close(dir_fd);
} else if (fchmod(dir_fd, safe_mode) != 0) {
char* escaped_path = output_escape(dir_path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "Failed to set directory mode on %s: %s",
escaped_path ? escaped_path : "<allocation failed>", strerror(errno));
free(escaped_path);
}
}
}
/* xattrs/ACLs last: a mode change can rewrite the ACL mask, so the ACL
xattrs must be (re)applied after fchmod. */
if (apply_xattrs && dir_fd >= 0 && list->xattrs)
xattr_apply_fd(dir_fd, list->xattrs[i]);
if (dir_fd >= 0)
close(dir_fd);
close(parent_fd);
free(leaf);
free(dir_path);
@@ -2634,6 +2703,11 @@ File* file_receive_directory(int file_descriptor, const Config* config) {
return NULL;
}
}
/* Directory xattrs/ACLs (-X/-A) ride after the metadata when negotiated. */
if (config && !receive_file_xattrs(file, file_descriptor, config)) {
file_destroy(file);
return NULL;
}
return file;
}
@@ -2670,6 +2744,12 @@ File* file_receive_dir_time(int file_descriptor, const Config* config) {
return NULL;
}
}
/* Directory xattrs/ACLs (-X/-A) ride after the metadata, mirroring the
sender's send_dir_times(). */
if (config && !receive_file_xattrs(file, file_descriptor, config)) {
file_destroy(file);
return NULL;
}
return file;
}
@@ -2845,8 +2925,10 @@ File* file_receive_special(int file_descriptor) {
/* Read a delete-manifest frame (the STATUS_MANIFEST leading code has already
been consumed): a keep-set entry count followed by that many
destination-relative paths, then a protected-prefix count followed by that
many destination-relative prefixes, then (protocol 2.10.0+) a missing-args
count followed by that many destination-relative delete paths. The frame is
many destination-relative prefixes, then a missing-args count followed by that
many destination-relative delete paths, then (protocol 2.23.0) a
synchronized-directory count followed by that many destination-relative
directory paths (the receive root is the "." sentinel). The frame is
self-delimiting (the counts are authoritative), so the caller decides what to
do next and continues reading the following STATUS_* frame. Every section is
validated identically: an entry must be non-empty, relative and traversal-free
@@ -2891,7 +2973,8 @@ DeleteManifest* receive_manifest_entries(int fd) {
manifest->keeps = array_list_create(free);
manifest->protected = array_list_create(free);
manifest->missing = array_list_create(free);
if (!manifest->keeps || !manifest->protected || !manifest->missing) {
manifest->dirs = array_list_create(free);
if (!manifest->keeps || !manifest->protected || !manifest->missing || !manifest->dirs) {
delete_manifest_free(manifest);
send_status(fd, STATUS_ERROR);
return NULL;
@@ -2900,7 +2983,8 @@ DeleteManifest* receive_manifest_entries(int fd) {
size_t manifest_entries = 0;
if (!receive_manifest_section(fd, manifest->keeps, &manifest_bytes, &manifest_entries) ||
!receive_manifest_section(fd, manifest->protected, &manifest_bytes, &manifest_entries) ||
!receive_manifest_section(fd, manifest->missing, &manifest_bytes, &manifest_entries)) {
!receive_manifest_section(fd, manifest->missing, &manifest_bytes, &manifest_entries) ||
!receive_manifest_section(fd, manifest->dirs, &manifest_bytes, &manifest_entries)) {
delete_manifest_free(manifest);
return NULL;
}
@@ -2913,18 +2997,33 @@ void delete_manifest_free(DeleteManifest* manifest) {
array_list_delete(manifest->keeps);
array_list_delete(manifest->protected);
array_list_delete(manifest->missing);
array_list_delete(manifest->dirs);
free(manifest);
}
/* Shared --max-delete budget for one receiver-side deletion commit. Both the
--delete-missing-args exact-path removals and the ordinary extras walk draw
from the same tally, matching rsync (whose --max-delete counts every deleted
file or directory). `max_delete` is SIZE_MAX for an unlimited budget. */
typedef struct {
size_t max_delete;
size_t deleted;
size_t skipped;
bool limit_hit;
} DeleteBudgetState;
/* Remove every destination entry under the receive root that is not in the
keep-set, bounded by MAX_SERVER_DELETE_COUNT (or a smaller client
--max-delete=NUM, which is all-or-nothing), using the symlink-safe delete
walker. With --delay-updates the not-yet-published staging directory is a
keep-set, bounded by the shared budget (a smaller client --max-delete=NUM
replaces the server hard bound; rsync deletes up to the bound and skips the
rest). With --delay-updates the not-yet-published staging directory is a
direct child of the receive root and must not be treated as a set of extras;
the manifest's protected prefixes (paths excluded on the source) and the
the manifest's protected prefixes (paths excluded on the source), the
size-pruned prefixes (--max-size/--min-size, always protected) and the
alternate basis directories are never destination content and are skipped at
any depth. Prints a notice and returns true on success. */
bool manifest_delete_extras(const Config* config, DeleteManifest* manifest) {
any depth. Returns true unless a traversal/unlink error aborted the walk;
the budget's limit_hit/skipped fields report a cap-stopped run. */
static bool delete_extras_budgeted(const Config* config, DeleteManifest* manifest,
DeleteBudgetState* budget) {
if (!config || !manifest || !manifest->keeps)
return false;
fprintf(stderr, "Deleting files not in manifest...\n");
@@ -2936,9 +3035,10 @@ bool manifest_delete_extras(const Config* config, DeleteManifest* manifest) {
at any depth: they are extra comparison snapshots the user pointed at,
not destination content, and deleting them would destroy the very files a
--link-dest run just linked into place;
- the sender-side protected prefixes (source paths excluded by filters), at
any depth, so an excluded destination mirror survives --delete unless
--delete-excluded opts back into removing it. */
- the sender-side protected prefixes (source paths excluded by filters and
paths pruned by --max-size/--min-size), at any depth, so their destination
mirror survives --delete unless --delete-excluded opts back into removing
the filter-excluded ones (size-pruned entries are always protected). */
int skip_count = (config->delay_updates ? 1 : 0) + config->basis_count +
(manifest->protected ? manifest->protected->size : 0);
DeleteSkipEntry* skips = NULL;
@@ -2963,30 +3063,26 @@ bool manifest_delete_extras(const Config* config, DeleteManifest* manifest) {
idx++;
}
}
/* A client --max-delete=NUM smaller than the server's hard bound replaces it
for this run; both still bound the walk. The walker is all-or-nothing, so
a run that would delete more than the bound removes nothing and fails with
an error that names the bound that was hit. */
bool user_limited =
config->max_delete >= 0 && (size_t)config->max_delete < MAX_SERVER_DELETE_COUNT;
size_t cap = user_limited ? (size_t)config->max_delete : MAX_SERVER_DELETE_COUNT;
size_t deleted_count = 0;
DeleteWalkResult result = delete_extras_limited(config->receive_root_directory, manifest->keeps,
cap, skips, skip_count, &deleted_count);
/* Clamp rather than subtract: an accounting bug where deleted already exceeds
max_delete must never underflow into an effectively unlimited budget. */
size_t remaining;
if (budget->max_delete == SIZE_MAX)
remaining = SIZE_MAX;
else if (budget->deleted >= budget->max_delete)
remaining = 0;
else
remaining = budget->max_delete - budget->deleted;
size_t deleted = 0;
size_t skipped = 0;
DeleteWalkResult result =
delete_extras_limited(config->receive_root_directory, manifest->keeps, manifest->dirs,
remaining, skips, skip_count, &deleted, &skipped);
free(skips);
if (result == DELETE_WALK_LIMIT_EXCEEDED) {
if (user_limited) {
log_message(LOG_LEVEL_ERROR,
"deletion stopped: the destination holds more than --max-delete=%d extraneous "
"entries; no files were deleted",
config->max_delete);
} else {
log_message(LOG_LEVEL_ERROR,
"deletion stopped: the destination holds more than %u extraneous entries "
"(server deletion limit); no files were deleted",
(unsigned)MAX_SERVER_DELETE_COUNT);
}
return false;
budget->deleted += deleted;
budget->skipped += skipped;
if (result == DELETE_WALK_LIMIT_REACHED) {
budget->limit_hit = true;
return true;
}
if (result != DELETE_WALK_OK) {
log_message(LOG_LEVEL_ERROR, "deletion failed while removing extraneous files");
@@ -3004,10 +3100,12 @@ bool manifest_delete_extras(const Config* config, DeleteManifest* manifest) {
removed recursively only when --delete or --force is in effect (rsync parity:
the man page says a non-empty directory mirror is only deleted with --force
or --delete); otherwise it is left with a warning and the run continues. A
mirror that does not exist is a no-op. Returns false only on a genuine error
(a confinement failure on a validated path or an I/O error), which fails the
run. */
bool manifest_delete_missing_args(const Config* config, DeleteManifest* manifest) {
mirror that does not exist is a no-op. Each removal draws from the shared
--max-delete budget: once it is exhausted the remaining requests are skipped
and counted. Returns false only on a genuine error (a confinement failure on
a validated path or an I/O error), which fails the run. */
static bool delete_missing_args_budgeted(const Config* config, DeleteManifest* manifest,
DeleteBudgetState* budget) {
if (!config || !manifest)
return false;
if (!manifest->missing || manifest->missing->size == 0)
@@ -3081,6 +3179,16 @@ bool manifest_delete_missing_args(const Config* config, DeleteManifest* manifest
free(full);
continue;
}
/* An entry that exists is one deletion: skip it (and count it) when the
shared --max-delete budget is already exhausted. */
if (budget->deleted >= budget->max_delete) {
budget->limit_hit = true;
budget->skipped++;
close(parent_fd);
free(leaf);
free(full);
continue;
}
bool removed = false;
if (S_ISDIR(st.st_mode)) {
if (unlinkat(parent_fd, leaf, AT_REMOVEDIR) == 0) {
@@ -3091,10 +3199,40 @@ bool manifest_delete_missing_args(const Config* config, DeleteManifest* manifest
free(leaf);
leaf = NULL;
if (config->use_delete || config->force_delete) {
if (!file_remove_tree_secure(full))
/* Remove the contents entry-by-entry through the budgeted extras
walker so every deleted file/dir counts toward --max-delete (rsync
parity); the now-empty directory itself costs one more. A run that
hits the cap leaves the remaining entries in place. */
ArrayList* no_keeps = array_list_create(free);
/* Never let an accounting slip (deleted > max_delete) underflow the
remaining budget into SIZE_MAX, which would grant unlimited
deletions. */
size_t remaining =
budget->deleted >= budget->max_delete ? 0 : budget->max_delete - budget->deleted;
size_t contents_deleted = 0;
size_t contents_skipped = 0;
DeleteWalkResult walk =
no_keeps ? delete_extras_limited(full, no_keeps, NULL, remaining, NULL, 0,
&contents_deleted, &contents_skipped)
: DELETE_WALK_ERROR;
if (no_keeps)
array_list_delete(no_keeps);
budget->deleted += contents_deleted;
budget->skipped += contents_skipped;
if (walk == DELETE_WALK_LIMIT_REACHED) {
budget->limit_hit = true;
} else if (walk != DELETE_WALK_OK) {
ok = false;
else
} else if (budget->deleted >= budget->max_delete) {
budget->limit_hit = true;
budget->skipped++;
} else if (file_remove_tree_secure(full)) {
/* The shared `if (removed)` tail charges this directory exactly
once; counting it here too would consume two budget units. */
removed = true;
} else {
ok = false;
}
} else {
char* escaped = output_escape(rel, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING,
@@ -3114,6 +3252,7 @@ bool manifest_delete_missing_args(const Config* config, DeleteManifest* manifest
}
}
if (removed) {
budget->deleted++;
char* escaped = output_escape(rel, log_get_8_bit_output());
fprintf(stderr, " Deleted: %s\n", escaped ? escaped : "<allocation failed>");
free(escaped);
@@ -3129,24 +3268,58 @@ bool manifest_delete_missing_args(const Config* config, DeleteManifest* manifest
return ok;
}
/* Public wrappers used outside the commit path (and by unit tests): no
--max-delete budget. */
bool manifest_delete_extras(const Config* config, DeleteManifest* manifest) {
DeleteBudgetState budget = {
.max_delete = SIZE_MAX, .deleted = 0, .skipped = 0, .limit_hit = false};
return delete_extras_budgeted(config, manifest, &budget);
}
bool manifest_delete_missing_args(const Config* config, DeleteManifest* manifest) {
DeleteBudgetState budget = {
.max_delete = SIZE_MAX, .deleted = 0, .skipped = 0, .limit_hit = false};
return delete_missing_args_budgeted(config, manifest, &budget);
}
/* Commit every deletion family the manifest carries. The --delete-missing-args
exact-path deletions run FIRST: they are explicit user requests and must not
be blocked by the extras walker's filter-exclusion protection (a protected
leftover inside a missing-argument directory must not make that user-requested
removal fail). The ordinary extras walk then runs when --delete is active.
Returns true when there was nothing to do or every requested deletion
committed. */
bool manifest_delete_all(const Config* config, DeleteManifest* manifest) {
Both draw from one --max-delete budget; the result reports a cap-stopped
(partial) commit distinctly so the client can exit 25 like rsync. */
DeleteCommitResult manifest_delete_all(const Config* config, DeleteManifest* manifest) {
if (!config || !manifest)
return false;
return DELETE_COMMIT_ERROR;
/* Central no-mutation guard: a dry-run never deletes. No manifest is sent on
the dry-run path, but a hostile/buggy peer could; treat it as a no-op so
the receiver can never remove anything. */
if (config->dry_run)
return true;
if (config->delete_missing_args && !manifest_delete_missing_args(config, manifest))
return false;
if (config->use_delete && !manifest_delete_extras(config, manifest))
return false;
return true;
return DELETE_COMMIT_OK;
/* A client --max-delete=NUM smaller than the server's hard bound replaces it
for this run; both still bound the commit. */
bool user_limited =
config->max_delete >= 0 && (size_t)config->max_delete < MAX_SERVER_DELETE_COUNT;
DeleteBudgetState budget = {.max_delete = user_limited ? (size_t)config->max_delete
: MAX_SERVER_DELETE_COUNT,
.deleted = 0,
.skipped = 0,
.limit_hit = false};
if (config->delete_missing_args && !delete_missing_args_budgeted(config, manifest, &budget))
return DELETE_COMMIT_ERROR;
if (config->use_delete && !delete_extras_budgeted(config, manifest, &budget))
return DELETE_COMMIT_ERROR;
if (budget.limit_hit) {
if (user_limited) {
log_message(LOG_LEVEL_ERROR, "Deletions stopped due to --max-delete limit (%zu skipped)",
budget.skipped);
} else {
log_message(LOG_LEVEL_ERROR,
"Deletions stopped due to the server deletion limit of %u (%zu skipped)",
(unsigned)MAX_SERVER_DELETE_COUNT, budget.skipped);
}
return DELETE_COMMIT_LIMIT_REACHED;
}
return DELETE_COMMIT_OK;
}
+38 -17
View File
@@ -40,8 +40,9 @@ File* receive_incremental_check_ex(int fd, const Config* config, bool* skipped,
* parent's mtime). -O/--omit-dir-times skips the application entirely. The
* list owns deep copies of the paths and metadata; freed on every path. */
typedef struct {
char** paths; /* owned, destination-relative wire paths */
FileMetadata* entries; /* owned, parallel to paths */
char** paths; /* owned, destination-relative wire paths */
FileMetadata* entries; /* owned, parallel to paths */
FileXattrList** xattrs; /* owned, parallel to paths; NULL when none */
size_t count;
size_t capacity;
size_t bytes; /* cumulative strlen of every retained path */
@@ -57,17 +58,19 @@ bool dir_metadata_should_capture(const Config* config);
void dir_time_list_init(DirTimeList* list);
void dir_time_list_free(DirTimeList* list);
/* Deep-copy one directory's path + metadata into the list. Returns false on
* allocation failure OR when the cumulative entry/byte caps would be exceeded
* (the caller fails the transfer). */
bool dir_time_list_add(DirTimeList* list, const char* wire_path, const FileMetadata* metadata);
/* Deep-copy one directory's path + metadata (and, when non-NULL, its captured
* xattr/ACL block) into the list. Returns false on allocation failure OR when
* the cumulative entry/byte caps would be exceeded (the caller fails the
* transfer). */
bool dir_time_list_add(DirTimeList* list, const char* wire_path, const FileMetadata* metadata,
const FileXattrList* xattrs);
/* Apply every accumulated directory's metadata beneath `root_directory`,
* confined fd-relative. Times (mtime, plus atime when -U captured one) are
* applied only when config->preserve_times && !config->omit_dir_times; the mode
* (through --chmod when configured) is applied only when config->preserve_perms.
* Best-effort per entry: an absent directory (an empty/pruned source dir that
* was deliberately not created) or a non-directory at the path is skipped
* QUIETLY, an unreachable one with a warning, and never fatal. */
* confined fd-relative: ownership through the negotiated identity policy,
* times (mtime, plus atime when -U captured one under -t), the mode (through
* --chmod when configured, under -p), and the captured xattrs/ACLs (under
* -X/-A). Best-effort per entry: an absent directory (an empty/pruned source
* dir that was deliberately not created) or a non-directory at the path is
* skipped QUIETLY, an unreachable one with a warning, and never fatal. */
void dir_metadata_list_apply(const DirTimeList* list, const char* root_directory,
const Config* config);
@@ -85,11 +88,18 @@ typedef struct DeleteManifest {
ArrayList* keeps;
ArrayList* protected;
ArrayList* missing;
/* Destination-relative paths of the directories the sender synchronized for
this run. The extras walker only removes entries directly inside one of
these (the receive root is the "." sentinel); `--files-from` runs therefore
leave untransmitted directories and the unlisted parts of listed ones
alone, matching rsync's "delete only in synchronized directories". */
ArrayList* dirs;
} DeleteManifest;
void delete_manifest_free(DeleteManifest* manifest);
/* Read a delete-manifest frame: keep count + keeps, then protected count +
protected prefixes, then missing count + missing paths (self-delimiting; the
/* Read a delete-manifest frame (protocol 2.23.0): keep count + keeps, then
protected count + protected prefixes, then missing count + missing paths,
then synchronized-directory count + directory paths (self-delimiting; the
leading STATUS_MANIFEST code has been consumed). Returns an owned
DeleteManifest, or NULL after signalling STATUS_ERROR on a malformed frame. */
DeleteManifest* receive_manifest_entries(int fd);
@@ -108,11 +118,22 @@ bool manifest_delete_extras(const Config* config, DeleteManifest* manifest);
confinement or I/O error (the run then fails); tolerated per-path cases are
reported and skipped. */
bool manifest_delete_missing_args(const Config* config, DeleteManifest* manifest);
/* Outcome of committing a delete manifest. LIMIT_REACHED reports rsync's
partial --max-delete result: the budget allowed some deletions and the rest
were skipped (the run still stores all file data but the client exits 25). */
typedef enum {
DELETE_COMMIT_OK = 0,
DELETE_COMMIT_LIMIT_REACHED,
DELETE_COMMIT_ERROR
} DeleteCommitResult;
/* Run every deletion family the manifest carries: the --delete-missing-args
exact-path deletions first (user requests are not blocked by exclusion
protection), then the ordinary extras walk when --delete is active. Returns
true when nothing to do or everything committed. */
bool manifest_delete_all(const Config* config, DeleteManifest* manifest);
protection), then the ordinary extras walk when --delete is active. Both
share one --max-delete budget. Returns DELETE_COMMIT_OK when nothing was to
do or everything committed, DELETE_COMMIT_LIMIT_REACHED when the budget
stopped part of the work, or DELETE_COMMIT_ERROR on a genuine failure. */
DeleteCommitResult manifest_delete_all(const Config* config, DeleteManifest* manifest);
/* Outcome of a single file_save_to_disk operation. The receiver needs to
distinguish "written" from "skipped" so --remove-source-files can be told
+7
View File
@@ -2,6 +2,7 @@
#define FILE_TYPES_H
#include "data.h"
#include "format.h"
#include "xattr.h"
#include <stdbool.h>
#include <sys/stat.h>
@@ -86,6 +87,12 @@ typedef struct {
* Receiver: parsed off the wire, attached here, and applied fd-relative on
* the written file. NULL/0 == the file carries no xattrs. */
FileXattrList* xattrs;
/* Sender-side output-parity state (never serialized): the receiver-reported
* pre-transfer destination snapshot for this entry, filled by the per-file
* STATUS_CHECK exchange when report_dest_info is set. `known` is false when
* no report was requested/received, in which case -i/--out-format treats the
* entry conservatively as newly created. */
OutputDestState dest_state;
} File;
/* The path that should be sent on the wire and used for the receiver-side
+103
View File
@@ -0,0 +1,103 @@
#include "format.h"
#include "protocol.h"
#include <stdio.h>
#include <string.h>
bool format_human_size_decimal(unsigned long long bytes, char* buffer, size_t buffer_size) {
if (!buffer || buffer_size == 0)
return false;
if (bytes < 1000ULL) {
int written = snprintf(buffer, buffer_size, "%llu", bytes);
return written >= 0 && (size_t)written < buffer_size;
}
static const char units[] = "KMGTPE";
double value = (double)bytes;
size_t divisions = 0;
while (value >= 1000.0 && divisions < sizeof(units) - 1) {
value /= 1000.0;
divisions++;
}
int written = snprintf(buffer, buffer_size, "%.2f%c", value, units[divisions - 1]);
return written >= 0 && (size_t)written < buffer_size;
}
bool format_big_num(unsigned long long value, bool human_readable, char* buffer,
size_t buffer_size) {
if (human_readable)
return format_human_size_decimal(value, buffer, buffer_size);
char digits[32];
int written = snprintf(digits, sizeof(digits), "%llu", value);
if (written < 0 || (size_t)written >= sizeof(digits))
return false;
size_t len = (size_t)written;
size_t separators = len > 1 ? (len - 1) / 3 : 0;
size_t total = len + separators;
if (total + 1 > buffer_size)
return false;
size_t out = total;
buffer[out] = '\0';
size_t digits_since_sep = 0;
for (size_t i = len; i > 0; i--) {
buffer[--out] = digits[i - 1];
digits_since_sep++;
if (digits_since_sep == 3 && i > 1) {
buffer[--out] = ',';
digits_since_sep = 0;
}
}
return true;
}
bool format_rsync_datetime(time_t when, bool dash, char* buffer, size_t buffer_size) {
if (!buffer || buffer_size == 0)
return false;
struct tm broken_down;
if (localtime_r(&when, &broken_down) == NULL)
return false;
const char* format = dash ? "%Y/%m/%d-%H:%M:%S" : "%Y/%m/%d %H:%M:%S";
return strftime(buffer, buffer_size, format, &broken_down) != 0;
}
bool format_dest_state_send(int fd, const OutputDestState* state) {
if (!state)
return false;
int32_t has_old = state->existed ? 1 : 0;
uint64_t size = (uint64_t)state->size;
int64_t mtime = (int64_t)state->mtime_sec;
int64_t mtime_nsec = state->mtime_nsec;
uint32_t mode = state->mode;
int32_t uid = state->uid;
int32_t gid = state->gid;
return send_n_data(fd, &has_old, sizeof(has_old)) && send_n_data(fd, &size, sizeof(size)) &&
send_n_data(fd, &mtime, sizeof(mtime)) &&
send_n_data(fd, &mtime_nsec, sizeof(mtime_nsec)) && send_n_data(fd, &mode, sizeof(mode)) &&
send_n_data(fd, &uid, sizeof(uid)) && send_n_data(fd, &gid, sizeof(gid));
}
bool format_dest_state_receive(int fd, OutputDestState* state) {
if (!state)
return false;
int32_t has_old = 0;
uint64_t size = 0;
int64_t mtime = 0;
int64_t mtime_nsec = 0;
uint32_t mode = 0;
int32_t uid = 0;
int32_t gid = 0;
if (!receive_n_data(fd, &has_old, sizeof(has_old)) || !receive_n_data(fd, &size, sizeof(size)) ||
!receive_n_data(fd, &mtime, sizeof(mtime)) ||
!receive_n_data(fd, &mtime_nsec, sizeof(mtime_nsec)) ||
!receive_n_data(fd, &mode, sizeof(mode)) || !receive_n_data(fd, &uid, sizeof(uid)) ||
!receive_n_data(fd, &gid, sizeof(gid)))
return false;
memset(state, 0, sizeof(*state));
state->known = true;
state->existed = has_old != 0;
state->size = size;
state->mtime_sec = mtime;
state->mtime_nsec = mtime_nsec;
state->mode = mode;
state->uid = uid;
state->gid = gid;
return true;
}
+59
View File
@@ -0,0 +1,59 @@
#ifndef FORMAT_H
#define FORMAT_H
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#include <time.h>
/* Low-level output-formatting primitives shared by the change-event model
* (change_list.c) and the transfer driver (client_send.c).
*
* The functions here are pure/string-level except for the STATUS_DEST_INFO
* codec, which lets the receiver report the pre-transfer destination entry so
* the sender can render rsync-accurate --itemize-changes / --out-format
* columns (see protocol.h). */
/* Pre-transfer destination snapshot, reported by the receiver when the wire
* config carries report_dest_info. `known` distinguishes "no report was
* requested/received" from "the destination did not exist" (`existed == false`
* with `known == true`). */
typedef struct {
bool known;
bool existed;
unsigned long long size;
long long mtime_sec;
long long mtime_nsec;
uint32_t mode;
int32_t uid;
int32_t gid;
} OutputDestState;
/* rsync's -h/--human-readable size (decimal, base 1000): integers below 1000
* print verbatim; larger values use the largest unit that keeps the value
* below 1000 (K/M/G/T/P/E) with exactly two decimals, so 1500000 -> "1.50M"
* and 999999 -> "1000.00K" (matching rsync's human_num). Returns false when
* the buffer is too small (nothing is written). */
bool format_human_size_decimal(unsigned long long bytes, char* buffer, size_t buffer_size);
/* rsync's general number formatting (big_num). When `human_readable` is true
* this is format_human_size_decimal; otherwise the integer is rendered with a
* ',' thousands separator every three digits (rsync's separator in the C
* locale). Returns false on an undersized buffer. */
bool format_big_num(unsigned long long value, bool human_readable, char* buffer,
size_t buffer_size);
/* rsync's %M/%t timestamp. When `dash` is true the separator between the date
* and the time is '-' (the %M form: "YYYY/MM/DD-HH:MM:SS"); otherwise it is a
* space (the %t form: "YYYY/MM/DD HH:MM:SS"). Local time. Returns false on a
* bad time or an undersized buffer. */
bool format_rsync_datetime(time_t when, bool dash, char* buffer, size_t buffer_size);
/* Fixed-width STATUS_DEST_INFO record codec (int32 has_old, uint64 size,
* int64 mtime, int64 mtime_nsec, uint32 mode, int32 uid, int32 gid). The
* status frame itself is sent/received by the caller. Returns false on I/O
* failure. */
bool format_dest_state_send(int fd, const OutputDestState* state);
bool format_dest_state_receive(int fd, OutputDestState* state);
#endif
+343 -97
View File
@@ -43,14 +43,27 @@ typedef struct {
* so they are tracked separately from the explicit ownership gate. */
bool preserve_owner;
bool preserve_group;
/* --fake-super: when active the receiver must only RECORD the (resolved)
* ownership in the reserved xattr, never perform a real chown. Snapshotted
* so the fd-relative ownership helpers can suppress the chown without a
* Config argument. */
bool fake_super;
bool set;
} IdentityActive;
static IdentityActive g_identity;
static void identity_active_reset(void) {
free(g_identity.usermap);
free(g_identity.groupmap);
if (g_identity.usermap) {
for (int i = 0; i < g_identity.usermap_count; i++)
free(g_identity.usermap[i].to_name);
free(g_identity.usermap);
}
if (g_identity.groupmap) {
for (int i = 0; i < g_identity.groupmap_count; i++)
free(g_identity.groupmap[i].to_name);
free(g_identity.groupmap);
}
g_identity.usermap = NULL;
g_identity.groupmap = NULL;
g_identity.usermap_count = 0;
@@ -66,6 +79,7 @@ static void identity_active_reset(void) {
g_identity.copy_as_gid = 0;
g_identity.preserve_owner = false;
g_identity.preserve_group = false;
g_identity.fake_super = false;
g_identity.set = false;
}
@@ -88,20 +102,35 @@ bool identity_set_active(const Config* config) {
g_identity.copy_as_gid = config->copy_as_gid;
g_identity.preserve_owner = config->preserve_owner;
g_identity.preserve_group = config->preserve_group;
g_identity.fake_super = config->fake_super;
if (config->usermap_count > 0) {
g_identity.usermap = calloc((size_t)config->usermap_count, sizeof(IdentityMap));
if (!g_identity.usermap)
goto alloc_failed;
memcpy(g_identity.usermap, config->usermap,
(size_t)config->usermap_count * sizeof(IdentityMap));
for (int i = 0; i < config->usermap_count; i++) {
g_identity.usermap[i] = config->usermap[i];
g_identity.usermap[i].to_name =
config->usermap[i].to_name ? str_dup(config->usermap[i].to_name) : NULL;
if (config->usermap[i].to_name && !g_identity.usermap[i].to_name) {
g_identity.usermap_count = i; /* free only the entries already duplicated */
goto alloc_failed;
}
}
g_identity.usermap_count = config->usermap_count;
}
if (config->groupmap_count > 0) {
g_identity.groupmap = calloc((size_t)config->groupmap_count, sizeof(IdentityMap));
if (!g_identity.groupmap)
goto alloc_failed;
memcpy(g_identity.groupmap, config->groupmap,
(size_t)config->groupmap_count * sizeof(IdentityMap));
for (int i = 0; i < config->groupmap_count; i++) {
g_identity.groupmap[i] = config->groupmap[i];
g_identity.groupmap[i].to_name =
config->groupmap[i].to_name ? str_dup(config->groupmap[i].to_name) : NULL;
if (config->groupmap[i].to_name && !g_identity.groupmap[i].to_name) {
g_identity.groupmap_count = i;
goto alloc_failed;
}
}
g_identity.groupmap_count = config->groupmap_count;
}
g_identity.set = true;
@@ -157,30 +186,28 @@ bool privilege_super_mode_permitted(SuperMode mode) {
}
bool identity_active_enabled(void) {
/* numeric_ids is included: this set only gates identity_apply_ownership,
which runs only when metadata is present (a -M/--preserve transfer). A
standalone --numeric-ids (no ownership-affecting flag) carries no
metadata, never reaches identity_apply_ownership, and therefore correctly
stays inert; combined with -M it activates raw-id application. --super /
--no-super does NOT enable ownership: it only permits or forbids the
already-requested super-user activities, so a --super with no explicit
identity flag must never silently apply client-chosen ownership. */
/* --numeric-ids is deliberately NOT included: it is a mapping MODIFIER (use
* the transmitted numeric id raw instead of a name lookup), not a request to
* change ownership. rsync's --numeric-ids on its own never chowns anything;
* it only changes how an already-requested -o/-g/map resolves. Ownership is
* activated only by an explicit request: --chown/--usermap/--groupmap/
* --copy-as or a preserve-source -o/--owner / -g/--group. --super/--no-super
* likewise does NOT enable ownership: it only permits or forbids the
* already-requested super-user activities. */
return g_identity.set &&
(g_identity.numeric_ids || g_identity.chown_uid_set || g_identity.chown_gid_set ||
g_identity.usermap_count > 0 || g_identity.groupmap_count > 0 || g_identity.copy_as_set ||
g_identity.preserve_owner || g_identity.preserve_group);
(g_identity.chown_uid_set || g_identity.chown_gid_set || g_identity.usermap_count > 0 ||
g_identity.groupmap_count > 0 || g_identity.copy_as_set || g_identity.preserve_owner ||
g_identity.preserve_group);
}
bool identity_owner_requested(void) {
return g_identity.set &&
(g_identity.copy_as_set || g_identity.chown_uid_set || g_identity.numeric_ids ||
g_identity.preserve_owner || g_identity.usermap_count > 0);
return g_identity.set && (g_identity.copy_as_set || g_identity.chown_uid_set ||
g_identity.preserve_owner || g_identity.usermap_count > 0);
}
bool identity_group_requested(void) {
return g_identity.set &&
(g_identity.copy_as_set || g_identity.chown_gid_set || g_identity.numeric_ids ||
g_identity.preserve_group || g_identity.groupmap_count > 0);
return g_identity.set && (g_identity.copy_as_set || g_identity.chown_gid_set ||
g_identity.preserve_group || g_identity.groupmap_count > 0);
}
bool identity_ownership_requested(const Config* config) {
@@ -228,6 +255,29 @@ bool identity_copy_as_refused(const Config* config) {
return geteuid() != 0 || config->super_mode == SUPER_MODE_OFF;
}
/* Validate one received FROM:TO map rule. `from` is a single id, the LOW end
* of an inclusive range, IDENTITY_MATCH_ANY, or IDENTITY_MATCH_UNNAMED; a
* sentinel FROM must carry the same value in from_hi. `to` is a non-negative
* id, IDENTITY_CURRENT, or ignored when a bounded receiver-resolved `to_name`
* is present. */
static bool identity_wire_map_valid(const IdentityMap* map) {
if (!map)
return false;
if (map->from < IDENTITY_MATCH_UNNAMED)
return false;
if (map->from < 0) {
if (map->from_hi != map->from)
return false;
} else if (map->from_hi < map->from) {
return false;
}
if (map->to < IDENTITY_CURRENT)
return false;
if (map->to_name && strlen(map->to_name) > 255)
return false;
return true;
}
bool identity_wire_valid(const Config* config) {
if (!config)
return false;
@@ -239,11 +289,11 @@ bool identity_wire_valid(const Config* config) {
if (config->chown_gid_set && config->chown_gid < IDENTITY_MATCH_ANY)
return false;
for (int i = 0; i < config->usermap_count; i++) {
if (config->usermap[i].from < IDENTITY_MATCH_ANY || config->usermap[i].to < IDENTITY_CURRENT)
if (!identity_wire_map_valid(&config->usermap[i]))
return false;
}
for (int i = 0; i < config->groupmap_count; i++) {
if (config->groupmap[i].from < IDENTITY_MATCH_ANY || config->groupmap[i].to < IDENTITY_CURRENT)
if (!identity_wire_map_valid(&config->groupmap[i]))
return false;
}
/* Defense-in-depth: a --copy-as block must never carry a negative (sentinel)
@@ -301,15 +351,137 @@ static int identity_resolve_token(const char* token, bool is_group, int32_t* out
return 0;
}
static int identity_append_rule(IdentityMap** map, int* count, int32_t from, int32_t to) {
static bool identity_all_digits(const char* token) {
if (!token || *token == '\0')
return false;
for (const char* p = token; *p; p++)
if (*p < '0' || *p > '9')
return false;
return true;
}
static bool identity_token_has_glob(const char* token) {
return token && (strchr(token, '*') || strchr(token, '?') || strchr(token, '['));
}
/* Parse a --usermap/--groupmap FROM token into a matcher (from/from_hi). rsync
* accepts a name, a numeric id, an inclusive LOW-HIGH range, '*' (any id), or an
* empty token (ids with no name on the sender). Returns 0 on success, -1 on a
* malformed token or an unresolvable sender-side name. */
static int identity_parse_from(const char* token, bool is_group, int32_t* out_from,
int32_t* out_hi) {
if (token[0] == '\0') {
*out_from = IDENTITY_MATCH_UNNAMED;
*out_hi = IDENTITY_MATCH_UNNAMED;
return 0;
}
if (strcmp(token, "*") == 0) {
*out_from = IDENTITY_MATCH_ANY;
*out_hi = IDENTITY_MATCH_ANY;
return 0;
}
const char* num = token[0] == '@' ? token + 1 : token;
if (identity_all_digits(num)) {
int32_t id;
if (identity_resolve_token(token, is_group, &id) != 0)
return -1;
*out_from = id;
*out_hi = id;
return 0;
}
/* An inclusive LOW-HIGH numeric range. */
const char* dash = strchr(num, '-');
if (dash && dash != num && dash[1] != '\0' && strchr(dash + 1, '-') == NULL) {
size_t lo_len = (size_t)(dash - num);
size_t hi_len = strlen(dash + 1);
char low[16];
char high[16];
if (lo_len < sizeof(low) && hi_len < sizeof(high)) {
memcpy(low, num, lo_len);
low[lo_len] = '\0';
memcpy(high, dash + 1, hi_len);
high[hi_len] = '\0';
if (identity_all_digits(low) && identity_all_digits(high)) {
char* endptr = NULL;
errno = 0;
long lo = strtol(low, &endptr, 10);
if (errno != 0 || !endptr || *endptr != '\0')
return -1;
errno = 0;
long hi = strtol(high, &endptr, 10);
if (errno != 0 || !endptr || *endptr != '\0' || hi < lo || hi > INT32_MAX)
return -1;
*out_from = (int32_t)lo;
*out_hi = (int32_t)hi;
return 0;
}
}
/* Not a numeric LOW-HIGH range: fall through and treat as a name (a
* hyphenated account name like "wayne-smith" must still resolve). */
}
/* A sender-side name. A wildcard other than the bare '*' is matched by rsync
* against the sender's names; because FastSync transmits numeric ids only, the
* receiver cannot evaluate it, so reject rather than silently mis-match. */
if (identity_token_has_glob(token)) {
log_message(LOG_LEVEL_ERROR,
"%smap FROM '%s': name wildcards other than '*' are not supported "
"(FastSync transmits numeric ids, so sender names are unavailable on the "
"receiver)",
is_group ? "--group" : "--user", token);
return -1;
}
int32_t id;
if (identity_resolve_token(token, is_group, &id) != 0)
return -1;
*out_from = id;
*out_hi = id;
return 0;
}
/* Parse a --usermap/--groupmap TO token. '*', a bare numeric id, or an @N id is
* stored numerically; every other non-empty token is a NAME resolved on the
* RECEIVER at apply time (rsync resolves TO names against the receiving side).
* Returns 0 on success, -1 on an empty/malformed token. */
static int identity_parse_to(const char* token, bool is_group, int32_t* out_to, char** out_name) {
if (token[0] == '\0') {
log_message(LOG_LEVEL_ERROR, "%smap TO value is missing", is_group ? "--group" : "--user");
return -1;
}
if (strcmp(token, "*") == 0) {
*out_to = IDENTITY_CURRENT;
*out_name = NULL;
return 0;
}
const char* num = token[0] == '@' ? token + 1 : token;
if (identity_all_digits(num)) {
int32_t id;
if (identity_resolve_token(token, is_group, &id) != 0)
return -1;
*out_to = id;
*out_name = NULL;
return 0;
}
if (identity_token_has_glob(token)) {
log_message(LOG_LEVEL_ERROR, "%smap TO '%s' may not contain a wildcard",
is_group ? "--group" : "--user", token);
return -1;
}
char* name = str_dup(token);
if (!name)
return -1;
*out_to = 0;
*out_name = name;
return 0;
}
static int identity_append_rule(IdentityMap** map, int* count, const IdentityMap* rule) {
if (*count >= MAX_IDENTITY_MAP)
return -1;
IdentityMap* grown = realloc(*map, (size_t)(*count + 1) * sizeof(IdentityMap));
if (!grown)
return -1;
*map = grown;
(*map)[*count].from = from;
(*map)[*count].to = to;
(*map)[*count] = *rule;
(*count)++;
return 0;
}
@@ -326,7 +498,7 @@ int identity_parse_map(Config* config, const char* value, bool is_group) {
char* saveptr = NULL;
for (char* rule = strtok_r(list, ",", &saveptr); rule; rule = strtok_r(NULL, ",", &saveptr)) {
char* colon = strchr(rule, ':');
if (!colon || colon == rule) {
if (!colon) {
/* Log before freeing: `rule` points into the str_dup'd list. */
log_message(LOG_LEVEL_ERROR, "%s rules must be FROM:TO (got '%s')", optname, rule);
free(list);
@@ -335,25 +507,25 @@ int identity_parse_map(Config* config, const char* value, bool is_group) {
*colon = '\0';
char* from_token = rule;
char* to_token = colon + 1;
if (*to_token == '\0') {
IdentityMap parsed;
memset(&parsed, 0, sizeof(parsed));
if (identity_parse_from(from_token, is_group, &parsed.from, &parsed.from_hi) != 0) {
log_message(LOG_LEVEL_ERROR,
"%s could not resolve FROM '%s' in '%s' (a name must exist on the "
"source; use @N for a numeric id)",
optname, from_token, value);
free(list);
log_message(LOG_LEVEL_ERROR, "%s rule 'FROM:' is missing the TO value (got '%s')", optname,
value);
return -1;
}
int32_t from_id, to_id;
if (identity_resolve_token(from_token, is_group, &from_id) != 0 ||
identity_resolve_token(to_token, is_group, &to_id) != 0) {
if (identity_parse_to(to_token, is_group, &parsed.to, &parsed.to_name) != 0) {
log_message(LOG_LEVEL_ERROR, "%s could not parse TO '%s' in '%s'", optname, to_token, value);
free(list);
log_message(LOG_LEVEL_ERROR,
"%s could not resolve '%s' (name must exist on the source; use "
"@N for a numeric id)",
optname, value);
return -1;
}
if (identity_append_rule(is_group ? &config->groupmap : &config->usermap,
is_group ? &config->groupmap_count : &config->usermap_count, from_id,
to_id) != 0) {
is_group ? &config->groupmap_count : &config->usermap_count,
&parsed) != 0) {
free(parsed.to_name);
free(list);
log_message(LOG_LEVEL_ERROR, "%s has too many rules (max %d)", optname, MAX_IDENTITY_MAP);
return -1;
@@ -628,17 +800,108 @@ done:
/* ---- Receiver-side ownership application ---- */
static bool identity_map_lookup(const IdentityMap* map, int count, int32_t source_id,
/* True when a map rule's FROM matcher accepts `id`. A sentinel FROM never
* carries a range. IDENTITY_MATCH_UNNAMED mirrors rsync's empty FROM: it
* matches only ids that have no name in the account database (rsync matches the
* sender's names; FastSync transmits numeric ids only, so it approximates this
* with the receiver's database -- documented in RSYNC_COMPAT.md). */
static bool identity_map_from_matches(const IdentityMap* map, int32_t id, bool is_group) {
if (map->from == IDENTITY_MATCH_ANY)
return true;
if (map->from == IDENTITY_MATCH_UNNAMED)
return is_group ? (getgrgid((gid_t)id) == NULL) : (getpwuid((uid_t)id) == NULL);
return id >= map->from && id <= map->from_hi;
}
/* First matching rule wins. A rule whose TO is a receiver-side name resolves it
* against the receiver's account database here; an unresolvable TO name is
* skipped with a warning and the next rule is considered (rsync prints "Unknown
* --usermap name on receiver" and leaves the id unmapped rather than aborting). */
static bool identity_map_lookup(const IdentityMap* map, int count, int32_t source_id, bool is_group,
int32_t* out_to) {
for (int i = 0; i < count; i++) {
if (map[i].from == IDENTITY_MATCH_ANY || map[i].from == source_id) {
if (!identity_map_from_matches(&map[i], source_id, is_group))
continue;
if (map[i].to_name) {
if (is_group) {
struct group* gr = getgrnam(map[i].to_name);
if (!gr) {
log_message(LOG_LEVEL_WARNING, "Unknown --groupmap name on receiver: %s", map[i].to_name);
continue;
}
*out_to = (int32_t)gr->gr_gid;
} else {
struct passwd* pw = getpwnam(map[i].to_name);
if (!pw) {
log_message(LOG_LEVEL_WARNING, "Unknown --usermap name on receiver: %s", map[i].to_name);
continue;
}
*out_to = (int32_t)pw->pw_uid;
}
} else {
*out_to = map[i].to;
return true;
}
return true;
}
return false;
}
/* Resolve the owner side from the negotiated policy. Sets *out and returns
* true when an owner-affecting request is active (a usermap, --chown USER, or
* -o/--owner); returns false (leaving *out untouched) when the owner side is
* not requested, so callers can pass (uid_t)-1 to fchown and leave it as-is.
* --numeric-ids only changes the RESOLUTION (raw id instead of a name lookup);
* it never makes the side requested. */
static bool identity_resolve_owner(int32_t source_uid, uid_t* out) {
if (!(g_identity.chown_uid_set || g_identity.preserve_owner || g_identity.usermap_count > 0))
return false;
int32_t target;
if (identity_map_lookup(g_identity.usermap, g_identity.usermap_count, source_uid, false,
&target)) {
*out = target == IDENTITY_CURRENT ? geteuid() : (uid_t)target;
} else if (g_identity.chown_uid_set) {
*out = g_identity.chown_uid == IDENTITY_CURRENT ? geteuid() : (uid_t)g_identity.chown_uid;
} else if (g_identity.numeric_ids) {
*out = (uid_t)source_uid;
} else {
/* Best-effort name mapping against the receiver's own database. When the
* transmitted (numeric) id has no name here, fall back to the raw numeric id
* so -o still preserves the source owner. */
struct passwd* pw = getpwuid((uid_t)source_uid);
if (pw) {
const struct passwd* mapped = getpwnam(pw->pw_name);
*out = mapped ? mapped->pw_uid : (uid_t)source_uid;
} else {
*out = (uid_t)source_uid;
}
}
return true;
}
/* Group-side counterpart of identity_resolve_owner(). */
static bool identity_resolve_group(int32_t source_gid, gid_t* out) {
if (!(g_identity.chown_gid_set || g_identity.preserve_group || g_identity.groupmap_count > 0))
return false;
int32_t target;
if (identity_map_lookup(g_identity.groupmap, g_identity.groupmap_count, source_gid, true,
&target)) {
*out = target == IDENTITY_CURRENT ? getegid() : (gid_t)target;
} else if (g_identity.chown_gid_set) {
*out = g_identity.chown_gid == IDENTITY_CURRENT ? getegid() : (gid_t)g_identity.chown_gid;
} else if (g_identity.numeric_ids) {
*out = (gid_t)source_gid;
} else {
struct group* gr = getgrgid((gid_t)source_gid);
if (gr) {
const struct group* mapped = getgrnam(gr->gr_name);
*out = mapped ? mapped->gr_gid : (gid_t)source_gid;
} else {
*out = (gid_t)source_gid;
}
}
return true;
}
/* Resolve the target ownership from the negotiated policy against the entry's
* current stat. Shared by the fd (regular file) and no-follow (symlink) apply
* paths. Returns false when no side is to be changed. */
@@ -662,58 +925,12 @@ static bool identity_resolve_targets(const struct stat* st, int32_t source_uid,
* request the owner/group respectively, and a side that is NOT requested must
* be left exactly as it is (`-1` to fchown on that side). This is what lets
* plain -g change only the group, or -o only the owner. */
bool owner_requested = g_identity.chown_uid_set || g_identity.numeric_ids ||
g_identity.preserve_owner || g_identity.usermap_count > 0;
bool group_requested = g_identity.chown_gid_set || g_identity.numeric_ids ||
g_identity.preserve_group || g_identity.groupmap_count > 0;
if (!owner_requested && !group_requested)
return false;
int32_t target;
uid_t uid = (uid_t)-1;
gid_t gid = (gid_t)-1;
/* Priority (unchanged): usermap/groupmap > --chown > --numeric-ids (raw) >
* name mapping on the transmitted numeric id, with a raw-id fallback when the
* receiver has no name for that id. */
if (owner_requested) {
if (identity_map_lookup(g_identity.usermap, g_identity.usermap_count, source_uid, &target)) {
uid = target == IDENTITY_CURRENT ? geteuid() : (uid_t)target;
} else if (g_identity.chown_uid_set) {
uid = g_identity.chown_uid == IDENTITY_CURRENT ? geteuid() : (uid_t)g_identity.chown_uid;
} else if (g_identity.numeric_ids) {
uid = (uid_t)source_uid;
} else {
/* Best-effort name mapping against the receiver's own database. When the
* transmitted (numeric) id has no name here, fall back to the raw numeric
* id so -o still preserves the source owner. */
struct passwd* pw = getpwuid((uid_t)source_uid);
if (pw) {
const struct passwd* mapped = getpwnam(pw->pw_name);
uid = mapped ? mapped->pw_uid : (uid_t)source_uid;
} else {
uid = (uid_t)source_uid;
}
}
}
if (group_requested) {
if (identity_map_lookup(g_identity.groupmap, g_identity.groupmap_count, source_gid, &target)) {
gid = target == IDENTITY_CURRENT ? getegid() : (gid_t)target;
} else if (g_identity.chown_gid_set) {
gid = g_identity.chown_gid == IDENTITY_CURRENT ? getegid() : (gid_t)g_identity.chown_gid;
} else if (g_identity.numeric_ids) {
gid = (gid_t)source_gid;
} else {
struct group* gr = getgrgid((gid_t)source_gid);
if (gr) {
const struct group* mapped = getgrnam(gr->gr_name);
gid = mapped ? mapped->gr_gid : (gid_t)source_gid;
} else {
gid = (gid_t)source_gid;
}
}
}
bool owner_requested = identity_resolve_owner(source_uid, &uid);
bool group_requested = identity_resolve_group(source_gid, &gid);
if (!owner_requested && !group_requested)
return false;
/* Only change ownership when a requested side actually differs (avoid
* needless syscalls and any chance of clearing setuid/setgid on an
@@ -726,6 +943,30 @@ static bool identity_resolve_targets(const struct stat* st, int32_t source_uid,
return true;
}
/* --fake-super storage resolution: the receiver records the ownership it WOULD
* have applied. A requested side uses the resolved mapping (--copy-as /
* usermap / --chown / -o/-g, with --numeric-ids as the raw-id modifier); a side
* that was not requested keeps the source's own id, so a plain --fake-super run
* records the source owner untouched. */
void identity_resolve_storage_ids(int32_t source_uid, int32_t source_gid, uint32_t* out_uid,
uint32_t* out_gid) {
if (g_identity.copy_as_set) {
*out_uid = (uint32_t)g_identity.copy_as_uid;
*out_gid = (uint32_t)g_identity.copy_as_gid;
return;
}
uid_t uid = (uid_t)source_uid;
gid_t gid = (gid_t)source_gid;
uid_t resolved_uid;
gid_t resolved_gid;
if (identity_resolve_owner(source_uid, &resolved_uid))
uid = resolved_uid;
if (identity_resolve_group(source_gid, &resolved_gid))
gid = resolved_gid;
*out_uid = (uint32_t)uid;
*out_gid = (uint32_t)gid;
}
static void identity_log_chown_failure(const char* what, uid_t uid, gid_t gid) {
/* EPERM/EACCES are expected when the receiver is not privileged (e.g. the CI
* `nobody` user): warn and continue, never abort the transfer. Any other
@@ -760,8 +1001,12 @@ bool identity_apply_ownership(int fd, int32_t source_uid, int32_t source_gid) {
/* Ownership application is OFF unless the client requested an identity flag.
* This is the controlled gate: a default (or plain -M) transfer never changes
* ownership, byte-for-byte preserving FastSync's existing behavior. --no-super
* additionally forbids it even when the receiver is root. */
if (!identity_active_enabled() || !privilege_super_permitted() || fd < 0)
* additionally forbids it even when the receiver is root. --fake-super never
* performs a REAL chown: that would defeat the point of the flag (record the
* source ownership on an unprivileged receiver for a later privileged
* restore); the resolved ownership is stored in the reserved xattr instead by
* fake_super_store_fd(). */
if (!identity_active_enabled() || g_identity.fake_super || !privilege_super_permitted() || fd < 0)
return true;
struct stat st;
if (fstat(fd, &st) != 0)
@@ -781,7 +1026,8 @@ bool identity_apply_ownership(int fd, int32_t source_uid, int32_t source_gid) {
bool identity_apply_ownership_link(int parent_fd, const char* leaf, int32_t source_uid,
int32_t source_gid) {
if (!identity_active_enabled() || !privilege_super_permitted() || parent_fd < 0 || !leaf)
if (!identity_active_enabled() || g_identity.fake_super || !privilege_super_permitted() ||
parent_fd < 0 || !leaf)
return true;
struct stat st;
if (fstatat(parent_fd, leaf, &st, AT_SYMLINK_NOFOLLOW) != 0)
+8
View File
@@ -127,6 +127,14 @@ bool identity_explicit_ownership_requested(const Config* config);
* is active returns true. */
bool identity_apply_ownership(int fd, int32_t source_uid, int32_t source_gid);
/* Resolve the ownership that --fake-super should RECORD in the reserved xattr
* (rather than chown for real). A requested side (--copy-as / usermap /
* --chown / -o / -g, with --numeric-ids as the raw-id modifier) yields the
* resolved target; a side that was not requested keeps the transmitted source
* id. Must be called after identity_set_active(). */
void identity_resolve_storage_ids(int32_t source_uid, int32_t source_gid, uint32_t* out_uid,
uint32_t* out_gid);
/* P7 Wave D: the no-follow (symlink) counterpart. Resolves the same
* usermap/groupmap/chown/numeric-ids/copy-as policy but applies it with
* fchownat(..., AT_SYMLINK_NOFOLLOW) so a symlink's own ownership is changed
+21 -16
View File
@@ -213,8 +213,11 @@ bool metadata_mode_for_policy(mode_t source_mode, mode_t current_mode, FileAttrP
mode_t* out_mode) {
const mode_t execute_bits = S_IXUSR | S_IXGRP | S_IXOTH;
if (policy.perms) {
/* Group/other write is never granted from a client-supplied mode. */
*out_mode = source_mode & 0777 & ~(S_IWGRP | S_IWOTH);
/* rsync --perms copies the source's permission and special bits exactly,
* including group/other write and setuid/setgid/sticky. The kernel may
* still clear setgid when the receiver is not in the file's group; the
* caller logs a failed chmod rather than silently masking the bits here. */
*out_mode = source_mode & (mode_t)(S_ISUID | S_ISGID | S_ISVTX | 0777);
return true;
}
if (policy.executability) {
@@ -223,9 +226,9 @@ bool metadata_mode_for_policy(mode_t source_mode, mode_t current_mode, FileAttrP
* bits from the DESTINATION's own read bits (so a class that can read may
* execute); otherwise clear every execute bit. This runs on the
* destination-derived base (pre-existing dest mode, or source&~umask for a
* new file), and leaves special bits untouched. --perms wins when both are
* set (handled above). */
mode_t base = current_mode & 0777;
* new file), and leaves the special bits untouched. --perms wins when both
* are set (handled above). */
mode_t base = current_mode & (mode_t)(S_ISUID | S_ISGID | S_ISVTX | 0777);
if (source_mode & 0111)
*out_mode = base | ((base & 0444) >> 2);
else
@@ -310,7 +313,7 @@ bool file_restore_symlink_metadata(const char* path, const FileMetadata* metadat
platforms that support it and quietly ignore the unsupported case so the
transfer never fails over it. */
if (policy.perms) {
mode_t link_mode = metadata->mode & 0777 & ~(S_IWGRP | S_IWOTH);
mode_t link_mode = metadata->mode & (mode_t)(S_ISUID | S_ISGID | S_ISVTX | 0777);
if (fchmodat(parent_fd, leaf, link_mode, AT_SYMLINK_NOFOLLOW) != 0 && errno != EOPNOTSUPP &&
errno != ENOTSUP && errno != ENOSYS) {
log_message(LOG_LEVEL_DEBUG, "Could not set symlink mode on %s: %s", path, strerror(errno));
@@ -343,15 +346,6 @@ bool file_restore_metadata_fd(int fd, const FileMetadata* metadata, FileAttrPoli
if (fd < 0 || metadata == NULL)
return metadata == NULL;
bool ok = true;
if (policy.perms || policy.executability) {
struct stat current;
if (fstat(fd, &current) != 0)
return false;
mode_t safe_mode = 0;
bool apply_mode = metadata_mode_for_policy(metadata->mode, current.st_mode, policy, &safe_mode);
if (apply_mode && fchmod(fd, safe_mode) != 0)
ok = false;
}
/* Client uid/gid values are deliberately not authoritative UNLESS the client
explicitly opted in with an identity flag (--numeric-ids / --usermap /
--groupmap / --chown / -o/-g). identity_apply_ownership is the controlled,
@@ -362,9 +356,20 @@ bool file_restore_metadata_fd(int fd, const FileMetadata* metadata, FileAttrPoli
marks this entry as failed instead of reporting a wrong-owner write as
success. With no identity flag set it is a no-op, so a default or plain -M
transfer keeps FastSync's existing behavior of never applying client
ownership. */
ownership. Ownership runs BEFORE the mode because a chown clears
setuid/setgid; rsync likewise chowns first and then restores the source
mode (including its special bits). */
if (!identity_apply_ownership(fd, (int32_t)metadata->uid, (int32_t)metadata->gid))
ok = false;
if (policy.perms || policy.executability) {
struct stat current;
if (fstat(fd, &current) != 0)
return false;
mode_t safe_mode = 0;
bool apply_mode = metadata_mode_for_policy(metadata->mode, current.st_mode, policy, &safe_mode);
if (apply_mode && fchmod(fd, safe_mode) != 0)
ok = false;
}
/* --crtimes captures and transmits the source birth time, but there is no
* portable way to set a birth time (utimensat can only set atime/mtime), so
* the receiver deliberately does NOT apply it. This is explicit, honest
+7
View File
@@ -30,6 +30,8 @@ PipelineContextSender* pipeline_context_sender_create(Config* config, Queue* que
context->max_queue_bytes = 0;
context->manifest = NULL;
context->excluded_paths = NULL;
context->size_skipped_paths = NULL;
context->synced_dirs = NULL;
context->missing_args = NULL;
context->scan_had_io_error = false;
context->remove_source_files = NULL;
@@ -44,6 +46,7 @@ PipelineContextSender* pipeline_context_sender_create(Config* config, Queue* que
protocol_session_set_max_alloc(&context->allocation_session, config->max_alloc);
context->dir_entries = NULL;
context->dir_entries_mutex_init = false;
context->delete_limit = false;
int init = 0;
if (config->use_metadata) {
context->dir_entries = array_list_create(file_destroy);
@@ -186,6 +189,10 @@ void pipeline_context_sender_destroy(PipelineContextSender* context) {
}
if (context->excluded_paths)
array_list_delete(context->excluded_paths);
if (context->size_skipped_paths)
array_list_delete(context->size_skipped_paths);
if (context->synced_dirs)
array_list_delete(context->synced_dirs);
if (context->missing_args)
array_list_delete(context->missing_args);
if (context->remove_source_files)
+16
View File
@@ -42,6 +42,18 @@ typedef struct {
scanner's exclusion sink) or, in the early modes, by the path-only pre-scan
on the calling thread before the pipeline starts. */
ArrayList* excluded_paths;
/* --max-size/--min-size pruned source paths. These are ALWAYS sent as
protected prefixes (even with --delete-excluded), so the destination
mirrors of size-skipped files survive --delete like rsync. Populated by
the scanner thread (workers append under mutex_scanner) or, in the early
modes, by the path-only pre-scan on the calling thread. */
ArrayList* size_skipped_paths;
/* Destination-relative paths of the directories the source scan synchronized
for this run (the receive root is the "." sentinel). Sent with the
manifest so the receiver confines its extras walk to them, matching rsync's
"delete only in synchronized directories" (notably for --files-from).
Populated by the scanner thread or the early pre-scan. */
ArrayList* synced_dirs;
/* --delete-missing-args: the destination-relative mirrors of the --files-from
entries that are missing under the source. Computed by the preflight on
the calling thread before the pipeline starts; the sender thread transmits
@@ -83,6 +95,10 @@ typedef struct {
ArrayList* dir_entries;
mtx_t dir_entries_mutex;
bool dir_entries_mutex_init;
/* Set by the sender thread when the receiver reported a --max-delete-capped
deletion (STATUS_DELETE_LIMIT): the transfer succeeded and the process must
exit 25 like rsync. Read by the caller after the sender thread is joined. */
bool delete_limit;
} PipelineContextSender;
/* `config` is borrowed and must outlive the context: destroy does NOT free it,
+8
View File
@@ -104,6 +104,10 @@ int protocol_get_io_timeout_sec(void) {
return session->io_timeout_sec > 0 ? session->io_timeout_sec : 0;
}
int protocol_server_io_timeout_sec(int client_timeout) {
return client_timeout > 0 ? client_timeout : SERVER_IO_TIMEOUT_SEC;
}
void protocol_session_set_max_alloc(ProtocolSession* session, unsigned long long max_alloc) {
if (!session)
session = bound_session ? bound_session : &legacy_io_session;
@@ -495,6 +499,10 @@ static const char* status_to_string(Status status) {
return "ERROR_DETAIL";
case STATUS_DRY_RUN_TRANSFER:
return "DRY_RUN_TRANSFER";
case STATUS_DELETE_LIMIT:
return "DELETE_LIMIT";
case STATUS_DEST_INFO:
return "DEST_INFO";
default:
return "UNKNOWN";
}
+43 -12
View File
@@ -34,6 +34,11 @@
#define DEFAULT_MAX_ALLOC (1ULL * 1024 * 1024 * 1024)
/* Server policy ceiling for a client-provided allocation limit. */
#define MAX_SERVER_ALLOC (256ULL * 1024 * 1024)
/* Server-owned floor for the per-message I/O deadline. A client --timeout=0
(rsync's default) disables the client's own deadlines, but a server session
must never be held open forever by a silent peer (slow-loris), so the server
floors the effective deadline at this value. */
#define SERVER_IO_TIMEOUT_SEC 60
/* Bounded cumulative per-connection receive budget. In-flight wire buffers,
decompression buffers and queued (not yet written) file payloads for a
connection must stay within this ceiling. */
@@ -59,10 +64,12 @@ typedef struct ProtocolSession {
bool eight_bit_output;
unsigned long long max_alloc;
/* Per-session deadline (seconds) applied to every protocol send/receive by
* protocol_send_n_data / protocol_receive_n_data. Defaults to the built-in
* 60 s window; a value <= 0 falls back to that default. Set from the
* negotiated Config->timeout so --timeout is honored by the poll()-driven
* protocol I/O, not just the socket SO_RCVTIMEO/SO_SNDTIMEO. */
* protocol_send_n_data / protocol_receive_n_data. The initialized default is
* the built-in 60 s window; a value <= 0 disables the deadline (rsync's
* --timeout=0). Set from the negotiated Config->timeout so --timeout is
* honored by the poll()-driven protocol I/O, not just the socket
* SO_RCVTIMEO/SO_SNDTIMEO. The server does not propagate a client 0 here: it
* installs protocol_server_io_timeout_sec() so its sessions keep a floor. */
int io_timeout_sec;
} ProtocolSession;
@@ -155,7 +162,26 @@ enum NET_STATUS {
* (the receiver reads none in dry-run). STATUS_OK keeps its meaning in this
* path ("already up to date / nothing to do"). Appended after
* STATUS_ERROR_DETAIL so no existing status is renumbered. */
STATUS_DRY_RUN_TRANSFER
STATUS_DRY_RUN_TRANSFER,
/* --max-delete budget exhausted (protocol 2.23.0). Sent by the receiver as
* the terminal success status INSTEAD of STATUS_OK when a --delete/
* --delete-missing-args commit removed up to the --max-delete bound but had
* to skip further extras. The transfer itself succeeded and all file data is
* stored; the sender maps this to rsync's exit code 25 ("the --max-delete
* limit stopped deletions"). Appended after STATUS_DRY_RUN_TRANSFER so no
* existing status is renumbered. */
STATUS_DELETE_LIMIT,
/* Destination-state report for output parity (protocol 2.23.0). When the
* wire config carries report_dest_info=true, the receiver answers every
* per-file STATUS_CHECK request with STATUS_DEST_INFO FIRST, followed by a
* fixed record describing the pre-transfer destination entry
* (int32 has_old; uint64 size; int64 mtime; int64 mtime_nsec; uint32 mode;
* int32 uid; int32 gid). The ordinary STATUS_OK/STATUS_NEXT/... verdict
* follows, so the sender can render rsync-accurate -i/--out-format columns
* (new vs modified, and which of size/time/perms/owner/group differ) without
* changing the transfer decision itself. Appended after
* STATUS_DELETE_LIMIT so no existing status is renumbered. */
STATUS_DEST_INFO
};
void io_set_fds(int read_fd, int write_fd);
@@ -170,15 +196,20 @@ void protocol_session_unbind(void);
void protocol_session_set_ssl(ProtocolSession* session, SSL* ssl);
void protocol_session_set_bwlimit(ProtocolSession* session, unsigned long long bytes_per_sec);
void protocol_session_set_max_alloc(ProtocolSession* session, unsigned long long max_alloc);
/* Override the per-message send/receive deadline for this session.
* `sec` <= 0 restores the built-in 60 s default (used for --timeout=0/unset).
* An explicit long deadline (e.g. the delete-ack wait) is applied per-call by
* protocol_receive_status_timed and is unaffected by this setter. */
/* Override the per-message send/receive deadline for this session. The value
* is stored verbatim: a positive value sets the deadline, `sec` <= 0 disables
* it (rsync's --timeout=0). An explicit long deadline (e.g. the delete-ack
* wait) is applied per-call by protocol_receive_status_timed and is unaffected
* by this setter. */
void protocol_session_set_io_timeout(ProtocolSession* session, int sec);
/* Effective per-message I/O deadline (seconds) for the currently-bound session,
* falling back to the built-in default. Used by the plaintext sendfile path
* which bypasses the protocol send primitive. */
/* Effective per-message I/O deadline (seconds) for the currently-bound session.
* Zero means the deadline is disabled (rsync's --timeout=0). Used by the
* plaintext sendfile path which bypasses the protocol send primitive. */
int protocol_get_io_timeout_sec(void);
/* The server-side effective deadline for a client-requested timeout: a positive
* client value is honored, otherwise the SERVER_IO_TIMEOUT_SEC floor applies so
* a silent peer can never hold a session open forever. */
int protocol_server_io_timeout_sec(int client_timeout);
void* protocol_alloc(size_t size);
void* protocol_realloc(void* ptr, size_t size);
void protocol_session_set_8_bit_output(ProtocolSession* session, bool enabled);
+109 -165
View File
@@ -584,22 +584,41 @@ bool path_under_skip_prefix(const char* child_rel, bool at_root, const DeleteSki
return false;
}
/* All-or-nothing max-delete needs to know BEFORE any unlink whether the run
would delete more than max_delete entries. This rehearsal pass walks the
destination with the same decisions as the delete pass but never touches the
filesystem: it counts every regular file the delete pass would unlink and
every directory it would rmdir (a directory is removed only once every entry
below it has been removed and nothing the walker leaves in place survives).
Entries the walker never removes (symlinks, manifest-listed files, protected
prefixes) mark the enclosing directory as surviving, exactly as they would
make a real rmdir fail with ENOTEMPTY. Stops early once *count reaches the
cap (sets *exceeds). Returns false on a traversal error. */
static bool count_extras_fd(int dirfd, const char* rel_path, const PathIndex* keep, size_t cap,
size_t* count, bool* exceeds, const DeleteSkipEntry* skips,
int skip_count, bool* survives) {
/* Per-run deletion budget and tallies. `max_delete` is the cap on the number
of entries the walker may remove (SIZE_MAX = unlimited); once it is reached
the remaining extras are counted in `skipped` and left in place, matching
rsync's partial --max-delete behavior. */
typedef struct {
size_t max_delete;
size_t deleted;
size_t skipped;
bool limit_hit;
} DeleteBudget;
/* True when direct children of the directory named by `rel` may be removed.
With no synchronization info (dirs == NULL) the whole tree is deletable; when
a dirs index is supplied only its exact entries are (the receive root is the
"." sentinel). */
static bool is_synced_dir(const PathIndex* dirs, const char* rel) {
if (!dirs)
return true;
return path_index_contains(dirs, rel[0] == '\0' ? "." : rel);
}
/* Remove the extras directly inside the directory open on `dirfd`, recursing
into every child directory so kept content below a synchronized prefix is
reached. `all_removed` reports whether every child entry was removed (so the
caller may rmdir this directory). A child directory is never removed when it
is itself a synchronized directory or holds kept content; with a dirs index
supplied, direct children of a non-synchronized directory are never extras at
all (they are left in place but still descended into). Symlinks are unlinked
like any other non-directory extra (never followed). */
static bool delete_extras_fd(int dirfd, const char* rel_path, const PathIndex* keep,
const PathIndex* dirs, DeleteBudget* budget,
const DeleteSkipEntry* skips, int skip_count, bool parent_deletable,
bool* all_removed) {
/* openat(dirfd, ".") opens an independent file description: a dup() would
share dirfd's file offset, and a prior rehearsal pass must not have drained
this directory's stream before the delete pass reads it again. */
share dirfd's file offset and a prior pass could leave the stream drained. */
int scanfd = openat(dirfd, ".", O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
if (scanfd < 0)
return false;
@@ -610,93 +629,10 @@ static bool count_extras_fd(int dirfd, const char* rel_path, const PathIndex* ke
}
bool operation_ok = true;
bool local_survives = false;
bool at_root = rel_path[0] == '\0';
const struct dirent* entry;
while ((entry = readdir(dir)) != NULL) {
if (strcmp(entry->d_name, ".") == 0 || strcmp(entry->d_name, "..") == 0)
continue;
if (*exceeds)
break;
char* child_rel = path_cat((char*)rel_path, entry->d_name);
if (!child_rel) {
operation_ok = false;
continue;
}
if (path_under_skip_prefix(child_rel, at_root, skips, skip_count)) {
local_survives = true;
free(child_rel);
continue;
}
struct stat st;
if (fstatat(dirfd, entry->d_name, &st, AT_SYMLINK_NOFOLLOW) != 0) {
if (errno != ENOENT)
operation_ok = false;
free(child_rel);
continue;
}
if (S_ISLNK(st.st_mode)) {
local_survives = true;
free(child_rel);
continue;
}
if (S_ISDIR(st.st_mode)) {
int childfd = openat(dirfd, entry->d_name, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
bool child_ok = true;
bool child_survives = true;
if (childfd >= 0) {
child_ok = count_extras_fd(childfd, child_rel, keep, cap, count, exceeds, skips, skip_count,
&child_survives);
close(childfd);
} else if (errno != ENOENT) {
operation_ok = false;
}
if (!child_ok)
operation_ok = false;
if (keep_is_dir(keep, child_rel)) {
/* A directory with kept content below it is never removed. */
local_survives = true;
} else if (child_survives) {
/* The directory still holds entries the walker leaves in place, so an
rmdir would fail with ENOTEMPTY; the delete pass leaves it behind
rather than reporting an error (matching rsync). */
local_survives = true;
} else {
if (*count >= cap) {
*exceeds = true;
} else {
(*count)++;
}
}
} else {
bool found = keep_is_file(keep, child_rel);
if (!found) {
if (*count >= cap) {
*exceeds = true;
} else {
(*count)++;
}
}
}
free(child_rel);
}
closedir(dir);
*survives = local_survives;
return operation_ok;
}
static bool delete_extras_fd(int dirfd, const char* rel_path, const PathIndex* keep,
size_t max_delete, size_t* deleted_count, const DeleteSkipEntry* skips,
int skip_count) {
/* Independent file description (see count_extras_fd). */
int scanfd = openat(dirfd, ".", O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
if (scanfd < 0)
return false;
DIR* dir = fdopendir(scanfd);
if (!dir) {
close(scanfd);
return false;
}
bool operation_ok = true;
/* A directory is deletable when it or ANY ancestor is synchronized; the
`parent_deletable` flag carries that down the recursion so dest-only
directories below a synchronized root are removed wholesale. */
bool deletable = parent_deletable || is_synced_dir(dirs, rel_path);
const struct dirent* entry;
while ((entry = readdir(dir)) != NULL) {
if (strcmp(entry->d_name, ".") == 0 || strcmp(entry->d_name, "..") == 0)
@@ -714,6 +650,7 @@ static bool delete_extras_fd(int dirfd, const char* rel_path, const PathIndex* k
destination directory that happens to be called .fastsync-stage is
ordinary content. */
if (path_under_skip_prefix(child_rel, rel_path[0] == '\0', skips, skip_count)) {
local_survives = true;
free(child_rel);
continue;
}
@@ -724,55 +661,58 @@ static bool delete_extras_fd(int dirfd, const char* rel_path, const PathIndex* k
free(child_rel);
continue;
}
// Skip symlinks to prevent following them outside the destination tree
if (S_ISLNK(st.st_mode)) {
free(child_rel);
continue;
}
if (S_ISDIR(st.st_mode)) {
int childfd = openat(dirfd, entry->d_name, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
bool child_removed = false;
bool child_all_removed = false;
if (childfd >= 0) {
child_removed = delete_extras_fd(childfd, child_rel, keep, max_delete, deleted_count, skips,
skip_count);
if (!child_removed)
if (!delete_extras_fd(childfd, child_rel, keep, dirs, budget, skips, skip_count, deletable,
&child_all_removed))
operation_ok = false;
close(childfd);
} else if (errno != ENOENT) {
operation_ok = false;
}
if (child_removed && !keep_is_dir(keep, child_rel)) {
if (*deleted_count >= max_delete) {
operation_ok = false;
bool child_synced = dirs && path_index_contains(dirs, child_rel);
if (child_synced || keep_is_dir(keep, child_rel)) {
/* A synchronized directory and a directory holding kept content are
never removed. */
local_survives = true;
} else if (child_all_removed && deletable) {
if (budget->deleted >= budget->max_delete) {
budget->limit_hit = true;
budget->skipped++;
local_survives = true;
} else if (unlinkat(dirfd, entry->d_name, AT_REMOVEDIR) != 0) {
/* ENOENT: already gone (fine). ENOTEMPTY/EEXIST: the directory
still holds entries the walker leaves in place (a protected
excluded prefix, a kept file the manifest protects, a symlink);
rsync leaves such a directory behind, so this is not an error.
Only genuine I/O failures abort the deletion. */
if (errno != ENOENT && errno != ENOTEMPTY && errno != EEXIST)
operation_ok = false;
local_survives = true;
} else {
if (unlinkat(dirfd, entry->d_name, AT_REMOVEDIR) != 0) {
/* ENOENT: already gone (fine). ENOTEMPTY/EEXIST: the directory
still holds entries the walker leaves in place (a protected
excluded prefix, a kept file the manifest protects, a symlink);
rsync leaves such a directory behind, so this is not an error.
Only genuine I/O failures abort the deletion. */
if (errno != ENOENT && errno != ENOTEMPTY && errno != EEXIST)
operation_ok = false;
} else {
(*deleted_count)++;
}
budget->deleted++;
}
} else {
local_survives = true;
}
} else {
// Check if relative path is in manifest
bool found = keep_is_file(keep, child_rel);
if (!found) {
if (*deleted_count >= max_delete) {
if (found || !deletable) {
/* Kept file, or a child of a directory that is not synchronized: never
an extra for this run. */
local_survives = true;
} else if (budget->deleted >= budget->max_delete) {
budget->limit_hit = true;
budget->skipped++;
local_survives = true;
} else if (unlinkat(dirfd, entry->d_name, 0) != 0) {
if (errno != ENOENT)
operation_ok = false;
free(child_rel);
continue;
}
if (unlinkat(dirfd, entry->d_name, 0) != 0) {
if (errno != ENOENT)
operation_ok = false;
} else {
(*deleted_count)++;
}
local_survives = true;
} else {
budget->deleted++;
char* escaped_path = output_escape(child_rel, log_get_8_bit_output());
fprintf(stderr, " Deleted: %s\n", escaped_path ? escaped_path : "<allocation failed>");
free(escaped_path);
@@ -781,21 +721,33 @@ static bool delete_extras_fd(int dirfd, const char* rel_path, const PathIndex* k
free(child_rel);
}
closedir(dir);
*all_removed = !local_survives;
return operation_ok;
}
DeleteWalkResult delete_extras_limited(const char* dest_root, const ArrayList* manifest,
size_t max_delete, const DeleteSkipEntry* skips,
int skip_count, size_t* deleted_out) {
const ArrayList* synced_dirs, size_t max_delete,
const DeleteSkipEntry* skips, int skip_count,
size_t* deleted_out, size_t* skipped_out) {
if (deleted_out)
*deleted_out = 0;
if (skipped_out)
*skipped_out = 0;
if (!manifest)
return DELETE_WALK_ERROR;
/* Index the keep-set once so both passes answer membership in O(path length)
instead of scanning every manifest entry for every destination entry. */
/* Index the keep-set (and the synchronized-dir set, when supplied) once so
membership is answered in O(path length) instead of scanning every entry
for every destination entry. */
PathIndex keep;
if (!build_keep_index(manifest, &keep))
return DELETE_WALK_ERROR;
PathIndex dirs;
bool have_dirs = synced_dirs != NULL;
if (have_dirs &&
!path_index_build(&dirs, (const char* const*)synced_dirs->items, (size_t)synced_dirs->size)) {
path_index_free(&keep);
return DELETE_WALK_ERROR;
}
int rootfd;
int root_fd = utils_get_authorized_root_fd();
if (root_fd >= 0) {
@@ -810,39 +762,31 @@ DeleteWalkResult delete_extras_limited(const char* dest_root, const ArrayList* m
}
if (rootfd < 0) {
path_index_free(&keep);
if (have_dirs)
path_index_free(&dirs);
return DELETE_WALK_ERROR;
}
if (max_delete != SIZE_MAX) {
/* Rehearse the deletion first so a run that would exceed the cap removes
nothing (rsync's all-or-nothing --max-delete contract). */
size_t count = 0;
bool exceeds = false;
bool survives = false;
bool counted_ok = count_extras_fd(rootfd, "", &keep, max_delete, &count, &exceeds, skips,
skip_count, &survives);
if (!counted_ok) {
close(rootfd);
path_index_free(&keep);
return DELETE_WALK_ERROR;
}
if (exceeds) {
close(rootfd);
path_index_free(&keep);
return DELETE_WALK_LIMIT_EXCEEDED;
}
}
size_t deleted_count = 0;
bool ok = delete_extras_fd(rootfd, "", &keep, max_delete, &deleted_count, skips, skip_count);
DeleteBudget budget = {.max_delete = max_delete, .deleted = 0, .skipped = 0, .limit_hit = false};
bool all_removed = false;
bool ok = delete_extras_fd(rootfd, "", &keep, have_dirs ? &dirs : NULL, &budget, skips,
skip_count, false, &all_removed);
if (close(rootfd) != 0)
ok = false;
path_index_free(&keep);
if (have_dirs)
path_index_free(&dirs);
if (deleted_out)
*deleted_out = deleted_count;
return ok ? DELETE_WALK_OK : DELETE_WALK_ERROR;
*deleted_out = budget.deleted;
if (skipped_out)
*skipped_out = budget.skipped;
if (!ok)
return DELETE_WALK_ERROR;
return budget.limit_hit ? DELETE_WALK_LIMIT_REACHED : DELETE_WALK_OK;
}
bool delete_extras(const char* dest_root, const ArrayList* manifest) {
return delete_extras_limited(dest_root, manifest, SIZE_MAX, NULL, 0, NULL) == DELETE_WALK_OK;
return delete_extras_limited(dest_root, manifest, NULL, SIZE_MAX, NULL, 0, NULL, NULL) ==
DELETE_WALK_OK;
}
bool has_path_traversal(const char* path) {
+19 -16
View File
@@ -98,10 +98,10 @@ bool glob_match(const char* pattern, const char* str);
typedef enum {
/* Every extra entry was removed (or there were none). */
DELETE_WALK_OK = 0,
/* The destination holds more extras than the numeric cap for this run. With
the all-or-nothing max-delete semantics NOTHING was removed (the walker
counts first and refuses to start when the run would exceed the limit). */
DELETE_WALK_LIMIT_EXCEEDED,
/* The numeric cap for this run was reached before every extra was removed.
The walker removed exactly the entries the cap allowed and skipped (without
removing) the rest, matching rsync's partial --max-delete behavior. */
DELETE_WALK_LIMIT_REACHED,
/* A traversal or unlink failure aborted the deletion (partial removal is
possible, mirroring the delete pass). */
DELETE_WALK_ERROR
@@ -122,19 +122,22 @@ typedef struct {
only DIRECT children of the destination root, i.e. child_rel has no '/'). */
bool path_under_skip_prefix(const char* child_rel, bool at_root, const DeleteSkipEntry* skips,
int skip_count);
/* Remove files/dirs under dest_root that are not listed in manifest without
ever descending into a protected prefix (see DeleteSkipEntry). When
max_delete is not SIZE_MAX the run is all-or-nothing: extras are counted
first and DELETE_WALK_LIMIT_EXCEEDED is returned (with nothing removed) when
the count would exceed the cap. `deleted_out` optionally receives the number
of entries actually removed. The all-or-nothing guarantee holds only while
the destination tree is not being concurrently modified: the rehearsal pass
and the delete pass are two separate walks, so a concurrent change between
them (another process adding/removing entries) can make the second pass
delete a different set than the first one counted. */
/* Remove files/dirs/symlinks under dest_root that are not listed in manifest
without ever descending into a protected prefix (see DeleteSkipEntry). When
`synced_dirs` is non-NULL, extras are only removed directly inside a directory
whose destination-relative path is an exact entry in that list (the receive
root is the "." sentinel); directories outside the synchronized set are still
descended into so kept content below a listed directory is preserved, but
nothing in them is removed. A NULL `synced_dirs` keeps the legacy behavior of
treating the whole destination tree as deletable. `max_delete` caps the
number of removed entries (SIZE_MAX = unlimited): the walker removes up to the
cap and returns DELETE_WALK_LIMIT_REACHED when more extras remained.
`deleted_out`/`skipped_out` optionally receive the number of entries removed
and the number skipped because of the cap. */
DeleteWalkResult delete_extras_limited(const char* dest_root, const ArrayList* manifest,
size_t max_delete, const DeleteSkipEntry* skips,
int skip_count, size_t* deleted_out);
const ArrayList* synced_dirs, size_t max_delete,
const DeleteSkipEntry* skips, int skip_count,
size_t* deleted_out, size_t* skipped_out);
bool delete_extras(const char* dest_root, const ArrayList* manifest);
bool utils_set_authorized_root(int fd, const char* canonical_path);
/* The fd-only compatibility form is fail-closed for path-based operations;
+27 -39
View File
@@ -38,6 +38,22 @@ void xattr_list_free(FileXattrList* list) {
free(list);
}
FileXattrList* xattr_list_clone(const FileXattrList* list) {
if (!list)
return NULL;
FileXattrList* clone = xattr_list_new();
if (!clone)
return NULL;
for (int i = 0; i < list->count; i++) {
if (!xattr_list_append(clone, list->items[i].name, list->items[i].value,
list->items[i].value_len)) {
xattr_list_free(clone);
return NULL;
}
}
return clone;
}
bool xattr_list_append(FileXattrList* list, const char* name, const void* value, size_t value_len) {
if (!list || !name || (!value && value_len != 0))
return false;
@@ -365,29 +381,11 @@ void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, int6
}
}
/* --fake-super replay: read the freshly-stored record and re-apply the source
* stat fd-relative. A privileged (root) run can actually change the owner;
* a non-root run silently skips the fchown on EPERM/EACCES (never fatal,
* mirroring the normal metadata identity path; other errors are logged) and
* still applies mode/mtime where permitted.
*
* The OWNER leg additionally honors three policies:
* - an ownership identity policy must be active: the explicit flags
* (--numeric-ids / --chown / --usermap / --groupmap / --copy-as) OR the
* preserve-source -o/--owner / -g/--group requests. --fake-super on its own
* only RECORDS the source owner; replaying that owner as a live chown
* without an ownership opt-in would be an un-gated client-chosen-ownership
* primitive. The owner and group sides are applied INDEPENDENTLY (through
* identity_owner_requested()/identity_group_requested()), so a plain -o or
* -g touches only the requested side and passes (uid_t)-1 / (gid_t)-1 for
* the other.
* - --no-super (privilege_super_permitted() false) suppresses it even for a
* root receiver, exactly like the normal metadata identity path.
* - an active --copy-as is AUTHORITATIVE: the identity path already forced the
* target owner, so replaying the recorded source owner here would silently
* override it. The xattr record is still stored/replayed for a later
* privileged restore; only the live chown is skipped. Mode/mtime remain
* applied either way so unprivileged --fake-super still works. */
/* --fake-super replay: read the freshly-stored record and re-apply mode/mtime
* fd-relative. The recorded uid/gid are retained for a later privileged
* restore but are NEVER chowned here: --fake-super only RECORDS ownership, it
* must not real-chown the recorded (resolved) owner. Mode/mtime still apply so
* unprivileged --fake-super keeps working. */
bool fake_super_restore_fd(int fd, FileAttrPolicy policy) {
if (fd < 0)
return false;
@@ -403,22 +401,12 @@ bool fake_super_restore_fd(int fd, FileAttrPolicy policy) {
5)
return false; /* malformed record: skip, never fatal */
/* Owner is applied best-effort only: a non-root process cannot chown and
must not abort the transfer for that reason (FastSync identity philosophy).
EPERM/EACCES (expected for a non-root receiver) are skipped silently; a
genuine EINVAL (an impossible stored id) is logged so the corruption is
not hidden. --no-super suppresses the owner leg even for root, and an
active --copy-as is authoritative so its forced owner must not be
overwritten by the recorded source owner. */
if (identity_active_enabled() && privilege_super_permitted() && !identity_copy_as_active()) {
/* Apply only the requested side(s): an unchosen side is passed as -1 so the
* kernel leaves it exactly as-is. */
uid_t owner = identity_owner_requested() ? (uid_t)ul_uid : (uid_t)-1;
gid_t group = identity_group_requested() ? (gid_t)ul_gid : (gid_t)-1;
if (fchown(fd, owner, group) != 0 && errno != EPERM && errno != EACCES)
log_message(LOG_LEVEL_WARNING,
"--fake-super: could not restore owner on destination file: %s", strerror(errno));
}
/* --fake-super NEVER performs a real chown: that would defeat the whole
point of the flag (record privileged ownership on an unprivileged receiver
for a later privileged restore). The uid/gid parsed above are retained in
the record for that later restore, but no ownership change happens here. */
(void)ul_uid;
(void)ul_gid;
/* Mode is applied only when the per-attribute policy asks for it, through the
SAME shared helper the normal metadata path uses (metadata_mode_for_policy):
group/other write bits are never granted, so a recorded source mode of 0666
+12 -11
View File
@@ -56,6 +56,8 @@ typedef struct {
FileXattrList* xattr_list_new(void);
void xattr_list_free(FileXattrList* list);
/* Deep-copy `list` (NULL in, NULL out). Returns NULL on allocation failure. */
FileXattrList* xattr_list_clone(const FileXattrList* list);
/* Append one entry (deep copy). Returns false on allocation failure. */
bool xattr_list_append(FileXattrList* list, const char* name, const void* value, size_t value_len);
@@ -96,17 +98,16 @@ void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, int6
int64_t mtime_nsec);
/* --fake-super replay: parse the FAKESUPER_XATTR record previously written on
* `fd` by fake_super_store_fd and re-apply uid/gid/mode/mtime fd-relative.
* Best-effort: absence of the xattr or a malformed record is a silent no-op
* that never fails the transfer. The OWNER leg is applied only when an explicit
* ownership identity policy is active (numeric-ids/chown/usermap/groupmap/
* copy-as/-o/-g), when super-user activities are permitted, and when --copy-as
* is not authoritative; a non-root EPERM/EACCES is skipped silently, matching
* FastSync's identity philosophy. The MODE leg is applied only when
* policy.perms||policy.executability and the MTIME leg only when policy.times,
* so the fake-super replay cannot bypass the per-attribute split; the mode is
* sanitized exactly like the normal metadata path (group/other write bits never
* granted). Returns true when the xattr was present and parsed. */
* `fd` by fake_super_store_fd and re-apply mode/mtime fd-relative. The
* recorded uid/gid are deliberately NOT chowned for real: --fake-super only
* RECORDS ownership (the caller stores the resolved mapping via
* identity_resolve_storage_ids), it never performs a real chown. Best-effort:
* absence of the xattr or a malformed record is a silent no-op that never fails
* the transfer. The MODE leg is applied only when policy.perms||policy.
* executability and the MTIME leg only when policy.times, so the fake-super
* replay cannot bypass the per-attribute split; the mode is sanitized exactly
* like the normal metadata path (group/other write bits never granted).
* Returns true when the xattr was present and parsed. */
bool fake_super_restore_fd(int fd, FileAttrPolicy policy);
#endif
+2
View File
@@ -115,7 +115,9 @@ static void build_canonical_frame(void) {
if (cfg->usermap) {
cfg->usermap_count = 1;
cfg->usermap[0].from = MAP_FROM;
cfg->usermap[0].from_hi = MAP_FROM;
cfg->usermap[0].to = MAP_TO;
cfg->usermap[0].to_name = NULL;
}
if (!cfg->send_directory || !cfg->receive_root_directory || !cfg->usermap) {
config_delete(cfg);
+7
View File
@@ -254,6 +254,13 @@ def _wait_for_port(port, timeout=5):
def _wait_proc(proc, timeout=5):
"""Stop a long-lived subprocess promptly. The server installs a SIGTERM
handler, so signal first and only escalate to SIGKILL if it does not exit;
waiting without signalling would burn the full timeout on every stop."""
if proc.poll() is not None:
proc.wait()
return
proc.terminate()
try:
proc.wait(timeout=timeout)
except subprocess.TimeoutExpired:
+1 -1
View File
@@ -36,7 +36,7 @@ from common import ( # noqa: E402
verify_transfer,
)
PROTOCOL_VERSION = b"2.22.0"
PROTOCOL_VERSION = b"2.23.0"
STATUS_MANIFEST = 5
STATUS_OK = 0
File diff suppressed because it is too large. Load diff
+282
View File
@@ -0,0 +1,282 @@
"""Output-parity tests (#291 selection/output, #292 output formatting).
These tests exercise rsync-style selection ordering and output formatting. The
differential tests run the SAME transfer with real ``rsync 3.4.1`` and with
fastsync and compare stdout, so they are skipped when rsync is unavailable.
"""
import os
import shutil
import subprocess
import sys
import pytest
sys.path.insert(0, os.path.dirname(__file__))
from common import TEST_DATA_DIR, run_client, clean_dir, get_dest_received_dir
RSYNC = shutil.which("rsync")
requires_rsync = pytest.mark.skipif(RSYNC is None, reason="rsync 3.4.1 not installed")
def _rsync(args):
env = dict(os.environ, LC_ALL="C")
return subprocess.run(
[RSYNC] + args, capture_output=True, text=True, env=env, timeout=120
)
def _make_selection_tree(root):
clean_dir(root)
os.makedirs(os.path.join(root, "sub"))
with open(os.path.join(root, "a.txt"), "wb") as fh:
fh.write(b"top text\n")
with open(os.path.join(root, "b.log"), "wb") as fh:
fh.write(b"log data\n")
with open(os.path.join(root, "sub", "c.txt"), "wb") as fh:
fh.write(b"nested text\n")
with open(os.path.join(root, "sub", "d.log"), "wb") as fh:
fh.write(b"nested log\n")
class TestSelectionOrdering:
"""#291: --include/--exclude compile into one ordered rule list."""
@pytest.mark.ci
def test_include_then_exclude_keeps_only_matching(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "out_inc_src")
dest = os.path.join(TEST_DATA_DIR, "out_inc_dst")
_make_selection_tree(source)
clean_dir(dest)
result, _ = run_client(
source, dest,
flags=["--preserve", "--include=*.txt", "--exclude=*"],
port=shared_server.port,
)
assert result.returncode == 0, f"include/exclude failed: {result.stderr[:300]}"
received = get_dest_received_dir(dest, source)
assert os.path.exists(os.path.join(received, "a.txt"))
# `*` also excludes the directory, so nothing below sub/ is sent.
assert not os.path.exists(os.path.join(received, "b.log"))
assert not os.path.exists(os.path.join(received, "sub", "c.txt"))
@pytest.mark.ci
def test_include_dirs_then_files_idiom(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "out_inc2_src")
dest = os.path.join(TEST_DATA_DIR, "out_inc2_dst")
_make_selection_tree(source)
clean_dir(dest)
result, _ = run_client(
source, dest,
flags=["--preserve", "--include=*/", "--include=*.txt", "--exclude=*"],
port=shared_server.port,
)
assert result.returncode == 0, f"include/exclude failed: {result.stderr[:300]}"
received = get_dest_received_dir(dest, source)
assert os.path.exists(os.path.join(received, "a.txt"))
assert os.path.exists(os.path.join(received, "sub", "c.txt"))
assert not os.path.exists(os.path.join(received, "b.log"))
assert not os.path.exists(os.path.join(received, "sub", "d.log"))
@requires_rsync
def test_include_idiom_matches_rsync_selection(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "out_inc3_src")
dest = os.path.join(TEST_DATA_DIR, "out_inc3_dst")
rdst = os.path.join(TEST_DATA_DIR, "out_inc3_rdst")
_make_selection_tree(source)
clean_dir(dest)
clean_dir(rdst)
flags = ["--include=*/", "--include=*.txt", "--exclude=*"]
rsync_result = _rsync(["-a"] + flags + [source + "/", rdst + "/"])
assert rsync_result.returncode == 0, rsync_result.stderr
result, _ = run_client(source, dest, flags=["--preserve"] + flags,
port=shared_server.port)
assert result.returncode == 0
received = get_dest_received_dir(dest, source)
assert os.path.exists(os.path.join(received, "a.txt"))
assert os.path.exists(os.path.join(received, "sub", "c.txt"))
assert not os.path.exists(os.path.join(received, "b.log"))
# rsync -a src/ dst/ writes directly into dst/
assert os.path.exists(os.path.join(rdst, "a.txt"))
assert os.path.exists(os.path.join(rdst, "sub", "c.txt"))
assert not os.path.exists(os.path.join(rdst, "b.log"))
class TestOneFileSystem:
"""#291: -x emits the mount-point directory but not its contents."""
def test_one_file_system_emits_mount_point_dir(self, shared_server):
local = os.stat(".")
shm = "/dev/shm"
try:
shm_stat = os.stat(shm)
except OSError:
pytest.skip("/dev/shm not available")
if shm_stat.st_dev == local.st_dev:
pytest.skip("no cross-device filesystem available")
source = os.path.join(TEST_DATA_DIR, "out_ofs_src")
dest = os.path.join(TEST_DATA_DIR, "out_ofs_dst")
clean_dir(source)
clean_dir(dest)
os.makedirs(os.path.join(source, "nested"))
os.makedirs(os.path.join(shm, "fastsync_ofs_probe"), exist_ok=True)
with open(os.path.join(source, "keep.txt"), "wb") as fh:
fh.write(b"keep\n")
with open(os.path.join(shm, "fastsync_ofs_probe", "inside.txt"), "wb") as fh:
fh.write(b"cross\n")
link = os.path.join(source, "nested", "link")
try:
os.symlink(os.path.join(shm, "fastsync_ofs_probe"), link)
except OSError:
pytest.skip("cannot create symlink")
try:
result, _ = run_client(
source, dest,
flags=["--preserve", "--copy-links", "-x"],
port=shared_server.port,
)
assert result.returncode == 0, f"-x failed: {result.stderr[:300]}"
received = get_dest_received_dir(dest, source)
assert os.path.exists(os.path.join(received, "keep.txt"))
# The mount-point directory entry is created but its contents are not.
assert os.path.isdir(os.path.join(received, "nested", "link"))
assert not os.path.exists(os.path.join(received, "nested", "link", "inside.txt"))
finally:
shutil.rmtree(os.path.join(shm, "fastsync_ofs_probe"), ignore_errors=True)
def _make_output_tree(root):
clean_dir(root)
os.makedirs(os.path.join(root, "sub"))
with open(os.path.join(root, "a.txt"), "wb") as fh:
fh.write(b"hello\n")
with open(os.path.join(root, "sub", "b.txt"), "wb") as fh:
fh.write("wörld\n".encode("utf-8"))
os.symlink("a.txt", os.path.join(root, "link"))
class TestItemizeParity:
"""#292: -i output matches rsync 3.4.1 for the cases fastsync can observe."""
@requires_rsync
@pytest.mark.ci
def test_itemize_first_transfer_matches_rsync(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "out_item_src")
dest = os.path.join(TEST_DATA_DIR, "out_item_dst")
rdst = os.path.join(TEST_DATA_DIR, "out_item_rdst")
_make_output_tree(source)
clean_dir(dest)
clean_dir(rdst)
rsync_result = _rsync(["-a", "-i", source + "/", rdst + "/"])
assert rsync_result.returncode == 0, rsync_result.stderr
rsync_lines = sorted(
line for line in rsync_result.stdout.splitlines()
if line.startswith(">f") or line.startswith("cL")
)
result, _ = run_client(source, dest, flags=["-a", "-i"],
port=shared_server.port)
assert result.returncode == 0, result.stderr[:300]
fast_lines = sorted(
line for line in result.stdout.splitlines()
if line.startswith(">f") or line.startswith("cL")
)
assert fast_lines == rsync_lines, f"rsync={rsync_lines} fastsync={fast_lines}"
@requires_rsync
@pytest.mark.ci
def test_itemize_modified_file_matches_rsync(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "out_item2_src")
dest = os.path.join(TEST_DATA_DIR, "out_item2_dst")
rdst = os.path.join(TEST_DATA_DIR, "out_item2_rdst")
_make_output_tree(source)
clean_dir(dest)
clean_dir(rdst)
seed = run_client(source, dest, flags=["-a"], port=shared_server.port)
assert seed[0].returncode == 0, seed[0].stderr[:300]
assert _rsync(["-a", source + "/", rdst + "/"]).returncode == 0
with open(os.path.join(source, "a.txt"), "wb") as fh:
fh.write(b"hello changed and longer\n")
# Pin the source mtime so rsync's `t` column is deterministic (a write
# that lands in the same whole second as the seed would not show `t`).
os.utime(os.path.join(source, "a.txt"), (1000000000, 1000000000))
rsync_result = _rsync(["-a", "-i", source + "/", rdst + "/"])
assert rsync_result.returncode == 0, rsync_result.stderr
rsync_lines = sorted(
line for line in rsync_result.stdout.splitlines() if line.startswith(">f")
)
result, _ = run_client(source, dest,
flags=["-a", "-i", "--incremental"],
port=shared_server.port)
assert result.returncode == 0, result.stderr[:300]
fast_lines = sorted(
line for line in result.stdout.splitlines() if line.startswith(">f")
)
assert fast_lines == rsync_lines, f"rsync={rsync_lines} fastsync={fast_lines}"
class TestOutFormatParity:
@requires_rsync
@pytest.mark.ci
def test_out_format_n_l_matches_rsync(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "out_fmt_src")
dest = os.path.join(TEST_DATA_DIR, "out_fmt_dst")
rdst = os.path.join(TEST_DATA_DIR, "out_fmt_rdst")
_make_output_tree(source)
clean_dir(dest)
clean_dir(rdst)
fmt = "%n %l"
rsync_result = _rsync(["-a", "--out-format=" + fmt, source + "/", rdst + "/"])
assert rsync_result.returncode == 0, rsync_result.stderr
rsync_lines = sorted(
line for line in rsync_result.stdout.splitlines()
if line and not line.split(" ", 1)[0].endswith("/")
)
result, _ = run_client(source, dest,
flags=["-a", "--out-format=" + fmt],
port=shared_server.port)
assert result.returncode == 0, result.stderr[:300]
fast_lines = sorted(
line for line in result.stdout.splitlines()
if line and not line.split(" ", 1)[0].endswith("/")
)
assert fast_lines == rsync_lines, f"rsync={rsync_lines} fastsync={fast_lines}"
@requires_rsync
@pytest.mark.ci
def test_out_format_M_datetime_shape(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "out_M_src")
dest = os.path.join(TEST_DATA_DIR, "out_M_dst")
_make_output_tree(source)
clean_dir(dest)
result, _ = run_client(source, dest,
flags=["-a", "--out-format=%M %f"],
port=shared_server.port)
assert result.returncode == 0, result.stderr[:300]
import re
pattern = re.compile(r"^\d{4}/\d{2}/\d{2}-\d{2}:\d{2}:\d{2} ")
for line in result.stdout.splitlines():
if line:
assert pattern.match(line), f"bad %M format: {line!r}"
class TestListOnlyParity:
@requires_rsync
@pytest.mark.ci
def test_list_only_matches_rsync(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "out_list_src")
dest = os.path.join(TEST_DATA_DIR, "out_list_dst")
_make_output_tree(source)
clean_dir(dest)
rsync_result = _rsync(["-r", "--list-only", source + "/"])
assert rsync_result.returncode == 0, rsync_result.stderr
rsync_lines = sorted(rsync_result.stdout.splitlines())
result, _ = run_client(source, dest, flags=["--list-only", "-l"],
port=shared_server.port)
assert result.returncode == 0, result.stderr[:300]
fast_lines = sorted(result.stdout.splitlines())
assert fast_lines == rsync_lines, (
f"rsync={rsync_lines}\nfastsync={fast_lines}"
)
+4 -4
View File
@@ -94,14 +94,14 @@ def _seed_protocol_source(source):
class TestProtocol:
@pytest.mark.ci
def test_protocol_current_version_accepted(self, shared_server):
"""--protocol=2.22.0 (the current PROTOCOL_VERSION) is accepted and the
"""--protocol=2.23.0 (the current PROTOCOL_VERSION) is accepted and the
transfer completes normally."""
source = os.path.join(TEST_DATA_DIR, "proto_ok_src")
dest = os.path.join(TEST_DATA_DIR, "proto_ok_dst")
shutil.rmtree(dest, ignore_errors=True)
os.makedirs(dest)
_seed_protocol_source(source)
result, _ = run_client(source, dest, flags=["--protocol=2.22.0"],
result, _ = run_client(source, dest, flags=["--protocol=2.23.0"],
port=shared_server.port)
assert result.returncode == 0, \
f"--protocol current run failed: {(result.stderr or result.stdout)[:400]}"
@@ -118,8 +118,8 @@ class TestProtocol:
shutil.rmtree(dest, ignore_errors=True)
os.makedirs(dest)
_seed_protocol_source(source)
for bad in ("2.21.0", "2.20.0", "2.19.0", "2.18.0", "2.17.0", "2.15.0", "2.16.0", "216",
"31"):
for bad in ("2.22.0", "2.21.0", "2.20.0", "2.19.0", "2.18.0", "2.17.0", "2.15.0", "2.16.0",
"216", "31"):
result, _ = run_client(source, dest, flags=[f"--protocol={bad}"],
port=shared_server.port)
assert result.returncode != 0, f"--protocol={bad} should be rejected"
+94 -28
View File
@@ -95,14 +95,14 @@ class TestPreservePerms:
source = os.path.join(TEST_DATA_DIR, "perms_nop_new_src")
dest = os.path.join(TEST_DATA_DIR, "perms_nop_new_dst")
# 0664 has group/other bits that the umask strips, so the result is not
# just the source mode. FastSync additionally never grants group/other
# write from a client-supplied mode (S_IWGRP|S_IWOTH are always
# cleared), so the expected mode masks those too.
# just the source mode. Under strict rsync parity the source mode is
# masked only by the umask (group/other write is no longer force-cleared
# on top of it).
_seed_file(source, dest, "f.txt", b"new\n", 0o664)
result, _ = run_client(source, dest, flags=["-t"], port=shared_server.port)
assert result.returncode == 0, f"-t failed: {(result.stderr or '')[:300]}"
want = 0o664 & ~_process_umask() & ~0o022
assert result.returncode == 0, f"-t failed: {(result.stderr or result.stdout)[:300]}"
want = 0o664 & ~_process_umask()
got = os.stat(_received(dest, source, "f.txt")).st_mode & 0o777
assert got == want, \
f"new no--p destination mode: want {oct(want)}, got {oct(got)}"
@@ -254,15 +254,15 @@ class TestDirectoryModes:
assert got == 0o750, f"-p must apply the source directory mode, got {oct(got)}"
@pytest.mark.ci
def test_p_sanitizes_directory_group_other_write(self, shared_server):
# A 0777 source directory must never produce a group/other-writable
# destination directory: the file-mode sanitization is applied to dirs.
source, dest, _ = self._tree("dirmode_sanitize", 0o777, pin_mtime=False)
def test_p_preserves_directory_group_other_write(self, shared_server):
# Strict rsync parity: -p copies the source directory mode exactly,
# including group/other write (the old sanitization is gone).
source, dest, _ = self._tree("dirmode_go_write", 0o777, pin_mtime=False)
result, _ = run_client(source, dest, flags=["-p"], port=shared_server.port)
assert result.returncode == 0, f"-p failed: {(result.stderr or result.stdout)[:300]}"
mode = os.stat(os.path.join(get_dest_received_dir(dest, source), "sub")).st_mode & 0o777
assert mode & 0o022 == 0, \
f"directory must never be group/other writable, got {oct(mode)}"
assert mode == 0o777, \
f"-p must preserve the source directory mode exactly, got {oct(mode)}"
@pytest.mark.ci
def test_omit_dir_times_suppresses_times_not_modes(self, shared_server):
@@ -340,16 +340,85 @@ class TestOwnershipRoot:
assert (st.st_uid, st.st_gid) == (33333, 44444), \
f"--chown must override -o, got uid={st.st_uid} gid={st.st_gid}"
def test_fake_super_o_does_not_change_group(self, shared_server):
# --fake-super replays the recorded source stat; with only -o requested
# it must apply the owner but leave the group untouched (MAJOR 1).
def test_fake_super_o_does_not_real_chown(self, shared_server):
# #294: --fake-super only RECORDS ownership; it must never real-chown the
# recorded source owner (that defeats the point of the flag). With -o the
# resolved owner is parked in the reserved xattr and the on-disk owner is
# left as the receiver's.
source, dest = self._seed_owned("fake_o", 12345, 54321)
result, _ = run_client(source, dest, flags=["--fake-super", "-o"],
port=shared_server.port)
assert result.returncode == 0, f"--fake-super -o failed: {(result.stderr or '')[:300]}"
dst = _received(dest, source, "f.txt")
st = os.stat(dst)
assert st.st_uid != 12345, \
f"--fake-super -o must NOT real-chown the source owner, got uid={st.st_uid}"
record = os.getxattr(dst, "user.fastsync.stat").decode()
fields = record.split(":")
assert fields[0] == "12345", \
f"--fake-super must record the resolved owner, got {fields[0]}"
def test_o_applies_directory_owner(self, shared_server):
"""#286.2: -o must apply the source owner to DIRECTORIES too (the
deferred directory-metadata application now runs the identity path)."""
source = os.path.join(TEST_DATA_DIR, "root_dir_o_src")
dest = os.path.join(TEST_DATA_DIR, "root_dir_o_dst")
clean_dir(source)
clean_dir(dest)
os.makedirs(os.path.join(source, "sub", "deep"))
with open(os.path.join(source, "sub", "deep", "f.txt"), "wb") as fh:
fh.write(b"dir owner\n")
os.chown(os.path.join(source, "sub"), 12345, 12346)
os.chown(os.path.join(source, "sub", "deep"), 23456, 34567)
result, _ = run_client(source, dest, flags=["-o", "-t"], port=shared_server.port)
assert result.returncode == 0, f"-o dir failed: {(result.stderr or '')[:300]}"
received = get_dest_received_dir(dest, source)
sub = os.stat(os.path.join(received, "sub"))
deep = os.stat(os.path.join(received, "sub", "deep"))
assert sub.st_uid == 12345, f"dir 'sub' owner not applied: {sub.st_uid}"
assert deep.st_uid == 23456, f"dir 'sub/deep' owner not applied: {deep.st_uid}"
# -o alone must not change the group.
assert sub.st_gid != 12346
def test_a_applies_directory_owner_and_group(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "root_dir_a_src")
dest = os.path.join(TEST_DATA_DIR, "root_dir_a_dst")
clean_dir(source)
clean_dir(dest)
os.makedirs(os.path.join(source, "sub"))
with open(os.path.join(source, "sub", "f.txt"), "wb") as fh:
fh.write(b"dir owner group\n")
os.chown(os.path.join(source, "sub"), 12345, 54321)
result, _ = run_client(source, dest, flags=["-a"], port=shared_server.port)
assert result.returncode == 0, f"-a dir failed: {(result.stderr or '')[:300]}"
received = get_dest_received_dir(dest, source)
st = os.stat(os.path.join(received, "sub"))
assert (st.st_uid, st.st_gid) == (12345, 54321), \
f"-a must apply dir owner+group, got uid={st.st_uid} gid={st.st_gid}"
def test_numeric_ids_alone_does_not_chown(self, shared_server):
"""#286.1: --numeric-ids is a mapping modifier, not an ownership request.
`-t --numeric-ids` must leave the receiver's ownership untouched."""
source, dest = self._seed_owned("num_only", 12345, 54321)
result, _ = run_client(source, dest, flags=["-t", "--numeric-ids"],
port=shared_server.port)
assert result.returncode == 0, \
f"-t --numeric-ids failed: {(result.stderr or '')[:300]}"
st = os.stat(_received(dest, source, "f.txt"))
assert st.st_uid == 12345, f"--fake-super -o must apply the owner, got uid={st.st_uid}"
assert st.st_gid != 54321, "--fake-super -o must not change the group"
assert st.st_uid != 12345, \
f"--numeric-ids alone must not chown, got uid={st.st_uid}"
def test_numeric_ids_with_o_uses_raw_id(self, shared_server):
source, dest = self._seed_owned("num_o", 12345, 54321)
result, _ = run_client(source, dest, flags=["-o", "-t", "--numeric-ids"],
port=shared_server.port)
assert result.returncode == 0, \
f"-o --numeric-ids failed: {(result.stderr or '')[:300]}"
st = os.stat(_received(dest, source, "f.txt"))
assert st.st_uid == 12345, \
f"-o --numeric-ids must apply the raw id, got uid={st.st_uid}"
class TestPreserveFeatureMatrix:
@@ -374,14 +443,13 @@ class TestPreserveFeatureMatrix:
class TestSpecialNodeModes:
"""Security: a client can never grant group/other write, including on a
recreated special node (FIFO). The special-node creation path sanitizes
S_IWGRP|S_IWOTH just like the regular-file and directory paths, so a source
FIFO with mode 0777 must land as 0755 (owner/group/other read+exec from the
source otherwise preserved). FIFOs are created unprivileged via mkfifo."""
"""Strict rsync parity: with -p the source FIFO mode is copied exactly,
including group/other write. Without -p the node follows the same
source & ~umask base as any other new entry. FIFOs are created
unprivileged via mkfifo."""
@pytest.mark.ci
def test_specials_p_sanitizes_fifo_group_other_write(self):
def test_specials_p_preserves_fifo_mode(self):
source = os.path.join(TEST_DATA_DIR, "specialmode_src")
dest = os.path.join(TEST_DATA_DIR, "specialmode_dst")
clean_dir(source)
@@ -395,8 +463,8 @@ class TestSpecialNodeModes:
# Production daemonizes with umask(0) (server.c) so the source mode is
# what reaches mkfifo. The session server runs in the foreground and
# would inherit the runner's umask, which alone would strip the write
# bits and mask a regression in the sanitization. Start a dedicated
# foreground server under umask(0) to exercise the real path.
# bits and mask a regression. Start a dedicated foreground server under
# umask(0) to exercise the real path.
server = ServerManager()
saved_umask = os.umask(0)
try:
@@ -417,7 +485,5 @@ class TestSpecialNodeModes:
st = os.lstat(received)
assert stat.S_ISFIFO(st.st_mode), f"received entry is not a FIFO: {oct(st.st_mode)}"
mode = st.st_mode & 0o777
assert mode & 0o022 == 0, \
f"recreated FIFO must never be group/other writable, got {oct(mode)}"
assert mode == 0o755, \
f"-p must preserve the source FIFO mode minus group/other write (want 0o755), got {oct(mode)}"
assert mode == 0o777, \
f"-p must preserve the source FIFO mode exactly (want 0o777), got {oct(mode)}"
+2
View File
@@ -15,6 +15,7 @@
#include "test_file.h"
#include "test_file_list.h"
#include "test_file_sendfile.h"
#include "test_format.h"
#include "test_fuzz_smoke.h"
#include "test_glob.h"
#include "test_hardlink.h"
@@ -58,6 +59,7 @@ int main() {
RUN_TEST(test_chunk);
RUN_TEST(test_batch);
RUN_TEST(test_change_list);
RUN_TEST(test_format);
RUN_TEST(test_config);
RUN_TEST(test_credentials);
RUN_TEST(test_compression);
+106 -16
View File
@@ -1,5 +1,6 @@
#include "test_change_list.h"
#include "change_list.h"
#include "config.h"
#include "test_utils.h"
#include "utils.h"
#include <stdlib.h>
@@ -9,64 +10,150 @@
static ChangeEvent sample_event(void) {
ChangeEvent event;
memset(&event, 0, sizeof(event));
event.path = "/srv/root/sub/file.txt";
event.path = "src/sub/file.txt";
event.name = "sub/file.txt";
event.decision = CHANGE_SENT;
event.is_directory = false;
event.size = 12345;
event.bytes_sent = 999;
event.mtime_sec = 1700000000;
event.mtime_nsec = 0;
event.mode = 0100644;
event.uid = 1000;
event.gid = 1000;
return event;
}
/* Expected %M expansion computed independently with localtime_r. */
static void expected_mtime(time_t when, char out[32]) {
struct tm broken_down;
localtime_r(&when, &broken_down);
strftime(out, 32, "%Y/%m/%d-%H:%M:%S", &broken_down);
}
static void test_format_tokens() {
ChangeEvent event = sample_event();
char* line = change_render_format("%f %n %l %b %M %%", &event);
Config* config = config_create();
char when[32];
expected_mtime(event.mtime_sec, when);
char* line = change_render_format("%f %n %l %b %M %%", config, &event);
EXPECT_NOT_NULL(line);
EXPECT_EQ_STR(line, "/srv/root/sub/file.txt file.txt 12345 999 1700000000 %");
char expected[256];
snprintf(expected, sizeof(expected), "src/sub/file.txt sub/file.txt 12345 999 %s %%", when);
EXPECT_EQ_STR(line, expected);
free(line);
config_delete(config);
}
static void test_format_unknown_tokens_preserved() {
ChangeEvent event = sample_event();
char* line = change_render_format("x%q=%f%z", &event);
Config* config = config_create();
char* line = change_render_format("x%q=%f%z", config, &event);
EXPECT_NOT_NULL(line);
EXPECT_EQ_STR(line, "x%q=/srv/root/sub/file.txt%z");
EXPECT_EQ_STR(line, "x%q=src/sub/file.txt%z");
free(line);
config_delete(config);
}
static void test_format_leaf_name() {
static void test_format_directory_name_has_trailing_slash() {
ChangeEvent event = sample_event();
event.path = "bare.txt";
char* line = change_render_format("%n|%f", &event);
event.is_directory = true;
event.path = "src/sub";
event.name = "sub";
Config* config = config_create();
char* line = change_render_format("%n|%f", config, &event);
EXPECT_NOT_NULL(line);
EXPECT_EQ_STR(line, "bare.txt|bare.txt");
EXPECT_EQ_STR(line, "sub/|src/sub");
free(line);
config_delete(config);
}
static void test_render_itemize_sent_file() {
ChangeEvent event = sample_event();
char* line = change_render_itemize(&event);
Config* config = config_create();
char* line = change_render_itemize(config, &event);
EXPECT_NOT_NULL(line);
EXPECT_EQ_STR(line, ">f+++++++++ /srv/root/sub/file.txt");
EXPECT_EQ_STR(line, ">f+++++++++ sub/file.txt");
free(line);
config_delete(config);
}
static void test_render_itemize_directory() {
ChangeEvent event = sample_event();
event.is_directory = true;
event.path = "src/sub";
event.name = "sub";
Config* config = config_create();
char* line = change_render_itemize(config, &event);
EXPECT_NOT_NULL(line);
EXPECT_EQ_STR(line, "cd+++++++++ sub/");
free(line);
config_delete(config);
}
static void test_render_itemize_symlink() {
ChangeEvent event = sample_event();
event.is_symlink = true;
event.path = "src/link";
event.name = "link";
event.symlink_target = "a.txt";
Config* config = config_create();
char* line = change_render_itemize(config, &event);
EXPECT_NOT_NULL(line);
EXPECT_EQ_STR(line, "cL+++++++++ link -> a.txt");
free(line);
config_delete(config);
}
static void test_render_itemize_compares_destination() {
ChangeEvent event = sample_event();
Config* config = config_create();
config->preserve_perms = true;
config->preserve_owner = true;
config->preserve_group = true;
event.dest.known = true;
event.dest.existed = true;
event.dest.size = 1;
event.dest.mtime_sec = 1700000000;
event.dest.mtime_nsec = 0;
event.dest.mode = 0100600;
event.dest.uid = 1;
event.dest.gid = 2;
char* line = change_render_itemize(config, &event);
EXPECT_NOT_NULL(line);
/* size, perms, owner and group differ; time matches. */
EXPECT_EQ_STR(line, ">f.s.pog... sub/file.txt");
free(line);
config_delete(config);
}
static void test_render_itemize_up_to_date_is_empty() {
ChangeEvent event = sample_event();
Config* config = config_create();
event.decision = CHANGE_UP_TO_DATE;
char* line = change_render_itemize(&event);
char* line = change_render_itemize(config, &event);
EXPECT_NOT_NULL(line);
EXPECT_EQ_STR(line, "");
free(line);
config_delete(config);
}
static void test_render_list_line() {
char* line = change_render_list_line(0100644, 4096, 1700000000, "/srv/x.txt");
ChangeEvent event;
memset(&event, 0, sizeof(event));
Config* config = config_create();
event.name = "sub/x.txt";
event.path = "sub/x.txt";
event.mode = 0100644;
event.size = 4096;
event.mtime_sec = 1700000000;
char* line = change_render_list_line(config, &event);
EXPECT_NOT_NULL(line);
EXPECT_TRUE(strncmp(line, "-rw-r--r--", 10) == 0);
EXPECT_TRUE(strstr(line, "4096") != NULL);
EXPECT_TRUE(strstr(line, "/srv/x.txt") != NULL);
EXPECT_TRUE(strstr(line, "4,096") != NULL);
EXPECT_TRUE(strstr(line, "sub/x.txt") != NULL);
free(line);
config_delete(config);
}
static void test_change_list_enabled() {
@@ -93,8 +180,11 @@ static void test_change_list_enabled() {
void test_change_list() {
test_format_tokens();
test_format_unknown_tokens_preserved();
test_format_leaf_name();
test_format_directory_name_has_trailing_slash();
test_render_itemize_sent_file();
test_render_itemize_directory();
test_render_itemize_symlink();
test_render_itemize_compares_destination();
test_render_itemize_up_to_date_is_empty();
test_render_list_line();
test_change_list_enabled();
+255 -10
View File
@@ -317,7 +317,7 @@ static void test_parse_args_protocol_accept_current() {
Config* cfg = valid_client_config();
EXPECT_NOT_NULL(cfg);
char* argv_equals[] = {"fastsync", "--source-dir", "/src",
"--dest-dir", "/dst", "--protocol=2.22.0"};
"--dest-dir", "/dst", "--protocol=2.23.0"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 6, argv_equals, positional_args, &positional_count), 0);
@@ -327,7 +327,7 @@ static void test_parse_args_protocol_accept_current() {
cfg = valid_client_config();
EXPECT_NOT_NULL(cfg);
char* argv_space[] = {"fastsync", "--source-dir", "/src", "--dest-dir",
"/dst", "--protocol", "2.22.0"};
"/dst", "--protocol", "2.23.0"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 7, argv_space, positional_args, &positional_count), 0);
EXPECT_EQ_STR(cfg->version, PROTOCOL_VERSION);
@@ -338,8 +338,8 @@ static void test_parse_args_protocol_accept_current() {
* failure (parse_args simply stores it; validate_config rejects it up front). */
static void test_parse_args_protocol_rejects_other_versions() {
static const char* const bad_versions[] = {"2.17", "2.16", "2.15.0", "2.16.0", "2.17.0",
"2.18.0", "2.19.0", "2.20.0", "2.21.0", "216",
"31", "abc", ""};
"2.18.0", "2.19.0", "2.20.0", "2.21.0", "2.22.0",
"216", "31", "abc", ""};
for (size_t i = 0; i < sizeof(bad_versions) / sizeof(bad_versions[0]); i++) {
Config* cfg = valid_client_config();
EXPECT_NOT_NULL(cfg);
@@ -530,7 +530,9 @@ static void test_parse_args_chmod() {
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv, positional_args, &positional_count), 0);
EXPECT_EQ_STR(cfg->chmod_spec, "u=rw,go=r");
EXPECT_TRUE(cfg->preserve_perms);
/* rsync's --chmod does NOT imply --perms: it only tweaks the mode used for a
* new destination unless -p is also given. */
EXPECT_FALSE(cfg->preserve_perms);
EXPECT_TRUE(cfg->use_metadata);
mode_t result;
EXPECT_TRUE(chmod_apply(0777, cfg->chmod_spec, &result));
@@ -545,14 +547,39 @@ static void test_parse_args_numeric_chmod() {
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, argv, positional_args, &positional_count), 0);
EXPECT_EQ_STR(cfg->chmod_spec, "7777");
EXPECT_TRUE(cfg->preserve_perms);
EXPECT_FALSE(cfg->preserve_perms);
EXPECT_TRUE(cfg->use_metadata);
config_delete(cfg);
}
static void test_parse_args_accepts_selector_chmod() {
Config* cfg = config_create();
char* argv[] = {"fastsync", "--chmod=Dg+s,Fo-w,+X", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv, positional_args, &positional_count), 0);
EXPECT_EQ_STR(cfg->chmod_spec, "Dg+s,Fo-w,+X");
EXPECT_FALSE(cfg->preserve_perms);
config_delete(cfg);
}
static void test_parse_args_appends_repeated_chmod() {
Config* cfg = config_create();
char* argv[] = {"fastsync", "--chmod=a+r", "--chmod=a-w", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, argv, positional_args, &positional_count), 0);
/* Repeated --chmod options accumulate (rsync >= 3.2.4) instead of replacing. */
EXPECT_EQ_STR(cfg->chmod_spec, "a+r,a-w");
mode_t result;
EXPECT_TRUE(chmod_apply(0644, cfg->chmod_spec, &result));
EXPECT_EQ_INT(result, 0444);
config_delete(cfg);
}
static void test_parse_args_rejects_invalid_chmod() {
Config* cfg = config_create();
char* argv[] = {"fastsync", "--chmod=a+X", "/src", "/dst"};
char* argv[] = {"fastsync", "--chmod=a+r,", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv, positional_args, &positional_count), -1);
@@ -1518,6 +1545,9 @@ static void test_parse_args_compress_choice_parity() {
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, argv, positional_args, &positional_count), 0);
/* "auto" is normalized to the canonical "zstd" the receiver accepts. */
EXPECT_EQ_STR(cfg->compress_choice, strcmp(good[i], "auto") == 0 ? "zstd" : good[i]);
EXPECT_EQ_INT(cfg->use_compression, strcmp(good[i], "none") != 0 ? 1 : 0);
config_delete(cfg);
}
static const char* const bad[] = {"lz4", "zlib", "zlibx", "bogus"};
@@ -2326,6 +2356,8 @@ static void test_parse_args_log_file_format() {
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv, positional_args, &positional_count), 0);
EXPECT_EQ_STR(cfg->log_file_format, "%n %M");
/* The format alone is inert (no --log-file): no destination report needed. */
EXPECT_FALSE(cfg->report_dest_info);
config_delete(cfg);
cfg = config_create();
@@ -2334,6 +2366,61 @@ static void test_parse_args_log_file_format() {
EXPECT_EQ_INT(parse_args(cfg, 5, separate_argv, positional_args, &positional_count), 0);
EXPECT_EQ_STR(cfg->log_file_format, "%n %M");
config_delete(cfg);
/* With --log-file the log-format is a real output mode whose %i/%n columns
need the receiver's destination snapshot (same as -i/--out-format). */
const char* log_path = "cli_log_fmt_test.txt";
cfg = config_create();
char log_arg[64];
snprintf(log_arg, sizeof(log_arg), "--log-file=%s", log_path);
char* both_argv[] = {"fastsync", log_arg, "--log-file-format=%i %n", "/src", "/dst"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, both_argv, positional_args, &positional_count), 0);
EXPECT_NOT_NULL(cfg->log_file);
EXPECT_TRUE(cfg->report_dest_info);
config_delete(cfg);
remove(log_path);
}
/* --skip-compress takes a separate value even when it starts with '-' (e.g. a
* suffix typed as "-foo"); the cluster expander must copy it verbatim rather
* than treat it as a short-option cluster. */
static void test_parse_args_skip_compress_dash_value() {
Config* cfg = config_create();
char* argv[] = {"fastsync", "--skip-compress", "-foo/bar", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, argv, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->skip_compress_set);
EXPECT_EQ_INT(cfg->skip_compress_count, 2);
EXPECT_EQ_STR(cfg->skip_compress_suffixes[0], "-foo");
EXPECT_EQ_STR(cfg->skip_compress_suffixes[1], "bar");
config_delete(cfg);
}
/* -i and --out-format also request the destination snapshot. */
static void test_parse_args_report_dest_info_modes() {
Config* cfg = config_create();
char* itemize_argv[] = {"fastsync", "-i", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 3, itemize_argv, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->report_dest_info);
config_delete(cfg);
cfg = config_create();
char* out_argv[] = {"fastsync", "--out-format=%n", "/src", "/dst"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 3, out_argv, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->report_dest_info);
config_delete(cfg);
cfg = config_create();
char* plain_argv[] = {"fastsync", "/src", "/dst"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 3, plain_argv, positional_args, &positional_count), 0);
EXPECT_FALSE(cfg->report_dest_info);
config_delete(cfg);
}
/* --delay-updates is a plain boolean receiver option. */
@@ -2602,10 +2689,13 @@ static void test_parse_args_delete_policy_invalid_values() {
EXPECT_EQ_INT(parse_args(cfg, 4, argv, positional_args, &positional_count), -1);
config_delete(cfg);
/* A negative --max-delete is rsync's deprecated "no client limit" spelling:
parse succeeds and every negative value clamps to -1. */
cfg = config_create();
char* argv2[] = {"fastsync", "--max-delete=-3", "/src", "/dst"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv2, positional_args, &positional_count), -1);
EXPECT_EQ_INT(parse_args(cfg, 4, argv2, positional_args, &positional_count), 0);
EXPECT_EQ_INT(cfg->max_delete, -1);
config_delete(cfg);
}
@@ -2861,6 +2951,92 @@ static void test_parse_args_usermap_name_resolution() {
config_delete(cfg);
}
/* #294: rsync map FROM forms -- inclusive numeric ranges, '*' (any), and the
* empty token (ids with no name on the sender). A TO name is transmitted as a
* NAME for the receiver to resolve (rsync resolves TO names on the receiving
* side), not resolved against the client's database. */
static void test_parse_args_usermap_rsync_forms() {
int positional_args[2];
Config* cfg = config_create();
int positional_count = 0;
char* argv[] = {"fastsync", "--usermap=0-99:nobody", "/src", "/dst"};
EXPECT_EQ_INT(parse_args(cfg, 4, argv, positional_args, &positional_count), 0);
EXPECT_EQ_INT(cfg->usermap_count, 1);
EXPECT_EQ_INT(cfg->usermap[0].from, 0);
EXPECT_EQ_INT(cfg->usermap[0].from_hi, 99);
EXPECT_TRUE(cfg->preserve_owner);
config_delete(cfg);
/* Empty FROM => IDENTITY_MATCH_UNNAMED. */
cfg = config_create();
positional_count = 0;
char* argv2[] = {"fastsync", "--usermap=:@0", "/src", "/dst"};
EXPECT_EQ_INT(parse_args(cfg, 4, argv2, positional_args, &positional_count), 0);
EXPECT_EQ_INT(cfg->usermap_count, 1);
EXPECT_EQ_INT(cfg->usermap[0].from, IDENTITY_MATCH_UNNAMED);
EXPECT_EQ_INT(cfg->usermap[0].from_hi, IDENTITY_MATCH_UNNAMED);
config_delete(cfg);
/* '*' FROM => IDENTITY_MATCH_ANY. */
cfg = config_create();
positional_count = 0;
char* argv3[] = {"fastsync", "--groupmap=*:@0", "/src", "/dst"};
EXPECT_EQ_INT(parse_args(cfg, 4, argv3, positional_args, &positional_count), 0);
EXPECT_EQ_INT(cfg->groupmap[0].from, IDENTITY_MATCH_ANY);
EXPECT_EQ_INT(cfg->groupmap[0].from_hi, IDENTITY_MATCH_ANY);
config_delete(cfg);
/* A TO name is kept as a receiver-resolved name, NOT resolved locally. */
cfg = config_create();
positional_count = 0;
char* argv4[] = {"fastsync", "--usermap=0:nobody", "/src", "/dst"};
EXPECT_EQ_INT(parse_args(cfg, 4, argv4, positional_args, &positional_count), 0);
EXPECT_EQ_INT(cfg->usermap_count, 1);
EXPECT_NOT_NULL(cfg->usermap[0].to_name);
if (cfg->usermap[0].to_name)
EXPECT_EQ_STR(cfg->usermap[0].to_name, "nobody");
config_delete(cfg);
}
/* #294: rsync refuses to mix --chown with --usermap/--groupmap on the same
* side (either order). --chown=USER conflicts with a prior --usermap;
* --chown=:GROUP conflicts with a prior --groupmap; the opposite side is fine. */
static void test_parse_args_identity_map_chown_conflict() {
int positional_args[2];
struct {
const char* a;
const char* b;
} bad[] = {
{"--usermap=0:1", "--chown=2:3"}, {"--chown=2:3", "--usermap=0:1"},
{"--chown=2", "--usermap=0:1"}, {"--chown=2:3", "--groupmap=0:1"},
{"--groupmap=0:1", "--chown=2:3"}, {"--chown=:3", "--groupmap=0:1"},
};
for (size_t i = 0; i < sizeof(bad) / sizeof(bad[0]); i++) {
Config* cfg = config_create();
char* argv[] = {"fastsync", (char*)bad[i].a, (char*)bad[i].b, "/src", "/dst"};
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, argv, positional_args, &positional_count), -1);
config_delete(cfg);
}
/* The opposite-side combinations rsync allows must still parse. */
struct {
const char* a;
const char* b;
} ok[] = {
{"--chown=2", "--groupmap=0:1"},
{"--chown=:3", "--usermap=0:1"},
};
for (size_t i = 0; i < sizeof(ok) / sizeof(ok[0]); i++) {
Config* cfg = config_create();
char* argv[] = {"fastsync", (char*)ok[i].a, (char*)ok[i].b, "/src", "/dst"};
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, argv, positional_args, &positional_count), 0);
config_delete(cfg);
}
}
/* --chown parses USER:GROUP / USER / :GROUP, numeric ids, and '*'. */
static void test_parse_args_chown() {
Config* cfg = config_create();
@@ -2978,8 +3154,10 @@ static void test_parse_args_rejects_malformed_identity() {
const char* val;
} bad[] = {
{"--usermap", "@1000"},
{"--usermap", ":1000"},
{"--usermap", "definitely_not_a_real_user_zzz:@1"},
{"--usermap", "0-"},
{"--usermap", "5-2:@1"},
{"--usermap", "roo*:@1"},
{"--groupmap", "@1"},
{"--groupmap", "no_such_group_qqq:x"},
{"--chown", "a:b:c"},
@@ -3253,6 +3431,24 @@ static void test_parse_args_remote_option_multiple() {
config_delete(cfg);
}
/* The -M=value and -Mvalue short forms are expanded by the cluster expander to
* "-M value" before parsing; both must still collect the remote option (there
* is no dedicated -M= branch). */
static void test_parse_args_remote_option_short_forms() {
static const char* const forms[] = {"-M=--allow-delete", "-M--allow-delete"};
for (size_t i = 0; i < sizeof(forms) / sizeof(forms[0]); i++) {
Config* cfg = valid_client_config();
EXPECT_NOT_NULL(cfg);
char* argv[] = {"fastsync", "--source-dir", "/src", "--dest-dir", "/dst", (char*)forms[i]};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 6, argv, positional_args, &positional_count), 0);
EXPECT_EQ_INT(cfg->remote_option_count, 1);
EXPECT_EQ_STR(cfg->remote_options[0], "--allow-delete");
config_delete(cfg);
}
}
/* Space-separated form "--remote-option OPT" also parses. */
static void test_parse_args_remote_option_space_form() {
Config* cfg = valid_client_config();
@@ -3833,7 +4029,7 @@ static void test_parse_args_attached_short_values() {
cfg = valid_client_config();
positional_count = 0;
char* argv_m[] = {"fastsync", "-Mfoo=bar", "--source-dir", "/src", "--dest-dir", "/dst"};
EXPECT_EQ_INT(parse_args(cfg, 7, argv_m, positional_args, &positional_count), 0);
EXPECT_EQ_INT(parse_args(cfg, 6, argv_m, positional_args, &positional_count), 0);
EXPECT_EQ_INT(cfg->remote_option_count, 1);
EXPECT_EQ_STR(cfg->remote_options[0], "foo=bar");
config_delete(cfg);
@@ -3851,6 +4047,12 @@ static void test_parse_args_inline_equals_forms() {
EXPECT_EQ_STR(cfg->exclude_patterns[0], "*.log");
EXPECT_EQ_INT(cfg->include_count, 1);
EXPECT_EQ_STR(cfg->include_patterns[0], "*.txt");
/* The same patterns are compiled, in command-line order, into the shared
* ordered --filter rule list (rsync first-match-wins). */
EXPECT_NOT_NULL(cfg->filters);
EXPECT_EQ_INT(cfg->filters->size, 2);
EXPECT_EQ_STR((char*)cfg->filters->items[0], "- *.log");
EXPECT_EQ_STR((char*)cfg->filters->items[1], "+ *.txt");
config_delete(cfg);
const char* list_path = "cli_inline_patterns.txt";
@@ -3898,6 +4100,41 @@ static void test_parse_args_inline_equals_forms() {
config_delete(cfg);
}
/* --exclude/--include compile into the SAME ordered filter list as --filter, so
* rsync's first-match-wins semantics hold: the common `--include='*.txt'
* --exclude='*'` idiom keeps the .txt files and drops the rest, and an
* --include rule with no matching exclude is not a mandatory whitelist. */
static void test_parse_args_include_exclude_order() {
Config* cfg = config_create();
int positional_args[2];
int positional_count = 0;
char* argv[] = {"fastsync", "--include=*.txt", "--exclude=*", "/src", "/dst"};
EXPECT_EQ_INT(parse_args(cfg, 5, argv, positional_args, &positional_count), 0);
EXPECT_NOT_NULL(cfg->filters);
EXPECT_EQ_INT(cfg->filters->size, 2);
EXPECT_EQ_STR((char*)cfg->filters->items[0], "+ *.txt");
EXPECT_EQ_STR((char*)cfg->filters->items[1], "- *");
/* The order is reversible on the command line and the list follows it. */
Config* cfg2 = config_create();
positional_count = 0;
char* argv2[] = {"fastsync", "--exclude=*", "--include=*.txt", "/src", "/dst"};
EXPECT_EQ_INT(parse_args(cfg2, 5, argv2, positional_args, &positional_count), 0);
EXPECT_EQ_INT(cfg2->filters->size, 2);
EXPECT_EQ_STR((char*)cfg2->filters->items[0], "- *");
EXPECT_EQ_STR((char*)cfg2->filters->items[1], "+ *.txt");
/* --filter and --exclude/--include interleave in command-line order. */
Config* cfg3 = config_create();
positional_count = 0;
char* argv3[] = {"fastsync", "--filter=- *.tmp", "--include=*.txt", "/src", "/dst"};
EXPECT_EQ_INT(parse_args(cfg3, 5, argv3, positional_args, &positional_count), 0);
EXPECT_EQ_INT(cfg3->filters->size, 2);
EXPECT_EQ_STR((char*)cfg3->filters->items[0], "- *.tmp");
EXPECT_EQ_STR((char*)cfg3->filters->items[1], "+ *.txt");
config_delete(cfg);
config_delete(cfg2);
config_delete(cfg3);
}
/* OPT_NOOP compatibility flags (-s/--secluded-args, -r/--recursive) must never
* swallow the next argv: `fastsync -s SRC DST` keeps both positionals. */
static void test_parse_args_noop_does_not_consume_argv() {
@@ -3959,6 +4196,8 @@ void test_client_cli() {
test_parse_args_usermap();
test_parse_args_groupmap();
test_parse_args_usermap_name_resolution();
test_parse_args_usermap_rsync_forms();
test_parse_args_identity_map_chown_conflict();
test_parse_args_chown();
test_parse_args_copy_as();
test_parse_args_rejects_malformed_identity();
@@ -4001,6 +4240,8 @@ void test_client_cli() {
test_parse_args_executability();
test_parse_args_chmod();
test_parse_args_numeric_chmod();
test_parse_args_accepts_selector_chmod();
test_parse_args_appends_repeated_chmod();
test_parse_args_rejects_invalid_chmod();
test_parse_args_invalid_port();
test_parse_args_non_numeric_port();
@@ -4091,6 +4332,8 @@ void test_client_cli() {
test_parse_args_list_only();
test_parse_args_out_format();
test_parse_args_log_file_format();
test_parse_args_report_dest_info_modes();
test_parse_args_skip_compress_dash_value();
test_parse_args_checksum_choice_aliases();
test_parse_args_checksum_choice_requires_value();
test_parse_args_checksum_choice_equals_forms();
@@ -4119,6 +4362,7 @@ void test_client_cli() {
test_parse_args_trust_sender_default_false();
test_parse_args_trust_sender();
test_parse_args_remote_option_multiple();
test_parse_args_remote_option_short_forms();
test_parse_args_remote_option_space_form();
test_parse_args_remote_option_missing_value();
test_parse_args_remote_option_rejects_bad_values();
@@ -4131,6 +4375,7 @@ void test_client_cli() {
test_parse_args_short_clustering();
test_parse_args_attached_short_values();
test_parse_args_inline_equals_forms();
test_parse_args_include_exclude_order();
test_parse_args_noop_does_not_consume_argv();
test_parse_args_backup_copy_links_shorts();
test_parse_args_rejects_unsupported_short();
+118 -15
View File
@@ -552,6 +552,83 @@ static void test_config_send_receive() {
}
}
/* #5: a received --max-alloc=0 (rsync's "no limit") is floored to the server
* ceiling on the receive path, so a client cannot disable it. */
static void test_config_receive_max_alloc_zero_floored() {
Config* send_cfg = config_create();
EXPECT_NOT_NULL(send_cfg);
send_cfg->send_directory = str_dup("/send/src");
send_cfg->receive_root_directory = str_dup("/send/dst");
send_cfg->max_alloc = 0;
int p[2];
EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0);
io_set_fds(p[0], p[1]);
io_set_bwlimit(0);
pid_t pid = fork();
if (pid == 0) {
close(p[1]);
io_set_fds(p[0], p[0]);
Config* recv_cfg = config_receive(p[0]);
bool ok = recv_cfg != NULL && recv_cfg->max_alloc == MAX_SERVER_ALLOC;
config_delete(recv_cfg);
close(p[0]);
close(p[1]);
_exit(ok ? 0 : 1);
} else {
close(p[0]);
io_set_fds(p[1], p[1]);
bool sent = config_send(p[1], send_cfg);
int status;
waitpid(pid, &status, 0);
close(p[0]);
close(p[1]);
config_delete(send_cfg);
EXPECT_TRUE(sent);
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
}
}
/* #4: a hostile/older client that still sends compress_choice=auto must be
* accepted (as zstd) rather than failing the whole transfer. */
static void test_config_receive_compress_choice_auto_canonicalized() {
Config* send_cfg = config_create();
EXPECT_NOT_NULL(send_cfg);
send_cfg->send_directory = str_dup("/send/src");
send_cfg->receive_root_directory = str_dup("/send/dst");
free(send_cfg->compress_choice);
send_cfg->compress_choice = str_dup("auto");
int p[2];
EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0);
io_set_fds(p[0], p[1]);
io_set_bwlimit(0);
pid_t pid = fork();
if (pid == 0) {
close(p[1]);
io_set_fds(p[0], p[0]);
Config* recv_cfg = config_receive(p[0]);
bool ok = recv_cfg != NULL && strcmp(recv_cfg->compress_choice, "zstd") == 0;
config_delete(recv_cfg);
close(p[0]);
close(p[1]);
_exit(ok ? 0 : 1);
} else {
close(p[0]);
io_set_fds(p[1], p[1]);
bool sent = config_send(p[1], send_cfg);
int status;
waitpid(pid, &status, 0);
close(p[0]);
close(p[1]);
config_delete(send_cfg);
EXPECT_TRUE(sent);
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
}
}
static void test_config_send_receive_version_mismatch() {
/* A peer using the previous wire format must be rejected. */
Config* cfg = config_create();
@@ -1345,13 +1422,19 @@ static void test_config_identity_wire_roundtrip() {
send_cfg->usermap_count = 2;
send_cfg->usermap = calloc(2, sizeof(IdentityMap));
send_cfg->usermap[0].from = IDENTITY_MATCH_ANY;
send_cfg->usermap[0].from_hi = IDENTITY_MATCH_ANY;
send_cfg->usermap[0].to = 65534;
send_cfg->usermap[0].to_name = NULL;
send_cfg->usermap[1].from = 1000;
send_cfg->usermap[1].from_hi = 1000;
send_cfg->usermap[1].to = 1000;
send_cfg->usermap[1].to_name = NULL;
send_cfg->groupmap_count = 1;
send_cfg->groupmap = calloc(1, sizeof(IdentityMap));
send_cfg->groupmap[0].from = 0;
send_cfg->groupmap[0].from_hi = 0;
send_cfg->groupmap[0].to = IDENTITY_CURRENT;
send_cfg->groupmap[0].to_name = str_dup("root");
int p[2];
EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0);
@@ -1367,9 +1450,12 @@ static void test_config_identity_wire_roundtrip() {
ok = recv->numeric_ids && recv->chown_uid_set && recv->chown_uid == 1001 &&
recv->chown_gid_set && recv->chown_gid == IDENTITY_CURRENT && recv->usermap_count == 2 &&
recv->groupmap_count == 1 && recv->usermap[0].from == IDENTITY_MATCH_ANY &&
recv->usermap[0].to == 65534 && recv->usermap[1].from == 1000 &&
recv->usermap[1].to == 1000 && recv->groupmap[0].from == 0 &&
recv->groupmap[0].to == IDENTITY_CURRENT;
recv->usermap[0].from_hi == IDENTITY_MATCH_ANY && recv->usermap[0].to == 65534 &&
recv->usermap[0].to_name == NULL && recv->usermap[1].from == 1000 &&
recv->usermap[1].from_hi == 1000 && recv->usermap[1].to == 1000 &&
recv->groupmap[0].from == 0 && recv->groupmap[0].from_hi == 0 &&
recv->groupmap[0].to == IDENTITY_CURRENT && recv->groupmap[0].to_name != NULL &&
strcmp(recv->groupmap[0].to_name, "root") == 0;
}
config_delete(recv);
close(p[0]);
@@ -1399,7 +1485,8 @@ static void test_config_receive_rejects_invalid_identity() {
c->receive_root_directory = str_dup("/dst");
c->usermap_count = 1;
c->usermap = calloc(1, sizeof(IdentityMap));
c->usermap[0].from = -2; /* below IDENTITY_MATCH_ANY */
c->usermap[0].from = -3; /* below IDENTITY_MATCH_UNNAMED */
c->usermap[0].from_hi = -3;
c->usermap[0].to = 0;
EXPECT_FALSE(roundtrip_config_ok(c));
config_delete(c);
@@ -2101,8 +2188,10 @@ static void test_identity_explicit_ownership_requested() {
}
/* P7 Wave E hardening (A3): --super no longer implies raw numeric-id
preservation, so it must never enable ownership application on its own; an
explicit identity flag is required. */
preservation, so it must never enable ownership application on its own.
#286: --numeric-ids is a mapping MODIFIER only and is likewise inert on its
own; a real ownership request (-o/-g or an explicit identity flag) is
required to activate chown. */
static void test_super_does_not_imply_numeric() {
Config* c = config_create();
EXPECT_NOT_NULL(c);
@@ -2112,6 +2201,9 @@ static void test_super_does_not_imply_numeric() {
EXPECT_FALSE(identity_active_enabled());
c->numeric_ids = true;
EXPECT_TRUE(identity_set_active(c));
EXPECT_FALSE(identity_active_enabled()); /* mapping modifier only */
c->preserve_owner = true;
EXPECT_TRUE(identity_set_active(c));
EXPECT_TRUE(identity_active_enabled());
identity_clear_active();
config_delete(c);
@@ -2635,13 +2727,19 @@ static void golden_config_populate(Config* c) {
c->usermap_count = 2;
c->usermap = calloc(2, sizeof(IdentityMap));
c->usermap[0].from = IDENTITY_MATCH_ANY;
c->usermap[0].from_hi = IDENTITY_MATCH_ANY;
c->usermap[0].to = 1000;
c->usermap[0].to_name = NULL;
c->usermap[1].from = 5;
c->usermap[1].from_hi = 9;
c->usermap[1].to = 6;
c->usermap[1].to_name = NULL;
c->groupmap_count = 1;
c->groupmap = calloc(1, sizeof(IdentityMap));
c->groupmap[0].from = 7;
c->groupmap[0].from_hi = 7;
c->groupmap[0].to = 8;
c->groupmap[0].to_name = str_dup("root");
c->preserve_atimes = true;
c->preserve_crtimes = false;
c->omit_dir_times = true;
@@ -2665,14 +2763,14 @@ static void golden_config_populate(Config* c) {
c->copy_as_gid = 222;
}
/* The pinned golden frame (protocol 2.22.0). The values below are the only
/* The pinned golden frame (protocol 2.23.0). The values below are the only
* thing that ties the generated table to the historical wire format; update
* them ONLY with a PROTOCOL_VERSION bump and a documented reason. The 2.22.0
* preserve-attribute split appends four serialized bools
* (preserve_perms/times/owner/group) to CONFIG_WIRE_METADATA_TIMES_FIELDS after
* omit_link_times. */
#define GOLDEN_WIRE_LEN 653
#define GOLDEN_WIRE_HASH 95530566005420798ULL
* them ONLY with a PROTOCOL_VERSION bump and a documented reason. The 2.23.0
* rsync-parity wave changes the config-frame layout (map-entry range + TO name,
* one report_dest_info bool, and other wire changes landing in this version);
* the byte-exact values are recomputed for the merged layout. */
#define GOLDEN_WIRE_LEN 697
#define GOLDEN_WIRE_HASH 7835017034643051109ULL
static unsigned long long fnv1a_64(const unsigned char* buf, size_t len) {
unsigned long long h = 1469598103934665603ULL;
@@ -2754,7 +2852,7 @@ static unsigned long long capture_wire_hash(const Config* cfg, size_t* out_len)
return h;
}
/* Byte-for-byte wire compatibility guard (protocol 2.22.0). The expected hash
/* Byte-for-byte wire compatibility guard (protocol 2.23.0). The expected hash
* pins the pre-X-macro byte stream; the refactor MUST NOT change it. */
static void test_config_wire_golden() {
if (is_running_under_valgrind())
@@ -2815,7 +2913,10 @@ static void test_config_wire_golden_receive() {
ok = ok && recv->super_mode == SUPER_MODE_ON;
ok = ok && recv->chown_uid == 1234 && recv->chown_gid == 5678;
ok = ok && recv->usermap_count == 2 && recv->usermap[0].from == IDENTITY_MATCH_ANY &&
recv->usermap[0].to == 1000 && recv->usermap[1].from == 5 && recv->usermap[1].to == 6;
recv->usermap[0].from_hi == IDENTITY_MATCH_ANY && recv->usermap[0].to == 1000 &&
recv->usermap[1].from == 5 && recv->usermap[1].from_hi == 9 && recv->usermap[1].to == 6;
ok = ok && recv->groupmap_count == 1 && recv->groupmap[0].from == 7 &&
recv->groupmap[0].to_name != NULL && strcmp(recv->groupmap[0].to_name, "root") == 0;
ok = ok && recv->basis_count == 2 && recv->basis_dirs[0].type == BASIS_DEST_COMPARE &&
recv->basis_dirs[1].type == BASIS_DEST_LINK;
ok = ok && recv->module != NULL && strcmp(recv->module, "goldenmod") == 0;
@@ -3027,6 +3128,8 @@ void test_config() {
test_pipeline_receiver_lifecycle();
if (!is_running_under_valgrind()) {
test_config_send_receive();
test_config_receive_max_alloc_zero_floored();
test_config_receive_compress_choice_auto_canonicalized();
test_config_local_only_fields_not_serialized();
test_config_send_receive_version_mismatch();
test_config_receive_truncated();
+168 -36
View File
@@ -28,6 +28,10 @@ static void test_file_create() {
EXPECT_NULL(f->data->data);
EXPECT_EQ_INT((int)f->data->size, 0);
EXPECT_NULL(f->metadata);
/* An unset destination snapshot must read as known == false, never
indeterminate bytes (-i/--out-format without --incremental). */
EXPECT_FALSE(f->dest_state.known);
EXPECT_FALSE(f->dest_state.existed);
file_destroy(f);
}
@@ -326,6 +330,52 @@ static void test_file_save_to_disk_partial_install() {
rmdir(root);
}
/* --temp-dir is a client-controlled wire value that must be confined below the
* receive root: an absolute or `..`-escaping value is rejected (a client must
* never make the receiver write scratch files in an arbitrary directory), while
* a relative one resolves under the root and is used for the atomic install. */
static void test_file_save_to_disk_temp_dir_confined() {
const char* root = "test_temp_confine_tmp";
const char* dest_file = "test_temp_confine_tmp/file.txt";
char outside[PATH_MAX];
snprintf(outside, sizeof(outside), "/tmp/fastsync_temp_outside_%d", (int)getpid());
unlink(dest_file);
rmdir("test_temp_confine_tmp/scratch");
rmdir(root);
mkdir(root, 0755);
mkdir("test_temp_confine_tmp/scratch", 0755);
mkdir(outside, 0755);
File* f = file_create("file.txt");
EXPECT_NOT_NULL(f);
const char* content = "confined temp dir";
f->data->data = malloc(strlen(content));
EXPECT_NOT_NULL(f->data->data);
memcpy(f->data->data, content, strlen(content));
f->data->size = strlen(content);
Config* config = config_create();
EXPECT_NOT_NULL(config);
config->temp_dir = str_dup(outside);
EXPECT_EQ_INT(file_save_to_disk_full(root, f, config), FILE_SAVE_ERROR);
EXPECT_EQ_INT(access(dest_file, F_OK), -1);
free(config->temp_dir);
config->temp_dir = str_dup("../escape");
EXPECT_EQ_INT(file_save_to_disk_full(root, f, config), FILE_SAVE_ERROR);
EXPECT_EQ_INT(access(dest_file, F_OK), -1);
free(config->temp_dir);
config->temp_dir = str_dup("scratch");
EXPECT_EQ_INT(file_save_to_disk_full(root, f, config), FILE_SAVE_WRITTEN);
EXPECT_EQ_INT(access(dest_file, F_OK), 0);
file_destroy(f);
config_delete(config);
unlink(dest_file);
rmdir("test_temp_confine_tmp/scratch");
rmdir(root);
rmdir(outside);
}
/* Issue #251: file_save_to_disk_full must distinguish receiver-side skips
(--existing/--ignore-existing/--update) from real writes so the sender can
decide whether --remove-source-files may unlink its source. */
@@ -957,7 +1007,8 @@ static void test_inplace_overwrite_metadata_strips_special_bits() {
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
/* Metadata-derived mode is applied and never includes setuid/setgid/sticky. */
/* No -p: the pre-existing destination mode (without its special bits) is
* restored; the source mode is not applied. */
EXPECT_EQ_INT((int)(st.st_mode & (S_ISUID | S_ISGID | S_ISVTX)), 0);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0755);
@@ -1039,12 +1090,11 @@ static void test_atomic_no_perms_preserves_destination_mode() {
unlink(fresh);
}
/* MAJOR 2: a brand-new destination file must never be created group/other
* writable from a client-supplied source mode. The daemon runs with umask(0),
* so without the explicit S_IWGRP|S_IWOTH strip a source 0666 (with no -p)
* would materialize as world-writable. */
static void test_new_file_mode_never_group_other_writable() {
const char* path = "test_new_file_no_go_write.bin";
/* Strict rsync parity: a brand-new destination file with no -p follows
* rsync's source_mode & ~umask base, so group/other write in the source mode is
* honored exactly as the umask allows (it is no longer force-cleared). */
static void test_new_file_mode_honors_source_and_umask() {
const char* path = "test_new_file_mode.bin";
unlink(path);
FileMetadata m;
memset(&m, 0, sizeof(m));
@@ -1058,19 +1108,15 @@ static void test_new_file_mode_never_group_other_writable() {
EXPECT_TRUE(ok);
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
EXPECT_EQ_INT((int)(st.st_mode & (S_IWGRP | S_IWOTH)), 0);
/* The rest of the source mode is still honored (owner write survives). */
EXPECT_EQ_INT((int)(st.st_mode & S_IWUSR), S_IWUSR);
EXPECT_EQ_INT((int)(st.st_mode & 0777), (int)(0666 & ~(mode_t)file_process_umask()));
unlink(path);
}
/* Security: a client-supplied special-node mode must never materialize a
* group/other-writable FIFO. file_save_special_to_disk() sanitizes the
* creation bits the same way the regular-file policy does: under -p the source
* mode loses S_IWGRP|S_IWOTH (0777 -> 0755), and without -p a safe 0644 default
* is used. The daemon runs with umask(0) (server.c), so the explicit strip is
* what keeps the node safe -- the test clears the umask to prove it. */
static void test_special_fifo_mode_never_group_other_writable_impl() {
/* Strict rsync parity for recreated special nodes: with -p the source mode is
* copied exactly (0777 -> 0777), and without -p the same source & ~umask base
* as any other new entry applies. The process umask is cleared so the source
* bits are what reaches mkfifo. */
static void test_special_fifo_mode_honors_source_and_umask_impl() {
const char* root = "test_special_mode_tmp";
const char* with_p = "test_special_mode_tmp/with_p.fifo";
const char* no_p = "test_special_mode_tmp/no_p.fifo";
@@ -1089,7 +1135,7 @@ static void test_special_fifo_mode_never_group_other_writable_impl() {
meta.gid = getegid();
meta.mtime_sec = 1000000000;
/* -p: the source mode is honored minus group/other write. */
/* -p: the source mode (including group/other write) is copied exactly. */
File* f = file_create("with_p.fifo");
EXPECT_NOT_NULL(f);
f->is_special = true;
@@ -1101,12 +1147,11 @@ static void test_special_fifo_mode_never_group_other_writable_impl() {
struct stat st;
EXPECT_EQ_INT(lstat(with_p, &st), 0);
EXPECT_TRUE(S_ISFIFO(st.st_mode));
EXPECT_EQ_INT((int)(st.st_mode & (S_IWGRP | S_IWOTH)), 0);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0755);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0777);
f->metadata = NULL;
file_destroy(f);
/* No -p: the fixed safe default, never the source's 0777. */
/* No -p: source & ~umask (umask is cleared, so 0777). */
f = file_create("no_p.fifo");
EXPECT_NOT_NULL(f);
f->is_special = true;
@@ -1115,8 +1160,7 @@ static void test_special_fifo_mode_never_group_other_writable_impl() {
EXPECT_EQ_INT(file_save_to_disk_full(root, f, cfg), FILE_SAVE_WRITTEN);
EXPECT_EQ_INT(lstat(no_p, &st), 0);
EXPECT_TRUE(S_ISFIFO(st.st_mode));
EXPECT_EQ_INT((int)(st.st_mode & (S_IWGRP | S_IWOTH)), 0);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0644);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0777);
f->metadata = NULL;
file_destroy(f);
@@ -1126,15 +1170,15 @@ static void test_special_fifo_mode_never_group_other_writable_impl() {
rmdir(root);
}
/* The receiver daemon runs umask(0), so an unsanitized source mode would reach
* mkfifo unmasked. Run the body with umask(0) to exercise the explicit strip,
* and restore the process umask from this wrapper so a failing EXPECT inside the
* body (which returns from the body only) cannot leak umask(0) into later
* tests. */
static void test_special_fifo_mode_never_group_other_writable() {
/* The receiver daemon runs umask(0), so the source mode reaches mkfifo
* unmasked. Run the body with umask(0) and refresh the cached process umask so
* file_process_umask() agrees, then restore both. */
static void test_special_fifo_mode_honors_source_and_umask() {
mode_t saved_umask = umask(0);
test_special_fifo_mode_never_group_other_writable_impl();
file_umask_capture();
test_special_fifo_mode_honors_source_and_umask_impl();
umask(saved_umask);
file_umask_capture();
}
/* --specials recreates a unix-domain socket via mknod(S_IFSOCK), which Linux
@@ -1659,9 +1703,19 @@ static void test_dir_time_list() {
dir_time_list_init(&list);
EXPECT_EQ_INT((int)list.count, 0);
FileMetadata metadata = {.mtime_sec = 1000000000, .mtime_nsec = 0};
EXPECT_TRUE(dir_time_list_add(&list, "sub", &metadata));
EXPECT_TRUE(dir_time_list_add(&list, "sub", &metadata));
EXPECT_TRUE(dir_time_list_add(&list, "sub", &metadata, NULL));
/* A captured xattr block is deep-copied into the list. */
FileXattrList* xl = xattr_list_new();
EXPECT_NOT_NULL(xl);
EXPECT_TRUE(xattr_list_append(xl, "user.dir", "v", 1));
EXPECT_TRUE(dir_time_list_add(&list, "sub", &metadata, xl));
xattr_list_free(xl); /* the list owns its own copy now */
EXPECT_EQ_INT((int)list.count, 2);
EXPECT_NOT_NULL(list.xattrs);
EXPECT_NOT_NULL(list.xattrs[1]);
EXPECT_EQ_INT(list.xattrs[1]->count, 1);
EXPECT_EQ_STR(list.xattrs[1]->items[0].name, "user.dir");
EXPECT_NULL(list.xattrs[0]);
Config* cfg = config_create();
EXPECT_NOT_NULL(cfg);
@@ -1677,6 +1731,7 @@ static void test_dir_time_list() {
EXPECT_EQ_INT((int)list.count, 0);
EXPECT_NULL(list.paths);
EXPECT_NULL(list.entries);
EXPECT_NULL(list.xattrs);
rmdir(sub);
rmdir(root);
@@ -1701,14 +1756,14 @@ static void test_dir_time_list_cap() {
for (size_t i = 0; i < MAX_DIR_TIME_ENTRIES + 1 && !rejected; i++) {
size_t before_count = list.count;
size_t before_bytes = list.bytes;
if (!dir_time_list_add(&list, path, &metadata)) {
if (!dir_time_list_add(&list, path, &metadata, NULL)) {
rejected = true;
/* The rejected add must not have partially mutated the list. */
EXPECT_TRUE(list.count == before_count);
EXPECT_TRUE(list.bytes == before_bytes);
} else {
EXPECT_TRUE(list.count == before_count + 1);
EXPECT_TRUE(list.bytes == before_bytes + path_len + sizeof(FileMetadata) + sizeof(char*));
EXPECT_TRUE(list.bytes == before_bytes + path_len + sizeof(FileMetadata) + 2 * sizeof(char*));
}
}
EXPECT_TRUE(rejected);
@@ -1872,6 +1927,81 @@ static void test_keep_dirlinks_secure_open() {
file_set_keep_dirlinks(false);
}
/* Build an ArrayList of str_dup'd strings (NULL on allocation failure). */
static ArrayList* make_manifest_string_list(const char* const* entries, int count) {
ArrayList* list = array_list_create(free);
if (!list)
return NULL;
for (int i = 0; i < count; i++) {
char* dup = str_dup(entries[i]);
if (!dup || !array_list_add(list, dup)) {
free(dup);
array_list_delete(list);
return NULL;
}
}
return list;
}
/* Regression (#3): a non-empty --delete-missing-args directory charges each
* removed entry exactly once. The directory itself must not be counted twice;
* if it were, `deleted` would exceed --max-delete and the extras walk would
* underflow its remaining budget and delete past the user's cap. */
static void test_manifest_delete_missing_dir_budget_double_count() {
char root[PATH_MAX];
snprintf(root, sizeof(root), "/tmp/fastsync_mgdir_%d", (int)getpid());
char* gone = path_cat(root, "gone");
char* gone_file = path_cat(gone, "f0");
char* extra = path_cat(root, "extra.txt");
EXPECT_NOT_NULL(gone);
EXPECT_NOT_NULL(gone_file);
EXPECT_NOT_NULL(extra);
mkdir(root, 0755);
mkdir(gone, 0755);
EXPECT_EQ_INT(access(extra, F_OK), -1);
EXPECT_TRUE(file_write_to_disk(extra, "extra", 5, false, false));
/* The missing-arg directory holds N-1 == 2 entries; with the directory itself
that is exactly --max-delete=3. */
EXPECT_TRUE(file_write_to_disk(gone_file, "x", 1, false, false));
char* gone_file2 = path_cat(gone, "f1");
EXPECT_TRUE(gone_file2 != NULL && file_write_to_disk(gone_file2, "x", 1, false, false));
Config* cfg = config_create();
EXPECT_NOT_NULL(cfg);
cfg->receive_root_directory = str_dup(root);
cfg->use_delete = true;
cfg->delete_missing_args = true;
cfg->max_delete = 3;
const char* missing_names[] = {"gone"};
const char* synced[] = {"."};
DeleteManifest manifest = {0};
manifest.keeps = make_manifest_string_list(NULL, 0);
manifest.missing = make_manifest_string_list(missing_names, 1);
manifest.dirs = make_manifest_string_list(synced, 1);
EXPECT_NOT_NULL(manifest.keeps);
EXPECT_NOT_NULL(manifest.missing);
EXPECT_NOT_NULL(manifest.dirs);
DeleteCommitResult result = manifest_delete_all(cfg, &manifest);
EXPECT_EQ_INT((int)result, (int)DELETE_COMMIT_LIMIT_REACHED);
/* The whole missing-arg directory is gone (dir + its 2 entries == 3). */
EXPECT_EQ_INT(access(gone, F_OK), -1);
/* The saturated budget must leave the in-scope extra untouched. */
EXPECT_EQ_INT(access(extra, F_OK), 0);
array_list_delete(manifest.keeps);
array_list_delete(manifest.missing);
array_list_delete(manifest.dirs);
config_delete(cfg);
unlink(extra);
free(gone);
free(gone_file);
free(gone_file2);
free(extra);
rmdir(root);
}
void test_file() {
test_file_create();
test_file_special_rdev_valid();
@@ -1885,6 +2015,7 @@ void test_file() {
test_file_save_to_disk_ignore_existing();
test_file_save_to_disk_ignore_existing_entry_types();
test_file_save_to_disk_partial_install();
test_file_save_to_disk_temp_dir_confined();
test_file_save_to_disk_reports_skips();
test_file_write_to_disk_sparse_preserves_holes();
test_file_write_to_disk_partial_retention();
@@ -1919,10 +2050,11 @@ void test_file() {
test_inplace_overwrite_clears_special_mode_bits();
test_inplace_overwrite_metadata_strips_special_bits();
test_atomic_no_perms_preserves_destination_mode();
test_new_file_mode_never_group_other_writable();
test_special_fifo_mode_never_group_other_writable();
test_new_file_mode_honors_source_and_umask();
test_special_fifo_mode_honors_source_and_umask();
test_special_socket_recreated();
test_inplace_overwrite_truncates_shorter_payload();
test_inplace_refuses_fifo_destination();
test_inplace_refuses_device_destination();
test_manifest_delete_missing_dir_budget_double_count();
}
+45
View File
@@ -148,6 +148,50 @@ static void test_ancestor_and_descendant_queries() {
remove(path);
}
/* The delete-walker's synchronized-directory predicate: a directory is in scope
only when it is a listed directory or lies below one, NOT when it is merely an
implied parent of a listed file. */
static void test_dir_in_scope() {
char err[160];
/* NULL set / empty list semantics. */
EXPECT_TRUE(file_list_dir_in_scope(NULL, "anything"));
const char* path = "test_file_list_dirscope.txt";
write_list(path, "d1/leaf.txt\n");
FileListSet* set = file_list_load(path, false, err, sizeof(err));
EXPECT_NOT_NULL(set);
/* d1 is only an implied parent of a listed FILE: not synchronized. */
EXPECT_FALSE(file_list_dir_in_scope(set, "d1"));
EXPECT_FALSE(file_list_dir_in_scope(set, "d1/sub"));
EXPECT_FALSE(file_list_dir_in_scope(set, "other"));
file_list_destroy(set);
remove(path);
/* A listed DIRECTORY synchronizes itself and its whole subtree. */
write_list(path, "d1/\nother\n");
set = file_list_load(path, false, err, sizeof(err));
EXPECT_NOT_NULL(set);
EXPECT_TRUE(file_list_dir_in_scope(set, "d1"));
EXPECT_TRUE(file_list_dir_in_scope(set, "d1/sub/deep"));
EXPECT_TRUE(file_list_dir_in_scope(set, "other"));
EXPECT_TRUE(file_list_dir_in_scope(set, "other/x"));
EXPECT_FALSE(file_list_dir_in_scope(set, "d2"));
EXPECT_FALSE(file_list_dir_in_scope(set, "d1x")); /* component boundary */
EXPECT_FALSE(file_list_dir_in_scope(set, ""));
file_list_destroy(set);
remove(path);
/* "." lists the whole tree. */
write_list(path, ".\n");
set = file_list_load(path, false, err, sizeof(err));
EXPECT_NOT_NULL(set);
EXPECT_TRUE(file_list_dir_in_scope(set, ""));
EXPECT_TRUE(file_list_dir_in_scope(set, "anything/at/all"));
file_list_destroy(set);
remove(path);
}
/* Regression for the remote OOM: an adversarial --files-from entry made of a
very deep chain of repeated components must be indexed with memory
proportional to the entry count. The old implementation stored one copied
@@ -219,6 +263,7 @@ static void test_oversized_entry_rejected() {
void test_file_list() {
test_membership_matches_reference();
test_ancestor_and_descendant_queries();
test_dir_in_scope();
test_deep_paths_are_bounded();
test_oversized_entry_rejected();
}
+94
View File
@@ -0,0 +1,94 @@
#include "test_format.h"
#include "format.h"
#include "test_utils.h"
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <time.h>
#include <unistd.h>
static void expect_big_num(unsigned long long value, bool human, const char* expected) {
char buffer[64];
EXPECT_TRUE(format_big_num(value, human, buffer, sizeof(buffer)));
EXPECT_EQ_STR(buffer, expected);
}
static void test_human_size_decimal() {
/* Values below 1000 print verbatim; larger values use the largest unit that
* keeps the value below 1000 and exactly two decimals (rsync human_num). */
expect_big_num(0, true, "0");
expect_big_num(999, true, "999");
expect_big_num(1000, true, "1.00K");
expect_big_num(1500, true, "1.50K");
expect_big_num(9999, true, "10.00K");
expect_big_num(999999, true, "1000.00K");
expect_big_num(1000000, true, "1.00M");
expect_big_num(1500000, true, "1.50M");
}
static void test_big_num_grouping() {
/* Non-human numbers are comma-grouped every three digits (rsync big_num). */
expect_big_num(0, false, "0");
expect_big_num(1, false, "1");
expect_big_num(999, false, "999");
expect_big_num(1000, false, "1,000");
expect_big_num(4096, false, "4,096");
expect_big_num(1234567, false, "1,234,567");
expect_big_num(1000000000ULL, false, "1,000,000,000");
}
static void test_datetime_format() {
char buffer[32];
time_t when = 1700000000;
EXPECT_TRUE(format_rsync_datetime(when, true, buffer, sizeof(buffer)));
/* %M shape: YYYY/MM/DD-HH:MM:SS */
EXPECT_EQ_INT(strlen(buffer), 19);
EXPECT_EQ_INT(buffer[4], '/');
EXPECT_EQ_INT(buffer[7], '/');
EXPECT_EQ_INT(buffer[10], '-');
EXPECT_EQ_INT(buffer[13], ':');
EXPECT_EQ_INT(buffer[16], ':');
char space_form[32];
EXPECT_TRUE(format_rsync_datetime(when, false, space_form, sizeof(space_form)));
EXPECT_EQ_INT(space_form[10], ' ');
}
static void test_dest_state_roundtrip() {
/* The wire codec is exercised over a socketpair so the real send/receive
* primitives run. */
int fds[2];
if (socketpair(AF_UNIX, SOCK_STREAM, 0, fds) != 0)
return;
OutputDestState out;
memset(&out, 0, sizeof(out));
out.known = true;
out.existed = true;
out.size = 123456789ULL;
out.mtime_sec = 1700000000;
out.mtime_nsec = 123456789;
out.mode = 0100644;
out.uid = 1000;
out.gid = 1000;
OutputDestState in;
memset(&in, 0, sizeof(in));
EXPECT_TRUE(format_dest_state_send(fds[0], &out));
EXPECT_TRUE(format_dest_state_receive(fds[1], &in));
EXPECT_TRUE(in.known);
EXPECT_TRUE(in.existed);
EXPECT_TRUE(in.size == out.size);
EXPECT_TRUE(in.mtime_sec == out.mtime_sec);
EXPECT_TRUE(in.mtime_nsec == out.mtime_nsec);
EXPECT_TRUE(in.mode == out.mode);
EXPECT_TRUE(in.uid == out.uid);
EXPECT_TRUE(in.gid == out.gid);
close(fds[0]);
close(fds[1]);
}
void test_format(void) {
test_human_size_decimal();
test_big_num_grouping();
test_datetime_format();
test_dest_state_roundtrip();
}
+6
View File
@@ -0,0 +1,6 @@
#ifndef TEST_FORMAT_H
#define TEST_FORMAT_H
void test_format(void);
#endif
+21 -13
View File
@@ -18,6 +18,9 @@
/* P8 config-frame tail: super_mode (4) + copy-as presence (4) + uid (4) + gid (4). */
#define P8_TAIL_BYTES 16
/* Protocol 2.23.0 appends one trailing bool (report_dest_info) AFTER the P8
* tail, so the P8 fields sit this many bytes before the end of the frame. */
#define OUTPUT_TAIL_BYTES 4
/* Smoke test for chunk_deserialize fuzz target */
static void test_fuzz_chunk_deserialize() {
@@ -334,31 +337,31 @@ static void test_fuzz_config_receive_p8_tail() {
/* super_mode outside the 0..2 tri-state is refused. */
memcpy(mut, frame, len);
put_i32(mut, len - P8_TAIL_BYTES, 99);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES, 99);
EXPECT_FALSE(receive_config_frame(mut, len));
put_i32(mut, len - P8_TAIL_BYTES, -1);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES, -1);
EXPECT_FALSE(receive_config_frame(mut, len));
/* A negative (sentinel) and an extreme copy-as uid/gid are refused. */
memcpy(mut, frame, len);
put_i32(mut, len - P8_TAIL_BYTES, SUPER_MODE_AUTO);
put_i32(mut, len - P8_TAIL_BYTES + 4, 1);
put_i32(mut, len - P8_TAIL_BYTES + 8, -1);
put_i32(mut, len - P8_TAIL_BYTES + 12, 0);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES, SUPER_MODE_AUTO);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES + 4, 1);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES + 8, -1);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES + 12, 0);
EXPECT_FALSE(receive_config_frame(mut, len));
put_i32(mut, len - P8_TAIL_BYTES + 8, 0);
put_i32(mut, len - P8_TAIL_BYTES + 12, INT32_MIN);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES + 8, 0);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES + 12, INT32_MIN);
EXPECT_FALSE(receive_config_frame(mut, len));
/* A presence int that is not a wire bool is refused. */
memcpy(mut, frame, len);
put_i32(mut, len - P8_TAIL_BYTES, SUPER_MODE_AUTO);
put_i32(mut, len - P8_TAIL_BYTES + 4, 2);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES, SUPER_MODE_AUTO);
put_i32(mut, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES + 4, 2);
EXPECT_FALSE(receive_config_frame(mut, len));
/* Truncating anywhere inside the P8 tail is refused. */
EXPECT_FALSE(receive_config_frame(frame, len - 2));
EXPECT_FALSE(receive_config_frame(frame, len - P8_TAIL_BYTES));
EXPECT_FALSE(receive_config_frame(frame, len - OUTPUT_TAIL_BYTES - P8_TAIL_BYTES));
free(mut);
free(frame);
@@ -379,7 +382,9 @@ static void test_fuzz_config_receive_huge_map_count() {
}
c->usermap_count = 1;
c->usermap[0].from = sentinel_from;
c->usermap[0].from_hi = sentinel_from;
c->usermap[0].to = sentinel_to;
c->usermap[0].to_name = NULL;
unsigned char* frame = NULL;
size_t len = 0;
@@ -390,9 +395,12 @@ static void test_fuzz_config_receive_huge_map_count() {
return;
}
unsigned char pattern[8];
/* One wire entry is [from][from_hi][to][to_name]; search the fixed-width
prefix (the to_name length-prefixed string follows). */
unsigned char pattern[12];
memcpy(pattern, &sentinel_from, sizeof(sentinel_from));
memcpy(pattern + sizeof(sentinel_from), &sentinel_to, sizeof(sentinel_to));
memcpy(pattern + sizeof(sentinel_from), &sentinel_from, sizeof(sentinel_from));
memcpy(pattern + 2 * sizeof(sentinel_from), &sentinel_to, sizeof(sentinel_to));
size_t entry_off = find_bytes(frame, len, pattern, sizeof(pattern));
if (entry_off == SIZE_MAX || entry_off < sizeof(int32_t)) {
free(frame);
+70 -5
View File
@@ -448,9 +448,9 @@ static void test_file_restore_executability_rsync_rule() {
/* The shared metadata_mode_for_policy() helper is the single source of truth
* used by both the normal metadata path and the --fake-super replay. It must
* reproduce the per-attribute split: no mode change when neither -p nor -E is
* set; -p applies the sanitized source mode (group/other write cleared)
* regardless of the destination; -E derives exec bits from the destination and
* --perms wins when both are set. */
* set; -p applies the source mode exactly (including group/other write and the
* setuid/setgid/sticky bits) regardless of the destination; -E derives exec
* bits from the destination and --perms wins when both are set. */
static void test_metadata_mode_for_policy() {
mode_t out = 0xdead;
EXPECT_FALSE(
@@ -459,7 +459,13 @@ static void test_metadata_mode_for_policy() {
EXPECT_TRUE(
metadata_mode_for_policy(0777, 0644, (FileAttrPolicy){true, false, false, false}, &out));
EXPECT_EQ_INT((int)(out & 0777), 0755); /* group/other write always cleared */
EXPECT_EQ_INT((int)(out & 0777), 0777); /* group/other write is preserved */
mode_t specials = (mode_t)(S_ISUID | S_ISGID | S_ISVTX | 0672);
EXPECT_TRUE(
metadata_mode_for_policy(specials, 0644, (FileAttrPolicy){true, false, false, false}, &out));
EXPECT_EQ_INT((int)(out & (S_ISUID | S_ISGID | S_ISVTX | 0777)),
(int)(S_ISUID | S_ISGID | S_ISVTX | 0672));
/* -E: exec bits derive from the DESTINATION's read bits. */
EXPECT_TRUE(
@@ -566,6 +572,27 @@ static void test_file_attr_policy_from_config() {
config_delete(c);
}
/* Strict rsync parity: -p copies the source's setuid/setgid/sticky bits (they
* are attempted, not masked away). On Linux these are settable on a file the
* receiving user owns; a mount that denies them would log a chmod failure. */
static void test_perms_preserves_special_bits() {
const char* path = "temp_special_bits.txt";
unlink(path);
FileMetadata m = {
.mode = (mode_t)(S_ISUID | S_ISGID | S_ISVTX | 0755), .uid = getuid(), .gid = getgid()};
bool ok = file_to_disk_secure_attrs(path, "x", 1, false, false, false, &m,
(FileAttrPolicy){true, false, false, false}, false, false,
false, NULL, false, false, NULL);
EXPECT_TRUE(ok);
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0755);
EXPECT_EQ_INT((int)(st.st_mode & (S_ISUID | S_ISGID | S_ISVTX)),
(int)(S_ISUID | S_ISGID | S_ISVTX));
unlink(path);
}
static void test_chmod_changes() {
mode_t result;
EXPECT_TRUE(chmod_apply(0777, "u=rw,go=r", &result));
@@ -583,8 +610,45 @@ static void test_chmod_changes() {
EXPECT_EQ_INT(result, 0755);
EXPECT_FALSE(chmod_apply(0777, "888", &result));
EXPECT_FALSE(chmod_apply(0777, "10000", &result));
EXPECT_FALSE(chmod_apply(0777, "a+X", &result));
EXPECT_FALSE(chmod_apply(0777, "a+r,", &result));
/* go+w is honored (rsync gives 0666 from a 0644 file). */
EXPECT_TRUE(chmod_apply(0644, "go+w", &result));
EXPECT_EQ_INT(result, 0666);
/* X only sets execute on directories or already-executable files. */
EXPECT_TRUE(chmod_apply(0644, "a+X", &result));
EXPECT_EQ_INT(result, 0644);
EXPECT_TRUE(chmod_apply(0755, "a+X", &result));
EXPECT_EQ_INT(result, 0755);
EXPECT_TRUE(chmod_apply((mode_t)(S_IFDIR | 0644), "a+X", &result));
EXPECT_EQ_INT((int)(result & 0777), 0755);
EXPECT_TRUE(S_ISDIR(result));
/* D/F selectors restrict a clause to directories/files. */
EXPECT_TRUE(chmod_apply((mode_t)(S_IFDIR | 0700), "Dg+s", &result));
EXPECT_EQ_INT((int)(result & 07777), 02700);
EXPECT_TRUE(chmod_apply((mode_t)(S_IFREG | 0644), "Dg+s", &result));
EXPECT_EQ_INT((int)(result & 07777), 0644);
EXPECT_TRUE(chmod_apply((mode_t)(S_IFREG | 0644), "Fo-w", &result));
EXPECT_EQ_INT((int)(result & 07777), 0644);
EXPECT_TRUE(chmod_apply((mode_t)(S_IFREG | 0666), "Fo-w", &result));
EXPECT_EQ_INT((int)(result & 07777), 0664);
EXPECT_TRUE(chmod_apply((mode_t)(S_IFDIR | 0666), "Fo-w", &result));
EXPECT_EQ_INT((int)(result & 07777), 0666);
EXPECT_FALSE(chmod_apply(0644, "DFu+w", &result));
/* Special bits: s/t map to setuid/setgid/sticky like rsync. */
EXPECT_TRUE(chmod_apply(0755, "u+s", &result));
EXPECT_EQ_INT((int)(result & 07777), 04755);
EXPECT_TRUE(chmod_apply(0755, "g+s", &result));
EXPECT_EQ_INT((int)(result & 07777), 02755);
EXPECT_TRUE(chmod_apply(0755, "a+t", &result));
EXPECT_EQ_INT((int)(result & 07777), 01755);
/* Comma-separated clauses accumulate (the CLI joins repeated options). */
EXPECT_TRUE(chmod_apply(0644, "g+w,u+x", &result));
EXPECT_EQ_INT((int)(result & 07777), 0764);
}
/* P7 Wave D: symlink metadata is applied with no-follow primitives, and -J
@@ -722,5 +786,6 @@ void test_metadata() {
test_file_restore_metadata_fd_attribute_split();
test_file_attr_policy_from_config();
test_file_restore_symlink_metadata();
test_perms_preserves_special_bits();
test_chmod_changes();
}
+13 -3
View File
@@ -522,10 +522,19 @@ static void test_data_create_starts_uncharged_and_unowned() {
data_destroy(reserved);
}
/* The server floors a client --timeout=0 at SERVER_IO_TIMEOUT_SEC so a silent
* peer can never hold a session slot forever (slow-loris). */
static void test_protocol_server_io_timeout_floor() {
EXPECT_EQ_INT(protocol_server_io_timeout_sec(0), SERVER_IO_TIMEOUT_SEC);
EXPECT_EQ_INT(protocol_server_io_timeout_sec(-7), SERVER_IO_TIMEOUT_SEC);
EXPECT_EQ_INT(protocol_server_io_timeout_sec(30), 30);
EXPECT_TRUE(SERVER_IO_TIMEOUT_SEC > 0);
}
static void test_protocol_session_io_timeout() {
/* Default is the built-in 60 s window; the setter stores exactly what it is
* given (<= 0 means "fall back to the default") so callers can propagate
* --timeout without special-casing 0. */
/* The default is the built-in 60 s window; the setter stores exactly what it
* is given (<= 0 disables the deadline, matching rsync's --timeout=0) so
* callers can propagate --timeout without special-casing 0. */
ProtocolSession session;
protocol_session_init(&session, -1, -1);
EXPECT_EQ_INT(session.io_timeout_sec, 60);
@@ -670,6 +679,7 @@ void test_protocol() {
test_send_receive_int();
test_send_receive_status();
test_protocol_session_io_timeout();
test_protocol_server_io_timeout_floor();
test_send_receive_status_timed();
test_receive_status_keepalive_skips_reply();
test_receive_status_keepalive_aborts();
+34 -3
View File
@@ -644,17 +644,19 @@ static void test_scanner_one_file_system_cross_device() {
EXPECT_EQ_INT(seq_off_rc, 0);
EXPECT_TRUE(seq_off_found);
EXPECT_EQ_INT(seq_off_total, 2);
/* Sequential: with -x the cross-device subtree is dropped, keep.txt remains. */
/* Sequential: with -x the cross-device subtree is not descended into, but
* rsync-compatible behavior still emits the mount-point directory entry as an
* empty directory File, so keep.txt plus that entry are present. */
EXPECT_EQ_INT(seq_on_rc, 0);
EXPECT_FALSE(seq_on_found);
EXPECT_EQ_INT(seq_on_total, 1);
EXPECT_EQ_INT(seq_on_total, 2);
/* Parallel: same behavior, worker path (depth > 1). */
EXPECT_EQ_INT(par_off_rc, 0);
EXPECT_TRUE(par_off_found);
EXPECT_EQ_INT(par_off_total, 2);
EXPECT_EQ_INT(par_on_rc, 0);
EXPECT_FALSE(par_on_found);
EXPECT_EQ_INT(par_on_total, 1);
EXPECT_EQ_INT(par_on_total, 2);
}
/* Collect emitted file paths (relative to `root`) from a sequential scan.
@@ -884,6 +886,35 @@ static void test_filter_rules(bool parallel) {
free_paths(paths, count);
filter_rule_list_free(base);
/* The common include idiom (the exact rule order the CLI compiles from
* --include='*.txt' --exclude='*'): only .txt files survive. */
const char* idiom[] = {"+ *.txt", "- *"};
base = filter_base_build(idiom, 2, false, err, sizeof(err));
EXPECT_NOT_NULL(base);
options.base_filters = base;
rc = parallel ? collect_files_parallel(root, &options, &paths, &count)
: collect_files(root, &options, &paths, &count);
EXPECT_EQ_INT(rc, 0);
EXPECT_EQ_INT(count, 2);
EXPECT_TRUE(has_path(paths, count, "a.txt"));
EXPECT_TRUE(has_path(paths, count, "c.txt"));
EXPECT_FALSE(has_path(paths, count, "b.tmp"));
free_paths(paths, count);
filter_rule_list_free(base);
/* An include rule alone is NOT a mandatory whitelist (rsync semantics): only
* the matching file is affected, everything else is still transferred. */
const char* include_alone[] = {"+ *.txt"};
base = filter_base_build(include_alone, 1, false, err, sizeof(err));
EXPECT_NOT_NULL(base);
options.base_filters = base;
rc = parallel ? collect_files_parallel(root, &options, &paths, &count)
: collect_files(root, &options, &paths, &count);
EXPECT_EQ_INT(rc, 0);
EXPECT_EQ_INT(count, 3);
free_paths(paths, count);
filter_rule_list_free(base);
unlink("test_scan_filter/a.txt");
unlink("test_scan_filter/b.tmp");
unlink("test_scan_filter/c.txt");
+16 -4
View File
@@ -582,6 +582,7 @@ static void test_late_manifest_abort_frees_keepset() {
EXPECT_TRUE(send_str(p[1], "keep.txt"));
EXPECT_TRUE(send_int(p[1], 0)); /* protected-prefix section is empty */
EXPECT_TRUE(send_int(p[1], 0)); /* missing-args section is empty */
EXPECT_TRUE(send_int(p[1], 0)); /* synchronized-directories section is empty */
EXPECT_TRUE(send_status(p[1], STATUS_ABORT));
DeleteManifest* pending = NULL;
@@ -606,6 +607,7 @@ static void test_late_manifest_eof_frees_keepset() {
EXPECT_TRUE(send_str(p[1], "keep.txt"));
EXPECT_TRUE(send_int(p[1], 0)); /* protected-prefix section is empty */
EXPECT_TRUE(send_int(p[1], 0)); /* missing-args section is empty */
EXPECT_TRUE(send_int(p[1], 0)); /* synchronized-directories section is empty */
shutdown(p[1], SHUT_WR);
DeleteManifest* pending = NULL;
@@ -630,11 +632,13 @@ static void test_late_second_manifest_frees_both() {
EXPECT_TRUE(send_str(p[1], "first.txt"));
EXPECT_TRUE(send_int(p[1], 0)); /* protected-prefix section is empty */
EXPECT_TRUE(send_int(p[1], 0)); /* missing-args section is empty */
EXPECT_TRUE(send_int(p[1], 0)); /* synchronized-directories section is empty */
EXPECT_TRUE(send_status(p[1], STATUS_MANIFEST));
EXPECT_TRUE(send_int(p[1], 1));
EXPECT_TRUE(send_str(p[1], "second.txt"));
EXPECT_TRUE(send_int(p[1], 0)); /* protected-prefix section is empty */
EXPECT_TRUE(send_int(p[1], 0)); /* missing-args section is empty */
EXPECT_TRUE(send_int(p[1], 0)); /* synchronized-directories section is empty */
DeleteManifest* pending = NULL;
EXPECT_EQ_INT(run_pending_receiver(cfg, p[0], &pending), -1);
@@ -645,9 +649,10 @@ static void test_late_second_manifest_frees_both() {
config_delete(cfg);
}
/* A delete-manifest frame with a third (missing-args) section round-trips: the
receiver keeps all three sections and the missing paths are confined exactly
like the keep-set (a traversal entry in the missing section is rejected).
/* A delete-manifest frame with all four sections round-trips: the receiver
keeps the keep-set, protected prefixes, missing-args paths and synchronized
directories, and every section is confined exactly like the keep-set (a
traversal entry in the missing section is rejected).
receive_manifest_entries() reads the counts directly (the leading
STATUS_MANIFEST code is consumed by the caller, so these frames do not send
it). */
@@ -666,6 +671,9 @@ static void test_receive_manifest_three_sections() {
EXPECT_TRUE(send_int(p[1], 2));
EXPECT_TRUE(send_str(p[1], "gone.txt"));
EXPECT_TRUE(send_str(p[1], "dir/gone.bin"));
EXPECT_TRUE(send_int(p[1], 2));
EXPECT_TRUE(send_str(p[1], "."));
EXPECT_TRUE(send_str(p[1], "dir"));
DeleteManifest* manifest = receive_manifest_entries(p[0]);
EXPECT_NOT_NULL(manifest);
@@ -676,9 +684,12 @@ static void test_receive_manifest_three_sections() {
EXPECT_EQ_INT(manifest->missing->size, 2);
EXPECT_EQ_STR((char*)manifest->missing->items[0], "gone.txt");
EXPECT_EQ_STR((char*)manifest->missing->items[1], "dir/gone.bin");
EXPECT_EQ_INT(manifest->dirs->size, 2);
EXPECT_EQ_STR((char*)manifest->dirs->items[0], ".");
EXPECT_EQ_STR((char*)manifest->dirs->items[1], "dir");
delete_manifest_free(manifest);
/* A traversal entry in the third section is rejected like every other. */
/* A traversal entry in the missing section is rejected like every other. */
EXPECT_TRUE(send_int(p[1], 0));
EXPECT_TRUE(send_int(p[1], 0));
EXPECT_TRUE(send_int(p[1], 1));
@@ -780,6 +791,7 @@ static void test_receiver_pending_commits_missing_args() {
EXPECT_TRUE(send_int(p[1], 2));
EXPECT_TRUE(send_str(p[1], "gone.txt"));
EXPECT_TRUE(send_str(p[1], "never_here.txt"));
EXPECT_TRUE(send_int(p[1], 0)); /* no synchronized directories */
EXPECT_TRUE(send_status(p[1], STATUS_FINISHED));
/* NULL pending: the single-threaded commit path deletes at FINISHED. The
+91 -59
View File
@@ -136,7 +136,8 @@ static void test_walker_removes_extras_keeps_manifest_and_protected() {
EXPECT_NOT_NULL(manifest);
DeleteSkipEntry skip = {"prot", false};
size_t deleted = 0;
DeleteWalkResult result = delete_extras_limited(root, manifest, 100000, &skip, 1, &deleted);
DeleteWalkResult result =
delete_extras_limited(root, manifest, NULL, 100000, &skip, 1, &deleted, NULL);
EXPECT_EQ_INT((int)result, (int)DELETE_WALK_OK);
EXPECT_FALSE(file_exists(root, "a.txt"));
EXPECT_TRUE(file_exists(root, "keep.txt"));
@@ -170,7 +171,8 @@ static void test_walker_keeps_nested_manifest_dirs() {
ArrayList* manifest = make_manifest_strings(keeps, 3);
EXPECT_NOT_NULL(manifest);
size_t deleted = 0;
DeleteWalkResult result = delete_extras_limited(root, manifest, 100000, NULL, 0, &deleted);
DeleteWalkResult result =
delete_extras_limited(root, manifest, NULL, 100000, NULL, 0, &deleted, NULL);
EXPECT_EQ_INT((int)result, (int)DELETE_WALK_OK);
EXPECT_FALSE(file_exists(root, "extra.txt"));
EXPECT_TRUE(file_exists(root, "keepdir/deep/keep.txt"));
@@ -187,7 +189,9 @@ static void test_walker_keeps_nested_manifest_dirs() {
free(root);
}
static void test_walker_max_delete_exceeded_deletes_nothing() {
/* --max-delete is a partial cap (rsync parity): delete up to the limit, skip
the rest, and report DELETE_WALK_LIMIT_REACHED. */
static void test_walker_max_delete_partial_deletes_up_to_cap() {
char* root = make_walk_root("maxdel");
EXPECT_NOT_NULL(root);
EXPECT_TRUE(write_file_at(root, "a.txt", "extra"));
@@ -197,12 +201,15 @@ static void test_walker_max_delete_exceeded_deletes_nothing() {
ArrayList* manifest = make_manifest_strings(keeps, 0);
EXPECT_NOT_NULL(manifest);
size_t deleted = 999;
DeleteWalkResult result = delete_extras_limited(root, manifest, 2, NULL, 0, &deleted);
EXPECT_EQ_INT((int)result, (int)DELETE_WALK_LIMIT_EXCEEDED);
EXPECT_EQ_INT((int)deleted, 0);
EXPECT_TRUE(file_exists(root, "a.txt"));
EXPECT_TRUE(file_exists(root, "b.txt"));
EXPECT_TRUE(file_exists(root, "c.txt"));
size_t skipped = 0;
DeleteWalkResult result =
delete_extras_limited(root, manifest, NULL, 2, NULL, 0, &deleted, &skipped);
EXPECT_EQ_INT((int)result, (int)DELETE_WALK_LIMIT_REACHED);
EXPECT_EQ_INT((int)deleted, 2);
EXPECT_EQ_INT((int)skipped, 1);
int remaining = (file_exists(root, "a.txt") ? 1 : 0) + (file_exists(root, "b.txt") ? 1 : 0) +
(file_exists(root, "c.txt") ? 1 : 0);
EXPECT_EQ_INT(remaining, 1);
array_list_delete(manifest);
remove_walk_tree(root);
free(root);
@@ -217,7 +224,7 @@ static void test_walker_max_delete_exact_bound_deletes() {
ArrayList* manifest = make_manifest_strings(keeps, 0);
EXPECT_NOT_NULL(manifest);
size_t deleted = 0;
DeleteWalkResult result = delete_extras_limited(root, manifest, 2, NULL, 0, &deleted);
DeleteWalkResult result = delete_extras_limited(root, manifest, NULL, 2, NULL, 0, &deleted, NULL);
EXPECT_EQ_INT((int)result, (int)DELETE_WALK_OK);
EXPECT_EQ_INT((int)deleted, 2);
EXPECT_FALSE(file_exists(root, "a.txt"));
@@ -227,6 +234,77 @@ static void test_walker_max_delete_exact_bound_deletes() {
free(root);
}
/* Extraneous destination symlinks (including one pointing at a directory) must
be unlinked, never followed, so their targets survive. */
static void test_walker_removes_extraneous_symlinks() {
char* root = make_walk_root("symlink");
char* outside = make_walk_root("symlink_out");
EXPECT_NOT_NULL(root);
EXPECT_NOT_NULL(outside);
EXPECT_TRUE(write_file_at(outside, "secret.txt", "keep"));
EXPECT_TRUE(write_file_at(root, "keep.txt", "kept"));
char* link_file = path_cat(root, "link_file");
char* link_dir = path_cat(root, "link_dir");
char* link_broken = path_cat(root, "link_broken");
EXPECT_NOT_NULL(link_file);
EXPECT_NOT_NULL(link_dir);
EXPECT_NOT_NULL(link_broken);
EXPECT_EQ_INT(symlink("keep.txt", link_file), 0);
EXPECT_EQ_INT(symlink(outside, link_dir), 0);
EXPECT_EQ_INT(symlink("/nonexistent-target", link_broken), 0);
const char* keeps[] = {"keep.txt"};
ArrayList* manifest = make_manifest_strings(keeps, 1);
EXPECT_NOT_NULL(manifest);
size_t deleted = 0;
DeleteWalkResult result =
delete_extras_limited(root, manifest, NULL, 100000, NULL, 0, &deleted, NULL);
EXPECT_EQ_INT((int)result, (int)DELETE_WALK_OK);
EXPECT_FALSE(file_exists(root, "link_file"));
EXPECT_FALSE(file_exists(root, "link_dir"));
EXPECT_FALSE(file_exists(root, "link_broken"));
EXPECT_TRUE(file_exists(root, "keep.txt"));
EXPECT_TRUE(file_exists(outside, "secret.txt"));
free(link_file);
free(link_dir);
free(link_broken);
array_list_delete(manifest);
remove_walk_tree(root);
remove_walk_tree(outside);
free(root);
free(outside);
}
/* With a synchronized-dir set, extras outside it survive while extras directly
inside a listed directory are removed; the receive root is the "." sentinel. */
static void test_walker_confines_deletion_to_synced_dirs() {
char* root = make_walk_root("synced");
EXPECT_NOT_NULL(root);
EXPECT_TRUE(write_file_at(root, "rootextra.txt", "keep"));
EXPECT_EQ_INT(make_subdir(root, "inscope"), 0);
EXPECT_TRUE(write_file_at(root, "inscope/extra.txt", "delete"));
EXPECT_TRUE(write_file_at(root, "inscope/keep.txt", "kept"));
EXPECT_EQ_INT(make_subdir(root, "outscope"), 0);
EXPECT_TRUE(write_file_at(root, "outscope/extra.txt", "keep"));
const char* keeps[] = {"inscope/keep.txt"};
ArrayList* manifest = make_manifest_strings(keeps, 1);
ArrayList* dirs = array_list_create(free);
EXPECT_NOT_NULL(manifest);
EXPECT_NOT_NULL(dirs);
EXPECT_TRUE(array_list_add(dirs, str_dup("inscope")));
size_t deleted = 0;
DeleteWalkResult result =
delete_extras_limited(root, manifest, dirs, 100000, NULL, 0, &deleted, NULL);
EXPECT_EQ_INT((int)result, (int)DELETE_WALK_OK);
EXPECT_TRUE(file_exists(root, "rootextra.txt"));
EXPECT_FALSE(file_exists(root, "inscope/extra.txt"));
EXPECT_TRUE(file_exists(root, "inscope/keep.txt"));
EXPECT_TRUE(file_exists(root, "outscope/extra.txt"));
array_list_delete(manifest);
array_list_delete(dirs);
remove_walk_tree(root);
free(root);
}
static void test_walker_unlimited_deletes_all() {
char* root = make_walk_root("unlim");
EXPECT_NOT_NULL(root);
@@ -245,53 +323,6 @@ static void test_walker_unlimited_deletes_all() {
free(root);
}
/* The 100000-entry server hard bound (MAX_SERVER_DELETE_COUNT, which this test
exercises through a literal to avoid reaching into file_receive.c) is also
all-or-nothing: a destination holding more extras than the bound must be left
completely untouched. Skipped under valgrind: 100k file creations would be
far too slow under instrumentation. */
static void test_walker_hard_bound_all_or_nothing() {
if (is_running_under_valgrind())
return;
enum { HARD_BOUND = 100000 };
char* root = make_walk_root("hardbound");
EXPECT_NOT_NULL(root);
int rootfd = open(root, O_RDONLY | O_DIRECTORY | O_CLOEXEC);
EXPECT_TRUE(rootfd >= 0);
bool created = true;
for (int i = 0; created && i < HARD_BOUND + 1; i++) {
char name[32];
snprintf(name, sizeof(name), "f%d", i);
int fd = openat(rootfd, name, O_WRONLY | O_CREAT | O_TRUNC, 0644);
if (fd < 0)
created = false;
else
close(fd);
}
EXPECT_TRUE(created);
const char* keeps[1] = {NULL};
ArrayList* manifest = make_manifest_strings(keeps, 0);
EXPECT_NOT_NULL(manifest);
size_t deleted = 999;
DeleteWalkResult result = delete_extras_limited(root, manifest, HARD_BOUND, NULL, 0, &deleted);
EXPECT_EQ_INT((int)result, (int)DELETE_WALK_LIMIT_EXCEEDED);
EXPECT_EQ_INT((int)deleted, 0);
EXPECT_TRUE(file_exists(root, "f0"));
EXPECT_TRUE(file_exists(root, "f100000"));
array_list_delete(manifest);
/* Fast cleanup: unlink every created name through the still-open root fd. */
if (rootfd >= 0) {
for (int i = 0; i < HARD_BOUND + 1; i++) {
char name[32];
snprintf(name, sizeof(name), "f%d", i);
(void)unlinkat(rootfd, name, 0);
}
close(rootfd);
}
rmdir(root);
free(root);
}
typedef struct {
bool eight_bit_output;
const char* expected;
@@ -553,10 +584,11 @@ void test_shared_utils() {
test_getdelim_bounded();
test_walker_removes_extras_keeps_manifest_and_protected();
test_walker_keeps_nested_manifest_dirs();
test_walker_max_delete_exceeded_deletes_nothing();
test_walker_max_delete_partial_deletes_up_to_cap();
test_walker_max_delete_exact_bound_deletes();
test_walker_removes_extraneous_symlinks();
test_walker_confines_deletion_to_synced_dirs();
test_walker_unlimited_deletes_all();
test_walker_hard_bound_all_or_nothing();
test_loopback_helpers();
test_fd_peer_ip();
+3
View File
@@ -156,6 +156,9 @@ static void test_tcp_set_timeouts() {
tcp_set_timeouts(-1, -1);
EXPECT_EQ_INT(tcp_get_timeout_sec(), 0);
EXPECT_EQ_INT(tcp_get_contimeout_sec(), 0);
/* Restore finite defaults so later tests that rely on a bounded connect/IO
* timeout (e.g. connecting to a non-routable address) cannot block forever. */
tcp_set_timeouts(30, 10);
}
/* Test client_connect with an invalid host (should fail gracefully) */
+106 -103
View File
@@ -377,13 +377,12 @@ static void test_fake_super_restore() {
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)(st.st_mode & 07777), 0751);
/* Mode sanitization: the normal metadata path never grants group/other write
bits, and fake-super replay must not re-add them (a recorded 0666 restores
as 0644, never as world-writable). */
/* Strict rsync parity: -p restores the recorded mode exactly, including
group/other write (a recorded 0666 restores as 0666). */
fake_super_store_fd(fd, 1001, 1002, 0666, 1700000000, 0);
EXPECT_TRUE(fake_super_restore_fd(fd, policy));
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0644);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0666);
/* Restore with a malformed record must skip without failing. */
time_t before = st.st_mtime;
@@ -400,14 +399,12 @@ static void test_fake_super_restore() {
unlink(path);
}
/* --fake-super owner replay must honor the super gate and copy-as authority:
--no-super suppresses the recorded-source-owner chown even for root, and an
active --copy-as keeps its forced owner (the recorded source owner must never
override it). Root-gated: only root can observe a chown actually landing. */
static void test_fake_super_owner_gate() {
if (geteuid() != 0)
return; /* non-root cannot observe ownership changes; skip silently */
const char* path = "test_fake_super_owner_gate.txt";
/* --fake-super must NEVER perform a real chown: fake_super_restore_fd applies
* only mode/mtime and leaves the entry's uid/gid exactly as they were, even
* when an explicit ownership policy is active and super_mode permits it. This
* is observable unprivileged (the file's owner is simply unchanged). */
static void test_fake_super_no_real_chown() {
const char* path = "test_fake_super_nochown.txt";
unlink(path);
int fd = open(path, O_WRONLY | O_CREAT | O_TRUNC, 0600);
if (fd < 0)
@@ -416,63 +413,34 @@ static void test_fake_super_owner_gate() {
if (has_xattr)
removexattr(path, "user.fastsync.xprobe");
if (!has_xattr) {
close(fd);
unlink(path);
return; /* filesystem without xattr support */
}
if (fchown(fd, 0, 0) != 0) {
close(fd);
unlink(path);
return;
}
struct stat before;
EXPECT_EQ_INT(fstat(fd, &before), 0);
fake_super_store_fd(fd, 12345, 12346, 0755, 1700000000, 0);
Config* c = config_create();
FileAttrPolicy policy = {true, true, false, false};
EXPECT_NOT_NULL(c);
/* An explicit ownership policy is required before fake-super replay may
chown; --fake-super alone only records the source owner (A2). */
c->numeric_ids = true;
/* --no-super: the owner leg is skipped even as root. */
c->super_mode = SUPER_MODE_OFF;
EXPECT_TRUE(identity_set_active(c));
EXPECT_TRUE(fake_super_restore_fd(fd, policy));
struct stat st;
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)st.st_uid, 0);
EXPECT_EQ_INT((int)st.st_gid, 0);
/* AUTO with an identity policy: the recorded source owner is applied. */
c->super_mode = SUPER_MODE_AUTO;
EXPECT_TRUE(identity_set_active(c));
EXPECT_TRUE(fake_super_restore_fd(fd, policy));
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)st.st_uid, 12345);
EXPECT_EQ_INT((int)st.st_gid, 12346);
/* --super / --fake-super with NO explicit identity flag must NOT apply a
client-chosen owner: super_mode alone never enables ownership. */
EXPECT_EQ_INT(fchown(fd, 0, 0), 0);
c->numeric_ids = false;
/* The strongest ownership request available plus permitted super mode. */
c->preserve_owner = true;
c->preserve_group = true;
c->chown_uid_set = true;
c->chown_uid = 12345;
c->chown_gid_set = true;
c->chown_gid = 12346;
c->super_mode = SUPER_MODE_ON;
c->fake_super = true;
EXPECT_TRUE(identity_set_active(c));
EXPECT_TRUE(fake_super_restore_fd(fd, policy));
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)st.st_uid, 0);
EXPECT_EQ_INT((int)st.st_gid, 0);
/* Active --copy-as is authoritative: the recorded source owner must not
override it, even with AUTO/ON. */
c->copy_as_set = true;
c->copy_as_uid = 777;
c->copy_as_gid = 778;
EXPECT_TRUE(identity_set_active(c));
EXPECT_TRUE(fake_super_restore_fd(fd, policy));
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)st.st_uid, 0);
EXPECT_EQ_INT((int)st.st_gid, 0);
struct stat after;
EXPECT_EQ_INT(fstat(fd, &after), 0);
EXPECT_EQ_INT((int)after.st_uid, (int)before.st_uid);
EXPECT_EQ_INT((int)after.st_gid, (int)before.st_gid);
/* Mode is still replayed (policy-gated). */
EXPECT_EQ_INT((int)(after.st_mode & 0777), 0755);
identity_clear_active();
config_delete(c);
@@ -480,64 +448,99 @@ static void test_fake_super_owner_gate() {
unlink(path);
}
/* MAJOR 1: the --fake-super owner replay must honor the per-side -o/-g split.
* With only -o (preserve_owner) requested the recorded GROUP must be left
* untouched, and with only -g (preserve_group) the recorded OWNER must be left
* untouched. Root-gated: only root can observe a chown actually landing. */
static void test_fake_super_owner_group_split() {
if (geteuid() != 0)
return; /* non-root cannot observe ownership changes; skip silently */
const char* path = "test_fake_super_owner_group_split.txt";
unlink(path);
int fd = open(path, O_WRONLY | O_CREAT | O_TRUNC, 0600);
if (fd < 0)
return;
bool has_xattr = setxattr(path, "user.fastsync.xprobe", "p", 1, 0) == 0;
if (has_xattr)
removexattr(path, "user.fastsync.xprobe");
if (!has_xattr) {
close(fd);
unlink(path);
return; /* filesystem without xattr support */
}
if (fchown(fd, 0, 0) != 0) {
close(fd);
unlink(path);
return;
}
fake_super_store_fd(fd, 12345, 12346, 0755, 1700000000, 0);
/* identity_resolve_storage_ids() is what --fake-super RECORDS: the resolved
* mapping for a requested side, and the source's own id for a side never
* requested. Also pins the #286 rule that --numeric-ids alone never activates
* ownership (it is only a mapping modifier). */
static void test_fake_super_storage_resolution() {
uint32_t uid = 0, gid = 0;
/* --numeric-ids alone is INERT: no ownership request, storage unchanged. */
Config* c = config_create();
FileAttrPolicy policy = {true, true, false, false};
EXPECT_NOT_NULL(c);
struct stat st;
c->numeric_ids = true;
EXPECT_TRUE(identity_set_active(c));
EXPECT_FALSE(identity_active_enabled());
EXPECT_FALSE(identity_owner_requested());
EXPECT_FALSE(identity_group_requested());
identity_resolve_storage_ids(12345, 6789, &uid, &gid);
EXPECT_EQ_INT((int)uid, 12345);
EXPECT_EQ_INT((int)gid, 6789);
config_delete(c);
/* -o only: the owner is applied, the group stays at its current value (0). */
/* --fake-super with no ownership request records the raw source ids. */
c = config_create();
EXPECT_NOT_NULL(c);
c->fake_super = true;
EXPECT_TRUE(identity_set_active(c));
identity_resolve_storage_ids(12345, 6789, &uid, &gid);
EXPECT_EQ_INT((int)uid, 12345);
EXPECT_EQ_INT((int)gid, 6789);
/* -o + --numeric-ids: raw owner, un-requested group stays the source gid. */
c->preserve_owner = true;
c->preserve_group = false;
c->numeric_ids = true;
EXPECT_TRUE(identity_set_active(c));
EXPECT_TRUE(fake_super_restore_fd(fd, policy));
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)st.st_uid, 12345);
EXPECT_EQ_INT((int)st.st_gid, 0);
identity_resolve_storage_ids(12345, 6789, &uid, &gid);
EXPECT_EQ_INT((int)uid, 12345);
EXPECT_EQ_INT((int)gid, 6789);
/* -g only: the group is applied, the owner stays at its current value (0). */
EXPECT_EQ_INT(fchown(fd, 0, 0), 0);
c->preserve_owner = false;
c->preserve_group = true;
/* --chown overrides both sides. */
c->chown_uid_set = true;
c->chown_uid = 777;
c->chown_gid_set = true;
c->chown_gid = 778;
EXPECT_TRUE(identity_set_active(c));
EXPECT_TRUE(fake_super_restore_fd(fd, policy));
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)st.st_uid, 0);
EXPECT_EQ_INT((int)st.st_gid, 12346);
identity_resolve_storage_ids(12345, 6789, &uid, &gid);
EXPECT_EQ_INT((int)uid, 777);
EXPECT_EQ_INT((int)gid, 778);
/* A usermap match beats --chown on the owner side only. */
c->usermap_count = 1;
c->usermap = calloc(1, sizeof(IdentityMap));
EXPECT_NOT_NULL(c->usermap);
c->usermap[0].from = IDENTITY_MATCH_ANY;
c->usermap[0].to = 999;
EXPECT_TRUE(identity_set_active(c));
identity_resolve_storage_ids(12345, 6789, &uid, &gid);
EXPECT_EQ_INT((int)uid, 999);
EXPECT_EQ_INT((int)gid, 778);
/* --copy-as is authoritative for both sides. */
c->copy_as_set = true;
c->copy_as_uid = 111;
c->copy_as_gid = 222;
EXPECT_TRUE(identity_set_active(c));
identity_resolve_storage_ids(12345, 6789, &uid, &gid);
EXPECT_EQ_INT((int)uid, 111);
EXPECT_EQ_INT((int)gid, 222);
identity_clear_active();
config_delete(c);
close(fd);
unlink(path);
}
/* xattr_list_clone deep-copies names/values (used by the deferred directory
* metadata accumulator), so the clone stays valid after the original is freed. */
static void test_xattr_list_clone() {
EXPECT_NULL(xattr_list_clone(NULL));
FileXattrList* list = xattr_list_new();
EXPECT_NOT_NULL(list);
EXPECT_TRUE(xattr_list_append(list, "user.a", "1", 1));
EXPECT_TRUE(xattr_list_append(list, "user.b", "22", 2));
FileXattrList* clone = xattr_list_clone(list);
EXPECT_NOT_NULL(clone);
EXPECT_EQ_INT(clone->count, 2);
EXPECT_EQ_STR(clone->items[0].name, "user.a");
EXPECT_EQ_INT((int)clone->items[1].value_len, 2);
EXPECT_TRUE(memcmp(clone->items[1].value, "22", 2) == 0);
EXPECT_TRUE(clone->items[0].name != list->items[0].name);
xattr_list_free(list);
EXPECT_EQ_STR(clone->items[0].name, "user.a");
xattr_list_free(clone);
}
void test_xattr() {
test_xattr_list_clone();
test_xattr_wire_roundtrip();
test_xattr_reject_privileged_namespace();
test_xattr_reject_oversized_value();
@@ -547,6 +550,6 @@ void test_xattr() {
test_xattr_receive_drops_acl_without_preserve_acls();
test_link_copy_fallback_preserves_xattrs();
test_fake_super_restore();
test_fake_super_owner_gate();
test_fake_super_owner_group_split();
test_fake_super_no_real_chown();
test_fake_super_storage_resolution();
}