Merge PR #309: parity burn-down
CI / lint (push) Successful in 1m46s
CI / parity-fast (push) Skipped
CI / parity-full (push) Successful in 22s
CI / sanitizers (address) (push) Failing after 54s
CI / sanitizers (undefined) (push) Successful in 43s
CI / build-and-test (push) Successful in 1m22s
CI / fuzz-build (push) Successful in 49s
CI / coverage (push) Successful in 44s
CI / valgrind (push) Successful in 2m19s
CI / lint (push) Successful in 1m46s
CI / parity-fast (push) Skipped
CI / parity-full (push) Successful in 22s
CI / sanitizers (address) (push) Failing after 54s
CI / sanitizers (undefined) (push) Successful in 43s
CI / build-and-test (push) Successful in 1m22s
CI / fuzz-build (push) Successful in 49s
CI / coverage (push) Successful in 44s
CI / valgrind (push) Successful in 2m19s
This commit was merged in pull request #309.
This commit is contained in:
@@ -79,6 +79,24 @@ of 157 rows.
|
||||
- **Daemon umask no longer forced to `0`.** `daemonize()` now sets the
|
||||
conventional `022`, so implied parent directories created without `-p` are no
|
||||
longer world-writable `0777`.
|
||||
- **Daemon modules are read-only by default.** A `--daemon` module is now
|
||||
served read-only unless it sets `read only = no` (or rsync's `write only =
|
||||
yes`), matching rsync: a real `rsyncd.conf` that omits `read only` is no
|
||||
longer silently writable. A global `read only` still sets the default for
|
||||
later modules, and an explicit module value wins. This is a behavior change
|
||||
for existing FastSync-native configs that relied on the old writable default;
|
||||
add `read only = no` to keep them writable. An rsync `write only = yes` is
|
||||
mapped to writability (FastSync is push-only, so a module can never be read
|
||||
from the network).
|
||||
- **Accepted-but-unenforced rsync security keys now warn at startup.** The
|
||||
rsync keys FastSync recognizes but does not implement — `secrets file`,
|
||||
`refuse options`, `exclude`/`include`/`filter`, `max size`/`min size`,
|
||||
`pre-xfer exec`/`post-xfer exec`, `incoming chmod`/`outgoing chmod`,
|
||||
`name converter`, `use chroot`, `uid`/`gid`, and the rest of the
|
||||
access-control set — load for migration compatibility but now emit a
|
||||
`WARN` naming the key (and module) so an operator does not believe the
|
||||
restriction is enforced. `auth users`/`secrets file` stay fail-closed: a
|
||||
module declaring `auth users` still requires a FastSync credential store.
|
||||
- **Credentials and signal handling hardened.** Secret files are opened with
|
||||
`O_NOFOLLOW|O_NONBLOCK` (while allowing fd-backed store paths and bound-waiting
|
||||
a FIFO read for ~3 s so a slow process substitution works but a connected-but-
|
||||
|
||||
@@ -187,7 +187,7 @@ This produces `./build/client` and `./build/server`. `compile_commands.json` is
|
||||
| `--groupmap=MAP` | Map group names when applying ownership |
|
||||
| `--numeric-ids` | Apply source numeric uid/gid directly instead of mapping by name |
|
||||
| `--copy-as=USER[:GROUP]` | Force every written entry to USER[:GROUP] (requires a privileged receiver) |
|
||||
| `--fake-super` | Record the resolved owner plus mode/time in a reserved `user.fastsync.stat` xattr and replay mode/time; never performs a real chown |
|
||||
| `--fake-super` | Record the resolved owner plus full mode/rdev in rsync's reserved `user.rsync.%stat` xattr (rsync 3.4.1 grammar) and replay the permission bits; never performs a real chown |
|
||||
| `--super` | Permit the receiver to attempt confined super-user activities (device nodes) |
|
||||
| `-D` | Preserve device and special files (implies `--devices --specials`) |
|
||||
| `--devices` | Recreate device nodes on the destination (privileged; skipped without `CAP_MKNOD`) |
|
||||
@@ -219,7 +219,7 @@ This produces `./build/client` and `./build/server`. `compile_commands.json` is
|
||||
| `--delete-excluded` | Also delete filter-excluded destination mirrors (size-pruned mirrors stay protected) |
|
||||
| `--max-delete <n>` | Delete at most n destination entries; the rest are skipped and the run exits 25 (partial), matching rsync |
|
||||
| `--delay-updates` | Put updated files into place only at the end of the transfer (`--force` is honored at publication; the fixed `.fastsync-stage` staging name diverges from rsync — see [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md)) |
|
||||
| `-T, --temp-dir <dir>` | Scratch directory for temp files before the atomic install; confined to the receive root (relative only), with an `EXDEV` non-atomic copy fallback |
|
||||
| `-T, --temp-dir <dir>` | Scratch directory for temp files before the atomic install; confined to the receive root (a relative path resolves below it; an absolute path is accepted only when it canonicalizes inside it), with an `EXDEV` non-atomic copy fallback |
|
||||
| `-n, --dry-run` | Report what would be transferred without mutating the destination. Since protocol 2.21.0 a server-routed target contacts the receiver and reports would-transfer based on receiver state; a plain local destination keeps the client-side scan. Never mutates or deletes. |
|
||||
| `-v, --verbose` | Enable debug logging |
|
||||
| `-q, --quiet` | Suppress non-error output |
|
||||
@@ -606,7 +606,7 @@ remote SSH argv is already built injection-safe.
|
||||
| `--max-alloc <SIZE>` | Maximum single allocation (binary units; default 1G; `0` = no local limit). |
|
||||
| `--max-depth <n>` | Limit recursive scanning depth; zero means unlimited. |
|
||||
| `-b, --backup` | Back up overwritten files. |
|
||||
| `-T, --temp-dir <dir>` | Scratch directory for temp files before the atomic install (confined to the receive root; `EXDEV` falls back to a non-atomic copy). |
|
||||
| `-T, --temp-dir <dir>` | Scratch directory for temp files before the atomic install (confined to the receive root: relative resolves below it, absolute must canonicalize inside it; `EXDEV` falls back to a non-atomic copy). |
|
||||
| `--backup-dir <dir>` | Store backups under a separate directory (requires `--backup`). |
|
||||
| `--suffix <suffix>` | Set the backup filename suffix (default: `~`). |
|
||||
| `--partial` | Select partial-transfer handling. On failed/interrupted writes the already-written temp file is retained (best-effort) for resumption. With `--partial --partial-dir <dir>`, completed files are written under the partial directory and installed atomically. |
|
||||
@@ -643,7 +643,7 @@ remote SSH argv is already built injection-safe.
|
||||
| `--groupmap=MAP` | Map group names when applying ownership (same syntax as `--usermap`). |
|
||||
| `--numeric-ids` | Mapping modifier: apply the source numeric uid/gid directly instead of mapping by name (combine with `-o`/`-g`, `-a`, or a map). |
|
||||
| `--copy-as=USER[:GROUP]` | Force every written entry to USER[:GROUP]; requires a privileged receiver. |
|
||||
| `--fake-super` | Record the resolved owner plus mode/time in a reserved `user.fastsync.stat` xattr and replay mode/time; never performs a real chown. |
|
||||
| `--fake-super` | Record the resolved owner plus full mode/rdev in rsync's reserved `user.rsync.%stat` xattr (rsync 3.4.1 grammar) and replay the permission bits; never performs a real chown. |
|
||||
| `--super` | Permit the receiver to attempt confined super-user activities (device nodes). |
|
||||
| `--no-super` | Forbid those super-user activities even when the receiver is root. |
|
||||
| `-l`, `--links` | Copy symlinks as symlinks; the target is stored verbatim (absolute and `..`-bearing targets included), matching rsync. |
|
||||
@@ -768,9 +768,17 @@ and `address`, the global section accepts:
|
||||
- `hosts allow` / `hosts deny` — comma- and/or whitespace-separated host access
|
||||
patterns.
|
||||
|
||||
A `[module]` requires `path`, and may also set `read only`, `client owner`,
|
||||
`auth users`, `max connections` (0 = unlimited; enforced per module across all
|
||||
connection children), and its own `hosts allow`/`hosts deny`.
|
||||
A `[module]` requires `path`, and may also set `read only`, `write only`,
|
||||
`client owner`, `auth users`, `max connections` (0 = unlimited; enforced per
|
||||
module across all connection children), and its own `hosts allow`/`hosts deny`.
|
||||
|
||||
Like rsync, a module is **read-only by default**: a bare `[module]` with only a
|
||||
`path` refuses a write transfer. Opt a module into writability explicitly with
|
||||
`read only = no` or `write only = yes`; a global `read only` value in the
|
||||
section before the first `[module]` sets the default for later modules, and a
|
||||
module's own `read only`/`write only = yes` always wins over it. An
|
||||
rsync-style `write only = yes` is mapped to writability because FastSync is
|
||||
push-only (a module can never be read from the network).
|
||||
|
||||
The per-host cap and the shared auth lockout identify a source by its numeric
|
||||
peer IP. **Loopback peers (127.0.0.0/8, IPv6 `::1`) are exempt**: every local
|
||||
|
||||
+66
-50
@@ -6,18 +6,18 @@ This document maps rsync's full feature set to FastSync's current implementation
|
||||
|
||||
| Status | Count | Description |
|
||||
|--------|-------|-------------|
|
||||
| ✅ Parity | 117 | Reproduces rsync's semantics for this option's scope |
|
||||
| ⚠️ Caveat | 13 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
|
||||
| ❌ Divergent | 27 | Rejected, an accepted no-op, deliberately non-rsync (native config/auth/batch, privileged namespaces, safe-subset privilege), or impossible on any portable filesystem call |
|
||||
| ✅ Parity | 119 | Reproduces rsync's semantics for this option's scope |
|
||||
| ⚠️ Caveat | 14 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
|
||||
| ❌ Divergent | 24 | Rejected, an accepted no-op, deliberately non-rsync (native config/auth/batch, privileged namespaces, safe-subset privilege), or impossible on any portable filesystem call |
|
||||
| **Total** | **157** | One row per rsync option/feature group; a row may name several spellings |
|
||||
|
||||
This matrix reports honest rsync parity, not "implemented" as a synonym for
|
||||
"parsed". A ✅ row matches rsync for the option's scope. A ⚠️ row is real and
|
||||
tested but diverges in at least one documented way. An ❌ row is either
|
||||
rejected (`--protocol` with any value but the current one, `--inc-recursive`),
|
||||
rejected (`--protocol` with any value but the current one),
|
||||
an accepted no-op (`-s`/`--secluded-args`, `--protect-args`, `--old-args`),
|
||||
deliberately non-rsync and non-interoperable (the FastSync daemon config/auth,
|
||||
the batch container, `--fake-super`'s xattr format, `--copy-as` credential
|
||||
the batch container, `--copy-as` credential
|
||||
switching), or impossible (`-N`/`--crtimes`). The counts are derived from the
|
||||
rows below; update them together with the table.
|
||||
|
||||
@@ -83,8 +83,8 @@ output, codec breadth, general `-R`/`-d`, the filter grammar (the unsupported
|
||||
rejected elsewhere — see the audit-cycle follow-up note above), receiver-side
|
||||
name resolution, absolute basis dirs, and the remaining client quick wins) and
|
||||
reclassified the inherently non-rsync rows as **divergent** (native daemon
|
||||
config/auth, the non-interoperable batch container, `--fake-super`'s xattr
|
||||
format, `-X`'s privileged namespaces, and the safe-subset device/privilege
|
||||
config/auth, the non-interoperable batch container,
|
||||
`-X`'s privileged namespaces, and the safe-subset device/privilege
|
||||
flags). It moved `PROTOCOL_VERSION` three times (`2.23.0 → 2.24.0` delete
|
||||
timing, `2.24.0 → 2.25.0` wire stats, `2.25.0 → 2.26.0` codecs). See the
|
||||
**Parity Completion Wave (protocol 2.26.0)** section near the end for the full
|
||||
@@ -161,7 +161,7 @@ Every one of those has an entry below with its remaining caveats.
|
||||
| `--no-implied-dirs` | Don't send implied dirs with -R | ✅ Parity | With `-R`, rsync creates the ancestor directories implied by a listed path and, with `--no-implied-dirs`, omits their attributes from the transfer so they keep the destination's own state (or are created with default attributes when absent). Protocol 2.26.0 matches this: without `--files-from` the implied-dir walk applies per-attribute metadata only to explicitly transferred directories, and with `-R --files-from` a listed file whose parent is not itself listed is placed normally — the missing implied parent is created with default attributes (not the source's) and the file transfers with `rc 0`, exactly like rsync 3.4.1 (a differential test verifies the modes and mtimes with and without the flag). Works single-threaded and under `-j`/`--threads` |
|
||||
| `-d`, `--dirs`, `--old-dirs`, `--old-d` | Transfer dirs without recursing | ✅ Parity | Protocol 2.26.0 implements rsync's one-level `-d` listing for `dir`, `dir/` and `.`: the source's immediate contents are transferred (files with content, directories as explicit entries), matching rsync's destination tree in a differential test. `--dirs --files-from` transfers exactly the listed items — a listed directory is created empty and a listed file with content — under the same `-R` layout rules. A plain recursive scan also recreates empty source directories now: the scanner emits a payload-less directory entry (with metadata) for every traversed directory that produced no transferred or descended child, unless `-m/--prune-empty-dirs` suppresses it or the run is `--files-from`/`--list-only` (a directory emptied by filtering is recreated too, matching rsync). Directory entries cross as `STATUS_MKDIR` and appear in the delete manifest, so `--delete` prunes correctly and an empty listed directory survives; an incoming directory replaces a destination regular file (rsync removes the non-directory and creates the directory), verified differentially. Directory times are applied at the end of the transfer; modes/ownership follow the per-attribute policy. Under `--delay-updates` directories are created immediately while only regular files are staged, exactly as rsync does |
|
||||
| `--mkpath` | Create missing path components | ✅ Parity | Wire option (client → server). At connection start the server creates the client's destination root directory (and any missing leading components below its own authorized root) when `--mkpath` is set, failing the connection cleanly if it cannot. Without `--mkpath` a destination root that does not exist yet is rejected up front (rsync semantics), so the flag is the only way to transfer into a not-yet-created destination directory. Creation is confined by the same secure mkdir walk as file writes (`O_NOFOLLOW`, no `..`) |
|
||||
| `--inc-recursive`, `--no-inc-recursive` | Incremental recursion mode | ❌ Divergent | rsync's man-page-only scanning-mode switch (and its short aliases). FastSync always performs a single full recursive scan, so both spellings are rejected as unknown options rather than accepted as a no-op; there is no incremental-recursion engine to toggle. A genuine implementation would be a scan-architecture change with no benefit for FastSync's push model |
|
||||
| `--inc-recursive`, `--no-inc-recursive` | Incremental recursion mode | ✅ Parity | rsync's man-page-only scanning-mode switch. FastSync always performs a single full recursive scan, so both spellings are accepted as inert no-ops and the destination is identical whichever mode the caller requests — the same treatment as `-r`/`--recursive`, which is likewise a no-op. The switch is a scan-implementation detail with no observable effect on the final tree (rsync's own `--no-inc-recursive` selects a full scan, which is exactly FastSync's behavior) |
|
||||
|
||||
## 5. Transfer Modifications
|
||||
|
||||
@@ -184,7 +184,7 @@ Every one of those has an entry below with its remaining caveats.
|
||||
| `--backup-dir=DIR` | Backup directory hierarchy | ✅ Parity | `backup_dir` config field |
|
||||
| `--suffix=SUFFIX` | Backup suffix (default ~) | ✅ Parity | `suffix` config field |
|
||||
| `--delay-updates` | Put updated files in place at end | ❌ Divergent | Successfully received files are staged under a private 0700 `.fastsync-stage` dir inside the receive root and atomically renamed into their final destinations only after the whole transfer (manifest/delete handling included) succeeds, just before the success/outcome frame is sent. The delete walker deliberately skips the staging dir at the receive root, so `--delete` removes genuine extras but never the staged files (deletion runs before publication; rsync's delete-after ordering is not implemented). `--existing`/`--ignore-existing`/`--update` decide against the final destination path at stage time; `--backup` moves the old file aside at publication, and **`--force` is honored at publication** (protocol 2.23.0): a staged regular file or symlink may replace a destination directory that blocks it. Incompatible with `--inplace` and with `--backup-dir=.fastsync-stage` (the internal staging name is reserved; both are rejected). The staging dir name is fixed, so two simultaneous delayed transfers to the same destination root are serialized with an exclusive advisory lock held for the whole transfer: the second session fails cleanly instead of corrupting the first. Aborting or failing before publication installs nothing and removes the staging tree; a crash between stage and publish leaves staged leftovers that the next delayed run wipes at start (process death releases the lock). A stage→publish failure aborts the transfer (best-effort cleanup of the not-yet-published staged files; already-published files are not rolled back). **Reclassified Divergent (differential evidence):** the staging name is fixed and a delayed run wipes a pre-existing destination tree of that name at start even without `--delete`, whereas rsync uses its own internal temp name and leaves a genuine destination entry named `.fastsync-stage` untouched (`test_delay_updates_staging_name_collision_residual`); deletion also runs before publication while rsync's `--delay-updates` implies `--delete-after`. Works in single-threaded and `-j`/`--threads` modes |
|
||||
| `-T`, `--temp-dir=DIR` | Create temporary files in DIR | ❌ Divergent | `--temp-dir` with the rsync short `-T` (the timeout alias moved to long-only `--timeout`). A **relative** dir matches rsync exactly: it is resolved below the receive/destination root and must already exist (differentially verified: `rsync -a --temp-dir=scratch src/ dst/` and FastSync produce identical trees and an empty scratch dir). **Reclassified as a deliberate divergence because an absolute `--temp-dir` is rejected by the receiver** — it is resolved verbatim by rsync standalone (which will use `/tmp` or any other absolute directory, including one outside the destination), but FastSync's security-reviewed receiver confines the scratch dir to the authorized receive root and rejects any absolute path or one containing `..`. **Audit-cycle hardening:** the opened dir is additionally judged by the real path of its fd (`/proc/self/fd`), so a client-planted symlink under the receive root cannot redirect receiver scratch files outside the authorized root (an escaping target is refused with `EACCES`), while an in-root symlink to another filesystem — the `EXDEV` fallback case — still works. A differential test confirms rsync exits 0 using an absolute scratch dir while FastSync refuses before writing anything into it (the scratch dir stays empty). Its daemon mode also confines relative to the module, but standalone rsync's absolute-temp-dir behavior is not reproduced because it would let a client place receiver scratch files outside the sandbox. Temp copies use a unique name in the scratch dir and are atomically renamed into place; **on `EXDEV` (scratch dir and destination on different filesystems, reachable via a confined relative symlink) the receiver falls back to a non-atomic copy instead of aborting**, matching rsync. `--inplace` and `--partial-dir` writes bypass the scratch dir |
|
||||
| `-T`, `--temp-dir=DIR` | Create temporary files in DIR | ❌ Divergent | `--temp-dir` with the rsync short `-T` (the timeout alias moved to long-only `--timeout`). A **relative** dir matches rsync exactly: it is resolved below the receive/destination root and must already exist (differentially verified: `rsync -a --temp-dir=scratch src/ dst/` and FastSync produce identical trees and an empty scratch dir). An **absolute** dir is accepted when it canonicalizes (`realpath(3)`) inside the receive root, so an in-root absolute scratch path is usable and used (unit- and integration-tested for both the local batch apply and a live TCP transfer). **Remaining divergence:** an absolute `--temp-dir` that escapes the receive root is rejected, and so is a relative one containing `..` — rsync standalone resolves an absolute `--temp-dir` verbatim (it will use `/tmp` or any other directory, including one outside the destination), but FastSync's security-reviewed receiver confines the scratch dir to the authorized receive root and refuses an out-of-root path before writing anything. **Audit-cycle hardening:** the opened dir is additionally judged by the real path of its fd (`/proc/self/fd`), so a client-planted symlink under the receive root cannot redirect receiver scratch files outside the authorized root (an escaping target is refused with `EACCES`), while an in-root symlink to another filesystem — the `EXDEV` fallback case — still works. A differential test confirms rsync exits 0 using an out-of-root absolute scratch dir while FastSync refuses before writing anything into it (the scratch dir stays empty). Its daemon mode also confines relative to the module. Temp copies use a unique name in the scratch dir and are atomically renamed into place; **on `EXDEV` (scratch dir and destination on different filesystems, reachable via a confined relative symlink) the receiver falls back to a non-atomic copy instead of aborting**, matching rsync. `--inplace` and `--partial-dir` writes bypass the scratch dir |
|
||||
| `--partial` | Keep partially transferred files | ✅ Parity | On a failed/interrupted write the already-written temp file is retained at the destination path (best-effort rename instead of unlink) so a later `--append`/`--append-verify` run can resume it. Retention never runs when no data was actually written or under `--ignore-existing`/`--existing` (the destination is not ours to overwrite), and it only ever renames the already-written temp. A failed rename falls back to the normal unlink |
|
||||
| `--partial-dir=DIR` | Keep partial files in DIR | ✅ Parity | The working file is written under the confined partial directory (a relative dir below the receive root) and atomically renamed into place once complete, so an interrupted transfer leaves a resumable copy there and completed transfers do not linger under it. `--inplace` bypasses the partial dir (rsync parity), and combining `--inplace` with `--partial-dir` is now **rejected up front** with rsync's message (`--inplace cannot be used with --partial-dir`) instead of silently ignoring the partial dir. **Implies `--partial`** (audit-cycle fix, matching rsync 3.4.1, which sets `keep_partial` after option parsing): `--partial-dir=DIR` alone retains an interrupted transfer's partial, and the implication wins over an explicit `--no-partial` regardless of order |
|
||||
|
||||
@@ -193,7 +193,7 @@ Every one of those has an entry below with its remaining caveats.
|
||||
| Flag | Rsync Description | FastSync Status | Notes |
|
||||
|------|-------------------|-----------------|-------|
|
||||
| `--delete` | Delete extraneous files from dest | ✅ Parity | `use_delete` config field. Deletion is always derived from the keep-set the sender actually transmitted (the per-directory `STATUS_DELETE_PLAN` set by default, or the whole-tree manifest for the late timings — never from unchecked input), runs through the symlink-safe walker bounded by `MAX_SERVER_DELETE_COUNT`, and skips the `.fastsync-stage` staging dir under `--delay-updates`. **Lockstep track 6 (protocol 2.28.0): plain `--delete` with no explicit timing flag now defaults to `--delete-during`**, exactly like rsync's `--del` (the client normalizes it to the existing `delete_during` wire bool; no new wire field). This frees destination space progressively during the transfer and avoids the whole-old+new-tree peak that could `ENOSPC` a tight destination. The old late whole-tree commit is opt-in via `--delete-after` or the FastSync-only long spelling `--delete-commit`. **Abort/ordering parity (parity-2.29):** the complete per-directory plan set is transmitted before the first data frame, so a mid-transfer abort has already applied every planned removal exactly like rsync's generator (which runs ahead of its throttled sender); `-d/--dirs` uses the same per-directory plans (the generator records only the directories whose direct children it enumerated, so an untraversed subdirectory's mirror is shielded); and the sorted depth-first traversal makes the removal order — and therefore the survivor set under a partial `--max-delete` — match rsync exactly (`test_delete_boundary_parity.py`, `test_parity_order.py`). By default the destination mirror of a path the source scan pruned (filter/exclude/size rules) is **protected** from deletion — matching rsync, which does not delete excluded files under `--delete`; `--delete-excluded` opts back into deleting them (see below). Deletion is scoped to the **synchronized directories** sent on the wire (protocol 2.23.0), so a `--files-from` subset no longer deletes untransmitted paths outside the listed directory subtrees. The walk is bounded: a client `--max-delete=NUM` (or the 100000-entry server bound) makes it **partial** — entries up to the bound are removed, the rest are skipped, and the client exits **25** (`RERR_PARTIAL`), matching rsync, rather than failing the transfer. Extraneous destination symlinks are unlinked by name (never followed); a directory still holding a kept/protected entry is left behind rather than failing |
|
||||
| `--delete-before` | Delete before transfer | ⚠️ Caveat | Implies `--delete`. The sender runs a full source pre-scan (paths only) and transmits the keep-set manifest BEFORE any file data; the receiver validates it, removes every destination entry not listed (bounded walk, staging-dir skip, protected prefixes honored), then acks `STATUS_OK`. The sender only starts streaming after the deletion committed, or aborts if the receiver reported a deletion error. By definition the deletions already happened when a later transfer phase fails — rsync's delete-before is destructive the same way; a subsequent failure does not restore the removed files. **Phase-0 divergence (sharpened):** rsync builds the full file list first, so a source file created after that scan is NOT transferred and its destination extra is deleted; FastSync's single-threaded data pass re-scans the source, so the late file IS transferred (a safe superset), while FastSync `--threads` pipelines the scan and matches rsync |
|
||||
| `--delete-before` | Delete before transfer | ✅ Parity | Implies `--delete`. The sender runs a full source pre-scan (paths only) and transmits the keep-set manifest BEFORE any file data; the receiver validates it, removes every destination entry not listed (bounded walk, staging-dir skip, protected prefixes honored), then acks `STATUS_OK`. The sender only starts streaming after the deletion committed, or aborts if the receiver reported a deletion error. By definition the deletions already happened when a later transfer phase fails — rsync's delete-before is destructive the same way; a subsequent failure does not restore the removed files. **Phase-0 divergence closed (no-wire):** both data passes now replay the exact file list the pre-scan built for the keep-set instead of re-reading the source — the single-threaded send loop and the `--threads` pipeline (whose scanner thread feeds the retained pre-scan chunks into the pipeline rather than re-scanning) — so a source file created after that scan is NOT transferred and its destination extra is deleted, exactly like rsync's single file list. The pre-scan captures the deferred directory times and the `--stats` directory count because no later scan runs (`test_delete_timing_parity.py::TestDeleteBeforeLateFileParity`, parametrized single-threaded vs `--threads=4`, differential vs rsync 3.4.1) |
|
||||
| `--del`, `--delete-during` | Delete during transfer | ✅ Parity | Both spellings accepted; imply `--delete`, and since lockstep track 6 this is also the default timing of a plain `--delete`. **Protocol 2.24.0 implements per-directory delete plans:** as the sender reaches each source directory it streams a `STATUS_DELETE_PLAN` for that directory and the receiver removes that directory's extras (verified with a byte-slicing proxy). The one-shot per-run config block (protected prefixes, size-pruned mirrors, `--delete-missing-args` exact paths) rides a dedicated config-only carrier frame with an `apply=false` flag, so it reaches the receiver even when the scope allows no directory plan at all (a `--files-from` list of bare files synchronizes no directory). **Abort/ordering parity (parity-2.29):** the complete plan set is transmitted before the first data frame, so on a mid-transfer abort every planned extra has already been removed exactly like rsync's generator (which runs ahead of its throttled sender); `-d/--dirs` no longer falls back to the end-of-transfer commit but records only the directories whose direct children it enumerated; and the sorted depth-first traversal makes the removal order — and the partial-`--max-delete` survivor set — identical to rsync (`test_delete_boundary_parity.py`, `test_parity_order.py`). `-R` plans are scoped to the transferred prefix subtree |
|
||||
| `--delete-delay` | Find deletions during, delete after | ✅ Parity | Implies `--delete`. **Protocol 2.24.0 implements rsync's delete-delay timing:** the sender records each directory's delete plan while scanning and the receiver commits those removals only after the whole transfer succeeds (per plan), so an extra created in the destination after its directory's plan survives while `--delete-after` re-scans and removes it, and a failed transfer removes nothing. The **reported** deleted count advances only on an actual removal. **Fixed (no-wire):** the `--max-delete` budget is now charged on ACTUAL removals (an unlink/rmdir that succeeded), not at plan/snapshot time, and a queued directory is re-scanned at commit and removed recursively (content created after the plan included), matching rsync: a snapshotted entry that fails or is skipped consumes no budget, so a later extra rsync would delete is still deleted. The deferred snapshot list keeps an independent hard cap (`DELETE_PLAN_SERVER_LIMIT`) so it cannot grow without bound now that the budget is no longer charged while scanning. A `--max-delete=2` partial delete reports exactly 2 and exits 25 in both tools, and the refilled-directory differential (late content removed, directory removed, budget shared) now matches rsync 3.4.1 on both sides (`test_delete_delay_budget_parity.py`, `test_delete_timing_parity.py`). Unit tests cover recursive removal, actual-removal charging, and the bounded deferred list. **Ordering parity (parity-2.29):** the sorted depth-first traversal plus the up-front plan set make the order in which extras are removed — and therefore the survivor set under a partial `--max-delete` — match rsync exactly (differential `test_parity_order.py::test_delete_delay_deletion_order_matches_rsync` and `::test_partial_max_delete_survivor_order_matches_rsync`) |
|
||||
| `--delete-after` | Delete after transfer | ✅ Parity | Implies `--delete`. Selects the late whole-tree commit: the keep-set manifest closes the data stream and the receiver commits the bounded deletion only after the terminal `STATUS_FINISHED` proves the whole transfer (every data frame received and stored) succeeded. A failed or aborted transfer removes nothing. Since lockstep track 6 a plain `--delete` defaults to delete-during (rsync's `--del`); `--delete-after` — or the FastSync-only `--delete-commit` spelling, which selects the identical timing — is the explicit way to keep the old commit-style behavior |
|
||||
@@ -342,7 +342,7 @@ why plain `--append` works on the normal atomic path, not only with `--inplace`.
|
||||
| `-X`, `--xattrs` | Preserve extended attributes | ❌ Divergent | Deliberately restricted to unprivileged `user.*` extended attributes plus the two POSIX ACL xattrs; `security.*` (SELinux, capabilities, ...) and `trusted.*` are **never** captured or applied — a client can never force a privileged attribute onto the destination, and the receiver independently re-validates every incoming name against the whitelist. This is a security-policy divergence from rsync, which can preserve the privileged namespaces with the needed privilege; implementing them would defeat FastSync's privilege-escalation guard. `user.*` capture/apply matches rsync in a differential test. Payloads are bounded on both ends. Incompatible with `-s`. **Also divergent: symlink xattrs/ACLs are not captured or applied** — `-X`/`-A` with `-l` carries only the link's owner/times/mode, not its xattrs (the capture uses path-following `listxattr`/`getxattr`, so the link's own xattrs are never read, and the receiver's symlink write path applies no xattr block). Closing this needs a dedicated symlink-xattr wire block and a `PROTOCOL_VERSION` bump |
|
||||
| `-H`, `--hard-links` | Preserve hard links | ✅ Parity | Files on the source that share an inode (`st_dev`+`st_ino`, e.g. a `cp -al` tree) are re-created as hard links to one another on the destination, so duplicate links stay deduplicated and only the first member's data is sent (later members are transmitted as payload-less `STATUS_HARDLINK` frames). The receiver links each sibling to the first member's installed file with an atomic link + rename; on `link()` failure it falls back to a byte-identical local copy of the first member, never a partial/corrupt file. Requires the sequential scan for ordering (the first member is always emitted and installed before any sibling is linked). Works single-threaded and under `-j`/`--threads`, `--inplace`, `--delay-updates` (links staged and published by rename) and `--partial`. Crosses the wire (`preserve_hard_links` bool; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0**, peers must match). Incompatible with `-s` (chunk serialization) and `--append`/`--append-verify`, rejected up front with a distinct error. See the Phase-4 hard-links notes below |
|
||||
| `-D` | Same as --devices --specials | ✅ Parity | Implies `--devices --specials`. `-D` was unassigned in FastSync (verified: no collision), so it is free to imply both device-node and special-file preservation. As of protocol 2.23.0 `--specials` genuinely covers **both FIFOs and unix sockets**, so `-D` covers the full rsync set. See the `--devices`/`--specials` rows and the Phase-4 devices notes below |
|
||||
| `--devices` | Preserve device files | ❌ Divergent | Recreates char/block device nodes with `mknodat` (type + rdev strictly validated, confined fd-relative below the receive root), but only when the receiver has `CAP_MKNOD`: a non-root receiver logs a warning and skips the entry instead of erroring, so a transfer with devices never aborts. Deliberate privilege-model divergence from rsync, which errors when it cannot create the node. `--specials` (FIFOs and unix sockets) is unprivileged and remains parity |
|
||||
| `--devices` | Preserve device files | ⚠️ Caveat | Recreates char/block device nodes with `mknodat` (type + rdev strictly validated, confined fd-relative below the receive root). A device whose `mknodat` fails with `EPERM`/`EACCES` (no `CAP_MKNOD`, or super-user activity forbidden) is a **per-entry failure**: FastSync logs `cannot create device ...` (rsync logs `mknod ... failed`), counts it, **continues with the remaining files**, and ends the run with a non-OK terminal status. rsync parity: rsync likewise continues and exits partial (23). Residuals: (1) FastSync's default AUTO still *attempts* the node on a non-root receiver and therefore reports the per-entry failure, whereas rsync without `--super` silently ignores `--devices` and skips the non-regular entry with exit 0 — use `--no-super` for rsync's silent-skip behavior; (2) FastSync's process exit code for a receiver-side per-entry failure is the general error code 1, not rsync's partial 23 (a client exit-code-mapping residual that applies to every receiver file error, not just this branch); (3) with `--remove-source-files`, the non-OK terminal status means successfully transferred sources are not removed on a partial run. `--specials` (FIFOs and unix sockets) keeps the unprivileged skip path and remains parity |
|
||||
| `--specials` | Preserve special files | ✅ Parity | **FIFO and unix-socket recreation work** (protocol 2.23.0): FIFOs are recreated with `mkfifoat`, and sockets with `mknodat(..., S_IFSOCK)` — the latter is unprivileged on Linux because it materializes the socket *node*, not a live bound socket, so it is a real, assertable behavior under CI (it matches rsync, which also recreates a socket by `mknod`). Node creation is confined below the receive root (fd-relative parent; no `..`, no symlink follow) and type/rdev are validated strictly; a matching existing node is left in place and an unrelated entry is never replaced. Crosses the wire like `--devices` (the `STATUS_SPECIAL` frame). See the Phase-4 devices notes |
|
||||
| `--copy-devices` | Copy device contents as file | ❌ Divergent | Copies a device/FIFO's reported `st_size` into an ordinary regular file and never reads an unbounded pseudo-device, so `--sendfile` cannot hang and the run always succeeds. Deliberate safe divergence from rsync's dd-like unbounded device read, which can block; the dangerous behavior will not be implemented |
|
||||
| `--write-devices` | Write to devices as files | ❌ Divergent | Writes only into an existing char/block node under the confined receive root (`O_NOFOLLOW` + `O_NONBLOCK`); a missing, symlinked, FIFO-with-no-reader, non-device, or otherwise unusable destination is skipped with a warning rather than allowed or aborted. Deliberate confinement divergence from rsync's more permissive behavior |
|
||||
@@ -351,7 +351,7 @@ why plain `--append` works on the normal atomic path, not only with `--inplace`.
|
||||
| `-O`, `--omit-dir-times` | Omit dirs from --times | ✅ Parity | Real modifier now that FastSync preserves directory times. With metadata on, the scanner captures every traversed source directory's mtime (and atime under `-U`) and the sender transmits them in trailing `STATUS_DIR_TIMES` frame(s) **after all file data and the optional delete manifest** (chunked at the receiver's `MAX_MANIFEST_ENTRIES` per-frame cap); a dir-time entry only RECORDS metadata and never creates the directory (an empty source directory is created by the separate `STATUS_MKDIR` entry the scanner now emits, and `-m/--prune-empty-dirs` suppresses that; the trailing dir-time simply re-applies the metadata). The receiver defers applying them until its delete / `--delay-updates` publication phases have committed, so writing or removing a child never clobbers a parent directory's mtime (rsync applies directory times at the end for exactly this reason). When `-O` is set (the boolean crosses the wire) the receiver does not apply any of them; without `-O` an `-a`/`--preserve` transfer now restores directory times (reversing the old "never preserves dir times" divergence). Wire change: the terminal `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** |
|
||||
| `-J`, `--omit-link-times` | Omit symlinks from --times | ✅ Parity | Real modifier now that FastSync preserves symlink times. Symlink entries already carried their metadata on `STATUS_SYMLINK`; the receiver now applies it with **no-follow primitives only** (`utimensat(..., AT_SYMLINK_NOFOLLOW)`, plus best-effort `fchmodat(..., AT_SYMLINK_NOFOLLOW)` and policy-gated `fchownat(..., AT_SYMLINK_NOFOLLOW)`), so the link itself is stamped without ever dereferencing it, confined fd-relative below the authorized receive root. A symlink has no children, so the times are applied immediately at creation. When `-J` is set (the boolean crosses the wire) the receiver skips the timestamps (mode/ownership are unaffected); without `-J` an `-a`/`-l` transfer restores symlink mtimes. Wire change alongside `-O`: the shared `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** |
|
||||
| `--super` | Receiver attempts super-user activities | ❌ Divergent | Safe-subset privilege model. `--super` permits the receiver to attempt already-confined super-user activities (ownership application, char/block device-node creation, `--write-devices`); `--no-super` forbids them even for root; `auto` keeps the historical best-effort attempt. **FastSync never elevates** — no `setuid`/`seteuid`/`setgid` — and `--super` never bypasses the confinement floor, so it diverges from rsync's real elevation. A server `--no-super` veto forces it off for every connection; a privileged standalone listener defaults off without `--allow-super`; daemon modules opt in with `client owner = yes` |
|
||||
| `--fake-super` | Store/recover privileged attrs via xattrs | ❌ Divergent | Records the resolved `uid:gid:mode:mtime_sec:mtime_nsec` in a reserved `user.fastsync.stat` xattr and immediately replays mode/times fd-relative, but **never performs a real `chown`** (the owner is recorded for a later privileged restore). The on-disk key and format are FastSync-native, not rsync's `user.rsync.%stat%`, so recordings are not interoperable with rsync — the same class as the native auth and batch formats. Implies metadata transmission; incompatible with `-s` |
|
||||
| `--fake-super` | Store/recover privileged attrs via xattrs | ⚠️ Caveat | Writes rsync 3.4.1's reserved `user.rsync.%stat` xattr with rsync's exact value grammar `<octal st_mode with S_IFMT> <rdev_major>,<rdev_minor> <uid>:<gid>` (e.g. `104711 0,0 1234:5678`), recording the RESOLVED owner (the `--chown`/`--usermap`/`--groupmap`/`--copy-as` mapping when active, else the source's own id) plus the full mode and rdev; it **never performs a real `chown`**. mtime is carried by the file's own timestamp, exactly as rsync does it (there is no mtime field). The receiver parses the same grammar and replays the permission bits fd-relative, stripping the recorded special bits on disk exactly like rsync's fake-super receiver. Regular files are interoperable with real rsync 3.4.1 in both directions (the differential test has rsync read a FastSync fake-super tree and re-emit the identical record). Char/block devices **are** faked: a device is written as a regular empty file and its `user.rsync.%stat` records the real `rdev` (e.g. `20644 1,3 0:0`), never `mknod`'d, on both privileged and unprivileged receivers, exactly as rsync does. The record parser range-checks every field (mode/rdev/uid/gid) and rejects malformed records cleanly. Residual: directories are not yet faked — no `%stat` record is written for a directory. Implies metadata transmission; incompatible with `-s` |
|
||||
| `--open-noatime` | Avoid changing access time when opening files | ✅ Parity | Sender-side policy: the sender opens source files with `O_NOATIME` (Linux) when reading them for transfer, so the open/read does NOT bump the source's on-disk access time. Degrades safely when `O_NOATIME` is unavailable (not defined) or refused (`EPERM`, since it needs `CAP_FOWNER` or file ownership): the code falls back to a normal open, so the data always transfers — only the atime-bump is skipped. It does not itself capture/preserve atime; it only avoids modifying it. **Client-only, never crosses the wire.** Exposed as `file_open_for_read()` and applied to both the buffered data path and the sendfile path |
|
||||
| `--numeric-ids` | Do not map uid/gid by name | ✅ Parity | **A mapping modifier only:** when ownership is being applied it uses the transmitted numeric uid/gid directly, skipping the name lookup. It does **not** request ownership application on its own — combine it with `-o`/`-g`, `-a`, or an explicit map (`--chown`/`--usermap`/`--groupmap`) — and it does not need any metadata flag merely to parse. Ownership is only applied when metadata (hence the source uid/gid) is actually transmitted (see the Phase-4 identity notes) |
|
||||
| `--usermap=STRING` | Map usernames | ✅ Parity | Opt-in ownership application. Comma-separated `FROM:TO` rules evaluated in order, first match wins. `FROM` accepts a source-resolved user name, a name **glob** (`*`/`?`/`[...]`, expanded sender-side at CLI-parse time against the sender's passwd/group database and collapsed into numeric `LOW-HIGH` ranges, bounded by `MAX_IDENTITY_MAP`), an `@N`/bare `N` numeric id, an inclusive `LOW-HIGH` id range, `*`, or an empty field (ids with no source name). `TO` accepts a receiver-resolved **name** (protocol 2.26.0 resolves it on the receiving side against the receiver's account database, matching rsync), an `@N`/bare `N` id, or `*` (the receiving process's euid). Rules travel as resolved numeric pairs plus an optional TO name; the receiver applies a matching rule, else falls back to `--chown`, `--numeric-ids`, then a best-effort name lookup, via fd-relative `fchown`. Malformed specs are clear errors. Implies metadata; only effective where the receiver can chown (otherwise a warning) |
|
||||
@@ -408,7 +408,7 @@ match, exactly as prior phases did).
|
||||
is refused) also re-applies the incoming (or, for `-H`, the first member's)
|
||||
xattrs and the `--fake-super` stat, so attributes are preserved rather than
|
||||
silently dropped when the link fails.
|
||||
- **Reserved fake-super key is receiver-only:** the `user.fastsync.stat` key is
|
||||
- **Reserved fake-super key is receiver-only:** the `user.rsync.%stat` key is
|
||||
excluded from sender capture AND from receiver application, so it can only be
|
||||
written by the receiver's own `--fake-super` handling. A source file that
|
||||
already carries such a record is never forwarded on a plain `-X` run, so it
|
||||
@@ -417,15 +417,19 @@ match, exactly as prior phases did).
|
||||
Applying an ACL is owner-privileged: `fsetxattr` failure (e.g. non-root,
|
||||
unsupported filesystem) is logged (collapsed to one line per file) and never
|
||||
fatal.
|
||||
- **`--fake-super`**: see the row above; the reserved key is `user.fastsync.stat`
|
||||
with the documented `uid:gid:mode:mtime_sec:mtime_nsec` (mode octal) format.
|
||||
**Replay exists**: after each stored record the receiver immediately re-applies
|
||||
the recorded mode and times fd-relative (`fake_super_restore_fd`), but it
|
||||
deliberately never performs a real `chown` — `--fake-super` only *records*
|
||||
the resolved owner (the active `--chown`/`--usermap`/`--groupmap`/`--copy-as`
|
||||
mapping when one is in effect, otherwise the source's own id) for a later
|
||||
privileged restore. The recording format diverges from rsync's
|
||||
`user.rsync.%stat%`; no cross-tool conversion is attempted.
|
||||
- **`--fake-super`**: see the row above; the reserved key is rsync's own
|
||||
`user.rsync.%stat` with rsync 3.4.1's exact `<octal st_mode> <rdev_major>,
|
||||
<rdev_minor> <uid>:<gid>` value (mtime is not stored — the file's own
|
||||
timestamp carries it, exactly as rsync does). **Replay exists**: after each
|
||||
stored record the receiver immediately re-applies the recorded permission bits
|
||||
fd-relative (`fake_super_restore_fd`, with the recorded special bits stripped
|
||||
on disk exactly like rsync), but it deliberately never performs a real
|
||||
`chown` — `--fake-super` only *records* the resolved owner (the active
|
||||
`--chown`/`--usermap`/`--groupmap`/`--copy-as` mapping when one is in effect,
|
||||
otherwise the source's own id) for a later privileged restore. Because the
|
||||
key and grammar are rsync's, a regular-file fake-super tree is interoperable
|
||||
with rsync 3.4.1 in both directions; directories and device nodes are not yet
|
||||
faked.
|
||||
- **Chunk serialization (`-s`) incompatibility:** the per-file xattr block rides
|
||||
the streaming per-file frame, which `-s` replaces with a fixed buffer format,
|
||||
so `-X` / `-A` combined with `-s` is rejected up front on both ends (mirroring
|
||||
@@ -554,18 +558,22 @@ marker + rdev so `--devices/--specials` also work under `-s`. `PROTOCOL_VERSION`
|
||||
was bumped **2.12.0 → 2.13.0** (peers must match, exactly as prior phases did).
|
||||
|
||||
**Privilege gating (the crux):** making a device node requires `CAP_MKNOD` (root).
|
||||
CI runs the integration suite as a NON-ROOT user (via setpriv), so `mknod` fails
|
||||
with `EPERM`. The receiver treats this as a graceful, logged *skip of the entry*
|
||||
returned as a success/skip outcome — the whole transfer NEVER aborts just because
|
||||
the environment cannot create the node. `mkfifo` (FIFOs) is unprivileged, so
|
||||
`--specials` FIFO creation is a real, assertable behavior under CI. **Sockets are
|
||||
recreated too** (protocol 2.23.0) with `mknodat(..., S_IFSOCK)`: Linux allows an
|
||||
unprivileged `mknod` of a socket node because no live bound socket is created,
|
||||
so a source socket materializes as a socket-type filesystem entry exactly as
|
||||
rsync does. The "device actually created" integration assertions are guarded to
|
||||
run only as root. User-facing expectation: point `--devices` at devices and a
|
||||
non-root receiver will faithfully skip them while transferring everything else;
|
||||
`--specials` recreates FIFOs and socket nodes for any receiver.
|
||||
When the receiver attempts a device `mknod` and the kernel refuses with
|
||||
`EPERM`/`EACCES`, FastSync now reports a genuine transfer error (the receiver's
|
||||
outcome aggregation fails the entry), matching rsync, which logs
|
||||
`mknod ... failed` and exits partial (23) whenever it attempts the node (as root
|
||||
or with `--super`); with `--no-super` the device entry is pre-skipped instead.
|
||||
Only `mkfifo` (FIFOs) is unprivileged, so `--specials` FIFO creation is a real,
|
||||
assertable behavior under CI. **Sockets are recreated too** (protocol 2.23.0)
|
||||
with `mknodat(..., S_IFSOCK)`: Linux allows an unprivileged `mknod` of a socket
|
||||
node because no live bound socket is created, so a source socket materializes as
|
||||
a socket-type filesystem entry exactly as rsync does. The "device actually
|
||||
created" integration assertions are guarded to run only as root; a root runner
|
||||
additionally drops the receiver to an unprivileged user (setpriv) to assert the
|
||||
`CAP_MKNOD` failure surfaces as a failed transfer rather than a silent skip.
|
||||
User-facing expectation: point `--devices` at devices and a receiver without
|
||||
`CAP_MKNOD` reports the failure, while `--specials` recreates FIFOs and socket
|
||||
nodes for any receiver.
|
||||
|
||||
**Confinement & validation:** a special/device node is created with
|
||||
`mknodat`/`mkfifoat` on the parent directory opened fd-relative below the receive
|
||||
@@ -730,8 +738,8 @@ targets verbatim, matching rsync.
|
||||
| Flag | Rsync Description | FastSync Status | Notes |
|
||||
|------|-------------------|-----------------|-------|
|
||||
| `--daemon` | Run as rsync daemon | ❌ Divergent | Wave A: a real persistent listener. `fastsync-server --daemon --config FILE` (plus `--no-detach` to stay foreground; without it the listener detaches to the background after binding) reads a FastSync-native module config file and serves each connection confined to the requested module's `path` root (never a client-chosen root; every client-chosen-ownership/super-user request (`--numeric-ids`/`--chown`/`--usermap`/`--groupmap`/`--fake-super`/`--copy-as`/explicit `--super`) is refused unless the module opts in with `client owner = yes`, and the operator `--no-super` veto is honored). TCP/TLS via the existing `--tls` stack; plaintext still requires `--allow-unauthenticated` (same secure default as the standalone server). Client destinations use rsync's `host::module/path` form. Wire/protocol: the config frame gained a trailing daemon-module string and `PROTOCOL_VERSION` was bumped **2.14.0 → 2.15.0** (see the Daemon Mode notes below). Daemon mode is built in FastSync's own protocol/config grammar, not rsync's SMB/daemon option encoding |
|
||||
| `--config=FILE` | Alternate rsyncd.conf file | ❌ Divergent | Wave A: selects the daemon config file. Default when omitted (in `--daemon` mode): `~/.config/fastsync/fastsyncd.conf` if it exists, else `/etc/fastsyncd.conf`. The grammar is FastSync-native (documented in the Daemon Mode notes below) and strictly rejects unknown keys so a typo can never silently change what a module serves; requires `--daemon` |
|
||||
| `--dparam=OVERRIDE` | Override global daemon config | ❌ Divergent | Wave A: overrides one global scalar from the command line (`--dparam port=8734` and `--dparam=KEY=VALUE` both work). Limited to the global keys the grammar defines (`port`, `motd file`, `address`, `max connections`, `max connections per host`, `auth failure delay`, `auth lockout threshold`, `auth lockout duration`, `hosts allow`, `hosts deny`); keys are case-insensitive and unknown keys/invalid values are rejected. Requires `--daemon` |
|
||||
| `--config=FILE` | Alternate rsyncd.conf file | ❌ Divergent | Wave A: selects the daemon config file. Default when omitted (in `--daemon` mode): `~/.config/fastsync/fastsyncd.conf` if it exists, else `/etc/fastsyncd.conf`. The grammar is FastSync-native (documented in the Daemon Mode notes below) and still strictly rejects a genuinely unknown key so a typo can never silently change what a module serves; requires `--daemon`. **rsync 3.4.1 key subset accepted:** the common rsyncd.conf GLOBAL keys (`port`, `address`, `motd file`, `max connections`, `hosts allow`/`hosts deny`, plus the inert `pid file`, `log file`, `socket options`/`sockopts`, `listen backlog`, `syslog facility`, `syslog tag`, `log format`, `use chroot`, `uid`, `gid`, `timeout`, `max verbosity`/`min verbosity`, `lock file`, `transfer logging`, `strict modes`, `reverse lookup`/`forward lookup`, `ignore errors`, `ignore nonreadable`, `dont compress`) and MODULE keys (`path`, `read only`, `max connections`, `auth users`, `hosts allow`/`hosts deny`, plus the inert `comment`, `use chroot`, `uid`/`gid`/`daemon uid`/`daemon gid`, `exclude`, `include`, `exclude from`/`include from`, `filter`, `secrets file`, `auth digest`, `max verbosity`/`min verbosity`, `lock file`, `transfer logging`, `log file`/`log format`/`syslog facility`/`syslog tag`, `timeout`, `strict modes`, `numeric ids`, `fake super`, `munge symlinks`, `write only`, `list`, `dont compress`, `charset`, `refuse options`, `incoming chmod`/`outgoing chmod`, `open noatime`, `max size`/`min size`, `temp dir`, `pre-xfer exec`/`post-xfer exec`, `name converter`, `proxy protocol`/`proxy protocol hosts`, `reverse lookup`/`forward lookup`, `ignore errors`, `ignore nonreadable`) are recognized. Keys with a FastSync equivalent map onto it. Modules are **read-only by default**, exactly like rsync: `read only = no` (or `write only = yes`, which FastSync maps to writability because it is push-only) opts a module in; a global `read only` sets the default for later modules, and an explicit module value always wins. Keys with no FastSync equivalent load **inert** (no effect) rather than failing the whole config, and every inert key whose intent is access control (`secrets file`, `refuse options`, `exclude`/`include`/`filter`, `max size`/`min size`, `pre-xfer exec`/`post-xfer exec`, `incoming chmod`/`outgoing chmod`, `name converter`, `use chroot`, `uid`/`gid`, ...) emits a startup **WARN** naming the key (and module), so an operator cannot mistake an unenforced restriction for an enforced one. Residual: the native grammar still differs from rsync's (no `\` line continuation, `%VAR%` expansion, `[global]` re-entry, or inline `#` comments), and the inert keys are genuinely not enforced — in particular a daemon-side `exclude`/`filter` is NOT applied and `secrets file` is NOT read (use `path`, `--password-file`, and client-side filters instead) |
|
||||
| `--dparam=OVERRIDE` | Override global daemon config | ❌ Divergent | Wave A: overrides one global scalar from the command line (`--dparam port=8734` and `--dparam=KEY=VALUE` both work). Reuses the exact same global-key dispatch as `--config`, so it accepts the native global keys (`port`, `motd file`, `address`, `read only`, `max connections`, `max connections per host`, `auth failure delay`, `auth lockout threshold`, `auth lockout duration`, `hosts allow`, `hosts deny`), the recognized inert rsync global keys, and rsync's compact spellings (`motdfile`, `pidfile`, `logfile`); keys are case-insensitive. `read only` sets the global default and re-applies it to every module that did not set its own value; the default is `yes` (rsync modules are read-only unless `read only = no` / `write only = yes`), so `--dparam read only=no` is required to make modules without their own value writable, and an inert security global key (`use chroot`, `uid`, `gid`, `strict modes`) emits the same startup **WARN** as `--config`. Genuinely unknown keys and invalid values are rejected. Requires `--daemon` |
|
||||
| `--no-detach` | Don't detach from parent | ✅ Parity | Wave A: with `--daemon`, keeps the listener in the foreground (what integration tests use). Without it the daemonizes (fork/setsid, stdio redirected to /dev/null) after the listening socket is bound. Requires `--daemon` |
|
||||
| `--password-file=FILE` | Read daemon password from file | ❌ Divergent | A7 daemon auth. Client: `--password-file` supplies `user:password` for a `host::module/path` destination (the username is taken from this file, so `user@host::module` stays rejected); the literal password is held client-side only for the SCRAM handshake and wiped at teardown. Server (`fastsync-server --daemon --password-file FILE`): the salted-PBKDF2 verifier store that modules with `auth users` are verified against. **Neither the password nor any replayable bearer value crosses the wire or is stored server-side** — the store holds a per-user salt plus derived keys, and the daemon proves the secret with a per-connection nonce challenge. The file must be private to its owner: both the client and server verify the exact inode they read (open-then-`fstat`, so the check cannot be raced) and refuse a `--password-file`/`--early-input` that is not owned by the current user or grants any group/other permission bit (mode 0600), mirroring the TLS private-key check. A process-substitution pipe (`--early-input <(vault ...)`) is still accepted when it satisfies those checks. **Hardening follow-up:** the file is opened with `O_NOFOLLOW`, so a symlinked credential path fails closed (`ELOOP`) instead of being followed before the owner/mode gate; literal fd-backed paths (`/dev/fd/<digits>`, `/proc/self/fd/<digits>`, which is what a bash process substitution passes) are exempt, so process substitution still works. A FIFO/process-substitution read now waits under a bounded ~3 s deadline for its writer, so a slow producer works while a connected-but-silent FIFO fails instead of hanging. See the Daemon Mode notes below for the file formats and the plaintext/TLS caveat |
|
||||
| `--early-input=FILE` | Use FILE for daemon early exec | ❌ Divergent | Server-only (requires `--daemon`): a second credential-store file, same new-format grammar as `--password-file`, read before the listener accepts connections (a secrets-manager / process-substitution source). Its entries layer over `--password-file`: byte-identical verifiers dedupe, a conflicting verifier for the same user is a startup error. Opened with the same `O_NOFOLLOW` hardening as `--password-file` (a symlinked path fails closed with `ELOOP`; fd-backed `/dev/fd/N`/`/proc/self/fd/N` process-substitution paths are exempt) and a FIFO read is bound-waited (~3 s) so a slow producer works while a writer-less FIFO cannot hang. A daemon whose modules declare `auth users` must be given at least one of the two, or it refuses to start (fail closed) |
|
||||
@@ -739,7 +747,7 @@ targets verbatim, matching rsync.
|
||||
|
||||
**Daemon Mode notes (Wave A protocol 2.15.0; A7 auth protocol 2.19.0; MOTD no bump):** FastSync daemon mode is supported in FastSync's own protocol/config grammar, not rsync's SMB/daemon option encoding.
|
||||
|
||||
- **Config grammar** (`fastsyncd.conf`): line-based; an implicit global section first, then `[module]` sections. Keys are case-insensitive, values are trimmed and may be wrapped in one layer of double quotes (`path = "/srv/my dir"`). `#` and `;` at the start of a line (after leading whitespace) are full-line comments; inline comments and `\` continuations are not supported. Lines are bounded (4096 chars), and at most 256 `[module]` sections are accepted. Global keys: `port` (default 873), `motd file` (the daemon sends its bounded, escaped content to a client after the module gate/auth accepts, unless the client passes `--no-motd`), `address` (optional bind address), `max connections` (positive integer cap on concurrent connections, default 100; 0/negative/garbage is a parse error), `max connections per host` (concurrent-connection cap per source IP, default 0 = unlimited), `auth failure delay` (milliseconds to sleep after a failed authentication, default 500; 0 disables, capped at 5000), `auth lockout threshold` (failed authentications from one source before lockout, default 10; 0 disables), `auth lockout duration` (seconds a locked-out source is refused, default 300), `hosts allow` and `hosts deny` (comma- and/or whitespace-separated host access patterns — see the host access control note below). Module keys: `path` (required; the daemon-side authorized root for that module), `read only` (yes/no/true/false/1/0, default no), `client owner` (yes/no/true/false/1/0, default no; opts the module into client-chosen ownership — see below), `auth users` (comma list), `max connections` (optional per-module cap, 0 = unlimited; enforced across all connection children), `hosts allow`/`hosts deny` (per-module host access lists). **Unknown keys and malformed lines are parse-and-reject errors** (never silently ignored), so a typo cannot change what a module serves.
|
||||
- **Config grammar** (`fastsyncd.conf`): line-based; an implicit global section first, then `[module]` sections. Keys are case-insensitive, values are trimmed and may be wrapped in one layer of double quotes (`path = "/srv/my dir"`). `#` and `;` at the start of a line (after leading whitespace) are full-line comments; inline comments and `\` continuations are not supported. Lines are bounded (4096 chars), and at most 256 `[module]` sections are accepted. Global keys: `port` (default 873), `motd file` (the daemon sends its bounded, escaped content to a client after the module gate/auth accepts, unless the client passes `--no-motd`), `address` (optional bind address), `max connections` (positive integer cap on concurrent connections, default 100; 0/negative/garbage is a parse error), `max connections per host` (concurrent-connection cap per source IP, default 0 = unlimited), `auth failure delay` (milliseconds to sleep after a failed authentication, default 500; 0 disables, capped at 5000), `auth lockout threshold` (failed authentications from one source before lockout, default 10; 0 disables), `auth lockout duration` (seconds a locked-out source is refused, default 300), `hosts allow` and `hosts deny` (comma- and/or whitespace-separated host access patterns — see the host access control note below). Module keys: `path` (required; the daemon-side authorized root for that module), `read only` (yes/no/true/false/1/0, default yes — rsync modules are read-only unless `read only = no`/`write only = yes`), `client owner` (yes/no/true/false/1/0, default no; opts the module into client-chosen ownership — see below), `auth users` (comma list), `max connections` (optional per-module cap, 0 = unlimited; enforced across all connection children), `hosts allow`/`hosts deny` (per-module host access lists). **Unknown keys and malformed lines are parse-and-reject errors** (never silently ignored), so a typo cannot change what a module serves. To reduce the rsync divergence, the parser additionally accepts the common rsync 3.4.1 GLOBAL and MODULE keys: the keys with a FastSync equivalent (`path`, `read only`, `max connections`, `auth users`, `hosts allow`/`hosts deny`, and the global `port`/`address`/`motd file`) map onto it, a global `read only` becomes the default for modules defined after it, and the keys with no FastSync equivalent (e.g. `pid file`, `log file`, `use chroot`, `uid`/`gid`, `comment`, `exclude`/`include`, `max verbosity`, `lock file`, `transfer logging`, `timeout`, `secrets file`) are recognized and loaded **inert** (accepted-but-ignored) instead of failing the whole file. `--dparam` reuses the same dispatch, so it also accepts the inert rsync global keys and the compact spellings `motdfile`/`pidfile`/`logfile`. A key outside both sets is still rejected. The inert keys are genuinely not enforced: a daemon-side `exclude`/`include`/`filter` is not applied and a `secrets file` is not read (use `--password-file`/`--early-input`), so an rsync config that relies on those must be edited rather than trusted.
|
||||
- **Host access control (`hosts allow`/`hosts deny`):** both keys accept a comma- and/or whitespace-separated list of patterns and may appear globally and/or per module (multiple config-file lines append; a `--dparam` override replaces). Supported patterns are `*` (match all), an IPv4 or IPv6 literal (`10.0.0.1`, `2001:db8::1`), and an IPv4/IPv6 CIDR (`10.0.0.0/8`, `2001:db8::/32`). Hostname patterns are **not** supported: because the peer is always a numeric address and no reverse DNS is performed, a hostname/glob pattern would silently never match, so it is rejected at load time (fail-closed) instead of being accepted as a dead rule. An IPv4 peer on a dual-stack IPv6 listener is normalized from its `::ffff:a.b.c.d` form so IPv4 patterns match it. rsync-like semantics: a matching `hosts deny` rejects; if any `hosts allow` entries exist, a peer matching none of them is rejected; deny takes precedence over allow. The daemon enforces the global list first, then the selected module's list, **before authentication** in `server_module_gate`, with an audit log line naming the peer, the module and the outcome. The numeric peer address is obtained with `getpeername`+`inet_ntop` (`utils_fd_peer_ip`, handling both address families); when it cannot be obtained a module with any ACL fails closed (refused), while an ACL-free module continues and logs at debug. A malformed pattern (e.g. an out-of-range CIDR prefix) is a parse error at load time.
|
||||
- **Connection caps, shared registry and auth lockout:** the global `max connections` key (default 100) is plumbed into the listener (`transport_tcp.c`), which rejects a connection once the accept-loop parent's active-child count reaches it; the IPv4/IPv6 peer is logged for every accepted connection. Because the listener forks one child per connection, the per-module `max connections` cap, the global `max connections per host` cap, and the auth-failure counter live in a fixed-size registry carved from an anonymous shared mapping (`daemon_limits.c`, `mmap(MAP_SHARED|MAP_ANONYMOUS)`) created by the parent before the accept loop, so every forked child shares the same counters (C11 atomics only — never a pthread lock, which can deadlock in a forked child). The parent reserves a registry slot per accepted connection and the child records the selected module and source IP once known; the parent's `SIGCHLD` handler reclaims the slot when the child dies (including `SIGKILL`) and re-derives the per-module and per-source occupancy counts from the surviving REGISTERED slots, so a child killed mid-registration cannot leak a count. The per-source table has a bounded lifetime: an entry with no live connection is reclaimed after its lockout expires or it has been idle (300 s); if the table is genuinely full the per-source cap/lockout fails open for new sources (per-module cap and ACLs still apply) with a rate-limited warning. The per-module cap (0 = unlimited) is enforced after the module lookup and before auth; per-source identity reuses the normalized numeric peer address (`utils_fd_peer_ip`, IPv4-mapped IPv6 collapsed to IPv4), and a trusted loopback peer (127.0.0.0/8 / `::1`, `utils_fd_peer_is_local`) is exempt from the per-source cap and the auth lockout because all local clients share one address (the per-module/global caps still apply). Clients behind a shared NAT/proxy address likewise share one per-source budget and lockout counter. A failed authentication increments the shared per-source failure count and, once `auth lockout threshold` (default 10; 0 disables) is reached, the source is refused for `auth lockout duration` seconds (default 300) before any challenge is sent, even when the next attempt is handled by a different forked child; a successful authentication clears the counter. On a failed authentication the per-connection child still sleeps the global `auth failure delay` (default 500 ms, 0 disables, capped at 5000) via `nanosleep`, rate-limiting online guessing without delaying a success. A missing registry (allocation failure) degrades to the global cap and host ACLs rather than refusing to start.
|
||||
- **Module selection & confinement:** the client requests a module with an rsync-style `host::module[/path]` destination. The module name crosses the wire as a trailing string on the config frame (bumping `PROTOCOL_VERSION` 2.14.0 → 2.15.0; the bump is required because the config-frame layout changed and the strict same-version handshake is what prevents a peer from desynchronizing on the new trailing field). The daemon looks the module up in ITS OWN config and uses the module's `path` as the authorized root through the exact same `configure_authorization` confinement the standalone server applies to `--destination-root` (`file_open_secure_parent`, `has_path_traversal`, `path_is_within`); the client never supplies the root, every client-chosen-ownership/super-user request is refused unless the module declares `client owner = yes` (the daemon's per-module opt-in, see below), and the operator `--no-super` veto forces super-user activities off for every daemon connection. The client's `/path` part is relative inside the module and is rejected if absolute or if it contains `..`. Unknown modules are refused before any data moves (the run fails cleanly at the config handshake). An absolute destination and a module request against a non-daemon server are also refused.
|
||||
@@ -936,7 +944,7 @@ These are the last compatibility items and the closing phase toward rsync flag p
|
||||
| `-T` / `--timeout` | `-T` = `--temp-dir` | → `--timeout` (long-only) |
|
||||
| `-a` / `--archive` (= `-c -m -M`) | `-a` = `-rlptD` | → becomes **real rsync `-a`** after the renames |
|
||||
|
||||
**Wave B — Output & filesystem completion (✅ implemented).** `-S`/`--sparse` (`⚠️→✅`): real hole preservation — a sparse-aware writer (`write_all_sparse`) skips all-zero runs ≥ 4096 bytes with `lseek(SEEK_CUR)` and `ftruncate`s the final size, wired into both the atomic temp+rename store and `--inplace` receiver-side with **no wire change** (the full file image is already in memory; the ftruncate presize is kept). `-P` (`⚠️→✅`): interrupted-write retention — on a save failure after data reached the temp fd, `--partial` now renames the already-written temp to the destination path (best-effort; falls through to the normal unlink on failure, never retains when `--partial` is off) so a later `--append`/`--append-verify` run can resume. `--block-size=SIZE` (`⚠️→✅`): promoted after verification — `--block-size` is now an alias for `--delta-block`, both set `config->delta_block_size`, which the delta engine already honored end-to-end (`delta_signature_create_seeded` + `delta_apply`); out-of-range values keep the default. `--fake-super` (`⚠️→✅`): added `fake_super_restore_fd` to parse and re-apply the recorded `user.fastsync.stat` record fd-relative (mode/time only — protocol 2.23.0: **never a real chown**; the resolved owner is recorded for a later privileged restore); a save under `--fake-super` now re-applies the recorded attrs instead of only recording them, with the recording format unchanged. `--stderr=client` (`⚠️→❌ Divergent`): FastSync has no rsync client-message channel, and `client` is rejected at CLI parse — the rejection is the documented behavior (unit-tested). `-N`/`--crtimes` (`⚠️→❌ Divergent`): birth-times cannot be set by any portable fs call (`utimensat` sets only atime/mtime); capture/transmit stays, setting is impossible, the flag is accepted and safely inert. Review-hardening (post-eval): fake-super replay applies the mode through the shared `metadata_mode_for_policy` helper (protocol 2.23.0: exactly the source mode under `-p`, with no masking); `--sparse` takes precedence over `--preallocate` (posix_fallocate skipped so holes survive) — **reversed by the parity-completion wave: `--preallocate` now wins, matching rsync**; `--partial` retention is disabled for `--no_replace` (ignore/existing) and only marks a write-attempt after the actual write begins; `--block-size=SIZE`/`--delta-block=SIZE` inline forms are accepted.
|
||||
**Wave B — Output & filesystem completion (✅ implemented).** `-S`/`--sparse` (`⚠️→✅`): real hole preservation — a sparse-aware writer (`write_all_sparse`) skips all-zero runs ≥ 4096 bytes with `lseek(SEEK_CUR)` and `ftruncate`s the final size, wired into both the atomic temp+rename store and `--inplace` receiver-side with **no wire change** (the full file image is already in memory; the ftruncate presize is kept). `-P` (`⚠️→✅`): interrupted-write retention — on a save failure after data reached the temp fd, `--partial` now renames the already-written temp to the destination path (best-effort; falls through to the normal unlink on failure, never retains when `--partial` is off) so a later `--append`/`--append-verify` run can resume. `--block-size=SIZE` (`⚠️→✅`): promoted after verification — `--block-size` is now an alias for `--delta-block`, both set `config->delta_block_size`, which the delta engine already honored end-to-end (`delta_signature_create_seeded` + `delta_apply`); out-of-range values keep the default. `--fake-super` (`⚠️→✅`): added `fake_super_restore_fd` to parse and re-apply the recorded `user.fastsync.stat` record fd-relative (mode/time only — protocol 2.23.0: **never a real chown**; the resolved owner is recorded for a later privileged restore); a save under `--fake-super` now re-applies the recorded attrs instead of only recording them. (The later fake-super xattr-interop pass replaced that native `user.fastsync.stat` format with rsync's `user.rsync.%stat` grammar — see the row and Phase-4 notes.) `--stderr=client` (`⚠️→❌ Divergent`): FastSync has no rsync client-message channel, and `client` is rejected at CLI parse — the rejection is the documented behavior (unit-tested). `-N`/`--crtimes` (`⚠️→❌ Divergent`): birth-times cannot be set by any portable fs call (`utimensat` sets only atime/mtime); capture/transmit stays, setting is impossible, the flag is accepted and safely inert. Review-hardening (post-eval): fake-super replay applies the mode through the shared `metadata_mode_for_policy` helper (protocol 2.23.0: exactly the source mode under `-p`, with no masking); `--sparse` takes precedence over `--preallocate` (posix_fallocate skipped so holes survive) — **reversed by the parity-completion wave: `--preallocate` now wins, matching rsync**; `--partial` retention is disabled for `--no_replace` (ignore/existing) and only marks a write-attempt after the actual write begins; `--block-size=SIZE`/`--delta-block=SIZE` inline forms are accepted.
|
||||
|
||||
**Wave C — Devices & special files (finalize statuses + tests) (✅ implemented).** The four special-file rows are finalized with coverage tests. `--devices`, `--copy-devices`, and `--write-devices` are **✅ Implemented**, each with a documented, safety-driven divergence: device-node creation is privilege-gated, so a receiver without `CAP_MKNOD` skips that entry with a warning (a per-entry skip, never a transfer failure); `--copy-devices` copies a device/FIFO's reported size into an ordinary regular file (a size-bounded safe divergence from rsync's unbounded dd-like read); `--write-devices` writes only into an existing char/block node under the confined receive root and skips every unusable target rather than clobbering or aborting. `--specials` reclassified from **⛔ Impossible/Divergence** to **✅ Parity** in protocol 2.23.0: **FIFO recreation works** (unprivileged `mkfifo`) **and unix sockets are recreated** with `mknod(S_IFSOCK)`, which Linux permits unprivileged (the flag previously assumed sockets were impossible — see the `--specials` row). Tests assert FIFO recreation, socket recreation, the regular-file result of `--copy-devices`, the skipped/missing and non-device `--write-devices` targets, and (root-gated) real device-node creation; a root runner additionally drops the receiver to an unprivileged user to assert the `CAP_MKNOD` skip is graceful. (The parity-completion wave later reclassified `--devices`, `--copy-devices`, and `--write-devices` as explicit **❌ Divergent** rows, because their safe subsets are deliberately not rsync's behavior; the implementation itself is unchanged.)
|
||||
|
||||
@@ -956,7 +964,7 @@ These are the last compatibility items and the closing phase toward rsync flag p
|
||||
|
||||
**Wire:** two trailing config-frame blocks after the `--iconv` spec, in fixed order — `send_privilege_options`/`receive_privilege_options` (one `super_mode` int, validated `0..2`), then `send_copy_as_options`/`receive_copy_as_options` (presence int + two int32 ids, validated `>= 0`, with `copy_as_set ⇒ use_metadata`). `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Divergences from rsync:** rsync's `--super` elevates the receiver and `--copy-as` actually switches its credentials; FastSync never elevates and only permits/forwards confined attempts, and `--copy-as` forces ownership rather than switching identity.
|
||||
|
||||
**Honest status after the parity 2.29 cycle (protocol 2.28.0, no wire change), updated by the parity cycle 2.29 pass, the audit-cycle follow-ups, and the triage cycle.** ✅ Parity 117 / ⚠️ Caveat 13 / ❌ Divergent 27 = 157 rows. The 2.29 cycle closed the scanner-order, delete-timing, relative-basis, and fuzzy-eligibility residuals (moving `-n`/`--delete`/`--del`/`--delete-delay` to ✅) and improved the `--info`/`--stats`/`--debug` partial rows; the triage cycle moved `-F` and `-i`/`--itemize-changes` ✅ → ⚠️ for their documented residuals. The remaining ⚠️ rows are `--info`, `--debug`, `--msgs2stderr`, `--stats`, `--progress`, `-i`, `--delete-before`, `--filter`, `-F`, the three basis-dir options, and `-y/--fuzzy`. Earlier: **Honest status after the parity 2.28.0 cycle (protocol 2.28.0), updated by the rsync-parity-stats, rsync-parity-options, rsync-parity-fs, parity-review, no-wire parity-track-1/2b and wire parity-track-4a/5a passes.** ✅ Parity 116 / ⚠️ Caveat 14 / ❌ Divergent 27 = 157 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. This reclassification makes every difference explicit. The completion wave closed 23 previously-caveated rows (9 that triage showed were already parity, plus 14 genuine fixes) and turned the 17 inherently non-rsync rows — native daemon config/auth, the FastSync batch container, the safe-subset device/privilege flags, `-X`'s privileged namespaces, `--fake-super`'s native xattr format, and the `--old-args` no-op — into explicit ❌ divergences. The stats pass flipped `--delete-delay` to ✅ (actual-removal accounting), but the parity-review pass moved it back to ⚠️ because FastSync charged the `--max-delete` budget at plan/snapshot time and left a refilled snapshotted directory in place, whereas rsync charges on actual removals and recursively removes a queued directory (including content created after its plan). The no-wire parity-track-1 pass fixed both (actual-removal charging plus recursive deferred removal with an independent deferred-list cap), narrowing the caveat to the partial-delete ordering. The stats pass also reclassified `--out-format` to ❌ (protocol-specific `%b`/delta-`%c`), and sharpened the `--stats`/`--progress`/`--checksum-choice` residuals. The options pass flipped `--bwlimit` and `--ignore-errors` to ✅ (rsync-exact size parsing and ~100 ms leaky-bucket throttling, and rsync's skip-unreadable-subdir plus IO-error-suppressed deletion with exit 23) and emits rsync-format `--info=name/flist/del/remove/nonreg/progress` lines (real-run `deleting`/`*deleting` carried over a new trailing `report_deletes` wire bool, `PROTOCOL_VERSION` 2.26.0 → 2.27.0), while reclassifying `-M` over daemon/TCP
|
||||
**Honest status after the parity 2.29 cycle (protocol 2.28.0, no wire change), updated by the parity cycle 2.29 pass, the audit-cycle follow-ups, the triage cycle, and a later no-wire parity pass.** ✅ Parity 119 / ⚠️ Caveat 14 / ❌ Divergent 24 = 157 rows. The no-wire parity pass accepted `--inc-recursive`/`--no-inc-recursive` as inert no-ops (❌ → ✅, since FastSync's full scan is rsync's `--no-inc-recursive` and the destination is identical), narrowed the `--temp-dir` divergence by accepting an absolute path that canonicalizes inside the receive root (the row stays ❌ for out-of-root absolute paths), closed the `--delete-before` phase-0 divergence (⚠️ → ✅: both the single-threaded and the `--threads` data passes now replay the pre-scan file list, so a source file created after the scan is neither transferred nor kept, matching rsync), and moved `--fake-super` and `--devices` ❌ → ⚠️ (`--fake-super` now writes/reads rsync's exact `user.rsync.%stat` key and `<octal-mode> <rdev_major>,<rdev_minor> <uid>:<gid>` grammar, interoperating with real rsync 3.4.1 for regular files and faking char/block devices as regular files carrying the real rdev; `--devices` now logs a failed device `mknod` as a per-entry failure that continues the transfer instead of a silent non-root skip — see those rows for the remaining directory-faking and exit-code residuals). A review pass then hardened the fake-super stat parser (strict range-checked parsing), made rsync-style daemon modules read-only by default with a startup warning for accepted-but-unenforced access-control keys, and extended the `--delete-before` replay to the `--threads` path. The 2.29 cycle closed the scanner-order, delete-timing, relative-basis, and fuzzy-eligibility residuals (moving `-n`/`--delete`/`--del`/`--delete-delay` to ✅) and improved the `--info`/`--stats`/`--debug` partial rows; the triage cycle moved `-F` and `-i`/`--itemize-changes` ✅ → ⚠️ for their documented residuals. The remaining ⚠️ rows are `--info`, `--debug`, `--msgs2stderr`, `--stats`, `--progress`, `-i`, `--filter`, `-F`, the three basis-dir options, `-y/--fuzzy`, `--fake-super`, and `--devices`. Earlier: **Honest status after the parity 2.28.0 cycle (protocol 2.28.0), updated by the rsync-parity-stats, rsync-parity-options, rsync-parity-fs, parity-review, no-wire parity-track-1/2b and wire parity-track-4a/5a passes.** ✅ Parity 116 / ⚠️ Caveat 14 / ❌ Divergent 27 = 157 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. This reclassification makes every difference explicit. The completion wave closed 23 previously-caveated rows (9 that triage showed were already parity, plus 14 genuine fixes) and turned the 17 inherently non-rsync rows — native daemon config/auth, the FastSync batch container, the safe-subset device/privilege flags, `-X`'s privileged namespaces, `--fake-super`'s native xattr format, and the `--old-args` no-op — into explicit ❌ divergences. The stats pass flipped `--delete-delay` to ✅ (actual-removal accounting), but the parity-review pass moved it back to ⚠️ because FastSync charged the `--max-delete` budget at plan/snapshot time and left a refilled snapshotted directory in place, whereas rsync charges on actual removals and recursively removes a queued directory (including content created after its plan). The no-wire parity-track-1 pass fixed both (actual-removal charging plus recursive deferred removal with an independent deferred-list cap), narrowing the caveat to the partial-delete ordering. The stats pass also reclassified `--out-format` to ❌ (protocol-specific `%b`/delta-`%c`), and sharpened the `--stats`/`--progress`/`--checksum-choice` residuals. The options pass flipped `--bwlimit` and `--ignore-errors` to ✅ (rsync-exact size parsing and ~100 ms leaky-bucket throttling, and rsync's skip-unreadable-subdir plus IO-error-suppressed deletion with exit 23) and emits rsync-format `--info=name/flist/del/remove/nonreg/progress` lines (real-run `deleting`/`*deleting` carried over a new trailing `report_deletes` wire bool, `PROTOCOL_VERSION` 2.26.0 → 2.27.0), while reclassifying `-M` over daemon/TCP
|
||||
and receiver-side `protect`/`risk` re-derivation to ❌ (no argv channel /
|
||||
receiver filter engine); the wire parity-track-4a pass later added that
|
||||
receiver filter engine, flipping `--filter=RULE` back to ✅ (see above; the
|
||||
@@ -1021,7 +1029,9 @@ integration tests unless it is explicitly listed as a limitation.
|
||||
### Filesystem and deletion semantics
|
||||
|
||||
- **`--temp-dir` is confined to the receive root on the receiver:** a relative
|
||||
dir resolves below it; an absolute path or one containing `..` is rejected.
|
||||
dir resolves below it, and an absolute path is accepted only when
|
||||
`realpath(3)` confirms it is inside the canonical receive root; an
|
||||
out-of-root absolute path or one containing `..` is rejected.
|
||||
An `EXDEV` install falls back to a non-atomic copy instead of aborting. (The
|
||||
confined receiver path cannot be mount-tested in the CI container — no
|
||||
`CAP_SYS_ADMIN` and unprivileged user namespaces are disabled — so the
|
||||
@@ -1057,9 +1067,11 @@ integration tests unless it is explicitly listed as a limitation.
|
||||
clear configuration error (matching rsync) instead of an order-dependent
|
||||
winner.
|
||||
- **`--fake-super` never real-chowns.** It records the *resolved* owner (the
|
||||
active mapping, else the source id) in `user.fastsync.stat` for a later
|
||||
privileged restore and replays only mode/times. Directory ownership and
|
||||
directory xattrs/ACLs are preserved alongside file entries.
|
||||
active mapping, else the source id) in rsync's `user.rsync.%stat` for a later
|
||||
privileged restore and replays only the permission bits (mtime travels through
|
||||
the normal metadata path). Directory ownership and
|
||||
directory xattrs/ACLs are preserved alongside file entries, though directories
|
||||
themselves are not yet given a `%stat%` record.
|
||||
- **`--chmod`** implements rsync's `D`/`F`/`X` selectors, `s`/`t`, append
|
||||
semantics, does not imply `-p`, and applies its changes without sanitization.
|
||||
|
||||
@@ -1095,8 +1107,9 @@ These remain after the wave; they are the reasons a row above is ⚠️.
|
||||
link-following tool can follow a link outside the receive root. Use
|
||||
`--safe-links` when the source is untrusted. `--trust-sender` does **not**
|
||||
affect symlink targets.
|
||||
- **`--temp-dir` absolute/foreign-filesystem paths are rejected by the
|
||||
receiver** (rsync's daemon also confines; standalone rsync differs).
|
||||
- **`--temp-dir` out-of-root absolute and foreign-filesystem paths are rejected
|
||||
by the receiver** (an absolute path that canonicalizes inside the receive root
|
||||
is accepted; rsync's daemon also confines; standalone rsync differs).
|
||||
- **`--copy-devices` reads a bounded `st_size`** rather than rsync's unbounded
|
||||
device read.
|
||||
- **A broken symlink referent under `--copy-links`/`--copy-unsafe-links` exits 0**
|
||||
@@ -1235,8 +1248,9 @@ These remain after the wave; the individual rows carry the precise wording.
|
||||
`--ignore-errors` exits 23 but its EACCES differential is not
|
||||
exercised in CI.
|
||||
- **`--delay-updates`** uses a fixed staging name with an advisory lock and
|
||||
deletes before publication; **`--temp-dir`** rejects absolute/foreign paths
|
||||
(deliberately confined, see the row); **`--remote-option`** is SSH-only.
|
||||
deletes before publication; **`--temp-dir`** rejects out-of-root absolute and
|
||||
foreign paths (in-root absolute paths are accepted; deliberately confined, see
|
||||
the row); **`--remote-option`** is SSH-only.
|
||||
**`--iconv`** now matches rsync's push direction (destination charset = the
|
||||
spec's REMOTE half; a server `--iconv` overrides it).
|
||||
- **Basis dirs** now use rsync's metadata quick-check by default (track 5a) and
|
||||
@@ -1247,15 +1261,17 @@ These remain after the wave; the individual rows carry the precise wording.
|
||||
engine (both files ≥ 16 KiB, size ratio ≤ 10×), a narrower window than
|
||||
rsync's, so the selected basis — and the `--stats` bandwidth counters —
|
||||
can differ while the tree stays byte-exact.
|
||||
- **`--inc-recursive`/`--no-inc-recursive`** are not implemented (rejected).
|
||||
- **`--inc-recursive`/`--no-inc-recursive`** are accepted as inert no-ops:
|
||||
FastSync always performs a single full recursive scan (equivalent to
|
||||
rsync's `--no-inc-recursive`), so the destination is identical either way.
|
||||
|
||||
### Intentional divergences (explicit ❌ rows)
|
||||
|
||||
Native daemon config/auth (`--daemon`, `--config`, `--dparam`,
|
||||
`--password-file`, `--early-input`, `--hash-credentials`/`--iterations`), the
|
||||
non-interoperable batch container (`--write-batch`/`--only-write-batch`/
|
||||
`--read-batch`), `--fake-super`'s native xattr format, `-X`'s privileged
|
||||
namespaces, `--devices`/`--copy-devices`/`--write-devices`'s safe subsets,
|
||||
`--read-batch`), `-X`'s privileged
|
||||
namespaces, `--copy-devices`/`--write-devices`'s safe subsets,
|
||||
`--super`/`--copy-as`'s refusal to elevate or switch credentials, and the
|
||||
`-s`/`--secluded-args`/`--protect-args`/`--old-args` accepted no-ops.
|
||||
|
||||
|
||||
@@ -962,6 +962,13 @@ static const OptionEntry OPTION_TABLE[] = {
|
||||
/* rsync -r/--recursive: FastSync is always recursive, so this is a
|
||||
* faithful no-op (accepted silently, never consumes an argument). */
|
||||
{"--recursive", "-r", OPT_NOOP, 0},
|
||||
/* rsync's incremental-recursion scan-mode switch. FastSync always performs
|
||||
* a single full recursive scan, so both spellings are accepted as no-ops:
|
||||
* the destination is identical whichever mode the caller requests.
|
||||
* --no-inc-recursive is handled before the generic --no-* negation branch
|
||||
* (see cli_handle_pre_negation) but is registered here for discoverability. */
|
||||
{"--inc-recursive", NULL, OPT_NOOP, 0},
|
||||
{"--no-inc-recursive", NULL, OPT_NOOP, 0},
|
||||
{"--update", "-u", OPT_FLAG, offsetof(Config, update)},
|
||||
/* rsync's --old-args: accepted for CLI compatibility as a documented no-op
|
||||
* (the remote server path is always safely quoted; see usage.c). It is
|
||||
@@ -1387,6 +1394,12 @@ static bool cli_handle_pre_negation(CliParseCtx* ctx) {
|
||||
ctx->no_delta = true;
|
||||
else if (strcmp(arg, "--no-incremental") == 0)
|
||||
ctx->no_incremental = true;
|
||||
/* Real rsync option names that merely start with "--no-" and are inert
|
||||
* no-ops (e.g. --no-inc-recursive) are registered as OPT_NOOP entries;
|
||||
* accept them before the generic negation table would reject the name. */
|
||||
const OptionEntry* noop = find_table_option(arg);
|
||||
if (noop && noop->kind == OPT_NOOP)
|
||||
return true;
|
||||
if (apply_negation(config, arg) != 0) {
|
||||
ctx->exit_code = -1;
|
||||
return true;
|
||||
|
||||
@@ -381,21 +381,28 @@ bool files_from_list_check(const Config* config, ArrayList* missing_dest, int* s
|
||||
paths, loading and sending nothing. --delete-before/--delete-during need the
|
||||
complete keep-set manifest before the first data byte, so it is built by a
|
||||
dedicated pre-scan pass and transmitted early; the data pass then re-scans
|
||||
with a fresh scanner. A source I/O error is fatal unless the options carry
|
||||
--ignore-errors, in which case the scan continues past the unreadable
|
||||
directory and *io_error_out reports it (the caller still performs the
|
||||
deletion but reports the run as errored). */
|
||||
with a fresh scanner. --delete-before additionally replays this very scan as
|
||||
its data pass (rsync's single file list), so `chunks_out` (optional) retains
|
||||
the scanned Chunk objects for the caller to send instead of destroying them;
|
||||
the caller owns the list and must give it a chunk_destroy destructor. A
|
||||
source I/O error is fatal unless the options carry --ignore-errors, in which
|
||||
case the scan continues past the unreadable directory and *io_error_out
|
||||
reports it (the caller still performs the deletion but reports the run as
|
||||
errored). */
|
||||
bool scan_paths_only(const Config* config, const ScannerOptions* options, ArrayList* manifest,
|
||||
DeletePlanSender* plans, bool* io_error_out,
|
||||
unsigned long long* non_dir_count_out) {
|
||||
unsigned long long* non_dir_count_out, ArrayList* chunks_out,
|
||||
bool emit_nonreg) {
|
||||
if (io_error_out)
|
||||
*io_error_out = false;
|
||||
if (non_dir_count_out)
|
||||
*non_dir_count_out = 0;
|
||||
ScannerOptions local = *options;
|
||||
/* The pre-scan is a paths-only pass with no client output; it must not emit
|
||||
--info=nonreg lines (the data pass does that once). */
|
||||
local.note_nonreg = false;
|
||||
/* The pre-scan is normally a paths-only pass with no client output: it must
|
||||
not emit --info=nonreg lines because the data pass re-scans and emits them
|
||||
once. When the caller replays this scan as the data pass (--delete-before)
|
||||
there is no later scan, so it opts in and the lines are emitted here. */
|
||||
local.note_nonreg = emit_nonreg && options->note_nonreg;
|
||||
DirectoryScanner* scanner = directory_scanner_create_with_options(config->send_directory, &local);
|
||||
if (!scanner)
|
||||
return false;
|
||||
@@ -430,7 +437,16 @@ bool scan_paths_only(const Config* config, const ScannerOptions* options, ArrayL
|
||||
break;
|
||||
}
|
||||
}
|
||||
chunk_destroy(chunk);
|
||||
if (chunks_out) {
|
||||
/* Retain the chunk for the caller's data pass; ownership moves with it. */
|
||||
if (!array_list_add(chunks_out, chunk)) {
|
||||
ok = false;
|
||||
chunk_destroy(chunk);
|
||||
break;
|
||||
}
|
||||
} else {
|
||||
chunk_destroy(chunk);
|
||||
}
|
||||
}
|
||||
if (ok) {
|
||||
/* Keep every traversed source directory, including empty ones, so a plan
|
||||
|
||||
+118
-22
@@ -1148,6 +1148,43 @@ send_fail:
|
||||
static int scan_directory_multithreaded(void* pipeline_context) {
|
||||
PipelineContextSender* context = (PipelineContextSender*)pipeline_context;
|
||||
protocol_session_bind(&context->allocation_session);
|
||||
if (context->prescan_chunks != NULL) {
|
||||
/* --delete-before replays the pre-scan that built the early keep-set as the
|
||||
data pass (rsync builds one file list). Feed the retained chunks straight
|
||||
into the pipeline instead of re-reading the source, so a file created
|
||||
after the pre-scan is neither transferred nor kept. The chunk also
|
||||
carries the directory times captured by that scan (there is no later
|
||||
scan), so no scanner is created here. */
|
||||
bool failed = false;
|
||||
for (int i = 0; i < context->prescan_chunks->size; i++) {
|
||||
Chunk* chunk = (Chunk*)context->prescan_chunks->items[i];
|
||||
/* Move ownership out of the retained list so a cleanup here never
|
||||
double-frees a chunk the queue now owns. */
|
||||
context->prescan_chunks->items[i] = NULL;
|
||||
if (chunk == NULL)
|
||||
continue;
|
||||
if (!queue_enqueue_multithreaded_cancel(
|
||||
context->queue_scanner, chunk, &context->mutex_scanner,
|
||||
&context->condition_not_empty_scanner, &context->condition_not_full_scanner,
|
||||
&context->cancelled)) {
|
||||
chunk_destroy(chunk);
|
||||
failed = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
mtx_lock(&context->mutex_scanner);
|
||||
context->scanner_done = true;
|
||||
cnd_broadcast(&context->condition_not_empty_scanner);
|
||||
cnd_broadcast(&context->condition_not_full_scanner);
|
||||
mtx_unlock(&context->mutex_scanner);
|
||||
if (failed) {
|
||||
pipeline_cancel(context);
|
||||
protocol_session_unbind();
|
||||
return thrd_error;
|
||||
}
|
||||
protocol_session_unbind();
|
||||
return thrd_success;
|
||||
}
|
||||
PreparedScanner prepared;
|
||||
/* -j/--threads=N sizes the parallel scanner's worker pool; 0 (bare -j) lets
|
||||
* the scanner apply its built-in default. */
|
||||
@@ -1394,6 +1431,10 @@ typedef struct {
|
||||
Client* client;
|
||||
DirectoryScanner* scanner;
|
||||
ArrayList* manifest;
|
||||
/* --delete-before: the pre-scan that built the keep-set, retained as the data
|
||||
pass's file list (owning Chunk*; consumed chunks are NULLed as they are
|
||||
sent). NULL in every other mode, where the data pass scans normally. */
|
||||
ArrayList* prescan_chunks;
|
||||
DeletePlanSender* plan_sender;
|
||||
ArrayList* remove_sources;
|
||||
ArrayList* dir_entries;
|
||||
@@ -1478,12 +1519,23 @@ static bool send_files_prepare_delete(Config* config, SendFilesState* state) {
|
||||
if (state->delete_early) {
|
||||
/* Pass 1: collect the complete keep-set (paths only, no data loaded) and
|
||||
transmit it now, before any file data. The receiver removes extras and
|
||||
acks; the transfer aborts here if the deletion could not commit. */
|
||||
acks; the transfer aborts here if the deletion could not commit. The
|
||||
scanned chunks are retained so the data pass can replay this exact list
|
||||
instead of re-reading the source (rsync builds one file list and never
|
||||
transfers a file created after it). */
|
||||
ArrayList* early_manifest = array_list_create(free);
|
||||
if (!early_manifest)
|
||||
ArrayList* prescan_chunks = array_list_create(chunk_destroy);
|
||||
if (!early_manifest || !prescan_chunks) {
|
||||
array_list_delete(early_manifest);
|
||||
array_list_delete(prescan_chunks);
|
||||
return false;
|
||||
}
|
||||
/* No later scan runs for --delete-before, so this pass must also capture the
|
||||
deferred directory times and the --stats directory count. */
|
||||
state->prepared.options.dir_entries = state->dir_entries;
|
||||
state->prepared.options.dir_count = config->stats ? &state->dir_count : NULL;
|
||||
bool prescan_ok = scan_paths_only(config, &state->prepared.options, early_manifest, NULL,
|
||||
&state->had_scan_io, NULL);
|
||||
&state->had_scan_io, NULL, prescan_chunks, true);
|
||||
bool early_ok = false;
|
||||
bool skip_delete = false;
|
||||
if (prescan_ok) {
|
||||
@@ -1514,8 +1566,13 @@ static bool send_files_prepare_delete(Config* config, SendFilesState* state) {
|
||||
state->prepared.options.excluded_paths = NULL;
|
||||
state->prepared.options.size_skipped_paths = NULL;
|
||||
state->prepared.options.synced_dirs = NULL;
|
||||
if (!prescan_ok || (!early_ok && !skip_delete))
|
||||
if (!prescan_ok || (!early_ok && !skip_delete)) {
|
||||
array_list_delete(prescan_chunks);
|
||||
return false;
|
||||
}
|
||||
/* Adopt the captured scan as the data pass's file list (including when an
|
||||
I/O error suppressed only the deletion: the list is still complete). */
|
||||
state->prescan_chunks = prescan_chunks;
|
||||
} else if (state->delete_per_dir) {
|
||||
/* --delete-during/--delete-delay: build one plan per source directory from a
|
||||
path-only pre-scan and transmit the COMPLETE plan set now, before any data,
|
||||
@@ -1527,8 +1584,9 @@ static bool send_files_prepare_delete(Config* config, SendFilesState* state) {
|
||||
if (!state->plan_sender || !state->plan_dirs)
|
||||
return false;
|
||||
state->prepared.options.plan_dirs = state->plan_dirs;
|
||||
bool prescan_ok = scan_paths_only(config, &state->prepared.options, NULL, state->plan_sender,
|
||||
&state->had_scan_io, &state->per_dir_non_dir_count);
|
||||
bool prescan_ok =
|
||||
scan_paths_only(config, &state->prepared.options, NULL, state->plan_sender,
|
||||
&state->had_scan_io, &state->per_dir_non_dir_count, NULL, false);
|
||||
bool plans_ok = false;
|
||||
bool skip_delete = false;
|
||||
if (prescan_ok) {
|
||||
@@ -1592,24 +1650,42 @@ static bool send_files_run(Config* config, SendFilesState* state) {
|
||||
state->stop = stop_condition_make(config->stop_after_mins > 0, config->stop_after_mins,
|
||||
config->cli.stop_at_set, config->stop_at, now_mono);
|
||||
state->prepared.options.stop_condition = &state->stop;
|
||||
/* The early-delete pre-scan above already ran; only the data pass should feed
|
||||
the directory-time list (otherwise every directory would be captured
|
||||
twice). */
|
||||
state->prepared.options.dir_entries = state->dir_entries;
|
||||
state->prepared.options.dir_count = config->stats ? &state->dir_count : NULL;
|
||||
state->scanner =
|
||||
directory_scanner_create_with_options(config->send_directory, &state->prepared.options);
|
||||
if (!state->scanner)
|
||||
return false;
|
||||
/* --delete-before reuses the pre-scan that built the keep-set as the data
|
||||
pass's file list, so a source file created after that scan is neither
|
||||
transferred nor kept (rsync builds one file list). That pre-scan captured
|
||||
the deferred directory times and the --stats directory count because no
|
||||
later scan runs; every other mode opens a fresh data scanner here. */
|
||||
if (state->prescan_chunks == NULL) {
|
||||
state->prepared.options.dir_entries = state->dir_entries;
|
||||
state->prepared.options.dir_count = config->stats ? &state->dir_count : NULL;
|
||||
state->scanner =
|
||||
directory_scanner_create_with_options(config->send_directory, &state->prepared.options);
|
||||
if (!state->scanner)
|
||||
return false;
|
||||
}
|
||||
|
||||
Chunk* current_chunk;
|
||||
int prescan_index = 0;
|
||||
memset(&state->transfer_stats, 0, sizeof(state->transfer_stats));
|
||||
state->start = time(NULL);
|
||||
client_progress_begin(config);
|
||||
/* True when the stop deadline cut the scan short so the keep-set manifest is
|
||||
only a prefix of the source. */
|
||||
bool send_failed = false;
|
||||
while ((current_chunk = directory_scanner_next(state->scanner)) != NULL) {
|
||||
while (true) {
|
||||
if (state->prescan_chunks != NULL) {
|
||||
if (prescan_index >= state->prescan_chunks->size)
|
||||
break;
|
||||
/* Move ownership out of the retained list so chunk_destroy below (and the
|
||||
cleanup tail for an early exit) never double-frees it. */
|
||||
current_chunk = (Chunk*)state->prescan_chunks->items[prescan_index];
|
||||
state->prescan_chunks->items[prescan_index] = NULL;
|
||||
prescan_index++;
|
||||
} else {
|
||||
current_chunk = directory_scanner_next(state->scanner);
|
||||
if (current_chunk == NULL)
|
||||
break;
|
||||
}
|
||||
/* Graceful abort (Ctrl-C/SIGTERM): notify the receiver and clean up. The
|
||||
session is active (config_send already succeeded); a send failure here is
|
||||
fine because the client is exiting anyway. */
|
||||
@@ -1667,10 +1743,14 @@ static bool send_files_run(Config* config, SendFilesState* state) {
|
||||
* Returns the rsync-compatible exit code. */
|
||||
static int send_files_finalize(const Config* config, SendFilesState* state) {
|
||||
Client* client = state->client;
|
||||
if (directory_scanner_failed(state->scanner))
|
||||
return 1;
|
||||
if (directory_scanner_had_io_error(state->scanner))
|
||||
state->had_scan_io = true;
|
||||
/* A --delete-before run replays the pre-scan and owns no data scanner; its
|
||||
I/O-error verdict was already recorded by that pre-scan. */
|
||||
if (state->scanner != NULL) {
|
||||
if (directory_scanner_failed(state->scanner))
|
||||
return 1;
|
||||
if (directory_scanner_had_io_error(state->scanner))
|
||||
state->had_scan_io = true;
|
||||
}
|
||||
/* An abort that arrived after the last chunk must still stop the completion
|
||||
tail (manifest/finalize) rather than let it run to success. */
|
||||
if (client_abort_pending()) {
|
||||
@@ -1774,6 +1854,8 @@ static int send_files_finalize(const Config* config, SendFilesState* state) {
|
||||
static void send_files_cleanup(SendFilesState* state) {
|
||||
if (state->manifest)
|
||||
array_list_delete(state->manifest);
|
||||
if (state->prescan_chunks)
|
||||
array_list_delete(state->prescan_chunks);
|
||||
if (state->plan_sender)
|
||||
delete_plan_sender_destroy(state->plan_sender);
|
||||
if (state->excluded)
|
||||
@@ -1987,13 +2069,27 @@ int send_files_multithreaded(Config* config) {
|
||||
if (prepared_ok)
|
||||
prepared.options.plan_dirs = context->plan_dirs;
|
||||
} else {
|
||||
/* --delete-before: retain the pre-scan chunks as the pipeline's data
|
||||
pass (rsync's single file list) so a source file created after the
|
||||
scan is not transferred. No later scan runs, so this pass must also
|
||||
capture the deferred directory times and the --stats directory
|
||||
count. */
|
||||
context->manifest = array_list_create(free);
|
||||
prepared_ok = prepared_ok && context->manifest != NULL;
|
||||
context->prescan_chunks = array_list_create(chunk_destroy);
|
||||
prepared_ok = prepared_ok && context->manifest != NULL && context->prescan_chunks != NULL;
|
||||
if (prepared_ok) {
|
||||
prepared.options.dir_entries = context->dir_entries;
|
||||
prepared.options.dir_entries_mutex = &context->dir_entries_mutex;
|
||||
prepared.options.dir_count = config->stats ? &context->dir_count : NULL;
|
||||
if (!append_implied_dir_times(config, context->dir_entries))
|
||||
prepared_ok = false;
|
||||
}
|
||||
}
|
||||
bool prebuilt =
|
||||
prepared_ok &&
|
||||
scan_paths_only(config, &prepared.options, context->manifest, context->delete_plans,
|
||||
&context->scan_had_io_error, &pre_scan_non_dir);
|
||||
&context->scan_had_io_error, &pre_scan_non_dir, context->prescan_chunks,
|
||||
context->prescan_chunks != NULL);
|
||||
prepared_scanner_destroy(&prepared);
|
||||
if (per_dir && prebuilt) {
|
||||
const char* walk_root = delete_plan_walk_root(config, context->synced_dirs);
|
||||
|
||||
@@ -44,7 +44,8 @@ const char* delete_plan_walk_root(const Config* config, const ArrayList* synced_
|
||||
bool files_from_list_check(const Config* config, ArrayList* missing_dest, int* skipped_out);
|
||||
bool scan_paths_only(const Config* config, const ScannerOptions* options, ArrayList* manifest,
|
||||
DeletePlanSender* plans, bool* io_error_out,
|
||||
unsigned long long* non_dir_count_out);
|
||||
unsigned long long* non_dir_count_out, ArrayList* chunks_out,
|
||||
bool emit_nonreg);
|
||||
|
||||
/* client_report.c */
|
||||
void log_server_rejection(const char* context);
|
||||
|
||||
+8
-4
@@ -26,6 +26,9 @@ void print_usage(void) {
|
||||
printf(" owner, group, devices and specials; not\n");
|
||||
printf(" compression/multithreading\n");
|
||||
printf(" -r, --recursive Recurse into directories (FastSync is always recursive)\n");
|
||||
printf(" --inc-recursive Accepted for rsync CLI compatibility; no effect (FastSync\n");
|
||||
printf(" always performs a full scan, so the destination is identical)\n");
|
||||
printf(" --no-inc-recursive Accepted for rsync CLI compatibility; no effect\n");
|
||||
printf(" -n, --dry-run Show what would be transferred\n");
|
||||
printf(" --remove-source-files Remove regular source files after successful transfer\n");
|
||||
printf(" -p, --perms Preserve permission bits\n");
|
||||
@@ -205,10 +208,11 @@ void print_usage(void) {
|
||||
printf(" -A, --acls Preserve POSIX ACLs (the system.posix_acl_* xattrs;\n");
|
||||
printf(" setting an ACL the receiver is not permitted to\n");
|
||||
printf(" set is warned and skipped, never fatal)\n");
|
||||
printf(" --fake-super Store the source uid/gid/mode/mtime in a reserved\n");
|
||||
printf(" user.fastsync.stat xattr on each written file and\n");
|
||||
printf(" re-apply it (fd-relative) on a privileged run; the\n");
|
||||
printf(" recording format diverges from rsync's user.rsync.%%stat%%\n");
|
||||
printf(" --fake-super Store the source mode/rdev/uid/gid in rsync's\n");
|
||||
printf(" reserved user.rsync.%%stat xattr on each written\n");
|
||||
printf(" file (interoperable with rsync); it never performs a\n");
|
||||
printf(" real chown, so an unprivileged receiver records the\n");
|
||||
printf(" privileged stat for a later restore\n");
|
||||
printf(" --super Permit the receiver to attempt super-user activities\n");
|
||||
printf(" (char/block device-node creation, --write-devices)\n");
|
||||
printf(" within the confined receive root. Never elevates\n");
|
||||
|
||||
+34
-2
@@ -53,7 +53,13 @@ bool receiver_send_final_success(int fd, const Config* config, const ReceiverOut
|
||||
return send_status(fd, final_status);
|
||||
size_t count = outcomes ? outcomes->count : 0;
|
||||
for (size_t i = 0; i < count; i++) {
|
||||
Status per_file = outcomes->entries[i] == FILE_SAVE_WRITTEN ? STATUS_NEXT : STATUS_OK;
|
||||
Status per_file;
|
||||
if (outcomes->entries[i] == FILE_SAVE_WRITTEN)
|
||||
per_file = STATUS_NEXT;
|
||||
else if (outcomes->entries[i] == FILE_SAVE_FAILED)
|
||||
per_file = STATUS_ERROR;
|
||||
else
|
||||
per_file = STATUS_OK;
|
||||
if (!send_status(fd, per_file))
|
||||
return false;
|
||||
}
|
||||
@@ -719,6 +725,11 @@ typedef struct {
|
||||
ArrayList* would_delete;
|
||||
/* --info=del: actually-removed paths collected during the delete commit. */
|
||||
ArrayList* deleted_paths;
|
||||
/* Per-run count of entries that failed to materialize without aborting the
|
||||
stream (currently ONLY a --devices mknod EPERM/EACCES). A nonzero count
|
||||
makes the terminal frame carry a non-OK status so the client exits
|
||||
non-zero, matching rsync's continue-and-exit-partial behavior. */
|
||||
size_t failed_entries;
|
||||
} ReceiverSaveContext;
|
||||
|
||||
static bool receiver_save_file(File* file, void* context_pointer) {
|
||||
@@ -742,6 +753,11 @@ static bool receiver_save_file(File* file, void* context_pointer) {
|
||||
count as matched data in the end-of-transfer report. */
|
||||
if (result != FILE_SAVE_ERROR && file->matched_bytes > 0)
|
||||
context->stats.matched_data += file->matched_bytes;
|
||||
/* --devices parity: a device node the receiver could not mknod (EPERM/EACCES)
|
||||
is counted per-run but does not abort the transfer. The terminal frame
|
||||
turns a nonzero count into a non-OK status so the client exits non-zero. */
|
||||
if (result == FILE_SAVE_FAILED)
|
||||
context->failed_entries++;
|
||||
/* Protocol 2.28.0: receiver-observed literal bytes and the created-entry
|
||||
breakdown (regular/dir/link/special) for the `--stats` report. */
|
||||
if (result == FILE_SAVE_WRITTEN)
|
||||
@@ -775,9 +791,25 @@ static void receiver_note_delete_limit(void* context_pointer) {
|
||||
context->delete_limit_reached = true;
|
||||
}
|
||||
|
||||
/* Terminal status for a run. A capped --delete limit wins (rsync exit 25);
|
||||
otherwise any per-entry failure (for example an unprivileged --devices
|
||||
mknod) makes the terminal frame non-OK so the client exits non-zero. rsync
|
||||
reports 23 here; mapping the client's exact exit code to 23 is a separate,
|
||||
pre-existing concern. A clean run keeps STATUS_OK. */
|
||||
static Status receiver_final_status(bool delete_limit_reached, size_t failed_entries) {
|
||||
if (delete_limit_reached)
|
||||
return STATUS_DELETE_LIMIT;
|
||||
return failed_entries > 0 ? STATUS_ERROR : STATUS_OK;
|
||||
}
|
||||
|
||||
static bool receiver_send_success_frame(int fd, void* context_pointer) {
|
||||
ReceiverSaveContext* context = context_pointer;
|
||||
Status final_status = context->delete_limit_reached ? STATUS_DELETE_LIMIT : STATUS_OK;
|
||||
if (context->failed_entries > 0)
|
||||
log_message(LOG_LEVEL_WARNING,
|
||||
"%zu entr%s failed to materialize; continuing (partial transfer)",
|
||||
context->failed_entries, context->failed_entries == 1 ? "y" : "ies");
|
||||
Status final_status =
|
||||
receiver_final_status(context->delete_limit_reached, context->failed_entries);
|
||||
if (!receiver_send_stats_frame(fd, context->config, &context->stats, context->would_delete,
|
||||
context->deleted_paths))
|
||||
return false;
|
||||
|
||||
@@ -29,6 +29,7 @@ PipelineContextReceiver* pipeline_context_receiver_create(Config* config, Queue*
|
||||
context->deferred_manifest = NULL;
|
||||
context->deferred_plans = NULL;
|
||||
context->delete_limit_reached = false;
|
||||
context->failed_entries = 0;
|
||||
memset(&context->stats, 0, sizeof(context->stats));
|
||||
context->would_delete = NULL;
|
||||
context->deleted_paths = NULL;
|
||||
@@ -264,6 +265,13 @@ int write_thread(void* pipeline_context) {
|
||||
receiver_stats_note_saved(&context->stats, file, created, created_dirs);
|
||||
mtx_unlock(&context->mutex);
|
||||
}
|
||||
/* --devices parity: a device node that could not be mknod'ed is counted
|
||||
per-run but does NOT abort the transfer. */
|
||||
if (result == FILE_SAVE_FAILED) {
|
||||
mtx_lock(&context->mutex);
|
||||
context->failed_entries++;
|
||||
mtx_unlock(&context->mutex);
|
||||
}
|
||||
if (result == FILE_SAVE_ERROR) {
|
||||
file_destroy(file);
|
||||
pipeline_context_receiver_note_bytes_released(context, file_bytes);
|
||||
|
||||
@@ -64,6 +64,11 @@ typedef struct PipelineContextReceiver {
|
||||
/* --info=del actually-removed path list, collected by the deferred delete
|
||||
commit in server.c and reported in the STATUS_STATS frame. */
|
||||
struct ArrayList* deleted_paths;
|
||||
/* Per-run count of entries that failed to materialize without aborting the
|
||||
stream (currently ONLY a --devices mknod EPERM/EACCES). write_thread
|
||||
increments it under `mutex`; server.c turns a nonzero count into a non-OK
|
||||
terminal status so the client exits non-zero. */
|
||||
size_t failed_entries;
|
||||
} PipelineContextReceiver;
|
||||
|
||||
PipelineContextReceiver* pipeline_context_receiver_create(Config* config, Queue* queue_receiver,
|
||||
|
||||
+7
-1
@@ -1044,7 +1044,13 @@ static void server_run_mt_receiver(ServerSession* state) {
|
||||
dir_metadata_list_apply(&context->dir_times, config->receive_root_directory, config);
|
||||
}
|
||||
if (transfer_ok) {
|
||||
Status final_status = context->delete_limit_reached ? STATUS_DELETE_LIMIT : STATUS_OK;
|
||||
if (context->failed_entries > 0)
|
||||
log_message(LOG_LEVEL_WARNING,
|
||||
"%zu entr%s failed to materialize; continuing (partial transfer)",
|
||||
context->failed_entries, context->failed_entries == 1 ? "y" : "ies");
|
||||
Status final_status = context->delete_limit_reached
|
||||
? STATUS_DELETE_LIMIT
|
||||
: (context->failed_entries > 0 ? STATUS_ERROR : STATUS_OK);
|
||||
/* Emit the optional wire-stats record first (protocol 2.25.0), then the
|
||||
success/outcome frame, exactly like the single-threaded receiver. */
|
||||
if (!receiver_send_stats_frame(state->fd, config, &context->stats, context->would_delete,
|
||||
|
||||
+6
-4
@@ -735,11 +735,13 @@ typedef struct Config {
|
||||
* --copy-as) imply it. */
|
||||
/* fake_super */
|
||||
/* --fake-super: receiver-only. When set, each written file additionally gets
|
||||
* a reserved user.fastsync.stat xattr recording the RESOLVED uid/gid (the
|
||||
* rsync's reserved user.rsync.%stat xattr recording the RESOLVED uid/gid (the
|
||||
* source's own when no ownership request is active, else the --chown/--usermap
|
||||
* result) plus mode/mtime so a later privileged restore could re-apply them.
|
||||
* It NEVER real-chowns: the point is to record the source ownership on an
|
||||
* unprivileged receiver. Crosses the wire. */
|
||||
* result) plus the full mode and rdev, in rsync 3.4.1's grammar, so the tree is
|
||||
* interoperable and a later privileged restore could re-apply them. mtime is
|
||||
* carried by the file's own timestamp, exactly as rsync does it. It NEVER
|
||||
* real-chowns: the point is to record the source ownership on an unprivileged
|
||||
* receiver. Crosses the wire. */
|
||||
/* module */
|
||||
/* Daemon module selection (Wave A, protocol 2.15.0). Client-composed from a
|
||||
* host::module/path destination; NULL or "" means "no module" (the ordinary
|
||||
|
||||
+216
-1
@@ -1,5 +1,6 @@
|
||||
#include "daemon_conf.h"
|
||||
#include "credentials.h"
|
||||
#include "log.h"
|
||||
#include "utils.h"
|
||||
#include <arpa/inet.h>
|
||||
#include <ctype.h>
|
||||
@@ -33,6 +34,158 @@ static bool key_equals(const char* key, const char* canonical) {
|
||||
return strcasecmp(key, canonical) == 0;
|
||||
}
|
||||
|
||||
/* True when `key` matches one of the NUL-terminated names in `list`. */
|
||||
static bool key_in_list(const char* key, const char* const* list, size_t count) {
|
||||
for (size_t i = 0; i < count; i++) {
|
||||
if (strcasecmp(key, list[i]) == 0)
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/* rsync 3.4.1 rsyncd.conf GLOBAL keys accepted in the pre-module section that
|
||||
* have no FastSync equivalent. They are recognized and documented as inert:
|
||||
* accepting a real rsync config must not fail on a logging/process key, but a
|
||||
* silently-reinterpreted key is never invented. `pidfile`/`logfile` are the
|
||||
* compact --dparam spellings rsync documents. The same list is used by the
|
||||
* `--dparam` dispatch (apply_global_key), so there is a single impl. */
|
||||
static const char* const kRsyncInertGlobalKeys[] = {
|
||||
"pid file",
|
||||
"pidfile",
|
||||
"log file",
|
||||
"logfile",
|
||||
"socket options",
|
||||
"sockopts",
|
||||
"listen backlog",
|
||||
"syslog facility",
|
||||
"syslog tag",
|
||||
"log format",
|
||||
"use chroot",
|
||||
"uid",
|
||||
"gid",
|
||||
"timeout",
|
||||
"max verbosity",
|
||||
"min verbosity",
|
||||
"lock file",
|
||||
"transfer logging",
|
||||
"strict modes",
|
||||
"reverse lookup",
|
||||
"forward lookup",
|
||||
"ignore errors",
|
||||
"ignore nonreadable",
|
||||
"dont compress",
|
||||
};
|
||||
|
||||
/* rsync 3.4.1 rsyncd.conf MODULE keys accepted in a [module] section that have
|
||||
* no FastSync equivalent (accepted-and-documented inert). Keys with a FastSync
|
||||
* meaning (`path`, `read only`, `write only`, `auth users`, `max connections`,
|
||||
* `hosts allow`/`hosts deny`, `client owner`) are handled by apply_module_key
|
||||
* before this list is consulted. Security-relevant keys (`exclude`, `filter`,
|
||||
* `secrets file`, `refuse options`, ...) are inert, so a daemon-side filter or
|
||||
* rsync secrets file is NOT enforced: each is loudly warned about at load time
|
||||
* (see kRsyncUnenforcedModuleSecurityKeys) and documented as a residual in
|
||||
* RSYNC_COMPAT.md. */
|
||||
static const char* const kRsyncInertModuleKeys[] = {
|
||||
"comment",
|
||||
"use chroot",
|
||||
"daemon chroot",
|
||||
"uid",
|
||||
"gid",
|
||||
"daemon uid",
|
||||
"daemon gid",
|
||||
"exclude",
|
||||
"include",
|
||||
"exclude from",
|
||||
"include from",
|
||||
"filter",
|
||||
"max verbosity",
|
||||
"min verbosity",
|
||||
"lock file",
|
||||
"transfer logging",
|
||||
"log file",
|
||||
"log format",
|
||||
"syslog facility",
|
||||
"syslog tag",
|
||||
"timeout",
|
||||
"secrets file",
|
||||
"auth digest",
|
||||
"strict modes",
|
||||
"numeric ids",
|
||||
"fake super",
|
||||
"munge symlinks",
|
||||
"list",
|
||||
"dont compress",
|
||||
"charset",
|
||||
"refuse options",
|
||||
"incoming chmod",
|
||||
"outgoing chmod",
|
||||
"open noatime",
|
||||
"max size",
|
||||
"min size",
|
||||
"temp dir",
|
||||
"pre-xfer exec",
|
||||
"post-xfer exec",
|
||||
"name converter",
|
||||
"proxy protocol",
|
||||
"proxy protocol hosts",
|
||||
"reverse lookup",
|
||||
"forward lookup",
|
||||
"ignore errors",
|
||||
"ignore nonreadable",
|
||||
};
|
||||
|
||||
/* Subset of the inert rsync keys whose intent is access control (data
|
||||
* visibility, credential source, transfer hooks, daemon privilege), plus the
|
||||
* global keys that shape the daemon's privilege/identity. These load for
|
||||
* rsync-config compatibility, but because FastSync ignores them an operator
|
||||
* migrating a hardened rsyncd.conf must not believe the restriction applies.
|
||||
* The loader emits one LOG_LEVEL_WARNING per occurrence naming the key (and the
|
||||
* module, for a module key). `write only` is deliberately absent: it is mapped
|
||||
* onto writability instead (FastSync is push-only, so a write-only module is
|
||||
* simply writable). */
|
||||
static const char* const kRsyncUnenforcedModuleSecurityKeys[] = {
|
||||
"secrets file",
|
||||
"auth digest",
|
||||
"refuse options",
|
||||
"exclude",
|
||||
"include",
|
||||
"exclude from",
|
||||
"include from",
|
||||
"filter",
|
||||
"max size",
|
||||
"min size",
|
||||
"pre-xfer exec",
|
||||
"post-xfer exec",
|
||||
"incoming chmod",
|
||||
"outgoing chmod",
|
||||
"name converter",
|
||||
"use chroot",
|
||||
"daemon chroot",
|
||||
"uid",
|
||||
"gid",
|
||||
"daemon uid",
|
||||
"daemon gid",
|
||||
"munge symlinks",
|
||||
"fake super",
|
||||
"strict modes",
|
||||
"proxy protocol",
|
||||
"proxy protocol hosts",
|
||||
};
|
||||
|
||||
static const char* const kRsyncUnenforcedGlobalSecurityKeys[] = {
|
||||
"use chroot",
|
||||
"uid",
|
||||
"gid",
|
||||
"strict modes",
|
||||
};
|
||||
|
||||
#define kRsyncInertGlobalCount (sizeof(kRsyncInertGlobalKeys) / sizeof(kRsyncInertGlobalKeys[0]))
|
||||
#define kRsyncInertModuleCount (sizeof(kRsyncInertModuleKeys) / sizeof(kRsyncInertModuleKeys[0]))
|
||||
#define kRsyncUnenforcedModuleSecurityCount \
|
||||
(sizeof(kRsyncUnenforcedModuleSecurityKeys) / sizeof(kRsyncUnenforcedModuleSecurityKeys[0]))
|
||||
#define kRsyncUnenforcedGlobalSecurityCount \
|
||||
(sizeof(kRsyncUnenforcedGlobalSecurityKeys) / sizeof(kRsyncUnenforcedGlobalSecurityKeys[0]))
|
||||
|
||||
static bool parse_bool_value(const char* value, bool* out) {
|
||||
if (strcasecmp(value, "yes") == 0 || strcasecmp(value, "true") == 0 || strcmp(value, "1") == 0) {
|
||||
*out = true;
|
||||
@@ -250,6 +403,10 @@ DaemonConf* daemon_conf_create(void) {
|
||||
if (!conf)
|
||||
return NULL;
|
||||
conf->global.port = DAEMON_CONF_DEFAULT_PORT;
|
||||
/* rsync modules are READ-ONLY unless `read only = no` (or `write only = yes`)
|
||||
* is set, so FastSync must default the same way: a migrated rsyncd.conf that
|
||||
* omits `read only` is served read-only, never writable. */
|
||||
conf->global.read_only_default = true;
|
||||
conf->global.max_connections = DAEMON_CONF_DEFAULT_MAX_CONNECTIONS;
|
||||
conf->global.auth_failure_delay_ms = DAEMON_CONF_DEFAULT_AUTH_FAILURE_DELAY_MS;
|
||||
conf->global.max_connections_per_host = DAEMON_CONF_DEFAULT_MAX_CONNECTIONS_PER_HOST;
|
||||
@@ -324,7 +481,7 @@ static bool apply_global_key(DaemonConf* conf, char* key, const char* value, boo
|
||||
char* err, size_t err_size) {
|
||||
if (key_equals(key, "port"))
|
||||
return store_port(&conf->global.port, value, err, err_size);
|
||||
if (key_equals(key, "motd file")) {
|
||||
if (key_equals(key, "motd file") || key_equals(key, "motdfile")) {
|
||||
if (!store_string(&conf->global.motd_file, value)) {
|
||||
set_error(err, err_size, "out of memory parsing 'motd file'");
|
||||
return false;
|
||||
@@ -338,6 +495,25 @@ static bool apply_global_key(DaemonConf* conf, char* key, const char* value, boo
|
||||
}
|
||||
return true;
|
||||
}
|
||||
/* rsync allows the `read only` module key in the global section as the
|
||||
* default for modules defined after it. Map it to that default (a later
|
||||
* --dparam re-applies it to modules that did not set their own value) so a
|
||||
* global `read only = yes` cannot be silently dropped into a writable
|
||||
* default. */
|
||||
if (key_equals(key, "read only")) {
|
||||
bool parsed;
|
||||
if (!parse_bool_value(value, &parsed)) {
|
||||
set_error(err, err_size, "global 'read only' must be yes/no (or true/false/1/0), got '%s'",
|
||||
value);
|
||||
return false;
|
||||
}
|
||||
conf->global.read_only_default = parsed;
|
||||
for (int i = 0; i < conf->module_count; i++) {
|
||||
if (!conf->modules[i].read_only_explicit)
|
||||
conf->modules[i].read_only = parsed;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
if (key_equals(key, "max connections"))
|
||||
return store_max_connections(&conf->global.max_connections, value, NULL, err, err_size);
|
||||
if (key_equals(key, "max connections per host"))
|
||||
@@ -360,6 +536,15 @@ static bool apply_global_key(DaemonConf* conf, char* key, const char* value, boo
|
||||
if (key_equals(key, "hosts deny"))
|
||||
return store_host_list(&conf->global.hosts_deny, &conf->global.hosts_deny_count, value,
|
||||
"hosts deny", NULL, replace_hosts, err, err_size);
|
||||
/* A recognized rsync global key with no FastSync equivalent loads inert. */
|
||||
if (key_in_list(key, kRsyncInertGlobalKeys, kRsyncInertGlobalCount)) {
|
||||
if (key_in_list(key, kRsyncUnenforcedGlobalSecurityKeys, kRsyncUnenforcedGlobalSecurityCount))
|
||||
log_message(LOG_LEVEL_WARNING,
|
||||
"daemon config: global key '%s' is accepted for rsync compatibility but is NOT "
|
||||
"enforced by FastSync; the restriction it expresses will not be applied",
|
||||
key);
|
||||
return true;
|
||||
}
|
||||
set_error(err, err_size, "unknown global key '%s'", key);
|
||||
return false;
|
||||
}
|
||||
@@ -388,6 +573,26 @@ static bool apply_module_key(DaemonModule* module, char* key, char* value, char*
|
||||
return false;
|
||||
}
|
||||
module->read_only = parsed;
|
||||
module->read_only_explicit = true;
|
||||
return true;
|
||||
}
|
||||
/* rsync's `write only = yes` makes the module client-writable. FastSync has
|
||||
* no read/pull path, so mapping it to writability is the exact
|
||||
* security-relevant effect; set `read_only_explicit` so a global default
|
||||
* cannot override the module's explicit choice. `write only = no` is the
|
||||
* rsync default and leaves the module's read-only state untouched. */
|
||||
if (key_equals(key, "write only")) {
|
||||
bool parsed;
|
||||
if (!parse_bool_value(value, &parsed)) {
|
||||
set_error(err, err_size,
|
||||
"module '%s': 'write only' must be yes/no (or true/false/1/0), got '%s'",
|
||||
module->name, value);
|
||||
return false;
|
||||
}
|
||||
if (parsed) {
|
||||
module->read_only = false;
|
||||
module->read_only_explicit = true;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
if (key_equals(key, "client owner")) {
|
||||
@@ -457,6 +662,15 @@ static bool apply_module_key(DaemonModule* module, char* key, char* value, char*
|
||||
if (key_equals(key, "hosts deny"))
|
||||
return store_host_list(&module->hosts_deny, &module->hosts_deny_count, value, "hosts deny",
|
||||
module->name, false, err, err_size);
|
||||
/* A recognized rsync module key with no FastSync equivalent loads inert. */
|
||||
if (key_in_list(key, kRsyncInertModuleKeys, kRsyncInertModuleCount)) {
|
||||
if (key_in_list(key, kRsyncUnenforcedModuleSecurityKeys, kRsyncUnenforcedModuleSecurityCount))
|
||||
log_message(LOG_LEVEL_WARNING,
|
||||
"daemon config: module '%s' key '%s' is accepted for rsync compatibility but is "
|
||||
"NOT enforced by FastSync; the restriction it expresses will not be applied",
|
||||
module->name, key);
|
||||
return true;
|
||||
}
|
||||
set_error(err, err_size, "unknown key '%s' in module '%s'", key, module->name);
|
||||
return false;
|
||||
}
|
||||
@@ -507,6 +721,7 @@ static int open_module(DaemonConf* conf, int* current_module, const char* name,
|
||||
}
|
||||
conf->modules = grown;
|
||||
memset(&conf->modules[conf->module_count], 0, sizeof(DaemonModule));
|
||||
conf->modules[conf->module_count].read_only = conf->global.read_only_default;
|
||||
conf->modules[conf->module_count].name = str_dup(name);
|
||||
if (!conf->modules[conf->module_count].name) {
|
||||
set_error(err, err_size, "out of memory adding module '%s'", name);
|
||||
|
||||
+39
-13
@@ -17,7 +17,20 @@
|
||||
* DAEMON_CONF_MAX_LINE all fail the whole load with a clear, line-numbered
|
||||
* error instead of being silently ignored. This keeps a typo from silently
|
||||
* changing what a module serves.
|
||||
*/
|
||||
*
|
||||
* rsync compatibility: to reduce the divergence from rsync 3.4.1's rsyncd.conf
|
||||
* grammar, the parser also ACCEPTS the common rsync GLOBAL and MODULE keys.
|
||||
* Keys with a FastSync equivalent are mapped onto it (the native spellings are
|
||||
* unchanged; `read only` defaults to yes like rsync, and `write only = yes`
|
||||
* opts a module into writability). Keys with no FastSync equivalent are
|
||||
* accepted and documented as inert (they load successfully but have no effect)
|
||||
* rather than failing the whole config; the accepted inert set is listed in
|
||||
* kRsyncInertGlobalKeys / kRsyncInertModuleKeys in daemon_conf.c and in
|
||||
* RSYNC_COMPAT.md. Every inert key whose intent is access control is loudly
|
||||
* warned about at load time (kRsyncUnenforced*SecurityKeys) so an operator
|
||||
* migrating a hardened rsyncd.conf is never misled into believing the
|
||||
* restriction is enforced. A key outside both the FastSync-native grammar and
|
||||
* the recognized rsync subset is still rejected as unknown. */
|
||||
|
||||
/* A daemon module's configured root is used exactly like the standalone
|
||||
* server's --destination-root: the daemon confines every connection that
|
||||
@@ -42,15 +55,21 @@
|
||||
* store refuses (fail closed) rather than falling open; see server.c. Auth is
|
||||
* never bypassed by ignoring the list. */
|
||||
typedef struct DaemonModule {
|
||||
char* name; /* module name, as the client requests it */
|
||||
char* path; /* module root (daemon-side authorized root) */
|
||||
bool read_only; /* `read only = yes/no`; default no */
|
||||
bool client_owner; /* `client owner = yes/no`; default no. Per-module opt-in
|
||||
that lets this module's clients choose ownership
|
||||
(--numeric-ids/--chown/--usermap/--groupmap/--fake-super/
|
||||
--copy-as) and request explicit --super super-user
|
||||
activities. Without it the daemon refuses all of them. */
|
||||
char** auth_users; /* `auth users = a,b`; Wave B credential list */
|
||||
char* name; /* module name, as the client requests it */
|
||||
char* path; /* module root (daemon-side authorized root) */
|
||||
bool read_only; /* `read only = yes/no`; defaults to the global `read only`
|
||||
default (rsync allows it in the global section), which is
|
||||
itself default YES (rsync modules are read-only unless
|
||||
`read only = no` / `write only = yes` opts in) */
|
||||
bool read_only_explicit; /* set when this module set its own `read only` or
|
||||
`write only = yes`, so a later global default (from a
|
||||
`--dparam read only=`) does not override it */
|
||||
bool client_owner; /* `client owner = yes/no`; default no. Per-module opt-in
|
||||
that lets this module's clients choose ownership
|
||||
(--numeric-ids/--chown/--usermap/--groupmap/--fake-super/
|
||||
--copy-as) and request explicit --super super-user
|
||||
activities. Without it the daemon refuses all of them. */
|
||||
char** auth_users; /* `auth users = a,b`; Wave B credential list */
|
||||
int auth_user_count;
|
||||
/* `max connections = N` (optional per-module cap). 0 means unlimited. The
|
||||
* per-connection child records the selected module in the shared registry
|
||||
@@ -69,6 +88,10 @@ typedef struct DaemonConfGlobals {
|
||||
int port; /* `port`, default DAEMON_CONF_DEFAULT_PORT (873) */
|
||||
char* motd_file; /* `motd file`, may be NULL */
|
||||
char* address; /* `address` (optional bind address), may be NULL */
|
||||
bool read_only_default; /* global `read only` default for modules defined
|
||||
after it (rsync allows the module key in the
|
||||
global section); default YES to match rsync's
|
||||
read-only modules */
|
||||
int max_connections; /* `max connections`, default
|
||||
DAEMON_CONF_DEFAULT_MAX_CONNECTIONS (100) */
|
||||
int auth_failure_delay_ms; /* `auth failure delay`, milliseconds; default
|
||||
@@ -152,10 +175,13 @@ const DaemonModule* daemon_conf_find_module(const DaemonConf* conf, const char*
|
||||
bool daemon_module_name_valid(const char* name);
|
||||
|
||||
/* Parse one --dparam=KEY=VALUE (or "--dparam KEY=VALUE") override string and
|
||||
* apply it to the global keys only. Keys are case-insensitive and limited to
|
||||
* the global keys defined by the grammar (port, motd file, address,
|
||||
* apply it to the global keys only. Keys are case-insensitive and cover the
|
||||
* global keys defined by the grammar (port, motd file, address, read only,
|
||||
* max connections, max connections per host, auth failure delay,
|
||||
* auth lockout threshold, auth lockout duration, hosts allow, hosts deny).
|
||||
* auth lockout threshold, auth lockout duration, hosts allow, hosts deny) plus
|
||||
* the recognized inert rsync global keys and the compact rsync spellings
|
||||
* (`motdfile`, `pidfile`, `logfile`). Applying `read only` sets the global
|
||||
* default and re-applies it to every module that did not set its own value.
|
||||
* Returns 0 on success, -1 on error (err filled). */
|
||||
int daemon_conf_apply_dparam(DaemonConf* conf, const char* assignment, char* err, size_t err_size);
|
||||
|
||||
|
||||
+31
-24
@@ -1182,21 +1182,24 @@ int file_open_temp_dir(const char* dir_path) {
|
||||
* destination file) and best-effort: a per-attribute or privilege failure is
|
||||
* logged and skipped, never fatal. */
|
||||
static void restore_extra_fd(int fd, const FileMetadata* metadata, const FileXattrList* xattrs,
|
||||
bool fake_super, FileAttrPolicy policy) {
|
||||
bool fake_super, FileAttrPolicy policy, uint32_t fake_super_rdev_major,
|
||||
uint32_t fake_super_rdev_minor) {
|
||||
xattr_apply_fd(fd, xattrs);
|
||||
if (fake_super && metadata) {
|
||||
/* Record the ownership that WOULD have been applied: when an explicit
|
||||
ownership request (--chown/--usermap/--groupmap/--copy-as or -o/-g) is
|
||||
active, the resolved mapping; otherwise the source's own id. The real
|
||||
chown is suppressed (identity_apply_ownership early-returns under
|
||||
--fake-super) so recording never defeats the flag. Mode/mtime are still
|
||||
replayed (policy-gated) so unprivileged --fake-super keeps working. */
|
||||
--fake-super) so recording never defeats the flag. The recorded stat is
|
||||
rsync's format; the permission bits are replayed (policy-gated) so
|
||||
unprivileged --fake-super keeps working while mtime comes from the
|
||||
normal metadata path above. */
|
||||
uint32_t store_uid;
|
||||
uint32_t store_gid;
|
||||
identity_resolve_storage_ids((int32_t)metadata->uid, (int32_t)metadata->gid, &store_uid,
|
||||
&store_gid);
|
||||
fake_super_store_fd(fd, store_uid, store_gid, (uint32_t)metadata->mode, metadata->mtime_sec,
|
||||
metadata->mtime_nsec);
|
||||
fake_super_store_fd(fd, store_uid, store_gid, (uint32_t)metadata->mode, fake_super_rdev_major,
|
||||
fake_super_rdev_minor);
|
||||
fake_super_restore_fd(fd, policy);
|
||||
}
|
||||
}
|
||||
@@ -1206,7 +1209,8 @@ file_to_disk_secure_impl(const char* path, const void* data, unsigned long long
|
||||
bool inplace, bool sparse, bool preallocate, const FileMetadata* metadata,
|
||||
FileAttrPolicy policy, bool update, bool no_replace, bool use_fsync,
|
||||
const char* temp_dir, const FileXattrList* xattrs, bool fake_super,
|
||||
bool keep_partial, unsigned* dirs_created, const char* count_floor) {
|
||||
bool keep_partial, unsigned* dirs_created, const char* count_floor,
|
||||
uint32_t fake_super_rdev_major, uint32_t fake_super_rdev_minor) {
|
||||
char* leaf = NULL;
|
||||
int dirfd = file_open_secure_parent_counted(path, &leaf, true, dirs_created, count_floor);
|
||||
if (dirfd < 0)
|
||||
@@ -1313,7 +1317,8 @@ file_to_disk_secure_impl(const char* path, const void* data, unsigned long long
|
||||
}
|
||||
}
|
||||
if (ok)
|
||||
restore_extra_fd(fd, metadata, xattrs, fake_super, policy);
|
||||
restore_extra_fd(fd, metadata, xattrs, fake_super, policy, fake_super_rdev_major,
|
||||
fake_super_rdev_minor);
|
||||
if (ok && use_fsync)
|
||||
ok = fsync(fd) == 0;
|
||||
}
|
||||
@@ -1431,7 +1436,8 @@ file_to_disk_secure_impl(const char* path, const void* data, unsigned long long
|
||||
}
|
||||
}
|
||||
if (ok)
|
||||
restore_extra_fd(fd, metadata, xattrs, fake_super, policy);
|
||||
restore_extra_fd(fd, metadata, xattrs, fake_super, policy, fake_super_rdev_major,
|
||||
fake_super_rdev_minor);
|
||||
if (ok && use_fsync)
|
||||
ok = fsync(fd) == 0;
|
||||
}
|
||||
@@ -1499,7 +1505,8 @@ file_to_disk_secure_impl(const char* path, const void* data, unsigned long long
|
||||
"non-atomic copy into the destination directory");
|
||||
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
|
||||
policy, update, no_replace, use_fsync, NULL, xattrs, fake_super,
|
||||
keep_partial, dirs_created, count_floor);
|
||||
keep_partial, dirs_created, count_floor, fake_super_rdev_major,
|
||||
fake_super_rdev_minor);
|
||||
}
|
||||
return ok;
|
||||
}
|
||||
@@ -1509,7 +1516,7 @@ bool file_to_disk_secure(const char* path, const void* data, unsigned long long
|
||||
FileAttrPolicy policy, const char* temp_dir) {
|
||||
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
|
||||
policy, false, false, false, temp_dir, NULL, false, false, NULL,
|
||||
NULL);
|
||||
NULL, 0, 0);
|
||||
}
|
||||
|
||||
bool file_to_disk_secure_update(const char* path, const void* data, unsigned long long data_size,
|
||||
@@ -1518,7 +1525,7 @@ bool file_to_disk_secure_update(const char* path, const void* data, unsigned lon
|
||||
const char* temp_dir) {
|
||||
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
|
||||
policy, true, false, false, temp_dir, NULL, false, false, NULL,
|
||||
NULL);
|
||||
NULL, 0, 0);
|
||||
}
|
||||
|
||||
bool file_to_disk_secure_with_fsync(const char* path, const void* data,
|
||||
@@ -1527,7 +1534,7 @@ bool file_to_disk_secure_with_fsync(const char* path, const void* data,
|
||||
FileAttrPolicy policy, bool use_fsync, const char* temp_dir) {
|
||||
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
|
||||
policy, false, false, use_fsync, temp_dir, NULL, false, false,
|
||||
NULL, NULL);
|
||||
NULL, NULL, 0, 0);
|
||||
}
|
||||
|
||||
bool file_to_disk_secure_no_replace(const char* path, const void* data,
|
||||
@@ -1536,7 +1543,7 @@ bool file_to_disk_secure_no_replace(const char* path, const void* data,
|
||||
const char* temp_dir) {
|
||||
return file_to_disk_secure_impl(path, data, data_size, false, sparse, preallocate, metadata,
|
||||
policy, false, true, false, temp_dir, NULL, false, false, NULL,
|
||||
NULL);
|
||||
NULL, 0, 0);
|
||||
}
|
||||
|
||||
/* Receiver write-path variant that also applies the per-file xattrs (-X/-A)
|
||||
@@ -1551,19 +1558,19 @@ bool file_to_disk_secure_attrs(const char* path, const void* data, unsigned long
|
||||
bool fake_super, bool keep_partial, const char* temp_dir) {
|
||||
return file_to_disk_secure_attrs_counted(path, data, data_size, inplace, sparse, preallocate,
|
||||
metadata, policy, update, no_replace, use_fsync, xattrs,
|
||||
fake_super, keep_partial, temp_dir, NULL, NULL);
|
||||
fake_super, keep_partial, temp_dir, NULL, NULL, 0, 0);
|
||||
}
|
||||
|
||||
bool file_to_disk_secure_attrs_counted(const char* path, const void* data,
|
||||
unsigned long long data_size, bool inplace, bool sparse,
|
||||
bool preallocate, const FileMetadata* metadata,
|
||||
FileAttrPolicy policy, bool update, bool no_replace,
|
||||
bool use_fsync, const FileXattrList* xattrs, bool fake_super,
|
||||
bool keep_partial, const char* temp_dir,
|
||||
unsigned* dirs_created, const char* count_floor) {
|
||||
bool file_to_disk_secure_attrs_counted(
|
||||
const char* path, const void* data, unsigned long long data_size, bool inplace, bool sparse,
|
||||
bool preallocate, const FileMetadata* metadata, FileAttrPolicy policy, bool update,
|
||||
bool no_replace, bool use_fsync, const FileXattrList* xattrs, bool fake_super,
|
||||
bool keep_partial, const char* temp_dir, unsigned* dirs_created, const char* count_floor,
|
||||
uint32_t fake_super_rdev_major, uint32_t fake_super_rdev_minor) {
|
||||
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
|
||||
policy, update, no_replace, use_fsync, temp_dir, xattrs,
|
||||
fake_super, keep_partial, dirs_created, count_floor);
|
||||
fake_super, keep_partial, dirs_created, count_floor,
|
||||
fake_super_rdev_major, fake_super_rdev_minor);
|
||||
}
|
||||
|
||||
/* Atomic --link-dest install. The destination is replaced (via a temporary
|
||||
@@ -1693,7 +1700,7 @@ static bool file_copy_basis_stream_impl(const char* path, const char* basis_path
|
||||
wrote = false;
|
||||
}
|
||||
if (wrote)
|
||||
restore_extra_fd(fd, metadata, xattrs, fake_super, policy);
|
||||
restore_extra_fd(fd, metadata, xattrs, fake_super, policy, 0, 0);
|
||||
if (wrote && use_fsync)
|
||||
wrote = fsync(fd) == 0;
|
||||
if (close(fd) != 0)
|
||||
@@ -1842,7 +1849,7 @@ static bool file_to_disk_secure_link_impl(const char* path, const char* basis_pa
|
||||
return true;
|
||||
return file_to_disk_secure_attrs_counted(
|
||||
path, data, data_size, false, false, preallocate, metadata, policy, false, false, use_fsync,
|
||||
xattrs, fake_super, false, temp_dir, dirs_created, count_floor);
|
||||
xattrs, fake_super, false, temp_dir, dirs_created, count_floor, 0, 0);
|
||||
}
|
||||
|
||||
if (scratch_dirfd >= 0)
|
||||
|
||||
+6
-7
@@ -182,13 +182,12 @@ bool file_copy_basis_stream_attrs(const char* path, const char* basis_path,
|
||||
* confined secure walk had to create that lie strictly below `count_floor` (a
|
||||
* receive-root-relative prefix, or NULL for all). Used to reproduce rsync's
|
||||
* `Number of created files` directory count on a fresh destination. */
|
||||
bool file_to_disk_secure_attrs_counted(const char* path, const void* data,
|
||||
unsigned long long data_size, bool inplace, bool sparse,
|
||||
bool preallocate, const FileMetadata* metadata,
|
||||
FileAttrPolicy policy, bool update, bool no_replace,
|
||||
bool use_fsync, const FileXattrList* xattrs, bool fake_super,
|
||||
bool keep_partial, const char* temp_dir,
|
||||
unsigned* dirs_created, const char* count_floor);
|
||||
bool file_to_disk_secure_attrs_counted(
|
||||
const char* path, const void* data, unsigned long long data_size, bool inplace, bool sparse,
|
||||
bool preallocate, const FileMetadata* metadata, FileAttrPolicy policy, bool update,
|
||||
bool no_replace, bool use_fsync, const FileXattrList* xattrs, bool fake_super,
|
||||
bool keep_partial, const char* temp_dir, unsigned* dirs_created, const char* count_floor,
|
||||
uint32_t fake_super_rdev_major, uint32_t fake_super_rdev_minor);
|
||||
bool file_to_disk_secure_link_attrs_counted(const char* path, const char* basis_path,
|
||||
const void* data, unsigned long long data_size,
|
||||
bool preallocate, const FileMetadata* metadata,
|
||||
|
||||
+124
-39
@@ -2,6 +2,7 @@
|
||||
#include <ctype.h>
|
||||
#include <dirent.h>
|
||||
#include <fcntl.h>
|
||||
#include <limits.h>
|
||||
#include <libgen.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
@@ -198,6 +199,56 @@ static FileSaveResult hardlink_sibling_absent_first(const char* destination_path
|
||||
return FILE_SAVE_ERROR;
|
||||
}
|
||||
|
||||
/* Resolve a user-supplied --temp-dir against the receive `root`.
|
||||
*
|
||||
* A relative, traversal-free name is joined below the root (the historical
|
||||
* behavior). An absolute path is canonicalized with realpath(3) and accepted
|
||||
* only when it lies inside the canonicalized receive root; this is the parity
|
||||
* win over rejecting every absolute path, without weakening the confinement
|
||||
* invariant: an absolute path that escapes the root (including one reached
|
||||
* through a symlinked component) is still refused. A `..` component in a
|
||||
* relative name is likewise refused. The root itself is treated as an
|
||||
* absolute path free of `..`; its realpath() resolves any symlinks so the
|
||||
* prefix comparison is against one canonical form.
|
||||
*
|
||||
* Logs a clear error on rejection (the scratch dir must stay confined) and
|
||||
* returns a newly allocated scratch path, or NULL on rejection/allocation
|
||||
* failure. */
|
||||
static char* file_save_resolve_temp_dir(const char* root, const char* temp_dir) {
|
||||
if (temp_dir[0] != '/') {
|
||||
if (has_path_traversal(temp_dir)) {
|
||||
log_message(
|
||||
LOG_LEVEL_ERROR,
|
||||
"receiver rejected --temp-dir '%s': a '..' component would escape the receive root",
|
||||
temp_dir);
|
||||
return NULL;
|
||||
}
|
||||
return path_cat(root, temp_dir);
|
||||
}
|
||||
char canonical_temp[PATH_MAX];
|
||||
char canonical_root[PATH_MAX];
|
||||
if (!realpath(temp_dir, canonical_temp)) {
|
||||
log_message(LOG_LEVEL_ERROR,
|
||||
"receiver rejected --temp-dir '%s': could not resolve the absolute path (%s)",
|
||||
temp_dir, strerror(errno));
|
||||
return NULL;
|
||||
}
|
||||
if (!realpath(root, canonical_root)) {
|
||||
log_message(LOG_LEVEL_ERROR,
|
||||
"receiver rejected --temp-dir '%s': could not resolve the receive root (%s)",
|
||||
temp_dir, strerror(errno));
|
||||
return NULL;
|
||||
}
|
||||
if (strcmp(canonical_root, "/") != 0 && !path_is_within_root(canonical_root, canonical_temp)) {
|
||||
log_message(LOG_LEVEL_ERROR,
|
||||
"receiver rejected --temp-dir '%s': an absolute temp dir must be inside the "
|
||||
"receive root '%s'",
|
||||
temp_dir, canonical_root);
|
||||
return NULL;
|
||||
}
|
||||
return str_dup(canonical_temp);
|
||||
}
|
||||
|
||||
/* Install a --hard-links/-H sibling: the destination entry is atomically
|
||||
replaced (temp + rename) with a hard link to the group's first member. The
|
||||
first member is guaranteed already installed at `hardlink_target` under the
|
||||
@@ -303,17 +354,13 @@ static FileSaveResult file_save_hardlink_sibling(const char* root_directory, con
|
||||
free(destination_path);
|
||||
return absent_result;
|
||||
}
|
||||
/* Resolve a relative --temp-dir under the destination root, exactly as the
|
||||
* primary save path does; an absolute or `..`-escaping value is rejected. */
|
||||
/* Resolve the --temp-dir under the destination root, exactly as the primary
|
||||
* save path does: a relative dir joins below the root, an absolute dir is
|
||||
* accepted only when it canonicalizes inside the root, and any escaping value
|
||||
* is rejected. */
|
||||
char* resolved_temp = NULL;
|
||||
if (cfg->temp_dir) {
|
||||
if (cfg->temp_dir[0] == '/' || has_path_traversal(cfg->temp_dir)) {
|
||||
free(content);
|
||||
free(first_disk);
|
||||
free(destination_path);
|
||||
return FILE_SAVE_ERROR;
|
||||
}
|
||||
resolved_temp = path_cat(root_directory, cfg->temp_dir);
|
||||
resolved_temp = file_save_resolve_temp_dir(root_directory, cfg->temp_dir);
|
||||
if (!resolved_temp) {
|
||||
free(content);
|
||||
free(first_disk);
|
||||
@@ -353,11 +400,14 @@ bool file_special_rdev_valid(int32_t major, int32_t minor, mode_t mode) {
|
||||
/* ---- Device/special node RECREATION (--devices/--specials), receiver side ----
|
||||
*
|
||||
* Privilege gating: making a real device node requires CAP_MKNOD (root); making
|
||||
* a FIFO works unprivileged (mkfifo). When the receiver lacks the capability,
|
||||
* mknodat() fails with EPERM and the entry is SKIPPED with a warning -- the
|
||||
* whole transfer must NOT abort just because the environment cannot make the
|
||||
* node. CI runs non-root, so device creation is expected to skip there and
|
||||
* only a FIFO is honestly assertable unprivileged.
|
||||
* a FIFO works unprivileged (mkfifo). A device node whose mknodat() fails with
|
||||
* EPERM/EACCES is a PER-ENTRY failure (rsync parity: rsync reports the mknod
|
||||
* failure, still transfers the rest, and exits partial, code 23), reported as
|
||||
* FILE_SAVE_FAILED so the receiver counts it and continues. Only the
|
||||
* unprivileged FIFO/socket (--specials) path keeps the best-effort skip,
|
||||
* because those are normally creatable without privilege and a failure there is
|
||||
* environmental. CI runs non-root, so device creation is expected to fail
|
||||
* there; only a FIFO is honestly assertable unprivileged.
|
||||
*
|
||||
* Confinement: the parent directory is opened fd-relative below the receive
|
||||
* root (file_open_secure_parent: O_NOFOLLOW, no "..", root-checked) and the
|
||||
@@ -495,13 +545,32 @@ static FileSaveResult file_save_special_to_disk(const char* root_directory, cons
|
||||
node_kind, escaped_path ? escaped_path : "<allocation failed>");
|
||||
free(escaped_path);
|
||||
} else if (errno == EPERM || errno == EACCES) {
|
||||
/* Missing CAP_MKNOD / parent write permission: the environment cannot
|
||||
create the node, so skip instead of failing the whole run. */
|
||||
char* escaped_path = output_escape(file->path, log_get_8_bit_output());
|
||||
const char* shown_path = escaped_path ? escaped_path : "<allocation failed>";
|
||||
if (is_char || is_blk) {
|
||||
/* rsync parity: a device node that cannot be created (no CAP_MKNOD, or
|
||||
* super-user activities not permitted) is a per-entry failure. rsync
|
||||
* logs `mknod ".../node" failed: ...`, still transfers the remaining
|
||||
* files, and exits partial (23); FastSync logs it, counts it, and
|
||||
* continues rather than aborting the stream. FIFO/socket creation
|
||||
* (--specials) keeps the best-effort skip path below. */
|
||||
log_message(LOG_LEVEL_ERROR,
|
||||
"cannot create %s %s: %s\n"
|
||||
" --devices node creation needs privilege (CAP_MKNOD)",
|
||||
node_kind, shown_path, strerror(errno));
|
||||
free(escaped_path);
|
||||
close(parent_fd);
|
||||
free(leaf);
|
||||
free(destination);
|
||||
return FILE_SAVE_FAILED;
|
||||
}
|
||||
/* Missing CAP_MKNOD / parent write permission for a FIFO/socket: the
|
||||
environment cannot create the node, so skip instead of failing the
|
||||
whole run. */
|
||||
log_message(LOG_LEVEL_WARNING,
|
||||
"skipping %s: cannot create %s node (%s)\n"
|
||||
" --devices/--specials node creation needs privilege (CAP_MKNOD)",
|
||||
escaped_path ? escaped_path : "<allocation failed>", node_kind, strerror(errno));
|
||||
" --specials node creation needs privilege (CAP_MKNOD)",
|
||||
shown_path, node_kind, strerror(errno));
|
||||
free(escaped_path);
|
||||
} else {
|
||||
char* escaped_path = output_escape(file->path, log_get_8_bit_output());
|
||||
@@ -868,6 +937,14 @@ static bool file_save_try_special_dispatch(const FileSavePlan* plan, bool* creat
|
||||
/* Device/special node (--devices/--specials): recreate the node instead of
|
||||
writing content (privilege-gated, confined, rdev-validated). */
|
||||
if (file->is_special) {
|
||||
/* Under --fake-super rsync never mknod()s a device: it writes a regular
|
||||
empty file and records the real rdev in user.rsync.%stat. Fall through to
|
||||
the ordinary writer so the device round-trips (its S_IFMT mode bits and
|
||||
rdev are parked in the record). Without --fake-super the node is
|
||||
recreated (or, when privilege is refused, handled per-entry). */
|
||||
mode_t special_mode = file->metadata ? file->metadata->mode : 0;
|
||||
if (config && config->fake_super && (S_ISCHR(special_mode) || S_ISBLK(special_mode)))
|
||||
return false;
|
||||
*out = file_save_special_to_disk(plan->root_directory, file, config, created);
|
||||
return true;
|
||||
}
|
||||
@@ -896,17 +973,17 @@ static bool file_save_try_special_dispatch(const FileSavePlan* plan, bool* creat
|
||||
and disk paths. Returns false on an invalid/escaping option or an
|
||||
allocation failure (the caller routes to the cleanup epilogue). */
|
||||
static bool file_save_resolve_paths(FileSavePlan* plan) {
|
||||
/* These options arrive from the client. --backup-dir, --partial-dir and
|
||||
--temp-dir are names below the server root, never independent filesystem
|
||||
roots: an absolute or `..`-escaping value is rejected outright (rsync's
|
||||
daemon confines temp-dir to the module the same way). A relative temp dir
|
||||
is resolved under the receive root below; if that resolution still lands on
|
||||
a different filesystem than the destination the install falls back to a
|
||||
non-atomic copy (see file_to_disk_secure_impl), never an abort. */
|
||||
/* These options arrive from the client. --backup-dir and --partial-dir are
|
||||
names below the server root, never independent filesystem roots: an
|
||||
absolute or `..`-escaping value is rejected outright. --temp-dir is
|
||||
resolved by file_save_resolve_temp_dir below: a relative name joins below
|
||||
the root, an absolute name is accepted only when it canonicalizes inside
|
||||
the root, and any escaping value is rejected. If the resolved scratch dir
|
||||
still lands on a different filesystem than the destination the install
|
||||
falls back to a non-atomic copy (see file_to_disk_secure_impl), never an
|
||||
abort. */
|
||||
if ((plan->backup_dir && (plan->backup_dir[0] == '/' || has_path_traversal(plan->backup_dir))) ||
|
||||
(plan->partial_dir &&
|
||||
(plan->partial_dir[0] == '/' || has_path_traversal(plan->partial_dir))) ||
|
||||
(plan->temp_dir && (plan->temp_dir[0] == '/' || has_path_traversal(plan->temp_dir))))
|
||||
(plan->partial_dir && (plan->partial_dir[0] == '/' || has_path_traversal(plan->partial_dir))))
|
||||
return false;
|
||||
if (plan->backup_dir &&
|
||||
!(plan->confined_backup = path_cat(plan->root_directory, plan->backup_dir)))
|
||||
@@ -914,6 +991,9 @@ static bool file_save_resolve_paths(FileSavePlan* plan) {
|
||||
if (plan->partial_dir &&
|
||||
!(plan->confined_partial = path_cat(plan->root_directory, plan->partial_dir)))
|
||||
return false;
|
||||
if (plan->temp_dir &&
|
||||
!(plan->confined_temp = file_save_resolve_temp_dir(plan->root_directory, plan->temp_dir)))
|
||||
return false;
|
||||
|
||||
const char* actual_root = plan->use_partial_root ? plan->confined_partial : plan->root_directory;
|
||||
plan->destination_path = path_cat(plan->root_directory, plan->file->path);
|
||||
@@ -1039,7 +1119,7 @@ static bool file_save_install_data(FileSavePlan* plan, const FileMetadata* metad
|
||||
config && config->preallocate, metadata, plan->policy, config && config->update,
|
||||
config && config->ignore_existing, config && config->use_fsync, file->xattrs,
|
||||
config ? config->fake_super : false, config ? config->partial : false, plan->confined_temp,
|
||||
created_dirs, count_floor);
|
||||
created_dirs, count_floor, (uint32_t)file->rdev_major, (uint32_t)file->rdev_minor);
|
||||
}
|
||||
free(count_floor);
|
||||
return ok;
|
||||
@@ -1146,17 +1226,22 @@ FileSaveResult file_save_to_disk_full_ex(const char* root_directory, const File*
|
||||
|
||||
/* A configured --temp-dir sends the temporary working copy to a scratch
|
||||
directory; the engine then atomically renames the completed file into the
|
||||
final destination directory. A relative temp dir is resolved under the
|
||||
receive root and must already exist (an absolute or `..`-escaping value was
|
||||
rejected above); the engine falls back to a non-atomic copy on EXDEV. The
|
||||
partial-dir flow already keeps its working copy in a separate directory and
|
||||
--inplace writes directly, so neither diverts through the scratch dir
|
||||
(matching rsync, where --inplace/--partial-dir supersede --temp-dir). */
|
||||
bool use_temp_dir = plan.temp_dir != NULL && !plan.inplace && !plan.use_partial_root;
|
||||
final destination directory. The scratch path was confined to the receive
|
||||
root (and canonicalized) in file_save_resolve_paths and must already exist;
|
||||
the engine falls back to a non-atomic copy on EXDEV. The partial-dir flow
|
||||
already keeps its working copy in a separate directory and --inplace writes
|
||||
directly, so neither diverts through the scratch dir (matching rsync, where
|
||||
--inplace/--partial-dir supersede --temp-dir). */
|
||||
/* --inplace and --partial-dir supersede --temp-dir in rsync, so the scratch
|
||||
dir is not used on those paths. The value was still validated/confined by
|
||||
file_save_resolve_paths; drop the resolved path so it is never handed to the
|
||||
install engine. */
|
||||
if (plan.confined_temp && (plan.inplace || plan.use_partial_root)) {
|
||||
free(plan.confined_temp);
|
||||
plan.confined_temp = NULL;
|
||||
}
|
||||
bool use_temp_dir = plan.confined_temp != NULL;
|
||||
if (use_temp_dir) {
|
||||
plan.confined_temp = path_cat(root_directory, plan.temp_dir);
|
||||
if (!plan.confined_temp)
|
||||
goto out;
|
||||
/* A user-supplied trailing slash would leave the scratch path ending in
|
||||
"/", which has no final component to create/open. Normalize it away. */
|
||||
size_t temp_len = strlen(plan.confined_temp);
|
||||
|
||||
+10
-2
@@ -12,8 +12,16 @@
|
||||
|
||||
/* Outcome of a single file_save_to_disk operation. The receiver needs to
|
||||
distinguish "written" from "skipped" so --remove-source-files can be told
|
||||
which sources were actually stored. */
|
||||
typedef enum { FILE_SAVE_ERROR = 0, FILE_SAVE_WRITTEN = 1, FILE_SAVE_SKIPPED = 2 } FileSaveResult;
|
||||
which sources were actually stored. FILE_SAVE_FAILED is a per-entry failure
|
||||
(for example a device node that mknodat() refused with EPERM/EACCES): it is
|
||||
logged and counted by the receiver but does NOT abort the transfer, matching
|
||||
rsync's continue-and-exit-partial behavior. */
|
||||
typedef enum {
|
||||
FILE_SAVE_ERROR = 0,
|
||||
FILE_SAVE_WRITTEN = 1,
|
||||
FILE_SAVE_SKIPPED = 2,
|
||||
FILE_SAVE_FAILED = 3
|
||||
} FileSaveResult;
|
||||
|
||||
bool file_special_rdev_valid(int32_t major, int32_t minor, mode_t mode);
|
||||
|
||||
|
||||
@@ -37,6 +37,7 @@ PipelineContextSender* pipeline_context_sender_create(Config* config, Queue* que
|
||||
context->scan_had_io_error = false;
|
||||
context->remove_source_files = NULL;
|
||||
context->early_delete = false;
|
||||
context->prescan_chunks = NULL;
|
||||
context->delete_plans = NULL;
|
||||
context->delete_suppressed = false;
|
||||
context->scan_stopped_early = false;
|
||||
@@ -194,6 +195,8 @@ void pipeline_context_sender_destroy(PipelineContextSender* context) {
|
||||
if (context->manifest) {
|
||||
array_list_delete(context->manifest);
|
||||
}
|
||||
if (context->prescan_chunks)
|
||||
array_list_delete(context->prescan_chunks);
|
||||
if (context->delete_plans)
|
||||
delete_plan_sender_destroy(context->delete_plans);
|
||||
if (context->excluded_paths)
|
||||
|
||||
@@ -78,6 +78,13 @@ typedef struct {
|
||||
path-only pre-scan on the calling thread and the pipeline scanner must not
|
||||
append to it. Set once before the worker threads start. */
|
||||
bool early_delete;
|
||||
/* --delete-before: the path-only pre-scan that built the early keep-set,
|
||||
retained as the pipeline's file list (owning Chunk*; consumed and NULLed by
|
||||
the scanner thread) so the data pass replays rsync's single file list
|
||||
instead of re-reading the source. NULL in every other mode, where the
|
||||
scanner thread scans normally. Set once before the worker threads start
|
||||
and freed with the context. */
|
||||
ArrayList* prescan_chunks;
|
||||
/* Non-NULL for --delete-during/--delete-delay: the per-directory plan set
|
||||
prebuilt by the path-only pre-scan on the calling thread. The sender
|
||||
thread transmits the root plan before any data and the remaining plans
|
||||
|
||||
+86
-33
@@ -7,6 +7,7 @@
|
||||
#include "utils.h"
|
||||
#include "file_types.h"
|
||||
#include <errno.h>
|
||||
#include <limits.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
@@ -363,16 +364,21 @@ bool xattr_apply_fd(int fd, const FileXattrList* list) {
|
||||
return true;
|
||||
}
|
||||
|
||||
/* ---- --fake-super: park ownership/mode/mtime in a reserved xattr ---- */
|
||||
/* ---- --fake-super: park ownership/mode/rdev in a reserved xattr ---- */
|
||||
|
||||
void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, int64_t mtime_sec,
|
||||
int64_t mtime_nsec) {
|
||||
void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, uint32_t rdev_major,
|
||||
uint32_t rdev_minor) {
|
||||
if (fd < 0)
|
||||
return;
|
||||
char record[128];
|
||||
int len =
|
||||
snprintf(record, sizeof(record), "%lu:%lu:%03o:%lld:%ld", (unsigned long)uid,
|
||||
(unsigned long)gid, (unsigned)mode & 0777U, (long long)mtime_sec, (long)mtime_nsec);
|
||||
/* rsync 3.4.1's exact grammar: "<octal full st_mode> <rdev_major>,<rdev_minor>
|
||||
* <uid>:<gid>". The octal mode carries the S_IFMT bits (e.g. 0104711 for a
|
||||
* setuid regular file, 020644 for a char device); the rdev pair is 0,0 for a
|
||||
* non-device. No mtime field: rsync leaves the file's own timestamp in
|
||||
* charge of mtime. This value is what rsync reads back to restore a
|
||||
* fake-super tree, so the field order and separators must not change. */
|
||||
char record[96];
|
||||
int len = snprintf(record, sizeof(record), "%o %u,%u %u:%u", (unsigned)mode, (unsigned)rdev_major,
|
||||
(unsigned)rdev_minor, (unsigned)uid, (unsigned)gid);
|
||||
if (len <= 0 || (size_t)len >= sizeof(record))
|
||||
return;
|
||||
if (fsetxattr(fd, FAKESUPER_XATTR, record, (size_t)len, 0) != 0) {
|
||||
@@ -381,11 +387,64 @@ void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, int6
|
||||
}
|
||||
}
|
||||
|
||||
/* --fake-super replay: read the freshly-stored record and re-apply mode/mtime
|
||||
* fd-relative. The recorded uid/gid are retained for a later privileged
|
||||
* restore but are NEVER chowned here: --fake-super only RECORDS ownership, it
|
||||
* must not real-chown the recorded (resolved) owner. Mode/mtime still apply so
|
||||
* unprivileged --fake-super keeps working. */
|
||||
/* Parse rsync's `user.rsync.%stat` grammar strictly:
|
||||
* "<octal st_mode> <rdev_major>,<rdev_minor> <uid>:<gid>"
|
||||
* Every field is parsed with strtoul() so an out-of-range value is a clean
|
||||
* rejection rather than the undefined behavior sscanf("%u") exhibited, each
|
||||
* field is range-checked against the same bounds the wire validator uses, and
|
||||
* the whole record must be consumed (only trailing whitespace is tolerated) so
|
||||
* trailing garbage is refused. Returns false on any malformed input. */
|
||||
static bool fake_super_parse_stat(const char* record, unsigned* mode_out, unsigned* rdev_major_out,
|
||||
unsigned* rdev_minor_out, unsigned* uid_out, unsigned* gid_out) {
|
||||
if (!record)
|
||||
return false;
|
||||
char* end = NULL;
|
||||
const char* p = record;
|
||||
errno = 0;
|
||||
unsigned long mode = strtoul(p, &end, 8);
|
||||
if (errno != 0 || end == p || mode > (unsigned long)UINT_MAX || *end != ' ')
|
||||
return false;
|
||||
p = end + 1;
|
||||
errno = 0;
|
||||
unsigned long rdev_major = strtoul(p, &end, 10);
|
||||
if (errno != 0 || end == p || rdev_major > 0xffffUL || *end != ',')
|
||||
return false;
|
||||
p = end + 1;
|
||||
errno = 0;
|
||||
unsigned long rdev_minor = strtoul(p, &end, 10);
|
||||
if (errno != 0 || end == p || rdev_minor > 0x00ffffffUL || *end != ' ')
|
||||
return false;
|
||||
p = end + 1;
|
||||
errno = 0;
|
||||
unsigned long uid = strtoul(p, &end, 10);
|
||||
if (errno != 0 || end == p || uid > (unsigned long)UINT_MAX || *end != ':')
|
||||
return false;
|
||||
p = end + 1;
|
||||
errno = 0;
|
||||
unsigned long gid = strtoul(p, &end, 10);
|
||||
if (errno != 0 || end == p || gid > (unsigned long)UINT_MAX)
|
||||
return false;
|
||||
p = end;
|
||||
while (*p == ' ' || *p == '\t' || *p == '\n' || *p == '\r')
|
||||
p++;
|
||||
if (*p != '\0')
|
||||
return false;
|
||||
*mode_out = (unsigned)mode;
|
||||
*rdev_major_out = (unsigned)rdev_major;
|
||||
*rdev_minor_out = (unsigned)rdev_minor;
|
||||
*uid_out = (unsigned)uid;
|
||||
*gid_out = (unsigned)gid;
|
||||
return true;
|
||||
}
|
||||
|
||||
/* --fake-super replay: read the freshly-stored record and re-apply its
|
||||
* permission bits fd-relative. The recorded uid/gid are retained for a later
|
||||
* privileged restore but are NEVER chowned here: --fake-super only RECORDS
|
||||
* ownership, it must not real-chown the recorded (resolved) owner. The
|
||||
* recorded rdev is likewise parsed for grammar compatibility but is not acted
|
||||
* on (device recreation is a separate, privilege-gated path). mtime is not in
|
||||
* the record: the normal metadata path applies it (policy.times), exactly as
|
||||
* rsync relies on the file's own timestamp. */
|
||||
bool fake_super_restore_fd(int fd, FileAttrPolicy policy) {
|
||||
if (fd < 0)
|
||||
return false;
|
||||
@@ -394,24 +453,25 @@ bool fake_super_restore_fd(int fd, FileAttrPolicy policy) {
|
||||
if (len < 0)
|
||||
return false; /* absent or filesystem without xattrs: silent no-op */
|
||||
record[len] = '\0';
|
||||
unsigned long ul_uid, ul_gid, ul_mode;
|
||||
long long mtime_sec;
|
||||
long mtime_nsec;
|
||||
if (sscanf(record, "%lu:%lu:%lo:%lld:%ld", &ul_uid, &ul_gid, &ul_mode, &mtime_sec, &mtime_nsec) !=
|
||||
5)
|
||||
unsigned ul_mode, rdev_major, rdev_minor, ul_uid, ul_gid;
|
||||
if (!fake_super_parse_stat(record, &ul_mode, &rdev_major, &rdev_minor, &ul_uid, &ul_gid))
|
||||
return false; /* malformed record: skip, never fatal */
|
||||
|
||||
/* --fake-super NEVER performs a real chown: that would defeat the whole
|
||||
point of the flag (record privileged ownership on an unprivileged receiver
|
||||
for a later privileged restore). The uid/gid parsed above are retained in
|
||||
the record for that later restore, but no ownership change happens here. */
|
||||
/* --fake-super NEVER performs a real chown: that would defeat the whole point
|
||||
of the flag (record privileged ownership on an unprivileged receiver for a
|
||||
later privileged restore). The uid/gid parsed above are retained in the
|
||||
record for that later restore, but no ownership change happens here. The
|
||||
rdev is retained for the same reason. */
|
||||
(void)rdev_major;
|
||||
(void)rdev_minor;
|
||||
(void)ul_uid;
|
||||
(void)ul_gid;
|
||||
/* Mode is applied only when the per-attribute policy asks for it, through the
|
||||
SAME shared helper the normal metadata path uses (metadata_mode_for_policy):
|
||||
under --perms the recorded source mode is copied exactly, including
|
||||
group/other write and setuid/setgid/sticky bits (rsync parity), and the -E
|
||||
rule derives exec bits from the destination's read bits exactly like
|
||||
SAME shared helper the normal metadata path uses (metadata_mode_for_policy).
|
||||
The recorded special bits are stripped first: rsync's fake-super receiver
|
||||
stores the full mode in the xattr but never installs setuid/setgid/sticky on
|
||||
the real file, so only the 0777 permission bits may be replayed. The -E
|
||||
rule then derives exec bits from the destination's read bits exactly like
|
||||
file_restore_metadata_fd. */
|
||||
if (policy.perms || policy.executability) {
|
||||
struct stat cur;
|
||||
@@ -419,19 +479,12 @@ bool fake_super_restore_fd(int fd, FileAttrPolicy policy) {
|
||||
if (fstat(fd, &cur) != 0) {
|
||||
log_message(LOG_LEVEL_WARNING, "--fake-super: could not read destination mode: %s",
|
||||
strerror(errno));
|
||||
} else if (metadata_mode_for_policy((mode_t)ul_mode, cur.st_mode, policy, &want)) {
|
||||
} else if (metadata_mode_for_policy((mode_t)(ul_mode & 0777U), cur.st_mode, policy, &want)) {
|
||||
if (fchmod(fd, want) != 0)
|
||||
log_message(LOG_LEVEL_WARNING,
|
||||
"--fake-super: could not restore mode on destination file: %s",
|
||||
strerror(errno));
|
||||
}
|
||||
}
|
||||
if (policy.times) {
|
||||
struct timespec times[2] = {{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
|
||||
{.tv_sec = (time_t)mtime_sec, .tv_nsec = mtime_nsec}};
|
||||
if (futimens(fd, times) != 0)
|
||||
log_message(LOG_LEVEL_WARNING,
|
||||
"--fake-super: could not restore mtime on destination file: %s", strerror(errno));
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
+28
-21
@@ -31,11 +31,14 @@
|
||||
*/
|
||||
|
||||
/* Reserved key used by --fake-super to park the source's privileged ownership
|
||||
* / mode / mtime on the destination file as an unprivileged user.* xattr, so a
|
||||
* later privileged restore could re-apply them. Exact documented format:
|
||||
* uid:gid:mode:mtime_sec:mtime_nsec (decimal, decimal, octal, dec, dec)
|
||||
* e.g. "1000:1000:644:1765238400:0". */
|
||||
#define FAKESUPER_XATTR "user.fastsync.stat"
|
||||
* / mode / rdev on the destination file as an unprivileged user.* xattr, so the
|
||||
* tree is interoperable with rsync 3.4.1 and a later privileged restore can
|
||||
* re-apply them. This is rsync's own key and value grammar exactly:
|
||||
* <octal st_mode with S_IFMT> <rdev_major>,<rdev_minor> <uid>:<gid>
|
||||
* e.g. "104711 0,0 1234:5678" for a setuid regular file owned by 1234:5678,
|
||||
* or "20644 1,3 111:222" for a char device. mtime is deliberately NOT part of
|
||||
* the record: exactly like rsync, the file's own timestamp carries it. */
|
||||
#define FAKESUPER_XATTR "user.rsync.%stat"
|
||||
|
||||
/* --- bounds --- */
|
||||
#define XATTR_NAME_MAX 255 /* xattr names are limited to 255 bytes */
|
||||
@@ -91,24 +94,28 @@ FileXattrList* xattr_receive(int fd, int* ok, bool preserve_acls);
|
||||
* true when apply was attempted (allowing callers to treat it as best-effort). */
|
||||
bool xattr_apply_fd(int fd, const FileXattrList* list);
|
||||
|
||||
/* --fake-super: write the source uid/gid/mode/mtime record into the reserved
|
||||
* FAKESUPER_XATTR on `fd`. Best-effort (logged, never fatal). Only meaningful
|
||||
* when metadata was transmitted so the values exist. */
|
||||
void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, int64_t mtime_sec,
|
||||
int64_t mtime_nsec);
|
||||
/* --fake-super: write the source uid/gid/mode/rdev record into the reserved
|
||||
* FAKESUPER_XATTR on `fd`, using rsync 3.4.1's exact grammar (see the key
|
||||
* comment above). `mode` is the full st_mode including its S_IFMT bits.
|
||||
* Best-effort (logged, never fatal). Only meaningful when metadata was
|
||||
* transmitted so the values exist. */
|
||||
void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, uint32_t rdev_major,
|
||||
uint32_t rdev_minor);
|
||||
|
||||
/* --fake-super replay: parse the FAKESUPER_XATTR record previously written on
|
||||
* `fd` by fake_super_store_fd and re-apply mode/mtime fd-relative. The
|
||||
* recorded uid/gid are deliberately NOT chowned for real: --fake-super only
|
||||
* RECORDS ownership (the caller stores the resolved mapping via
|
||||
* identity_resolve_storage_ids), it never performs a real chown. Best-effort:
|
||||
* absence of the xattr or a malformed record is a silent no-op that never fails
|
||||
* the transfer. The MODE leg is applied only when policy.perms||policy.
|
||||
* executability and the MTIME leg only when policy.times, so the fake-super
|
||||
* replay cannot bypass the per-attribute split; the mode follows the normal
|
||||
* metadata path exactly (under --perms the source mode is copied verbatim,
|
||||
* special and group/other write bits included).
|
||||
* Returns true when the xattr was present and parsed. */
|
||||
* `fd` by fake_super_store_fd and re-apply the recorded permission bits
|
||||
* fd-relative. The recorded uid/gid are deliberately NOT chowned for real:
|
||||
* --fake-super only RECORDS ownership (the caller stores the resolved mapping
|
||||
* via identity_resolve_storage_ids), it never performs a real chown. The
|
||||
* recorded rdev is retained for a later privileged restore but is not acted on
|
||||
* here. Best-effort: absence of the xattr or a malformed record is a silent
|
||||
* no-op that never fails the transfer. The MODE leg is applied only when
|
||||
* policy.perms||policy.executability, and the recorded special bits
|
||||
* (setuid/setgid/sticky) are NOT applied to the real file -- exactly like
|
||||
* rsync's fake-super receiver, which stores the full mode in the xattr but
|
||||
* strips the special bits on disk. mtime is not part of the record; the normal
|
||||
* metadata path carries it (policy.times) exactly as rsync sets the file's own
|
||||
* timestamp. Returns true when the xattr was present and parsed. */
|
||||
bool fake_super_restore_fd(int fd, FileAttrPolicy policy);
|
||||
|
||||
#endif
|
||||
@@ -141,6 +141,7 @@ class DaemonManager:
|
||||
def __init__(self):
|
||||
self._proc = None
|
||||
self._port = None
|
||||
self.log_path = None
|
||||
|
||||
def start(self, config_path, port_override=None, extra_args=None, log_path=None):
|
||||
self.stop()
|
||||
@@ -154,7 +155,11 @@ class DaemonManager:
|
||||
if extra_args:
|
||||
cmd += extra_args
|
||||
if log_path is None:
|
||||
log_path = os.path.join(TEST_DATA_DIR, "fastsyncd.log")
|
||||
# A unique log per manager: several managers run in one xdist
|
||||
# worker, and a shared log lets one daemon's truncate/write offset
|
||||
# corrupt the other's appended lines (a flaky log assertion).
|
||||
log_path = os.path.join(TEST_DATA_DIR, f"fastsyncd_{id(self):x}.log")
|
||||
self.log_path = log_path
|
||||
log = open(log_path, "w")
|
||||
self._proc = subprocess.Popen(
|
||||
cmd, stdout=log, stderr=log, stdin=subprocess.DEVNULL, start_new_session=True)
|
||||
@@ -224,6 +229,7 @@ def daemon_env():
|
||||
"\n"
|
||||
"[files]\n"
|
||||
"path = %s\n"
|
||||
"read only = no\n"
|
||||
"\n"
|
||||
"[readonly]\n"
|
||||
"path = %s\n"
|
||||
@@ -231,18 +237,22 @@ def daemon_env():
|
||||
"\n"
|
||||
"[locked]\n"
|
||||
"path = %s\n"
|
||||
"read only = no\n"
|
||||
"auth users = alice\n"
|
||||
"\n"
|
||||
"[team]\n"
|
||||
"path = %s\n"
|
||||
"read only = no\n"
|
||||
"auth users = alice,bob\n"
|
||||
"\n"
|
||||
"[owner]\n"
|
||||
"path = %s\n"
|
||||
"read only = no\n"
|
||||
"client owner = yes\n"
|
||||
"\n"
|
||||
"[denied]\n"
|
||||
"path = %s\n"
|
||||
"read only = no\n"
|
||||
"hosts deny = 127.0.0.1\n"
|
||||
% (config_port, FILES_MODULE, READONLY_MODULE, AUTH_MODULE, TEAM_MODULE, OWNER_MODULE,
|
||||
DENIED_MODULE))
|
||||
@@ -261,7 +271,8 @@ def daemon_env():
|
||||
global DETACH_PORT
|
||||
DETACH_PORT = _find_free_port()
|
||||
with open(DETACH_CONF, "w") as f:
|
||||
f.write("port = %d\n\n[detach]\npath = %s\n" % (DETACH_PORT, DETACH_MODULE))
|
||||
f.write("port = %d\n\n[detach]\npath = %s\nread only = no\n"
|
||||
% (DETACH_PORT, DETACH_MODULE))
|
||||
|
||||
yield
|
||||
_kill_by_cmdline_marker(DETACH_CONF)
|
||||
@@ -348,7 +359,8 @@ class TestDaemonModuleSelection:
|
||||
the fix regresses."""
|
||||
port = _find_free_port()
|
||||
with open(UMASK_CONF, "w") as f:
|
||||
f.write("port = %d\n\n[files]\npath = %s\n" % (port, FILES_MODULE))
|
||||
f.write("port = %d\n\n[files]\npath = %s\nread only = no\n"
|
||||
% (port, FILES_MODULE))
|
||||
sub = os.path.join(FILES_MODULE, "umask_check")
|
||||
shutil.rmtree(sub, ignore_errors=True)
|
||||
os.makedirs(sub, exist_ok=True)
|
||||
@@ -375,6 +387,58 @@ class TestDaemonModuleSelection:
|
||||
proc.kill()
|
||||
|
||||
|
||||
class TestRsyncConfigCompat:
|
||||
"""A real rsyncd.conf can be pointed at FastSync: the common rsync GLOBAL
|
||||
and MODULE keys are accepted, the ones with a FastSync equivalent (port,
|
||||
path, read only, max connections) take effect, and the inert ones (pid
|
||||
file, log file, comment, use chroot, uid, gid, exclude, timeout, ...) are
|
||||
documented no-ops. --dparam accepts the same expanded key set."""
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_rsync_style_config_round_trip(self):
|
||||
module = os.path.join(MODULE_ROOT, "rsync_style")
|
||||
shutil.rmtree(module, ignore_errors=True)
|
||||
os.makedirs(module, exist_ok=True)
|
||||
port = _find_free_port()
|
||||
conf = os.path.join(TEST_DATA_DIR, "fastsyncd_rsync_style.conf")
|
||||
with open(conf, "w") as f:
|
||||
f.write(
|
||||
"# an rsync 3.4.1-style rsyncd.conf\n"
|
||||
"pid file = /tmp/fastsyncd_rsync_style.pid\n"
|
||||
"log file = /tmp/fastsyncd_rsync_style.log\n"
|
||||
"socket options = TCP_NODELAY\n"
|
||||
"use chroot = no\n"
|
||||
"uid = nobody\n"
|
||||
"gid = nogroup\n"
|
||||
"timeout = 600\n"
|
||||
"max verbosity = 2\n"
|
||||
"transfer logging = yes\n"
|
||||
"port = %d\n"
|
||||
"\n"
|
||||
"[rsync_style]\n"
|
||||
"path = %s\n"
|
||||
"comment = rsync-style module\n"
|
||||
"use chroot = no\n"
|
||||
"exclude = *.tmp\n"
|
||||
"read only = no\n"
|
||||
"max connections = 4\n"
|
||||
% (port, module))
|
||||
d = DaemonManager()
|
||||
# --dparam borrows rsync's compact spelling; `pidfile` is inert but must
|
||||
# not be rejected, proving dparam reuses the expanded global key set.
|
||||
d.start(conf, extra_args=["--dparam", "pidfile=/tmp/rsync_style.pid"],
|
||||
log_path=os.path.join(TEST_DATA_DIR, "fastsyncd_rsync_style.log"))
|
||||
try:
|
||||
result = _push("127.0.0.1::rsync_style", d.port)
|
||||
assert result.returncode == 0, result.stderr or result.stdout
|
||||
received = get_dest_received_dir(module, SOURCE_DIR)
|
||||
mismatches, missing = verify_transfer(SOURCE_DIR, received)
|
||||
assert not missing, f"missing: {missing[:5]}"
|
||||
assert not mismatches, f"mismatch: {mismatches[:5]}"
|
||||
finally:
|
||||
d.stop()
|
||||
|
||||
|
||||
class TestDaemonRejection:
|
||||
def _tree_files(self):
|
||||
"""Snapshot every file path (module-relative) currently under the module
|
||||
@@ -477,7 +541,7 @@ class TestDaemonRejection:
|
||||
before any data lands. `accept` lists the log phrases that count as the
|
||||
refusal (a non-root daemon refuses --copy-as earlier, at the privilege
|
||||
check, so the caller accepts that phrase too)."""
|
||||
log_path = os.path.join(TEST_DATA_DIR, "fastsyncd.log")
|
||||
log_path = daemon.log_path
|
||||
before = os.path.getsize(log_path) if os.path.exists(log_path) else 0
|
||||
before_files = self._tree_files()
|
||||
result, _ = run_client(SOURCE_DIR, f"127.0.0.1::{module}", port=daemon.port, flags=flags)
|
||||
@@ -514,14 +578,13 @@ class TestDaemonRejection:
|
||||
the refusal into a silent accept."""
|
||||
port = _find_free_port()
|
||||
d = DaemonManager()
|
||||
log_path = os.path.join(TEST_DATA_DIR, "fastsyncd.log")
|
||||
try:
|
||||
d.start(CONF_FILE, port_override=port,
|
||||
extra_args=["--password-file", CRED_FILE, "--no-super"])
|
||||
result, _ = run_client(SOURCE_DIR, "127.0.0.1::files", port=d.port,
|
||||
flags=["--super", "--preserve"])
|
||||
assert result.returncode != 0, "the --no-super daemon must refuse --super"
|
||||
with open(log_path, "rb") as f:
|
||||
with open(d.log_path, "rb") as f:
|
||||
tail = f.read().decode("utf-8", "replace")
|
||||
assert "client-chosen ownership" in tail, (
|
||||
f"daemon did not log the --super refusal: {tail[-400:]!r}"
|
||||
@@ -1010,7 +1073,7 @@ class TestDaemonAuthentication:
|
||||
|
||||
def test_auth_log_does_not_leak_password(self, daemon):
|
||||
"""The daemon log must never contain the password or the store verifier."""
|
||||
log_path = os.path.join(TEST_DATA_DIR, "fastsyncd.log")
|
||||
log_path = daemon.log_path
|
||||
before = os.path.getsize(log_path) if os.path.exists(log_path) else 0
|
||||
_push_with_creds("127.0.0.1::locked", daemon.port, "alice", WRONG_PASS)
|
||||
_push_with_creds("127.0.0.1::locked", daemon.port, "alice", ALICE_PASS)
|
||||
@@ -1037,7 +1100,7 @@ class TestDaemonAuthentication:
|
||||
_push_with_creds("127.0.0.1::locked", port, "alice", ALICE_PASS)
|
||||
_push_with_creds("127.0.0.1::locked", port, "alice", WRONG_PASS)
|
||||
time.sleep(0.3)
|
||||
log_path = os.path.join(TEST_DATA_DIR, "fastsyncd.log")
|
||||
log_path = d.log_path
|
||||
with open(log_path, "rb") as f:
|
||||
log = f.read().decode("utf-8", "replace")
|
||||
finally:
|
||||
@@ -1072,7 +1135,8 @@ class TestDaemonMotd:
|
||||
motd_line = "motd file = %s\n" % motd_path if motd_path else ""
|
||||
os.makedirs(self.MOTD_MODULE, exist_ok=True)
|
||||
with open(self.MOTD_CONF, "w") as f:
|
||||
f.write("port = %d\n%s\n[files]\npath = %s\n" % (port, motd_line, self.MOTD_MODULE))
|
||||
f.write("port = %d\n%s\n[files]\npath = %s\nread only = no\n"
|
||||
% (port, motd_line, self.MOTD_MODULE))
|
||||
d = DaemonManager()
|
||||
d.start(self.MOTD_CONF, port_override=port)
|
||||
return d, port
|
||||
@@ -1256,12 +1320,12 @@ class TestDaemonTLSAuth:
|
||||
_write_client_password_file(client_creds, "alice", ALICE_PASS)
|
||||
d = DaemonManager()
|
||||
port = _find_free_port()
|
||||
log_path = os.path.join(TEST_DATA_DIR, "fastsyncd.log")
|
||||
try:
|
||||
d.start(CONF_FILE, port_override=port, extra_args=[
|
||||
"--tls", "--cert", certs["server_cert"], "--key", certs["server_key"],
|
||||
"--ca", certs["ca"], "--client-cn", "fastsync-client",
|
||||
"--password-file", CRED_FILE])
|
||||
log_path = d.log_path
|
||||
before_files = _tree_file_count(AUTH_MODULE)
|
||||
log_before = os.path.getsize(log_path) if os.path.exists(log_path) else 0
|
||||
tls_flags = ["--tls",
|
||||
@@ -1309,6 +1373,7 @@ class TestDaemonConnectionLimits:
|
||||
"\n"
|
||||
"[locked]\n"
|
||||
"path = %s\n"
|
||||
"read only = no\n"
|
||||
"auth users = alice\n"
|
||||
% (port, AUTH_MODULE))
|
||||
d = DaemonManager()
|
||||
@@ -1346,6 +1411,7 @@ class TestDaemonConnectionLimits:
|
||||
"\n"
|
||||
"[files]\n"
|
||||
"path = %s\n"
|
||||
"read only = no\n"
|
||||
"max connections = 2\n"
|
||||
% (port, FILES_MODULE))
|
||||
d = DaemonManager()
|
||||
|
||||
@@ -111,13 +111,20 @@ class _SlicingProxy:
|
||||
"""
|
||||
|
||||
def __init__(self, target_port, forward_limit=None, hook=None, hook_after=0,
|
||||
throttle=0.0, wait_for_reply=False):
|
||||
throttle=0.0, wait_for_reply=False, hook_after_config_ack=False):
|
||||
self.target = ("127.0.0.1", target_port)
|
||||
self.forward_limit = forward_limit
|
||||
self.hook = hook
|
||||
self.hook_after = hook_after
|
||||
self.throttle = throttle
|
||||
self.wait_for_reply = wait_for_reply
|
||||
# When set, the hook fires on the FIRST client->server bytes that follow
|
||||
# the config-frame ack, BEFORE they are forwarded. For --delete-before
|
||||
# those bytes are the keep-set manifest, so this runs the hook after the
|
||||
# client's source pre-scan but before the receiver's delete ack releases
|
||||
# the client into its data pass -- a deterministic late-file window.
|
||||
self.hook_after_config_ack = hook_after_config_ack
|
||||
self.config_acked = False
|
||||
self.server_replied = threading.Event()
|
||||
self.hook_called = threading.Event()
|
||||
self.listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
@@ -165,6 +172,13 @@ class _SlicingProxy:
|
||||
socks = []
|
||||
break
|
||||
data = data[:room]
|
||||
if (self.hook_after_config_ack and self.config_acked and self.hook is not None
|
||||
and not self.hook_called.is_set()):
|
||||
# The first client bytes after the config ack are the
|
||||
# pre-scan keep-set manifest: run the injection before
|
||||
# forwarding so it is causally after the source scan.
|
||||
self.hook()
|
||||
self.hook_called.set()
|
||||
backend.sendall(data)
|
||||
forwarded += len(data)
|
||||
self._maybe_hook(forwarded)
|
||||
@@ -177,6 +191,7 @@ class _SlicingProxy:
|
||||
client.sendall(data)
|
||||
# Any server reply proves the receiver consumed the
|
||||
# frames that precede it, so the hook barrier is met.
|
||||
self.config_acked = True
|
||||
self.server_replied.set()
|
||||
self._maybe_hook(forwarded)
|
||||
except OSError:
|
||||
@@ -198,8 +213,11 @@ class _SlicingProxy:
|
||||
def _maybe_hook(self, forwarded):
|
||||
"""Fire the one-shot hook once its barrier is satisfied: enough client
|
||||
bytes have been forwarded and, when ``wait_for_reply`` is set, the
|
||||
server has sent a reply proving it processed the preceding frames."""
|
||||
if self.hook is None or self.hook_called.is_set():
|
||||
server has sent a reply proving it processed the preceding frames.
|
||||
|
||||
``hook_after_config_ack`` uses its own barrier (see ``_serve``), so the
|
||||
byte/reply heuristic is bypassed entirely."""
|
||||
if self.hook is None or self.hook_called.is_set() or self.hook_after_config_ack:
|
||||
return
|
||||
if forwarded < self.hook_after:
|
||||
return
|
||||
@@ -537,3 +555,76 @@ class TestDeleteDelayMaxDeleteRefilledDir:
|
||||
assert os.path.isdir(later_dir), "later extra was not skipped by the budget"
|
||||
# The one actual removal is reported.
|
||||
assert _deleted_count(result.stdout) == 1, result.stdout
|
||||
|
||||
|
||||
class TestDeleteBeforeLateFileParity:
|
||||
"""rsync builds its file list once, so a source file created after that scan
|
||||
is NOT transferred and its destination extra is deleted. FastSync used to
|
||||
re-scan the source in its data pass (single-threaded) or pipeline a fresh
|
||||
re-scan against the pre-scan keep-set (``--threads``) and would transfer the
|
||||
late file (a safe superset); both paths now replay the pre-scan file list
|
||||
instead, matching rsync.
|
||||
|
||||
The late file is injected through the config-ack barrier: the first client
|
||||
bytes after the config ack are the pre-scan keep-set manifest, so the hook
|
||||
runs causally after the source scan and before the receiver's delete ack
|
||||
releases the client into its data pass.
|
||||
|
||||
For ``--threads`` the pipeline scanner runs concurrently with the sender, so
|
||||
the injection must land while that re-scan is still in flight to be observed
|
||||
by it. The source is therefore a tree of ``_N_DIRS`` directories: the
|
||||
injection writes the late file into EVERY directory, so it is enough that
|
||||
any one directory is still unscanned when the hook fires. The tree is sized
|
||||
so the hook (a localhost round trip) lands long before a full scan finishes;
|
||||
a re-scanning pipeline then transfers the late files for the directories it
|
||||
has not yet reached, which the tree comparison catches.
|
||||
"""
|
||||
|
||||
_N_DIRS = 2000
|
||||
|
||||
@requires_rsync
|
||||
@pytest.mark.parametrize("mt", [False, True])
|
||||
def test_late_source_file_not_transferred_and_extra_deleted(self, mt):
|
||||
tag = f"dblate_mt{int(mt)}"
|
||||
source = os.path.join(TEST_DATA_DIR, f"{tag}_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, f"{tag}_dst")
|
||||
rsync_dst = os.path.join(TEST_DATA_DIR, f"{tag}_rsync_dst")
|
||||
clean_dir(source)
|
||||
clean_dir(dest)
|
||||
clean_dir(rsync_dst)
|
||||
for i in range(self._N_DIRS):
|
||||
_write(os.path.join(source, f"dir{i:05d}", "keep.txt"), b"kept payload\n")
|
||||
# Both destinations carry the would-be late file as an extra.
|
||||
for root in (dest, rsync_dst):
|
||||
received = get_dest_received_dir(root, source)
|
||||
for i in range(self._N_DIRS):
|
||||
_write(os.path.join(received, f"dir{i:05d}", "late.txt"), b"stale extra\n")
|
||||
|
||||
# rsync reference: the same source with no late file; the extras are
|
||||
# removed and nothing is transferred for the (never-scanned) late paths.
|
||||
rsync_result = _rsync(["-a", "--delete-before", source + "/", rsync_dst + "/"])
|
||||
assert rsync_result.returncode == 0, rsync_result.stderr
|
||||
rsync_tree = _tree(rsync_dst)
|
||||
assert "dir00000/late.txt" not in rsync_tree
|
||||
|
||||
received = get_dest_received_dir(dest, source)
|
||||
|
||||
def hook():
|
||||
# Runs after the pre-scan and before the receiver's delete ack.
|
||||
for i in range(self._N_DIRS):
|
||||
_write(os.path.join(source, f"dir{i:05d}", "late.txt"),
|
||||
b"created after the scan\n")
|
||||
|
||||
with ServerManager() as server:
|
||||
server.start(extra_args=["--allow-delete"])
|
||||
proxy = _SlicingProxy(server.port, hook=hook, hook_after_config_ack=True)
|
||||
flags = ["--delete-before"] + (["--threads=4"] if mt else [])
|
||||
result, _ = run_client(source, dest, flags=flags, port=proxy.port)
|
||||
proxy.finish()
|
||||
assert result.returncode == 0, (result.stderr or result.stdout)[:400]
|
||||
assert proxy.hook_called.is_set(), "late-file hook never fired"
|
||||
assert _tree(received) == rsync_tree, (
|
||||
f"late source {'multithreaded' if mt else 'single-threaded'} data pass re-scanned: "
|
||||
f"{sum(1 for p in _tree(received) if p.endswith('late.txt'))} late files were "
|
||||
"transferred"
|
||||
)
|
||||
|
||||
@@ -183,10 +183,11 @@ class TestDeviceSpecial:
|
||||
)
|
||||
|
||||
@pytest.mark.setpriv
|
||||
def test_devices_nonroot_receiver_skips_safely(self):
|
||||
"""A receiver without CAP_MKNOD must skip a device entry with a warning
|
||||
and never abort. A root runner drops the receiver (server) to nobody
|
||||
via setpriv; on a non-root runner (or without setpriv) the test skips."""
|
||||
def test_devices_nonroot_receiver_errors_like_rsync(self):
|
||||
"""A receiver without CAP_MKNOD must report the failed device mknod as a
|
||||
transfer error (rsync parity, partial failure) instead of silently
|
||||
succeeding. A root runner drops the receiver (server) to nobody via
|
||||
setpriv; on a non-root runner (or without setpriv) the test skips."""
|
||||
if os.geteuid() != 0 or shutil.which("setpriv") is None:
|
||||
pytest.skip("requires root + setpriv to run the receiver unprivileged")
|
||||
self._setup()
|
||||
@@ -202,16 +203,16 @@ class TestDeviceSpecial:
|
||||
flags=["--devices"], port=port)
|
||||
finally:
|
||||
out, err = _stop_captured_server(server)
|
||||
assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr[:300]}"
|
||||
received = get_dest_received_dir(DEVICE_DEST, DEVICE_SOURCE)
|
||||
with open(os.path.join(received, "plain.txt")) as f:
|
||||
assert f.read() == "regular content\n"
|
||||
assert not os.path.lexists(os.path.join(received, "chardev")), (
|
||||
"a receiver without CAP_MKNOD must skip the device node, not create it"
|
||||
assert result.returncode != 0, (
|
||||
f"a failed device mknod must be a transfer error like rsync (got exit 0): "
|
||||
f"{(out + err)[:300]}"
|
||||
)
|
||||
assert ("cannot create device node" in (out + err)
|
||||
or "device-node creation is not permitted" in (out + err)), (
|
||||
f"receiver did not log the documented device skip: out={out!r} err={err!r}"
|
||||
received = get_dest_received_dir(DEVICE_DEST, DEVICE_SOURCE)
|
||||
assert not os.path.lexists(os.path.join(received, "chardev")), (
|
||||
"a receiver without CAP_MKNOD must not create the device node"
|
||||
)
|
||||
assert "cannot create device" in (out + err), (
|
||||
f"receiver did not log the device creation error: out={out!r} err={err!r}"
|
||||
)
|
||||
|
||||
@pytest.mark.skipif(os.geteuid() != 0, reason="requires root to create device nodes")
|
||||
@@ -6599,7 +6600,7 @@ class TestExtendedAttributes:
|
||||
os.getxattr(os.path.join(received, "data.txt"), "user.foo")
|
||||
|
||||
def test_reserved_fake_super_key_not_forwarded(self, shared_server):
|
||||
"""A source file that already carries the reserved user.fastsync.stat
|
||||
"""A source file that already carries the reserved user.rsync.%stat
|
||||
record must NOT have it planted on the receiver during a plain -X run
|
||||
(it is receiver-only, so it cannot be spoofed for a later privileged
|
||||
restore)."""
|
||||
@@ -6609,7 +6610,7 @@ class TestExtendedAttributes:
|
||||
fh.write(b"reserved\n")
|
||||
if not _xattr_supported(f):
|
||||
pytest.skip("filesystem does not support user xattrs")
|
||||
os.setxattr(f, "user.fastsync.stat", b"0:0:644:0:0")
|
||||
os.setxattr(f, "user.rsync.%stat", b"100644 0,0 0:0")
|
||||
# A normal user.* attr still travels alongside.
|
||||
os.setxattr(f, "user.keep", b"yes")
|
||||
|
||||
@@ -6619,7 +6620,7 @@ class TestExtendedAttributes:
|
||||
received = get_dest_received_dir(dest, source)
|
||||
assert os.getxattr(os.path.join(received, "data.txt"), "user.keep") == b"yes"
|
||||
with pytest.raises(OSError):
|
||||
os.getxattr(os.path.join(received, "data.txt"), "user.fastsync.stat")
|
||||
os.getxattr(os.path.join(received, "data.txt"), "user.rsync.%stat")
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_xattrs_multithreaded(self, shared_server):
|
||||
@@ -6708,10 +6709,17 @@ class TestExtendedAttributes:
|
||||
assert result.returncode == 0, \
|
||||
f"--fake-super sync failed: {(result.stderr or result.stdout)[:300]}"
|
||||
received = get_dest_received_dir(dest, source)
|
||||
record = os.getxattr(os.path.join(received, "data.txt"), "user.fastsync.stat").decode()
|
||||
fields = record.split(":")
|
||||
assert len(fields) == 5
|
||||
assert fields[0] == str(uid), f"reserved uid field {fields[0]} != source uid {uid}"
|
||||
record = os.getxattr(os.path.join(received, "data.txt"), "user.rsync.%stat").decode()
|
||||
# rsync 3.4.1 grammar: "<octal st_mode> <rdev_major>,<rdev_minor> <uid>:<gid>".
|
||||
fields = record.split()
|
||||
assert len(fields) == 3, f"unexpected rsync fake-super record {record!r}"
|
||||
mode_field, rdev_field, owner_field = fields
|
||||
assert rdev_field == "0,0", f"regular file rdev must be 0,0, got {rdev_field!r}"
|
||||
assert int(mode_field, 8) & 0o170000 == stat.S_IFREG, (
|
||||
f"recorded mode {mode_field!r} must carry S_IFREG"
|
||||
)
|
||||
assert owner_field.split(":")[0] == str(uid), \
|
||||
f"recorded uid {owner_field!r} != source uid {uid}"
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_fake_super_records_resolved_chown_without_real_chown(self, shared_server):
|
||||
@@ -6730,12 +6738,71 @@ class TestExtendedAttributes:
|
||||
assert result.returncode == 0, \
|
||||
f"--fake-super --chown sync failed: {(result.stderr or result.stdout)[:300]}"
|
||||
dst = os.path.join(get_dest_received_dir(dest, source), "data.txt")
|
||||
record = os.getxattr(dst, "user.fastsync.stat").decode().split(":")
|
||||
assert record[0] == "33333", f"recorded owner {record[0]} != resolved 33333"
|
||||
assert record[1] == "44444", f"recorded group {record[1]} != resolved 44444"
|
||||
record = os.getxattr(dst, "user.rsync.%stat").decode().split()
|
||||
owner = record[2].split(":")
|
||||
assert owner == ["33333", "44444"], (
|
||||
f"recorded owner {record[2]!r} != resolved 33333:44444"
|
||||
)
|
||||
st = os.stat(dst)
|
||||
assert st.st_uid != 33333, "--fake-super must not real-chown the recorded owner"
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_fake_super_rsync_interop(self, shared_server):
|
||||
"""A fake-super tree written by FastSync is readable by rsync 3.4.1:
|
||||
rsync reads the `user.rsync.%stat` record (mode/rdev/uid:gid) and, when
|
||||
it re-emits a fake-super tree, reproduces the same record. This pins
|
||||
the on-disk key and value grammar against the real tool."""
|
||||
rsync = shutil.which("rsync")
|
||||
if rsync is None:
|
||||
pytest.skip("rsync not installed")
|
||||
source, dest = self._source_and_dest("fakesuper_interop")
|
||||
f = os.path.join(source, "data.txt")
|
||||
with open(f, "wb") as fh:
|
||||
fh.write(b"interop\n")
|
||||
if not _xattr_supported(f):
|
||||
pytest.skip("filesystem does not support user xattrs")
|
||||
# rsync's fake-super receiver only writes a %stat% record when it has
|
||||
# something to fake; a root-owned file with a matching root stat is
|
||||
# a no-op. When privileged, record a non-root owner so the round-trip
|
||||
# actually exercises the parser (non-root CI already has a non-zero uid).
|
||||
if os.geteuid() == 0:
|
||||
try:
|
||||
os.chown(f, 12345, 12346)
|
||||
except OSError:
|
||||
pass
|
||||
# A setuid bit exercises the full st_mode encoding; set it AFTER any
|
||||
# chown (chown clears setuid/setgid), and note that neither tool installs
|
||||
# it on the real destination file.
|
||||
os.chmod(f, 0o4711)
|
||||
|
||||
result, _ = run_client(source, dest, flags=["--fake-super"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode == 0, \
|
||||
f"--fake-super sync failed: {(result.stderr or result.stdout)[:300]}"
|
||||
received = get_dest_received_dir(dest, source)
|
||||
rec = os.getxattr(os.path.join(received, "data.txt"), "user.rsync.%stat").decode()
|
||||
rec_fields = rec.split()
|
||||
assert len(rec_fields) == 3 and rec_fields[1] == "0,0", (
|
||||
f"FastSync did not write rsync's stat grammar: {rec!r}"
|
||||
)
|
||||
assert int(rec_fields[0], 8) & 0o7777 == 0o4711, (
|
||||
f"FastSync did not record the source mode in rsync's grammar: {rec!r}"
|
||||
)
|
||||
|
||||
out = os.path.join(TEST_DATA_DIR, "fakesuper_interop_rsync")
|
||||
clean_dir(out)
|
||||
rs = subprocess.run([rsync, "-aX", "--fake-super",
|
||||
received + "/", out + "/"],
|
||||
capture_output=True, text=True, timeout=120)
|
||||
assert rs.returncode == 0, (
|
||||
f"rsync could not read FastSync's fake-super tree: {rs.stderr[:300]}"
|
||||
)
|
||||
out_rec = os.getxattr(os.path.join(out, "data.txt"), "user.rsync.%stat").decode()
|
||||
assert out_rec == rec, (
|
||||
"rsync re-emitted a different fake-super record; FastSync's grammar "
|
||||
f"is not interoperable: ours={rec!r} rsync={out_rec!r}"
|
||||
)
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_directory_xattrs_preserved(self, shared_server):
|
||||
"""#286.3: -aX must preserve user.* xattrs on DIRECTORIES, not just files."""
|
||||
@@ -7255,9 +7322,10 @@ class TestCopyAs:
|
||||
)
|
||||
received = get_dest_received_dir(dest, source)
|
||||
dst = os.path.join(received, "mixed.txt")
|
||||
record = os.getxattr(dst, "user.fastsync.stat").decode().split(":")
|
||||
assert (record[0], record[1]) == ("65534", "65534"), (
|
||||
f"fake-super must record the resolved copy-as ownership: {record[:2]}"
|
||||
record = os.getxattr(dst, "user.rsync.%stat").decode().split()
|
||||
owner = record[2].split(":")
|
||||
assert owner == ["65534", "65534"], (
|
||||
f"fake-super must record the resolved copy-as ownership: {owner}"
|
||||
)
|
||||
st = os.lstat(dst)
|
||||
assert (st.st_uid, st.st_gid) != (12345, 12346), (
|
||||
|
||||
@@ -353,10 +353,10 @@ class TestOwnershipRoot:
|
||||
st = os.stat(dst)
|
||||
assert st.st_uid != 12345, \
|
||||
f"--fake-super -o must NOT real-chown the source owner, got uid={st.st_uid}"
|
||||
record = os.getxattr(dst, "user.fastsync.stat").decode()
|
||||
fields = record.split(":")
|
||||
assert fields[0] == "12345", \
|
||||
f"--fake-super must record the resolved owner, got {fields[0]}"
|
||||
record = os.getxattr(dst, "user.rsync.%stat").decode()
|
||||
owner = record.split()[2].split(":")
|
||||
assert owner[0] == "12345", \
|
||||
f"--fake-super must record the resolved owner, got {owner[0]}"
|
||||
|
||||
def test_o_applies_directory_owner(self, shared_server):
|
||||
"""#286.2: -o must apply the source owner to DIRECTORIES too (the
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
"""Parity coverage for an absolute ``--temp-dir`` that lies inside the receive root.
|
||||
|
||||
FastSync confines the ``--temp-dir`` scratch directory to the receive root. It
|
||||
previously rejected *every* absolute path; it now canonicalizes an absolute path
|
||||
with ``realpath(3)`` and accepts it when it resolves inside the canonical receive
|
||||
root (the destination is identical, so this is a pure parity win), while an
|
||||
absolute path that escapes the root stays rejected with a clear error.
|
||||
|
||||
Two paths exercise the same receiver-side resolution:
|
||||
|
||||
* the local ``--read-batch`` apply (no network; the batch destination is the
|
||||
receive root), and
|
||||
* a real TCP transfer against the shared server (the destination root is the
|
||||
client-supplied absolute path).
|
||||
|
||||
The out-of-root case asserts the run fails without writing a single scratch
|
||||
file, so the confinement invariant is preserved.
|
||||
"""
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
from common import CLIENT_CMD, TEST_DATA_DIR, clean_dir, get_dest_received_dir, run_client
|
||||
|
||||
FILES = {
|
||||
"top.txt": b"top level\n",
|
||||
"sub/nested.txt": b"nested file\n" * 16,
|
||||
}
|
||||
|
||||
|
||||
def _run(args):
|
||||
return subprocess.run(CLIENT_CMD + args, capture_output=True, text=True, timeout=180)
|
||||
|
||||
|
||||
def _seed_source(root):
|
||||
clean_dir(root)
|
||||
for rel, data in FILES.items():
|
||||
full = os.path.join(root, rel)
|
||||
os.makedirs(os.path.dirname(full), exist_ok=True)
|
||||
with open(full, "wb") as fh:
|
||||
fh.write(data)
|
||||
|
||||
|
||||
def _make_batch(tmp, source):
|
||||
batch = os.path.join(tmp, "tree.batch")
|
||||
result = _run(["--only-write-batch", batch, source])
|
||||
assert result.returncode == 0, (result.stdout, result.stderr)
|
||||
return batch
|
||||
|
||||
|
||||
def _read(path):
|
||||
with open(path, "rb") as fh:
|
||||
return fh.read()
|
||||
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_read_batch_absolute_temp_dir_inside_root_accepted(tmp_path):
|
||||
source = os.path.join(tmp_path, "src")
|
||||
dest = os.path.join(tmp_path, "dst")
|
||||
_seed_source(source)
|
||||
clean_dir(dest)
|
||||
scratch = os.path.join(dest, "scratch")
|
||||
os.makedirs(scratch)
|
||||
# Stamp the scratch dir with an old mtime so the test can prove the receiver
|
||||
# really created (and then removed) its temp file there: the directory mtime
|
||||
# changes when an entry is created/removed, so a silently-ignored --temp-dir
|
||||
# would leave the stamp untouched. An empty scratch dir alone does not
|
||||
# distinguish "used and cleaned up" from "never used".
|
||||
stale_mtime = 946684800 # 2000-01-01
|
||||
os.utime(scratch, (stale_mtime, stale_mtime))
|
||||
|
||||
batch = _make_batch(str(tmp_path), source)
|
||||
result = _run(["--read-batch", batch, dest, "--temp-dir", scratch])
|
||||
assert result.returncode == 0, (result.stdout, result.stderr)
|
||||
|
||||
received = get_dest_received_dir(dest, source)
|
||||
for rel, data in FILES.items():
|
||||
assert _read(os.path.join(received, rel)) == data, f"content mismatch for {rel}"
|
||||
assert os.listdir(scratch) == [], "scratch dir was not left clean"
|
||||
assert os.stat(scratch).st_mtime != stale_mtime, (
|
||||
"--temp-dir scratch dir was never written to (temp file not created there)"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_read_batch_absolute_temp_dir_outside_root_rejected(tmp_path):
|
||||
source = os.path.join(tmp_path, "src")
|
||||
dest = os.path.join(tmp_path, "dst")
|
||||
_seed_source(source)
|
||||
clean_dir(dest)
|
||||
outside = os.path.join(tmp_path, "outside")
|
||||
os.makedirs(outside)
|
||||
|
||||
batch = _make_batch(str(tmp_path), source)
|
||||
result = _run(["--read-batch", batch, dest, "--temp-dir", outside])
|
||||
assert result.returncode != 0, "an absolute temp dir outside the receive root must be rejected"
|
||||
assert os.listdir(outside) == [], "receiver wrote into an unconfined temp dir"
|
||||
assert "temp-dir" in (result.stdout + result.stderr), (result.stdout, result.stderr)
|
||||
|
||||
|
||||
def test_tcp_absolute_temp_dir_inside_root_accepted(shared_server):
|
||||
source = os.path.join(TEST_DATA_DIR, "tempdir_abs_in_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "tempdir_abs_in_dst")
|
||||
_seed_source(source)
|
||||
clean_dir(dest)
|
||||
scratch = os.path.join(dest, "scratch")
|
||||
os.makedirs(scratch)
|
||||
# See test_read_batch_absolute_temp_dir_inside_root_accepted: the stale
|
||||
# mtime makes actual scratch usage observable (the temp file creation and
|
||||
# removal bump the directory mtime).
|
||||
stale_mtime = 946684800 # 2000-01-01
|
||||
os.utime(scratch, (stale_mtime, stale_mtime))
|
||||
|
||||
result, _ = run_client(source, dest, flags=["--temp-dir", scratch], port=shared_server.port)
|
||||
assert result.returncode == 0, (result.stdout, result.stderr)[:300]
|
||||
received = get_dest_received_dir(dest, source)
|
||||
for rel, data in FILES.items():
|
||||
assert _read(os.path.join(received, rel)) == data, f"content mismatch for {rel}"
|
||||
assert os.listdir(scratch) == [], "scratch dir was not left clean"
|
||||
assert os.stat(scratch).st_mtime != stale_mtime, (
|
||||
"--temp-dir scratch dir was never written to (temp file not created there)"
|
||||
)
|
||||
|
||||
|
||||
def test_tcp_absolute_temp_dir_outside_root_rejected(shared_server):
|
||||
source = os.path.join(TEST_DATA_DIR, "tempdir_abs_out_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "tempdir_abs_out_dst")
|
||||
_seed_source(source)
|
||||
clean_dir(dest)
|
||||
outside = os.path.join(TEST_DATA_DIR, "tempdir_abs_out_scratch")
|
||||
clean_dir(outside)
|
||||
|
||||
result, _ = run_client(source, dest, flags=["--temp-dir", outside], port=shared_server.port)
|
||||
assert result.returncode != 0, "an absolute temp dir outside the receive root must be rejected"
|
||||
assert os.listdir(outside) == [], "receiver wrote into an unconfined temp dir"
|
||||
@@ -5040,10 +5040,12 @@ static void test_parse_args_include_exclude_order() {
|
||||
config_delete(cfg3);
|
||||
}
|
||||
|
||||
/* OPT_NOOP compatibility flags (-s/--secluded-args, -r/--recursive) must never
|
||||
* swallow the next argv: `fastsync -s SRC DST` keeps both positionals. */
|
||||
/* OPT_NOOP compatibility flags (-s/--secluded-args, -r/--recursive, and the
|
||||
* --inc-recursive/--no-inc-recursive scan-mode pair) must never swallow the
|
||||
* next argv: `fastsync -s SRC DST` keeps both positionals. */
|
||||
static void test_parse_args_noop_does_not_consume_argv() {
|
||||
static const char* const noops[] = {"-s", "--secluded-args", "-r", "--recursive"};
|
||||
static const char* const noops[] = {"-s", "--secluded-args", "-r",
|
||||
"--recursive", "--inc-recursive", "--no-inc-recursive"};
|
||||
for (size_t i = 0; i < sizeof(noops) / sizeof(noops[0]); i++) {
|
||||
Config* cfg = config_create();
|
||||
int positional_args[2];
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
#include "test_daemon_conf.h"
|
||||
#include "credentials.h"
|
||||
#include "daemon_conf.h"
|
||||
#include "log.h"
|
||||
#include "test_utils.h"
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
@@ -31,6 +32,9 @@ static void test_daemon_conf_create_defaults() {
|
||||
EXPECT_EQ_INT(conf->global.port, DAEMON_CONF_DEFAULT_PORT);
|
||||
EXPECT_NULL(conf->global.motd_file);
|
||||
EXPECT_NULL(conf->global.address);
|
||||
/* rsync modules are read-only unless they opt in, so the default must be
|
||||
* true. */
|
||||
EXPECT_TRUE(conf->global.read_only_default);
|
||||
EXPECT_EQ_INT(conf->global.max_connections, DAEMON_CONF_DEFAULT_MAX_CONNECTIONS);
|
||||
EXPECT_EQ_INT(conf->global.auth_failure_delay_ms, DAEMON_CONF_DEFAULT_AUTH_FAILURE_DELAY_MS);
|
||||
EXPECT_EQ_INT(conf->global.max_connections_per_host,
|
||||
@@ -626,6 +630,348 @@ static void test_daemon_conf_module_count_capped() {
|
||||
EXPECT_TRUE(strstr(err, "too many modules") != NULL);
|
||||
}
|
||||
|
||||
/* rsync rsyncd.conf compatibility: the common GLOBAL keys FastSync does not
|
||||
* implement (pid file, log file, use chroot, uid/gid, timeout, ...) are
|
||||
* accepted as documented inert keys, while the keys with a FastSync equivalent
|
||||
* keep working and the compact rsync --dparam spellings (`pidfile`, `logfile`,
|
||||
* `motdfile`) are recognized. A global `read only` is rsync's module default
|
||||
* and must not be silently dropped. */
|
||||
static void test_daemon_conf_rsync_global_keys() {
|
||||
char* path;
|
||||
char err[256];
|
||||
EXPECT_EQ_INT(write_conf("pid file = /run/fastsyncd.pid\n"
|
||||
"log file = /var/log/fastsyncd.log\n"
|
||||
"socket options = TCP_NODELAY\n"
|
||||
"listen backlog = 10\n"
|
||||
"syslog facility = daemon\n"
|
||||
"syslog tag = fastsyncd\n"
|
||||
"use chroot = no\n"
|
||||
"uid = nobody\n"
|
||||
"gid = nogroup\n"
|
||||
"timeout = 600\n"
|
||||
"max verbosity = 3\n"
|
||||
"lock file = /var/run/fastsyncd.lock\n"
|
||||
"transfer logging = yes\n"
|
||||
"strict modes = yes\n"
|
||||
"reverse lookup = no\n"
|
||||
"dont compress = *.gz\n"
|
||||
"read only = yes\n"
|
||||
"port = 8734\n"
|
||||
"address = 127.0.0.1\n"
|
||||
"pidfile = /run/other.pid\n"
|
||||
"logfile = /var/log/other.log\n"
|
||||
"motdfile = /etc/fastsync/motd.alt\n"
|
||||
"\n"
|
||||
"[pub]\n"
|
||||
"path = /srv/pub\n"
|
||||
"\n"
|
||||
"[explicit]\n"
|
||||
"path = /srv/explicit\n"
|
||||
"read only = no\n",
|
||||
&path),
|
||||
0);
|
||||
DaemonConf* conf = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NOT_NULL(conf);
|
||||
/* Mapped globals took effect; the compact aliases too. */
|
||||
EXPECT_EQ_INT(conf->global.port, 8734);
|
||||
EXPECT_EQ_STR(conf->global.address, "127.0.0.1");
|
||||
EXPECT_EQ_STR(conf->global.motd_file, "/etc/fastsync/motd.alt");
|
||||
/* The global `read only = yes` is the default for modules defined after it. */
|
||||
EXPECT_TRUE(conf->global.read_only_default);
|
||||
EXPECT_EQ_INT(conf->module_count, 2);
|
||||
EXPECT_TRUE(conf->modules[0].read_only);
|
||||
/* An explicit per-module value wins over the global default. */
|
||||
EXPECT_FALSE(conf->modules[1].read_only);
|
||||
daemon_conf_free(conf);
|
||||
}
|
||||
|
||||
/* rsync module keys with no FastSync equivalent load inert; the keys with a
|
||||
* FastSync meaning still map onto their native fields. */
|
||||
static void test_daemon_conf_rsync_module_keys() {
|
||||
char* path;
|
||||
char err[256];
|
||||
EXPECT_EQ_INT(write_conf("[data]\n"
|
||||
"path = /srv/data\n"
|
||||
"comment = Public data\n"
|
||||
"use chroot = yes\n"
|
||||
"uid = nobody\n"
|
||||
"gid = nogroup\n"
|
||||
"exclude = *.tmp\n"
|
||||
"include = keep.tmp\n"
|
||||
"exclude from = /etc/rsync.exclude\n"
|
||||
"max verbosity = 2\n"
|
||||
"lock file = /var/run/rsyncd.lock\n"
|
||||
"transfer logging = yes\n"
|
||||
"timeout = 300\n"
|
||||
"secrets file = /etc/rsyncd.secrets\n"
|
||||
"auth digest = sha256\n"
|
||||
"numeric ids = yes\n"
|
||||
"write only = no\n"
|
||||
"list = yes\n"
|
||||
"dont compress = *.gz\n"
|
||||
"refuse options = delete\n"
|
||||
"read only = yes\n"
|
||||
"max connections = 5\n"
|
||||
"hosts allow = 10.0.0.0/8\n"
|
||||
"auth users = alice\n",
|
||||
&path),
|
||||
0);
|
||||
DaemonConf* conf = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NOT_NULL(conf);
|
||||
EXPECT_EQ_STR(conf->modules[0].path, "/srv/data");
|
||||
EXPECT_TRUE(conf->modules[0].read_only);
|
||||
EXPECT_EQ_INT(conf->modules[0].max_connections, 5);
|
||||
EXPECT_EQ_INT(conf->modules[0].hosts_allow_count, 1);
|
||||
EXPECT_EQ_STR(conf->modules[0].hosts_allow[0], "10.0.0.0/8");
|
||||
EXPECT_EQ_INT(conf->modules[0].auth_user_count, 1);
|
||||
EXPECT_EQ_STR(conf->modules[0].auth_users[0], "alice");
|
||||
daemon_conf_free(conf);
|
||||
}
|
||||
|
||||
/* A genuinely unknown key is still rejected in both contexts, so accepting the
|
||||
* rsync subset did not turn typos into silent no-ops. */
|
||||
static void test_daemon_conf_rsync_unknown_keys_rejected() {
|
||||
char* path;
|
||||
char err[256];
|
||||
EXPECT_EQ_INT(write_conf("bogus rsync key = 1\n", &path), 0);
|
||||
const DaemonConf* conf = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NULL(conf);
|
||||
EXPECT_TRUE(strstr(err, "unknown global key") != NULL);
|
||||
|
||||
EXPECT_EQ_INT(write_conf("[m]\npath = /x\nnot a real key = 1\n", &path), 0);
|
||||
conf = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NULL(conf);
|
||||
EXPECT_TRUE(strstr(err, "unknown key 'not a real key'") != NULL);
|
||||
}
|
||||
|
||||
/* A recognized rsync key with an invalid value is still a clear parse error. */
|
||||
static void test_daemon_conf_rsync_read_only_invalid() {
|
||||
char* path;
|
||||
char err[256];
|
||||
EXPECT_EQ_INT(write_conf("read only = maybe\n[m]\npath = /x\n", &path), 0);
|
||||
const DaemonConf* conf = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NULL(conf);
|
||||
EXPECT_TRUE(strstr(err, "read only") != NULL);
|
||||
|
||||
EXPECT_EQ_INT(write_conf("[m]\npath = /x\nread only = maybe\n", &path), 0);
|
||||
conf = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NULL(conf);
|
||||
EXPECT_TRUE(strstr(err, "read only") != NULL);
|
||||
}
|
||||
|
||||
/* --dparam reuses the same global dispatch: it accepts the compact rsync
|
||||
* spellings and the inert rsync global keys, and `read only` sets the default
|
||||
* for modules that did not set their own value. */
|
||||
static void test_daemon_conf_dparam_rsync_keys() {
|
||||
DaemonConf* conf = daemon_conf_create();
|
||||
EXPECT_NOT_NULL(conf);
|
||||
char err[256];
|
||||
|
||||
EXPECT_EQ_INT(daemon_conf_apply_dparam(conf, "pidfile=/run/x.pid", err, sizeof(err)), 0);
|
||||
EXPECT_EQ_INT(daemon_conf_apply_dparam(conf, "pid file=/run/y.pid", err, sizeof(err)), 0);
|
||||
EXPECT_EQ_INT(daemon_conf_apply_dparam(conf, "logfile=/tmp/x.log", err, sizeof(err)), 0);
|
||||
EXPECT_EQ_INT(daemon_conf_apply_dparam(conf, "log file=/tmp/y.log", err, sizeof(err)), 0);
|
||||
EXPECT_EQ_INT(daemon_conf_apply_dparam(conf, "motdfile=/tmp/alt.motd", err, sizeof(err)), 0);
|
||||
EXPECT_EQ_STR(conf->global.motd_file, "/tmp/alt.motd");
|
||||
EXPECT_EQ_INT(daemon_conf_apply_dparam(conf, "timeout=600", err, sizeof(err)), 0);
|
||||
EXPECT_EQ_INT(daemon_conf_apply_dparam(conf, "use chroot=no", err, sizeof(err)), 0);
|
||||
|
||||
/* A module already parsed without an explicit `read only` takes the
|
||||
* --dparam default; an explicit module value is preserved. */
|
||||
{
|
||||
char* path;
|
||||
EXPECT_EQ_INT(write_conf("[plain]\npath = /p\n[explicit]\npath = /e\nread only = no\n", &path),
|
||||
0);
|
||||
DaemonConf* loaded = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NOT_NULL(loaded);
|
||||
EXPECT_EQ_INT(daemon_conf_apply_dparam(loaded, "read only=yes", err, sizeof(err)), 0);
|
||||
EXPECT_TRUE(loaded->global.read_only_default);
|
||||
EXPECT_TRUE(loaded->modules[0].read_only);
|
||||
EXPECT_FALSE(loaded->modules[1].read_only);
|
||||
daemon_conf_free(loaded);
|
||||
}
|
||||
|
||||
/* Invalid values and genuinely unknown keys are still rejected. */
|
||||
EXPECT_EQ_INT(daemon_conf_apply_dparam(conf, "read only=maybe", err, sizeof(err)), -1);
|
||||
EXPECT_EQ_INT(daemon_conf_apply_dparam(conf, "definitely not rsync=1", err, sizeof(err)), -1);
|
||||
EXPECT_TRUE(strstr(err, "unknown global key") != NULL);
|
||||
|
||||
daemon_conf_free(conf);
|
||||
}
|
||||
|
||||
/* rsync modules default to READ-ONLY; `read only = no` / `write only = yes`
|
||||
* opt a module into writability, and an explicit module value wins over a
|
||||
* global default. */
|
||||
static void test_daemon_conf_read_only_default_and_opt_in() {
|
||||
char* path;
|
||||
char err[256];
|
||||
DaemonConf* conf;
|
||||
|
||||
/* A module that never mentions read only/write only is READ-ONLY, matching
|
||||
* rsync (a migrated rsyncd.conf must not be served writable). */
|
||||
EXPECT_EQ_INT(write_conf("[m]\npath = /x\n", &path), 0);
|
||||
conf = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NOT_NULL(conf);
|
||||
EXPECT_TRUE(conf->modules[0].read_only);
|
||||
EXPECT_FALSE(conf->modules[0].read_only_explicit);
|
||||
daemon_conf_free(conf);
|
||||
|
||||
/* `read only = no` opts in to writable. */
|
||||
EXPECT_EQ_INT(write_conf("[m]\npath = /x\nread only = no\n", &path), 0);
|
||||
conf = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NOT_NULL(conf);
|
||||
EXPECT_FALSE(conf->modules[0].read_only);
|
||||
EXPECT_TRUE(conf->modules[0].read_only_explicit);
|
||||
daemon_conf_free(conf);
|
||||
|
||||
/* `write only = yes` opts in to writable (FastSync is push-only). */
|
||||
EXPECT_EQ_INT(write_conf("[m]\npath = /x\nwrite only = yes\n", &path), 0);
|
||||
conf = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NOT_NULL(conf);
|
||||
EXPECT_FALSE(conf->modules[0].read_only);
|
||||
EXPECT_TRUE(conf->modules[0].read_only_explicit);
|
||||
daemon_conf_free(conf);
|
||||
|
||||
/* `write only = no` is rsync's default and does not undo read-only. */
|
||||
EXPECT_EQ_INT(write_conf("[m]\npath = /x\nwrite only = no\n", &path), 0);
|
||||
conf = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NOT_NULL(conf);
|
||||
EXPECT_TRUE(conf->modules[0].read_only);
|
||||
EXPECT_FALSE(conf->modules[0].read_only_explicit);
|
||||
daemon_conf_free(conf);
|
||||
|
||||
/* A global `read only = no` is the default for later modules; an explicit
|
||||
* module `read only`/`write only = yes` still wins. */
|
||||
EXPECT_EQ_INT(write_conf("read only = no\n[a]\npath = /a\n"
|
||||
"[b]\npath = /b\nread only = yes\n"
|
||||
"[c]\npath = /c\nwrite only = yes\n",
|
||||
&path),
|
||||
0);
|
||||
conf = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NOT_NULL(conf);
|
||||
EXPECT_FALSE(conf->global.read_only_default);
|
||||
EXPECT_FALSE(conf->modules[0].read_only);
|
||||
EXPECT_TRUE(conf->modules[1].read_only);
|
||||
EXPECT_FALSE(conf->modules[2].read_only);
|
||||
daemon_conf_free(conf);
|
||||
|
||||
/* A global `read only = yes` keeps modules without an explicit value
|
||||
* read-only. */
|
||||
EXPECT_EQ_INT(write_conf("read only = yes\n[a]\npath = /a\n"
|
||||
"[b]\npath = /b\nread only = no\n"
|
||||
"[c]\npath = /c\nwrite only = yes\n",
|
||||
&path),
|
||||
0);
|
||||
conf = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NOT_NULL(conf);
|
||||
EXPECT_TRUE(conf->global.read_only_default);
|
||||
EXPECT_TRUE(conf->modules[0].read_only);
|
||||
EXPECT_FALSE(conf->modules[1].read_only);
|
||||
EXPECT_FALSE(conf->modules[2].read_only);
|
||||
daemon_conf_free(conf);
|
||||
|
||||
/* An invalid `write only` value is a clear parse error. */
|
||||
EXPECT_EQ_INT(write_conf("[m]\npath = /x\nwrite only = maybe\n", &path), 0);
|
||||
conf = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NULL(conf);
|
||||
EXPECT_TRUE(strstr(err, "write only") != NULL);
|
||||
}
|
||||
|
||||
/* Security-relevant rsync keys are accepted for migration but have no FastSync
|
||||
* effect, so loading must warn loudly (naming the key and module) rather than
|
||||
* letting an operator believe the restriction is enforced. */
|
||||
static void test_daemon_conf_unenforced_security_keys_warned() {
|
||||
char* path;
|
||||
char err[256];
|
||||
EXPECT_EQ_INT(write_conf("use chroot = yes\n"
|
||||
"uid = nobody\n"
|
||||
"[m]\n"
|
||||
"path = /x\n"
|
||||
"read only = no\n"
|
||||
"secrets file = /etc/rsyncd.secrets\n"
|
||||
"refuse options = delete\n"
|
||||
"exclude = *.tmp\n"
|
||||
"max size = 1M\n"
|
||||
"pre-xfer exec = /bin/true\n"
|
||||
"incoming chmod = F644\n"
|
||||
"name converter = sh\n",
|
||||
&path),
|
||||
0);
|
||||
|
||||
set_log_level(LOG_LEVEL_WARNING);
|
||||
FILE* log_capture = tmpfile();
|
||||
EXPECT_NOT_NULL(log_capture);
|
||||
log_set_file(log_capture);
|
||||
DaemonConf* conf = daemon_conf_load(path, err, sizeof(err));
|
||||
fflush(log_capture);
|
||||
rewind(log_capture);
|
||||
log_set_file(NULL);
|
||||
free(path);
|
||||
|
||||
/* Inert security keys must never fail the load. */
|
||||
EXPECT_NOT_NULL(conf);
|
||||
EXPECT_FALSE(conf->modules[0].read_only);
|
||||
|
||||
bool saw_secrets = false;
|
||||
bool saw_refuse = false;
|
||||
bool saw_filter = false;
|
||||
bool saw_size = false;
|
||||
bool saw_hook = false;
|
||||
bool saw_chmod = false;
|
||||
bool saw_converter = false;
|
||||
bool saw_global_chroot = false;
|
||||
bool saw_global_uid = false;
|
||||
char line[512];
|
||||
while (fgets(line, sizeof(line), log_capture) != NULL) {
|
||||
if (strstr(line, "NOT enforced") == NULL)
|
||||
continue;
|
||||
if (strstr(line, "module 'm'") && strstr(line, "secrets file"))
|
||||
saw_secrets = true;
|
||||
if (strstr(line, "module 'm'") && strstr(line, "refuse options"))
|
||||
saw_refuse = true;
|
||||
if (strstr(line, "module 'm'") && strstr(line, "exclude"))
|
||||
saw_filter = true;
|
||||
if (strstr(line, "module 'm'") && strstr(line, "max size"))
|
||||
saw_size = true;
|
||||
if (strstr(line, "module 'm'") && strstr(line, "pre-xfer exec"))
|
||||
saw_hook = true;
|
||||
if (strstr(line, "module 'm'") && strstr(line, "incoming chmod"))
|
||||
saw_chmod = true;
|
||||
if (strstr(line, "module 'm'") && strstr(line, "name converter"))
|
||||
saw_converter = true;
|
||||
if (strstr(line, "global key 'use chroot'"))
|
||||
saw_global_chroot = true;
|
||||
if (strstr(line, "global key 'uid'"))
|
||||
saw_global_uid = true;
|
||||
}
|
||||
fclose(log_capture);
|
||||
|
||||
EXPECT_TRUE(saw_secrets);
|
||||
EXPECT_TRUE(saw_refuse);
|
||||
EXPECT_TRUE(saw_filter);
|
||||
EXPECT_TRUE(saw_size);
|
||||
EXPECT_TRUE(saw_hook);
|
||||
EXPECT_TRUE(saw_chmod);
|
||||
EXPECT_TRUE(saw_converter);
|
||||
EXPECT_TRUE(saw_global_chroot);
|
||||
EXPECT_TRUE(saw_global_uid);
|
||||
daemon_conf_free(conf);
|
||||
}
|
||||
|
||||
void test_daemon_conf() {
|
||||
test_daemon_conf_create_defaults();
|
||||
test_daemon_conf_full_parse();
|
||||
@@ -645,4 +991,11 @@ void test_daemon_conf() {
|
||||
test_daemon_conf_module_count_capped();
|
||||
test_daemon_hosts_allowed();
|
||||
test_daemon_module_name_valid();
|
||||
test_daemon_conf_rsync_global_keys();
|
||||
test_daemon_conf_rsync_module_keys();
|
||||
test_daemon_conf_rsync_unknown_keys_rejected();
|
||||
test_daemon_conf_rsync_read_only_invalid();
|
||||
test_daemon_conf_dparam_rsync_keys();
|
||||
test_daemon_conf_read_only_default_and_opt_in();
|
||||
test_daemon_conf_unenforced_security_keys_warned();
|
||||
}
|
||||
@@ -9,7 +9,9 @@
|
||||
#include "charset.h"
|
||||
#include "utils.h"
|
||||
#include "protocol.h"
|
||||
#include "xattr.h"
|
||||
#include "test_utils.h"
|
||||
#include <errno.h>
|
||||
#include <fcntl.h>
|
||||
#include <limits.h>
|
||||
#include <stdlib.h>
|
||||
@@ -17,6 +19,7 @@
|
||||
#include <sys/stat.h>
|
||||
#include <sys/sysmacros.h>
|
||||
#include <sys/wait.h>
|
||||
#include <sys/xattr.h>
|
||||
#include <time.h>
|
||||
#include <unistd.h>
|
||||
|
||||
@@ -339,12 +342,17 @@ static void test_file_save_to_disk_temp_dir_confined() {
|
||||
const char* root = "test_temp_confine_tmp";
|
||||
const char* dest_file = "test_temp_confine_tmp/file.txt";
|
||||
char outside[PATH_MAX];
|
||||
char inside_abs[PATH_MAX];
|
||||
snprintf(outside, sizeof(outside), "/tmp/fastsync_temp_outside_%d", (int)getpid());
|
||||
unlink(dest_file);
|
||||
rmdir("test_temp_confine_tmp/scratch");
|
||||
rmdir("test_temp_confine_tmp/abs_scratch");
|
||||
rmdir(root);
|
||||
mkdir(root, 0755);
|
||||
mkdir("test_temp_confine_tmp/scratch", 0755);
|
||||
mkdir("test_temp_confine_tmp/abs_scratch", 0755);
|
||||
if (!realpath("test_temp_confine_tmp/abs_scratch", inside_abs))
|
||||
EXPECT_FAIL("realpath(abs_scratch) failed; inside_abs would be uninitialized");
|
||||
mkdir(outside, 0755);
|
||||
|
||||
File* f = file_create("file.txt");
|
||||
@@ -364,6 +372,13 @@ static void test_file_save_to_disk_temp_dir_confined() {
|
||||
config->temp_dir = str_dup("../escape");
|
||||
EXPECT_EQ_INT(file_save_to_disk_full(root, f, config), FILE_SAVE_ERROR);
|
||||
EXPECT_EQ_INT(access(dest_file, F_OK), -1);
|
||||
/* An absolute temp dir that canonicalizes INSIDE the receive root is
|
||||
accepted and used (the parity win); destination is still written. */
|
||||
free(config->temp_dir);
|
||||
config->temp_dir = str_dup(inside_abs);
|
||||
EXPECT_EQ_INT(file_save_to_disk_full(root, f, config), FILE_SAVE_WRITTEN);
|
||||
EXPECT_EQ_INT(access(dest_file, F_OK), 0);
|
||||
unlink(dest_file);
|
||||
free(config->temp_dir);
|
||||
config->temp_dir = str_dup("scratch");
|
||||
EXPECT_EQ_INT(file_save_to_disk_full(root, f, config), FILE_SAVE_WRITTEN);
|
||||
@@ -373,6 +388,7 @@ static void test_file_save_to_disk_temp_dir_confined() {
|
||||
config_delete(config);
|
||||
unlink(dest_file);
|
||||
rmdir("test_temp_confine_tmp/scratch");
|
||||
rmdir("test_temp_confine_tmp/abs_scratch");
|
||||
rmdir(root);
|
||||
rmdir(outside);
|
||||
}
|
||||
@@ -1315,6 +1331,103 @@ static void test_special_socket_recreated() {
|
||||
rmdir(root);
|
||||
}
|
||||
|
||||
/* --fake-super device round-trip (rsync parity): a char/block device must be
|
||||
* materialized as a REGULAR empty file whose user.rsync.%stat records the real
|
||||
* rdev -- never as an mknod'ed node -- even on a privileged receiver. This is
|
||||
* the non-privileged unit counterpart to the setpriv integration test (which
|
||||
* the PR gate excludes). */
|
||||
static void test_fake_super_device_writes_regular_file_with_rdev() {
|
||||
const char* root = "test_fake_super_dev_tmp";
|
||||
const char* node = "test_fake_super_dev_tmp/cdev";
|
||||
unlink(node);
|
||||
rmdir(root);
|
||||
EXPECT_EQ_INT(mkdir(root, 0700), 0);
|
||||
|
||||
Config* cfg = config_create();
|
||||
EXPECT_NOT_NULL(cfg);
|
||||
cfg->fake_super = true;
|
||||
cfg->preserve_devices = true;
|
||||
cfg->preserve_perms = true;
|
||||
cfg->use_metadata = true;
|
||||
cfg->use_xattrs = true;
|
||||
|
||||
FileMetadata meta;
|
||||
memset(&meta, 0, sizeof(meta));
|
||||
meta.mode = S_IFCHR | 0644;
|
||||
|
||||
File* f = file_create("cdev");
|
||||
EXPECT_NOT_NULL(f);
|
||||
f->is_special = true;
|
||||
f->rdev_major = 1;
|
||||
f->rdev_minor = 3;
|
||||
f->metadata = &meta;
|
||||
|
||||
EXPECT_EQ_INT(file_save_to_disk_full(root, f, cfg), FILE_SAVE_WRITTEN);
|
||||
|
||||
struct stat st;
|
||||
EXPECT_EQ_INT(lstat(node, &st), 0);
|
||||
EXPECT_TRUE(S_ISREG(st.st_mode)); /* never a real device node */
|
||||
EXPECT_EQ_INT((int)st.st_size, 0);
|
||||
|
||||
char value[128] = {0};
|
||||
ssize_t got = getxattr(node, FAKESUPER_XATTR, value, sizeof(value) - 1);
|
||||
EXPECT_TRUE(got > 0);
|
||||
EXPECT_EQ_STR(value, "20644 1,3 0:0"); /* the REAL rdev, not 0,0 */
|
||||
|
||||
f->metadata = NULL;
|
||||
file_destroy(f);
|
||||
config_delete(cfg);
|
||||
unlink(node);
|
||||
rmdir(root);
|
||||
}
|
||||
|
||||
/* A char/block device that mknodat() refuses (EPERM/EACCES on an unprivileged
|
||||
* receiver) must be a PER-ENTRY failure -- FILE_SAVE_FAILED, which the receiver
|
||||
* counts and continues past -- never the fatal FILE_SAVE_ERROR that aborts the
|
||||
* stream. The unit suite normally runs as root, so drop the effective uid to
|
||||
* make the kernel refusal deterministic. */
|
||||
static void test_device_mknod_failure_is_per_entry() {
|
||||
const char* root = "test_device_eperm_tmp";
|
||||
const char* node = "test_device_eperm_tmp/cdev";
|
||||
unlink(node);
|
||||
rmdir(root);
|
||||
EXPECT_EQ_INT(mkdir(root, 0777), 0);
|
||||
|
||||
Config* cfg = config_create();
|
||||
EXPECT_NOT_NULL(cfg);
|
||||
cfg->preserve_devices = true;
|
||||
cfg->use_metadata = true;
|
||||
|
||||
FileMetadata meta;
|
||||
memset(&meta, 0, sizeof(meta));
|
||||
meta.mode = S_IFCHR | 0644;
|
||||
|
||||
File* f = file_create("cdev");
|
||||
EXPECT_NOT_NULL(f);
|
||||
f->is_special = true;
|
||||
f->rdev_major = 1;
|
||||
f->rdev_minor = 3;
|
||||
f->metadata = &meta;
|
||||
|
||||
uid_t saved = geteuid();
|
||||
bool dropped = false;
|
||||
if (saved == 0 && seteuid(65534) == 0)
|
||||
dropped = true;
|
||||
FileSaveResult result = file_save_to_disk_full(root, f, cfg);
|
||||
if (dropped)
|
||||
EXPECT_EQ_INT(seteuid(saved), 0);
|
||||
|
||||
EXPECT_EQ_INT(result, FILE_SAVE_FAILED);
|
||||
/* Nothing was created: no device node and no regular-file fallback. */
|
||||
struct stat st;
|
||||
EXPECT_EQ_INT(lstat(node, &st), -1);
|
||||
|
||||
f->metadata = NULL;
|
||||
file_destroy(f);
|
||||
config_delete(cfg);
|
||||
rmdir(root);
|
||||
}
|
||||
|
||||
static void test_inplace_overwrite_truncates_shorter_payload() {
|
||||
const char* root = "test_inplace_trunc_tmp";
|
||||
const char* path = "test_inplace_trunc_tmp/big.txt";
|
||||
@@ -2396,6 +2509,8 @@ void test_file() {
|
||||
test_new_file_mode_honors_source_and_umask();
|
||||
test_special_fifo_mode_honors_source_and_umask();
|
||||
test_special_socket_recreated();
|
||||
test_fake_super_device_writes_regular_file_with_rdev();
|
||||
test_device_mknod_failure_is_per_entry();
|
||||
test_inplace_overwrite_truncates_shorter_payload();
|
||||
test_inplace_refuses_fifo_destination();
|
||||
test_inplace_refuses_device_destination();
|
||||
|
||||
+80
-8
@@ -156,8 +156,8 @@ static void test_xattr_capture_and_appliable() {
|
||||
EXPECT_TRUE(xattr_name_appliable("user.foo", false));
|
||||
EXPECT_TRUE(xattr_name_appliable("user.foo", true));
|
||||
/* The reserved fake-super key is receiver-only and never forwarded/applied. */
|
||||
EXPECT_FALSE(xattr_name_appliable("user.fastsync.stat", false));
|
||||
EXPECT_FALSE(xattr_name_appliable("user.fastsync.stat", true));
|
||||
EXPECT_FALSE(xattr_name_appliable("user.rsync.%stat", false));
|
||||
EXPECT_FALSE(xattr_name_appliable("user.rsync.%stat", true));
|
||||
/* B4: the ACL names require --acls; -X alone must not authorize them. */
|
||||
EXPECT_FALSE(xattr_name_appliable("system.posix_acl_access", false));
|
||||
EXPECT_FALSE(xattr_name_appliable("system.posix_acl_default", false));
|
||||
@@ -351,9 +351,10 @@ static void test_xattr_capture_filters_acls() {
|
||||
}
|
||||
|
||||
/* --fake-super replay: fake_super_store_fd records the source stat into the
|
||||
* reserved xattr, and fake_super_restore_fd re-applies mode/mtime (and owner,
|
||||
* when the process may) fd-relative. Restore must also be a safe no-op with no
|
||||
* xattr present. Guarded on filesystem xattr support. */
|
||||
* reserved xattr, and fake_super_restore_fd re-applies the permission bits
|
||||
* fd-relative (mtime travels through the normal metadata path; the owner is
|
||||
* never chowned). Restore must also be a safe no-op with no xattr present.
|
||||
* Guarded on filesystem xattr support. */
|
||||
static void test_fake_super_restore() {
|
||||
const char* path = "test_fake_super_restore.txt";
|
||||
unlink(path);
|
||||
@@ -373,7 +374,7 @@ static void test_fake_super_restore() {
|
||||
FileAttrPolicy policy = {true, true, false, false, true};
|
||||
EXPECT_FALSE(fake_super_restore_fd(fd, policy));
|
||||
|
||||
fake_super_store_fd(fd, 1001, 1002, 0751, 1700000000, 123456789);
|
||||
fake_super_store_fd(fd, 1001, 1002, S_IFREG | 0751, 0, 0);
|
||||
EXPECT_TRUE(fake_super_restore_fd(fd, policy));
|
||||
struct stat st;
|
||||
EXPECT_EQ_INT(fstat(fd, &st), 0);
|
||||
@@ -381,7 +382,7 @@ static void test_fake_super_restore() {
|
||||
|
||||
/* Strict rsync parity: -p restores the recorded mode exactly, including
|
||||
group/other write (a recorded 0666 restores as 0666). */
|
||||
fake_super_store_fd(fd, 1001, 1002, 0666, 1700000000, 0);
|
||||
fake_super_store_fd(fd, 1001, 1002, S_IFREG | 0666, 0, 0);
|
||||
EXPECT_TRUE(fake_super_restore_fd(fd, policy));
|
||||
EXPECT_EQ_INT(fstat(fd, &st), 0);
|
||||
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0666);
|
||||
@@ -401,6 +402,76 @@ static void test_fake_super_restore() {
|
||||
unlink(path);
|
||||
}
|
||||
|
||||
/* The stored record is rsync 3.4.1's exact grammar
|
||||
* "<octal st_mode with S_IFMT> <rdev_major>,<rdev_minor> <uid>:<gid>"
|
||||
* so a fake-super tree is readable by rsync. Also pins two rsync parity
|
||||
* rules: the special bits are stored in the record but NOT applied to the real
|
||||
* file, and a device record's rdev round-trips through the parser. Guarded on
|
||||
* filesystem xattr support. */
|
||||
static void test_fake_super_rsync_format() {
|
||||
const char* path = "test_fake_super_format.txt";
|
||||
unlink(path);
|
||||
int fd = open(path, O_WRONLY | O_CREAT | O_TRUNC, 0600);
|
||||
if (fd < 0)
|
||||
return;
|
||||
bool has_xattr = setxattr(path, "user.fastsync.xprobe", "p", 1, 0) == 0;
|
||||
if (has_xattr)
|
||||
removexattr(path, "user.fastsync.xprobe");
|
||||
if (!has_xattr) {
|
||||
close(fd);
|
||||
unlink(path);
|
||||
return; /* skip silently when the filesystem has no xattr support */
|
||||
}
|
||||
|
||||
/* A setuid regular file: the full st_mode (with S_IFMT + special bits) is
|
||||
recorded, rdev is 0,0, and the owner is uid:gid. */
|
||||
fake_super_store_fd(fd, 1234, 5678, S_IFREG | 04711, 0, 0);
|
||||
char value[128];
|
||||
ssize_t got = fgetxattr(fd, FAKESUPER_XATTR, value, sizeof(value));
|
||||
EXPECT_EQ_INT((int)got, 20);
|
||||
EXPECT_TRUE(got == 20 && memcmp(value, "104711 0,0 1234:5678", 20) == 0);
|
||||
|
||||
/* The special bits in the record are NOT installed on the real file. */
|
||||
FileAttrPolicy policy = {true, true, false, false, true};
|
||||
EXPECT_TRUE(fake_super_restore_fd(fd, policy));
|
||||
struct stat st;
|
||||
EXPECT_EQ_INT(fstat(fd, &st), 0);
|
||||
EXPECT_EQ_INT((int)(st.st_mode & 07777), 0711);
|
||||
EXPECT_EQ_INT((int)(st.st_mode & (S_ISUID | S_ISGID | S_ISVTX)), 0);
|
||||
|
||||
/* A device record (char 1,3, uid 111, gid 222) parses without error and
|
||||
still never real-chowns or installs the device's mode bits verbatim. */
|
||||
EXPECT_EQ_INT((int)fsetxattr(fd, FAKESUPER_XATTR, "20644 1,3 111:222", 17, 0), 0);
|
||||
struct stat before;
|
||||
fstat(fd, &before);
|
||||
EXPECT_TRUE(fake_super_restore_fd(fd, policy));
|
||||
fstat(fd, &st);
|
||||
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0644);
|
||||
EXPECT_EQ_INT((int)st.st_uid, (int)before.st_uid);
|
||||
EXPECT_EQ_INT((int)st.st_gid, (int)before.st_gid);
|
||||
|
||||
/* Hardened parser: an out-of-range field (previously UB via sscanf("%u")),
|
||||
a missing field, or trailing garbage is rejected cleanly instead of being
|
||||
silently accepted. */
|
||||
const char* malformed[] = {
|
||||
"20644 65536,3 111:222", /* major > 0xffff */
|
||||
"20644 1,16777216 111:222", /* minor > 0xffffff */
|
||||
"20644 1,3 111:222 trailing", /* trailing garbage */
|
||||
"20644 1,3 111", /* missing gid */
|
||||
"20644 1,3 4294967296:222", /* uid > UINT_MAX */
|
||||
"20644 1,3 111:4294967296", /* gid > UINT_MAX */
|
||||
"99999999999999999999 1,3 0:0", /* mode overflow */
|
||||
"", /* empty record */
|
||||
};
|
||||
for (size_t i = 0; i < sizeof(malformed) / sizeof(malformed[0]); i++) {
|
||||
EXPECT_EQ_INT((int)fsetxattr(fd, FAKESUPER_XATTR, malformed[i], strlen(malformed[i]), 0), 0);
|
||||
EXPECT_FALSE(fake_super_restore_fd(fd, policy));
|
||||
}
|
||||
|
||||
close(fd);
|
||||
unlink(path);
|
||||
}
|
||||
|
||||
/* --fake-super must NEVER perform a real chown: fake_super_restore_fd applies
|
||||
* only mode/mtime and leaves the entry's uid/gid exactly as they were, even
|
||||
* when an explicit ownership policy is active and super_mode permits it. This
|
||||
@@ -421,7 +492,7 @@ static void test_fake_super_no_real_chown() {
|
||||
}
|
||||
struct stat before;
|
||||
EXPECT_EQ_INT(fstat(fd, &before), 0);
|
||||
fake_super_store_fd(fd, 12345, 12346, 0755, 1700000000, 0);
|
||||
fake_super_store_fd(fd, 12345, 12346, S_IFREG | 0755, 0, 0);
|
||||
|
||||
Config* c = config_create();
|
||||
FileAttrPolicy policy = {true, true, false, false, true};
|
||||
@@ -600,6 +671,7 @@ void test_xattr() {
|
||||
test_xattr_receive_drops_acl_without_preserve_acls();
|
||||
test_link_copy_fallback_preserves_xattrs();
|
||||
test_fake_super_restore();
|
||||
test_fake_super_rsync_format();
|
||||
test_fake_super_no_real_chown();
|
||||
test_fake_super_storage_resolution();
|
||||
test_file_save_directory_applies_xattrs();
|
||||
|
||||
Reference in New Issue
Block a user