Release v2.29.0 #312

Merged
TapTap merged 123 commits from dev into main 2026-09-23 02:05:14 +02:00
8 changed files with 257 additions and 46 deletions
Showing only changes of commit 8792e3265a - Show all commits
+2 -2
View File
@@ -219,7 +219,7 @@ This produces `./build/client` and `./build/server`. `compile_commands.json` is
| `--delete-excluded` | Also delete filter-excluded destination mirrors (size-pruned mirrors stay protected) |
| `--max-delete <n>` | Delete at most n destination entries; the rest are skipped and the run exits 25 (partial), matching rsync |
| `--delay-updates` | Put updated files into place only at the end of the transfer (`--force` is honored at publication; the fixed `.fastsync-stage` staging name diverges from rsync — see [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md)) |
| `-T, --temp-dir <dir>` | Scratch directory for temp files before the atomic install; confined to the receive root (relative only), with an `EXDEV` non-atomic copy fallback |
| `-T, --temp-dir <dir>` | Scratch directory for temp files before the atomic install; confined to the receive root (a relative path resolves below it; an absolute path is accepted only when it canonicalizes inside it), with an `EXDEV` non-atomic copy fallback |
| `-n, --dry-run` | Report what would be transferred without mutating the destination. Since protocol 2.21.0 a server-routed target contacts the receiver and reports would-transfer based on receiver state; a plain local destination keeps the client-side scan. Never mutates or deletes. |
| `-v, --verbose` | Enable debug logging |
| `-q, --quiet` | Suppress non-error output |
@@ -606,7 +606,7 @@ remote SSH argv is already built injection-safe.
| `--max-alloc <SIZE>` | Maximum single allocation (binary units; default 1G; `0` = no local limit). |
| `--max-depth <n>` | Limit recursive scanning depth; zero means unlimited. |
| `-b, --backup` | Back up overwritten files. |
| `-T, --temp-dir <dir>` | Scratch directory for temp files before the atomic install (confined to the receive root; `EXDEV` falls back to a non-atomic copy). |
| `-T, --temp-dir <dir>` | Scratch directory for temp files before the atomic install (confined to the receive root: relative resolves below it, absolute must canonicalize inside it; `EXDEV` falls back to a non-atomic copy). |
| `--backup-dir <dir>` | Store backups under a separate directory (requires `--backup`). |
| `--suffix <suffix>` | Set the backup filename suffix (default: `~`). |
| `--partial` | Select partial-transfer handling. On failed/interrupted writes the already-written temp file is retained (best-effort) for resumption. With `--partial --partial-dir <dir>`, completed files are written under the partial directory and installed atomically. |
+18 -12
View File
@@ -6,15 +6,15 @@ This document maps rsync's full feature set to FastSync's current implementation
| Status | Count | Description |
|--------|-------|-------------|
| ✅ Parity | 117 | Reproduces rsync's semantics for this option's scope |
| ✅ Parity | 118 | Reproduces rsync's semantics for this option's scope |
| ⚠️ Caveat | 13 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
| ❌ Divergent | 27 | Rejected, an accepted no-op, deliberately non-rsync (native config/auth/batch, privileged namespaces, safe-subset privilege), or impossible on any portable filesystem call |
| ❌ Divergent | 26 | Rejected, an accepted no-op, deliberately non-rsync (native config/auth/batch, privileged namespaces, safe-subset privilege), or impossible on any portable filesystem call |
| **Total** | **157** | One row per rsync option/feature group; a row may name several spellings |
This matrix reports honest rsync parity, not "implemented" as a synonym for
"parsed". A ✅ row matches rsync for the option's scope. A ⚠️ row is real and
tested but diverges in at least one documented way. An ❌ row is either
rejected (`--protocol` with any value but the current one, `--inc-recursive`),
rejected (`--protocol` with any value but the current one),
an accepted no-op (`-s`/`--secluded-args`, `--protect-args`, `--old-args`),
deliberately non-rsync and non-interoperable (the FastSync daemon config/auth,
the batch container, `--fake-super`'s xattr format, `--copy-as` credential
@@ -161,7 +161,7 @@ Every one of those has an entry below with its remaining caveats.
| `--no-implied-dirs` | Don't send implied dirs with -R | ✅ Parity | With `-R`, rsync creates the ancestor directories implied by a listed path and, with `--no-implied-dirs`, omits their attributes from the transfer so they keep the destination's own state (or are created with default attributes when absent). Protocol 2.26.0 matches this: without `--files-from` the implied-dir walk applies per-attribute metadata only to explicitly transferred directories, and with `-R --files-from` a listed file whose parent is not itself listed is placed normally — the missing implied parent is created with default attributes (not the source's) and the file transfers with `rc 0`, exactly like rsync 3.4.1 (a differential test verifies the modes and mtimes with and without the flag). Works single-threaded and under `-j`/`--threads` |
| `-d`, `--dirs`, `--old-dirs`, `--old-d` | Transfer dirs without recursing | ✅ Parity | Protocol 2.26.0 implements rsync's one-level `-d` listing for `dir`, `dir/` and `.`: the source's immediate contents are transferred (files with content, directories as explicit entries), matching rsync's destination tree in a differential test. `--dirs --files-from` transfers exactly the listed items — a listed directory is created empty and a listed file with content — under the same `-R` layout rules. A plain recursive scan also recreates empty source directories now: the scanner emits a payload-less directory entry (with metadata) for every traversed directory that produced no transferred or descended child, unless `-m/--prune-empty-dirs` suppresses it or the run is `--files-from`/`--list-only` (a directory emptied by filtering is recreated too, matching rsync). Directory entries cross as `STATUS_MKDIR` and appear in the delete manifest, so `--delete` prunes correctly and an empty listed directory survives; an incoming directory replaces a destination regular file (rsync removes the non-directory and creates the directory), verified differentially. Directory times are applied at the end of the transfer; modes/ownership follow the per-attribute policy. Under `--delay-updates` directories are created immediately while only regular files are staged, exactly as rsync does |
| `--mkpath` | Create missing path components | ✅ Parity | Wire option (client → server). At connection start the server creates the client's destination root directory (and any missing leading components below its own authorized root) when `--mkpath` is set, failing the connection cleanly if it cannot. Without `--mkpath` a destination root that does not exist yet is rejected up front (rsync semantics), so the flag is the only way to transfer into a not-yet-created destination directory. Creation is confined by the same secure mkdir walk as file writes (`O_NOFOLLOW`, no `..`) |
| `--inc-recursive`, `--no-inc-recursive` | Incremental recursion mode | ❌ Divergent | rsync's man-page-only scanning-mode switch (and its short aliases). FastSync always performs a single full recursive scan, so both spellings are rejected as unknown options rather than accepted as a no-op; there is no incremental-recursion engine to toggle. A genuine implementation would be a scan-architecture change with no benefit for FastSync's push model |
| `--inc-recursive`, `--no-inc-recursive` | Incremental recursion mode | ✅ Parity | rsync's man-page-only scanning-mode switch. FastSync always performs a single full recursive scan, so both spellings are accepted as inert no-ops and the destination is identical whichever mode the caller requests — the same treatment as `-r`/`--recursive`, which is likewise a no-op. The switch is a scan-implementation detail with no observable effect on the final tree (rsync's own `--no-inc-recursive` selects a full scan, which is exactly FastSync's behavior) |
## 5. Transfer Modifications
@@ -184,7 +184,7 @@ Every one of those has an entry below with its remaining caveats.
| `--backup-dir=DIR` | Backup directory hierarchy | ✅ Parity | `backup_dir` config field |
| `--suffix=SUFFIX` | Backup suffix (default ~) | ✅ Parity | `suffix` config field |
| `--delay-updates` | Put updated files in place at end | ❌ Divergent | Successfully received files are staged under a private 0700 `.fastsync-stage` dir inside the receive root and atomically renamed into their final destinations only after the whole transfer (manifest/delete handling included) succeeds, just before the success/outcome frame is sent. The delete walker deliberately skips the staging dir at the receive root, so `--delete` removes genuine extras but never the staged files (deletion runs before publication; rsync's delete-after ordering is not implemented). `--existing`/`--ignore-existing`/`--update` decide against the final destination path at stage time; `--backup` moves the old file aside at publication, and **`--force` is honored at publication** (protocol 2.23.0): a staged regular file or symlink may replace a destination directory that blocks it. Incompatible with `--inplace` and with `--backup-dir=.fastsync-stage` (the internal staging name is reserved; both are rejected). The staging dir name is fixed, so two simultaneous delayed transfers to the same destination root are serialized with an exclusive advisory lock held for the whole transfer: the second session fails cleanly instead of corrupting the first. Aborting or failing before publication installs nothing and removes the staging tree; a crash between stage and publish leaves staged leftovers that the next delayed run wipes at start (process death releases the lock). A stage→publish failure aborts the transfer (best-effort cleanup of the not-yet-published staged files; already-published files are not rolled back). **Reclassified Divergent (differential evidence):** the staging name is fixed and a delayed run wipes a pre-existing destination tree of that name at start even without `--delete`, whereas rsync uses its own internal temp name and leaves a genuine destination entry named `.fastsync-stage` untouched (`test_delay_updates_staging_name_collision_residual`); deletion also runs before publication while rsync's `--delay-updates` implies `--delete-after`. Works in single-threaded and `-j`/`--threads` modes |
| `-T`, `--temp-dir=DIR` | Create temporary files in DIR | ❌ Divergent | `--temp-dir` with the rsync short `-T` (the timeout alias moved to long-only `--timeout`). A **relative** dir matches rsync exactly: it is resolved below the receive/destination root and must already exist (differentially verified: `rsync -a --temp-dir=scratch src/ dst/` and FastSync produce identical trees and an empty scratch dir). **Reclassified as a deliberate divergence because an absolute `--temp-dir` is rejected by the receiver** — it is resolved verbatim by rsync standalone (which will use `/tmp` or any other absolute directory, including one outside the destination), but FastSync's security-reviewed receiver confines the scratch dir to the authorized receive root and rejects any absolute path or one containing `..`. **Audit-cycle hardening:** the opened dir is additionally judged by the real path of its fd (`/proc/self/fd`), so a client-planted symlink under the receive root cannot redirect receiver scratch files outside the authorized root (an escaping target is refused with `EACCES`), while an in-root symlink to another filesystem — the `EXDEV` fallback case — still works. A differential test confirms rsync exits 0 using an absolute scratch dir while FastSync refuses before writing anything into it (the scratch dir stays empty). Its daemon mode also confines relative to the module, but standalone rsync's absolute-temp-dir behavior is not reproduced because it would let a client place receiver scratch files outside the sandbox. Temp copies use a unique name in the scratch dir and are atomically renamed into place; **on `EXDEV` (scratch dir and destination on different filesystems, reachable via a confined relative symlink) the receiver falls back to a non-atomic copy instead of aborting**, matching rsync. `--inplace` and `--partial-dir` writes bypass the scratch dir |
| `-T`, `--temp-dir=DIR` | Create temporary files in DIR | ❌ Divergent | `--temp-dir` with the rsync short `-T` (the timeout alias moved to long-only `--timeout`). A **relative** dir matches rsync exactly: it is resolved below the receive/destination root and must already exist (differentially verified: `rsync -a --temp-dir=scratch src/ dst/` and FastSync produce identical trees and an empty scratch dir). An **absolute** dir is accepted when it canonicalizes (`realpath(3)`) inside the receive root, so an in-root absolute scratch path is usable and used (unit- and integration-tested for both the local batch apply and a live TCP transfer). **Remaining divergence:** an absolute `--temp-dir` that escapes the receive root is rejected, and so is a relative one containing `..` — rsync standalone resolves an absolute `--temp-dir` verbatim (it will use `/tmp` or any other directory, including one outside the destination), but FastSync's security-reviewed receiver confines the scratch dir to the authorized receive root and refuses an out-of-root path before writing anything. **Audit-cycle hardening:** the opened dir is additionally judged by the real path of its fd (`/proc/self/fd`), so a client-planted symlink under the receive root cannot redirect receiver scratch files outside the authorized root (an escaping target is refused with `EACCES`), while an in-root symlink to another filesystem — the `EXDEV` fallback case — still works. A differential test confirms rsync exits 0 using an out-of-root absolute scratch dir while FastSync refuses before writing anything into it (the scratch dir stays empty). Its daemon mode also confines relative to the module. Temp copies use a unique name in the scratch dir and are atomically renamed into place; **on `EXDEV` (scratch dir and destination on different filesystems, reachable via a confined relative symlink) the receiver falls back to a non-atomic copy instead of aborting**, matching rsync. `--inplace` and `--partial-dir` writes bypass the scratch dir |
| `--partial` | Keep partially transferred files | ✅ Parity | On a failed/interrupted write the already-written temp file is retained at the destination path (best-effort rename instead of unlink) so a later `--append`/`--append-verify` run can resume it. Retention never runs when no data was actually written or under `--ignore-existing`/`--existing` (the destination is not ours to overwrite), and it only ever renames the already-written temp. A failed rename falls back to the normal unlink |
| `--partial-dir=DIR` | Keep partial files in DIR | ✅ Parity | The working file is written under the confined partial directory (a relative dir below the receive root) and atomically renamed into place once complete, so an interrupted transfer leaves a resumable copy there and completed transfers do not linger under it. `--inplace` bypasses the partial dir (rsync parity), and combining `--inplace` with `--partial-dir` is now **rejected up front** with rsync's message (`--inplace cannot be used with --partial-dir`) instead of silently ignoring the partial dir. **Implies `--partial`** (audit-cycle fix, matching rsync 3.4.1, which sets `keep_partial` after option parsing): `--partial-dir=DIR` alone retains an interrupted transfer's partial, and the implication wins over an explicit `--no-partial` regardless of order |
@@ -956,7 +956,7 @@ These are the last compatibility items and the closing phase toward rsync flag p
**Wire:** two trailing config-frame blocks after the `--iconv` spec, in fixed order — `send_privilege_options`/`receive_privilege_options` (one `super_mode` int, validated `0..2`), then `send_copy_as_options`/`receive_copy_as_options` (presence int + two int32 ids, validated `>= 0`, with `copy_as_set ⇒ use_metadata`). `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Divergences from rsync:** rsync's `--super` elevates the receiver and `--copy-as` actually switches its credentials; FastSync never elevates and only permits/forwards confined attempts, and `--copy-as` forces ownership rather than switching identity.
**Honest status after the parity 2.29 cycle (protocol 2.28.0, no wire change), updated by the parity cycle 2.29 pass, the audit-cycle follow-ups, and the triage cycle.** ✅ Parity 117 / ⚠️ Caveat 13 / ❌ Divergent 27 = 157 rows. The 2.29 cycle closed the scanner-order, delete-timing, relative-basis, and fuzzy-eligibility residuals (moving `-n`/`--delete`/`--del`/`--delete-delay` to ✅) and improved the `--info`/`--stats`/`--debug` partial rows; the triage cycle moved `-F` and `-i`/`--itemize-changes` ✅ → ⚠️ for their documented residuals. The remaining ⚠️ rows are `--info`, `--debug`, `--msgs2stderr`, `--stats`, `--progress`, `-i`, `--delete-before`, `--filter`, `-F`, the three basis-dir options, and `-y/--fuzzy`. Earlier: **Honest status after the parity 2.28.0 cycle (protocol 2.28.0), updated by the rsync-parity-stats, rsync-parity-options, rsync-parity-fs, parity-review, no-wire parity-track-1/2b and wire parity-track-4a/5a passes.** ✅ Parity 116 / ⚠️ Caveat 14 / ❌ Divergent 27 = 157 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. This reclassification makes every difference explicit. The completion wave closed 23 previously-caveated rows (9 that triage showed were already parity, plus 14 genuine fixes) and turned the 17 inherently non-rsync rows — native daemon config/auth, the FastSync batch container, the safe-subset device/privilege flags, `-X`'s privileged namespaces, `--fake-super`'s native xattr format, and the `--old-args` no-op — into explicit ❌ divergences. The stats pass flipped `--delete-delay` to ✅ (actual-removal accounting), but the parity-review pass moved it back to ⚠️ because FastSync charged the `--max-delete` budget at plan/snapshot time and left a refilled snapshotted directory in place, whereas rsync charges on actual removals and recursively removes a queued directory (including content created after its plan). The no-wire parity-track-1 pass fixed both (actual-removal charging plus recursive deferred removal with an independent deferred-list cap), narrowing the caveat to the partial-delete ordering. The stats pass also reclassified `--out-format` to ❌ (protocol-specific `%b`/delta-`%c`), and sharpened the `--stats`/`--progress`/`--checksum-choice` residuals. The options pass flipped `--bwlimit` and `--ignore-errors` to ✅ (rsync-exact size parsing and ~100 ms leaky-bucket throttling, and rsync's skip-unreadable-subdir plus IO-error-suppressed deletion with exit 23) and emits rsync-format `--info=name/flist/del/remove/nonreg/progress` lines (real-run `deleting`/`*deleting` carried over a new trailing `report_deletes` wire bool, `PROTOCOL_VERSION` 2.26.0 → 2.27.0), while reclassifying `-M` over daemon/TCP
**Honest status after the parity 2.29 cycle (protocol 2.28.0, no wire change), updated by the parity cycle 2.29 pass, the audit-cycle follow-ups, the triage cycle, and a later no-wire CLI parity fix.** ✅ Parity 118 / ⚠️ Caveat 13 / ❌ Divergent 26 = 157 rows. A no-wire CLI-parity pass accepted `--inc-recursive`/`--no-inc-recursive` as inert no-ops (❌ → ✅, since FastSync's full scan is rsync's `--no-inc-recursive` and the destination is identical) and narrowed the `--temp-dir` divergence by accepting an absolute path that canonicalizes inside the receive root (the row stays ❌ for out-of-root absolute paths). The 2.29 cycle closed the scanner-order, delete-timing, relative-basis, and fuzzy-eligibility residuals (moving `-n`/`--delete`/`--del`/`--delete-delay` to ✅) and improved the `--info`/`--stats`/`--debug` partial rows; the triage cycle moved `-F` and `-i`/`--itemize-changes` ✅ → ⚠️ for their documented residuals. The remaining ⚠️ rows are `--info`, `--debug`, `--msgs2stderr`, `--stats`, `--progress`, `-i`, `--delete-before`, `--filter`, `-F`, the three basis-dir options, and `-y/--fuzzy`. Earlier: **Honest status after the parity 2.28.0 cycle (protocol 2.28.0), updated by the rsync-parity-stats, rsync-parity-options, rsync-parity-fs, parity-review, no-wire parity-track-1/2b and wire parity-track-4a/5a passes.** ✅ Parity 116 / ⚠️ Caveat 14 / ❌ Divergent 27 = 157 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. This reclassification makes every difference explicit. The completion wave closed 23 previously-caveated rows (9 that triage showed were already parity, plus 14 genuine fixes) and turned the 17 inherently non-rsync rows — native daemon config/auth, the FastSync batch container, the safe-subset device/privilege flags, `-X`'s privileged namespaces, `--fake-super`'s native xattr format, and the `--old-args` no-op — into explicit ❌ divergences. The stats pass flipped `--delete-delay` to ✅ (actual-removal accounting), but the parity-review pass moved it back to ⚠️ because FastSync charged the `--max-delete` budget at plan/snapshot time and left a refilled snapshotted directory in place, whereas rsync charges on actual removals and recursively removes a queued directory (including content created after its plan). The no-wire parity-track-1 pass fixed both (actual-removal charging plus recursive deferred removal with an independent deferred-list cap), narrowing the caveat to the partial-delete ordering. The stats pass also reclassified `--out-format` to ❌ (protocol-specific `%b`/delta-`%c`), and sharpened the `--stats`/`--progress`/`--checksum-choice` residuals. The options pass flipped `--bwlimit` and `--ignore-errors` to ✅ (rsync-exact size parsing and ~100 ms leaky-bucket throttling, and rsync's skip-unreadable-subdir plus IO-error-suppressed deletion with exit 23) and emits rsync-format `--info=name/flist/del/remove/nonreg/progress` lines (real-run `deleting`/`*deleting` carried over a new trailing `report_deletes` wire bool, `PROTOCOL_VERSION` 2.26.0 → 2.27.0), while reclassifying `-M` over daemon/TCP
and receiver-side `protect`/`risk` re-derivation to ❌ (no argv channel /
receiver filter engine); the wire parity-track-4a pass later added that
receiver filter engine, flipping `--filter=RULE` back to ✅ (see above; the
@@ -1021,7 +1021,9 @@ integration tests unless it is explicitly listed as a limitation.
### Filesystem and deletion semantics
- **`--temp-dir` is confined to the receive root on the receiver:** a relative
dir resolves below it; an absolute path or one containing `..` is rejected.
dir resolves below it, and an absolute path is accepted only when
`realpath(3)` confirms it is inside the canonical receive root; an
out-of-root absolute path or one containing `..` is rejected.
An `EXDEV` install falls back to a non-atomic copy instead of aborting. (The
confined receiver path cannot be mount-tested in the CI container — no
`CAP_SYS_ADMIN` and unprivileged user namespaces are disabled — so the
@@ -1095,8 +1097,9 @@ These remain after the wave; they are the reasons a row above is ⚠️.
link-following tool can follow a link outside the receive root. Use
`--safe-links` when the source is untrusted. `--trust-sender` does **not**
affect symlink targets.
- **`--temp-dir` absolute/foreign-filesystem paths are rejected by the
receiver** (rsync's daemon also confines; standalone rsync differs).
- **`--temp-dir` out-of-root absolute and foreign-filesystem paths are rejected
by the receiver** (an absolute path that canonicalizes inside the receive root
is accepted; rsync's daemon also confines; standalone rsync differs).
- **`--copy-devices` reads a bounded `st_size`** rather than rsync's unbounded
device read.
- **A broken symlink referent under `--copy-links`/`--copy-unsafe-links` exits 0**
@@ -1235,8 +1238,9 @@ These remain after the wave; the individual rows carry the precise wording.
`--ignore-errors` exits 23 but its EACCES differential is not
exercised in CI.
- **`--delay-updates`** uses a fixed staging name with an advisory lock and
deletes before publication; **`--temp-dir`** rejects absolute/foreign paths
(deliberately confined, see the row); **`--remote-option`** is SSH-only.
deletes before publication; **`--temp-dir`** rejects out-of-root absolute and
foreign paths (in-root absolute paths are accepted; deliberately confined, see
the row); **`--remote-option`** is SSH-only.
**`--iconv`** now matches rsync's push direction (destination charset = the
spec's REMOTE half; a server `--iconv` overrides it).
- **Basis dirs** now use rsync's metadata quick-check by default (track 5a) and
@@ -1247,7 +1251,9 @@ These remain after the wave; the individual rows carry the precise wording.
engine (both files ≥ 16 KiB, size ratio ≤ 10×), a narrower window than
rsync's, so the selected basis — and the `--stats` bandwidth counters —
can differ while the tree stays byte-exact.
- **`--inc-recursive`/`--no-inc-recursive`** are not implemented (rejected).
- **`--inc-recursive`/`--no-inc-recursive`** are accepted as inert no-ops:
FastSync always performs a single full recursive scan (equivalent to
rsync's `--no-inc-recursive`), so the destination is identical either way.
### Intentional divergences (explicit ❌ rows)
+13
View File
@@ -962,6 +962,13 @@ static const OptionEntry OPTION_TABLE[] = {
/* rsync -r/--recursive: FastSync is always recursive, so this is a
* faithful no-op (accepted silently, never consumes an argument). */
{"--recursive", "-r", OPT_NOOP, 0},
/* rsync's incremental-recursion scan-mode switch. FastSync always performs
* a single full recursive scan, so both spellings are accepted as no-ops:
* the destination is identical whichever mode the caller requests.
* --no-inc-recursive is handled before the generic --no-* negation branch
* (see cli_handle_pre_negation) but is registered here for discoverability. */
{"--inc-recursive", NULL, OPT_NOOP, 0},
{"--no-inc-recursive", NULL, OPT_NOOP, 0},
{"--update", "-u", OPT_FLAG, offsetof(Config, update)},
/* rsync's --old-args: accepted for CLI compatibility as a documented no-op
* (the remote server path is always safely quoted; see usage.c). It is
@@ -1387,6 +1394,12 @@ static bool cli_handle_pre_negation(CliParseCtx* ctx) {
ctx->no_delta = true;
else if (strcmp(arg, "--no-incremental") == 0)
ctx->no_incremental = true;
/* Real rsync option names that merely start with "--no-" and are inert
* no-ops (e.g. --no-inc-recursive) are registered as OPT_NOOP entries;
* accept them before the generic negation table would reject the name. */
const OptionEntry* noop = find_table_option(arg);
if (noop && noop->kind == OPT_NOOP)
return true;
if (apply_negation(config, arg) != 0) {
ctx->exit_code = -1;
return true;
+3
View File
@@ -26,6 +26,9 @@ void print_usage(void) {
printf(" owner, group, devices and specials; not\n");
printf(" compression/multithreading\n");
printf(" -r, --recursive Recurse into directories (FastSync is always recursive)\n");
printf(" --inc-recursive Accepted for rsync CLI compatibility; no effect (FastSync\n");
printf(" always performs a full scan, so the destination is identical)\n");
printf(" --no-inc-recursive Accepted for rsync CLI compatibility; no effect\n");
printf(" -n, --dry-run Show what would be transferred\n");
printf(" --remove-source-files Remove regular source files after successful transfer\n");
printf(" -p, --perms Preserve permission bits\n");
+84 -29
View File
@@ -2,6 +2,7 @@
#include <ctype.h>
#include <dirent.h>
#include <fcntl.h>
#include <limits.h>
#include <libgen.h>
#include <stdio.h>
#include <stdlib.h>
@@ -198,6 +199,56 @@ static FileSaveResult hardlink_sibling_absent_first(const char* destination_path
return FILE_SAVE_ERROR;
}
/* Resolve a user-supplied --temp-dir against the receive `root`.
*
* A relative, traversal-free name is joined below the root (the historical
* behavior). An absolute path is canonicalized with realpath(3) and accepted
* only when it lies inside the canonicalized receive root; this is the parity
* win over rejecting every absolute path, without weakening the confinement
* invariant: an absolute path that escapes the root (including one reached
* through a symlinked component) is still refused. A `..` component in a
* relative name is likewise refused. The root itself is treated as an
* absolute path free of `..`; its realpath() resolves any symlinks so the
* prefix comparison is against one canonical form.
*
* Logs a clear error on rejection (the scratch dir must stay confined) and
* returns a newly allocated scratch path, or NULL on rejection/allocation
* failure. */
static char* file_save_resolve_temp_dir(const char* root, const char* temp_dir) {
if (temp_dir[0] != '/') {
if (has_path_traversal(temp_dir)) {
log_message(
LOG_LEVEL_ERROR,
"receiver rejected --temp-dir '%s': a '..' component would escape the receive root",
temp_dir);
return NULL;
}
return path_cat(root, temp_dir);
}
char canonical_temp[PATH_MAX];
char canonical_root[PATH_MAX];
if (!realpath(temp_dir, canonical_temp)) {
log_message(LOG_LEVEL_ERROR,
"receiver rejected --temp-dir '%s': could not resolve the absolute path (%s)",
temp_dir, strerror(errno));
return NULL;
}
if (!realpath(root, canonical_root)) {
log_message(LOG_LEVEL_ERROR,
"receiver rejected --temp-dir '%s': could not resolve the receive root (%s)",
temp_dir, strerror(errno));
return NULL;
}
if (strcmp(canonical_root, "/") != 0 && !path_is_within_root(canonical_root, canonical_temp)) {
log_message(LOG_LEVEL_ERROR,
"receiver rejected --temp-dir '%s': an absolute temp dir must be inside the "
"receive root '%s'",
temp_dir, canonical_root);
return NULL;
}
return str_dup(canonical_temp);
}
/* Install a --hard-links/-H sibling: the destination entry is atomically
replaced (temp + rename) with a hard link to the group's first member. The
first member is guaranteed already installed at `hardlink_target` under the
@@ -303,17 +354,13 @@ static FileSaveResult file_save_hardlink_sibling(const char* root_directory, con
free(destination_path);
return absent_result;
}
/* Resolve a relative --temp-dir under the destination root, exactly as the
* primary save path does; an absolute or `..`-escaping value is rejected. */
/* Resolve the --temp-dir under the destination root, exactly as the primary
* save path does: a relative dir joins below the root, an absolute dir is
* accepted only when it canonicalizes inside the root, and any escaping value
* is rejected. */
char* resolved_temp = NULL;
if (cfg->temp_dir) {
if (cfg->temp_dir[0] == '/' || has_path_traversal(cfg->temp_dir)) {
free(content);
free(first_disk);
free(destination_path);
return FILE_SAVE_ERROR;
}
resolved_temp = path_cat(root_directory, cfg->temp_dir);
resolved_temp = file_save_resolve_temp_dir(root_directory, cfg->temp_dir);
if (!resolved_temp) {
free(content);
free(first_disk);
@@ -896,17 +943,17 @@ static bool file_save_try_special_dispatch(const FileSavePlan* plan, bool* creat
and disk paths. Returns false on an invalid/escaping option or an
allocation failure (the caller routes to the cleanup epilogue). */
static bool file_save_resolve_paths(FileSavePlan* plan) {
/* These options arrive from the client. --backup-dir, --partial-dir and
--temp-dir are names below the server root, never independent filesystem
roots: an absolute or `..`-escaping value is rejected outright (rsync's
daemon confines temp-dir to the module the same way). A relative temp dir
is resolved under the receive root below; if that resolution still lands on
a different filesystem than the destination the install falls back to a
non-atomic copy (see file_to_disk_secure_impl), never an abort. */
/* These options arrive from the client. --backup-dir and --partial-dir are
names below the server root, never independent filesystem roots: an
absolute or `..`-escaping value is rejected outright. --temp-dir is
resolved by file_save_resolve_temp_dir below: a relative name joins below
the root, an absolute name is accepted only when it canonicalizes inside
the root, and any escaping value is rejected. If the resolved scratch dir
still lands on a different filesystem than the destination the install
falls back to a non-atomic copy (see file_to_disk_secure_impl), never an
abort. */
if ((plan->backup_dir && (plan->backup_dir[0] == '/' || has_path_traversal(plan->backup_dir))) ||
(plan->partial_dir &&
(plan->partial_dir[0] == '/' || has_path_traversal(plan->partial_dir))) ||
(plan->temp_dir && (plan->temp_dir[0] == '/' || has_path_traversal(plan->temp_dir))))
(plan->partial_dir && (plan->partial_dir[0] == '/' || has_path_traversal(plan->partial_dir))))
return false;
if (plan->backup_dir &&
!(plan->confined_backup = path_cat(plan->root_directory, plan->backup_dir)))
@@ -914,6 +961,9 @@ static bool file_save_resolve_paths(FileSavePlan* plan) {
if (plan->partial_dir &&
!(plan->confined_partial = path_cat(plan->root_directory, plan->partial_dir)))
return false;
if (plan->temp_dir &&
!(plan->confined_temp = file_save_resolve_temp_dir(plan->root_directory, plan->temp_dir)))
return false;
const char* actual_root = plan->use_partial_root ? plan->confined_partial : plan->root_directory;
plan->destination_path = path_cat(plan->root_directory, plan->file->path);
@@ -1146,17 +1196,22 @@ FileSaveResult file_save_to_disk_full_ex(const char* root_directory, const File*
/* A configured --temp-dir sends the temporary working copy to a scratch
directory; the engine then atomically renames the completed file into the
final destination directory. A relative temp dir is resolved under the
receive root and must already exist (an absolute or `..`-escaping value was
rejected above); the engine falls back to a non-atomic copy on EXDEV. The
partial-dir flow already keeps its working copy in a separate directory and
--inplace writes directly, so neither diverts through the scratch dir
(matching rsync, where --inplace/--partial-dir supersede --temp-dir). */
bool use_temp_dir = plan.temp_dir != NULL && !plan.inplace && !plan.use_partial_root;
final destination directory. The scratch path was confined to the receive
root (and canonicalized) in file_save_resolve_paths and must already exist;
the engine falls back to a non-atomic copy on EXDEV. The partial-dir flow
already keeps its working copy in a separate directory and --inplace writes
directly, so neither diverts through the scratch dir (matching rsync, where
--inplace/--partial-dir supersede --temp-dir). */
/* --inplace and --partial-dir supersede --temp-dir in rsync, so the scratch
dir is not used on those paths. The value was still validated/confined by
file_save_resolve_paths; drop the resolved path so it is never handed to the
install engine. */
if (plan.confined_temp && (plan.inplace || plan.use_partial_root)) {
free(plan.confined_temp);
plan.confined_temp = NULL;
}
bool use_temp_dir = plan.confined_temp != NULL;
if (use_temp_dir) {
plan.confined_temp = path_cat(root_directory, plan.temp_dir);
if (!plan.confined_temp)
goto out;
/* A user-supplied trailing slash would leave the scratch path ending in
"/", which has no final component to create/open. Normalize it away. */
size_t temp_len = strlen(plan.confined_temp);
+120
View File
@@ -0,0 +1,120 @@
"""Parity coverage for an absolute ``--temp-dir`` that lies inside the receive root.
FastSync confines the ``--temp-dir`` scratch directory to the receive root. It
previously rejected *every* absolute path; it now canonicalizes an absolute path
with ``realpath(3)`` and accepts it when it resolves inside the canonical receive
root (the destination is identical, so this is a pure parity win), while an
absolute path that escapes the root stays rejected with a clear error.
Two paths exercise the same receiver-side resolution:
* the local ``--read-batch`` apply (no network; the batch destination is the
receive root), and
* a real TCP transfer against the shared server (the destination root is the
client-supplied absolute path).
The out-of-root case asserts the run fails without writing a single scratch
file, so the confinement invariant is preserved.
"""
import os
import subprocess
import sys
import pytest
sys.path.insert(0, os.path.dirname(__file__))
from common import CLIENT_CMD, TEST_DATA_DIR, clean_dir, get_dest_received_dir, run_client
FILES = {
"top.txt": b"top level\n",
"sub/nested.txt": b"nested file\n" * 16,
}
def _run(args):
return subprocess.run(CLIENT_CMD + args, capture_output=True, text=True, timeout=180)
def _seed_source(root):
clean_dir(root)
for rel, data in FILES.items():
full = os.path.join(root, rel)
os.makedirs(os.path.dirname(full), exist_ok=True)
with open(full, "wb") as fh:
fh.write(data)
def _make_batch(tmp, source):
batch = os.path.join(tmp, "tree.batch")
result = _run(["--only-write-batch", batch, source])
assert result.returncode == 0, (result.stdout, result.stderr)
return batch
def _read(path):
with open(path, "rb") as fh:
return fh.read()
@pytest.mark.ci
def test_read_batch_absolute_temp_dir_inside_root_accepted(tmp_path):
source = os.path.join(tmp_path, "src")
dest = os.path.join(tmp_path, "dst")
_seed_source(source)
clean_dir(dest)
scratch = os.path.join(dest, "scratch")
os.makedirs(scratch)
batch = _make_batch(str(tmp_path), source)
result = _run(["--read-batch", batch, dest, "--temp-dir", scratch])
assert result.returncode == 0, (result.stdout, result.stderr)
received = get_dest_received_dir(dest, source)
for rel, data in FILES.items():
assert _read(os.path.join(received, rel)) == data, f"content mismatch for {rel}"
assert os.listdir(scratch) == [], "scratch dir was not left clean"
@pytest.mark.ci
def test_read_batch_absolute_temp_dir_outside_root_rejected(tmp_path):
source = os.path.join(tmp_path, "src")
dest = os.path.join(tmp_path, "dst")
_seed_source(source)
clean_dir(dest)
outside = os.path.join(tmp_path, "outside")
os.makedirs(outside)
batch = _make_batch(str(tmp_path), source)
result = _run(["--read-batch", batch, dest, "--temp-dir", outside])
assert result.returncode != 0, "an absolute temp dir outside the receive root must be rejected"
assert os.listdir(outside) == [], "receiver wrote into an unconfined temp dir"
assert "temp-dir" in (result.stdout + result.stderr), (result.stdout, result.stderr)
def test_tcp_absolute_temp_dir_inside_root_accepted(shared_server):
source = os.path.join(TEST_DATA_DIR, "tempdir_abs_in_src")
dest = os.path.join(TEST_DATA_DIR, "tempdir_abs_in_dst")
_seed_source(source)
clean_dir(dest)
scratch = os.path.join(dest, "scratch")
os.makedirs(scratch)
result, _ = run_client(source, dest, flags=["--temp-dir", scratch], port=shared_server.port)
assert result.returncode == 0, (result.stdout, result.stderr)[:300]
received = get_dest_received_dir(dest, source)
for rel, data in FILES.items():
assert _read(os.path.join(received, rel)) == data, f"content mismatch for {rel}"
assert os.listdir(scratch) == [], "scratch dir was not left clean"
def test_tcp_absolute_temp_dir_outside_root_rejected(shared_server):
source = os.path.join(TEST_DATA_DIR, "tempdir_abs_out_src")
dest = os.path.join(TEST_DATA_DIR, "tempdir_abs_out_dst")
_seed_source(source)
clean_dir(dest)
outside = os.path.join(TEST_DATA_DIR, "tempdir_abs_out_scratch")
clean_dir(outside)
result, _ = run_client(source, dest, flags=["--temp-dir", outside], port=shared_server.port)
assert result.returncode != 0, "an absolute temp dir outside the receive root must be rejected"
assert os.listdir(outside) == [], "receiver wrote into an unconfined temp dir"
+5 -3
View File
@@ -5040,10 +5040,12 @@ static void test_parse_args_include_exclude_order() {
config_delete(cfg3);
}
/* OPT_NOOP compatibility flags (-s/--secluded-args, -r/--recursive) must never
* swallow the next argv: `fastsync -s SRC DST` keeps both positionals. */
/* OPT_NOOP compatibility flags (-s/--secluded-args, -r/--recursive, and the
* --inc-recursive/--no-inc-recursive scan-mode pair) must never swallow the
* next argv: `fastsync -s SRC DST` keeps both positionals. */
static void test_parse_args_noop_does_not_consume_argv() {
static const char* const noops[] = {"-s", "--secluded-args", "-r", "--recursive"};
static const char* const noops[] = {"-s", "--secluded-args", "-r",
"--recursive", "--inc-recursive", "--no-inc-recursive"};
for (size_t i = 0; i < sizeof(noops) / sizeof(noops[0]); i++) {
Config* cfg = config_create();
int positional_args[2];
+12
View File
@@ -339,12 +339,16 @@ static void test_file_save_to_disk_temp_dir_confined() {
const char* root = "test_temp_confine_tmp";
const char* dest_file = "test_temp_confine_tmp/file.txt";
char outside[PATH_MAX];
char inside_abs[PATH_MAX];
snprintf(outside, sizeof(outside), "/tmp/fastsync_temp_outside_%d", (int)getpid());
unlink(dest_file);
rmdir("test_temp_confine_tmp/scratch");
rmdir("test_temp_confine_tmp/abs_scratch");
rmdir(root);
mkdir(root, 0755);
mkdir("test_temp_confine_tmp/scratch", 0755);
mkdir("test_temp_confine_tmp/abs_scratch", 0755);
EXPECT_NOT_NULL(realpath("test_temp_confine_tmp/abs_scratch", inside_abs));
mkdir(outside, 0755);
File* f = file_create("file.txt");
@@ -364,6 +368,13 @@ static void test_file_save_to_disk_temp_dir_confined() {
config->temp_dir = str_dup("../escape");
EXPECT_EQ_INT(file_save_to_disk_full(root, f, config), FILE_SAVE_ERROR);
EXPECT_EQ_INT(access(dest_file, F_OK), -1);
/* An absolute temp dir that canonicalizes INSIDE the receive root is
accepted and used (the parity win); destination is still written. */
free(config->temp_dir);
config->temp_dir = str_dup(inside_abs);
EXPECT_EQ_INT(file_save_to_disk_full(root, f, config), FILE_SAVE_WRITTEN);
EXPECT_EQ_INT(access(dest_file, F_OK), 0);
unlink(dest_file);
free(config->temp_dir);
config->temp_dir = str_dup("scratch");
EXPECT_EQ_INT(file_save_to_disk_full(root, f, config), FILE_SAVE_WRITTEN);
@@ -373,6 +384,7 @@ static void test_file_save_to_disk_temp_dir_confined() {
config_delete(config);
unlink(dest_file);
rmdir("test_temp_confine_tmp/scratch");
rmdir("test_temp_confine_tmp/abs_scratch");
rmdir(root);
rmdir(outside);
}