Release v2.29.0 #312

Merged
TapTap merged 123 commits from dev into main 2026-09-23 02:05:14 +02:00
11 changed files with 318 additions and 152 deletions
Showing only changes of commit 0b40c47d6a - Show all commits
+2 -2
View File
@@ -187,7 +187,7 @@ This produces `./build/client` and `./build/server`. `compile_commands.json` is
| `--groupmap=MAP` | Map group names when applying ownership | | `--groupmap=MAP` | Map group names when applying ownership |
| `--numeric-ids` | Apply source numeric uid/gid directly instead of mapping by name | | `--numeric-ids` | Apply source numeric uid/gid directly instead of mapping by name |
| `--copy-as=USER[:GROUP]` | Force every written entry to USER[:GROUP] (requires a privileged receiver) | | `--copy-as=USER[:GROUP]` | Force every written entry to USER[:GROUP] (requires a privileged receiver) |
| `--fake-super` | Record the resolved owner plus mode/time in a reserved `user.fastsync.stat` xattr and replay mode/time; never performs a real chown | | `--fake-super` | Record the resolved owner plus full mode/rdev in rsync's reserved `user.rsync.%stat` xattr (rsync 3.4.1 grammar) and replay the permission bits; never performs a real chown |
| `--super` | Permit the receiver to attempt confined super-user activities (device nodes) | | `--super` | Permit the receiver to attempt confined super-user activities (device nodes) |
| `-D` | Preserve device and special files (implies `--devices --specials`) | | `-D` | Preserve device and special files (implies `--devices --specials`) |
| `--devices` | Recreate device nodes on the destination (privileged; skipped without `CAP_MKNOD`) | | `--devices` | Recreate device nodes on the destination (privileged; skipped without `CAP_MKNOD`) |
@@ -643,7 +643,7 @@ remote SSH argv is already built injection-safe.
| `--groupmap=MAP` | Map group names when applying ownership (same syntax as `--usermap`). | | `--groupmap=MAP` | Map group names when applying ownership (same syntax as `--usermap`). |
| `--numeric-ids` | Mapping modifier: apply the source numeric uid/gid directly instead of mapping by name (combine with `-o`/`-g`, `-a`, or a map). | | `--numeric-ids` | Mapping modifier: apply the source numeric uid/gid directly instead of mapping by name (combine with `-o`/`-g`, `-a`, or a map). |
| `--copy-as=USER[:GROUP]` | Force every written entry to USER[:GROUP]; requires a privileged receiver. | | `--copy-as=USER[:GROUP]` | Force every written entry to USER[:GROUP]; requires a privileged receiver. |
| `--fake-super` | Record the resolved owner plus mode/time in a reserved `user.fastsync.stat` xattr and replay mode/time; never performs a real chown. | | `--fake-super` | Record the resolved owner plus full mode/rdev in rsync's reserved `user.rsync.%stat` xattr (rsync 3.4.1 grammar) and replay the permission bits; never performs a real chown. |
| `--super` | Permit the receiver to attempt confined super-user activities (device nodes). | | `--super` | Permit the receiver to attempt confined super-user activities (device nodes). |
| `--no-super` | Forbid those super-user activities even when the receiver is root. | | `--no-super` | Forbid those super-user activities even when the receiver is root. |
| `-l`, `--links` | Copy symlinks as symlinks; the target is stored verbatim (absolute and `..`-bearing targets included), matching rsync. | | `-l`, `--links` | Copy symlinks as symlinks; the target is stored verbatim (absolute and `..`-bearing targets included), matching rsync. |
+47 -37
View File
@@ -6,9 +6,9 @@ This document maps rsync's full feature set to FastSync's current implementation
| Status | Count | Description | | Status | Count | Description |
|--------|-------|-------------| |--------|-------|-------------|
| ✅ Parity | 118 | Reproduces rsync's semantics for this option's scope | | ✅ Parity | 119 | Reproduces rsync's semantics for this option's scope |
| ⚠️ Caveat | 13 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) | | ⚠️ Caveat | 14 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
| ❌ Divergent | 26 | Rejected, an accepted no-op, deliberately non-rsync (native config/auth/batch, privileged namespaces, safe-subset privilege), or impossible on any portable filesystem call | | ❌ Divergent | 24 | Rejected, an accepted no-op, deliberately non-rsync (native config/auth/batch, privileged namespaces, safe-subset privilege), or impossible on any portable filesystem call |
| **Total** | **157** | One row per rsync option/feature group; a row may name several spellings | | **Total** | **157** | One row per rsync option/feature group; a row may name several spellings |
This matrix reports honest rsync parity, not "implemented" as a synonym for This matrix reports honest rsync parity, not "implemented" as a synonym for
@@ -17,7 +17,7 @@ tested but diverges in at least one documented way. An ❌ row is either
rejected (`--protocol` with any value but the current one), rejected (`--protocol` with any value but the current one),
an accepted no-op (`-s`/`--secluded-args`, `--protect-args`, `--old-args`), an accepted no-op (`-s`/`--secluded-args`, `--protect-args`, `--old-args`),
deliberately non-rsync and non-interoperable (the FastSync daemon config/auth, deliberately non-rsync and non-interoperable (the FastSync daemon config/auth,
the batch container, `--fake-super`'s xattr format, `--copy-as` credential the batch container, `--copy-as` credential
switching), or impossible (`-N`/`--crtimes`). The counts are derived from the switching), or impossible (`-N`/`--crtimes`). The counts are derived from the
rows below; update them together with the table. rows below; update them together with the table.
@@ -83,8 +83,8 @@ output, codec breadth, general `-R`/`-d`, the filter grammar (the unsupported
rejected elsewhere — see the audit-cycle follow-up note above), receiver-side rejected elsewhere — see the audit-cycle follow-up note above), receiver-side
name resolution, absolute basis dirs, and the remaining client quick wins) and name resolution, absolute basis dirs, and the remaining client quick wins) and
reclassified the inherently non-rsync rows as **divergent** (native daemon reclassified the inherently non-rsync rows as **divergent** (native daemon
config/auth, the non-interoperable batch container, `--fake-super`'s xattr config/auth, the non-interoperable batch container,
format, `-X`'s privileged namespaces, and the safe-subset device/privilege `-X`'s privileged namespaces, and the safe-subset device/privilege
flags). It moved `PROTOCOL_VERSION` three times (`2.23.0 → 2.24.0` delete flags). It moved `PROTOCOL_VERSION` three times (`2.23.0 → 2.24.0` delete
timing, `2.24.0 → 2.25.0` wire stats, `2.25.0 → 2.26.0` codecs). See the timing, `2.24.0 → 2.25.0` wire stats, `2.25.0 → 2.26.0` codecs). See the
**Parity Completion Wave (protocol 2.26.0)** section near the end for the full **Parity Completion Wave (protocol 2.26.0)** section near the end for the full
@@ -342,7 +342,7 @@ why plain `--append` works on the normal atomic path, not only with `--inplace`.
| `-X`, `--xattrs` | Preserve extended attributes | ❌ Divergent | Deliberately restricted to unprivileged `user.*` extended attributes plus the two POSIX ACL xattrs; `security.*` (SELinux, capabilities, ...) and `trusted.*` are **never** captured or applied — a client can never force a privileged attribute onto the destination, and the receiver independently re-validates every incoming name against the whitelist. This is a security-policy divergence from rsync, which can preserve the privileged namespaces with the needed privilege; implementing them would defeat FastSync's privilege-escalation guard. `user.*` capture/apply matches rsync in a differential test. Payloads are bounded on both ends. Incompatible with `-s`. **Also divergent: symlink xattrs/ACLs are not captured or applied** — `-X`/`-A` with `-l` carries only the link's owner/times/mode, not its xattrs (the capture uses path-following `listxattr`/`getxattr`, so the link's own xattrs are never read, and the receiver's symlink write path applies no xattr block). Closing this needs a dedicated symlink-xattr wire block and a `PROTOCOL_VERSION` bump | | `-X`, `--xattrs` | Preserve extended attributes | ❌ Divergent | Deliberately restricted to unprivileged `user.*` extended attributes plus the two POSIX ACL xattrs; `security.*` (SELinux, capabilities, ...) and `trusted.*` are **never** captured or applied — a client can never force a privileged attribute onto the destination, and the receiver independently re-validates every incoming name against the whitelist. This is a security-policy divergence from rsync, which can preserve the privileged namespaces with the needed privilege; implementing them would defeat FastSync's privilege-escalation guard. `user.*` capture/apply matches rsync in a differential test. Payloads are bounded on both ends. Incompatible with `-s`. **Also divergent: symlink xattrs/ACLs are not captured or applied** — `-X`/`-A` with `-l` carries only the link's owner/times/mode, not its xattrs (the capture uses path-following `listxattr`/`getxattr`, so the link's own xattrs are never read, and the receiver's symlink write path applies no xattr block). Closing this needs a dedicated symlink-xattr wire block and a `PROTOCOL_VERSION` bump |
| `-H`, `--hard-links` | Preserve hard links | ✅ Parity | Files on the source that share an inode (`st_dev`+`st_ino`, e.g. a `cp -al` tree) are re-created as hard links to one another on the destination, so duplicate links stay deduplicated and only the first member's data is sent (later members are transmitted as payload-less `STATUS_HARDLINK` frames). The receiver links each sibling to the first member's installed file with an atomic link + rename; on `link()` failure it falls back to a byte-identical local copy of the first member, never a partial/corrupt file. Requires the sequential scan for ordering (the first member is always emitted and installed before any sibling is linked). Works single-threaded and under `-j`/`--threads`, `--inplace`, `--delay-updates` (links staged and published by rename) and `--partial`. Crosses the wire (`preserve_hard_links` bool; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0**, peers must match). Incompatible with `-s` (chunk serialization) and `--append`/`--append-verify`, rejected up front with a distinct error. See the Phase-4 hard-links notes below | | `-H`, `--hard-links` | Preserve hard links | ✅ Parity | Files on the source that share an inode (`st_dev`+`st_ino`, e.g. a `cp -al` tree) are re-created as hard links to one another on the destination, so duplicate links stay deduplicated and only the first member's data is sent (later members are transmitted as payload-less `STATUS_HARDLINK` frames). The receiver links each sibling to the first member's installed file with an atomic link + rename; on `link()` failure it falls back to a byte-identical local copy of the first member, never a partial/corrupt file. Requires the sequential scan for ordering (the first member is always emitted and installed before any sibling is linked). Works single-threaded and under `-j`/`--threads`, `--inplace`, `--delay-updates` (links staged and published by rename) and `--partial`. Crosses the wire (`preserve_hard_links` bool; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0**, peers must match). Incompatible with `-s` (chunk serialization) and `--append`/`--append-verify`, rejected up front with a distinct error. See the Phase-4 hard-links notes below |
| `-D` | Same as --devices --specials | ✅ Parity | Implies `--devices --specials`. `-D` was unassigned in FastSync (verified: no collision), so it is free to imply both device-node and special-file preservation. As of protocol 2.23.0 `--specials` genuinely covers **both FIFOs and unix sockets**, so `-D` covers the full rsync set. See the `--devices`/`--specials` rows and the Phase-4 devices notes below | | `-D` | Same as --devices --specials | ✅ Parity | Implies `--devices --specials`. `-D` was unassigned in FastSync (verified: no collision), so it is free to imply both device-node and special-file preservation. As of protocol 2.23.0 `--specials` genuinely covers **both FIFOs and unix sockets**, so `-D` covers the full rsync set. See the `--devices`/`--specials` rows and the Phase-4 devices notes below |
| `--devices` | Preserve device files | ❌ Divergent | Recreates char/block device nodes with `mknodat` (type + rdev strictly validated, confined fd-relative below the receive root), but only when the receiver has `CAP_MKNOD`: a non-root receiver logs a warning and skips the entry instead of erroring, so a transfer with devices never aborts. Deliberate privilege-model divergence from rsync, which errors when it cannot create the node. `--specials` (FIFOs and unix sockets) is unprivileged and remains parity | | `--devices` | Preserve device files | ⚠️ Caveat | Recreates char/block device nodes with `mknodat` (type + rdev strictly validated, confined fd-relative below the receive root). A device whose `mknodat` fails with `EPERM`/`EACCES` (no `CAP_MKNOD`, or super-user activity forbidden) is now a **transfer error** surfaced through the receiver's outcome aggregation — rsync parity: rsync reports `mknod ... failed` and exits partial (23) when it attempts the node (as root or with `--super`). Residual: FastSync's default AUTO still *attempts* the node on a non-root receiver and therefore errors, whereas rsync without `--super` silently ignores `--devices` and skips the non-regular entry with exit 0; use `--no-super` for rsync's silent-skip behavior. (FastSync's process exit code for a receiver-side transfer error is the general error code 1, not rsync's partial 23 — a client exit-code-mapping residual that applies to every receiver file error, not just this branch.) `--specials` (FIFOs and unix sockets) keeps the unprivileged skip path and remains parity |
| `--specials` | Preserve special files | ✅ Parity | **FIFO and unix-socket recreation work** (protocol 2.23.0): FIFOs are recreated with `mkfifoat`, and sockets with `mknodat(..., S_IFSOCK)` — the latter is unprivileged on Linux because it materializes the socket *node*, not a live bound socket, so it is a real, assertable behavior under CI (it matches rsync, which also recreates a socket by `mknod`). Node creation is confined below the receive root (fd-relative parent; no `..`, no symlink follow) and type/rdev are validated strictly; a matching existing node is left in place and an unrelated entry is never replaced. Crosses the wire like `--devices` (the `STATUS_SPECIAL` frame). See the Phase-4 devices notes | | `--specials` | Preserve special files | ✅ Parity | **FIFO and unix-socket recreation work** (protocol 2.23.0): FIFOs are recreated with `mkfifoat`, and sockets with `mknodat(..., S_IFSOCK)` — the latter is unprivileged on Linux because it materializes the socket *node*, not a live bound socket, so it is a real, assertable behavior under CI (it matches rsync, which also recreates a socket by `mknod`). Node creation is confined below the receive root (fd-relative parent; no `..`, no symlink follow) and type/rdev are validated strictly; a matching existing node is left in place and an unrelated entry is never replaced. Crosses the wire like `--devices` (the `STATUS_SPECIAL` frame). See the Phase-4 devices notes |
| `--copy-devices` | Copy device contents as file | ❌ Divergent | Copies a device/FIFO's reported `st_size` into an ordinary regular file and never reads an unbounded pseudo-device, so `--sendfile` cannot hang and the run always succeeds. Deliberate safe divergence from rsync's dd-like unbounded device read, which can block; the dangerous behavior will not be implemented | | `--copy-devices` | Copy device contents as file | ❌ Divergent | Copies a device/FIFO's reported `st_size` into an ordinary regular file and never reads an unbounded pseudo-device, so `--sendfile` cannot hang and the run always succeeds. Deliberate safe divergence from rsync's dd-like unbounded device read, which can block; the dangerous behavior will not be implemented |
| `--write-devices` | Write to devices as files | ❌ Divergent | Writes only into an existing char/block node under the confined receive root (`O_NOFOLLOW` + `O_NONBLOCK`); a missing, symlinked, FIFO-with-no-reader, non-device, or otherwise unusable destination is skipped with a warning rather than allowed or aborted. Deliberate confinement divergence from rsync's more permissive behavior | | `--write-devices` | Write to devices as files | ❌ Divergent | Writes only into an existing char/block node under the confined receive root (`O_NOFOLLOW` + `O_NONBLOCK`); a missing, symlinked, FIFO-with-no-reader, non-device, or otherwise unusable destination is skipped with a warning rather than allowed or aborted. Deliberate confinement divergence from rsync's more permissive behavior |
@@ -351,7 +351,7 @@ why plain `--append` works on the normal atomic path, not only with `--inplace`.
| `-O`, `--omit-dir-times` | Omit dirs from --times | ✅ Parity | Real modifier now that FastSync preserves directory times. With metadata on, the scanner captures every traversed source directory's mtime (and atime under `-U`) and the sender transmits them in trailing `STATUS_DIR_TIMES` frame(s) **after all file data and the optional delete manifest** (chunked at the receiver's `MAX_MANIFEST_ENTRIES` per-frame cap); a dir-time entry only RECORDS metadata and never creates the directory (an empty source directory is created by the separate `STATUS_MKDIR` entry the scanner now emits, and `-m/--prune-empty-dirs` suppresses that; the trailing dir-time simply re-applies the metadata). The receiver defers applying them until its delete / `--delay-updates` publication phases have committed, so writing or removing a child never clobbers a parent directory's mtime (rsync applies directory times at the end for exactly this reason). When `-O` is set (the boolean crosses the wire) the receiver does not apply any of them; without `-O` an `-a`/`--preserve` transfer now restores directory times (reversing the old "never preserves dir times" divergence). Wire change: the terminal `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** | | `-O`, `--omit-dir-times` | Omit dirs from --times | ✅ Parity | Real modifier now that FastSync preserves directory times. With metadata on, the scanner captures every traversed source directory's mtime (and atime under `-U`) and the sender transmits them in trailing `STATUS_DIR_TIMES` frame(s) **after all file data and the optional delete manifest** (chunked at the receiver's `MAX_MANIFEST_ENTRIES` per-frame cap); a dir-time entry only RECORDS metadata and never creates the directory (an empty source directory is created by the separate `STATUS_MKDIR` entry the scanner now emits, and `-m/--prune-empty-dirs` suppresses that; the trailing dir-time simply re-applies the metadata). The receiver defers applying them until its delete / `--delay-updates` publication phases have committed, so writing or removing a child never clobbers a parent directory's mtime (rsync applies directory times at the end for exactly this reason). When `-O` is set (the boolean crosses the wire) the receiver does not apply any of them; without `-O` an `-a`/`--preserve` transfer now restores directory times (reversing the old "never preserves dir times" divergence). Wire change: the terminal `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** |
| `-J`, `--omit-link-times` | Omit symlinks from --times | ✅ Parity | Real modifier now that FastSync preserves symlink times. Symlink entries already carried their metadata on `STATUS_SYMLINK`; the receiver now applies it with **no-follow primitives only** (`utimensat(..., AT_SYMLINK_NOFOLLOW)`, plus best-effort `fchmodat(..., AT_SYMLINK_NOFOLLOW)` and policy-gated `fchownat(..., AT_SYMLINK_NOFOLLOW)`), so the link itself is stamped without ever dereferencing it, confined fd-relative below the authorized receive root. A symlink has no children, so the times are applied immediately at creation. When `-J` is set (the boolean crosses the wire) the receiver skips the timestamps (mode/ownership are unaffected); without `-J` an `-a`/`-l` transfer restores symlink mtimes. Wire change alongside `-O`: the shared `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** | | `-J`, `--omit-link-times` | Omit symlinks from --times | ✅ Parity | Real modifier now that FastSync preserves symlink times. Symlink entries already carried their metadata on `STATUS_SYMLINK`; the receiver now applies it with **no-follow primitives only** (`utimensat(..., AT_SYMLINK_NOFOLLOW)`, plus best-effort `fchmodat(..., AT_SYMLINK_NOFOLLOW)` and policy-gated `fchownat(..., AT_SYMLINK_NOFOLLOW)`), so the link itself is stamped without ever dereferencing it, confined fd-relative below the authorized receive root. A symlink has no children, so the times are applied immediately at creation. When `-J` is set (the boolean crosses the wire) the receiver skips the timestamps (mode/ownership are unaffected); without `-J` an `-a`/`-l` transfer restores symlink mtimes. Wire change alongside `-O`: the shared `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** |
| `--super` | Receiver attempts super-user activities | ❌ Divergent | Safe-subset privilege model. `--super` permits the receiver to attempt already-confined super-user activities (ownership application, char/block device-node creation, `--write-devices`); `--no-super` forbids them even for root; `auto` keeps the historical best-effort attempt. **FastSync never elevates** — no `setuid`/`seteuid`/`setgid` — and `--super` never bypasses the confinement floor, so it diverges from rsync's real elevation. A server `--no-super` veto forces it off for every connection; a privileged standalone listener defaults off without `--allow-super`; daemon modules opt in with `client owner = yes` | | `--super` | Receiver attempts super-user activities | ❌ Divergent | Safe-subset privilege model. `--super` permits the receiver to attempt already-confined super-user activities (ownership application, char/block device-node creation, `--write-devices`); `--no-super` forbids them even for root; `auto` keeps the historical best-effort attempt. **FastSync never elevates** — no `setuid`/`seteuid`/`setgid` — and `--super` never bypasses the confinement floor, so it diverges from rsync's real elevation. A server `--no-super` veto forces it off for every connection; a privileged standalone listener defaults off without `--allow-super`; daemon modules opt in with `client owner = yes` |
| `--fake-super` | Store/recover privileged attrs via xattrs | ❌ Divergent | Records the resolved `uid:gid:mode:mtime_sec:mtime_nsec` in a reserved `user.fastsync.stat` xattr and immediately replays mode/times fd-relative, but **never performs a real `chown`** (the owner is recorded for a later privileged restore). The on-disk key and format are FastSync-native, not rsync's `user.rsync.%stat%`, so recordings are not interoperable with rsync — the same class as the native auth and batch formats. Implies metadata transmission; incompatible with `-s` | | `--fake-super` | Store/recover privileged attrs via xattrs | ⚠️ Caveat | Writes rsync 3.4.1's reserved `user.rsync.%stat` xattr with rsync's exact value grammar `<octal st_mode with S_IFMT> <rdev_major>,<rdev_minor> <uid>:<gid>` (e.g. `104711 0,0 1234:5678`), recording the RESOLVED owner (the `--chown`/`--usermap`/`--groupmap`/`--copy-as` mapping when active, else the source's own id) plus the full mode and rdev; it **never performs a real `chown`**. mtime is carried by the file's own timestamp, exactly as rsync does it (there is no mtime field). The receiver parses the same grammar and replays the permission bits fd-relative, stripping the recorded special bits on disk exactly like rsync's fake-super receiver. Regular files are interoperable with real rsync 3.4.1 in both directions (the differential test has rsync read a FastSync fake-super tree and re-emit the identical record). Residual: directories and device nodes are not yet faked — no `%stat` record is written for a directory, and a char/block node is still recreated/skipped rather than stored as a regular file carrying the stat. Implies metadata transmission; incompatible with `-s` |
| `--open-noatime` | Avoid changing access time when opening files | ✅ Parity | Sender-side policy: the sender opens source files with `O_NOATIME` (Linux) when reading them for transfer, so the open/read does NOT bump the source's on-disk access time. Degrades safely when `O_NOATIME` is unavailable (not defined) or refused (`EPERM`, since it needs `CAP_FOWNER` or file ownership): the code falls back to a normal open, so the data always transfers — only the atime-bump is skipped. It does not itself capture/preserve atime; it only avoids modifying it. **Client-only, never crosses the wire.** Exposed as `file_open_for_read()` and applied to both the buffered data path and the sendfile path | | `--open-noatime` | Avoid changing access time when opening files | ✅ Parity | Sender-side policy: the sender opens source files with `O_NOATIME` (Linux) when reading them for transfer, so the open/read does NOT bump the source's on-disk access time. Degrades safely when `O_NOATIME` is unavailable (not defined) or refused (`EPERM`, since it needs `CAP_FOWNER` or file ownership): the code falls back to a normal open, so the data always transfers — only the atime-bump is skipped. It does not itself capture/preserve atime; it only avoids modifying it. **Client-only, never crosses the wire.** Exposed as `file_open_for_read()` and applied to both the buffered data path and the sendfile path |
| `--numeric-ids` | Do not map uid/gid by name | ✅ Parity | **A mapping modifier only:** when ownership is being applied it uses the transmitted numeric uid/gid directly, skipping the name lookup. It does **not** request ownership application on its own — combine it with `-o`/`-g`, `-a`, or an explicit map (`--chown`/`--usermap`/`--groupmap`) — and it does not need any metadata flag merely to parse. Ownership is only applied when metadata (hence the source uid/gid) is actually transmitted (see the Phase-4 identity notes) | | `--numeric-ids` | Do not map uid/gid by name | ✅ Parity | **A mapping modifier only:** when ownership is being applied it uses the transmitted numeric uid/gid directly, skipping the name lookup. It does **not** request ownership application on its own — combine it with `-o`/`-g`, `-a`, or an explicit map (`--chown`/`--usermap`/`--groupmap`) — and it does not need any metadata flag merely to parse. Ownership is only applied when metadata (hence the source uid/gid) is actually transmitted (see the Phase-4 identity notes) |
| `--usermap=STRING` | Map usernames | ✅ Parity | Opt-in ownership application. Comma-separated `FROM:TO` rules evaluated in order, first match wins. `FROM` accepts a source-resolved user name, a name **glob** (`*`/`?`/`[...]`, expanded sender-side at CLI-parse time against the sender's passwd/group database and collapsed into numeric `LOW-HIGH` ranges, bounded by `MAX_IDENTITY_MAP`), an `@N`/bare `N` numeric id, an inclusive `LOW-HIGH` id range, `*`, or an empty field (ids with no source name). `TO` accepts a receiver-resolved **name** (protocol 2.26.0 resolves it on the receiving side against the receiver's account database, matching rsync), an `@N`/bare `N` id, or `*` (the receiving process's euid). Rules travel as resolved numeric pairs plus an optional TO name; the receiver applies a matching rule, else falls back to `--chown`, `--numeric-ids`, then a best-effort name lookup, via fd-relative `fchown`. Malformed specs are clear errors. Implies metadata; only effective where the receiver can chown (otherwise a warning) | | `--usermap=STRING` | Map usernames | ✅ Parity | Opt-in ownership application. Comma-separated `FROM:TO` rules evaluated in order, first match wins. `FROM` accepts a source-resolved user name, a name **glob** (`*`/`?`/`[...]`, expanded sender-side at CLI-parse time against the sender's passwd/group database and collapsed into numeric `LOW-HIGH` ranges, bounded by `MAX_IDENTITY_MAP`), an `@N`/bare `N` numeric id, an inclusive `LOW-HIGH` id range, `*`, or an empty field (ids with no source name). `TO` accepts a receiver-resolved **name** (protocol 2.26.0 resolves it on the receiving side against the receiver's account database, matching rsync), an `@N`/bare `N` id, or `*` (the receiving process's euid). Rules travel as resolved numeric pairs plus an optional TO name; the receiver applies a matching rule, else falls back to `--chown`, `--numeric-ids`, then a best-effort name lookup, via fd-relative `fchown`. Malformed specs are clear errors. Implies metadata; only effective where the receiver can chown (otherwise a warning) |
@@ -408,7 +408,7 @@ match, exactly as prior phases did).
is refused) also re-applies the incoming (or, for `-H`, the first member's) is refused) also re-applies the incoming (or, for `-H`, the first member's)
xattrs and the `--fake-super` stat, so attributes are preserved rather than xattrs and the `--fake-super` stat, so attributes are preserved rather than
silently dropped when the link fails. silently dropped when the link fails.
- **Reserved fake-super key is receiver-only:** the `user.fastsync.stat` key is - **Reserved fake-super key is receiver-only:** the `user.rsync.%stat` key is
excluded from sender capture AND from receiver application, so it can only be excluded from sender capture AND from receiver application, so it can only be
written by the receiver's own `--fake-super` handling. A source file that written by the receiver's own `--fake-super` handling. A source file that
already carries such a record is never forwarded on a plain `-X` run, so it already carries such a record is never forwarded on a plain `-X` run, so it
@@ -417,15 +417,19 @@ match, exactly as prior phases did).
Applying an ACL is owner-privileged: `fsetxattr` failure (e.g. non-root, Applying an ACL is owner-privileged: `fsetxattr` failure (e.g. non-root,
unsupported filesystem) is logged (collapsed to one line per file) and never unsupported filesystem) is logged (collapsed to one line per file) and never
fatal. fatal.
- **`--fake-super`**: see the row above; the reserved key is `user.fastsync.stat` - **`--fake-super`**: see the row above; the reserved key is rsync's own
with the documented `uid:gid:mode:mtime_sec:mtime_nsec` (mode octal) format. `user.rsync.%stat` with rsync 3.4.1's exact `<octal st_mode> <rdev_major>,
**Replay exists**: after each stored record the receiver immediately re-applies <rdev_minor> <uid>:<gid>` value (mtime is not stored — the file's own
the recorded mode and times fd-relative (`fake_super_restore_fd`), but it timestamp carries it, exactly as rsync does). **Replay exists**: after each
deliberately never performs a real `chown` — `--fake-super` only *records* stored record the receiver immediately re-applies the recorded permission bits
the resolved owner (the active `--chown`/`--usermap`/`--groupmap`/`--copy-as` fd-relative (`fake_super_restore_fd`, with the recorded special bits stripped
mapping when one is in effect, otherwise the source's own id) for a later on disk exactly like rsync), but it deliberately never performs a real
privileged restore. The recording format diverges from rsync's `chown` — `--fake-super` only *records* the resolved owner (the active
`user.rsync.%stat%`; no cross-tool conversion is attempted. `--chown`/`--usermap`/`--groupmap`/`--copy-as` mapping when one is in effect,
otherwise the source's own id) for a later privileged restore. Because the
key and grammar are rsync's, a regular-file fake-super tree is interoperable
with rsync 3.4.1 in both directions; directories and device nodes are not yet
faked.
- **Chunk serialization (`-s`) incompatibility:** the per-file xattr block rides - **Chunk serialization (`-s`) incompatibility:** the per-file xattr block rides
the streaming per-file frame, which `-s` replaces with a fixed buffer format, the streaming per-file frame, which `-s` replaces with a fixed buffer format,
so `-X` / `-A` combined with `-s` is rejected up front on both ends (mirroring so `-X` / `-A` combined with `-s` is rejected up front on both ends (mirroring
@@ -554,18 +558,22 @@ marker + rdev so `--devices/--specials` also work under `-s`. `PROTOCOL_VERSION`
was bumped **2.12.0 → 2.13.0** (peers must match, exactly as prior phases did). was bumped **2.12.0 → 2.13.0** (peers must match, exactly as prior phases did).
**Privilege gating (the crux):** making a device node requires `CAP_MKNOD` (root). **Privilege gating (the crux):** making a device node requires `CAP_MKNOD` (root).
CI runs the integration suite as a NON-ROOT user (via setpriv), so `mknod` fails When the receiver attempts a device `mknod` and the kernel refuses with
with `EPERM`. The receiver treats this as a graceful, logged *skip of the entry* `EPERM`/`EACCES`, FastSync now reports a genuine transfer error (the receiver's
returned as a success/skip outcome — the whole transfer NEVER aborts just because outcome aggregation fails the entry), matching rsync, which logs
the environment cannot create the node. `mkfifo` (FIFOs) is unprivileged, so `mknod ... failed` and exits partial (23) whenever it attempts the node (as root
`--specials` FIFO creation is a real, assertable behavior under CI. **Sockets are or with `--super`); with `--no-super` the device entry is pre-skipped instead.
recreated too** (protocol 2.23.0) with `mknodat(..., S_IFSOCK)`: Linux allows an Only `mkfifo` (FIFOs) is unprivileged, so `--specials` FIFO creation is a real,
unprivileged `mknod` of a socket node because no live bound socket is created, assertable behavior under CI. **Sockets are recreated too** (protocol 2.23.0)
so a source socket materializes as a socket-type filesystem entry exactly as with `mknodat(..., S_IFSOCK)`: Linux allows an unprivileged `mknod` of a socket
rsync does. The "device actually created" integration assertions are guarded to node because no live bound socket is created, so a source socket materializes as
run only as root. User-facing expectation: point `--devices` at devices and a a socket-type filesystem entry exactly as rsync does. The "device actually
non-root receiver will faithfully skip them while transferring everything else; created" integration assertions are guarded to run only as root; a root runner
`--specials` recreates FIFOs and socket nodes for any receiver. additionally drops the receiver to an unprivileged user (setpriv) to assert the
`CAP_MKNOD` failure surfaces as a failed transfer rather than a silent skip.
User-facing expectation: point `--devices` at devices and a receiver without
`CAP_MKNOD` reports the failure, while `--specials` recreates FIFOs and socket
nodes for any receiver.
**Confinement & validation:** a special/device node is created with **Confinement & validation:** a special/device node is created with
`mknodat`/`mkfifoat` on the parent directory opened fd-relative below the receive `mknodat`/`mkfifoat` on the parent directory opened fd-relative below the receive
@@ -936,7 +944,7 @@ These are the last compatibility items and the closing phase toward rsync flag p
| `-T` / `--timeout` | `-T` = `--temp-dir` | → `--timeout` (long-only) | | `-T` / `--timeout` | `-T` = `--temp-dir` | → `--timeout` (long-only) |
| `-a` / `--archive` (= `-c -m -M`) | `-a` = `-rlptD` | → becomes **real rsync `-a`** after the renames | | `-a` / `--archive` (= `-c -m -M`) | `-a` = `-rlptD` | → becomes **real rsync `-a`** after the renames |
**Wave B — Output & filesystem completion (✅ implemented).** `-S`/`--sparse` (`⚠️→✅`): real hole preservation — a sparse-aware writer (`write_all_sparse`) skips all-zero runs ≥ 4096 bytes with `lseek(SEEK_CUR)` and `ftruncate`s the final size, wired into both the atomic temp+rename store and `--inplace` receiver-side with **no wire change** (the full file image is already in memory; the ftruncate presize is kept). `-P` (`⚠️→✅`): interrupted-write retention — on a save failure after data reached the temp fd, `--partial` now renames the already-written temp to the destination path (best-effort; falls through to the normal unlink on failure, never retains when `--partial` is off) so a later `--append`/`--append-verify` run can resume. `--block-size=SIZE` (`⚠️→✅`): promoted after verification — `--block-size` is now an alias for `--delta-block`, both set `config->delta_block_size`, which the delta engine already honored end-to-end (`delta_signature_create_seeded` + `delta_apply`); out-of-range values keep the default. `--fake-super` (`⚠️→✅`): added `fake_super_restore_fd` to parse and re-apply the recorded `user.fastsync.stat` record fd-relative (mode/time only — protocol 2.23.0: **never a real chown**; the resolved owner is recorded for a later privileged restore); a save under `--fake-super` now re-applies the recorded attrs instead of only recording them, with the recording format unchanged. `--stderr=client` (`⚠️→❌ Divergent`): FastSync has no rsync client-message channel, and `client` is rejected at CLI parse — the rejection is the documented behavior (unit-tested). `-N`/`--crtimes` (`⚠️→❌ Divergent`): birth-times cannot be set by any portable fs call (`utimensat` sets only atime/mtime); capture/transmit stays, setting is impossible, the flag is accepted and safely inert. Review-hardening (post-eval): fake-super replay applies the mode through the shared `metadata_mode_for_policy` helper (protocol 2.23.0: exactly the source mode under `-p`, with no masking); `--sparse` takes precedence over `--preallocate` (posix_fallocate skipped so holes survive) — **reversed by the parity-completion wave: `--preallocate` now wins, matching rsync**; `--partial` retention is disabled for `--no_replace` (ignore/existing) and only marks a write-attempt after the actual write begins; `--block-size=SIZE`/`--delta-block=SIZE` inline forms are accepted. **Wave B — Output & filesystem completion (✅ implemented).** `-S`/`--sparse` (`⚠️→✅`): real hole preservation — a sparse-aware writer (`write_all_sparse`) skips all-zero runs ≥ 4096 bytes with `lseek(SEEK_CUR)` and `ftruncate`s the final size, wired into both the atomic temp+rename store and `--inplace` receiver-side with **no wire change** (the full file image is already in memory; the ftruncate presize is kept). `-P` (`⚠️→✅`): interrupted-write retention — on a save failure after data reached the temp fd, `--partial` now renames the already-written temp to the destination path (best-effort; falls through to the normal unlink on failure, never retains when `--partial` is off) so a later `--append`/`--append-verify` run can resume. `--block-size=SIZE` (`⚠️→✅`): promoted after verification — `--block-size` is now an alias for `--delta-block`, both set `config->delta_block_size`, which the delta engine already honored end-to-end (`delta_signature_create_seeded` + `delta_apply`); out-of-range values keep the default. `--fake-super` (`⚠️→✅`): added `fake_super_restore_fd` to parse and re-apply the recorded `user.fastsync.stat` record fd-relative (mode/time only — protocol 2.23.0: **never a real chown**; the resolved owner is recorded for a later privileged restore); a save under `--fake-super` now re-applies the recorded attrs instead of only recording them. (The later fake-super xattr-interop pass replaced that native `user.fastsync.stat` format with rsync's `user.rsync.%stat` grammar — see the row and Phase-4 notes.) `--stderr=client` (`⚠️→❌ Divergent`): FastSync has no rsync client-message channel, and `client` is rejected at CLI parse — the rejection is the documented behavior (unit-tested). `-N`/`--crtimes` (`⚠️→❌ Divergent`): birth-times cannot be set by any portable fs call (`utimensat` sets only atime/mtime); capture/transmit stays, setting is impossible, the flag is accepted and safely inert. Review-hardening (post-eval): fake-super replay applies the mode through the shared `metadata_mode_for_policy` helper (protocol 2.23.0: exactly the source mode under `-p`, with no masking); `--sparse` takes precedence over `--preallocate` (posix_fallocate skipped so holes survive) — **reversed by the parity-completion wave: `--preallocate` now wins, matching rsync**; `--partial` retention is disabled for `--no_replace` (ignore/existing) and only marks a write-attempt after the actual write begins; `--block-size=SIZE`/`--delta-block=SIZE` inline forms are accepted.
**Wave C — Devices & special files (finalize statuses + tests) (✅ implemented).** The four special-file rows are finalized with coverage tests. `--devices`, `--copy-devices`, and `--write-devices` are **✅ Implemented**, each with a documented, safety-driven divergence: device-node creation is privilege-gated, so a receiver without `CAP_MKNOD` skips that entry with a warning (a per-entry skip, never a transfer failure); `--copy-devices` copies a device/FIFO's reported size into an ordinary regular file (a size-bounded safe divergence from rsync's unbounded dd-like read); `--write-devices` writes only into an existing char/block node under the confined receive root and skips every unusable target rather than clobbering or aborting. `--specials` reclassified from **⛔ Impossible/Divergence** to **✅ Parity** in protocol 2.23.0: **FIFO recreation works** (unprivileged `mkfifo`) **and unix sockets are recreated** with `mknod(S_IFSOCK)`, which Linux permits unprivileged (the flag previously assumed sockets were impossible — see the `--specials` row). Tests assert FIFO recreation, socket recreation, the regular-file result of `--copy-devices`, the skipped/missing and non-device `--write-devices` targets, and (root-gated) real device-node creation; a root runner additionally drops the receiver to an unprivileged user to assert the `CAP_MKNOD` skip is graceful. (The parity-completion wave later reclassified `--devices`, `--copy-devices`, and `--write-devices` as explicit **❌ Divergent** rows, because their safe subsets are deliberately not rsync's behavior; the implementation itself is unchanged.) **Wave C — Devices & special files (finalize statuses + tests) (✅ implemented).** The four special-file rows are finalized with coverage tests. `--devices`, `--copy-devices`, and `--write-devices` are **✅ Implemented**, each with a documented, safety-driven divergence: device-node creation is privilege-gated, so a receiver without `CAP_MKNOD` skips that entry with a warning (a per-entry skip, never a transfer failure); `--copy-devices` copies a device/FIFO's reported size into an ordinary regular file (a size-bounded safe divergence from rsync's unbounded dd-like read); `--write-devices` writes only into an existing char/block node under the confined receive root and skips every unusable target rather than clobbering or aborting. `--specials` reclassified from **⛔ Impossible/Divergence** to **✅ Parity** in protocol 2.23.0: **FIFO recreation works** (unprivileged `mkfifo`) **and unix sockets are recreated** with `mknod(S_IFSOCK)`, which Linux permits unprivileged (the flag previously assumed sockets were impossible — see the `--specials` row). Tests assert FIFO recreation, socket recreation, the regular-file result of `--copy-devices`, the skipped/missing and non-device `--write-devices` targets, and (root-gated) real device-node creation; a root runner additionally drops the receiver to an unprivileged user to assert the `CAP_MKNOD` skip is graceful. (The parity-completion wave later reclassified `--devices`, `--copy-devices`, and `--write-devices` as explicit **❌ Divergent** rows, because their safe subsets are deliberately not rsync's behavior; the implementation itself is unchanged.)
@@ -956,7 +964,7 @@ These are the last compatibility items and the closing phase toward rsync flag p
**Wire:** two trailing config-frame blocks after the `--iconv` spec, in fixed order — `send_privilege_options`/`receive_privilege_options` (one `super_mode` int, validated `0..2`), then `send_copy_as_options`/`receive_copy_as_options` (presence int + two int32 ids, validated `>= 0`, with `copy_as_set ⇒ use_metadata`). `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Divergences from rsync:** rsync's `--super` elevates the receiver and `--copy-as` actually switches its credentials; FastSync never elevates and only permits/forwards confined attempts, and `--copy-as` forces ownership rather than switching identity. **Wire:** two trailing config-frame blocks after the `--iconv` spec, in fixed order — `send_privilege_options`/`receive_privilege_options` (one `super_mode` int, validated `0..2`), then `send_copy_as_options`/`receive_copy_as_options` (presence int + two int32 ids, validated `>= 0`, with `copy_as_set ⇒ use_metadata`). `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Divergences from rsync:** rsync's `--super` elevates the receiver and `--copy-as` actually switches its credentials; FastSync never elevates and only permits/forwards confined attempts, and `--copy-as` forces ownership rather than switching identity.
**Honest status after the parity 2.29 cycle (protocol 2.28.0, no wire change), updated by the parity cycle 2.29 pass, the audit-cycle follow-ups, the triage cycle, and a later no-wire parity pass.** ✅ Parity 119 / ⚠️ Caveat 12 / ❌ Divergent 26 = 157 rows. The no-wire parity pass accepted `--inc-recursive`/`--no-inc-recursive` as inert no-ops (❌ → ✅, since FastSync's full scan is rsync's `--no-inc-recursive` and the destination is identical), narrowed the `--temp-dir` divergence by accepting an absolute path that canonicalizes inside the receive root (the row stays ❌ for out-of-root absolute paths), and closed the `--delete-before` phase-0 divergence (⚠️ → ✅: the single-threaded data pass now replays the pre-scan file list, so a source file created after the scan is neither transferred nor kept, matching rsync). The 2.29 cycle closed the scanner-order, delete-timing, relative-basis, and fuzzy-eligibility residuals (moving `-n`/`--delete`/`--del`/`--delete-delay` to ✅) and improved the `--info`/`--stats`/`--debug` partial rows; the triage cycle moved `-F` and `-i`/`--itemize-changes` ✅ → ⚠️ for their documented residuals. The remaining ⚠️ rows are `--info`, `--debug`, `--msgs2stderr`, `--stats`, `--progress`, `-i`, `--filter`, `-F`, the three basis-dir options, and `-y/--fuzzy`. Earlier: **Honest status after the parity 2.28.0 cycle (protocol 2.28.0), updated by the rsync-parity-stats, rsync-parity-options, rsync-parity-fs, parity-review, no-wire parity-track-1/2b and wire parity-track-4a/5a passes.** ✅ Parity 116 / ⚠️ Caveat 14 / ❌ Divergent 27 = 157 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. This reclassification makes every difference explicit. The completion wave closed 23 previously-caveated rows (9 that triage showed were already parity, plus 14 genuine fixes) and turned the 17 inherently non-rsync rows — native daemon config/auth, the FastSync batch container, the safe-subset device/privilege flags, `-X`'s privileged namespaces, `--fake-super`'s native xattr format, and the `--old-args` no-op — into explicit ❌ divergences. The stats pass flipped `--delete-delay` to ✅ (actual-removal accounting), but the parity-review pass moved it back to ⚠️ because FastSync charged the `--max-delete` budget at plan/snapshot time and left a refilled snapshotted directory in place, whereas rsync charges on actual removals and recursively removes a queued directory (including content created after its plan). The no-wire parity-track-1 pass fixed both (actual-removal charging plus recursive deferred removal with an independent deferred-list cap), narrowing the caveat to the partial-delete ordering. The stats pass also reclassified `--out-format` to ❌ (protocol-specific `%b`/delta-`%c`), and sharpened the `--stats`/`--progress`/`--checksum-choice` residuals. The options pass flipped `--bwlimit` and `--ignore-errors` to ✅ (rsync-exact size parsing and ~100 ms leaky-bucket throttling, and rsync's skip-unreadable-subdir plus IO-error-suppressed deletion with exit 23) and emits rsync-format `--info=name/flist/del/remove/nonreg/progress` lines (real-run `deleting`/`*deleting` carried over a new trailing `report_deletes` wire bool, `PROTOCOL_VERSION` 2.26.0 → 2.27.0), while reclassifying `-M` over daemon/TCP **Honest status after the parity 2.29 cycle (protocol 2.28.0, no wire change), updated by the parity cycle 2.29 pass, the audit-cycle follow-ups, the triage cycle, and a later no-wire parity pass.** ✅ Parity 119 / ⚠️ Caveat 14 / ❌ Divergent 24 = 157 rows. The no-wire parity pass accepted `--inc-recursive`/`--no-inc-recursive` as inert no-ops (❌ → ✅, since FastSync's full scan is rsync's `--no-inc-recursive` and the destination is identical), narrowed the `--temp-dir` divergence by accepting an absolute path that canonicalizes inside the receive root (the row stays ❌ for out-of-root absolute paths), closed the `--delete-before` phase-0 divergence (⚠️ → ✅: the single-threaded data pass now replays the pre-scan file list, so a source file created after the scan is neither transferred nor kept, matching rsync), and moved `--fake-super` and `--devices` ❌ → ⚠️ (`--fake-super` now writes/reads rsync's exact `user.rsync.%stat` key and `<octal-mode> <rdev_major>,<rdev_minor> <uid>:<gid>` grammar, interoperating with real rsync 3.4.1 for regular files; `--devices` now surfaces a failed device `mknod` as a transfer error instead of a silent non-root skip — see those rows for the remaining directory/device-faking and exit-code residuals). The 2.29 cycle closed the scanner-order, delete-timing, relative-basis, and fuzzy-eligibility residuals (moving `-n`/`--delete`/`--del`/`--delete-delay` to ✅) and improved the `--info`/`--stats`/`--debug` partial rows; the triage cycle moved `-F` and `-i`/`--itemize-changes` ✅ → ⚠️ for their documented residuals. The remaining ⚠️ rows are `--info`, `--debug`, `--msgs2stderr`, `--stats`, `--progress`, `-i`, `--filter`, `-F`, the three basis-dir options, `-y/--fuzzy`, `--fake-super`, and `--devices`. Earlier: **Honest status after the parity 2.28.0 cycle (protocol 2.28.0), updated by the rsync-parity-stats, rsync-parity-options, rsync-parity-fs, parity-review, no-wire parity-track-1/2b and wire parity-track-4a/5a passes.** ✅ Parity 116 / ⚠️ Caveat 14 / ❌ Divergent 27 = 157 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. This reclassification makes every difference explicit. The completion wave closed 23 previously-caveated rows (9 that triage showed were already parity, plus 14 genuine fixes) and turned the 17 inherently non-rsync rows — native daemon config/auth, the FastSync batch container, the safe-subset device/privilege flags, `-X`'s privileged namespaces, `--fake-super`'s native xattr format, and the `--old-args` no-op — into explicit ❌ divergences. The stats pass flipped `--delete-delay` to ✅ (actual-removal accounting), but the parity-review pass moved it back to ⚠️ because FastSync charged the `--max-delete` budget at plan/snapshot time and left a refilled snapshotted directory in place, whereas rsync charges on actual removals and recursively removes a queued directory (including content created after its plan). The no-wire parity-track-1 pass fixed both (actual-removal charging plus recursive deferred removal with an independent deferred-list cap), narrowing the caveat to the partial-delete ordering. The stats pass also reclassified `--out-format` to ❌ (protocol-specific `%b`/delta-`%c`), and sharpened the `--stats`/`--progress`/`--checksum-choice` residuals. The options pass flipped `--bwlimit` and `--ignore-errors` to ✅ (rsync-exact size parsing and ~100 ms leaky-bucket throttling, and rsync's skip-unreadable-subdir plus IO-error-suppressed deletion with exit 23) and emits rsync-format `--info=name/flist/del/remove/nonreg/progress` lines (real-run `deleting`/`*deleting` carried over a new trailing `report_deletes` wire bool, `PROTOCOL_VERSION` 2.26.0 → 2.27.0), while reclassifying `-M` over daemon/TCP
and receiver-side `protect`/`risk` re-derivation to ❌ (no argv channel / and receiver-side `protect`/`risk` re-derivation to ❌ (no argv channel /
receiver filter engine); the wire parity-track-4a pass later added that receiver filter engine); the wire parity-track-4a pass later added that
receiver filter engine, flipping `--filter=RULE` back to ✅ (see above; the receiver filter engine, flipping `--filter=RULE` back to ✅ (see above; the
@@ -1059,9 +1067,11 @@ integration tests unless it is explicitly listed as a limitation.
clear configuration error (matching rsync) instead of an order-dependent clear configuration error (matching rsync) instead of an order-dependent
winner. winner.
- **`--fake-super` never real-chowns.** It records the *resolved* owner (the - **`--fake-super` never real-chowns.** It records the *resolved* owner (the
active mapping, else the source id) in `user.fastsync.stat` for a later active mapping, else the source id) in rsync's `user.rsync.%stat` for a later
privileged restore and replays only mode/times. Directory ownership and privileged restore and replays only the permission bits (mtime travels through
directory xattrs/ACLs are preserved alongside file entries. the normal metadata path). Directory ownership and
directory xattrs/ACLs are preserved alongside file entries, though directories
themselves are not yet given a `%stat%` record.
- **`--chmod`** implements rsync's `D`/`F`/`X` selectors, `s`/`t`, append - **`--chmod`** implements rsync's `D`/`F`/`X` selectors, `s`/`t`, append
semantics, does not imply `-p`, and applies its changes without sanitization. semantics, does not imply `-p`, and applies its changes without sanitization.
@@ -1260,8 +1270,8 @@ These remain after the wave; the individual rows carry the precise wording.
Native daemon config/auth (`--daemon`, `--config`, `--dparam`, Native daemon config/auth (`--daemon`, `--config`, `--dparam`,
`--password-file`, `--early-input`, `--hash-credentials`/`--iterations`), the `--password-file`, `--early-input`, `--hash-credentials`/`--iterations`), the
non-interoperable batch container (`--write-batch`/`--only-write-batch`/ non-interoperable batch container (`--write-batch`/`--only-write-batch`/
`--read-batch`), `--fake-super`'s native xattr format, `-X`'s privileged `--read-batch`), `-X`'s privileged
namespaces, `--devices`/`--copy-devices`/`--write-devices`'s safe subsets, namespaces, `--copy-devices`/`--write-devices`'s safe subsets,
`--super`/`--copy-as`'s refusal to elevate or switch credentials, and the `--super`/`--copy-as`'s refusal to elevate or switch credentials, and the
`-s`/`--secluded-args`/`--protect-args`/`--old-args` accepted no-ops. `-s`/`--secluded-args`/`--protect-args`/`--old-args` accepted no-ops.
+5 -4
View File
@@ -208,10 +208,11 @@ void print_usage(void) {
printf(" -A, --acls Preserve POSIX ACLs (the system.posix_acl_* xattrs;\n"); printf(" -A, --acls Preserve POSIX ACLs (the system.posix_acl_* xattrs;\n");
printf(" setting an ACL the receiver is not permitted to\n"); printf(" setting an ACL the receiver is not permitted to\n");
printf(" set is warned and skipped, never fatal)\n"); printf(" set is warned and skipped, never fatal)\n");
printf(" --fake-super Store the source uid/gid/mode/mtime in a reserved\n"); printf(" --fake-super Store the source mode/rdev/uid/gid in rsync's\n");
printf(" user.fastsync.stat xattr on each written file and\n"); printf(" reserved user.rsync.%%stat xattr on each written\n");
printf(" re-apply it (fd-relative) on a privileged run; the\n"); printf(" file (interoperable with rsync); it never performs a\n");
printf(" recording format diverges from rsync's user.rsync.%%stat%%\n"); printf(" real chown, so an unprivileged receiver records the\n");
printf(" privileged stat for a later restore\n");
printf(" --super Permit the receiver to attempt super-user activities\n"); printf(" --super Permit the receiver to attempt super-user activities\n");
printf(" (char/block device-node creation, --write-devices)\n"); printf(" (char/block device-node creation, --write-devices)\n");
printf(" within the confined receive root. Never elevates\n"); printf(" within the confined receive root. Never elevates\n");
+6 -4
View File
@@ -735,11 +735,13 @@ typedef struct Config {
* --copy-as) imply it. */ * --copy-as) imply it. */
/* fake_super */ /* fake_super */
/* --fake-super: receiver-only. When set, each written file additionally gets /* --fake-super: receiver-only. When set, each written file additionally gets
* a reserved user.fastsync.stat xattr recording the RESOLVED uid/gid (the * rsync's reserved user.rsync.%stat xattr recording the RESOLVED uid/gid (the
* source's own when no ownership request is active, else the --chown/--usermap * source's own when no ownership request is active, else the --chown/--usermap
* result) plus mode/mtime so a later privileged restore could re-apply them. * result) plus the full mode and rdev, in rsync 3.4.1's grammar, so the tree is
* It NEVER real-chowns: the point is to record the source ownership on an * interoperable and a later privileged restore could re-apply them. mtime is
* unprivileged receiver. Crosses the wire. */ * carried by the file's own timestamp, exactly as rsync does it. It NEVER
* real-chowns: the point is to record the source ownership on an unprivileged
* receiver. Crosses the wire. */
/* module */ /* module */
/* Daemon module selection (Wave A, protocol 2.15.0). Client-composed from a /* Daemon module selection (Wave A, protocol 2.15.0). Client-composed from a
* host::module/path destination; NULL or "" means "no module" (the ordinary * host::module/path destination; NULL or "" means "no module" (the ordinary
+5 -4
View File
@@ -1189,14 +1189,15 @@ static void restore_extra_fd(int fd, const FileMetadata* metadata, const FileXat
ownership request (--chown/--usermap/--groupmap/--copy-as or -o/-g) is ownership request (--chown/--usermap/--groupmap/--copy-as or -o/-g) is
active, the resolved mapping; otherwise the source's own id. The real active, the resolved mapping; otherwise the source's own id. The real
chown is suppressed (identity_apply_ownership early-returns under chown is suppressed (identity_apply_ownership early-returns under
--fake-super) so recording never defeats the flag. Mode/mtime are still --fake-super) so recording never defeats the flag. The recorded stat is
replayed (policy-gated) so unprivileged --fake-super keeps working. */ rsync's format; the permission bits are replayed (policy-gated) so
unprivileged --fake-super keeps working while mtime comes from the
normal metadata path above. */
uint32_t store_uid; uint32_t store_uid;
uint32_t store_gid; uint32_t store_gid;
identity_resolve_storage_ids((int32_t)metadata->uid, (int32_t)metadata->gid, &store_uid, identity_resolve_storage_ids((int32_t)metadata->uid, (int32_t)metadata->gid, &store_uid,
&store_gid); &store_gid);
fake_super_store_fd(fd, store_uid, store_gid, (uint32_t)metadata->mode, metadata->mtime_sec, fake_super_store_fd(fd, store_uid, store_gid, (uint32_t)metadata->mode, 0, 0);
metadata->mtime_nsec);
fake_super_restore_fd(fd, policy); fake_super_restore_fd(fd, policy);
} }
} }
+30 -9
View File
@@ -400,11 +400,13 @@ bool file_special_rdev_valid(int32_t major, int32_t minor, mode_t mode) {
/* ---- Device/special node RECREATION (--devices/--specials), receiver side ---- /* ---- Device/special node RECREATION (--devices/--specials), receiver side ----
* *
* Privilege gating: making a real device node requires CAP_MKNOD (root); making * Privilege gating: making a real device node requires CAP_MKNOD (root); making
* a FIFO works unprivileged (mkfifo). When the receiver lacks the capability, * a FIFO works unprivileged (mkfifo). A device node whose mknodat() fails with
* mknodat() fails with EPERM and the entry is SKIPPED with a warning -- the * EPERM/EACCES is a genuine transfer error (rsync parity: rsync reports the
* whole transfer must NOT abort just because the environment cannot make the * mknod failure and the run exits partial, code 23). Only the unprivileged
* node. CI runs non-root, so device creation is expected to skip there and * FIFO/socket (--specials) path keeps the best-effort skip, because those are
* only a FIFO is honestly assertable unprivileged. * normally creatable without privilege and a failure there is environmental.
* CI runs non-root, so device creation is expected to fail there; only a FIFO
* is honestly assertable unprivileged.
* *
* Confinement: the parent directory is opened fd-relative below the receive * Confinement: the parent directory is opened fd-relative below the receive
* root (file_open_secure_parent: O_NOFOLLOW, no "..", root-checked) and the * root (file_open_secure_parent: O_NOFOLLOW, no "..", root-checked) and the
@@ -542,13 +544,32 @@ static FileSaveResult file_save_special_to_disk(const char* root_directory, cons
node_kind, escaped_path ? escaped_path : "<allocation failed>"); node_kind, escaped_path ? escaped_path : "<allocation failed>");
free(escaped_path); free(escaped_path);
} else if (errno == EPERM || errno == EACCES) { } else if (errno == EPERM || errno == EACCES) {
/* Missing CAP_MKNOD / parent write permission: the environment cannot
create the node, so skip instead of failing the whole run. */
char* escaped_path = output_escape(file->path, log_get_8_bit_output()); char* escaped_path = output_escape(file->path, log_get_8_bit_output());
const char* shown_path = escaped_path ? escaped_path : "<allocation failed>";
if (is_char || is_blk) {
/* rsync parity: a device node that cannot be created (no CAP_MKNOD, or
* super-user activities not permitted) is a genuine transfer error.
* rsync reports `mknod ".../node" failed: ...` and the run exits
* partial (23); FastSync surfaces it through the outcome aggregation
* instead of silently skipping the entry. FIFO/socket creation
* (--specials) keeps the best-effort skip path below. */
log_message(LOG_LEVEL_ERROR,
"cannot create %s %s: %s\n"
" --devices node creation needs privilege (CAP_MKNOD)",
node_kind, shown_path, strerror(errno));
free(escaped_path);
close(parent_fd);
free(leaf);
free(destination);
return FILE_SAVE_ERROR;
}
/* Missing CAP_MKNOD / parent write permission for a FIFO/socket: the
environment cannot create the node, so skip instead of failing the
whole run. */
log_message(LOG_LEVEL_WARNING, log_message(LOG_LEVEL_WARNING,
"skipping %s: cannot create %s node (%s)\n" "skipping %s: cannot create %s node (%s)\n"
" --devices/--specials node creation needs privilege (CAP_MKNOD)", " --specials node creation needs privilege (CAP_MKNOD)",
escaped_path ? escaped_path : "<allocation failed>", node_kind, strerror(errno)); shown_path, node_kind, strerror(errno));
free(escaped_path); free(escaped_path);
} else { } else {
char* escaped_path = output_escape(file->path, log_get_8_bit_output()); char* escaped_path = output_escape(file->path, log_get_8_bit_output());
+35 -33
View File
@@ -363,16 +363,21 @@ bool xattr_apply_fd(int fd, const FileXattrList* list) {
return true; return true;
} }
/* ---- --fake-super: park ownership/mode/mtime in a reserved xattr ---- */ /* ---- --fake-super: park ownership/mode/rdev in a reserved xattr ---- */
void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, int64_t mtime_sec, void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, uint32_t rdev_major,
int64_t mtime_nsec) { uint32_t rdev_minor) {
if (fd < 0) if (fd < 0)
return; return;
char record[128]; /* rsync 3.4.1's exact grammar: "<octal full st_mode> <rdev_major>,<rdev_minor>
int len = * <uid>:<gid>". The octal mode carries the S_IFMT bits (e.g. 0104711 for a
snprintf(record, sizeof(record), "%lu:%lu:%03o:%lld:%ld", (unsigned long)uid, * setuid regular file, 020644 for a char device); the rdev pair is 0,0 for a
(unsigned long)gid, (unsigned)mode & 0777U, (long long)mtime_sec, (long)mtime_nsec); * non-device. No mtime field: rsync leaves the file's own timestamp in
* charge of mtime. This value is what rsync reads back to restore a
* fake-super tree, so the field order and separators must not change. */
char record[96];
int len = snprintf(record, sizeof(record), "%o %u,%u %u:%u", (unsigned)mode, (unsigned)rdev_major,
(unsigned)rdev_minor, (unsigned)uid, (unsigned)gid);
if (len <= 0 || (size_t)len >= sizeof(record)) if (len <= 0 || (size_t)len >= sizeof(record))
return; return;
if (fsetxattr(fd, FAKESUPER_XATTR, record, (size_t)len, 0) != 0) { if (fsetxattr(fd, FAKESUPER_XATTR, record, (size_t)len, 0) != 0) {
@@ -381,11 +386,14 @@ void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, int6
} }
} }
/* --fake-super replay: read the freshly-stored record and re-apply mode/mtime /* --fake-super replay: read the freshly-stored record and re-apply its
* fd-relative. The recorded uid/gid are retained for a later privileged * permission bits fd-relative. The recorded uid/gid are retained for a later
* restore but are NEVER chowned here: --fake-super only RECORDS ownership, it * privileged restore but are NEVER chowned here: --fake-super only RECORDS
* must not real-chown the recorded (resolved) owner. Mode/mtime still apply so * ownership, it must not real-chown the recorded (resolved) owner. The
* unprivileged --fake-super keeps working. */ * recorded rdev is likewise parsed for grammar compatibility but is not acted
* on (device recreation is a separate, privilege-gated path). mtime is not in
* the record: the normal metadata path applies it (policy.times), exactly as
* rsync relies on the file's own timestamp. */
bool fake_super_restore_fd(int fd, FileAttrPolicy policy) { bool fake_super_restore_fd(int fd, FileAttrPolicy policy) {
if (fd < 0) if (fd < 0)
return false; return false;
@@ -394,24 +402,25 @@ bool fake_super_restore_fd(int fd, FileAttrPolicy policy) {
if (len < 0) if (len < 0)
return false; /* absent or filesystem without xattrs: silent no-op */ return false; /* absent or filesystem without xattrs: silent no-op */
record[len] = '\0'; record[len] = '\0';
unsigned long ul_uid, ul_gid, ul_mode; unsigned ul_mode, rdev_major, rdev_minor, ul_uid, ul_gid;
long long mtime_sec; if (sscanf(record, "%o %u,%u %u:%u", &ul_mode, &rdev_major, &rdev_minor, &ul_uid, &ul_gid) != 5)
long mtime_nsec;
if (sscanf(record, "%lu:%lu:%lo:%lld:%ld", &ul_uid, &ul_gid, &ul_mode, &mtime_sec, &mtime_nsec) !=
5)
return false; /* malformed record: skip, never fatal */ return false; /* malformed record: skip, never fatal */
/* --fake-super NEVER performs a real chown: that would defeat the whole /* --fake-super NEVER performs a real chown: that would defeat the whole point
point of the flag (record privileged ownership on an unprivileged receiver of the flag (record privileged ownership on an unprivileged receiver for a
for a later privileged restore). The uid/gid parsed above are retained in later privileged restore). The uid/gid parsed above are retained in the
the record for that later restore, but no ownership change happens here. */ record for that later restore, but no ownership change happens here. The
rdev is retained for the same reason. */
(void)rdev_major;
(void)rdev_minor;
(void)ul_uid; (void)ul_uid;
(void)ul_gid; (void)ul_gid;
/* Mode is applied only when the per-attribute policy asks for it, through the /* Mode is applied only when the per-attribute policy asks for it, through the
SAME shared helper the normal metadata path uses (metadata_mode_for_policy): SAME shared helper the normal metadata path uses (metadata_mode_for_policy).
under --perms the recorded source mode is copied exactly, including The recorded special bits are stripped first: rsync's fake-super receiver
group/other write and setuid/setgid/sticky bits (rsync parity), and the -E stores the full mode in the xattr but never installs setuid/setgid/sticky on
rule derives exec bits from the destination's read bits exactly like the real file, so only the 0777 permission bits may be replayed. The -E
rule then derives exec bits from the destination's read bits exactly like
file_restore_metadata_fd. */ file_restore_metadata_fd. */
if (policy.perms || policy.executability) { if (policy.perms || policy.executability) {
struct stat cur; struct stat cur;
@@ -419,19 +428,12 @@ bool fake_super_restore_fd(int fd, FileAttrPolicy policy) {
if (fstat(fd, &cur) != 0) { if (fstat(fd, &cur) != 0) {
log_message(LOG_LEVEL_WARNING, "--fake-super: could not read destination mode: %s", log_message(LOG_LEVEL_WARNING, "--fake-super: could not read destination mode: %s",
strerror(errno)); strerror(errno));
} else if (metadata_mode_for_policy((mode_t)ul_mode, cur.st_mode, policy, &want)) { } else if (metadata_mode_for_policy((mode_t)(ul_mode & 0777U), cur.st_mode, policy, &want)) {
if (fchmod(fd, want) != 0) if (fchmod(fd, want) != 0)
log_message(LOG_LEVEL_WARNING, log_message(LOG_LEVEL_WARNING,
"--fake-super: could not restore mode on destination file: %s", "--fake-super: could not restore mode on destination file: %s",
strerror(errno)); strerror(errno));
} }
} }
if (policy.times) {
struct timespec times[2] = {{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
{.tv_sec = (time_t)mtime_sec, .tv_nsec = mtime_nsec}};
if (futimens(fd, times) != 0)
log_message(LOG_LEVEL_WARNING,
"--fake-super: could not restore mtime on destination file: %s", strerror(errno));
}
return true; return true;
} }
+28 -21
View File
@@ -31,11 +31,14 @@
*/ */
/* Reserved key used by --fake-super to park the source's privileged ownership /* Reserved key used by --fake-super to park the source's privileged ownership
* / mode / mtime on the destination file as an unprivileged user.* xattr, so a * / mode / rdev on the destination file as an unprivileged user.* xattr, so the
* later privileged restore could re-apply them. Exact documented format: * tree is interoperable with rsync 3.4.1 and a later privileged restore can
* uid:gid:mode:mtime_sec:mtime_nsec (decimal, decimal, octal, dec, dec) * re-apply them. This is rsync's own key and value grammar exactly:
* e.g. "1000:1000:644:1765238400:0". */ * <octal st_mode with S_IFMT> <rdev_major>,<rdev_minor> <uid>:<gid>
#define FAKESUPER_XATTR "user.fastsync.stat" * e.g. "104711 0,0 1234:5678" for a setuid regular file owned by 1234:5678,
* or "20644 1,3 111:222" for a char device. mtime is deliberately NOT part of
* the record: exactly like rsync, the file's own timestamp carries it. */
#define FAKESUPER_XATTR "user.rsync.%stat"
/* --- bounds --- */ /* --- bounds --- */
#define XATTR_NAME_MAX 255 /* xattr names are limited to 255 bytes */ #define XATTR_NAME_MAX 255 /* xattr names are limited to 255 bytes */
@@ -91,24 +94,28 @@ FileXattrList* xattr_receive(int fd, int* ok, bool preserve_acls);
* true when apply was attempted (allowing callers to treat it as best-effort). */ * true when apply was attempted (allowing callers to treat it as best-effort). */
bool xattr_apply_fd(int fd, const FileXattrList* list); bool xattr_apply_fd(int fd, const FileXattrList* list);
/* --fake-super: write the source uid/gid/mode/mtime record into the reserved /* --fake-super: write the source uid/gid/mode/rdev record into the reserved
* FAKESUPER_XATTR on `fd`. Best-effort (logged, never fatal). Only meaningful * FAKESUPER_XATTR on `fd`, using rsync 3.4.1's exact grammar (see the key
* when metadata was transmitted so the values exist. */ * comment above). `mode` is the full st_mode including its S_IFMT bits.
void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, int64_t mtime_sec, * Best-effort (logged, never fatal). Only meaningful when metadata was
int64_t mtime_nsec); * transmitted so the values exist. */
void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, uint32_t rdev_major,
uint32_t rdev_minor);
/* --fake-super replay: parse the FAKESUPER_XATTR record previously written on /* --fake-super replay: parse the FAKESUPER_XATTR record previously written on
* `fd` by fake_super_store_fd and re-apply mode/mtime fd-relative. The * `fd` by fake_super_store_fd and re-apply the recorded permission bits
* recorded uid/gid are deliberately NOT chowned for real: --fake-super only * fd-relative. The recorded uid/gid are deliberately NOT chowned for real:
* RECORDS ownership (the caller stores the resolved mapping via * --fake-super only RECORDS ownership (the caller stores the resolved mapping
* identity_resolve_storage_ids), it never performs a real chown. Best-effort: * via identity_resolve_storage_ids), it never performs a real chown. The
* absence of the xattr or a malformed record is a silent no-op that never fails * recorded rdev is retained for a later privileged restore but is not acted on
* the transfer. The MODE leg is applied only when policy.perms||policy. * here. Best-effort: absence of the xattr or a malformed record is a silent
* executability and the MTIME leg only when policy.times, so the fake-super * no-op that never fails the transfer. The MODE leg is applied only when
* replay cannot bypass the per-attribute split; the mode follows the normal * policy.perms||policy.executability, and the recorded special bits
* metadata path exactly (under --perms the source mode is copied verbatim, * (setuid/setgid/sticky) are NOT applied to the real file -- exactly like
* special and group/other write bits included). * rsync's fake-super receiver, which stores the full mode in the xattr but
* Returns true when the xattr was present and parsed. */ * strips the special bits on disk. mtime is not part of the record; the normal
* metadata path carries it (policy.times) exactly as rsync sets the file's own
* timestamp. Returns true when the xattr was present and parsed. */
bool fake_super_restore_fd(int fd, FileAttrPolicy policy); bool fake_super_restore_fd(int fd, FileAttrPolicy policy);
#endif #endif
+94 -26
View File
@@ -183,10 +183,11 @@ class TestDeviceSpecial:
) )
@pytest.mark.setpriv @pytest.mark.setpriv
def test_devices_nonroot_receiver_skips_safely(self): def test_devices_nonroot_receiver_errors_like_rsync(self):
"""A receiver without CAP_MKNOD must skip a device entry with a warning """A receiver without CAP_MKNOD must report the failed device mknod as a
and never abort. A root runner drops the receiver (server) to nobody transfer error (rsync parity, partial failure) instead of silently
via setpriv; on a non-root runner (or without setpriv) the test skips.""" succeeding. A root runner drops the receiver (server) to nobody via
setpriv; on a non-root runner (or without setpriv) the test skips."""
if os.geteuid() != 0 or shutil.which("setpriv") is None: if os.geteuid() != 0 or shutil.which("setpriv") is None:
pytest.skip("requires root + setpriv to run the receiver unprivileged") pytest.skip("requires root + setpriv to run the receiver unprivileged")
self._setup() self._setup()
@@ -202,16 +203,16 @@ class TestDeviceSpecial:
flags=["--devices"], port=port) flags=["--devices"], port=port)
finally: finally:
out, err = _stop_captured_server(server) out, err = _stop_captured_server(server)
assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr[:300]}" assert result.returncode != 0, (
received = get_dest_received_dir(DEVICE_DEST, DEVICE_SOURCE) f"a failed device mknod must be a transfer error like rsync (got exit 0): "
with open(os.path.join(received, "plain.txt")) as f: f"{(out + err)[:300]}"
assert f.read() == "regular content\n"
assert not os.path.lexists(os.path.join(received, "chardev")), (
"a receiver without CAP_MKNOD must skip the device node, not create it"
) )
assert ("cannot create device node" in (out + err) received = get_dest_received_dir(DEVICE_DEST, DEVICE_SOURCE)
or "device-node creation is not permitted" in (out + err)), ( assert not os.path.lexists(os.path.join(received, "chardev")), (
f"receiver did not log the documented device skip: out={out!r} err={err!r}" "a receiver without CAP_MKNOD must not create the device node"
)
assert "cannot create device" in (out + err), (
f"receiver did not log the device creation error: out={out!r} err={err!r}"
) )
@pytest.mark.skipif(os.geteuid() != 0, reason="requires root to create device nodes") @pytest.mark.skipif(os.geteuid() != 0, reason="requires root to create device nodes")
@@ -6599,7 +6600,7 @@ class TestExtendedAttributes:
os.getxattr(os.path.join(received, "data.txt"), "user.foo") os.getxattr(os.path.join(received, "data.txt"), "user.foo")
def test_reserved_fake_super_key_not_forwarded(self, shared_server): def test_reserved_fake_super_key_not_forwarded(self, shared_server):
"""A source file that already carries the reserved user.fastsync.stat """A source file that already carries the reserved user.rsync.%stat
record must NOT have it planted on the receiver during a plain -X run record must NOT have it planted on the receiver during a plain -X run
(it is receiver-only, so it cannot be spoofed for a later privileged (it is receiver-only, so it cannot be spoofed for a later privileged
restore).""" restore)."""
@@ -6609,7 +6610,7 @@ class TestExtendedAttributes:
fh.write(b"reserved\n") fh.write(b"reserved\n")
if not _xattr_supported(f): if not _xattr_supported(f):
pytest.skip("filesystem does not support user xattrs") pytest.skip("filesystem does not support user xattrs")
os.setxattr(f, "user.fastsync.stat", b"0:0:644:0:0") os.setxattr(f, "user.rsync.%stat", b"100644 0,0 0:0")
# A normal user.* attr still travels alongside. # A normal user.* attr still travels alongside.
os.setxattr(f, "user.keep", b"yes") os.setxattr(f, "user.keep", b"yes")
@@ -6619,7 +6620,7 @@ class TestExtendedAttributes:
received = get_dest_received_dir(dest, source) received = get_dest_received_dir(dest, source)
assert os.getxattr(os.path.join(received, "data.txt"), "user.keep") == b"yes" assert os.getxattr(os.path.join(received, "data.txt"), "user.keep") == b"yes"
with pytest.raises(OSError): with pytest.raises(OSError):
os.getxattr(os.path.join(received, "data.txt"), "user.fastsync.stat") os.getxattr(os.path.join(received, "data.txt"), "user.rsync.%stat")
@pytest.mark.ci @pytest.mark.ci
def test_xattrs_multithreaded(self, shared_server): def test_xattrs_multithreaded(self, shared_server):
@@ -6708,10 +6709,17 @@ class TestExtendedAttributes:
assert result.returncode == 0, \ assert result.returncode == 0, \
f"--fake-super sync failed: {(result.stderr or result.stdout)[:300]}" f"--fake-super sync failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source) received = get_dest_received_dir(dest, source)
record = os.getxattr(os.path.join(received, "data.txt"), "user.fastsync.stat").decode() record = os.getxattr(os.path.join(received, "data.txt"), "user.rsync.%stat").decode()
fields = record.split(":") # rsync 3.4.1 grammar: "<octal st_mode> <rdev_major>,<rdev_minor> <uid>:<gid>".
assert len(fields) == 5 fields = record.split()
assert fields[0] == str(uid), f"reserved uid field {fields[0]} != source uid {uid}" assert len(fields) == 3, f"unexpected rsync fake-super record {record!r}"
mode_field, rdev_field, owner_field = fields
assert rdev_field == "0,0", f"regular file rdev must be 0,0, got {rdev_field!r}"
assert int(mode_field, 8) & 0o170000 == stat.S_IFREG, (
f"recorded mode {mode_field!r} must carry S_IFREG"
)
assert owner_field.split(":")[0] == str(uid), \
f"recorded uid {owner_field!r} != source uid {uid}"
@pytest.mark.ci @pytest.mark.ci
def test_fake_super_records_resolved_chown_without_real_chown(self, shared_server): def test_fake_super_records_resolved_chown_without_real_chown(self, shared_server):
@@ -6730,12 +6738,71 @@ class TestExtendedAttributes:
assert result.returncode == 0, \ assert result.returncode == 0, \
f"--fake-super --chown sync failed: {(result.stderr or result.stdout)[:300]}" f"--fake-super --chown sync failed: {(result.stderr or result.stdout)[:300]}"
dst = os.path.join(get_dest_received_dir(dest, source), "data.txt") dst = os.path.join(get_dest_received_dir(dest, source), "data.txt")
record = os.getxattr(dst, "user.fastsync.stat").decode().split(":") record = os.getxattr(dst, "user.rsync.%stat").decode().split()
assert record[0] == "33333", f"recorded owner {record[0]} != resolved 33333" owner = record[2].split(":")
assert record[1] == "44444", f"recorded group {record[1]} != resolved 44444" assert owner == ["33333", "44444"], (
f"recorded owner {record[2]!r} != resolved 33333:44444"
)
st = os.stat(dst) st = os.stat(dst)
assert st.st_uid != 33333, "--fake-super must not real-chown the recorded owner" assert st.st_uid != 33333, "--fake-super must not real-chown the recorded owner"
@pytest.mark.ci
def test_fake_super_rsync_interop(self, shared_server):
"""A fake-super tree written by FastSync is readable by rsync 3.4.1:
rsync reads the `user.rsync.%stat` record (mode/rdev/uid:gid) and, when
it re-emits a fake-super tree, reproduces the same record. This pins
the on-disk key and value grammar against the real tool."""
rsync = shutil.which("rsync")
if rsync is None:
pytest.skip("rsync not installed")
source, dest = self._source_and_dest("fakesuper_interop")
f = os.path.join(source, "data.txt")
with open(f, "wb") as fh:
fh.write(b"interop\n")
if not _xattr_supported(f):
pytest.skip("filesystem does not support user xattrs")
# rsync's fake-super receiver only writes a %stat% record when it has
# something to fake; a root-owned file with a matching root stat is
# a no-op. When privileged, record a non-root owner so the round-trip
# actually exercises the parser (non-root CI already has a non-zero uid).
if os.geteuid() == 0:
try:
os.chown(f, 12345, 12346)
except OSError:
pass
# A setuid bit exercises the full st_mode encoding; set it AFTER any
# chown (chown clears setuid/setgid), and note that neither tool installs
# it on the real destination file.
os.chmod(f, 0o4711)
result, _ = run_client(source, dest, flags=["--fake-super"],
port=shared_server.port)
assert result.returncode == 0, \
f"--fake-super sync failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
rec = os.getxattr(os.path.join(received, "data.txt"), "user.rsync.%stat").decode()
rec_fields = rec.split()
assert len(rec_fields) == 3 and rec_fields[1] == "0,0", (
f"FastSync did not write rsync's stat grammar: {rec!r}"
)
assert int(rec_fields[0], 8) & 0o7777 == 0o4711, (
f"FastSync did not record the source mode in rsync's grammar: {rec!r}"
)
out = os.path.join(TEST_DATA_DIR, "fakesuper_interop_rsync")
clean_dir(out)
rs = subprocess.run([rsync, "-aX", "--fake-super",
received + "/", out + "/"],
capture_output=True, text=True, timeout=120)
assert rs.returncode == 0, (
f"rsync could not read FastSync's fake-super tree: {rs.stderr[:300]}"
)
out_rec = os.getxattr(os.path.join(out, "data.txt"), "user.rsync.%stat").decode()
assert out_rec == rec, (
"rsync re-emitted a different fake-super record; FastSync's grammar "
f"is not interoperable: ours={rec!r} rsync={out_rec!r}"
)
@pytest.mark.ci @pytest.mark.ci
def test_directory_xattrs_preserved(self, shared_server): def test_directory_xattrs_preserved(self, shared_server):
"""#286.3: -aX must preserve user.* xattrs on DIRECTORIES, not just files.""" """#286.3: -aX must preserve user.* xattrs on DIRECTORIES, not just files."""
@@ -7255,9 +7322,10 @@ class TestCopyAs:
) )
received = get_dest_received_dir(dest, source) received = get_dest_received_dir(dest, source)
dst = os.path.join(received, "mixed.txt") dst = os.path.join(received, "mixed.txt")
record = os.getxattr(dst, "user.fastsync.stat").decode().split(":") record = os.getxattr(dst, "user.rsync.%stat").decode().split()
assert (record[0], record[1]) == ("65534", "65534"), ( owner = record[2].split(":")
f"fake-super must record the resolved copy-as ownership: {record[:2]}" assert owner == ["65534", "65534"], (
f"fake-super must record the resolved copy-as ownership: {owner}"
) )
st = os.lstat(dst) st = os.lstat(dst)
assert (st.st_uid, st.st_gid) != (12345, 12346), ( assert (st.st_uid, st.st_gid) != (12345, 12346), (
+4 -4
View File
@@ -353,10 +353,10 @@ class TestOwnershipRoot:
st = os.stat(dst) st = os.stat(dst)
assert st.st_uid != 12345, \ assert st.st_uid != 12345, \
f"--fake-super -o must NOT real-chown the source owner, got uid={st.st_uid}" f"--fake-super -o must NOT real-chown the source owner, got uid={st.st_uid}"
record = os.getxattr(dst, "user.fastsync.stat").decode() record = os.getxattr(dst, "user.rsync.%stat").decode()
fields = record.split(":") owner = record.split()[2].split(":")
assert fields[0] == "12345", \ assert owner[0] == "12345", \
f"--fake-super must record the resolved owner, got {fields[0]}" f"--fake-super must record the resolved owner, got {owner[0]}"
def test_o_applies_directory_owner(self, shared_server): def test_o_applies_directory_owner(self, shared_server):
"""#286.2: -o must apply the source owner to DIRECTORIES too (the """#286.2: -o must apply the source owner to DIRECTORIES too (the
+62 -8
View File
@@ -156,8 +156,8 @@ static void test_xattr_capture_and_appliable() {
EXPECT_TRUE(xattr_name_appliable("user.foo", false)); EXPECT_TRUE(xattr_name_appliable("user.foo", false));
EXPECT_TRUE(xattr_name_appliable("user.foo", true)); EXPECT_TRUE(xattr_name_appliable("user.foo", true));
/* The reserved fake-super key is receiver-only and never forwarded/applied. */ /* The reserved fake-super key is receiver-only and never forwarded/applied. */
EXPECT_FALSE(xattr_name_appliable("user.fastsync.stat", false)); EXPECT_FALSE(xattr_name_appliable("user.rsync.%stat", false));
EXPECT_FALSE(xattr_name_appliable("user.fastsync.stat", true)); EXPECT_FALSE(xattr_name_appliable("user.rsync.%stat", true));
/* B4: the ACL names require --acls; -X alone must not authorize them. */ /* B4: the ACL names require --acls; -X alone must not authorize them. */
EXPECT_FALSE(xattr_name_appliable("system.posix_acl_access", false)); EXPECT_FALSE(xattr_name_appliable("system.posix_acl_access", false));
EXPECT_FALSE(xattr_name_appliable("system.posix_acl_default", false)); EXPECT_FALSE(xattr_name_appliable("system.posix_acl_default", false));
@@ -351,9 +351,10 @@ static void test_xattr_capture_filters_acls() {
} }
/* --fake-super replay: fake_super_store_fd records the source stat into the /* --fake-super replay: fake_super_store_fd records the source stat into the
* reserved xattr, and fake_super_restore_fd re-applies mode/mtime (and owner, * reserved xattr, and fake_super_restore_fd re-applies the permission bits
* when the process may) fd-relative. Restore must also be a safe no-op with no * fd-relative (mtime travels through the normal metadata path; the owner is
* xattr present. Guarded on filesystem xattr support. */ * never chowned). Restore must also be a safe no-op with no xattr present.
* Guarded on filesystem xattr support. */
static void test_fake_super_restore() { static void test_fake_super_restore() {
const char* path = "test_fake_super_restore.txt"; const char* path = "test_fake_super_restore.txt";
unlink(path); unlink(path);
@@ -373,7 +374,7 @@ static void test_fake_super_restore() {
FileAttrPolicy policy = {true, true, false, false, true}; FileAttrPolicy policy = {true, true, false, false, true};
EXPECT_FALSE(fake_super_restore_fd(fd, policy)); EXPECT_FALSE(fake_super_restore_fd(fd, policy));
fake_super_store_fd(fd, 1001, 1002, 0751, 1700000000, 123456789); fake_super_store_fd(fd, 1001, 1002, S_IFREG | 0751, 0, 0);
EXPECT_TRUE(fake_super_restore_fd(fd, policy)); EXPECT_TRUE(fake_super_restore_fd(fd, policy));
struct stat st; struct stat st;
EXPECT_EQ_INT(fstat(fd, &st), 0); EXPECT_EQ_INT(fstat(fd, &st), 0);
@@ -381,7 +382,7 @@ static void test_fake_super_restore() {
/* Strict rsync parity: -p restores the recorded mode exactly, including /* Strict rsync parity: -p restores the recorded mode exactly, including
group/other write (a recorded 0666 restores as 0666). */ group/other write (a recorded 0666 restores as 0666). */
fake_super_store_fd(fd, 1001, 1002, 0666, 1700000000, 0); fake_super_store_fd(fd, 1001, 1002, S_IFREG | 0666, 0, 0);
EXPECT_TRUE(fake_super_restore_fd(fd, policy)); EXPECT_TRUE(fake_super_restore_fd(fd, policy));
EXPECT_EQ_INT(fstat(fd, &st), 0); EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0666); EXPECT_EQ_INT((int)(st.st_mode & 0777), 0666);
@@ -401,6 +402,58 @@ static void test_fake_super_restore() {
unlink(path); unlink(path);
} }
/* The stored record is rsync 3.4.1's exact grammar
* "<octal st_mode with S_IFMT> <rdev_major>,<rdev_minor> <uid>:<gid>"
* so a fake-super tree is readable by rsync. Also pins two rsync parity
* rules: the special bits are stored in the record but NOT applied to the real
* file, and a device record's rdev round-trips through the parser. Guarded on
* filesystem xattr support. */
static void test_fake_super_rsync_format() {
const char* path = "test_fake_super_format.txt";
unlink(path);
int fd = open(path, O_WRONLY | O_CREAT | O_TRUNC, 0600);
if (fd < 0)
return;
bool has_xattr = setxattr(path, "user.fastsync.xprobe", "p", 1, 0) == 0;
if (has_xattr)
removexattr(path, "user.fastsync.xprobe");
if (!has_xattr) {
close(fd);
unlink(path);
return; /* skip silently when the filesystem has no xattr support */
}
/* A setuid regular file: the full st_mode (with S_IFMT + special bits) is
recorded, rdev is 0,0, and the owner is uid:gid. */
fake_super_store_fd(fd, 1234, 5678, S_IFREG | 04711, 0, 0);
char value[128];
ssize_t got = fgetxattr(fd, FAKESUPER_XATTR, value, sizeof(value));
EXPECT_EQ_INT((int)got, 20);
EXPECT_TRUE(got == 20 && memcmp(value, "104711 0,0 1234:5678", 20) == 0);
/* The special bits in the record are NOT installed on the real file. */
FileAttrPolicy policy = {true, true, false, false, true};
EXPECT_TRUE(fake_super_restore_fd(fd, policy));
struct stat st;
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)(st.st_mode & 07777), 0711);
EXPECT_EQ_INT((int)(st.st_mode & (S_ISUID | S_ISGID | S_ISVTX)), 0);
/* A device record (char 1,3, uid 111, gid 222) parses without error and
still never real-chowns or installs the device's mode bits verbatim. */
EXPECT_EQ_INT((int)fsetxattr(fd, FAKESUPER_XATTR, "20644 1,3 111:222", 17, 0), 0);
struct stat before;
fstat(fd, &before);
EXPECT_TRUE(fake_super_restore_fd(fd, policy));
fstat(fd, &st);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0644);
EXPECT_EQ_INT((int)st.st_uid, (int)before.st_uid);
EXPECT_EQ_INT((int)st.st_gid, (int)before.st_gid);
close(fd);
unlink(path);
}
/* --fake-super must NEVER perform a real chown: fake_super_restore_fd applies /* --fake-super must NEVER perform a real chown: fake_super_restore_fd applies
* only mode/mtime and leaves the entry's uid/gid exactly as they were, even * only mode/mtime and leaves the entry's uid/gid exactly as they were, even
* when an explicit ownership policy is active and super_mode permits it. This * when an explicit ownership policy is active and super_mode permits it. This
@@ -421,7 +474,7 @@ static void test_fake_super_no_real_chown() {
} }
struct stat before; struct stat before;
EXPECT_EQ_INT(fstat(fd, &before), 0); EXPECT_EQ_INT(fstat(fd, &before), 0);
fake_super_store_fd(fd, 12345, 12346, 0755, 1700000000, 0); fake_super_store_fd(fd, 12345, 12346, S_IFREG | 0755, 0, 0);
Config* c = config_create(); Config* c = config_create();
FileAttrPolicy policy = {true, true, false, false, true}; FileAttrPolicy policy = {true, true, false, false, true};
@@ -600,6 +653,7 @@ void test_xattr() {
test_xattr_receive_drops_acl_without_preserve_acls(); test_xattr_receive_drops_acl_without_preserve_acls();
test_link_copy_fallback_preserves_xattrs(); test_link_copy_fallback_preserves_xattrs();
test_fake_super_restore(); test_fake_super_restore();
test_fake_super_rsync_format();
test_fake_super_no_real_chown(); test_fake_super_no_real_chown();
test_fake_super_storage_resolution(); test_fake_super_storage_resolution();
test_file_save_directory_applies_xattrs(); test_file_save_directory_applies_xattrs();