7 Commits
Author SHA1 Message Date
TapTap 743b00ffdd docs: recount RSYNC_COMPAT summary after Phase-4 wave C (symlink-trust + devices + acl/xattr)
CI / lint (push) Failing after 1m8s
CI / build-and-test (push) Skipped
CI / sanitizers (address) (push) Skipped
CI / sanitizers (undefined) (push) Skipped
CI / fuzz-build (push) Skipped
CI / coverage (push) Skipped
CI / valgrind (push) Skipped
Wave C moved 12 rows: -l/--links now real (was Partial). New Implemented (7):
--munge-links, -k/--copy-dirlinks, -K/--keep-dirlinks, -D, -A/--acls, -X/--xattrs
(links was Partial->Implemented). New Partial (5): --devices, --specials,
--copy-devices, --write-devices, --fake-super. Summary: Implemented 94->101,
Partial 6->10, Not Implemented 41->30 (Total 147).

Final Phase-4 summary: 101/3/10/3/30 = 147; PROTOCOL_VERSION 2.13.0.
2026-09-08 22:42:02 +02:00
TapTap 94b6662018 Merge feat/p4-acl-xattr: -X/-A/--fake-super
# Conflicts:
#	src/shared/config.c
#	src/shared/file.c
#	src/shared/file_types.h
#	tests/integration/test_features.py
#	tests/test_client_cli.c
#	tests/test_config.c
2026-09-08 22:38:15 +02:00
TapTap 5bbdc5b450 Merge feat/p4-devices
# Conflicts:
#	src/client/client_send.c
#	src/server/receiver.c
#	src/shared/chunk.c
#	src/shared/file.c
#	src/shared/file_receive.c
#	src/shared/file_receive.h
#	src/shared/file_types.h
#	src/shared/protocol.h
#	tests/test_chunk.c
2026-09-08 22:33:52 +02:00
TapTap e600b56f10 Merge feat/p4-symlink-trust 2026-09-08 22:27:56 +02:00
TapTap 747946c318 acls/xattrs: -X/--xattrs, -A/--acls, --fake-super
CI / lint (pull_request) Failing after 55s
CI / build-and-test (pull_request) Skipped
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
New src/shared/xattr.{c,h}: capture user.* + POSIX ACL xattrs, transmit a bounded
per-file block, re-apply fd-relative. security.*/trusted.*/other system.* never
transmitted/applied (receiver re-validates). Bounds: name<=255 value<=1MiB count
<=256 total<=4MiB. --fake-super records uid:gid:mode:mtime in reserved
user.fastsync.stat (receiver-only). PROTOCOL_VERSION 2.12.0->2.13.0. Review
fixes: reserved key not forwardable, link/hardlink copy-fallback preserves
xattrs, no const-param mutation, per-file warning dedup.
2026-09-08 22:27:53 +02:00
TapTap 007e8f90f2 devices: --devices/--specials/-D/--copy-devices/--write-devices
CI / lint (pull_request) Failing after 56s
CI / build-and-test (pull_request) Skipped
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
Recreate char/block nodes via mknodat (privilege-gated, EPERM->warn+skip) and
FIFOs via mkfifoat; new STATUS_SPECIAL frame + validated rdev; sockets skipped;
-copy-devices copies st_size; -write-devices O_NOFOLLOW+O_NONBLOCK warn+skip.
preserve_specials/copy_devices/write_devices cross the wire. PROTOCOL_VERSION
2.12.0->2.13.0. Review fixes: -m source-removal keeps recreated specials, FIFO
ENXIO skip, rdev bounds at chunk_deserialize, STATUS_ERROR on receive branch,
scanner_prepare_special dedup.
2026-09-08 22:27:53 +02:00
TapTap 820188c2cc symlink-trust: -k/--copy-dirlinks, -K/--keep-dirlinks, --munge-links (+real -l/--links)
CI / lint (pull_request) Successful in 1m1s
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
CI / build-and-test (pull_request) Successful in 1m19s
Adds symlink-target transmission (File is_symlink+symlink_target, STATUS_SYMLINK
frame, chunk type 2), munge-links sender containment + receiver-side symmetric
target containment, keep-dirlinks confined dir-symlink following (O_NOFOLLOW
realpath-rechecked), and fixes -l to copy symlinks as symlinks. munge_links +
keep_dirlinks cross the wire; copy_dirlinks client-only. PROTOCOL_VERSION
2.12.0->2.13.0. Review fixes: receiver rejects absolute/.. targets, gated unmunge,
-K O_NOFOLLOW+re-fstat, keep_dirlinks set once at config-accept, rel_buf overflow
fails the walk.
2026-09-08 22:27:53 +02:00
31 changed files with 3144 additions and 112 deletions

No files matched your search

+184 -16
View File
@@ -6,11 +6,11 @@ This document maps rsync's full feature set to FastSync's current implementation
| Status | Count | Description | | Status | Count | Description |
|--------|-------|-------------| |--------|-------|-------------|
| ✅ Implemented | 94 | Feature works end-to-end | | ✅ Implemented | 101 | Feature works end-to-end |
| 🔀 Alt Arg | 3 | Functionality exists but under different flag/semantics | | 🔀 Alt Arg | 3 | Functionality exists but under different flag/semantics |
| ⚠️ Partial | 6 | Flag parsed/stored but behavior incomplete | | ⚠️ Partial | 10 | Flag parsed/stored but behavior incomplete |
| 🔄 Compatibility No-op | 3 | Flag is accepted for CLI compatibility but has no effect | | 🔄 Compatibility No-op | 3 | Flag is accepted for CLI compatibility but has no effect |
| ❌ Not Implemented | 41 | Flag not recognized or no behavior | | ❌ Not Implemented | 30 | Flag not recognized or no behavior |
| **Total** | **147** | | | **Total** | **147** | |
--- ---
@@ -244,20 +244,20 @@ why plain `--append` works on the normal atomic path, not only with `--inplace`.
| `-t`, `--times` | Preserve modification times | ✅ Implemented | Part of -M | | `-t`, `--times` | Preserve modification times | ✅ Implemented | Part of -M |
| `-E`, `--executability` | Preserve executability | ✅ Implemented | Preserves executable permission bits (implies metadata preservation) | | `-E`, `--executability` | Preserve executability | ✅ Implemented | Preserves executable permission bits (implies metadata preservation) |
| `--chmod=CHMOD` | Affect file permissions | ✅ Implemented | Supports numeric and symbolic `ugo` `rwx` changes; retains receiver safety masking | | `--chmod=CHMOD` | Affect file permissions | ✅ Implemented | Supports numeric and symbolic `ugo` `rwx` changes; retains receiver safety masking |
| `-A`, `--acls` | Preserve ACLs | ❌ Not Implemented | Removed because it had no effect | | `-A`, `--acls` | Preserve ACLs | ✅ Implemented | Implemented on Linux via the POSIX-ACL xattr representation: the sender captures the `system.posix_acl_access` / `system.posix_acl_default` xattrs into the same bounded whitelisted set as `-X`, transmits them per-file, and the receiver re-applies them fd-relative. Setting an ACL the receiver is not permitted to set (non-root on a file it does not own, unsupported filesystem) is logged and skipped, never fatal. libacl is **not** required. Only the `system.posix_acl_*` namespaces plus `user.*` are ever applied; privileged namespaces are never applied (see the Phase-4 xattr/ACL notes below). Implies metadata transmission |
| `-X`, `--xattrs` | Preserve extended attributes | ❌ Not Implemented | Removed because it had no effect | | `-X`, `--xattrs` | Preserve extended attributes | ✅ Implemented | Preserves unprivileged `user.*` extended attributes (Linux `listxattr`/`getxattr` on capture, `fsetxattr` on the written destination fd). Both capture (sender) and application (receiver) are restricted to the `user.*` namespace and the two POSIX ACL xattrs, so a client can **never** force a `security.*`/`trusted.*`/privileged attribute onto the destination; the receiver independently re-validates every incoming name against this whitelist and rejects anything else. Payloads are bounded (per-name ≤255B, per-value ≤1MiB, per-file count ≤256 total bytes ≤4MiB) on both ends, and an oversized/malformed frame is a clean protocol rejection (no OOM). Applied fd-relative to the exact written file. Implies metadata transmission. Incompatible with `-s` (chunk serialization), rejected up front (see the notes); a `--link-dest`/`-H` hard-link copy fallback re-applies the attributes so they are not dropped when a link is refused |
| `-H`, `--hard-links` | Preserve hard links | ✅ Implemented | Files on the source that share an inode (`st_dev`+`st_ino`, e.g. a `cp -al` tree) are re-created as hard links to one another on the destination, so duplicate links stay deduplicated and only the first member's data is sent (later members are transmitted as payload-less `STATUS_HARDLINK` frames). The receiver links each sibling to the first member's installed file with an atomic link + rename; on `link()` failure it falls back to a byte-identical local copy of the first member, never a partial/corrupt file. Requires the sequential scan for ordering (the first member is always emitted and installed before any sibling is linked). Works single-threaded and under `-m`, `--inplace`, `--delay-updates` (links staged and published by rename) and `--partial`. Crosses the wire (`preserve_hard_links` bool; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0**, peers must match). Incompatible with `-s` (chunk serialization) and `--append`/`--append-verify`, rejected up front with a distinct error. See the Phase-4 hard-links notes below | | `-H`, `--hard-links` | Preserve hard links | ✅ Implemented | Files on the source that share an inode (`st_dev`+`st_ino`, e.g. a `cp -al` tree) are re-created as hard links to one another on the destination, so duplicate links stay deduplicated and only the first member's data is sent (later members are transmitted as payload-less `STATUS_HARDLINK` frames). The receiver links each sibling to the first member's installed file with an atomic link + rename; on `link()` failure it falls back to a byte-identical local copy of the first member, never a partial/corrupt file. Requires the sequential scan for ordering (the first member is always emitted and installed before any sibling is linked). Works single-threaded and under `-m`, `--inplace`, `--delay-updates` (links staged and published by rename) and `--partial`. Crosses the wire (`preserve_hard_links` bool; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0**, peers must match). Incompatible with `-s` (chunk serialization) and `--append`/`--append-verify`, rejected up front with a distinct error. See the Phase-4 hard-links notes below |
| `-D` | Same as --devices --specials | ❌ Not Implemented | Removed because device-file handling is not implemented | | `-D` | Same as --devices --specials | ✅ Implemented | Implies `--devices --specials`. `-D` was unassigned in FastSync (verified: no collision), so it is free to imply both device-node and special-file preservation. See the `--devices`/`--specials` rows and the Phase-4 devices notes below |
| `--devices` | Preserve device files | ❌ Not Implemented | Removed because it had no effect | | `--devices` | Preserve device files | ⚠️ Partial | Recreates char/block device nodes on the destination via `mknod` instead of transferring content. Type + rdev are validated strictly (S_IFMT from the transmitted mode; major/minor range-checked, non-negative), and creation is **privilege-gated**: `mknod` needs `CAP_MKNOD`, so a non-root receiver (CI runs via setpriv as non-root) logs a warning and **skips the device entry safely** — the whole transfer never aborts just because the node could not be made. The node is created fd-relative below the receive root (`mknodat` on the confined secure parent), so it can never be placed outside the authorized root, never follows a symlink, and never replaces an existing directory. Only a char/block mode is honored. Crosses the wire (a new `STATUS_SPECIAL` frame carries the path + metadata mode + rdev; `PROTOCOL_VERSION` bumped **2.12.0 → 2.13.0**). Divergence: per-entry skip (not a hard error) when the receiver lacks `CAP_MKNOD`, documented in the Phase-4 devices notes |
| `--specials` | Preserve special files | ❌ Not Implemented | | | `--specials` | Preserve special files | ⚠️ Partial | Recreates **FIFOs** on the destination via `mkfifo` (unprivileged, so this is a real, assertable behavior under CI). Sockets cannot be recreated by any standard filesystem call and are skipped with an explicit note (best-effort / unsupported, matching the plan). FIFO creation is privileged-gated only in the sense of graceful skip on any permission failure. Node creation is confined below the receive root (`mkfifoat` on the secure fd-relative parent; no `..`, no symlink follow). Crosses the wire like `--devices` (the `STATUS_SPECIAL` frame; `PROTOCOL_VERSION` bumped **2.12.0 → 2.13.0**). See the Phase-4 devices notes |
| `--copy-devices` | Copy device contents as file | ❌ Not Implemented | | | `--copy-devices` | Copy device contents as file | ⚠️ Partial | Copy a device's CONTENT into an ordinary regular file on the destination instead of recreating the node — non-privileged and safe. FastSync scans a device/FIFO as a regular file: its reported size (`st_size`, typically 0 for char devices and FIFOs) is copied, so a FIFO or a non-readable device becomes an empty (or size-bounded) regular file without ever blocking or reading unbounded pseudo-device streams. The run always succeeds and never crashes on such input. **Deliberate, safe divergence from rsync's dd-like unbounded device read.** See the Phase-4 devices notes |
| `--write-devices` | Write to devices as files | ❌ Not Implemented | | | `--write-devices` | Write to devices as files | ⚠️ Partial | Write the received data directly into an **existing** device node on the destination instead of creating a regular file. Restricted and best-effort: the destination must already exist and be a char/block device (opened only under the confined receive root, with `O_NOFOLLOW` + `O_NONBLOCK`); a missing, symlinked, FIFO-with-no-reader (`ENXIO`), non-device destination, or any write failure is **skipped with a warning** rather than allowed, so a run can never clobber the system, never blocks on a special-file target, and never aborts on an unusable target. See the Phase-4 devices notes |
| `-U`, `--atimes` | Preserve access times | ✅ Implemented | Captures the source access time (from the scanner's pre-read stat, so it is not clobbered by reading the file for transfer) and transmits it over the wire; the receiver restores it together with the mtime via `futimens`/`utimensat`. Implies metadata transmission (the times travel inside the `-M` metadata payload), but does not enable ownership application (that stays opt-in via the identity flags). Wire: new `atime` fields on the metadata frame + a `preserve_atimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** | | `-U`, `--atimes` | Preserve access times | ✅ Implemented | Captures the source access time (from the scanner's pre-read stat, so it is not clobbered by reading the file for transfer) and transmits it over the wire; the receiver restores it together with the mtime via `futimens`/`utimensat`. Implies metadata transmission (the times travel inside the `-M` metadata payload), but does not enable ownership application (that stays opt-in via the identity flags). Wire: new `atime` fields on the metadata frame + a `preserve_atimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** |
| `-N`, `--crtimes` | Preserve create times | ⚠️ Partial | Captures the source birth time via `statx(STATX_BTIME)` on Linux and transmits it (recorded as a wire field), but there is **no portable way to set a birth time** (`utimensat` can only set atime/mtime), so the receiver explicitly does NOT apply it: it logs a debug note and continues — never failing the transfer and never pretending it worked. On platforms without `statx` it parses as a documented no-op (flag accepted; nothing is captured). Implies metadata transmission. Wire: new `crtime` fields + a `preserve_crtimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** (see the Phase-4 metadata-time notes) | | `-N`, `--crtimes` | Preserve create times | ⚠️ Partial | Captures the source birth time via `statx(STATX_BTIME)` on Linux and transmits it (recorded as a wire field), but there is **no portable way to set a birth time** (`utimensat` can only set atime/mtime), so the receiver explicitly does NOT apply it: it logs a debug note and continues — never failing the transfer and never pretending it worked. On platforms without `statx` it parses as a documented no-op (flag accepted; nothing is captured). Implies metadata transmission. Wire: new `crtime` fields + a `preserve_crtimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** (see the Phase-4 metadata-time notes) |
| `-O`, `--omit-dir-times` | Omit dirs from --times | 🔄 Compatibility No-op | Accepted and parsed for CLI compatibility, and the config boolean crosses the wire, but it has **no effect**: FastSync never preserves directory mtimes in the first place (directories are created via `mkdir` with no metadata, a documented divergence under `-d`/recursive), so there is nothing for an "omit" to suppress. It never breaks a normal run | | `-O`, `--omit-dir-times` | Omit dirs from --times | 🔄 Compatibility No-op | Accepted and parsed for CLI compatibility, and the config boolean crosses the wire, but it has **no effect**: FastSync never preserves directory mtimes in the first place (directories are created via `mkdir` with no metadata, a documented divergence under `-d`/recursive), so there is nothing for an "omit" to suppress. It never breaks a normal run |
| `-J`, `--omit-link-times` | Omit symlinks from --times | 🔄 Compatibility No-op | Accepted and parsed for CLI compatibility, and the config boolean crosses the wire, but it has **no effect**: FastSync never sets symlink times (`-l`/`--links` still only includes symlinks without transmitting a target; `--copy-links` dereferences), so there is nothing for an "omit" to suppress. It never breaks a normal run | | `-J`, `--omit-link-times` | Omit symlinks from --times | 🔄 Compatibility No-op | Accepted and parsed for CLI compatibility, and the config boolean crosses the wire, but it has **no effect**: FastSync never sets symlink times (`-l`/`--links` copies symlinks as symlinks but the receiver does not apply timestamps/owner to symlink entries), so there is nothing for an "omit" to suppress. It never breaks a normal run |
| `--super` | Receiver attempts super-user activities | ❌ Not Implemented | | | `--super` | Receiver attempts super-user activities | ❌ Not Implemented | |
| `--fake-super` | Store/recover privileged attrs via xattrs | ❌ Not Implemented | | | `--fake-super` | Store/recover privileged attrs via xattrs | ⚠️ Partial | Honest, limited subset. The receiver records the source `uid:gid:mode:mtime_sec:mtime_nsec` into a reserved `user.fastsync.stat` xattr on each written file (best-effort, fd-relative), so a later privileged restore could re-apply them — without attempting the (typically failing as non-root) `chown`. Full rsync fake-super **replay** (parsing that xattr to actually re-apply ownership on a later privileged run) is out of scope and is **divergent** from rsync, which uses its own `user.rsync.%stat%` format; no cross-tool conversion is attempted. Implies metadata transmission so the source uid/gid/mode/mtime are available. Both it and `-X`/`-A` are incompatible with `-s` (chunk serialization), rejected up front |
| `--open-noatime` | Avoid changing access time when opening files | ✅ Implemented | Sender-side policy: the sender opens source files with `O_NOATIME` (Linux) when reading them for transfer, so the open/read does NOT bump the source's on-disk access time. Degrades safely when `O_NOATIME` is unavailable (not defined) or refused (`EPERM`, since it needs `CAP_FOWNER` or file ownership): the code falls back to a normal open, so the data always transfers — only the atime-bump is skipped. It does not itself capture/preserve atime; it only avoids modifying it. **Client-only, never crosses the wire.** Exposed as `file_open_for_read()` and applied to both the buffered data path and the sendfile path | | `--open-noatime` | Avoid changing access time when opening files | ✅ Implemented | Sender-side policy: the sender opens source files with `O_NOATIME` (Linux) when reading them for transfer, so the open/read does NOT bump the source's on-disk access time. Degrades safely when `O_NOATIME` is unavailable (not defined) or refused (`EPERM`, since it needs `CAP_FOWNER` or file ownership): the code falls back to a normal open, so the data always transfers — only the atime-bump is skipped. It does not itself capture/preserve atime; it only avoids modifying it. **Client-only, never crosses the wire.** Exposed as `file_open_for_read()` and applied to both the buffered data path and the sendfile path |
| `--numeric-ids` | Do not map uid/gid by name | ✅ Implemented | Ownership is applied through FastSync's opt-in identity path (see the Phase-4 identity notes below). `--numeric-ids` is a mapping-policy modifier: when applying ownership it uses the transmitted numeric uid/gid directly, skipping the name lookup. Without an ownership-affecting option it is inert (FastSync only applies ownership when the user opts in). It does not need `-M` to be parsed, but ownership is only applied when metadata (hence the source uid/gid) is actually transmitted (see the notes) | | `--numeric-ids` | Do not map uid/gid by name | ✅ Implemented | Ownership is applied through FastSync's opt-in identity path (see the Phase-4 identity notes below). `--numeric-ids` is a mapping-policy modifier: when applying ownership it uses the transmitted numeric uid/gid directly, skipping the name lookup. Without an ownership-affecting option it is inert (FastSync only applies ownership when the user opts in). It does not need `-M` to be parsed, but ownership is only applied when metadata (hence the source uid/gid) is actually transmitted (see the notes) |
| `--usermap=STRING` | Map usernames | ✅ Implemented | Opt-in ownership application. rsync subset implemented: comma-separated `FROM:TO` rules evaluated in order, first match wins; `FROM`/`TO` are group/user names (resolved on the SOURCE machine at parse time), `*` (FROM matches any id / TO = the receiving process's current euid), and an `@N` or bare `N` numeric id. Rules are carried over the wire as resolved numeric id pairs; the receiver applies a matching rule (else falls back to `--chown`, `--numeric-ids`, then a best-effort name lookup) via an fd-relative `fchown`. Malformed/unresolvable specs are rejected with a clear error, never a silent no-op. Implies metadata preservation so the source uid/gid travel. Only effective when the receiver can actually change ownership (root or membership); otherwise it warns and continues | | `--usermap=STRING` | Map usernames | ✅ Implemented | Opt-in ownership application. rsync subset implemented: comma-separated `FROM:TO` rules evaluated in order, first match wins; `FROM`/`TO` are group/user names (resolved on the SOURCE machine at parse time), `*` (FROM matches any id / TO = the receiving process's current euid), and an `@N` or bare `N` numeric id. Rules are carried over the wire as resolved numeric id pairs; the receiver applies a matching rule (else falls back to `--chown`, `--numeric-ids`, then a best-effort name lookup) via an fd-relative `fchown`. Malformed/unresolvable specs are rejected with a clear error, never a silent no-op. Implies metadata preservation so the source uid/gid travel. Only effective when the receiver can actually change ownership (root or membership); otherwise it warns and continues |
@@ -284,6 +284,54 @@ receiver (apply), so they and their metadata fields cross the wire;
(mirroring the existing convention where `ignore_errors` is client-only while (mirroring the existing convention where `ignore_errors` is client-only while
`force_delete` crosses the wire). `force_delete` crosses the wire).
**Phase-4 xattr/ACL notes (`-X/--xattrs`, `-A/--acls`, `--fake-super`):** these
are new in protocol 2.13.0 and add a bounded per-file xattr block to the
per-file metadata frame (count + each `name`/`value`, sent only when xattr
transport is enabled, i.e. with zero overhead on unaffected runs). The config
frame carries `preserve_xattrs`, `preserve_acls` (in the existing file-options
block) and a trailing `fake_super` boolean — all CROSS the wire so the receiver
knows the negotiated behavior; the derived `use_xattrs` flag is recomputed on
the receiver. `PROTOCOL_VERSION` was bumped **2.12.0 → 2.13.0** (peers must
match, exactly as prior phases did).
- **Security model (both `-X` and `-A`):** only `user.*` and the
`system.posix_acl_access` / `system.posix_acl_default` namespaces are ever
captured (sender) or applied (receiver). `security.*` (SELinux, capabilities,
...), `trusted.*`, and all other `system.*` attributes are never transmitted
or applied, so a client can never compel the receiver to set a privileged
xattr. The receiver re-validates each incoming name against this whitelist
even though the sender already filtered, so a malicious/compromised sender's
`security.capability` payload is rejected outright (a clean protocol error),
never applied.
- **Bounds / memory safety:** per-name length ≤ 255 B, per-value ≤ 1 MiB,
per-file count ≤ 256 names, per-file name+value total ≤ 4 MiB. Both the
sender (during capture) and the receiver (during receive) enforce these; an
oversized or malformed frame is rejected, never a large allocation.
- **Confined application:** xattrs are applied with `fsetxattr` on the exact
just-written destination file fd (before the atomic rename), never on a
caller-controlled path; this is the same confinement as mode/time restore.
The `--link-dest` / `-H` hard-link copy fallback (a byte copy when `link()`
is refused) also re-applies the incoming (or, for `-H`, the first member's)
xattrs and the `--fake-super` stat, so attributes are preserved rather than
silently dropped when the link fails.
- **Reserved fake-super key is receiver-only:** the `user.fastsync.stat` key is
excluded from sender capture AND from receiver application, so it can only be
written by the receiver's own `--fake-super` handling. A source file that
already carries such a record is never forwarded on a plain `-X` run, so it
cannot be spoofed to mislead a later privileged restore.
- **`-A` requires no libacl** — ACLs travel as the `system.posix_acl_*` xattrs.
Applying an ACL is owner-privileged: `fsetxattr` failure (e.g. non-root,
unsupported filesystem) is logged (collapsed to one line per file) and never
fatal.
- **`--fake-super`**: see the row above; the reserved key is `user.fastsync.stat`
with the documented `uid:gid:mode:mtime_sec:mtime_nsec` (mode octal) format.
It is honest but partial — there is no replay, and it does not interoperate
with rsync's `user.rsync.%stat%`.
- **Chunk serialization (`-s`) incompatibility:** the per-file xattr block rides
the streaming per-file frame, which `-s` replaces with a fixed buffer format,
so `-X` / `-A` combined with `-s` is rejected up front on both ends (mirroring
the existing `-H` + `-s` rejection) rather than silently dropping attributes.
**atime capture does not clobber the source atime:** the sender records the **atime capture does not clobber the source atime:** the sender records the
access time from the **same pre-read stat the scanner already took** (inside access time from the **same pre-read stat the scanner already took** (inside
`file_metadata_create`), before any file data is read for transfer. So `-U` `file_metadata_create`), before any file data is read for transfer. So `-U`
@@ -392,17 +440,137 @@ front with a distinct error on the client, and re-checked on receive): `-H` with
`-s` chunk serialization (the chunk wire has no per-file hard-link info) and `-H` `-s` chunk serialization (the chunk wire has no per-file hard-link info) and `-H`
with `--append`/`--append-verify` (a payload-less sibling cannot be tail-resumed). with `--append`/`--append-verify` (a payload-less sibling cannot be tail-resumed).
**Phase-4 devices notes:** `--devices`, `--specials`, `-D`, `--copy-devices`,
and `--write-devices` are new. They change the wire: the config frame grows three
booleans — `preserve_specials`, `copy_devices`, `write_devices` — that CROSS the
wire (`preserve_devices` already existed), and a new `STATUS_SPECIAL` frame (used
by `--devices`/`--specials`/`-D`) carries a special/device entry: the destination
path, the metadata frame (whose mode's S_IFMT bits carry the node kind, requiring
the flags to imply metadata transmission), and two int32 `rdev` major/minor
fields. The chunk-serialized wire (`-s`) grows a matching per-file special
marker + rdev so `--devices/--specials` also work under `-s`. `PROTOCOL_VERSION`
was bumped **2.12.0 → 2.13.0** (peers must match, exactly as prior phases did).
**Privilege gating (the crux):** making a device node requires `CAP_MKNOD` (root).
CI runs the integration suite as a NON-ROOT user (via setpriv), so `mknod` fails
with `EPERM`. The receiver treats this as a graceful, logged *skip of the entry*
returned as a success/skip outcome — the whole transfer NEVER aborts just because
the environment cannot create the node. `mkfifo` (FIFOs) is unprivileged, so
`--specials` FIFO creation is a real, assertable behavior under CI; sockets cannot
be recreated by any standard filesystem call and are skipped with an explicit
note. The "device actually created" integration assertions are guarded to run
only as root. User-facing expectation: point `--devices` at devices and a
non-root receiver will faithfully skip them while transferring everything else.
**Confinement & validation:** a special/device node is created with
`mknodat`/`mkfifoat` on the parent directory opened fd-relative below the receive
root (`file_open_secure_parent`: `O_NOFOLLOW`, no `..` components, root-checked),
so a node can never be created outside the authorized destination root and never
through a symlinked parent. The transmitted type is derived ONLY from the
validated S_IFMT bits of the metadata mode (char/block/FIFO honored, socket
skipped, regular/dir rejected as an invalid special), and the transmitted rdev is
validated both on the wire (`file_receive_special`, `chunk_deserialize`) and at
the creation site (`file_special_rdev_valid`): a negative, oversize, or
non-device-carrying rdev is rejected outright (receiver aborts the frame), and a
node is never replaced over an existing directory or unrelated entry (a matching
existing node is left in place). `--write-devices` is the deliberately restricted
danger path: it only ever opens an existing char/block node under the confined
root, and every failure mode (missing, non-device, write error, EPERM) is a
warning + skip, never a system-clobbering write or an abort.
**Documented divergences (honest subset):**
- A device entry the receiver cannot create (missing `CAP_MKNOD`) is *skipped*,
not a transfer failure — rsync under the same conditions would error.
- `--copy-devices` copies the device's *reported size* (typically 0 for char
devices/FIFOs) into a regular file and never reads an unbounded pseudo-device;
this is the safe, non-hanging alternative to rsync's dd-like read.
- `--write-devices` requires the device to already exist at the destination and
never creates it; unsupported/inaccessible targets are skipped, not written.
- Ownership is not applied to recreated nodes (identity `fchown` needs an fd and
would require opening the node); permissions and mtime are applied at
creation / via `utimensat`.
## 9. Symlink Handling ## 9. Symlink Handling
| Flag | Rsync Description | FastSync Status | Notes | | Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------| |------|-------------------|-----------------|-------|
| `-l`, `--links` | Copy symlinks as symlinks | ⚠️ Partial | Scanner includes symlinks; target path not transmitted | | `-l`, `--links` | Copy symlinks as symlinks | ✅ Implemented | A symlink is transmitted as a real symlink: its target string crosses the wire (a new `STATUS_SYMLINK` frame / chunk entry type) and the receiver creates it with `symlinkat` beneath the receive root. This makes the previously-`-l`-included-but-targetless symlink handling complete. See the Phase-4 symlink-trust notes |
| `-L`, `--copy-links` | Transform symlink to referent | ✅ Implemented | `copy_links` config field | | `-L`, `--copy-links` | Transform symlink to referent | ✅ Implemented | `copy_links` config field |
| `--copy-unsafe-links` | Transform unsafe symlinks | ✅ Implemented | `copy_unsafe_links` config field | | `--copy-unsafe-links` | Transform unsafe symlinks | ✅ Implemented | `copy_unsafe_links` config field |
| `--safe-links` | Ignore symlinks outside tree | ✅ Implemented | `safe_links` config field | | `--safe-links` | Ignore symlinks outside tree | ✅ Implemented | `safe_links` config field |
| `--munge-links` | Munge symlinks for safety | ❌ Not Implemented | | | `--munge-links` | Munge symlinks for safety | ✅ Implemented | Sender rewrites each transmitted symlink target with a `#SYMLINK/` marker; a target that could escape the receive root (absolute or containing `..`) is never transmitted (contained/skipped); the receiver strips the marker to restore the real target. See the Phase-4 symlink-trust notes |
| `-k`, `--copy-dirlinks` | Transform symlink to dir | ❌ Not Implemented | | | `-k`, `--copy-dirlinks` | Transform symlink to dir | ✅ Implemented | A symlink whose referent is a directory is dereferenced and recursed as a real directory; a symlink to a regular file stays a symlink. Sender-side only. See the Phase-4 symlink-trust notes |
| `-K`, `--keep-dirlinks` | Treat symlinked dir as dir | ❌ Not Implemented | | | `-K`, `--keep-dirlinks` | Treat symlinked dir as dir | ✅ Implemented | On the receiver, an existing destination symlink-to-a-directory is used as that directory (followed) instead of being replaced; it is followed only when it resolves to a directory that stays beneath the receive root. See the Phase-4 symlink-trust notes |
**Phase-4 symlink-trust notes:** `-l/--links`, `-k/--copy-dirlinks`,
`-K/--keep-dirlinks`, and `--munge-links` form the "symlink trust boundaries"
row. Making all three new flags have an observable, security-sane effect
required transmitting symlink targets, so FastSync's `-l/--links` is now real:
a symlink-type entry carries its target on the wire (a new `STATUS_SYMLINK`
frame for the per-file path, and a new entry type `2` in the `-s` chunk
serializer) and the receiver creates it with `symlinkat` under an `O_NOFOLLOW`
parent walk, never following the target. Wire changes: `STATUS_SYMLINK`,
the chunk entry type `2`, a per-entry symlink-target string, and two new config
booleans that CROSS the wire — `munge_links` and `keep_dirlinks`; `PROTOCOL_VERSION`
was bumped **2.12.0 → 2.13.0** (peers must match, exactly as prior phases did).
**Per-flag semantics and divergences.**
- **`-l/--links`** copies a symlink as a symlink: the scanner `readlink`s the
target, the sender transmits it, and the receiver `symlinkat`s it. FastSync
`-l` never preserved symlink targets before (the flag was documented partial
and, in fact, tried to read the referent as file data); it now does, matching
rsync. Divergences: because the receiver enforces the symlink containment
predicate unconditionally, a plain `-l` sync **refuses to round-trip a
legitimate absolute symlink target** (it is dropped, never created pointing
outside the root — see the `--munge-links` note for the symmetric trust
boundary); a relative in-root target is copied as-is. FastSync also does not
set timestamps/owner on symlinks (no symlink-mode metadata application),
matching its existing no-op `--omit-link-times`.
- **`-k/--copy-dirlinks`** (sender): a symlink whose referent is a directory is
dereferenced and recursed into as a real directory; a symlink to a regular
file (or any non-directory) is kept as a symlink. This is rsync's `-k`. When
`-L/--copy-links` or `--safe-links`/`--copy-unsafe-links` are active, their
(dereference) semantics take precedence, so `-k` is subsumed exactly as in
rsync.
- **`-K/--keep-dirlinks`** (receiver, crosses the wire): when a directory is to
be created (on-demand parent creation for a child write) and the destination
path is already an existing symlink that resolves to a directory *within* the
receive root, that symlinked directory is used (followed) instead of being
replaced by a real directory; new entries are written beneath it. The follow
is confined: it only happens where `realpath` of the symlink resolves to a
still-within-root real directory, so a malicious link pointing outside the
root is never followed. Scope: `-K` acts on the write path (parent/`mkdir`
creation); the delete walker still never follows symlinks (a documented
divergence for `--delete` over an existing symlinked dir). Without `-K` the
destination symlink is not followed (the O_NOFOLLOW walk fails the write),
which is the safe default.
- **`--munge-links`** (sender security rewrite; crosses the wire so the receiver
unmunges): every transmitted symlink target is prefixed with the marker
`#SYMLINK/`; the receiver strips the marker (only when the negotiated
`munge_links` policy is on — a plain `-l` run never strips the prefix, so a
source symlink that genuinely begins with `#SYMLINK/` round-trips verbatim)
and restores the exact real target. The trust boundary is **symmetric and
enforced receiver-side**, independent of the sender: `file_symlink_at_secure`
refuses any target that `file_symlink_target_contained` rejects (absolute
`/...` or relative with a `..` component), and `file_save_to_disk_full`
contains such an entry (skipped) rather than materializing it. A deliberate confinement trade-off: because the receiver
enforces containment unconditionally, a plain `-l` (no `--munge-links`) sync
*refuses to round-trip a legitimate absolute symlink target* — such target is
dropped, never created pointing outside the root. This is a stricter subset of
rsync: rsync stores munged targets on the RECEIVING side and depends on both
ends running `--munge-links`; FastSync additionally enforces the containment
predicate at the receiver regardless of what the sender transmitted. When no
symlink is being transmitted (`-l`/`-k`/`-a` off) `--munge-links` has nothing
to rewrite and is inert. -*K/`--keep-dirlinks` policy is installed per
connection at config-accept (stable for the whole transfer, never racy under
`-m`), and only ever follows an in-root symlink-to-directory.*
**Compatibility (byte-identical when all three are absent):** `-k`, `-K` and
`--munge-links` are opt-in. Without them the scanner's link handling, the wire
frames, and the receiver's writes are unchanged for every other option set, so a
run that previously worked continues to behave identically. `-l/--links` itself
now transmits targets (the prior behavior was broken/partial); its status moved
`⚠️ Partial → ✅ Implemented`.
## 10. Sparse & Device ## 10. Sparse & Device
+40
View File
@@ -476,6 +476,9 @@ static const OptionEntry OPTION_TABLE[] = {
{"--copy-links", NULL, OPT_FLAG, offsetof(Config, copy_links)}, {"--copy-links", NULL, OPT_FLAG, offsetof(Config, copy_links)},
{"--safe-links", NULL, OPT_FLAG, offsetof(Config, safe_links)}, {"--safe-links", NULL, OPT_FLAG, offsetof(Config, safe_links)},
{"--copy-unsafe-links", NULL, OPT_FLAG, offsetof(Config, copy_unsafe_links)}, {"--copy-unsafe-links", NULL, OPT_FLAG, offsetof(Config, copy_unsafe_links)},
{"--copy-dirlinks", "-k", OPT_FLAG, offsetof(Config, copy_dirlinks)},
{"--keep-dirlinks", "-K", OPT_FLAG, offsetof(Config, keep_dirlinks)},
{"--munge-links", NULL, OPT_FLAG, offsetof(Config, munge_links)},
{"--hard-links", "-H", OPT_FLAG, offsetof(Config, preserve_hard_links)}, {"--hard-links", "-H", OPT_FLAG, offsetof(Config, preserve_hard_links)},
{"--sparse", "-S", OPT_FLAG, offsetof(Config, preserve_sparse)}, {"--sparse", "-S", OPT_FLAG, offsetof(Config, preserve_sparse)},
{"--inplace", NULL, OPT_FLAG, offsetof(Config, inplace)}, {"--inplace", NULL, OPT_FLAG, offsetof(Config, inplace)},
@@ -537,9 +540,17 @@ static const OptionEntry OPTION_TABLE[] = {
{"--numeric-ids", NULL, OPT_FLAG, offsetof(Config, numeric_ids)}, {"--numeric-ids", NULL, OPT_FLAG, offsetof(Config, numeric_ids)},
{"--atimes", "-U", OPT_FLAG, offsetof(Config, preserve_atimes)}, {"--atimes", "-U", OPT_FLAG, offsetof(Config, preserve_atimes)},
{"--crtimes", "-N", OPT_FLAG, offsetof(Config, preserve_crtimes)}, {"--crtimes", "-N", OPT_FLAG, offsetof(Config, preserve_crtimes)},
/* -D is handled separately (it implies both --devices and --specials). */
{"--devices", NULL, OPT_FLAG, offsetof(Config, preserve_devices)},
{"--specials", NULL, OPT_FLAG, offsetof(Config, preserve_specials)},
{"--copy-devices", NULL, OPT_FLAG, offsetof(Config, copy_devices)},
{"--write-devices", NULL, OPT_FLAG, offsetof(Config, write_devices)},
{"--omit-dir-times", "-O", OPT_FLAG, offsetof(Config, omit_dir_times)}, {"--omit-dir-times", "-O", OPT_FLAG, offsetof(Config, omit_dir_times)},
{"--omit-link-times", "-J", OPT_FLAG, offsetof(Config, omit_link_times)}, {"--omit-link-times", "-J", OPT_FLAG, offsetof(Config, omit_link_times)},
{"--open-noatime", NULL, OPT_FLAG, offsetof(Config, open_noatime)}, {"--open-noatime", NULL, OPT_FLAG, offsetof(Config, open_noatime)},
{"--xattrs", "-X", OPT_FLAG, offsetof(Config, preserve_xattrs)},
{"--acls", "-A", OPT_FLAG, offsetof(Config, preserve_acls)},
{"--fake-super", NULL, OPT_FLAG, offsetof(Config, fake_super)},
}; };
/* Only boolean options with no required argument are safe to negate. */ /* Only boolean options with no required argument are safe to negate. */
@@ -574,6 +585,9 @@ static const NegatableOption NEGATABLE_OPTIONS[] = {
{"preserve", "M", offsetof(Config, use_metadata)}, {"preserve", "M", offsetof(Config, use_metadata)},
{"sendfile", "f", offsetof(Config, use_sendfile)}, {"sendfile", "f", offsetof(Config, use_sendfile)},
{"chunk-serialization", "s", offsetof(Config, use_chunk_serialization)}, {"chunk-serialization", "s", offsetof(Config, use_chunk_serialization)},
{"xattrs", "X", offsetof(Config, preserve_xattrs)},
{"acls", "A", offsetof(Config, preserve_acls)},
{"fake-super", NULL, offsetof(Config, fake_super)},
}; };
static bool opt_is(const char* arg, const char* name, const char* alias) { static bool opt_is(const char* arg, const char* name, const char* alias) {
@@ -810,6 +824,13 @@ int parse_args(Config* config, int argc, char* argv[], int* positional_args,
if (entry->offset == offsetof(Config, preserve_atimes) || if (entry->offset == offsetof(Config, preserve_atimes) ||
entry->offset == offsetof(Config, preserve_crtimes)) entry->offset == offsetof(Config, preserve_crtimes))
config->use_metadata = true; config->use_metadata = true;
if (entry->offset == offsetof(Config, preserve_xattrs) ||
entry->offset == offsetof(Config, preserve_acls)) {
config->use_metadata = true;
config->use_xattrs = config->preserve_acls || config->preserve_xattrs;
}
if (entry->offset == offsetof(Config, fake_super))
config->use_metadata = true;
continue; continue;
} }
@@ -831,6 +852,12 @@ int parse_args(Config* config, int argc, char* argv[], int* positional_args,
} else if (opt_is(argv[i], "-V", "--version")) { } else if (opt_is(argv[i], "-V", "--version")) {
printf("fastsync version %s\n", PROTOCOL_VERSION); printf("fastsync version %s\n", PROTOCOL_VERSION);
return 1; return 1;
} else if (opt_is(argv[i], "-D", NULL)) {
/* rsync -D == --devices --specials. -D is otherwise unassigned in
FastSync (verified: no collision), so it is free to imply both. */
config->preserve_devices = true;
config->preserve_specials = true;
log_info_message(LOG_INFO_MISC, "Enabled preservation of device and special files (-D)");
} else if (opt_is(argv[i], "-a", "--archive")) { } else if (opt_is(argv[i], "-a", "--archive")) {
config->use_compression = config->use_compression =
!config->compress_choice || strcmp(config->compress_choice, "zstd") == 0; !config->compress_choice || strcmp(config->compress_choice, "zstd") == 0;
@@ -1200,6 +1227,15 @@ int parse_args(Config* config, int argc, char* argv[], int* positional_args,
config->files_from_set = set; config->files_from_set = set;
} }
/* Device/special preservation recreates a node from its metadata mode (whose
S_IFMT bits carry the node kind), so --devices/--specials/-D imply metadata
transmission. --copy-devices/--write-devices treat the entry as data but a
mtime/mode-preserving transfer still benefits from metadata, so all four
imply it (FastSync's broad -M bundle; ownership stays opt-in). */
if (config->preserve_devices || config->preserve_specials || config->copy_devices ||
config->write_devices)
config->use_metadata = true;
/* The "unchanged" decision for --compare-dest/--copy-dest/--link-dest must /* The "unchanged" decision for --compare-dest/--copy-dest/--link-dest must
* be made on the receiver against the basis directories, which requires the * be made on the receiver against the basis directories, which requires the
* per-file STATUS_CHECK handshake: basis-dir options therefore imply * per-file STATUS_CHECK handshake: basis-dir options therefore imply
@@ -1242,6 +1278,10 @@ int parse_args(Config* config, int argc, char* argv[], int* positional_args,
log_message(LOG_LEVEL_INFO, "Enabling metadata preservation for incremental/delta transfer"); log_message(LOG_LEVEL_INFO, "Enabling metadata preservation for incremental/delta transfer");
config->use_metadata = true; config->use_metadata = true;
} }
/* Recompute the derived xattr flag from the FINAL preserve flags (after any
* --no-xattrs/--no-acls negation) so the sender's wire gate always matches
* the flags the receiver will recompute from the received config. */
config->use_xattrs = config->preserve_acls || config->preserve_xattrs;
return 0; return 0;
} }
+49 -5
View File
@@ -20,6 +20,7 @@
#include "transport_ssh.h" #include "transport_ssh.h"
#include "transport_tls.h" #include "transport_tls.h"
#include "utils.h" #include "utils.h"
#include "xattr.h"
#include <fcntl.h> #include <fcntl.h>
#include <limits.h> #include <limits.h>
#include <stdio.h> #include <stdio.h>
@@ -89,6 +90,8 @@ static bool prepare_scanner(const Config* config, int num_threads, PreparedScann
options->use_metadata = config->use_metadata; options->use_metadata = config->use_metadata;
options->preserve_atimes = config->preserve_atimes; options->preserve_atimes = config->preserve_atimes;
options->preserve_crtimes = config->preserve_crtimes; options->preserve_crtimes = config->preserve_crtimes;
options->preserve_xattrs = config->preserve_xattrs;
options->preserve_acls = config->preserve_acls;
options->chunk_size = config->chunk_size; options->chunk_size = config->chunk_size;
options->exclude_patterns = config->exclude_patterns; options->exclude_patterns = config->exclude_patterns;
options->exclude_count = config->exclude_count; options->exclude_count = config->exclude_count;
@@ -102,8 +105,13 @@ static bool prepare_scanner(const Config* config, int num_threads, PreparedScann
options->copy_links = config->copy_links; options->copy_links = config->copy_links;
options->safe_links = config->safe_links; options->safe_links = config->safe_links;
options->copy_unsafe_links = config->copy_unsafe_links; options->copy_unsafe_links = config->copy_unsafe_links;
options->copy_dirlinks = config->copy_dirlinks;
options->munge_links = config->munge_links;
options->checksum = config->checksum; options->checksum = config->checksum;
options->one_file_system = config->one_file_system; options->one_file_system = config->one_file_system;
options->preserve_devices = config->preserve_devices;
options->preserve_specials = config->preserve_specials;
options->copy_devices = config->copy_devices;
options->file_list = (const FileListSet*)config->files_from_set; options->file_list = (const FileListSet*)config->files_from_set;
options->base_filters = out->base_filters; options->base_filters = out->base_filters;
options->per_dir_filters = config->per_dir_filter; options->per_dir_filters = config->per_dir_filter;
@@ -958,6 +966,9 @@ static int send_delta(Client* client, File* file, DeltaSignature* sig, Config* c
if (ok && config->use_metadata) if (ok && config->use_metadata)
ok = metadata_send(client->file_descriptor, file->metadata); ok = metadata_send(client->file_descriptor, file->metadata);
if (ok && config->use_xattrs)
ok = xattr_send(client->file_descriptor, file->xattrs);
data_destroy(to_send); data_destroy(to_send);
return ok ? 0 : -1; return ok ? 0 : -1;
} }
@@ -999,7 +1010,7 @@ static int send_append(const Client* client, File* file, Config* config,
transfer (byte-identical, never a corrupt prefix+tail blend). */ transfer (byte-identical, never a corrupt prefix+tail blend). */
int rc = file_send_single_calls_with_skip(file, fd, config->use_metadata, compression_level, int rc = file_send_single_calls_with_skip(file, fd, config->use_metadata, compression_level,
false, config->skip_compress_suffixes, skip_count, false, config->skip_compress_suffixes, skip_count,
config->compression_threads) config->compression_threads, config->use_xattrs)
? 1 ? 1
: -1; : -1;
return rc; return rc;
@@ -1016,6 +1027,9 @@ static int send_append(const Client* client, File* file, Config* config,
if (config->use_metadata && !metadata_send(fd, file->metadata)) { if (config->use_metadata && !metadata_send(fd, file->metadata)) {
return -1; return -1;
} }
if (config->use_xattrs && !xattr_send(fd, file->xattrs)) {
return -1;
}
bool ok; bool ok;
if (compress) { if (compress) {
/* Compression needs an owned copy of the tail to compress. */ /* Compression needs an owned copy of the tail to compress. */
@@ -1053,7 +1067,7 @@ static bool send_file_direct(File* file, int fd, bool use_metadata, int compress
int skip_count = config->skip_compress_set ? config->skip_compress_count : -1; int skip_count = config->skip_compress_set ? config->skip_compress_count : -1;
return file_send_single_calls_with_skip(file, fd, use_metadata, compression_level, true, return file_send_single_calls_with_skip(file, fd, use_metadata, compression_level, true,
config->skip_compress_suffixes, skip_count, config->skip_compress_suffixes, skip_count,
config->compression_threads); config->compression_threads, config->use_xattrs);
} }
/* Transmit one explicit directory entry (--dirs): a STATUS_MKDIR frame whose /* Transmit one explicit directory entry (--dirs): a STATUS_MKDIR frame whose
@@ -1067,6 +1081,20 @@ static bool send_directory_entry(Client* client, File* file) {
return send_str(client->file_descriptor, file_wire_path(file)); return send_str(client->file_descriptor, file_wire_path(file));
} }
/* Transmit one symlink entry: a STATUS_SYMLINK frame carrying the destination
* path, the (sender-munged, if --munge-links) target string, and metadata when
* negotiated. The receiver unmunges the target and creates the symlink beneath
* its root. Symlinks never need an incremental check or data payload. */
static bool send_symlink_entry(const Client* client, File* file, const Config* config) {
if (!file || !file_wire_path(file) || !file->symlink_target)
return false;
int fd = client->file_descriptor;
if (!send_status(fd, STATUS_SYMLINK) || !send_str(fd, file_wire_path(file)) ||
!send_str(fd, file->symlink_target))
return false;
return !config->use_metadata || metadata_send(fd, file->metadata);
}
// Send a single file directly via sendfile (non-incremental path). // Send a single file directly via sendfile (non-incremental path).
static bool send_file_direct_sendfile(File* file, int fd, bool use_metadata, const Config* config) { static bool send_file_direct_sendfile(File* file, int fd, bool use_metadata, const Config* config) {
if (!send_status(fd, STATUS_NEXT)) if (!send_status(fd, STATUS_NEXT))
@@ -1074,7 +1102,7 @@ static bool send_file_direct_sendfile(File* file, int fd, bool use_metadata, con
int skip_count = config->skip_compress_set ? config->skip_compress_count : -1; int skip_count = config->skip_compress_set ? config->skip_compress_count : -1;
return file_send_sendfile_with_skip(file, fd, use_metadata, 0, true, return file_send_sendfile_with_skip(file, fd, use_metadata, 0, true,
config->skip_compress_suffixes, skip_count, config->skip_compress_suffixes, skip_count,
config->compression_threads); config->compression_threads, config->use_xattrs);
} }
// Process one file in a chunk: either via incremental check or direct send. // Process one file in a chunk: either via incremental check or direct send.
@@ -1132,7 +1160,7 @@ static int send_single_file(Client* client, File* file, Config* config, bool use
int skip_count = config->skip_compress_set ? config->skip_compress_count : -1; int skip_count = config->skip_compress_set ? config->skip_compress_count : -1;
if (!file_send_sendfile_with_skip(file, client->file_descriptor, config->use_metadata, 0, false, if (!file_send_sendfile_with_skip(file, client->file_descriptor, config->use_metadata, 0, false,
config->skip_compress_suffixes, skip_count, config->skip_compress_suffixes, skip_count,
config->compression_threads)) config->compression_threads, config->use_xattrs))
return -1; return -1;
return 0; return 0;
} }
@@ -1181,7 +1209,8 @@ static int send_single_file(Client* client, File* file, Config* config, bool use
int skip_count = config->skip_compress_set ? config->skip_compress_count : -1; int skip_count = config->skip_compress_set ? config->skip_compress_count : -1;
if (!file_send_single_calls_with_skip(file, client->file_descriptor, config->use_metadata, if (!file_send_single_calls_with_skip(file, client->file_descriptor, config->use_metadata,
compression_level, false, config->skip_compress_suffixes, compression_level, false, config->skip_compress_suffixes,
skip_count, config->compression_threads)) skip_count, config->compression_threads,
config->use_xattrs))
return -1; return -1;
return 0; return 0;
} }
@@ -1248,6 +1277,21 @@ static int send_chunk_with_removal(Client* client, Chunk* chunk, Config* config,
change_emit_file_sent(config, f); change_emit_file_sent(config, f);
continue; continue;
} }
/* Symlink entry (-l / -k keep-as-symlink): only the target rides the wire. */
if (f->is_symlink) {
if (!send_symlink_entry(client, f, config))
return -1;
change_emit_file_sent(config, f);
continue;
}
/* --devices/--specials: a device/special node is recreated on the receiver,
not transferred as content. Send the dedicated STATUS_SPECIAL frame. */
if (f->is_special) {
if (!file_send_special(f, client->file_descriptor, config->use_metadata))
return -1;
change_emit_file_sent(config, f);
continue;
}
bool stream = f->data->data == NULL && f->data->size > 0; bool stream = f->data->data == NULL && f->data->size > 0;
bool use_sendfile = bool use_sendfile =
(config->use_sendfile && !config->use_compression) || (stream && !config->use_compression); (config->use_sendfile && !config->use_compression) || (stream && !config->use_compression);
+8
View File
@@ -79,6 +79,14 @@ bool validate_config(const Config* config) {
"--hard-links/-H cannot be combined with -s (chunk serialization)"); "--hard-links/-H cannot be combined with -s (chunk serialization)");
return false; return false;
} }
/* -X/-A ride the per-file metadata frame; the buffer-based chunk-serialization
wire format does not carry the xattr block, so the pair is rejected up front
(mirroring -H + -s) rather than silently dropping attributes. */
if ((config->preserve_xattrs || config->preserve_acls) && config->use_chunk_serialization) {
log_message(LOG_LEVEL_ERROR,
"--xattrs/-X and --acls/-A cannot be combined with -s (chunk serialization)");
return false;
}
if (config->preserve_hard_links && (config->append || config->append_verify)) { if (config->preserve_hard_links && (config->append || config->append_verify)) {
log_message(LOG_LEVEL_ERROR, log_message(LOG_LEVEL_ERROR,
"--hard-links/-H cannot be combined with --append/--append-verify"); "--hard-links/-H cannot be combined with --append/--append-verify");
+147 -21
View File
@@ -10,10 +10,13 @@
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
#include <sys/stat.h> #include <sys/stat.h>
#include <sys/sysmacros.h>
#include <threads.h> #include <threads.h>
#include <unistd.h> #include <unistd.h>
#include <limits.h> #include <limits.h>
#include "xattr.h"
typedef struct { typedef struct {
char* path; char* path;
int depth; int depth;
@@ -112,6 +115,12 @@ typedef struct {
char* path; char* path;
struct stat stats; struct stat stats;
bool is_directory; bool is_directory;
/* True when the entry should be carried through as a SYMLINK (is_symlink)
rather than a dereferenced file/directory. When true, `link_target` holds
the owned target string to transmit (sender-munged under --munge-links);
ownership transfers to the File built from this entry. */
bool is_symlink;
char* link_target;
/* True when the entry was pruned by a user selection rule (--filter/-C/per-dir /* True when the entry was pruned by a user selection rule (--filter/-C/per-dir
rules, the --exclude/--include layer, or --max-size/--min-size) rather than rules, the --exclude/--include layer, or --max-size/--min-size) rather than
skipped for another reason (unreadable, symlink policy, not applicable). */ skipped for another reason (unreadable, symlink policy, not applicable). */
@@ -165,6 +174,14 @@ static bool entry_passes_selection(const FileListSet* file_list, const FilterRul
return true; return true;
} }
/* Best-effort capture of the file's whitelisted xattrs (-X/-A). A failure to
* read xattrs is non-fatal: the file is transferred without them. */
static void scanner_capture_xattrs(const DirectoryScanner* scanner, File* file) {
if (!scanner || !file || !(scanner->preserve_xattrs || scanner->preserve_acls))
return;
file->xattrs = xattr_capture_path(file->path);
}
/* Apply --hard-links (-H) detection to one regular File. On a sibling (a /* Apply --hard-links (-H) detection to one regular File. On a sibling (a
* later member of an already-seen source inode) the File keeps the group id * later member of an already-seen source inode) the File keeps the group id
* and the first member's wire path but carries NO data payload (size 0); the * and the first member's wire path but carries NO data payload (size 0); the
@@ -193,6 +210,34 @@ static void scanner_assign_hardlink(DirectoryScanner* scanner, HardLinkTable* ta
} }
} }
/* Phase 4 special/devices: detect a device (char/block), FIFO or socket entry
and, when the matching --devices/--specials flag asks it be preserved,
convert the File into a node to recreate (is_special, empty payload) with its
device rdev captured from the source stat. When the entry is not preserved
(or --copy-devices instead copies its content as an ordinary regular file)
the File is left as a normal data file. Returns true when converted. */
static bool scanner_prepare_special(bool preserve_devices, bool preserve_specials, File* file,
const struct stat* stats) {
if (!file || !stats)
return false;
bool is_device = S_ISCHR(stats->st_mode) || S_ISBLK(stats->st_mode);
bool is_fifo = S_ISFIFO(stats->st_mode);
bool is_socket = S_ISSOCK(stats->st_mode);
if (!is_device && !is_fifo && !is_socket)
return false;
bool preserve = is_device ? preserve_devices : preserve_specials;
if (!preserve)
return false;
file->is_special = true;
file->data->size = 0;
file->data->data = NULL;
if (is_device) {
file->rdev_major = (int32_t)major(stats->st_rdev);
file->rdev_minor = (int32_t)minor(stats->st_rdev);
}
return true;
}
/* Append `rel` to the caller's exclusion sink, taking `mtx` when shared across /* Append `rel` to the caller's exclusion sink, taking `mtx` when shared across
parallel worker threads. Returns false on allocation failure (list left parallel worker threads. Returns false on allocation failure (list left
unchanged). */ unchanged). */
@@ -263,6 +308,8 @@ static int scanner_inspect_entry(const ScannerOptions* options, const char* sour
const char* containing_dir, const char* name, const char* containing_dir, const char* name,
ScannerEntry* entry) { ScannerEntry* entry) {
entry->excluded = false; entry->excluded = false;
entry->is_symlink = false;
entry->link_target = NULL;
entry->path = path_cat(containing_dir, name); entry->path = path_cat(containing_dir, name);
if (!entry->path) if (!entry->path)
return -1; return -1;
@@ -273,38 +320,81 @@ static int scanner_inspect_entry(const ScannerOptions* options, const char* sour
return 0; return 0;
} }
bool is_symlink = S_ISLNK(link_stats.st_mode); bool is_symlink = S_ISLNK(link_stats.st_mode);
if (is_symlink && !options->follow_symlinks && !options->copy_links && !options->safe_links && if (!is_symlink)
!options->copy_unsafe_links) goto regular;
/* Symlink: choose between dereferencing (---copy-links / --safe-links /
--copy-unsafe-links, plus -k for symlinks-to-directories) and carrying the
link through as a symlink (-l, and -k for symlinks-to-files). No link
option means the symlink is skipped entirely (pre-existing behavior). */
const bool any_link_option = options->follow_symlinks || options->copy_links ||
options->safe_links || options->copy_unsafe_links ||
options->copy_dirlinks;
if (!any_link_option)
goto skip; goto skip;
if (is_symlink && options->safe_links) { char link_target[4096];
char link_target[4096]; ssize_t length = readlink(entry->path, link_target, sizeof(link_target) - 1);
ssize_t length = readlink(entry->path, link_target, sizeof(link_target) - 1); if (length < 0)
if (length < 0) goto skip;
goto skip; link_target[length] = '\0';
link_target[length] = '\0';
if (options->safe_links) {
if (link_target[0] == '/' || !safe_relative_link(source_root, containing_dir, link_target)) if (link_target[0] == '/' || !safe_relative_link(source_root, containing_dir, link_target))
goto skip; goto skip;
} }
if (options->copy_unsafe_links && !options->copy_links) {
if (is_symlink && options->copy_unsafe_links && !options->copy_links) {
char link_target[4096];
ssize_t length = readlink(entry->path, link_target, sizeof(link_target) - 1);
if (length < 0)
goto skip;
link_target[length] = '\0';
if (link_target[0] != '/') if (link_target[0] != '/')
goto skip; goto skip;
} }
if (is_symlink && options->follow_symlinks && !options->copy_links) bool emit_symlink = false;
entry->stats = link_stats; if (options->copy_links) {
else if (stat(entry->path, &entry->stats) != 0) emit_symlink = false; /* --copy-links dereferences every referent */
goto skip; } else if (options->safe_links || options->copy_unsafe_links) {
emit_symlink = false; /* preserve pre-existing dereference behavior */
} else if (options->copy_dirlinks) {
struct stat ref;
if (stat(entry->path, &ref) == 0 && S_ISDIR(ref.st_mode))
emit_symlink = false; /* -k: symlink to a directory recurses as a dir */
else
emit_symlink = true; /* -k: symlink to a file stays a symlink */
} else if (options->follow_symlinks) {
emit_symlink = true; /* -l: copy symlink as symlink */
}
if (!emit_symlink) {
if (stat(entry->path, &entry->stats) != 0)
goto skip;
entry->is_directory = S_ISDIR(entry->stats.st_mode);
if (entry->is_directory)
return 1;
goto apply_filters;
}
/* Carry the link as a symlink. --munge-links containment: a target that
could escape the receive root (absolute or containing "..") is never
transmitted -- the entry is merely skipped ("contained"). */
if (link_target[0] == '\0' ||
(options->munge_links && !file_symlink_target_contained(link_target)))
goto skip;
entry->is_symlink = true;
entry->stats = link_stats;
entry->is_directory = false;
entry->link_target =
options->munge_links ? file_symlink_munge(link_target) : str_dup(link_target);
if (!entry->link_target)
goto skip;
goto apply_filters;
regular:
if (stat(entry->path, &entry->stats) != 0)
goto skip;
entry->is_directory = S_ISDIR(entry->stats.st_mode); entry->is_directory = S_ISDIR(entry->stats.st_mode);
if (entry->is_directory) if (entry->is_directory)
return 1; return 1;
apply_filters:
for (int i = 0; i < options->exclude_count; i++) for (int i = 0; i < options->exclude_count; i++)
if (glob_match(options->exclude_patterns[i], name)) { if (glob_match(options->exclude_patterns[i], name)) {
entry->excluded = true; entry->excluded = true;
@@ -330,6 +420,8 @@ static int scanner_inspect_entry(const ScannerOptions* options, const char* sour
skip: skip:
free(entry->path); free(entry->path);
entry->path = NULL; entry->path = NULL;
free(entry->link_target);
entry->link_target = NULL;
return 0; return 0;
} }
@@ -350,6 +442,8 @@ DirectoryScanner* directory_scanner_create_with_options(const char* root_directo
scanner->use_metadata = options->use_metadata; scanner->use_metadata = options->use_metadata;
scanner->preserve_atimes = options->preserve_atimes; scanner->preserve_atimes = options->preserve_atimes;
scanner->preserve_crtimes = options->preserve_crtimes; scanner->preserve_crtimes = options->preserve_crtimes;
scanner->preserve_xattrs = options->preserve_xattrs;
scanner->preserve_acls = options->preserve_acls;
scanner->chunk_size = options->chunk_size > 0 ? options->chunk_size : DESIRED_CHUNK_SIZE; scanner->chunk_size = options->chunk_size > 0 ? options->chunk_size : DESIRED_CHUNK_SIZE;
scanner->exclude_patterns = options->exclude_patterns; scanner->exclude_patterns = options->exclude_patterns;
scanner->exclude_count = options->exclude_count; scanner->exclude_count = options->exclude_count;
@@ -363,8 +457,13 @@ DirectoryScanner* directory_scanner_create_with_options(const char* root_directo
scanner->copy_links = options->copy_links; scanner->copy_links = options->copy_links;
scanner->safe_links = options->safe_links; scanner->safe_links = options->safe_links;
scanner->copy_unsafe_links = options->copy_unsafe_links; scanner->copy_unsafe_links = options->copy_unsafe_links;
scanner->copy_dirlinks = options->copy_dirlinks;
scanner->munge_links = options->munge_links;
scanner->checksum = options->checksum; scanner->checksum = options->checksum;
scanner->one_file_system = options->one_file_system; scanner->one_file_system = options->one_file_system;
scanner->preserve_devices = options->preserve_devices;
scanner->preserve_specials = options->preserve_specials;
scanner->copy_devices = options->copy_devices;
scanner->failed = false; scanner->failed = false;
scanner->root_path = str_dup(root_directory); scanner->root_path = str_dup(root_directory);
if (!scanner->root_path) { if (!scanner->root_path) {
@@ -822,6 +921,8 @@ Chunk* directory_scanner_next(DirectoryScanner* scanner) {
.copy_links = scanner->copy_links, .copy_links = scanner->copy_links,
.safe_links = scanner->safe_links, .safe_links = scanner->safe_links,
.copy_unsafe_links = scanner->copy_unsafe_links, .copy_unsafe_links = scanner->copy_unsafe_links,
.copy_dirlinks = scanner->copy_dirlinks,
.munge_links = scanner->munge_links,
.checksum = scanner->checksum, .checksum = scanner->checksum,
.one_file_system = scanner->one_file_system, .one_file_system = scanner->one_file_system,
.file_list = scanner->file_list, .file_list = scanner->file_list,
@@ -917,14 +1018,25 @@ Chunk* directory_scanner_next(DirectoryScanner* scanner) {
free(cur_path); free(cur_path);
if (file == NULL) { if (file == NULL) {
free(rel_copy); free(rel_copy);
free(inspected.link_target);
inspected.link_target = NULL;
scanner->failed = true; scanner->failed = true;
continue; continue;
} }
file->data->size = stats.st_size; if (inspected.is_symlink) {
file->is_symlink = true;
file->symlink_target = inspected.link_target;
inspected.link_target = NULL;
} else {
file->data->size = stats.st_size;
}
if (scanner->relative_mode) { if (scanner->relative_mode) {
file->send_path = rel_copy; file->send_path = rel_copy;
rel_copy = NULL; rel_copy = NULL;
} }
/* --devices/--specials: a device/FIFO/socket entry marked for preservation
becomes a node to recreate (is_special, no data, rdev captured). */
scanner_prepare_special(scanner->preserve_devices, scanner->preserve_specials, file, &stats);
if (scanner->hardlinks && S_ISREG(stats.st_mode)) if (scanner->hardlinks && S_ISREG(stats.st_mode))
scanner_assign_hardlink(scanner, scanner->hardlinks, file, &stats); scanner_assign_hardlink(scanner, scanner->hardlinks, file, &stats);
if (scanner->use_metadata) if (scanner->use_metadata)
@@ -936,6 +1048,8 @@ Chunk* directory_scanner_next(DirectoryScanner* scanner) {
scanner->failed = true; scanner->failed = true;
break; break;
} }
if (!(file->link_group != 0 && !file->link_first))
scanner_capture_xattrs(scanner, file);
if (!array_list_add(chunk_data, file)) { if (!array_list_add(chunk_data, file)) {
free(rel_copy); free(rel_copy);
file_destroy(file); file_destroy(file);
@@ -1235,14 +1349,23 @@ static void scan_root_entry(const ScannerOptions* options, const FilterNode* roo
free(cur_path); free(cur_path);
if (!file) { if (!file) {
free(rel); free(rel);
free(inspected.link_target);
inspected.link_target = NULL;
ps->failed = true; ps->failed = true;
return; return;
} }
file->data->size = st.st_size; if (inspected.is_symlink) {
file->is_symlink = true;
file->symlink_target = inspected.link_target;
inspected.link_target = NULL;
} else {
file->data->size = st.st_size;
}
if (use_rel) { if (use_rel) {
file->send_path = rel; file->send_path = rel;
rel = NULL; rel = NULL;
} }
scanner_prepare_special(options->preserve_devices, options->preserve_specials, file, &st);
if (options->hardlinks && S_ISREG(st.st_mode)) { if (options->hardlinks && S_ISREG(st.st_mode)) {
int gid; int gid;
bool is_first; bool is_first;
@@ -1270,6 +1393,9 @@ static void scan_root_entry(const ScannerOptions* options, const FilterNode* roo
ps->failed = true; ps->failed = true;
return; return;
} }
if ((options->preserve_xattrs || options->preserve_acls) &&
!(file->link_group != 0 && !file->link_first))
file->xattrs = xattr_capture_path(file->path);
if (!array_list_add(root_files, file)) { if (!array_list_add(root_files, file)) {
free(rel); free(rel);
file_destroy(file); file_destroy(file);
+26
View File
@@ -19,6 +19,10 @@ typedef struct {
* capture the source access / birth time into each entry's FileMetadata. */ * capture the source access / birth time into each entry's FileMetadata. */
bool preserve_atimes; bool preserve_atimes;
bool preserve_crtimes; bool preserve_crtimes;
/* Phase 4 xattrs: when preserve_xattrs || preserve_acls is set the scanner
* captures each regular file's whitelisted xattr set onto the File. */
bool preserve_xattrs;
bool preserve_acls;
unsigned long long chunk_size; unsigned long long chunk_size;
char** exclude_patterns; char** exclude_patterns;
int exclude_count; int exclude_count;
@@ -32,8 +36,22 @@ typedef struct {
bool copy_links; bool copy_links;
bool safe_links; bool safe_links;
bool copy_unsafe_links; bool copy_unsafe_links;
/* Phase 4 symlink-trust sender options: -k/--copy-dirlinks (dereference a
* symlink to a directory as a directory, keeping symlinks-to-files as
* symlinks) and --munge-links (rewrite each transmitted symlink target with a
* marker; escaping targets are never transmitted). Both are client/sender
* side only and never serialized to the wire (keep_dirlinks is the
* receiver-side counterpart). */
bool copy_dirlinks;
bool munge_links;
bool checksum; bool checksum;
bool one_file_system; bool one_file_system;
/* Phase 4 special/devices: whether device nodes (--devices) and special files
* (--specials) are preserved via recreation, and whether --copy-devices
* copies a device's content as an ordinary regular file. */
bool preserve_devices;
bool preserve_specials;
bool copy_devices;
/* Phase 2 (files-from / filter layer). All pointers are shared read-only /* Phase 2 (files-from / filter layer). All pointers are shared read-only
* across scanner instances and worker threads; ownership stays with the * across scanner instances and worker threads; ownership stays with the
* caller (client_send). */ * caller (client_send). */
@@ -87,6 +105,8 @@ typedef struct {
bool use_metadata; bool use_metadata;
bool preserve_atimes; bool preserve_atimes;
bool preserve_crtimes; bool preserve_crtimes;
bool preserve_xattrs;
bool preserve_acls;
unsigned long long chunk_size; unsigned long long chunk_size;
char** exclude_patterns; char** exclude_patterns;
int exclude_count; int exclude_count;
@@ -100,10 +120,16 @@ typedef struct {
bool copy_links; bool copy_links;
bool safe_links; bool safe_links;
bool copy_unsafe_links; bool copy_unsafe_links;
bool copy_dirlinks;
bool munge_links;
bool checksum; bool checksum;
bool one_file_system; bool one_file_system;
dev_t root_dev; dev_t root_dev;
bool failed; bool failed;
/* Phase 4 special/devices (see ScannerOptions). */
bool preserve_devices;
bool preserve_specials;
bool copy_devices;
/* Phase 2 (files-from / filter layer). */ /* Phase 2 (files-from / filter layer). */
char* root_path; /* transfer root (fs path) for rel computation */ char* root_path; /* transfer root (fs path) for rel computation */
char* current_rel; /* rel path of the open directory ("" == root) */ char* current_rel; /* rel path of the open directory ("" == root) */
+22
View File
@@ -128,6 +128,16 @@ void print_usage(void) {
printf(" none suppresses info even with --verbose\n"); printf(" none suppresses info even with --verbose\n");
printf(" -M, --preserve Preserve file metadata\n"); printf(" -M, --preserve Preserve file metadata\n");
printf(" -E, --executability Preserve executable permission bits\n"); printf(" -E, --executability Preserve executable permission bits\n");
printf(" -X, --xattrs Preserve user extended attributes (user.* only;\n");
printf(" privileged security.*/trusted.* namespaces are\n");
printf(" never captured or applied)\n");
printf(" -A, --acls Preserve POSIX ACLs (the system.posix_acl_* xattrs;\n");
printf(" setting an ACL the receiver is not permitted to\n");
printf(" set is warned and skipped, never fatal)\n");
printf(" --fake-super Store the source uid/gid/mode/mtime in a reserved\n");
printf(" user.fastsync.stat xattr on each written file instead\n");
printf(" of applying ownership (for a later privileged restore);\n");
printf(" partial: full rsync fake-super replay is out of scope\n");
printf(" --chmod <changes> Modify transferred permissions (rsync syntax)\n"); printf(" --chmod <changes> Modify transferred permissions (rsync syntax)\n");
printf(" --numeric-ids Do not map uid/gid by name: use the source numeric\n"); printf(" --numeric-ids Do not map uid/gid by name: use the source numeric\n");
printf(" ids directly when applying ownership\n"); printf(" ids directly when applying ownership\n");
@@ -180,8 +190,20 @@ void print_usage(void) {
printf(" --copy-links Transform symlinks into referent files\n"); printf(" --copy-links Transform symlinks into referent files\n");
printf(" --safe-links Skip symlinks that point outside transfer tree\n"); printf(" --safe-links Skip symlinks that point outside transfer tree\n");
printf(" --copy-unsafe-links Only transform unsafe symlinks into referent files\n"); printf(" --copy-unsafe-links Only transform unsafe symlinks into referent files\n");
printf(" -k, --copy-dirlinks Transform symlinks to directories into real dirs\n");
printf(" -K, --keep-dirlinks Keep an existing symlink-to-dir as that dir\n");
printf(" --munge-links Munge symlink targets on the wire (sender)\n");
printf(" -H, --hard-links Preserve hard-link relationships across the transfer\n"); printf(" -H, --hard-links Preserve hard-link relationships across the transfer\n");
printf(" -S, --sparse Handle sparse files efficiently\n"); printf(" -S, --sparse Handle sparse files efficiently\n");
printf(
" -D Preserve device and special files (implies --devices --specials)\n");
printf(
" --devices Recreate device nodes on the destination (privileged; skipped when\n");
printf(" the receiver lacks CAP_MKNOD)\n");
printf(" --specials Recreate special files (FIFOs) on the destination (sockets "
"skipped)\n");
printf(" --copy-devices Copy a source device's content as a regular file instead\n");
printf(" --write-devices Write received data into an existing destination device node\n");
printf(" --inplace Update files in-place (no temp+rename)\n"); printf(" --inplace Update files in-place (no temp+rename)\n");
printf( printf(
" --preallocate Allocate destination file space up front (fail-fast on full disk)\n"); " --preallocate Allocate destination file space up front (fail-fast on full disk)\n");
+12 -2
View File
@@ -154,7 +154,8 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver
DeleteManifest* deferred_manifest = NULL; DeleteManifest* deferred_manifest = NULL;
while (status == STATUS_NEXT || status == STATUS_CHUNK || status == STATUS_CHECK || while (status == STATUS_NEXT || status == STATUS_CHUNK || status == STATUS_CHECK ||
status == STATUS_KEEPALIVE || status == STATUS_ABORT || status == STATUS_CHECK_BATCH || status == STATUS_KEEPALIVE || status == STATUS_ABORT || status == STATUS_CHECK_BATCH ||
status == STATUS_MKDIR || status == STATUS_MANIFEST || status == STATUS_HARDLINK) { status == STATUS_MKDIR || status == STATUS_MANIFEST || status == STATUS_HARDLINK ||
status == STATUS_SYMLINK || status == STATUS_SPECIAL) {
if (status == STATUS_KEEPALIVE) { if (status == STATUS_KEEPALIVE) {
if (!send_status(file_descriptor, STATUS_KEEPALIVE)) if (!send_status(file_descriptor, STATUS_KEEPALIVE))
goto fail; goto fail;
@@ -185,6 +186,14 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver
File* file = file_receive_hardlink(file_descriptor); File* file = file_receive_hardlink(file_descriptor);
if (!file || !sink->store_file(file, sink->context)) if (!file || !sink->store_file(file, sink->context))
goto receive_error; goto receive_error;
} else if (status == STATUS_SYMLINK) {
File* sym = file_receive_symlink(file_descriptor, config);
if (!sym || !sink->store_file(sym, sink->context))
goto receive_error;
} else if (status == STATUS_SPECIAL) {
File* file = file_receive_special(file_descriptor);
if (!file || !sink->store_file(file, sink->context))
goto receive_error;
} else if (status == STATUS_MANIFEST) { } else if (status == STATUS_MANIFEST) {
DeleteManifest* manifest = receive_manifest_entries(file_descriptor); DeleteManifest* manifest = receive_manifest_entries(file_descriptor);
if (!manifest) if (!manifest)
@@ -309,7 +318,8 @@ static bool receiver_save_file(File* file, void* context_pointer) {
result = file_save_to_disk_full(context->config->receive_root_directory, file, context->config); result = file_save_to_disk_full(context->config->receive_root_directory, file, context->config);
} }
if (result != FILE_SAVE_ERROR && context->config->remove_source_files && !file->is_dir && if (result != FILE_SAVE_ERROR && context->config->remove_source_files && !file->is_dir &&
!file->skip && !receiver_outcomes_append(&context->outcomes, (unsigned char)result)) { !file->is_special && !file->skip &&
!receiver_outcomes_append(&context->outcomes, (unsigned char)result)) {
file_destroy(file); file_destroy(file);
return false; return false;
} }
+5
View File
@@ -215,6 +215,11 @@ void handler(int file_descriptor) {
apply path. Each connection is its own forked process, so this apply path. Each connection is its own forked process, so this
per-process snapshot never races another connection. */ per-process snapshot never races another connection. */
identity_set_active(config); identity_set_active(config);
/* Persist the negotiated --keep-dirlinks policy once, here at config-accept,
before any multithreaded receiver/writer threads are spawned, so the
fd-walk reads a stable value during the whole transfer (and never bleeds
across the per-connection forked processes). */
file_set_keep_dirlinks(config->keep_dirlinks);
if (config->use_multithreading) { if (config->use_multithreading) {
Queue* q = queue_create(100, file_destroy); Queue* q = queue_create(100, file_destroy);
if (q == NULL) { if (q == NULL) {
+108 -4
View File
@@ -74,16 +74,35 @@ static unsigned long long per_file_serialize_size(File* file, bool use_metadata)
if (metadata_size > ULLONG_MAX - size) if (metadata_size > ULLONG_MAX - size)
return 0; return 0;
size += metadata_size; size += metadata_size;
/* Entry type marker: 0 = regular file, 1 = explicit directory entry. */ /* Entry type marker: 0 = regular file, 1 = explicit directory entry,
2 = symlink entry (carries its target string), 3 = special/device node
(recreated by the receiver). */
if (sizeof(int) > ULLONG_MAX - size) if (sizeof(int) > ULLONG_MAX - size)
return 0; return 0;
size += sizeof(int); size += sizeof(int);
/* A special node also carries its rdev major/minor. */
if (file->is_special) {
if (2 * sizeof(int32_t) > ULLONG_MAX - size)
return 0;
size += 2 * sizeof(int32_t);
}
if (sizeof(size_t) > ULLONG_MAX - size) if (sizeof(size_t) > ULLONG_MAX - size)
return 0; return 0;
size += sizeof(size_t); size += sizeof(size_t);
if ((unsigned long long)file->data->size > ULLONG_MAX - size) if ((unsigned long long)file->data->size > ULLONG_MAX - size)
return 0; return 0;
return size + file->data->size; size += file->data->size;
/* Symlink entries append the target string (length-prefixed). */
if (file->is_symlink) {
size_t target_len = file->symlink_target ? strlen(file->symlink_target) : 0;
if (sizeof(size_t) > ULLONG_MAX - size)
return 0;
size += sizeof(size_t);
if ((unsigned long long)target_len > ULLONG_MAX - size)
return 0;
size += target_len;
}
return size;
} }
Data* chunk_serialize(Chunk* chunk, bool use_metadata) { Data* chunk_serialize(Chunk* chunk, bool use_metadata) {
@@ -116,10 +135,19 @@ Data* chunk_serialize(Chunk* chunk, bool use_metadata) {
memcpy(data_pointer, wire_path, path_len); memcpy(data_pointer, wire_path, path_len);
data_pointer += path_len; data_pointer += path_len;
int entry_type = file->is_dir ? 1 : 0; int entry_type = file->is_dir ? 1 : (file->is_symlink ? 2 : (file->is_special ? 3 : 0));
memcpy(data_pointer, &entry_type, sizeof(int)); memcpy(data_pointer, &entry_type, sizeof(int));
data_pointer += sizeof(int); data_pointer += sizeof(int);
if (file->is_special) {
int32_t special_major = file->rdev_major;
int32_t special_minor = file->rdev_minor;
memcpy(data_pointer, &special_major, sizeof(special_major));
data_pointer += sizeof(special_major);
memcpy(data_pointer, &special_minor, sizeof(special_minor));
data_pointer += sizeof(special_minor);
}
if (use_metadata) if (use_metadata)
metadata_to_buf(&data_pointer, file->metadata); metadata_to_buf(&data_pointer, file->metadata);
@@ -129,6 +157,15 @@ Data* chunk_serialize(Chunk* chunk, bool use_metadata) {
if (file_data_size > 0) if (file_data_size > 0)
memcpy(data_pointer, file->data->data, file_data_size); memcpy(data_pointer, file->data->data, file_data_size);
data_pointer += file_data_size; data_pointer += file_data_size;
if (file->is_symlink) {
size_t target_len = file->symlink_target ? strlen(file->symlink_target) : 0;
memcpy(data_pointer, &target_len, sizeof(size_t));
data_pointer += sizeof(size_t);
if (target_len > 0)
memcpy(data_pointer, file->symlink_target, target_len);
data_pointer += target_len;
}
} }
return data; return data;
} }
@@ -206,16 +243,46 @@ Chunk* chunk_deserialize(Data* data, bool use_metadata) {
} }
int entry_type; int entry_type;
memcpy(&entry_type, data_pointer, sizeof(int)); memcpy(&entry_type, data_pointer, sizeof(int));
if (entry_type != 0 && entry_type != 1) { if (entry_type != 0 && entry_type != 1 && entry_type != 2 && entry_type != 3) {
log_message(LOG_LEVEL_ERROR, "Invalid chunk format: bad entry type"); log_message(LOG_LEVEL_ERROR, "Invalid chunk format: bad entry type");
file_destroy(file); file_destroy(file);
array_list_delete(files); array_list_delete(files);
return NULL; return NULL;
} }
file->is_dir = entry_type == 1; file->is_dir = entry_type == 1;
file->is_symlink = entry_type == 2;
file->is_special = entry_type == 3;
data_pointer += sizeof(int); data_pointer += sizeof(int);
remaining_size -= sizeof(int); remaining_size -= sizeof(int);
if (file->is_special) {
if (remaining_size < 2 * (int32_t)sizeof(int32_t)) {
log_message(LOG_LEVEL_ERROR, "Invalid chunk format: not enough data for special rdev");
file_destroy(file);
array_list_delete(files);
return NULL;
}
int32_t special_major, special_minor;
memcpy(&special_major, data_pointer, sizeof(special_major));
data_pointer += sizeof(special_major);
memcpy(&special_minor, data_pointer, sizeof(special_minor));
data_pointer += sizeof(special_minor);
remaining_size -= 2 * sizeof(int32_t);
/* Reject an out-of-range/negative rdev here as a malformed chunk (the
same 0xffff / 0x00ffffff bounds file_special_rdev_valid uses), so a
bogus large-but-positive rdev is refused cleanly instead of being
deferred to the creation site where it would abort after the frame. */
if (special_major < 0 || special_minor < 0 || special_major > 0xffff ||
special_minor > 0x00ffffff) {
log_message(LOG_LEVEL_ERROR, "Invalid chunk format: out-of-range special rdev");
file_destroy(file);
array_list_delete(files);
return NULL;
}
file->rdev_major = special_major;
file->rdev_minor = special_minor;
}
if (use_metadata) { if (use_metadata) {
if (remaining_size < sizeof(int)) { if (remaining_size < sizeof(int)) {
log_message(LOG_LEVEL_ERROR, "Invalid chunk format: not enough data for metadata"); log_message(LOG_LEVEL_ERROR, "Invalid chunk format: not enough data for metadata");
@@ -293,6 +360,43 @@ Chunk* chunk_deserialize(Data* data, bool use_metadata) {
data_pointer += file_data_size; data_pointer += file_data_size;
remaining_size -= file_data_size; remaining_size -= file_data_size;
if (file->is_symlink) {
if (remaining_size < sizeof(size_t)) {
log_message(LOG_LEVEL_ERROR, "Invalid chunk format: not enough data for symlink target");
file_destroy(file);
array_list_delete(files);
return NULL;
}
size_t target_len;
memcpy(&target_len, data_pointer, sizeof(size_t));
data_pointer += sizeof(size_t);
remaining_size -= sizeof(size_t);
if (target_len == 0 || remaining_size < target_len) {
log_message(LOG_LEVEL_ERROR, "Invalid chunk format: bad symlink target");
file_destroy(file);
array_list_delete(files);
return NULL;
}
char* target = protocol_alloc(target_len + 1);
if (!target) {
log_perror("Could not allocate memory for symlink target");
file_destroy(file);
array_list_delete(files);
return NULL;
}
memcpy(target, data_pointer, target_len);
target[target_len] = '\0';
if (memchr(target, '\0', target_len) != NULL) {
free(target);
file_destroy(file);
array_list_delete(files);
return NULL;
}
file->symlink_target = target;
data_pointer += target_len;
remaining_size -= target_len;
}
if (!array_list_add(files, file)) { if (!array_list_add(files, file)) {
file_destroy(file); file_destroy(file);
array_list_delete(files); array_list_delete(files);
+64 -14
View File
@@ -71,11 +71,17 @@ static void config_set_defaults(Config* config) {
config->copy_links = false; config->copy_links = false;
config->safe_links = false; config->safe_links = false;
config->copy_unsafe_links = false; config->copy_unsafe_links = false;
config->copy_dirlinks = false;
config->munge_links = false;
config->keep_dirlinks = false;
config->preserve_hard_links = false; config->preserve_hard_links = false;
config->preserve_acls = false; config->preserve_acls = false;
config->preserve_xattrs = false; config->preserve_xattrs = false;
config->preserve_devices = false; config->preserve_devices = false;
config->preserve_sparse = false; config->preserve_sparse = false;
config->preserve_specials = false;
config->copy_devices = false;
config->write_devices = false;
config->itemize_changes = false; config->itemize_changes = false;
config->out_format = NULL; config->out_format = NULL;
config->log_file_format = NULL; config->log_file_format = NULL;
@@ -153,6 +159,8 @@ static void config_set_defaults(Config* config) {
config->omit_dir_times = false; config->omit_dir_times = false;
config->omit_link_times = false; config->omit_link_times = false;
config->open_noatime = false; config->open_noatime = false;
config->use_xattrs = false;
config->fake_super = false;
} }
static bool valid_wire_bool(int value) { static bool valid_wire_bool(int value) {
@@ -180,16 +188,18 @@ static bool validate_received_config(const Config* config) {
valid_wire_bool(config->safe_links) && valid_wire_bool(config->copy_unsafe_links) && valid_wire_bool(config->safe_links) && valid_wire_bool(config->copy_unsafe_links) &&
valid_wire_bool(config->preserve_hard_links) && valid_wire_bool(config->preserve_acls) && valid_wire_bool(config->preserve_hard_links) && valid_wire_bool(config->preserve_acls) &&
valid_wire_bool(config->preserve_xattrs) && valid_wire_bool(config->preserve_devices) && valid_wire_bool(config->preserve_xattrs) && valid_wire_bool(config->preserve_devices) &&
valid_wire_bool(config->preserve_sparse) && valid_wire_bool(config->ignore_existing) && valid_wire_bool(config->preserve_sparse) && valid_wire_bool(config->preserve_specials) &&
valid_wire_bool(config->existing) && valid_wire_bool(config->update) && valid_wire_bool(config->copy_devices) && valid_wire_bool(config->write_devices) &&
valid_wire_bool(config->inplace) && valid_wire_bool(config->append) && valid_wire_bool(config->ignore_existing) && valid_wire_bool(config->existing) &&
valid_wire_bool(config->use_fsync) && valid_wire_bool(config->append_verify) && valid_wire_bool(config->update) && valid_wire_bool(config->inplace) &&
valid_wire_bool(config->delete_excluded) && valid_wire_bool(config->force_delete) && valid_wire_bool(config->append) && valid_wire_bool(config->use_fsync) &&
valid_wire_bool(config->delete_missing_args) && valid_wire_bool(config->delete_after) && valid_wire_bool(config->append_verify) && valid_wire_bool(config->delete_excluded) &&
valid_wire_bool(config->preallocate) && valid_wire_bool(config->delete_delay) && valid_wire_bool(config->force_delete) && valid_wire_bool(config->delete_missing_args) &&
valid_wire_bool(config->delete_during) && valid_wire_bool(config->relative) && valid_wire_bool(config->delete_after) && valid_wire_bool(config->preallocate) &&
valid_wire_bool(config->prune_empty_dirs) && valid_wire_bool(config->delay_updates) && valid_wire_bool(config->delete_delay) && valid_wire_bool(config->delete_during) &&
valid_wire_bool(config->mkpath) && !(config->delay_updates && config->inplace) && valid_wire_bool(config->relative) && valid_wire_bool(config->prune_empty_dirs) &&
valid_wire_bool(config->delay_updates) && valid_wire_bool(config->mkpath) &&
!(config->delay_updates && config->inplace) &&
!(config->delay_updates && delay_updates_staging_name_conflict(config->backup_dir)) && !(config->delay_updates && delay_updates_staging_name_conflict(config->backup_dir)) &&
valid_wire_bool(config->partial) && valid_wire_bool(config->delete_before) && valid_wire_bool(config->partial) && valid_wire_bool(config->delete_before) &&
valid_wire_bool(config->checksum) && valid_wire_bool(config->eight_bit_output) && valid_wire_bool(config->checksum) && valid_wire_bool(config->eight_bit_output) &&
@@ -202,8 +212,12 @@ static bool validate_received_config(const Config* config) {
!((config->append || config->append_verify) && config->use_chunk_serialization) && !((config->append || config->append_verify) && config->use_chunk_serialization) &&
!(config->preserve_hard_links && config->use_chunk_serialization) && !(config->preserve_hard_links && config->use_chunk_serialization) &&
!(config->preserve_hard_links && (config->append || config->append_verify)) && !(config->preserve_hard_links && (config->append || config->append_verify)) &&
/* The xattr block rides the per-file streaming frame, which -s drops. */
!((config->preserve_xattrs || config->preserve_acls) && config->use_chunk_serialization) &&
valid_wire_bool(config->preserve_atimes) && valid_wire_bool(config->preserve_crtimes) && valid_wire_bool(config->preserve_atimes) && valid_wire_bool(config->preserve_crtimes) &&
valid_wire_bool(config->omit_dir_times) && valid_wire_bool(config->omit_link_times) && valid_wire_bool(config->omit_dir_times) && valid_wire_bool(config->omit_link_times) &&
valid_wire_bool(config->munge_links) && valid_wire_bool(config->keep_dirlinks) &&
valid_wire_bool(config->fake_super) &&
(!config->use_compression || (!config->use_compression ||
(config->compression_level >= 1 && config->compression_level <= 22)) && (config->compression_level >= 1 && config->compression_level <= 22)) &&
config->chunk_size > 0 && config->chunk_size <= MAX_CHUNK_SIZE && config->chunk_size > 0 && config->chunk_size <= MAX_CHUNK_SIZE &&
@@ -444,12 +458,16 @@ static bool send_delta_fields(int fd, const Config* c) {
} }
static bool send_file_options(int fd, const Config* c) { static bool send_file_options(int fd, const Config* c) {
/* Device/special preservation flags cross the wire so the receiver knows a
* special/device entry must be recreated. Trailing fields; protocol 2.13.0. */
return send_int(fd, c->backup) && send_str(fd, c->backup_dir ? c->backup_dir : "") && return send_int(fd, c->backup) && send_str(fd, c->backup_dir ? c->backup_dir : "") &&
send_int(fd, c->remove_source_files) && send_int(fd, c->follow_symlinks) && send_int(fd, c->remove_source_files) && send_int(fd, c->follow_symlinks) &&
send_int(fd, c->copy_links) && send_int(fd, c->safe_links) && send_int(fd, c->copy_links) && send_int(fd, c->safe_links) &&
send_int(fd, c->copy_unsafe_links) && send_int(fd, c->preserve_hard_links) && send_int(fd, c->copy_unsafe_links) && send_int(fd, c->preserve_hard_links) &&
send_int(fd, c->preserve_acls) && send_int(fd, c->preserve_xattrs) && send_int(fd, c->preserve_acls) && send_int(fd, c->preserve_xattrs) &&
send_int(fd, c->preserve_devices) && send_int(fd, c->preserve_sparse); send_int(fd, c->preserve_devices) && send_int(fd, c->preserve_sparse) &&
send_int(fd, c->preserve_specials) && send_int(fd, c->copy_devices) &&
send_int(fd, c->write_devices);
} }
static bool send_selection_options(int fd, const Config* c) { static bool send_selection_options(int fd, const Config* c) {
@@ -569,7 +587,8 @@ static bool receive_file_options(int fd, Config* c) {
return false; return false;
bool* flags[] = {&c->follow_symlinks, &c->copy_links, &c->safe_links, bool* flags[] = {&c->follow_symlinks, &c->copy_links, &c->safe_links,
&c->copy_unsafe_links, &c->preserve_hard_links, &c->preserve_acls, &c->copy_unsafe_links, &c->preserve_hard_links, &c->preserve_acls,
&c->preserve_xattrs, &c->preserve_devices, &c->preserve_sparse}; &c->preserve_xattrs, &c->preserve_devices, &c->preserve_sparse,
&c->preserve_specials, &c->copy_devices, &c->write_devices};
for (size_t i = 0; i < sizeof(flags) / sizeof(flags[0]); i++) { for (size_t i = 0; i < sizeof(flags) / sizeof(flags[0]); i++) {
if (!receive_wire_bool(fd, flags[i])) if (!receive_wire_bool(fd, flags[i]))
return false; return false;
@@ -771,6 +790,33 @@ static bool receive_metadata_times_options(int fd, Config* c) {
receive_wire_bool(fd, &c->omit_link_times); receive_wire_bool(fd, &c->omit_link_times);
} }
/* Phase 4 symlink-trust: --munge-links and -K/--keep-dirlinks. Both CROSS the
* wire (the receiver unmunges symlink targets and, with -K, follows an in-root
* destination symlink-to-directory). -k/--copy-dirlinks is sender-only and is
* never serialized. Trailing fields; protocol 2.13.0. */
static bool send_symlink_trust_options(int fd, const Config* c) {
return send_int(fd, c->munge_links) && send_int(fd, c->keep_dirlinks);
}
static bool receive_symlink_trust_options(int fd, Config* c) {
return receive_wire_bool(fd, &c->munge_links) && receive_wire_bool(fd, &c->keep_dirlinks);
}
/* -X/--xattrs, -A/--acls, --fake-super (Phase-4). The receiver learns
* preserve_xattrs/preserve_acls from the earlier file-options block and
* recomputes the derived use_xattrs there; only --fake-super (receiver-side
* behavior) needs an extra wire bit. Trailing field; protocol 2.13.0. */
static bool send_phase4_xattr_options(int fd, const Config* c) {
return send_int(fd, c->fake_super);
}
static bool receive_phase4_xattr_options(int fd, Config* c) {
if (!receive_wire_bool(fd, &c->fake_super))
return false;
c->use_xattrs = c->preserve_acls || c->preserve_xattrs;
return true;
}
bool config_send(int file_descriptor, const Config* config) { bool config_send(int file_descriptor, const Config* config) {
protocol_session_set_max_alloc(NULL, config->max_alloc); protocol_session_set_max_alloc(NULL, config->max_alloc);
if (!send_core_fields(file_descriptor, config) || !send_delta_fields(file_descriptor, config) || if (!send_core_fields(file_descriptor, config) || !send_delta_fields(file_descriptor, config) ||
@@ -780,7 +826,9 @@ bool config_send(int file_descriptor, const Config* config) {
!send_basis_options(file_descriptor, config) || !send_fuzzy_option(file_descriptor, config) || !send_basis_options(file_descriptor, config) || !send_fuzzy_option(file_descriptor, config) ||
!send_checksum_options(file_descriptor, config) || !send_checksum_options(file_descriptor, config) ||
!send_identity_options(file_descriptor, config) || !send_identity_options(file_descriptor, config) ||
!send_metadata_times_options(file_descriptor, config)) !send_metadata_times_options(file_descriptor, config) ||
!send_symlink_trust_options(file_descriptor, config) ||
!send_phase4_xattr_options(file_descriptor, config))
return false; return false;
Status status; Status status;
if (!receive_status(file_descriptor, &status)) if (!receive_status(file_descriptor, &status))
@@ -817,7 +865,9 @@ Config* config_receive(int file_descriptor) {
!receive_fuzzy_option(file_descriptor, config) || !receive_fuzzy_option(file_descriptor, config) ||
!receive_checksum_options(file_descriptor, config) || !receive_checksum_options(file_descriptor, config) ||
!receive_identity_options(file_descriptor, config) || !receive_identity_options(file_descriptor, config) ||
!receive_metadata_times_options(file_descriptor, config)) !receive_metadata_times_options(file_descriptor, config) ||
!receive_symlink_trust_options(file_descriptor, config) ||
!receive_phase4_xattr_options(file_descriptor, config))
goto error; goto error;
if (config->compress_choice[0] != '\0' && strcmp(config->compress_choice, "zstd") != 0 && if (config->compress_choice[0] != '\0' && strcmp(config->compress_choice, "zstd") != 0 &&
strcmp(config->compress_choice, "none") != 0) { strcmp(config->compress_choice, "none") != 0) {
+41 -1
View File
@@ -108,6 +108,15 @@ typedef struct Config {
bool copy_links; bool copy_links;
bool safe_links; bool safe_links;
bool copy_unsafe_links; bool copy_unsafe_links;
/* Phase 4 symlink-trust. -k/--copy-dirlinks and --munge-links are
* CLIENT/sender-side only (they decide how the SENDER scans and rewrites
* symlinks; the receiver never reads them), so they never cross the wire.
* -K/--keep-dirlinks is a RECEIVER-side policy (follow an in-root destination
* symlink-to-directory as a directory) and CROSSES the wire along with
* --munge-links (so the receiver knows to unmunge). */
bool copy_dirlinks; /* client-only, sender-side (-k) */
bool munge_links; /* crosses the wire */
bool keep_dirlinks; /* crosses the wire (-K) */
// Issue #121: Extended metadata preservation // Issue #121: Extended metadata preservation
bool preserve_hard_links; bool preserve_hard_links;
@@ -115,6 +124,22 @@ typedef struct Config {
bool preserve_xattrs; bool preserve_xattrs;
bool preserve_devices; bool preserve_devices;
bool preserve_sparse; bool preserve_sparse;
/* Phase 4 special/devices: preserve special files (FIFOs, sockets) and device
* nodes on the destination by recreating them (mknod/mkfifo) instead of
* transferring content. preserve_specials mirrors rsync --specials (the
* special-file half of -D); preserve_devices mirrors --devices (the device
* half of -D); both CROSS the wire so the receiver knows a special/device
* entry must be recreated rather than written as a regular file. */
bool preserve_specials;
/* --copy-devices: copy the CONTENT of a source device as an ordinary regular
* file on the destination (rsync's non-privileged safe mode), instead of
* recreating the device node. CROSSES the wire (receiver treats the entry as
* a regular file, which is the default, so this is belt-and-braces). */
bool copy_devices;
/* --write-devices: write the received data directly INTO an existing device
* node on the destination instead of creating a regular file. Dangeroud;
* see RSYNC_COMPAT.md for the tight gating. CROSSES the wire. */
bool write_devices;
// Issue #122: Output/logging options // Issue #122: Output/logging options
bool itemize_changes; bool itemize_changes;
@@ -310,9 +335,24 @@ typedef struct Config {
* source files with O_NOATIME so reading for transfer does not bump the * source files with O_NOATIME so reading for transfer does not bump the
* source access time. */ * source access time. */
bool open_noatime; bool open_noatime;
// Phase 4: xattr / ACL / fake-super preservation.
/* -X/--xattrs and -A/--acls toggle the sender's capture and the receiver's
* application of per-file extended attributes (xattrs). Both cross the wire:
* the sender only transmits the bounded, whitelisted attribute set it
* captures and the receiver re-validates namespaces/sizes before applying
* fd-relative. With neither set (the default) no xattr block is sent, so the
* wire is byte-identical to prior protocol versions for unaffected runs. */
/* true when preserve_xattrs || preserve_acls; the sender/receiver gate the
* xattr wire block on this single flag. */
bool use_xattrs;
/* --fake-super: receiver-only. When set, each written file additionally gets
* a reserved user.fastsync.stat xattr recording the source uid/gid/mode/mtime
* so a later privileged restore could re-apply them. Crosses the wire. */
bool fake_super;
} Config; } Config;
#define PROTOCOL_VERSION "2.12.0" #define PROTOCOL_VERSION "2.13.0"
#define DEFAULT_CHUNK_SIZE (10 * 1024 * 1024) #define DEFAULT_CHUNK_SIZE (10 * 1024 * 1024)
/* Upper bound on total basis-dir entries (rsync caps --link-dest at 20). */ /* Upper bound on total basis-dir entries (rsync caps --link-dest at 20). */
#define MAX_BASIS_DIRS 64 #define MAX_BASIS_DIRS 64
+238 -12
View File
@@ -20,6 +20,7 @@
#include "metadata.h" #include "metadata.h"
#include "utils.h" #include "utils.h"
#include "protocol.h" #include "protocol.h"
#include "xattr.h"
static bool write_all(int fd, const void* data, unsigned long long size) { static bool write_all(int fd, const void* data, unsigned long long size) {
const unsigned char* p = data; const unsigned char* p = data;
@@ -114,6 +115,12 @@ File* file_create(const char* path) {
file->link_group = 0; file->link_group = 0;
file->link_first = false; file->link_first = false;
file->hardlink_target = NULL; file->hardlink_target = NULL;
file->is_symlink = false;
file->symlink_target = NULL;
file->is_special = false;
file->rdev_major = 0;
file->rdev_minor = 0;
file->xattrs = NULL;
return file; return file;
} }
@@ -133,6 +140,10 @@ void file_destroy(void* item) {
file->basis_link = NULL; file->basis_link = NULL;
free(file->hardlink_target); free(file->hardlink_target);
file->hardlink_target = NULL; file->hardlink_target = NULL;
free(file->symlink_target);
file->symlink_target = NULL;
xattr_list_free(file->xattrs);
file->xattrs = NULL;
free(file); free(file);
} }
@@ -336,6 +347,108 @@ bool file_destination_is_newer_secure(const char* path, const FileMetadata* meta
return file_stat_secure(path, &st) && stat_is_newer(&st, metadata); return file_stat_secure(path, &st) && stat_is_newer(&st, metadata);
} }
/* --keep-dirlinks (-K) receiver process-wide policy: when set, a destination
* path component that is itself a symlink to an in-root directory is followed
* (used as that directory) instead of failing the O_NOFOLLOW walk. Only ever
* honoured when the resolved target is a directory that stays beneath the
* authorized root, so a malicious symlink can never redirect the write outside
* it. Client of record is the server's receiver. */
static bool file_keep_dirlinks = false;
void file_set_keep_dirlinks(bool enable) {
file_keep_dirlinks = enable;
}
bool file_get_keep_dirlinks(void) {
return file_keep_dirlinks;
}
/* True when `target` is a lexical symlink target that can never escape the
* receive root once created beneath it: relative (not absolute) and containing
* no ".." path component. Used by --munge-links' sender-side containment: an
* escaping target is never transmitted (the entry is skipped/contained). */
bool file_symlink_target_contained(const char* target) {
if (!target || target[0] == '\0' || target[0] == '/')
return false;
const char* p = target;
while (*p) {
const char* slash = strchr(p, '/');
size_t comp_len = slash ? (size_t)(slash - p) : strlen(p);
if (comp_len == 2 && p[0] == '.' && p[1] == '.')
return false;
if (!slash)
break;
p = slash + 1;
}
return true;
}
/* Remove a leading symlink munge marker (if present); returns true when the
* marker was stripped. `target` is a mutable NUL-terminated buffer. */
bool file_symlink_unmunge(char* target) {
if (!target)
return false;
static const char* const marker = SYMLINK_MUNGE_PREFIX;
size_t marker_len = strlen(marker);
if (strncmp(target, marker, marker_len) != 0)
return false;
size_t rest = strlen(target + marker_len) + 1;
memmove(target, target + marker_len, rest);
return true;
}
/* Owned copy of `target` prefixed with SYMLINK_MUNGE_PREFIX (the sender-side
* --munge-links rewriting). Returns NULL on allocation failure. */
char* file_symlink_munge(const char* target) {
if (!target)
return NULL;
static const char* const marker = SYMLINK_MUNGE_PREFIX;
size_t marker_len = strlen(marker);
size_t target_len = strlen(target);
char* out = malloc(marker_len + target_len + 1);
if (!out)
return NULL;
memcpy(out, marker, marker_len);
memcpy(out + marker_len, target, target_len + 1);
return out;
}
/* Create a symlink at `path` pointing to `target`, confined below the
* authorized root: the parent directory is opened with an O_NOFOLLOW fd walk
* and the link is created with symlinkat so neither the destination chain nor
* the target is ever followed. The final component is never dereferenced: an
* existing non-directory entry at `path` is unlinked by name before the link is
* placed; an existing directory there is left untouched (returns false, so a
* caller can treat it as a collision). As a receiver-side trust-boundary
* invariant, `target` must be file_symlink_target_contained() (relative and
* ".."-free): an absolute or escaping target is rejected outright (returns
* false) so a malicious sender can never materialize a symlink that points
* outside the receive root. */
bool file_symlink_at_secure(const char* path, const char* target) {
if (!path || !target || has_path_traversal(path) || !file_symlink_target_contained(target))
return false;
char* leaf = NULL;
int parent_fd = file_open_secure_parent(path, &leaf, true);
if (parent_fd < 0)
return false;
bool ok = false;
struct stat st;
bool exists = fstatat(parent_fd, leaf, &st, AT_SYMLINK_NOFOLLOW) == 0;
if (exists && S_ISDIR(st.st_mode)) {
/* A directory already at this path cannot be replaced atomically with a
symlink without --force semantics; leave it and report the collision. */
ok = false;
} else {
if (exists && unlinkat(parent_fd, leaf, 0) != 0 && errno != ENOENT)
goto out;
ok = symlinkat(target, parent_fd, leaf) == 0;
}
out:
close(parent_fd);
free(leaf);
return ok;
}
int file_open_secure_parent(const char* path, char** leaf_out, bool create_dirs) { int file_open_secure_parent(const char* path, char** leaf_out, bool create_dirs) {
char* copy = str_dup(path); char* copy = str_dup(path);
if (!copy) if (!copy)
@@ -383,6 +496,7 @@ int file_open_secure_parent(const char* path, char** leaf_out, bool create_dirs)
} }
char* save = NULL; char* save = NULL;
char* component = strtok_r(parent, "/", &save); char* component = strtok_r(parent, "/", &save);
char rel_buf[PATH_MAX] = "";
while (component) { while (component) {
if (strcmp(component, "..") == 0) { if (strcmp(component, "..") == 0) {
close(fd); close(fd);
@@ -392,10 +506,45 @@ int file_open_secure_parent(const char* path, char** leaf_out, bool create_dirs)
} }
if (strcmp(component, ".") != 0) { if (strcmp(component, ".") != 0) {
int next = openat(fd, component, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC); int next = openat(fd, component, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
if (create_dirs && next < 0 && errno == ENOENT) { if (next < 0 && create_dirs && errno == ENOENT) {
if (mkdirat(fd, component, 0755) == 0 || errno == EEXIST) if (mkdirat(fd, component, 0755) == 0 || errno == EEXIST)
next = openat(fd, component, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC); next = openat(fd, component, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
} }
/* --keep-dirlinks (-K): a path component that is an existing symlink to
an in-root directory is used as THAT directory rather than failing the
O_NOFOLLOW walk. Only honoured when the symlink resolves to a
directory that stays beneath the authorized root, so a malicious link
can never redirect the write outside it. */
if (next < 0 && file_keep_dirlinks && authorized_root_path != NULL &&
(errno == ELOOP || errno == ENOTDIR || errno == EACCES)) {
struct stat lst;
if (fstatat(fd, component, &lst, AT_SYMLINK_NOFOLLOW) == 0 && S_ISLNK(lst.st_mode)) {
char candidate[PATH_MAX];
char root[PATH_MAX];
if (realpath(authorized_root_path, root) &&
snprintf(candidate, sizeof(candidate), "%s%s/%s", root, rel_buf, component) <
(int)sizeof(candidate)) {
char resolved[PATH_MAX];
if (realpath(candidate, resolved) && strcmp(resolved, root) != 0 &&
strncmp(root, resolved, strlen(root)) == 0 &&
(resolved[strlen(root)] == '/' || resolved[strlen(root)] == '\0')) {
struct stat rst;
if (stat(resolved, &rst) == 0 && S_ISDIR(rst.st_mode)) {
/* Re-open the resolved directory WITHOUT following a symlink and
re-verify it is still a directory inode, so a symlink swapped
in between realpath() and open() (TOCTOU) cannot redirect this
fd outside the root. */
next = open(resolved, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
struct stat ofst;
if (next >= 0 && (fstat(next, &ofst) != 0 || !S_ISDIR(ofst.st_mode))) {
close(next);
next = -1;
}
}
}
}
}
}
if (next < 0) { if (next < 0) {
close(fd); close(fd);
free(copy); free(copy);
@@ -404,6 +553,20 @@ int file_open_secure_parent(const char* path, char** leaf_out, bool create_dirs)
} }
close(fd); close(fd);
fd = next; fd = next;
/* Track the walked relative prefix so the -K candidate path can be
reconstructed. An overflow while building it means the whole path is
at the PATH_MAX edge, so fail hard rather than silently building a
wrong (truncated) candidate for a later -K follow. */
size_t need = strlen(rel_buf) + strlen(component) + 2;
if (need <= sizeof(rel_buf)) {
strcat(rel_buf, "/");
strcat(rel_buf, component);
} else if (file_keep_dirlinks) {
close(fd);
free(copy);
free(leaf);
return -1;
}
} }
component = strtok_r(NULL, "/", &save); component = strtok_r(NULL, "/", &save);
} }
@@ -608,11 +771,25 @@ int file_open_private_dir(const char* dir_path) {
return fd; return fd;
} }
/* After the content and mode/times are restored on the just-written file, apply
* the per-file xattrs (-X/-A) and, for --fake-super, park the source's
* uid/gid/mode/mtime in the reserved xattr. All fd-relative (confined to the
* destination file) and best-effort: a per-attribute or privilege failure is
* logged and skipped, never fatal. */
static void restore_extra_fd(int fd, const FileMetadata* metadata, const FileXattrList* xattrs,
bool fake_super) {
xattr_apply_fd(fd, xattrs);
if (fake_super && metadata)
fake_super_store_fd(fd, (uint32_t)metadata->uid, (uint32_t)metadata->gid,
(uint32_t)metadata->mode, metadata->mtime_sec, metadata->mtime_nsec);
}
static bool file_to_disk_secure_impl(const char* path, const void* data, static bool file_to_disk_secure_impl(const char* path, const void* data,
unsigned long long data_size, bool inplace, bool sparse, unsigned long long data_size, bool inplace, bool sparse,
bool preallocate, const FileMetadata* metadata, bool preallocate, const FileMetadata* metadata,
bool preserve_executability, bool update, bool no_replace, bool preserve_executability, bool update, bool no_replace,
bool use_fsync, const char* temp_dir) { bool use_fsync, const char* temp_dir,
const FileXattrList* xattrs, bool fake_super) {
char* leaf = NULL; char* leaf = NULL;
int dirfd = file_open_secure_parent(path, &leaf, true); int dirfd = file_open_secure_parent(path, &leaf, true);
if (dirfd < 0) if (dirfd < 0)
@@ -662,6 +839,8 @@ static bool file_to_disk_secure_impl(const char* path, const void* data,
else if (fchmod(fd, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH) != 0) else if (fchmod(fd, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH) != 0)
ok = false; ok = false;
} }
if (ok)
restore_extra_fd(fd, metadata, xattrs, fake_super);
if (ok && use_fsync) if (ok && use_fsync)
ok = fsync(fd) == 0; ok = fsync(fd) == 0;
} }
@@ -749,6 +928,8 @@ static bool file_to_disk_secure_impl(const char* path, const void* data,
ok = write_all(fd, data, data_size); ok = write_all(fd, data, data_size);
if (ok && metadata) if (ok && metadata)
ok = file_restore_metadata_fd(fd, metadata, preserve_executability); ok = file_restore_metadata_fd(fd, metadata, preserve_executability);
if (ok)
restore_extra_fd(fd, metadata, xattrs, fake_super);
if (ok && use_fsync) if (ok && use_fsync)
ok = fsync(fd) == 0; ok = fsync(fd) == 0;
} }
@@ -802,7 +983,8 @@ bool file_to_disk_secure(const char* path, const void* data, unsigned long long
bool inplace, bool sparse, bool preallocate, const FileMetadata* metadata, bool inplace, bool sparse, bool preallocate, const FileMetadata* metadata,
bool preserve_executability, const char* temp_dir) { bool preserve_executability, const char* temp_dir) {
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata, return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
preserve_executability, false, false, false, temp_dir); preserve_executability, false, false, false, temp_dir, NULL,
false);
} }
bool file_to_disk_secure_update(const char* path, const void* data, unsigned long long data_size, bool file_to_disk_secure_update(const char* path, const void* data, unsigned long long data_size,
@@ -810,7 +992,8 @@ bool file_to_disk_secure_update(const char* path, const void* data, unsigned lon
const FileMetadata* metadata, bool preserve_executability, const FileMetadata* metadata, bool preserve_executability,
const char* temp_dir) { const char* temp_dir) {
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata, return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
preserve_executability, true, false, false, temp_dir); preserve_executability, true, false, false, temp_dir, NULL,
false);
} }
bool file_to_disk_secure_with_fsync(const char* path, const void* data, bool file_to_disk_secure_with_fsync(const char* path, const void* data,
@@ -819,7 +1002,8 @@ bool file_to_disk_secure_with_fsync(const char* path, const void* data,
bool preserve_executability, bool use_fsync, bool preserve_executability, bool use_fsync,
const char* temp_dir) { const char* temp_dir) {
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata, return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
preserve_executability, false, false, use_fsync, temp_dir); preserve_executability, false, false, use_fsync, temp_dir, NULL,
false);
} }
bool file_to_disk_secure_no_replace(const char* path, const void* data, bool file_to_disk_secure_no_replace(const char* path, const void* data,
@@ -827,7 +1011,22 @@ bool file_to_disk_secure_no_replace(const char* path, const void* data,
const FileMetadata* metadata, bool preserve_executability, const FileMetadata* metadata, bool preserve_executability,
const char* temp_dir) { const char* temp_dir) {
return file_to_disk_secure_impl(path, data, data_size, false, sparse, preallocate, metadata, return file_to_disk_secure_impl(path, data, data_size, false, sparse, preallocate, metadata,
preserve_executability, false, true, false, temp_dir); preserve_executability, false, true, false, temp_dir, NULL,
false);
}
/* Receiver write-path variant that also applies the per-file xattrs (-X/-A)
* and, under --fake-super, parks the source stat in the reserved xattr, on the
* just-written file descriptor before the final rename. `no_replace` / `update`
* mirror the plain wrappers; see file_to_disk_secure_impl for the semantics. */
bool file_to_disk_secure_attrs(const char* path, const void* data, unsigned long long data_size,
bool inplace, bool sparse, bool preallocate,
const FileMetadata* metadata, bool preserve_executability,
bool update, bool no_replace, bool use_fsync,
const FileXattrList* xattrs, bool fake_super, const char* temp_dir) {
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
preserve_executability, update, no_replace, use_fsync, temp_dir,
xattrs, fake_super);
} }
/* Atomic --link-dest install. The destination is replaced (via a temporary /* Atomic --link-dest install. The destination is replaced (via a temporary
@@ -839,10 +1038,18 @@ bool file_to_disk_secure_no_replace(const char* path, const void* data,
* fallback; a successful hard link keeps the basis inode's own attributes * fallback; a successful hard link keeps the basis inode's own attributes
* (applying metadata through the shared inode would mutate the basis file). * (applying metadata through the shared inode would mutate the basis file).
* Returns false only when both the link and the copy fallback fail. */ * Returns false only when both the link and the copy fallback fail. */
bool file_to_disk_secure_link(const char* path, const char* basis_path, const void* data, /* --link-dest / -H hardlink install with a byte-copy fallback. `metadata` is
unsigned long long data_size, bool preallocate, * applied only on the copy fallback; a successful hard link keeps the basis
const FileMetadata* metadata, bool preserve_executability, * inode's own attributes (applying through the shared inode would mutate the
bool use_fsync, const char* temp_dir) { * basis). Likewise `xattrs`/`fake_super` are applied only on the copy
* fallback, so a fallback copy preserves the per-file attributes instead of
* silently dropping them. */
static bool file_to_disk_secure_link_impl(const char* path, const char* basis_path,
const void* data, unsigned long long data_size,
bool preallocate, const FileMetadata* metadata,
bool preserve_executability, bool use_fsync,
const FileXattrList* xattrs, bool fake_super,
const char* temp_dir) {
if (!path || !basis_path) if (!path || !basis_path)
return false; return false;
char* leaf = NULL; char* leaf = NULL;
@@ -920,8 +1127,9 @@ bool file_to_disk_secure_link(const char* path, const char* basis_path, const vo
free(leaf); free(leaf);
/* The basis file could not be linked in (missing, cross-device, refused /* The basis file could not be linked in (missing, cross-device, refused
by the filesystem). Write a byte-identical local copy instead. */ by the filesystem). Write a byte-identical local copy instead. */
return file_to_disk_secure_with_fsync(path, data, data_size, false, false, preallocate, return file_to_disk_secure_attrs(path, data, data_size, false, false, preallocate, metadata,
metadata, preserve_executability, use_fsync, temp_dir); preserve_executability, false, false, use_fsync, xattrs,
fake_super, temp_dir);
} }
if (scratch_dirfd >= 0) if (scratch_dirfd >= 0)
@@ -931,6 +1139,24 @@ bool file_to_disk_secure_link(const char* path, const char* basis_path, const vo
return true; return true;
} }
bool file_to_disk_secure_link(const char* path, const char* basis_path, const void* data,
unsigned long long data_size, bool preallocate,
const FileMetadata* metadata, bool preserve_executability,
bool use_fsync, const char* temp_dir) {
return file_to_disk_secure_link_impl(path, basis_path, data, data_size, preallocate, metadata,
preserve_executability, use_fsync, NULL, false, temp_dir);
}
bool file_to_disk_secure_link_attrs(const char* path, const char* basis_path, const void* data,
unsigned long long data_size, bool preallocate,
const FileMetadata* metadata, bool preserve_executability,
bool use_fsync, const FileXattrList* xattrs, bool fake_super,
const char* temp_dir) {
return file_to_disk_secure_link_impl(path, basis_path, data, data_size, preallocate, metadata,
preserve_executability, use_fsync, xattrs, fake_super,
temp_dir);
}
bool file_write_to_disk(const char* path, const void* data, unsigned long long data_size, bool file_write_to_disk(const char* path, const void* data, unsigned long long data_size,
bool inplace, bool sparse) { bool inplace, bool sparse) {
if (!path || (!data && data_size != 0) || has_path_traversal(path)) if (!path || (!data && data_size != 0) || has_path_traversal(path))
+38
View File
@@ -33,6 +33,27 @@ int file_open_for_read(const char* path);
bool file_write_to_disk(const char* path, const void* data, unsigned long long data_size, bool file_write_to_disk(const char* path, const void* data, unsigned long long data_size,
bool inplace, bool sparse); bool inplace, bool sparse);
/* Symlink trust-boundary helpers (Phase 4, symlink wave). --munge-links
* sender-side marker: every transmitted symlink target is prefixed with this
* while the flag is on; the receiver strips it to restore the real target. */
#define SYMLINK_MUNGE_PREFIX "#SYMLINK/"
char* file_symlink_munge(const char* target);
/* True when a lexical target is relative and contains no ".." component, so it
* can never escape the receive root once created beneath it. */
bool file_symlink_target_contained(const char* target);
/* Strip a leading SYMLINK_MUNGE_PREFIX from `target` (mutable, in place);
* returns true when a marker was removed. */
bool file_symlink_unmunge(char* target);
/* Create a symlink at `path` -> `target`, confined below the authorized root
* (O_NOFOLLOW parent walk, symlinkat; the target is never followed). Returns
* false when a directory already occupies `path`. */
bool file_symlink_at_secure(const char* path, const char* target);
/* --keep-dirlinks (-K) receiver process-wide policy: allow an in-root existing
* symlink-to-directory to be followed as a directory. */
void file_set_keep_dirlinks(bool enable);
bool file_get_keep_dirlinks(void);
/* A configured fd without a canonical identity deliberately rejects paths. */ /* A configured fd without a canonical identity deliberately rejects paths. */
bool file_set_authorized_root(int fd, const char* canonical_path); bool file_set_authorized_root(int fd, const char* canonical_path);
@@ -81,6 +102,14 @@ bool file_to_disk_secure_no_replace(const char* path, const void* data,
unsigned long long data_size, bool sparse, bool preallocate, unsigned long long data_size, bool sparse, bool preallocate,
const FileMetadata* metadata, bool preserve_executability, const FileMetadata* metadata, bool preserve_executability,
const char* temp_dir); const char* temp_dir);
/* Receiver write-path variant that also applies per-file xattrs (-X/-A) and the
* --fake-super stat xattr fd-relative before the final rename. `update` /
* `no_replace` / `use_fsync` mirror the plain wrappers above. */
bool file_to_disk_secure_attrs(const char* path, const void* data, unsigned long long data_size,
bool inplace, bool sparse, bool preallocate,
const FileMetadata* metadata, bool preserve_executability,
bool update, bool no_replace, bool use_fsync,
const FileXattrList* xattrs, bool fake_super, const char* temp_dir);
/* Atomic --link-dest install: replace `path` with a hard link to `basis_path` /* Atomic --link-dest install: replace `path` with a hard link to `basis_path`
(via a temp name + rename); fall back to a byte-identical local copy from (via a temp name + rename); fall back to a byte-identical local copy from
`data` when the link is impossible (EXDEV/EPERM/unsupported filesystem). `data` when the link is impossible (EXDEV/EPERM/unsupported filesystem).
@@ -91,5 +120,14 @@ bool file_to_disk_secure_link(const char* path, const char* basis_path, const vo
unsigned long long data_size, bool preallocate, unsigned long long data_size, bool preallocate,
const FileMetadata* metadata, bool preserve_executability, const FileMetadata* metadata, bool preserve_executability,
bool use_fsync, const char* temp_dir); bool use_fsync, const char* temp_dir);
/* Like file_to_disk_secure_link, but the byte-copy fallback also applies the
* per-file xattrs (-X/-A) and --fake-super stat xattr (fd-relative). On a
* successful hard link no attributes are applied (the shared inode already
* carries the basis's). */
bool file_to_disk_secure_link_attrs(const char* path, const char* basis_path, const void* data,
unsigned long long data_size, bool preallocate,
const FileMetadata* metadata, bool preserve_executability,
bool use_fsync, const FileXattrList* xattrs, bool fake_super,
const char* temp_dir);
#endif #endif
+477 -24
View File
@@ -6,6 +6,7 @@
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
#include <sys/stat.h> #include <sys/stat.h>
#include <sys/sysmacros.h>
#include <unistd.h> #include <unistd.h>
#include "array_list.h" #include "array_list.h"
@@ -20,6 +21,7 @@
#include "metadata.h" #include "metadata.h"
#include "protocol.h" #include "protocol.h"
#include "utils.h" #include "utils.h"
#include "xattr.h"
#define MAX_SERVER_DELETE_COUNT 100000U #define MAX_SERVER_DELETE_COUNT 100000U
#define MAX_FILE_DATA_SIZE MAX_RECEIVE_WHOLE_FILE_SIZE #define MAX_FILE_DATA_SIZE MAX_RECEIVE_WHOLE_FILE_SIZE
@@ -84,9 +86,10 @@ static FileSaveResult file_stage_delayed_update(const char* root_directory,
config->preallocate, metadata, preserve_executability, config->preallocate, metadata, preserve_executability,
config->use_fsync, NULL); config->use_fsync, NULL);
} else { } else {
ok = file_to_disk_secure_with_fsync(staged_path, file->data->data, file->data->size, false, ok =
sparse, config->preallocate, metadata, file_to_disk_secure_attrs(staged_path, file->data->data, file->data->size, false, sparse,
preserve_executability, config->use_fsync, NULL); config->preallocate, metadata, preserve_executability, false,
false, config->use_fsync, file->xattrs, config->fake_super, NULL);
} }
if (!ok) { if (!ok) {
free(staged_path); free(staged_path);
@@ -249,9 +252,11 @@ static FileSaveResult file_save_hardlink_sibling(const char* root_directory, con
free(destination_path); free(destination_path);
return absent_result; return absent_result;
} }
bool ok = FileXattrList* sibling_xattrs = cfg->use_xattrs ? xattr_capture_path(staged_first) : NULL;
file_to_disk_secure_link(staged_sibling, staged_first, content, content_size, preallocate, bool ok = file_to_disk_secure_link_attrs(
file->metadata, preserve_executability, use_fsync, NULL); staged_sibling, staged_first, content, content_size, preallocate, file->metadata,
preserve_executability, use_fsync, sibling_xattrs, cfg ? cfg->fake_super : false, NULL);
xattr_list_free(sibling_xattrs);
free(content); free(content);
if (ok) if (ok)
ok = delay_updates_record(cfg->delay_context, staged_sibling, destination_path, file->path); ok = delay_updates_record(cfg->delay_context, staged_sibling, destination_path, file->path);
@@ -279,15 +284,242 @@ static FileSaveResult file_save_hardlink_sibling(const char* root_directory, con
return absent_result; return absent_result;
} }
const char* temp_dir = (cfg && cfg->temp_dir) ? cfg->temp_dir : NULL; const char* temp_dir = (cfg && cfg->temp_dir) ? cfg->temp_dir : NULL;
bool ok = FileXattrList* sibling_xattrs = cfg->use_xattrs ? xattr_capture_path(first_disk) : NULL;
file_to_disk_secure_link(destination_path, first_disk, content, content_size, preallocate, bool ok = file_to_disk_secure_link_attrs(
file->metadata, preserve_executability, use_fsync, temp_dir); destination_path, first_disk, content, content_size, preallocate, file->metadata,
preserve_executability, use_fsync, sibling_xattrs, cfg ? cfg->fake_super : false, temp_dir);
xattr_list_free(sibling_xattrs);
free(content); free(content);
free(first_disk); free(first_disk);
free(destination_path); free(destination_path);
return ok ? FILE_SAVE_WRITTEN : FILE_SAVE_ERROR; return ok ? FILE_SAVE_WRITTEN : FILE_SAVE_ERROR;
} }
/* Validate a transmitted special rdev against the node kind implied by `mode`'s
* S_IFMT bits. Char/block devices require a legal major/minor pair (non-negative,
* range-checked); a non-device special (FIFO/socket) must carry an empty rdev.
* Used identically on the wire path and at the secure recreation site so a
* malicious/bogus rdev can never drive a dangerous node. */
bool file_special_rdev_valid(int32_t major, int32_t minor, mode_t mode) {
bool is_device = S_ISCHR(mode) || S_ISBLK(mode);
if (is_device)
return major >= 0 && minor >= 0 && major <= 0xffff && minor <= 0x00ffffff;
/* A non-device entry must actually be a special (FIFO/socket) and carry no
rdev; a regular/dir mode is never a valid special node. */
return (S_ISFIFO(mode) || S_ISSOCK(mode)) && major == 0 && minor == 0;
}
/* ---- Device/special node RECREATION (--devices/--specials), receiver side ----
*
* Privilege gating: making a real device node requires CAP_MKNOD (root); making
* a FIFO works unprivileged (mkfifo). When the receiver lacks the capability,
* mknodat() fails with EPERM and the entry is SKIPPED with a warning -- the
* whole transfer must NOT abort just because the environment cannot make the
* node. CI runs non-root, so device creation is expected to skip there and
* only a FIFO is honestly assertable unprivileged.
*
* Confinement: the parent directory is opened fd-relative below the receive
* root (file_open_secure_parent: O_NOFOLLOW, no "..", root-checked) and the
* node is created with mknodat()/mkfifoat(), so it can never be placed outside
* the confined root and never follows a symlink.
*
* rdev validation: a malicious/bogus rdev (negative, out-of-range) is rejected
* here as well as on the wire (file_receive_special / chunk_deserialize), and a
* non-device entry must carry an empty rdev.
*/
static FileSaveResult file_save_special_to_disk(const char* root_directory, const File* file,
const Config* config) {
if (!root_directory || !file || !file->path || file->path[0] == '\0' ||
has_path_traversal(file->path) || !file->metadata)
return FILE_SAVE_ERROR;
mode_t mode = file->metadata->mode;
bool is_char = S_ISCHR(mode);
bool is_blk = S_ISBLK(mode);
bool is_fifo = S_ISFIFO(mode);
bool is_sock = S_ISSOCK(mode);
if (!is_char && !is_blk && !is_fifo && !is_sock) {
log_message(LOG_LEVEL_ERROR, "Special node has no device/FIFO/socket mode");
return FILE_SAVE_ERROR;
}
if (is_sock) {
/* No standard filesystem call recreates a socket; best-effort unsupported. */
log_message(LOG_LEVEL_WARNING, "socket not recreated: %s (unsupported; skipped)", file->path);
return FILE_SAVE_SKIPPED;
}
if (is_char || is_blk) {
if (!config || !config->preserve_devices)
return FILE_SAVE_SKIPPED;
} else if (is_fifo) {
if (!config || !config->preserve_specials)
return FILE_SAVE_SKIPPED;
}
/* Defense-in-depth rdev/type validation (also done on the wire path). */
if (!file_special_rdev_valid(file->rdev_major, file->rdev_minor, mode)) {
log_message(LOG_LEVEL_ERROR, "Rejected out-of-range device rdev %d:%d", file->rdev_major,
file->rdev_minor);
return FILE_SAVE_ERROR;
}
char* destination = path_cat(root_directory, file->path);
if (!destination)
return FILE_SAVE_ERROR;
char* leaf = NULL;
int parent_fd = file_open_secure_parent(destination, &leaf, true);
if (parent_fd < 0) {
free(destination);
return FILE_SAVE_ERROR;
}
/* --existing / --ignore-existing / --update decide against the node that
would be replaced, mirroring the regular-file path. */
if (config->existing && !file_path_exists_secure(destination)) {
close(parent_fd);
free(leaf);
free(destination);
return FILE_SAVE_SKIPPED;
}
if (config->ignore_existing && file_path_exists_secure(destination)) {
close(parent_fd);
free(leaf);
free(destination);
return FILE_SAVE_SKIPPED;
}
if (config->update && file_destination_is_newer_secure(destination, file->metadata)) {
close(parent_fd);
free(leaf);
free(destination);
return FILE_SAVE_SKIPPED;
}
dev_t rdev = 0;
mode_t create_mode;
if (is_char) {
create_mode = S_IFCHR;
rdev = makedev((unsigned)file->rdev_major, (unsigned)file->rdev_minor);
} else if (is_blk) {
create_mode = S_IFBLK;
rdev = makedev((unsigned)file->rdev_major, (unsigned)file->rdev_minor);
} else {
create_mode = S_IFIFO;
}
mode_t perms = mode & 0777;
int rc = is_fifo ? mkfifoat(parent_fd, leaf, perms)
: mknodat(parent_fd, leaf, create_mode | perms, rdev);
if (rc != 0) {
if (errno == EEXIST) {
/* An entry already exists: only skip when it already is a matching node;
never replace an existing directory or unrelated entry with the node. */
struct stat st;
if (fstatat(parent_fd, leaf, &st, AT_SYMLINK_NOFOLLOW) == 0 &&
((is_char && S_ISCHR(st.st_mode)) || (is_blk && S_ISBLK(st.st_mode)) ||
(is_fifo && S_ISFIFO(st.st_mode)))) {
close(parent_fd);
free(leaf);
free(destination);
return FILE_SAVE_SKIPPED;
}
log_message(LOG_LEVEL_WARNING, "refusing to replace existing entry with %s: %s (skipped)",
is_fifo ? "FIFO" : "device", file->path);
} else if (errno == EPERM || errno == EACCES) {
/* Missing CAP_MKNOD / parent write permission: the environment cannot
create the node, so skip instead of failing the whole run. */
log_message(LOG_LEVEL_WARNING,
"skipping %s: cannot create %s node (%s)\n"
" --devices/--specials node creation needs privilege (CAP_MKNOD)",
file->path, is_fifo ? "FIFO" : "device", strerror(errno));
} else {
log_message(LOG_LEVEL_WARNING, "failed to create %s %s: %s (skipped)",
is_fifo ? "FIFO" : "device", file->path, strerror(errno));
}
close(parent_fd);
free(leaf);
free(destination);
return FILE_SAVE_SKIPPED;
}
/* Apply mtime on the fresh node (utimensat, no-follow). Ownership is not
applied -- identity fchown needs an fd and would require opening the node. */
struct timespec times[2] = {
{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
{.tv_sec = file->metadata->mtime_sec, .tv_nsec = file->metadata->mtime_nsec}};
utimensat(parent_fd, leaf, times, AT_SYMLINK_NOFOLLOW);
close(parent_fd);
free(leaf);
free(destination);
return FILE_SAVE_WRITTEN;
}
/* --write-devices (receiver): write the received data directly into an EXISTING
* device node on the destination instead of creating a regular file. The node
* must already exist and be a char/block device (the device itself is opened and
* followed); it is confined to the receive root via file_open_secure_parent.
* Dangerous by nature, so deliberately restricted: a missing/non-device
* destination, or a write failure, is SKIPPED with a warning rather than
* allowed. On environments without device access the run still succeeds (the
* entry is skipped), never aborts. */
static FileSaveResult file_save_write_device(const char* root_directory, const File* file) {
if (!root_directory || !file || !file->path || file->path[0] == '\0' ||
has_path_traversal(file->path))
return FILE_SAVE_ERROR;
if (!file->data)
return FILE_SAVE_ERROR;
char* destination = path_cat(root_directory, file->path);
if (!destination)
return FILE_SAVE_ERROR;
char* leaf = NULL;
int parent_fd = file_open_secure_parent(destination, &leaf, false);
if (parent_fd < 0) {
free(destination);
return FILE_SAVE_SKIPPED;
}
/* O_NONBLOCK: a pre-existing FIFO at the target would otherwise block the
receive thread forever on open(2). With it the open only succeeds for a
readerless FIFO with O_RDWR (which the device fstat gate rejects anyway)
or fails with ENXIO/EAGAIN, both treated as a normal skip below. */
int fd = openat(parent_fd, leaf, O_WRONLY | O_CLOEXEC | O_NOFOLLOW | O_NONBLOCK);
int saved_errno = errno;
free(leaf);
close(parent_fd);
if (fd < 0) {
free(destination);
if (saved_errno == ENXIO || saved_errno == EAGAIN) {
/* A FIFO with no reader / an unreadable special: skip like every other
unusable write-devices target instead of blocking or failing. */
log_message(LOG_LEVEL_WARNING, "write-devices: %s not writable (%s); skipped", file->path,
strerror(saved_errno));
} else {
log_message(LOG_LEVEL_WARNING, "write-devices: cannot open %s (%s); skipped", file->path,
strerror(saved_errno));
}
return FILE_SAVE_SKIPPED;
}
struct stat st;
if (fstat(fd, &st) != 0 || !(S_ISCHR(st.st_mode) || S_ISBLK(st.st_mode))) {
close(fd);
free(destination);
log_message(LOG_LEVEL_WARNING, "write-devices: %s is not a device node; skipped", file->path);
return FILE_SAVE_SKIPPED;
}
bool ok = true;
if (file->data->size > 0) {
size_t total = (size_t)file->data->size;
size_t written = 0;
while (written < total) {
ssize_t n = write(fd, (char*)file->data->data + written, total - written);
if (n <= 0) {
ok = false;
break;
}
written += (size_t)n;
}
}
close(fd);
free(destination);
return ok ? FILE_SAVE_WRITTEN : FILE_SAVE_SKIPPED;
}
FileSaveResult file_save_to_disk_full(const char* root_directory, const File* file, FileSaveResult file_save_to_disk_full(const char* root_directory, const File* file,
const Config* config) { const Config* config) {
/* Backups are incompatible with ignore-existing: moving the entry first /* Backups are incompatible with ignore-existing: moving the entry first
@@ -314,6 +546,14 @@ FileSaveResult file_save_to_disk_full(const char* root_directory, const File* fi
return FILE_SAVE_ERROR; return FILE_SAVE_ERROR;
} }
/* Device/special node (--devices/--specials): recreate the node instead of
writing content (privilege-gated, confined, rdev-validated). */
if (file->is_special)
return file_save_special_to_disk(root_directory, file, config);
/* --write-devices: write straight into an existing device node. */
if (config && config->write_devices)
return file_save_write_device(root_directory, file);
/* Explicit directory entries (--dirs) carry an empty payload; the entry is /* Explicit directory entries (--dirs) carry an empty payload; the entry is
created as a directory under the receive root, applying the same secure created as a directory under the receive root, applying the same secure
mkdir-parent semantics as regular writes. Directories are created mkdir-parent semantics as regular writes. Directories are created
@@ -332,6 +572,49 @@ FileSaveResult file_save_to_disk_full(const char* root_directory, const File* fi
return ok ? FILE_SAVE_WRITTEN : FILE_SAVE_ERROR; return ok ? FILE_SAVE_WRITTEN : FILE_SAVE_ERROR;
} }
/* Symlink entry. (The process-wide --keep-dirlinks policy is set once by the
connection handler from the negotiated config, before any receiver/writer
threads start, so it is stable throughout this walk.) */
if (file->is_symlink) {
if (!file->symlink_target || file->path[0] == '\0' || has_path_traversal(file->path)) {
log_message(LOG_LEVEL_ERROR, "Invalid symlink entry received");
return FILE_SAVE_ERROR;
}
char* link_path = path_cat(root_directory, file->path);
if (!link_path)
return FILE_SAVE_ERROR;
/* Restore the real target by stripping the sender's --munge-links marker.
Only unmunge when the policy was negotiated: a plain -l run must preserve
a source symlink whose target genuinely begins with the marker verbatim. */
char* target = str_dup(file->symlink_target);
bool ok = target != NULL;
if (ok && config && config->munge_links)
file_symlink_unmunge(target);
/* Receiver-side trust boundary (independent of the sender): a target that
could escape the receive root (absolute, or relative-with-"..") is never
materialized. It is contained (the entry is skipped) rather than failing
the whole transfer, so a hostile sender can inject a broken symlink but
can never redirect it outside the root. */
if (ok && !file_symlink_target_contained(target))
ok = false;
if (!ok) {
/* Skip the escaping/empty target (contained) rather than abort. */
free(target);
free(link_path);
return FILE_SAVE_SKIPPED;
}
char* parent = str_dup(link_path);
if (parent) {
file_ensure_directory_secure(dirname(parent));
free(parent);
}
ok = file_symlink_at_secure(link_path, target);
free(target);
free(link_path);
return ok ? FILE_SAVE_WRITTEN : FILE_SAVE_ERROR;
}
/* --hard-links/-H sibling: a later member of a link group arrives with no /* --hard-links/-H sibling: a later member of a link group arrives with no
payload and is installed as a hard link to (or, on link() failure, a payload and is installed as a hard link to (or, on link() failure, a
byte-identical copy of) the group's first member. Handled entirely here, byte-identical copy of) the group's first member. Handled entirely here,
@@ -496,22 +779,18 @@ FileSaveResult file_save_to_disk_full(const char* root_directory, const File* fi
policy decision. */ policy decision. */
bool ok; bool ok;
if (config && file->basis_link) { if (config && file->basis_link) {
ok = file_to_disk_secure_link(disk_path, file->basis_link, file->data->data, file->data->size, ok = file_to_disk_secure_link_attrs(disk_path, file->basis_link, file->data->data,
config->preallocate, metadata, preserve_executability, file->data->size, config->preallocate, metadata,
config->use_fsync, confined_temp); preserve_executability, config->use_fsync, file->xattrs,
config->fake_super, confined_temp);
} else { } else {
ok = config && config->ignore_existing /* The plain no-replace / update / with-fsync engines, plus per-file xattr
? file_to_disk_secure_no_replace(disk_path, file->data->data, file->data->size, sparse, (-X/-A) and --fake-super application on the written fd. */
config && config->preallocate, metadata, ok = file_to_disk_secure_attrs(disk_path, file->data->data, file->data->size, inplace, sparse,
preserve_executability, confined_temp) config && config->preallocate, metadata, preserve_executability,
: config && config->update config && config->update, config && config->ignore_existing,
? file_to_disk_secure_update(disk_path, file->data->data, file->data->size, inplace, config && config->use_fsync, file->xattrs,
sparse, config && config->preallocate, metadata, config ? config->fake_super : false, confined_temp);
preserve_executability, confined_temp)
: file_to_disk_secure_with_fsync(disk_path, file->data->data, file->data->size,
inplace, sparse, config && config->preallocate,
metadata, preserve_executability,
config && config->use_fsync, confined_temp);
} }
free(confined_temp); free(confined_temp);
confined_temp = NULL; confined_temp = NULL;
@@ -545,6 +824,21 @@ fail:
return FILE_SAVE_ERROR; return FILE_SAVE_ERROR;
} }
/* Receive a file's xattr block (when the config enables xattr transport) and
* attach it to `file`. Returns false on a malformed/oversized frame. */
static bool receive_file_xattrs(File* file, int fd, const Config* config) {
if (!config->use_xattrs)
return true;
int xok = 0;
FileXattrList* list = xattr_receive(fd, &xok);
if (!xok) {
xattr_list_free(list);
return false;
}
file->xattrs = list;
return true;
}
static File* receive_delta_file(int fd, const Config* config, const char* check_path, static File* receive_delta_file(int fd, const Config* config, const char* check_path,
void* old_data, unsigned long long old_size, bool* failed) { void* old_data, unsigned long long old_size, bool* failed) {
if (!old_data) { if (!old_data) {
@@ -660,6 +954,14 @@ static File* receive_delta_file(int fd, const Config* config, const char* check_
return NULL; return NULL;
} }
} }
if (!receive_file_xattrs(file, fd, config)) {
file_destroy(file);
free(new_data);
free(old_data);
delta_signature_destroy(sig);
*failed = true;
return NULL;
}
Data* replacement = data_create(new_data, (size_t)new_size); Data* replacement = data_create(new_data, (size_t)new_size);
if (replacement == NULL) { if (replacement == NULL) {
@@ -697,6 +999,11 @@ static File* receive_delta_file(int fd, const Config* config, const char* check_
return NULL; return NULL;
} }
} }
if (!receive_file_xattrs(file, fd, config)) {
file_destroy(file);
*failed = true;
return NULL;
}
Data* file_data = receive_data_limited(fd, MAX_RECEIVE_WHOLE_FILE_SIZE); Data* file_data = receive_data_limited(fd, MAX_RECEIVE_WHOLE_FILE_SIZE);
if (file_data == NULL) { if (file_data == NULL) {
@@ -1202,6 +1509,10 @@ static File* receive_full_file(int fd, const Config* config, const char* path) {
return NULL; return NULL;
} }
} }
if (!receive_file_xattrs(file, fd, config)) {
file_destroy(file);
return NULL;
}
Data* file_data = receive_data_limited(fd, MAX_RECEIVE_WHOLE_FILE_SIZE); Data* file_data = receive_data_limited(fd, MAX_RECEIVE_WHOLE_FILE_SIZE);
if (file_data == NULL) { if (file_data == NULL) {
file_destroy(file); file_destroy(file);
@@ -1574,6 +1885,7 @@ File* receive_incremental_check(int fd, const Config* config, bool* skipped) {
return NULL; return NULL;
} }
FileMetadata* meta = NULL; FileMetadata* meta = NULL;
FileXattrList* append_xattrs = NULL;
if (config->use_metadata) { if (config->use_metadata) {
int meta_ok = 1; int meta_ok = 1;
meta = metadata_receive(fd, &meta_ok); meta = metadata_receive(fd, &meta_ok);
@@ -1585,8 +1897,21 @@ File* receive_incremental_check(int fd, const Config* config, bool* skipped) {
return NULL; return NULL;
} }
} }
if (config->use_xattrs) {
int xok = 0;
append_xattrs = xattr_receive(fd, &xok);
if (!xok) {
xattr_list_free(append_xattrs);
close(old_fd);
free(full_path);
free(check_path);
free(old_data);
return NULL;
}
}
Data* tail = receive_data_limited(fd, MAX_RECEIVE_WHOLE_FILE_SIZE); Data* tail = receive_data_limited(fd, MAX_RECEIVE_WHOLE_FILE_SIZE);
if (tail == NULL) { if (tail == NULL) {
xattr_list_free(append_xattrs);
close(old_fd); close(old_fd);
free(full_path); free(full_path);
free(check_path); free(check_path);
@@ -1600,6 +1925,7 @@ File* receive_incremental_check(int fd, const Config* config, bool* skipped) {
Data* uncompressed = data_decompress_limited(tail, MAX_RECEIVE_WHOLE_FILE_SIZE); Data* uncompressed = data_decompress_limited(tail, MAX_RECEIVE_WHOLE_FILE_SIZE);
data_destroy(tail); data_destroy(tail);
if (uncompressed == NULL) { if (uncompressed == NULL) {
xattr_list_free(append_xattrs);
close(old_fd); close(old_fd);
free(full_path); free(full_path);
free(check_path); free(check_path);
@@ -1608,6 +1934,7 @@ File* receive_incremental_check(int fd, const Config* config, bool* skipped) {
} }
if (uncompressed->size > MAX_FILE_DATA_SIZE) { if (uncompressed->size > MAX_FILE_DATA_SIZE) {
data_destroy(uncompressed); data_destroy(uncompressed);
xattr_list_free(append_xattrs);
close(old_fd); close(old_fd);
free(full_path); free(full_path);
free(check_path); free(check_path);
@@ -1623,6 +1950,7 @@ File* receive_incremental_check(int fd, const Config* config, bool* skipped) {
tail->size != (size_t)expected_tail) { tail->size != (size_t)expected_tail) {
send_status(fd, STATUS_ERROR); send_status(fd, STATUS_ERROR);
data_destroy(tail); data_destroy(tail);
xattr_list_free(append_xattrs);
close(old_fd); close(old_fd);
free(full_path); free(full_path);
free(check_path); free(check_path);
@@ -1633,6 +1961,7 @@ File* receive_incremental_check(int fd, const Config* config, bool* skipped) {
void* full = protocol_alloc(full_size ? full_size : 1); void* full = protocol_alloc(full_size ? full_size : 1);
if (!full) { if (!full) {
data_destroy(tail); data_destroy(tail);
xattr_list_free(append_xattrs);
close(old_fd); close(old_fd);
free(full_path); free(full_path);
free(check_path); free(check_path);
@@ -1650,12 +1979,15 @@ File* receive_incremental_check(int fd, const Config* config, bool* skipped) {
File* file = file_create(check_path); File* file = file_create(check_path);
if (!file) { if (!file) {
free(full); free(full);
xattr_list_free(append_xattrs);
close(old_fd); close(old_fd);
free(full_path); free(full_path);
free(check_path); free(check_path);
return NULL; return NULL;
} }
file->metadata = meta; file->metadata = meta;
file->xattrs = append_xattrs;
append_xattrs = NULL;
file->data = data_create(full, full_size); file->data = data_create(full, full_size);
if (!file->data) { /* data_create already freed full on failure */ if (!file->data) { /* data_create already freed full on failure */
file_destroy(file); file_destroy(file);
@@ -1764,6 +2096,10 @@ File* file_receive(const Config* config, int file_descriptor) {
return NULL; return NULL;
} }
} }
if (!receive_file_xattrs(file, file_descriptor, config)) {
file_destroy(file);
return NULL;
}
Data* file_data = receive_data_limited(file_descriptor, MAX_RECEIVE_WHOLE_FILE_SIZE); Data* file_data = receive_data_limited(file_descriptor, MAX_RECEIVE_WHOLE_FILE_SIZE);
if (file_data == NULL) { if (file_data == NULL) {
file_destroy(file); file_destroy(file);
@@ -1868,6 +2204,123 @@ File* file_receive_hardlink(int file_descriptor) {
return file; return file;
} }
/* Receive a symlink entry (the leading STATUS_SYMLINK code has already been
consumed): the destination path and the (sender-munged, if --munge-links)
symlink target string, then metadata when negotiated. The created File is
routed through the regular store_file sink, which creates the link beneath
the receive root (unmungeing the target first). */
File* file_receive_symlink(int file_descriptor, const Config* config) {
char* path = receive_str(file_descriptor);
if (path == NULL)
return NULL;
if (path[0] == '\0' || has_path_traversal(path)) {
char* escaped_path = output_escape(path, log_get_8_bit_output());
log_message(LOG_LEVEL_ERROR, "Invalid received symlink path: %s",
escaped_path ? escaped_path : "<allocation failed>");
free(escaped_path);
free(path);
send_status(file_descriptor, STATUS_ERROR);
return NULL;
}
char* target = receive_str(file_descriptor);
if (!target) {
free(path);
return NULL;
}
if (target[0] == '\0') {
char* escaped = output_escape(target, log_get_8_bit_output());
log_message(LOG_LEVEL_ERROR, "Invalid received symlink target: %s",
escaped ? escaped : "<allocation failed>");
free(escaped);
free(target);
free(path);
send_status(file_descriptor, STATUS_ERROR);
return NULL;
}
File* file = file_create(path);
free(path);
if (!file) {
free(target);
return NULL;
}
if (config && config->use_metadata) {
int meta_ok = 1;
file->metadata = metadata_receive(file_descriptor, &meta_ok);
if (!meta_ok) {
file_destroy(file);
free(target);
return NULL;
}
}
file->is_symlink = true;
file->symlink_target = target;
return file;
}
/* Receive a device/special node frame (--devices/--specials): the leading
* STATUS_SPECIAL code has already been consumed. Payload: the destination path,
* the metadata frame (whose mode's S_IFMT bits carry the node kind), and two
* int32 rdev major/minor fields. The created File carries no payload and is
* recreated by file_save_to_disk_full (mknod/mkfifo, privilege-gated and
* confined). rdev is validated here (non-negative, range-checked) so a bogus
* value cannot drive a dangerous node on the receiver. */
File* file_receive_special(int file_descriptor) {
char* path = receive_str(file_descriptor);
if (path == NULL)
return NULL;
if (path[0] == '\0' || has_path_traversal(path)) {
char* escaped_path = output_escape(path, log_get_8_bit_output());
log_message(LOG_LEVEL_ERROR, "Invalid received special path: %s",
escaped_path ? escaped_path : "<allocation failed>");
free(escaped_path);
free(path);
send_status(file_descriptor, STATUS_ERROR);
return NULL;
}
int meta_ok = 1;
FileMetadata* metadata = metadata_receive(file_descriptor, &meta_ok);
if (!meta_ok) {
free(path);
send_status(file_descriptor, STATUS_ERROR);
return NULL;
}
int32_t major = 0;
int32_t minor = 0;
if (!receive_n_data(file_descriptor, &major, sizeof(major)) ||
!receive_n_data(file_descriptor, &minor, sizeof(minor))) {
free(path);
file_metadata_destroy(metadata);
send_status(file_descriptor, STATUS_ERROR);
return NULL;
}
/* A node kind must be present; without metadata mode there is no S_IFMT to
recreate from. */
if (!metadata) {
log_message(LOG_LEVEL_ERROR, "Special node sent without metadata (mode)");
free(path);
send_status(file_descriptor, STATUS_ERROR);
return NULL;
}
if (!file_special_rdev_valid(major, minor, metadata->mode)) {
log_message(LOG_LEVEL_ERROR, "Invalid special rdev received (%d:%d)", (int)major, (int)minor);
free(path);
file_metadata_destroy(metadata);
send_status(file_descriptor, STATUS_ERROR);
return NULL;
}
File* file = file_create(path);
free(path);
if (file == NULL) {
file_metadata_destroy(metadata);
return NULL;
}
file->metadata = metadata;
file->is_special = true;
file->rdev_major = major;
file->rdev_minor = minor;
return file;
}
/* Read a delete-manifest frame (the STATUS_MANIFEST leading code has already /* Read a delete-manifest frame (the STATUS_MANIFEST leading code has already
been consumed): a keep-set entry count followed by that many been consumed): a keep-set entry count followed by that many
destination-relative paths, then a protected-prefix count followed by that destination-relative paths, then a protected-prefix count followed by that
+3
View File
@@ -10,6 +10,9 @@
File* file_receive(const Config* config, int file_descriptor); File* file_receive(const Config* config, int file_descriptor);
File* file_receive_directory(int file_descriptor); File* file_receive_directory(int file_descriptor);
File* file_receive_hardlink(int file_descriptor); File* file_receive_hardlink(int file_descriptor);
File* file_receive_symlink(int file_descriptor, const Config* config);
File* file_receive_special(int file_descriptor);
bool file_special_rdev_valid(int32_t major, int32_t minor, mode_t mode);
File* receive_incremental_check(int fd, const Config* config, bool* skipped); File* receive_incremental_check(int fd, const Config* config, bool* skipped);
/* A received delete-manifest frame: the keep-set (`keeps`, destination-relative /* A received delete-manifest frame: the keep-set (`keeps`, destination-relative
+31 -5
View File
@@ -16,17 +16,37 @@
#include "log.h" #include "log.h"
#include "metadata.h" #include "metadata.h"
#include "protocol.h" #include "protocol.h"
#include "xattr.h"
/* Transmit a device/special node (--devices / --specials) as a STATUS_SPECIAL
* frame: the destination path, the metadata frame (whose mode's S_IFMT bits
* carry the node kind) and the device rdev major/minor. The receiver validates
* the kind and rdev and recreates the node (privilege-gating the mknod). */
bool file_send_special(File* file, int file_descriptor, bool use_metadata) {
if (!file || !file_wire_path(file))
return false;
if (!send_status(file_descriptor, STATUS_SPECIAL))
return false;
if (!send_str(file_descriptor, file_wire_path(file)))
return false;
if (use_metadata && !metadata_send(file_descriptor, file->metadata))
return false;
int32_t major = file->rdev_major;
int32_t minor = file->rdev_minor;
return send_n_data(file_descriptor, &major, sizeof(major)) &&
send_n_data(file_descriptor, &minor, sizeof(minor));
}
bool file_send_single_calls(File* file, int file_descriptor, bool use_metadata, bool file_send_single_calls(File* file, int file_descriptor, bool use_metadata,
int compression_level, bool send_path) { int compression_level, bool send_path) {
return file_send_single_calls_with_skip(file, file_descriptor, use_metadata, compression_level, return file_send_single_calls_with_skip(file, file_descriptor, use_metadata, compression_level,
send_path, NULL, -1, 0); send_path, NULL, -1, 0, false);
} }
bool file_send_single_calls_with_skip(File* file, int file_descriptor, bool use_metadata, bool file_send_single_calls_with_skip(File* file, int file_descriptor, bool use_metadata,
int compression_level, bool send_path, int compression_level, bool send_path,
char* const* skip_suffixes, int skip_count, char* const* skip_suffixes, int skip_count,
int compression_threads) { int compression_threads, bool send_xattrs) {
if (!file || !file->path || !file->data || (file->data->size != 0 && !file->data->data)) if (!file || !file->path || !file->data || (file->data->size != 0 && !file->data->data))
return false; return false;
const Data* data_to_send = file->data; const Data* data_to_send = file->data;
@@ -49,6 +69,10 @@ bool file_send_single_calls_with_skip(File* file, int file_descriptor, bool use_
data_destroy(compressed_data); data_destroy(compressed_data);
return false; return false;
} }
if (send_xattrs && !xattr_send(file_descriptor, file ? file->xattrs : NULL)) {
data_destroy(compressed_data);
return false;
}
if (!send_data(file_descriptor, data_to_send)) { if (!send_data(file_descriptor, data_to_send)) {
data_destroy(compressed_data); data_destroy(compressed_data);
return false; return false;
@@ -60,23 +84,25 @@ bool file_send_single_calls_with_skip(File* file, int file_descriptor, bool use_
bool file_send_sendfile(File* file, int file_descriptor, bool use_metadata, int compression_level, bool file_send_sendfile(File* file, int file_descriptor, bool use_metadata, int compression_level,
bool send_path) { bool send_path) {
return file_send_sendfile_with_skip(file, file_descriptor, use_metadata, compression_level, return file_send_sendfile_with_skip(file, file_descriptor, use_metadata, compression_level,
send_path, NULL, -1, 0); send_path, NULL, -1, 0, false);
} }
bool file_send_sendfile_with_skip(File* file, int file_descriptor, bool use_metadata, bool file_send_sendfile_with_skip(File* file, int file_descriptor, bool use_metadata,
int compression_level, bool send_path, char* const* skip_suffixes, int compression_level, bool send_path, char* const* skip_suffixes,
int skip_count, int compression_threads) { int skip_count, int compression_threads, bool send_xattrs) {
if (!file || !file->path || !file->data) if (!file || !file->path || !file->data)
return false; return false;
if (compression_level > 0) if (compression_level > 0)
return file_send_single_calls_with_skip(file, file_descriptor, use_metadata, compression_level, return file_send_single_calls_with_skip(file, file_descriptor, use_metadata, compression_level,
send_path, skip_suffixes, skip_count, send_path, skip_suffixes, skip_count,
compression_threads); compression_threads, send_xattrs);
if (send_path && !send_str(file_descriptor, file_wire_path(file))) if (send_path && !send_str(file_descriptor, file_wire_path(file)))
return false; return false;
if (use_metadata && !metadata_send(file_descriptor, file->metadata)) if (use_metadata && !metadata_send(file_descriptor, file->metadata))
return false; return false;
if (send_xattrs && !xattr_send(file_descriptor, file ? file->xattrs : NULL))
return false;
int fd = file_open_for_read(file->path); int fd = file_open_for_read(file->path);
if (fd == -1) { if (fd == -1) {
+3 -2
View File
@@ -6,16 +6,17 @@
/* Client-side file send path. */ /* Client-side file send path. */
bool file_send_special(File* file, int file_descriptor, bool use_metadata);
bool file_send_single_calls(File* file, int file_descriptor, bool use_metadata, bool file_send_single_calls(File* file, int file_descriptor, bool use_metadata,
int compression_level, bool send_path); int compression_level, bool send_path);
bool file_send_single_calls_with_skip(File* file, int file_descriptor, bool use_metadata, bool file_send_single_calls_with_skip(File* file, int file_descriptor, bool use_metadata,
int compression_level, bool send_path, int compression_level, bool send_path,
char* const* skip_suffixes, int skip_count, char* const* skip_suffixes, int skip_count,
int compression_threads); int compression_threads, bool send_xattrs);
bool file_send_sendfile(File* file, int file_descriptor, bool use_metadata, int compression_level, bool file_send_sendfile(File* file, int file_descriptor, bool use_metadata, int compression_level,
bool send_path); bool send_path);
bool file_send_sendfile_with_skip(File* file, int file_descriptor, bool use_metadata, bool file_send_sendfile_with_skip(File* file, int file_descriptor, bool use_metadata,
int compression_level, bool send_path, char* const* skip_suffixes, int compression_level, bool send_path, char* const* skip_suffixes,
int skip_count, int compression_threads); int skip_count, int compression_threads, bool send_xattrs);
#endif #endif
+22
View File
@@ -2,6 +2,7 @@
#define FILE_TYPES_H #define FILE_TYPES_H
#include "data.h" #include "data.h"
#include "xattr.h"
#include <stdbool.h> #include <stdbool.h>
#include <sys/stat.h> #include <sys/stat.h>
@@ -56,6 +57,27 @@ typedef struct {
int link_group; int link_group;
bool link_first; bool link_first;
char* hardlink_target; char* hardlink_target;
/* Symlink-type entry (-l/--links, or -k/--copy-dirlinks' keep-as-symlink
* branch). When true, `symlink_target` holds the (sender-munged, if
* --munge-links) target string that is carried on the wire; the receiver
* creates a symlink to (an unmunged) target instead of writing regular-file
* data. `data` is empty for a symlink entry. Sender + receiver state. */
bool is_symlink;
char* symlink_target;
/* Phase 4 special/devices: when `is_special` is true this entry is a device
* or special node to be RECREATED on the destination (mknod/mkfifo) rather
* than written from `data`. The concrete node kind is derived from the
* metadata mode's S_IFMT bits (receiver-validated), and rdev_major/minor
* carry the device major/minor numbers for char/block devices. CROSSES the
* wire (protocol 2.13.0). */
bool is_special;
int32_t rdev_major;
int32_t rdev_minor;
/* Phase-4 xattrs (-X/--xattrs, -A/--acls). Sender: captured from the source
* file when use_xattrs is set; transmitted in the per-file metadata frame.
* Receiver: parsed off the wire, attached here, and applied fd-relative on
* the written file. NULL/0 == the file carries no xattrs. */
FileXattrList* xattrs;
} File; } File;
/* The path that should be sent on the wire and used for the receiver-side /* The path that should be sent on the wire and used for the receiver-side
+3 -2
View File
@@ -308,8 +308,9 @@ int write_thread(void* pipeline_context) {
} }
/* Record the per-file outcome so a --remove-source-files sender learns /* Record the per-file outcome so a --remove-source-files sender learns
which sources were actually written versus skipped on the receiver. which sources were actually written versus skipped on the receiver.
Explicit directory entries have no source and are never acknowledged. */ Explicit directory entries and recreated device/special nodes have no
if (context->config->remove_source_files && !file->is_dir && !file->skip && source and are never acknowledged (mirrors receiver.c). */
if (context->config->remove_source_files && !file->is_dir && !file->is_special && !file->skip &&
!receiver_outcomes_append(&context->outcomes, (unsigned char)result)) { !receiver_outcomes_append(&context->outcomes, (unsigned char)result)) {
file_destroy(file); file_destroy(file);
pipeline_context_receiver_note_bytes_released(context, file_bytes); pipeline_context_receiver_note_bytes_released(context, file_bytes);
+14 -1
View File
@@ -90,7 +90,20 @@ enum NET_STATUS {
* first (data-carrying) member's destination-relative wire path; the receiver * first (data-carrying) member's destination-relative wire path; the receiver
* creates this entry as a hard link to the first member's installed file * creates this entry as a hard link to the first member's installed file
* (falling back to a byte-identical copy if link() fails). Protocol 2.12.0. */ * (falling back to a byte-identical copy if link() fails). Protocol 2.12.0. */
STATUS_HARDLINK STATUS_HARDLINK,
/* A symlink-type entry (-l/--links, -k/--copy-dirlinks' keep-as-symlink
* branch). The sender transmits the destination path, the (sender-munged,
* if --munge-links) symlink target, and optional metadata; the receiver
* creates a symlink to the unmunged target beneath the receive root (see
* file_receive_symlink). Protocol 2.13.0. */
STATUS_SYMLINK,
/* --devices / --specials (-D): a device or special node the sender wants
* recreated (not written from content). Payload: destination path, the
* metadata frame (whose mode's S_IFMT bits carry the node kind), and two
* int32 rdev major/minor fields. The receiver validates the kind and rdev,
* confines the node below the receive root, and recreates it (mknod/mkfifo),
* privilege-gating the mknod. Protocol 2.13.0. */
STATUS_SPECIAL
}; };
void io_set_fds(int read_fd, int write_fd); void io_set_fds(int read_fd, int write_fd);
+331
View File
@@ -0,0 +1,331 @@
#define _GNU_SOURCE
#include "xattr.h"
#include "log.h"
#include "protocol.h"
#include "utils.h"
#include "file_types.h"
#include <errno.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/xattr.h>
/* ---- lifecycle ---- */
FileXattrList* xattr_list_new(void) {
FileXattrList* list = protocol_alloc(sizeof(FileXattrList));
if (!list)
return NULL;
list->items = NULL;
list->count = 0;
return list;
}
void xattr_list_free(FileXattrList* list) {
if (!list)
return;
for (int i = 0; i < list->count; i++) {
free(list->items[i].name);
free(list->items[i].value);
}
free(list->items);
free(list);
}
bool xattr_list_append(FileXattrList* list, const char* name, const void* value, size_t value_len) {
if (!list || !name || (!value && value_len != 0))
return false;
if (list->count >= XATTR_MAX_COUNT)
return false;
FileXattr* grown = realloc(list->items, ((size_t)list->count + 1) * sizeof(FileXattr));
if (!grown)
return false;
list->items = grown;
size_t name_len = strlen(name);
char* name_copy = malloc(name_len + 1);
if (!name_copy) {
return false;
}
unsigned char* value_copy = NULL;
if (value_len > 0) {
value_copy = malloc(value_len);
if (!value_copy) {
free(name_copy);
return false;
}
memcpy(value_copy, value, value_len);
}
memcpy(name_copy, name, name_len);
name_copy[name_len] = '\0';
list->items[list->count].name = name_copy;
list->items[list->count].value = value_copy;
list->items[list->count].value_len = value_len;
list->count++;
return true;
}
/* ---- namespace / length validation ---- */
/* A Linux xattr name is "namespace.name" with an optional leading "trusted.",
* "system.", "security.", "user.", or "trusted." prefix. We only ever touch
* the unprivileged "user.*" namespace and the two POSIX ACL xattrs carried in
* the "system." namespace. Everything else -- especially "security.*" (ACLs,
* capabilities, SELinux labels) and "trusted.*" -- is refused so a client can
* never compel the receiver to apply a privileged attribute it would not
* otherwise be able to set (and which would be a local privilege escalation if
* it could). */
bool xattr_name_appliable(const char* name) {
if (!name || name[0] == '\0')
return false;
size_t len = strlen(name);
if (len > XATTR_NAME_MAX)
return false;
/* The reserved --fake-super key is exclusively the RECEIVER's: it records the
* source stat for a later privileged restore. A plain -X run must never
* forward a source file that already carries this key (spoofable) onto the
* destination, so it is excluded from capture AND from application. Only
* fake_super_store_fd() writes it. */
if (strcmp(name, FAKESUPER_XATTR) == 0)
return false;
if (strncmp(name, "user.", 5) == 0)
return name[5] != '\0';
if (strcmp(name, "system.posix_acl_access") == 0)
return true;
if (strcmp(name, "system.posix_acl_default") == 0)
return true;
return false;
}
/* ---- SENDER: capture ---- */
FileXattrList* xattr_capture_path(const char* path) {
if (!path)
return NULL;
ssize_t list_size = listxattr(path, NULL, 0);
if (list_size <= 0)
return NULL; /* no xattrs, ENOTSUP, or error: nothing appliable */
char* names = malloc((size_t)list_size);
if (!names)
return NULL;
ssize_t got = listxattr(path, names, (size_t)list_size);
if (got < 0) {
free(names);
return NULL;
}
FileXattrList* list = xattr_list_new();
if (!list) {
free(names);
return NULL;
}
size_t budget = 0;
ssize_t offset = 0;
while (offset < got) {
const char* name = names + offset;
size_t name_len = strlen(name);
if (name_len == 0)
break; /* trailing double NUL not expected; stop */
offset += (ssize_t)name_len + 1;
if (!xattr_name_appliable(name))
continue;
ssize_t value_size = getxattr(path, name, NULL, 0);
if (value_size < 0)
continue;
if (value_size > XATTR_VALUE_MAX)
continue; /* oversize value is refused up front (bounded capture) */
if (name_len + (size_t)value_size > XATTR_TOTAL_MAX - budget)
continue; /* would exceed the per-file budget: skip, keep the rest */
unsigned char* buffer = malloc(value_size > 0 ? (size_t)value_size : 1);
if (!buffer) {
xattr_list_free(list);
free(names);
return NULL;
}
ssize_t read_len = getxattr(path, name, buffer, (size_t)value_size);
if (read_len < 0 || read_len != value_size) {
free(buffer);
continue;
}
if (!xattr_list_append(list, name, buffer, (size_t)value_size)) {
free(buffer);
xattr_list_free(list);
free(names);
return NULL;
}
free(buffer);
budget += name_len + (size_t)value_size;
}
free(names);
if (list->count == 0) {
xattr_list_free(list);
return NULL;
}
return list;
}
/* ---- WIRE ---- */
bool xattr_send(int fd, const FileXattrList* list) {
int count = list ? list->count : 0;
if (!send_int(fd, count))
return false;
for (int i = 0; i < count; i++) {
const FileXattr* xa = &list->items[i];
size_t name_len = strlen(xa->name);
if (name_len > INT32_MAX)
return false;
int32_t name_len32 = (int32_t)name_len;
if (xa->value_len > INT32_MAX)
return false;
int32_t value_len32 = (int32_t)xa->value_len;
if (!send_n_data(fd, &name_len32, sizeof(name_len32)) || !send_n_data(fd, xa->name, name_len) ||
!send_n_data(fd, &value_len32, sizeof(value_len32)) ||
(value_len32 > 0 && !send_n_data(fd, xa->value, (size_t)value_len32)))
return false;
}
return true;
}
FileXattrList* xattr_receive(int fd, int* ok) {
if (ok)
*ok = 0;
int count;
if (!receive_int(fd, &count))
return NULL;
if (count < 0 || count > XATTR_MAX_COUNT) {
log_message(LOG_LEVEL_ERROR, "rejected xattr block: invalid attribute count %d", count);
return NULL;
}
FileXattrList* list = xattr_list_new();
if (!list)
return NULL;
size_t budget = 0;
for (int i = 0; i < count; i++) {
int32_t name_len32;
if (!receive_n_data(fd, &name_len32, sizeof(name_len32))) {
xattr_list_free(list);
return NULL;
}
if (name_len32 <= 0 || name_len32 > XATTR_NAME_MAX) {
log_message(LOG_LEVEL_ERROR, "rejected xattr block: invalid name length %d", name_len32);
xattr_list_free(list);
return NULL;
}
char* name = protocol_alloc((size_t)name_len32 + 1);
if (!name) {
xattr_list_free(list);
return NULL;
}
if (!receive_n_data(fd, name, (size_t)name_len32)) {
free(name);
xattr_list_free(list);
return NULL;
}
name[name_len32] = '\0';
if (memchr(name, '\0', (size_t)name_len32) != NULL) {
/* embedded NUL in the name: malformed, reject */
free(name);
xattr_list_free(list);
return NULL;
}
if (!xattr_name_appliable(name)) {
log_message(LOG_LEVEL_ERROR, "rejected xattr block: disallowed namespace for '%s'", name);
free(name);
xattr_list_free(list);
return NULL;
}
int32_t value_len32;
if (!receive_n_data(fd, &value_len32, sizeof(value_len32))) {
free(name);
xattr_list_free(list);
return NULL;
}
if (value_len32 < 0 || value_len32 > XATTR_VALUE_MAX) {
log_message(LOG_LEVEL_ERROR, "rejected xattr block: invalid value length %d for '%s'",
value_len32, name);
free(name);
xattr_list_free(list);
return NULL;
}
if ((size_t)name_len32 + (size_t)value_len32 > XATTR_TOTAL_MAX - budget) {
log_message(LOG_LEVEL_ERROR, "rejected xattr block: total size budget exceeded for '%s'",
name);
free(name);
xattr_list_free(list);
return NULL;
}
unsigned char* value = NULL;
if (value_len32 > 0) {
value = protocol_alloc((size_t)value_len32);
if (!value) {
free(name);
xattr_list_free(list);
return NULL;
}
if (!receive_n_data(fd, value, (size_t)value_len32)) {
free(value);
free(name);
xattr_list_free(list);
return NULL;
}
}
if (!xattr_list_append(list, name, value, (size_t)value_len32)) {
free(value);
free(name);
xattr_list_free(list);
return NULL;
}
free(value);
free(name);
budget += (size_t)name_len32 + (size_t)value_len32;
}
if (ok)
*ok = 1;
return list;
}
/* ---- RECEIVER: apply (fd-relative, best-effort) ---- */
bool xattr_apply_fd(int fd, const FileXattrList* list) {
if (fd < 0 || !list)
return false;
bool warned = false;
int first_errno = 0;
for (int i = 0; i < list->count; i++) {
const FileXattr* xa = &list->items[i];
/* Defense in depth: even a hand-crafted list can never apply the reserved
--fake-super key (only fake_super_store_fd may write it). */
if (strcmp(xa->name, FAKESUPER_XATTR) == 0)
continue;
if (fsetxattr(fd, xa->name, xa->value, xa->value_len, 0) != 0) {
if (!warned) {
warned = true;
first_errno = errno;
}
}
}
/* Collapse potentially many per-attribute failures into one per-file warning
so a run with many unsettable attributes does not spam the log. */
if (warned)
log_message(LOG_LEVEL_WARNING, "could not set one or more xattrs on the destination file: %s",
strerror(first_errno));
return true;
}
/* ---- --fake-super: park ownership/mode/mtime in a reserved xattr ---- */
void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, int64_t mtime_sec,
int64_t mtime_nsec) {
if (fd < 0)
return;
char record[128];
int len =
snprintf(record, sizeof(record), "%lu:%lu:%03o:%lld:%ld", (unsigned long)uid,
(unsigned long)gid, (unsigned)mode & 0777U, (long long)mtime_sec, (long)mtime_nsec);
if (len <= 0 || (size_t)len >= sizeof(record))
return;
if (fsetxattr(fd, FAKESUPER_XATTR, record, (size_t)len, 0) != 0) {
log_message(LOG_LEVEL_WARNING, "--fake-super: could not store %s on destination file: %s",
FAKESUPER_XATTR, strerror(errno));
}
}
+89
View File
@@ -0,0 +1,89 @@
#ifndef XATTR_H
#define XATTR_H
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
/*
* Portable extended-attribute (xattr) and POSIX-ACL preservation (Phase 4,
* protocol 2.13.0). --xattrs/-X and --acls/-A are implemented on top of the
* xattr machinery: the SENDER captures a bounded, namespace-whitelisted set of
* `name = value` pairs per file, transmits them in a per-file wire block, and
* the RECEIVER re-applies them fd-relative on the just-written file. Linux
* xattr syscalls are used; libacl is NOT required (ACLs travel as the
* system.posix_acl_access / system.posix_acl_default xattrs).
*
* Security model:
* * A client can never force a `security.*` / privileged xattr onto the
* destination: both capture (sender) and apply (receiver) are restricted to
* the unprivileged `user.*` namespace and the two POSIX ACL xattrs. The
* receiver independently re-validates every incoming name against this
* whitelist, so a malicious sender's `security.capability` payload is
* rejected, not applied.
* * Payloads are bounded (per-name length, per-value length, per-file count
* and total bytes) on BOTH ends to prevent OOM/memory abuse; an oversized
* or malformed frame is a clean protocol rejection, never an allocation
* blowup.
* * Application is confined to the exact destination file descriptor
* (fsetxattr on the just-written fd), never a caller-controlled path.
*/
/* Reserved key used by --fake-super to park the source's privileged ownership
* / mode / mtime on the destination file as an unprivileged user.* xattr, so a
* later privileged restore could re-apply them. Exact documented format:
* uid:gid:mode:mtime_sec:mtime_nsec (decimal, decimal, octal, dec, dec)
* e.g. "1000:1000:644:1765238400:0". */
#define FAKESUPER_XATTR "user.fastsync.stat"
/* --- bounds --- */
#define XATTR_NAME_MAX 255 /* xattr names are limited to 255 bytes */
#define XATTR_VALUE_MAX (1024 * 1024) /* per-value cap (1 MiB) */
#define XATTR_TOTAL_MAX (4 * 1024 * 1024) /* per-file total name+value bytes */
#define XATTR_MAX_COUNT 256
typedef struct {
char* name; /* owned, NUL-terminated */
unsigned char* value; /* owned, may hold embedded NULs */
size_t value_len;
} FileXattr;
typedef struct {
FileXattr* items;
int count;
} FileXattrList;
FileXattrList* xattr_list_new(void);
void xattr_list_free(FileXattrList* list);
/* Append one entry (deep copy). Returns false on allocation failure. */
bool xattr_list_append(FileXattrList* list, const char* name, const void* value, size_t value_len);
/* True when `name` is a well-formed xattr name AND belongs to a namespace this
* build is authorized to apply (user.* or the two POSIX ACL xattrs). Used for
* both capture and receiver-side validation. */
bool xattr_name_appliable(const char* name);
/* Sender: read the whitelisted xattrs of `path` into a new list. Returns NULL
* when the path has no appliable xattrs (or the filesystem has no xattr
* support); an empty-but-valid list is never returned distinct from NULL. */
FileXattrList* xattr_capture_path(const char* path);
/* Wire: bounded serialization. xattr_send returns false on write failure; an
* empty/NULL list transmits a zero-count block. xattr_receive returns NULL and
* sets *ok = 0 on any malformed / oversized / non-whitelisted entry. */
bool xattr_send(int fd, const FileXattrList* list);
FileXattrList* xattr_receive(int fd, int* ok);
/* Receiver: apply every entry fd-relative (fsetxattr) to the just-written file
* descriptor. A per-attribute failure (e.g. ACL set refused for non-root on a
* file the process does not own) is logged and skipped, never fatal. Returns
* true when apply was attempted (allowing callers to treat it as best-effort). */
bool xattr_apply_fd(int fd, const FileXattrList* list);
/* --fake-super: write the source uid/gid/mode/mtime record into the reserved
* FAKESUPER_XATTR on `fd`. Best-effort (logged, never fatal). Only meaningful
* when metadata was transmitted so the values exist. */
void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, int64_t mtime_sec,
int64_t mtime_nsec);
#endif
+476
View File
@@ -3,6 +3,7 @@ import filecmp
import os import os
import random import random
import shutil import shutil
import stat
import subprocess import subprocess
import sys import sys
import time import time
@@ -18,6 +19,159 @@ from common import (
SOURCE_DIR = os.path.join(TEST_DATA_DIR, "feature_source") SOURCE_DIR = os.path.join(TEST_DATA_DIR, "feature_source")
DEST_DIR = os.path.join(TEST_DATA_DIR, "feature_dest") DEST_DIR = os.path.join(TEST_DATA_DIR, "feature_dest")
DEVICE_SOURCE = os.path.join(TEST_DATA_DIR, "device_source")
DEVICE_DEST = os.path.join(TEST_DATA_DIR, "device_dest")
class TestDeviceSpecial:
"""Phase 4: --devices / --specials / -D / --copy-devices / --write-devices.
Device node CREATION (mknod) is privileged (CAP_MKNOD); CI runs non-root, so
only the FIFO path (mkfifo, unprivileged) is asserted unconditionally. The
real-device-created assertions are guarded to run only as root. Everything
else must simply succeed / skip without aborting.
"""
def _setup(self):
clean_dir(DEVICE_SOURCE)
clean_dir(DEVICE_DEST)
with open(os.path.join(DEVICE_SOURCE, "plain.txt"), "wb") as f:
f.write(b"regular content\n")
def test_specials_recreates_fifo(self, shared_server):
self._setup()
os.mkfifo(os.path.join(DEVICE_SOURCE, "pipe.fifo"))
result, _ = run_client(DEVICE_SOURCE, DEVICE_DEST,
flags=["--specials"], port=shared_server.port)
assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr[:200]}"
received = get_dest_received_dir(DEVICE_DEST, DEVICE_SOURCE)
fifo = os.path.join(received, "pipe.fifo")
assert os.path.exists(fifo) and stat.S_ISFIFO(os.stat(fifo).st_mode), (
"source FIFO was not recreated as a FIFO on the destination"
)
# The regular file alongside it still transferred normally.
with open(os.path.join(received, "plain.txt")) as f:
assert f.read() == "regular content\n"
def test_D_implies_devices_and_specials_fifo(self, shared_server):
"""-D implies --devices --specials; a FIFO is preserved without a crash
even though no device mknod is attempted on the (non-root) receiver."""
self._setup()
os.mkfifo(os.path.join(DEVICE_SOURCE, "pipe.fifo"))
result, _ = run_client(DEVICE_SOURCE, DEVICE_DEST,
flags=["-D"], port=shared_server.port)
assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr[:200]}"
received = get_dest_received_dir(DEVICE_DEST, DEVICE_SOURCE)
assert stat.S_ISFIFO(os.stat(os.path.join(received, "pipe.fifo")).st_mode)
def test_copy_devices_non_crash(self, shared_server):
"""--copy-devices treats a special/device source as a regular-file copy;
a FIFO (st_size 0) must transfer without hanging or crashing."""
self._setup()
os.mkfifo(os.path.join(DEVICE_SOURCE, "device_copy.fifo"))
result, _ = run_client(DEVICE_SOURCE, DEVICE_DEST,
flags=["--copy-devices"], port=shared_server.port)
assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr[:200]}"
def test_write_devices_non_crash(self, shared_server):
"""--write-devices writes into an existing device only; when the
destination holds no device node the entry is skipped safely and the
run still succeeds (never aborts)."""
self._setup()
# Destination already holds a regular file at the source FIFO's path:
# the receiver must not clobber it and must not crash.
os.mkfifo(os.path.join(DEVICE_SOURCE, "target.fifo"))
result, _ = run_client(DEVICE_SOURCE, DEVICE_DEST,
flags=["--write-devices"], port=shared_server.port)
assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr[:200]}"
@pytest.mark.skipif(os.geteuid() != 0, reason="requires root to create device nodes")
def test_devices_recreates_real_char_device(self, shared_server):
"""Root-only: a source char device node is recreated on the destination
with the same type and rdev (privilege-gated mknod path)."""
self._setup()
src_dev = os.path.join(DEVICE_SOURCE, "realdev")
os.mknod(src_dev, stat.S_IFCHR | 0o666, os.makedev(1, 3))
result, _ = run_client(DEVICE_SOURCE, DEVICE_DEST,
flags=["--devices"], port=shared_server.port)
assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr[:200]}"
received = get_dest_received_dir(DEVICE_DEST, DEVICE_SOURCE)
st = os.lstat(os.path.join(received, "realdev"))
assert stat.S_ISCHR(st.st_mode)
assert os.major(st.st_rdev) == 1 and os.minor(st.st_rdev) == 3
def test_m_remove_source_files_keeps_recreated_fifo(self, shared_server):
"""-m --remove-source-files --specials: a recreated FIFO must NOT be
acknowledged as a removable source (its outcome must not shift the
per-file status stream, which would break the run and mis-remove the
adjacent regular file). The regular file is removed; the FIFO stays."""
self._setup()
os.mkfifo(os.path.join(DEVICE_SOURCE, "pipe.fifo"))
result, _ = run_client(DEVICE_SOURCE, DEVICE_DEST,
flags=["-m", "--remove-source-files", "--specials"],
port=shared_server.port)
assert result.returncode == 0, (
f"Exit {result.returncode}: {result.stderr[:300]}"
)
assert not os.path.exists(os.path.join(DEVICE_SOURCE, "plain.txt")), (
"regular source file should have been removed"
)
assert os.path.exists(os.path.join(DEVICE_SOURCE, "pipe.fifo")), (
"recreated FIFO source must never be removed"
)
@pytest.mark.skipif(os.geteuid() != 0, reason="requires root to create device nodes")
def test_m_remove_source_files_keeps_recreated_device(self, shared_server):
"""Root-only: -m --remove-source-files --devices must not remove a
source device node the receiver recreated (mirrors the single-threaded
behavior; the special is never acknowledged as a removable source)."""
self._setup()
src_dev = os.path.join(DEVICE_SOURCE, "realdev")
os.mknod(src_dev, stat.S_IFCHR | 0o666, os.makedev(1, 3))
result, _ = run_client(DEVICE_SOURCE, DEVICE_DEST,
flags=["-m", "--remove-source-files", "--devices"],
port=shared_server.port)
assert result.returncode == 0, (
f"Exit {result.returncode}: {result.stderr[:300]}"
)
assert not os.path.exists(os.path.join(DEVICE_SOURCE, "plain.txt")), (
"regular source file should have been removed"
)
assert os.path.exists(src_dev) and stat.S_ISCHR(os.lstat(src_dev).st_mode), (
"recreated device source must never be removed"
)
def test_write_devices_fifo_target_skips_not_hangs(self, shared_server):
"""--write-devices must never block on a pre-existing FIFO at the
destination mirror: opening with O_NONBLOCK fails with ENXIO and the
entry is skipped (the FIFO is left untouched and the run succeeds)."""
self._setup()
# Pre-plant a FIFO at the destination mirror of the source file's path.
received = get_dest_received_dir(DEVICE_DEST, DEVICE_SOURCE)
os.makedirs(received, exist_ok=True)
target = os.path.join(received, "plain.txt")
os.mkfifo(target)
result, dur = run_client(DEVICE_SOURCE, DEVICE_DEST,
flags=["--write-devices"], port=shared_server.port)
assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr[:200]}"
assert stat.S_ISFIFO(os.lstat(target).st_mode), "FIFO target was clobbered"
assert dur < 60, "write-devices hung on a FIFO target"
def test_special_confined_to_receive_root(self, shared_server):
"""A special node is created only under the receive root; nothing is
ever materialized outside it (the receiver is confined to its
authorized root)."""
self._setup()
os.mkfifo(os.path.join(DEVICE_SOURCE, "confined.fifo"))
result, _ = run_client(DEVICE_SOURCE, DEVICE_DEST,
flags=["--specials"], port=shared_server.port)
assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr[:200]}"
# The only new FIFO is under the receive tree; its sibling watchers
# confirm the confined dest layout (no stray node at the source root).
source_fifo_escaped = os.path.join(DEVICE_DEST, "confined.fifo")
assert not os.path.lexists(source_fifo_escaped), "special escaped the receive root"
received = get_dest_received_dir(DEVICE_DEST, DEVICE_SOURCE)
assert stat.S_ISFIFO(os.stat(os.path.join(received, "confined.fifo")).st_mode)
@pytest.fixture(scope="module", autouse=True) @pytest.fixture(scope="module", autouse=True)
@@ -4128,3 +4282,325 @@ class TestOmitTimes:
received = get_dest_received_dir(dest, source) received = get_dest_received_dir(dest, source)
mismatches, missing = verify_transfer(source, received) mismatches, missing = verify_transfer(source, received)
assert not missing and not mismatches, f"missing={missing} mismatches={mismatches}" assert not missing and not mismatches, f"missing={missing} mismatches={mismatches}"
class TestSymlinkTrust:
"""Phase-4 symlink trust boundaries: -k/--copy-dirlinks, -K/--keep-dirlinks
and --munge-links. Destination paths mirror the absolute source path below
the destination root (run_client uses absolute --source-dir/--dest-dir)."""
def test_copy_dirlinks_dereferences_dir_symlink(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "symlink_trust_copy_dirlinks")
dest = os.path.join(TEST_DATA_DIR, "symlink_trust_copy_dirlinks_dst")
clean_dir(source)
clean_dir(dest)
os.makedirs(os.path.join(source, "realdir"))
with open(os.path.join(source, "realfile.txt"), "wb") as f:
f.write(b"real file\n")
with open(os.path.join(source, "realdir", "inside.txt"), "wb") as f:
f.write(b"inside dir\n")
os.symlink("realfile.txt", os.path.join(source, "link_file"))
os.symlink("realdir", os.path.join(source, "link_dir"))
result, _ = run_client(source, dest, flags=["-k"], port=shared_server.port)
assert result.returncode == 0, f"-k failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
# link -> realdir dereferences into a real directory tree...
link_dir = os.path.join(received, "link_dir")
assert os.path.isdir(link_dir)
assert not os.path.islink(link_dir)
assert os.path.isfile(os.path.join(link_dir, "inside.txt"))
# ... while a symlink to a regular file stays a symlink.
link_file = os.path.join(received, "link_file")
assert os.path.islink(link_file)
assert os.readlink(link_file) == "realfile.txt"
def test_keep_dirlinks_keeps_dest_symlink_to_dir(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "symlink_trust_keep_dirlinks")
dest = os.path.join(TEST_DATA_DIR, "symlink_trust_keep_dirlinks_dst")
clean_dir(source)
clean_dir(dest)
os.makedirs(os.path.join(source, "sub"))
with open(os.path.join(source, "sub", "file.txt"), "wb") as f:
f.write(b"under the kept symlinked dir\n")
# Plant the destination's symlink-to-directory at the exact mirror path:
# sub -> realdir (relative, both siblings under the mirror parent).
parent = os.path.join(dest, os.path.abspath(source).lstrip(os.sep))
os.makedirs(parent)
os.makedirs(os.path.join(parent, "realdir"))
os.symlink("realdir", os.path.join(parent, "sub"))
result, _ = run_client(source, dest, flags=["-K"], port=shared_server.port)
assert result.returncode == 0, f"-K failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
sub = os.path.join(received, "sub")
# sub stays a symlink to the directory rather than being replaced...
assert os.path.islink(sub)
assert os.readlink(sub) == "realdir"
# ... and the file is written beneath it, through to the referent dir.
assert os.path.isfile(os.path.join(parent, "realdir", "file.txt"))
def test_munge_links_unmunged_target_and_containment(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "symlink_trust_munge")
dest = os.path.join(TEST_DATA_DIR, "symlink_trust_munge_dst")
clean_dir(source)
clean_dir(dest)
with open(os.path.join(source, "a.txt"), "wb") as f:
f.write(b"a\n")
os.symlink("a.txt", os.path.join(source, "good"))
os.symlink("/etc/passwd", os.path.join(source, "abs_escape"))
os.symlink("../../escape", os.path.join(source, "dotdot_escape"))
result, _ = run_client(source, dest, flags=["-l", "--munge-links"],
port=shared_server.port)
assert result.returncode == 0, f"--munge-links failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
# The safe symlink is created with its correct (unmunged) target.
good = os.path.join(received, "good")
assert os.path.islink(good)
assert os.readlink(good) == "a.txt"
# A target that would escape the receive root is contained (skip: never
# transmitted, so nothing is created at the destination).
assert not os.path.lexists(os.path.join(received, "abs_escape"))
assert not os.path.lexists(os.path.join(received, "dotdot_escape"))
assert os.path.isfile(os.path.join(received, "a.txt"))
def test_links_copies_symlinks_as_symlinks(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "symlink_trust_links")
dest = os.path.join(TEST_DATA_DIR, "symlink_trust_links_dst")
clean_dir(source)
clean_dir(dest)
os.makedirs(os.path.join(source, "realdir"))
with open(os.path.join(source, "realfile.txt"), "wb") as f:
f.write(b"real\n")
with open(os.path.join(source, "realdir", "x.txt"), "wb") as f:
f.write(b"x\n")
os.symlink("realfile.txt", os.path.join(source, "lf"))
os.symlink("realdir", os.path.join(source, "ld"))
result, _ = run_client(source, dest, flags=["-l"], port=shared_server.port)
assert result.returncode == 0, f"-l failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
assert os.path.islink(os.path.join(received, "lf"))
assert os.readlink(os.path.join(received, "lf")) == "realfile.txt"
assert os.path.islink(os.path.join(received, "ld"))
assert os.readlink(os.path.join(received, "ld")) == "realdir"
def test_receiver_contains_absolute_target_even_without_munge(self, shared_server):
# The trust boundary is symmetric and enforced receiver-side: a plain -l
# (no --munge-links) run must refuse to materialize an out-of-root
# absolute symlink target, while still copying a legitimate in-root one.
source = os.path.join(TEST_DATA_DIR, "symlink_trust_abs")
dest = os.path.join(TEST_DATA_DIR, "symlink_trust_abs_dst")
clean_dir(source)
clean_dir(dest)
with open(os.path.join(source, "a.txt"), "wb") as f:
f.write(b"a\n")
os.symlink("a.txt", os.path.join(source, "good"))
os.symlink("/etc/passwd", os.path.join(source, "unsafe_abs"))
result, _ = run_client(source, dest, flags=["-l"], port=shared_server.port)
assert result.returncode == 0, f"-l failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
good = os.path.join(received, "good")
assert os.path.islink(good)
assert os.readlink(good) == "a.txt"
# The absolute (non-contained) target was not materialized at the dest.
assert not os.path.lexists(os.path.join(received, "unsafe_abs"))
def test_links_does_not_strip_munge_prefix_without_munge(self, shared_server):
# A source symlink whose target genuinely begins with the #SYMLINK/ marker
# must round-trip verbatim under plain -l: the receiver only unmunges when
# the negotiated --munge-links policy is on, never unconditionally.
source = os.path.join(TEST_DATA_DIR, "symlink_trust_prefix")
dest = os.path.join(TEST_DATA_DIR, "symlink_trust_prefix_dst")
clean_dir(source)
clean_dir(dest)
with open(os.path.join(source, "realfile.txt"), "wb") as f:
f.write(b"real\n")
os.symlink("#SYMLINK/realfile.txt", os.path.join(source, "prefixed"))
result, _ = run_client(source, dest, flags=["-l"], port=shared_server.port)
assert result.returncode == 0, f"-l failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
prefixed = os.path.join(received, "prefixed")
assert os.path.islink(prefixed)
assert os.readlink(prefixed) == "#SYMLINK/realfile.txt"
def _xattr_supported(path):
"""True when the filesystem hosting `path` supports user xattrs."""
try:
os.setxattr(path, "user.fastsync-probe", b"p")
os.removexattr(path, "user.fastsync-probe")
return True
except (OSError, AttributeError):
return False
class TestExtendedAttributes:
"""-X/--xattrs, -A/--acls, --fake-super: portable extended metadata.
Runs unprivileged (CI is non-root). Everything is best-effort and guarded:
a filesystem without xattr support, or an ACL toolchain/POSIX-ACL
filesystem feature that is missing, is skipped rather than failed. The
security boundary (only user.* and the system.posix_acl_* namespaces are
ever applied) is asserted alongside the happy path."""
def _source_and_dest(self, name):
source = os.path.join(TEST_DATA_DIR, name + "_src")
dest = os.path.join(TEST_DATA_DIR, name + "_dst")
clean_dir(source)
clean_dir(dest)
return source, dest
@pytest.mark.ci
def test_xattrs_preserves_user_namespace(self, shared_server):
source, dest = self._source_and_dest("xattr")
f = os.path.join(source, "data.txt")
with open(f, "wb") as fh:
fh.write(b"xattr payload\n")
if not _xattr_supported(f):
pytest.skip("filesystem does not support user xattrs")
os.setxattr(f, "user.foo", b"preserved-value")
result, _ = run_client(source, dest, flags=["-X"], port=shared_server.port)
assert result.returncode == 0, \
f"-X sync failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
assert os.getxattr(os.path.join(received, "data.txt"), "user.foo") == b"preserved-value"
def test_without_xattrs_does_not_carry(self, shared_server):
source, dest = self._source_and_dest("xattr_ctrl")
f = os.path.join(source, "data.txt")
with open(f, "wb") as fh:
fh.write(b"plain\n")
if not _xattr_supported(f):
pytest.skip("filesystem does not support user xattrs")
os.setxattr(f, "user.foo", b"must-not-travel")
result, _ = run_client(source, dest, port=shared_server.port)
assert result.returncode == 0, \
f"control sync failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
with pytest.raises(OSError):
os.getxattr(os.path.join(received, "data.txt"), "user.foo")
def test_reserved_fake_super_key_not_forwarded(self, shared_server):
"""A source file that already carries the reserved user.fastsync.stat
record must NOT have it planted on the receiver during a plain -X run
(it is receiver-only, so it cannot be spoofed for a later privileged
restore)."""
source, dest = self._source_and_dest("xattr_reserved")
f = os.path.join(source, "data.txt")
with open(f, "wb") as fh:
fh.write(b"reserved\n")
if not _xattr_supported(f):
pytest.skip("filesystem does not support user xattrs")
os.setxattr(f, "user.fastsync.stat", b"0:0:644:0:0")
# A normal user.* attr still travels alongside.
os.setxattr(f, "user.keep", b"yes")
result, _ = run_client(source, dest, flags=["-X"], port=shared_server.port)
assert result.returncode == 0, \
f"-X reserved-key sync failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
assert os.getxattr(os.path.join(received, "data.txt"), "user.keep") == b"yes"
with pytest.raises(OSError):
os.getxattr(os.path.join(received, "data.txt"), "user.fastsync.stat")
@pytest.mark.ci
def test_xattrs_multithreaded(self, shared_server):
source, dest = self._source_and_dest("xattr_mt")
f = os.path.join(source, "data.txt")
with open(f, "wb") as fh:
fh.write(b"mt xattr\n")
if not _xattr_supported(f):
pytest.skip("filesystem does not support user xattrs")
os.setxattr(f, "user.k", b"v")
result, _ = run_client(source, dest, flags=["-X", "-m"], port=shared_server.port)
assert result.returncode == 0, \
f"-X -m sync failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
assert os.getxattr(os.path.join(received, "data.txt"), "user.k") == b"v"
@pytest.mark.ci
def test_acls_via_posix_acl_xattr(self, shared_server):
source, dest = self._source_and_dest("acl")
f = os.path.join(source, "data.txt")
with open(f, "wb") as fh:
fh.write(b"acl payload\n")
if not _xattr_supported(f):
pytest.skip("filesystem does not support xattrs")
acl_blob = None
if shutil.which("setfacl") is not None:
acl = subprocess.run(["setfacl", "-m", "o::r", f], capture_output=True, text=True)
if acl.returncode == 0:
try:
acl_blob = os.getxattr(f, "system.posix_acl_access")
except OSError:
acl_blob = None
if acl_blob is None:
# No setfacl (common in the minimal CI image): synthesize a valid
# non-trivial POSIX ACL ("u:current-uid:r") xattr blob directly.
import struct
try:
uid_for_acl = os.geteuid() if os.geteuid() != 0 else 65534
struct_entry = struct.pack("<HHI", 0x01, 0x4, 0xFFFFFFFF) # USER_OBJ r
struct_entry += struct.pack("<HHI", 0x02, 0x4, uid_for_acl) # USER r
struct_entry += struct.pack("<HHI", 0x04, 0x4, 0xFFFFFFFF) # GROUP_OBJ r
struct_entry += struct.pack("<HHI", 0x10, 0x4, 0xFFFFFFFF) # MASK r
struct_entry += struct.pack("<HHI", 0x20, 0x0, 0xFFFFFFFF) # OTHER ---
blob = struct.pack("<I", 2) + struct_entry
os.setxattr(f, "system.posix_acl_access", blob)
acl_blob = os.getxattr(f, "system.posix_acl_access")
except (OSError, struct.error) as e:
pytest.skip(f"cannot set a POSIX ACL unprivileged: {e}")
result, _ = run_client(source, dest, flags=["-A"], port=shared_server.port)
assert result.returncode == 0, \
f"-A sync failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
assert os.getxattr(os.path.join(received, "data.txt"),
"system.posix_acl_access") == acl_blob
@pytest.mark.ci
def test_acls_imply_xattr_transport(self, shared_server):
"""-A and -X enable the shared xattr transport; both attributes travel
together, and a security.* attribute a malicious peer would send is
never applied (receiver whitelist)."""
source, dest = self._source_and_dest("acl_xattr")
f = os.path.join(source, "data.txt")
with open(f, "wb") as fh:
fh.write(b"combined\n")
if not _xattr_supported(f):
pytest.skip("filesystem does not support xattrs")
os.setxattr(f, "user.for-acl-flag", b"yes")
result, _ = run_client(source, dest, flags=["-A", "-X"], port=shared_server.port)
assert result.returncode == 0, \
f"-A -X sync failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
assert os.getxattr(os.path.join(received, "data.txt"), "user.for-acl-flag") == b"yes"
@pytest.mark.ci
def test_fake_super_stores_source_stat(self, shared_server):
source, dest = self._source_and_dest("fakesuper")
f = os.path.join(source, "data.txt")
with open(f, "wb") as fh:
fh.write(b"fake-super\n")
if not _xattr_supported(f):
pytest.skip("filesystem does not support xattrs")
uid = os.stat(f).st_uid
result, _ = run_client(source, dest, flags=["--fake-super"], port=shared_server.port)
assert result.returncode == 0, \
f"--fake-super sync failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
record = os.getxattr(os.path.join(received, "data.txt"), "user.fastsync.stat").decode()
fields = record.split(":")
assert len(fields) == 5
assert fields[0] == str(uid), f"reserved uid field {fields[0]} != source uid {uid}"
+2
View File
@@ -27,6 +27,7 @@
#include "test_transport_ssh.h" #include "test_transport_ssh.h"
#include "test_transport_tls.h" #include "test_transport_tls.h"
#include "test_utils.h" #include "test_utils.h"
#include "test_xattr.h"
#include <stdio.h> #include <stdio.h>
#include <signal.h> #include <signal.h>
@@ -67,6 +68,7 @@ int main() {
RUN_TEST(test_client_cli); RUN_TEST(test_client_cli);
RUN_TEST(test_server); RUN_TEST(test_server);
RUN_TEST(test_fuzz_smoke); RUN_TEST(test_fuzz_smoke);
RUN_TEST(test_xattr);
printf("\n\033[1;36m=== TEST SUMMARY ===\033[0m\n"); printf("\n\033[1;36m=== TEST SUMMARY ===\033[0m\n");
printf("Total Tests Run: %d\n", tests_run); printf("Total Tests Run: %d\n", tests_run);
+124
View File
@@ -159,8 +159,132 @@ static void test_chunk_dir_entry_roundtrip() {
rmdir(dir_path); rmdir(dir_path);
} }
static void test_chunk_symlink_roundtrip() {
const char* file_path = "temp_chunk_symlink_file.txt";
const char* link_path = "temp_chunk_symlink";
const char* content = "regular payload";
const char* target = "temp_chunk_symlink_file.txt";
rmdir(link_path);
unlink(file_path);
file_write_to_disk(file_path, content, strlen(content), false, false);
for (int use_metadata = 0; use_metadata <= 1; use_metadata++) {
struct stat st;
EXPECT_EQ_INT(stat(file_path, &st), 0);
File* reg = file_create(file_path);
EXPECT_NOT_NULL(reg);
reg->data->size = (unsigned long long)st.st_size;
EXPECT_TRUE(file_load_data(reg));
File* link = file_create(link_path);
EXPECT_NOT_NULL(link);
link->is_symlink = true;
link->symlink_target = str_dup(target);
EXPECT_NOT_NULL(link->symlink_target);
if (use_metadata) {
reg->metadata = file_metadata_create(file_path, &st, false, false);
EXPECT_NOT_NULL(reg->metadata);
link->metadata = file_metadata_create(file_path, &st, false, false);
EXPECT_NOT_NULL(link->metadata);
}
File* files[2] = {reg, link};
Chunk* chunk = chunk_create(files, 2);
EXPECT_NOT_NULL(chunk);
Data* serialized = chunk_serialize(chunk, use_metadata != 0);
EXPECT_NOT_NULL(serialized);
Chunk* deserialized = chunk_deserialize(serialized, use_metadata != 0);
EXPECT_NOT_NULL(deserialized);
EXPECT_EQ_INT(deserialized->element_count, 2);
EXPECT_FALSE(deserialized->items[0]->is_symlink);
EXPECT_TRUE(deserialized->items[1]->is_symlink);
EXPECT_NULL(deserialized->items[0]->symlink_target);
EXPECT_EQ_STR(deserialized->items[1]->symlink_target, target);
EXPECT_EQ_INT((int)deserialized->items[1]->data->size, 0);
data_destroy(serialized);
chunk_destroy(deserialized);
chunk_destroy(chunk); /* frees reg and link */
}
unlink(file_path);
rmdir(link_path);
}
/* A --devices/--specials special entry (is_special + rdev) must round-trip
* through the chunk wire with a legal rdev. */
static void test_chunk_special_rdev_roundtrip() {
const char* path = "temp_chunk_special_node";
unlink(path);
File* special = file_create(path);
EXPECT_NOT_NULL(special);
special->is_special = true;
special->rdev_major = 1;
special->rdev_minor = 3;
struct stat st;
EXPECT_EQ_INT(stat("/dev/null", &st), 0);
special->metadata = file_metadata_create(path, &st, false, false);
EXPECT_NOT_NULL(special->metadata);
File* files[1] = {special};
Chunk* chunk = chunk_create(files, 1);
EXPECT_NOT_NULL(chunk);
Data* serialized = chunk_serialize(chunk, true);
EXPECT_NOT_NULL(serialized);
Chunk* deserialized = chunk_deserialize(serialized, true);
EXPECT_NOT_NULL(deserialized);
EXPECT_EQ_INT(deserialized->element_count, 1);
EXPECT_TRUE(deserialized->items[0]->is_special);
EXPECT_FALSE(deserialized->items[0]->is_dir);
EXPECT_EQ_INT((int)deserialized->items[0]->data->size, 0);
EXPECT_EQ_INT(deserialized->items[0]->rdev_major, 1);
EXPECT_EQ_INT(deserialized->items[0]->rdev_minor, 3);
EXPECT_NOT_NULL(deserialized->items[0]->metadata);
data_destroy(serialized);
chunk_destroy(deserialized);
chunk_destroy(chunk);
}
/* A special entry carrying an out-of-range rdev is a malformed chunk and must be
* rejected at deserialize (bounded by the same 0xffff / 0x00ffffff limits
* file_special_rdev_valid uses on the per-file wire), not deferred to the
* creation site. */
static void test_chunk_special_rdev_out_of_range_rejected() {
const char* path = "temp_chunk_special_bad_rdev";
unlink(path);
File* special = file_create(path);
EXPECT_NOT_NULL(special);
special->is_special = true;
special->rdev_major = 0x10000; /* > 0xffff */
special->rdev_minor = 3;
struct stat st;
EXPECT_EQ_INT(stat("/dev/null", &st), 0);
special->metadata = file_metadata_create(path, &st, false, false);
EXPECT_NOT_NULL(special->metadata);
File* files[1] = {special};
Chunk* chunk = chunk_create(files, 1);
EXPECT_NOT_NULL(chunk);
Data* serialized = chunk_serialize(chunk, true);
EXPECT_NOT_NULL(serialized);
Chunk* deserialized = chunk_deserialize(serialized, true);
EXPECT_NULL(deserialized);
data_destroy(serialized);
chunk_destroy(chunk);
}
void test_chunk() { void test_chunk() {
test_file_operations(); test_file_operations();
test_chunk_operations(); test_chunk_operations();
test_chunk_dir_entry_roundtrip(); test_chunk_dir_entry_roundtrip();
test_chunk_symlink_roundtrip();
test_chunk_special_rdev_roundtrip();
test_chunk_special_rdev_out_of_range_rejected();
} }
+138
View File
@@ -210,6 +210,60 @@ static void test_parse_args_version() {
config_delete(cfg); config_delete(cfg);
} }
/* --xattrs/-X and --acls/-A preserve per-file xattrs and both imply metadata
* transmission (the xattr block rides the metadata/per-file frame); each is
* individually negatable and the derived use_xattrs follows the flags. */
static void test_parse_args_xattrs_acls() {
Config* cfg = config_create();
char* argv[] = {"fastsync", "-X", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->preserve_xattrs);
EXPECT_FALSE(cfg->preserve_acls);
EXPECT_TRUE(cfg->use_xattrs);
EXPECT_TRUE(cfg->use_metadata);
config_delete(cfg);
cfg = config_create();
positional_count = 0;
char* argv_long[] = {"fastsync", "--acls", "/src", "/dst"};
EXPECT_EQ_INT(parse_args(cfg, 4, argv_long, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->preserve_acls);
EXPECT_TRUE(cfg->use_xattrs);
EXPECT_TRUE(cfg->use_metadata);
config_delete(cfg);
cfg = config_create();
positional_count = 0;
char* argv_neg[] = {"fastsync", "-X", "-A", "--no-xattrs", "/src", "/dst"};
EXPECT_EQ_INT(parse_args(cfg, 6, argv_neg, positional_args, &positional_count), 0);
EXPECT_FALSE(cfg->preserve_xattrs);
EXPECT_TRUE(cfg->preserve_acls);
EXPECT_TRUE(cfg->use_xattrs);
config_delete(cfg);
}
/* --fake-super is a receiver-side preference that parks the source
* uid/gid/mode/mtime in a reserved xattr; it implies metadata transmission. */
static void test_parse_args_fake_super() {
Config* cfg = config_create();
char* argv[] = {"fastsync", "--fake-super", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->fake_super);
EXPECT_TRUE(cfg->use_metadata);
config_delete(cfg);
cfg = config_create();
positional_count = 0;
char* argv_neg[] = {"fastsync", "--fake-super", "--no-fake-super", "/src", "/dst"};
EXPECT_EQ_INT(parse_args(cfg, 5, argv_neg, positional_args, &positional_count), 0);
EXPECT_FALSE(cfg->fake_super);
config_delete(cfg);
}
/* Test parse_args with valid port */ /* Test parse_args with valid port */
static void test_parse_args_valid_port() { static void test_parse_args_valid_port() {
Config* cfg = config_create(); Config* cfg = config_create();
@@ -1219,6 +1273,40 @@ static void test_parse_args_short_s_remains_chunk_serialization() {
config_delete(cfg); config_delete(cfg);
} }
/* Phase 4 symlink-trust flags: -k/--copy-dirlinks, -K/--keep-dirlinks and
--munge-links must parse into their Config fields. */
static void test_parse_args_symlink_trust() {
Config* cfg = config_create();
char* argv[] = {"fastsync", "-k", "-K", "--munge-links", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 6, argv, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->copy_dirlinks);
EXPECT_TRUE(cfg->keep_dirlinks);
EXPECT_TRUE(cfg->munge_links);
config_delete(cfg);
cfg = config_create();
char* long_argv[] = {"fastsync", "--copy-dirlinks", "--keep-dirlinks", "/src", "/dst"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, long_argv, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->copy_dirlinks);
EXPECT_TRUE(cfg->keep_dirlinks);
EXPECT_FALSE(cfg->munge_links);
config_delete(cfg);
/* Without any of the flags they stay off (additive, opt-in). */
cfg = config_create();
char* plain_argv[] = {"fastsync", "/src", "/dst"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 3, plain_argv, positional_args, &positional_count), 0);
EXPECT_FALSE(cfg->copy_dirlinks);
EXPECT_FALSE(cfg->keep_dirlinks);
EXPECT_FALSE(cfg->munge_links);
config_delete(cfg);
}
static void test_parse_args_8_bit_output() { static void test_parse_args_8_bit_output() {
Config* cfg = config_create(); Config* cfg = config_create();
char* long_argv[] = {"fastsync", "--8-bit-output", "/src", "/dst"}; char* long_argv[] = {"fastsync", "--8-bit-output", "/src", "/dst"};
@@ -2346,6 +2434,52 @@ static void test_parse_args_omit_link_times_long() {
config_delete(cfg); config_delete(cfg);
} }
/* --devices / --specials / -D / --copy-devices / --write-devices parse into the
config, and the preserved flags imply metadata transmission. */
static void test_parse_args_devices_specials() {
Config* cfg = config_create();
char* argv[] = {"fastsync", "--devices", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->preserve_devices);
EXPECT_FALSE(cfg->preserve_specials);
EXPECT_TRUE(cfg->use_metadata);
config_delete(cfg);
cfg = config_create();
char* argv2[] = {"fastsync", "--specials", "/src", "/dst"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv2, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->preserve_specials);
EXPECT_FALSE(cfg->preserve_devices);
EXPECT_TRUE(cfg->use_metadata);
config_delete(cfg);
cfg = config_create();
char* argv3[] = {"fastsync", "-D", "/src", "/dst"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv3, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->preserve_devices);
EXPECT_TRUE(cfg->preserve_specials);
EXPECT_TRUE(cfg->use_metadata);
config_delete(cfg);
cfg = config_create();
char* argv4[] = {"fastsync", "--copy-devices", "/src", "/dst"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv4, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->copy_devices);
config_delete(cfg);
cfg = config_create();
char* argv5[] = {"fastsync", "--write-devices", "/src", "/dst"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv5, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->write_devices);
config_delete(cfg);
}
void test_client_cli() { void test_client_cli() {
test_validate_config_required_paths(); test_validate_config_required_paths();
test_parse_args_numeric_ids(); test_parse_args_numeric_ids();
@@ -2356,6 +2490,7 @@ void test_client_cli() {
test_parse_args_rejects_malformed_identity(); test_parse_args_rejects_malformed_identity();
test_parse_args_preallocate(); test_parse_args_preallocate();
test_parse_args_metadata_times(); test_parse_args_metadata_times();
test_parse_args_devices_specials();
test_parse_args_atimes_long_and_short(); test_parse_args_atimes_long_and_short();
test_parse_args_omit_link_times_long(); test_parse_args_omit_link_times_long();
test_parse_args_append(); test_parse_args_append();
@@ -2429,6 +2564,7 @@ void test_client_cli() {
test_parse_args_rejects_unsupported_stderr_modes(); test_parse_args_rejects_unsupported_stderr_modes();
test_parse_args_secluded_args(); test_parse_args_secluded_args();
test_parse_args_short_s_remains_chunk_serialization(); test_parse_args_short_s_remains_chunk_serialization();
test_parse_args_symlink_trust();
test_parse_args_whole_file(); test_parse_args_whole_file();
test_parse_args_fuzzy_implies_delta(); test_parse_args_fuzzy_implies_delta();
test_parse_args_fuzzy_negation(); test_parse_args_fuzzy_negation();
@@ -2446,6 +2582,8 @@ void test_client_cli() {
test_parse_args_table_equals_size_options(); test_parse_args_table_equals_size_options();
test_parse_args_table_equals_string_and_int_options(); test_parse_args_table_equals_string_and_int_options();
test_parse_args_missing_argument_diagnostic(); test_parse_args_missing_argument_diagnostic();
test_parse_args_xattrs_acls();
test_parse_args_fake_super();
test_parse_args_partial_progress(); test_parse_args_partial_progress();
test_parse_args_itemize_changes(); test_parse_args_itemize_changes();
test_parse_args_list_only(); test_parse_args_list_only();
+142 -3
View File
@@ -622,9 +622,60 @@ static void test_config_delete_policy_wire_roundtrip() {
} }
} }
/* --delete-missing-args crosses the wire (the receiver executes the exact-path /* Phase 4 symlink-trust wire split: --munge-links and -K/--keep-dirlinks CROSS
deletions) while --ignore-missing-args is client-only: the receiver must the wire (the receiver unmunges targets and follows an in-root dir-link),
observe delete_missing_args unchanged and ignore_missing_args always false. */ while -k/--copy-dirlinks is client/sender-only and must NOT reach the
receiver (it would observe it false). */
static void test_config_symlink_trust_wire_roundtrip() {
if (is_running_under_valgrind())
return;
struct {
bool munge_links, keep_dirlinks, copy_dirlinks;
} cases[] = {
{false, false, false},
{true, false, false},
{false, true, false},
{true, true, true},
};
for (size_t i = 0; i < sizeof(cases) / sizeof(cases[0]); i++) {
int p[2];
EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0);
pid_t pid = fork();
if (pid == 0) {
close(p[1]);
io_set_fds(p[0], p[0]);
Config* recv = config_receive(p[0]);
bool ok = recv != NULL;
if (ok) {
ok = recv->munge_links == cases[i].munge_links &&
recv->keep_dirlinks == cases[i].keep_dirlinks &&
/* copy_dirlinks never crosses the wire. */
recv->copy_dirlinks == false;
}
config_delete(recv);
close(p[0]);
_exit(ok ? 0 : 1);
} else {
close(p[0]);
io_set_fds(p[1], p[1]);
Config* send_cfg = config_create();
EXPECT_NOT_NULL(send_cfg);
send_cfg->send_directory = str_dup("/src");
send_cfg->receive_root_directory = str_dup("/dst");
send_cfg->munge_links = cases[i].munge_links;
send_cfg->keep_dirlinks = cases[i].keep_dirlinks;
send_cfg->copy_dirlinks = cases[i].copy_dirlinks;
bool sent = config_send(p[1], send_cfg);
int status;
waitpid(pid, &status, 0);
close(p[1]);
config_delete(send_cfg);
EXPECT_TRUE(sent);
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
}
}
}
static void test_config_delete_missing_args_wire_roundtrip() { static void test_config_delete_missing_args_wire_roundtrip() {
if (is_running_under_valgrind()) if (is_running_under_valgrind())
return; return;
@@ -1090,6 +1141,91 @@ static void test_config_preallocate_wire_roundtrip() {
} }
} }
} }
static void test_config_devices_wire_roundtrip() {
if (is_running_under_valgrind())
return;
Config* send_cfg = config_create();
EXPECT_NOT_NULL(send_cfg);
send_cfg->send_directory = str_dup("/send/src");
send_cfg->receive_root_directory = str_dup("/send/dst");
send_cfg->preserve_devices = true;
send_cfg->preserve_specials = true;
send_cfg->copy_devices = true;
send_cfg->write_devices = true;
int p[2];
EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0);
io_set_fds(p[0], p[1]);
io_set_bwlimit(0);
pid_t pid = fork();
if (pid == 0) {
close(p[1]);
io_set_fds(p[0], p[0]);
Config* recv = config_receive(p[0]);
bool ok = recv != NULL;
if (ok) {
ok = recv->preserve_devices && recv->preserve_specials && recv->copy_devices &&
recv->write_devices;
}
config_delete(recv);
close(p[0]);
_exit(ok ? 0 : 1);
} else {
close(p[0]);
io_set_fds(p[1], p[1]);
bool sent = config_send(p[1], send_cfg);
int status;
waitpid(pid, &status, 0);
close(p[1]);
config_delete(send_cfg);
EXPECT_TRUE(sent);
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
}
}
/* Phase-4: preserve_xattrs/--acls (in file options) and --fake-super (trailing)
* cross the config wire; the receiver recomputes the derived use_xattrs. */
static void test_config_phase4_xattr_wire_roundtrip() {
if (is_running_under_valgrind())
return;
Config* send_cfg = config_create();
EXPECT_NOT_NULL(send_cfg);
send_cfg->send_directory = str_dup("/send/src");
send_cfg->receive_root_directory = str_dup("/send/dst");
send_cfg->preserve_xattrs = true;
send_cfg->preserve_acls = true;
send_cfg->fake_super = true;
int p[2];
EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0);
io_set_fds(p[0], p[1]);
io_set_bwlimit(0);
pid_t pid = fork();
if (pid == 0) {
close(p[1]);
io_set_fds(p[0], p[0]);
Config* recv = config_receive(p[0]);
bool ok = recv != NULL;
if (ok) {
ok = recv->preserve_xattrs && recv->preserve_acls && recv->fake_super && recv->use_xattrs;
}
config_delete(recv);
close(p[0]);
close(p[1]);
_exit(ok ? 0 : 1);
} else {
close(p[0]);
io_set_fds(p[1], p[1]);
bool sent = config_send(p[1], send_cfg);
int status;
waitpid(pid, &status, 0);
close(p[1]);
config_delete(send_cfg);
EXPECT_TRUE(sent);
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
}
}
void test_config() { void test_config() {
test_config_lifecycle(); test_config_lifecycle();
test_config_ssh_dest(); test_config_ssh_dest();
@@ -1107,6 +1243,7 @@ void test_config() {
test_config_delete_timing_wire_roundtrip(); test_config_delete_timing_wire_roundtrip();
test_config_delete_timing_conflict_rejected(); test_config_delete_timing_conflict_rejected();
test_config_delete_policy_wire_roundtrip(); test_config_delete_policy_wire_roundtrip();
test_config_symlink_trust_wire_roundtrip();
test_config_delete_missing_args_wire_roundtrip(); test_config_delete_missing_args_wire_roundtrip();
test_config_append_wire_roundtrip(); test_config_append_wire_roundtrip();
test_config_basis_roundtrip(); test_config_basis_roundtrip();
@@ -1117,7 +1254,9 @@ void test_config() {
test_config_identity_wire_roundtrip(); test_config_identity_wire_roundtrip();
test_config_receive_rejects_invalid_identity(); test_config_receive_rejects_invalid_identity();
test_config_metadata_times_wire_roundtrip(); test_config_metadata_times_wire_roundtrip();
test_config_devices_wire_roundtrip();
test_config_preallocate_wire_roundtrip(); test_config_preallocate_wire_roundtrip();
test_config_phase4_xattr_wire_roundtrip();
} }
test_config_delete_timing_early_helper(); test_config_delete_timing_early_helper();
test_config_is_remote_dest(); test_config_is_remote_dest();
+67
View File
@@ -25,6 +25,26 @@ static void test_file_create() {
file_destroy(f); file_destroy(f);
} }
/* rdev/type validation shared by the wire path and the secure recreation site:
* a legal char/block major/minor pair is accepted, out-of-range / negative
* values and non-device entries carrying an rdev are rejected. */
static void test_file_special_rdev_valid() {
mode_t fake_char = S_IFCHR | 0600;
mode_t fake_blk = S_IFBLK | 0600;
mode_t fake_fifo = S_IFIFO | 0600;
/* char/block devices: accept a legal pair, reject negative / oversized. */
EXPECT_TRUE(file_special_rdev_valid(1, 3, fake_char));
EXPECT_TRUE(file_special_rdev_valid(0xffff, 0x00ffffff, fake_blk));
EXPECT_FALSE(file_special_rdev_valid(-1, 3, fake_char));
EXPECT_FALSE(file_special_rdev_valid(1, -1, fake_char));
EXPECT_FALSE(file_special_rdev_valid(0x10000, 3, fake_char));
EXPECT_FALSE(file_special_rdev_valid(1, 0x1000000, fake_char));
/* FIFOs/sockets must carry an empty rdev. */
EXPECT_TRUE(file_special_rdev_valid(0, 0, fake_fifo));
EXPECT_FALSE(file_special_rdev_valid(1, 0, fake_fifo));
EXPECT_FALSE(file_special_rdev_valid(0, 0, (mode_t)(S_IFREG | 0600)));
}
static void test_file_destroy_null() { static void test_file_destroy_null() {
file_destroy(NULL); file_destroy(NULL);
} }
@@ -468,6 +488,50 @@ static void test_file_write_to_disk_does_not_follow_symlink() {
unlink(link); unlink(link);
} }
static void test_file_symlink_helpers() {
/* Munge/unmunge round-trip restores the original target. */
char* munged = file_symlink_munge("target.txt");
EXPECT_NOT_NULL(munged);
EXPECT_EQ_INT(memcmp(munged, SYMLINK_MUNGE_PREFIX, strlen(SYMLINK_MUNGE_PREFIX)), 0);
EXPECT_TRUE(file_symlink_unmunge(munged));
EXPECT_EQ_STR(munged, "target.txt");
free(munged);
char noop[] = "plain-target";
EXPECT_FALSE(file_symlink_unmunge(noop));
EXPECT_EQ_STR(noop, "plain-target");
/* Containment: relative targets without ".." are safe; absolute or
".."-escaping targets are not. */
EXPECT_TRUE(file_symlink_target_contained("a.txt"));
EXPECT_TRUE(file_symlink_target_contained("sub/dir/file"));
EXPECT_FALSE(file_symlink_target_contained("/etc/passwd"));
EXPECT_FALSE(file_symlink_target_contained("../escape"));
EXPECT_FALSE(file_symlink_target_contained("a/../b"));
EXPECT_FALSE(file_symlink_target_contained(""));
}
static void test_file_symlink_at_secure() {
const char* link = "test_symlink_at_secure_link";
const char* outside = "test_symlink_at_secure_outside.txt";
unlink(link);
unlink(outside);
EXPECT_TRUE(file_write_to_disk(outside, "out", 3, false, false));
EXPECT_TRUE(file_symlink_at_secure(link, "outside.text"));
struct stat st;
EXPECT_EQ_INT(lstat(link, &st), 0);
EXPECT_TRUE(S_ISLNK(st.st_mode));
/* Replacing an existing non-directory entry is fine. */
EXPECT_TRUE(file_symlink_at_secure(link, "other.txt"));
EXPECT_EQ_INT(lstat(link, &st), 0);
EXPECT_TRUE(S_ISLNK(st.st_mode));
unlink(link);
unlink(outside);
}
static void test_file_content_to_buffer() { static void test_file_content_to_buffer() {
const char* content = "Buffer content test"; const char* content = "Buffer content test";
EXPECT_TRUE(file_write_to_disk("test_buffer_file.txt", content, strlen(content), false, false)); EXPECT_TRUE(file_write_to_disk("test_buffer_file.txt", content, strlen(content), false, false));
@@ -960,6 +1024,7 @@ static void test_dir_entry_save_to_disk() {
void test_file() { void test_file() {
test_file_create(); test_file_create();
test_file_special_rdev_valid();
test_file_destroy_null(); test_file_destroy_null();
test_file_destroy_normal(); test_file_destroy_normal();
test_file_load_data(); test_file_load_data();
@@ -978,6 +1043,8 @@ void test_file() {
test_file_write_to_disk_creates_dirs(); test_file_write_to_disk_creates_dirs();
test_file_write_to_disk_does_not_follow_symlink(); test_file_write_to_disk_does_not_follow_symlink();
test_file_content_to_buffer(); test_file_content_to_buffer();
test_file_symlink_helpers();
test_file_symlink_at_secure();
test_file_save_to_disk_path_traversal(); test_file_save_to_disk_path_traversal();
test_file_save_to_disk_deep_traversal(); test_file_save_to_disk_deep_traversal();
test_dir_entry_save_to_disk(); test_dir_entry_save_to_disk();
+234
View File
@@ -0,0 +1,234 @@
#include "test_xattr.h"
#include "xattr.h"
#include "file.h"
#include "protocol.h"
#include "test_utils.h"
#include <fcntl.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <sys/xattr.h>
#include <unistd.h>
static void run_recv_helper(int fd) {
int ok = 0;
FileXattrList* list = xattr_receive(fd, &ok);
if (!ok)
_exit(1);
if (!list) {
/* NULL list only on error, already handled above. */
_exit(1);
}
if (list->count != 2)
_exit(1);
if (strcmp(list->items[0].name, "user.foo") != 0 || list->items[0].value_len != 3 ||
memcmp(list->items[0].value, "bar", 3) != 0)
_exit(1);
if (strcmp(list->items[1].name, "user.empty") != 0 || list->items[1].value_len != 0)
_exit(1);
xattr_list_free(list);
_exit(0);
}
static void test_xattr_wire_roundtrip() {
/* Round-trip a user.* list incl. an empty value. */
int p[2];
EXPECT_EQ_INT(pipe(p), 0);
io_set_fds(p[0], p[1]);
io_set_bwlimit(0);
pid_t pid = fork();
if (pid == 0) {
close(p[1]);
io_set_fds(p[0], p[0]);
run_recv_helper(p[0]);
}
close(p[0]);
io_set_fds(p[1], p[1]);
FileXattrList* list = xattr_list_new();
EXPECT_NOT_NULL(list);
EXPECT_TRUE(xattr_list_append(list, "user.foo", "bar", 3));
EXPECT_TRUE(xattr_list_append(list, "user.empty", NULL, 0));
EXPECT_TRUE(xattr_send(p[1], list));
xattr_list_free(list);
int status;
waitpid(pid, &status, 0);
close(p[1]);
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
}
static void run_recv_must_fail(int fd) {
int ok = 0;
FileXattrList* list = xattr_receive(fd, &ok);
/* A NULL list with ok==0 is the expected rejection. */
if (ok == 0 && list == NULL)
_exit(0);
xattr_list_free(list);
_exit(1);
}
/* A receiver must reject a security.* (privileged-namespace) attribute, never
* apply it: the send side can be malicious, so only the receiver whitelist
* matters. */
static void test_xattr_reject_privileged_namespace() {
int p[2];
EXPECT_EQ_INT(pipe(p), 0);
io_set_fds(p[0], p[1]);
io_set_bwlimit(0);
pid_t pid = fork();
if (pid == 0) {
close(p[1]);
io_set_fds(p[0], p[0]);
run_recv_must_fail(p[0]);
}
close(p[0]);
io_set_fds(p[1], p[1]);
FileXattrList* list = xattr_list_new();
EXPECT_NOT_NULL(list);
/* security.capability must be rejected by the receiver. */
EXPECT_TRUE(xattr_list_append(list, "security.capability", "\x01\x00", 2));
xattr_send(p[1], list); /* receiver rejects at the name check and exits */
xattr_list_free(list);
int status;
waitpid(pid, &status, 0);
close(p[1]);
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
}
/* An oversized value (beyond XATTR_VALUE_MAX) must be rejected on receive. */
static void test_xattr_reject_oversized_value() {
int p[2];
EXPECT_EQ_INT(pipe(p), 0);
io_set_fds(p[0], p[1]);
io_set_bwlimit(0);
pid_t pid = fork();
if (pid == 0) {
close(p[1]);
io_set_fds(p[0], p[0]);
run_recv_must_fail(p[0]);
}
close(p[0]);
io_set_fds(p[1], p[1]);
FileXattrList* list = xattr_list_new();
EXPECT_NOT_NULL(list);
size_t huge = (size_t)XATTR_VALUE_MAX + 1;
unsigned char* blob = calloc(1, huge);
EXPECT_NOT_NULL(blob);
EXPECT_TRUE(xattr_list_append(list, "user.huge", blob, huge));
xattr_send(p[1], list); /* send is best-effort; the receiver rejects and exits */
free(blob);
xattr_list_free(list);
int status;
waitpid(pid, &status, 0);
close(p[1]);
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
}
/* The list append enforces the count bound (defense in depth). */
static void test_xattr_count_bound() {
FileXattrList* list = xattr_list_new();
EXPECT_NOT_NULL(list);
bool all_ok = true;
for (int i = 0; i < XATTR_MAX_COUNT + 1; i++) {
char name[32];
snprintf(name, sizeof(name), "user.k%d", i);
if (!xattr_list_append(list, name, "v", 1))
all_ok = false;
}
EXPECT_FALSE(all_ok);
EXPECT_EQ_INT(list->count, XATTR_MAX_COUNT);
xattr_list_free(list);
}
/* The captured list on a plain file reflects only whitelisted namespaces
* (Linux only; skipped when the filesystem has no xattr support). */
static void test_xattr_capture_and_appliable() {
EXPECT_FALSE(xattr_name_appliable(NULL));
EXPECT_FALSE(xattr_name_appliable(""));
EXPECT_FALSE(xattr_name_appliable("security.selinux"));
EXPECT_FALSE(xattr_name_appliable("trusted.blob"));
EXPECT_TRUE(xattr_name_appliable("user.foo"));
/* The reserved fake-super key is receiver-only and never forwarded/applied. */
EXPECT_FALSE(xattr_name_appliable("user.fastsync.stat"));
EXPECT_TRUE(xattr_name_appliable("system.posix_acl_access"));
EXPECT_TRUE(xattr_name_appliable("system.posix_acl_default"));
}
/* MINOR-2: a --link-dest / -H copy fallback (linkat refused) must still apply
* the per-file xattrs and --fake-super stat. A DIRECTORY basis forces linkat
* to fail with EPERM, exercising the byte-copy fallback deterministically.
* Guarded on filesystem xattr support. */
static void test_link_copy_fallback_preserves_xattrs() {
const char* dest = "test_link_xattr_dest.txt";
const char* basis_dir = "test_link_xattr_basis_dir";
unlink(dest);
rmdir(basis_dir);
EXPECT_EQ_INT(mkdir(basis_dir, 0700), 0);
/* Probe xattr support on the cwd filesystem using the destination file. */
int probe = open(dest, O_WRONLY | O_CREAT | O_TRUNC, 0600);
bool has_xattr = probe >= 0 && setxattr(dest, "user.fastsync.xprobe", "p", 1, 0) == 0;
if (probe >= 0)
close(probe);
if (!has_xattr) {
removexattr(dest, "user.fastsync.xprobe");
unlink(dest);
rmdir(basis_dir);
return; /* skip silently when the filesystem has no xattr support */
}
removexattr(dest, "user.fastsync.xprobe");
FileXattrList* xattrs = xattr_list_new();
EXPECT_NOT_NULL(xattrs);
EXPECT_TRUE(xattr_list_append(xattrs, "user.fallback", "kept", 4));
FileMetadata m;
memset(&m, 0, sizeof(m));
m.mode = 0640;
m.uid = 1001;
m.gid = 1002;
m.mtime_sec = 1234567890;
m.mtime_nsec = 0;
m.atime_valid = false;
m.crtime_valid = false;
bool ok = file_to_disk_secure_link_attrs(dest, basis_dir, "payload", 7, false, &m, false, false,
xattrs, true, NULL);
xattr_list_free(xattrs);
EXPECT_TRUE(ok);
/* Content landed (the copy fallback wrote the caller's bytes). */
int fd = open(dest, O_RDONLY);
EXPECT_TRUE(fd >= 0);
if (fd >= 0) {
char buf[16];
ssize_t n = read(fd, buf, sizeof(buf));
close(fd);
EXPECT_EQ_INT((int)strlen("payload"), (int)n);
if (n == 7)
EXPECT_TRUE(memcmp(buf, "payload", 7) == 0);
}
/* Per-file xattr applied on the copy. */
char vbuf[16];
ssize_t vlen = getxattr(dest, "user.fallback", vbuf, sizeof(vbuf));
EXPECT_EQ_INT(4, (int)vlen);
if (vlen == 4)
EXPECT_TRUE(memcmp(vbuf, "kept", 4) == 0);
/* fake-super stat parked by the receiver. */
EXPECT_TRUE((int)getxattr(dest, FAKESUPER_XATTR, NULL, 0) > 0);
unlink(dest);
rmdir(basis_dir);
}
void test_xattr() {
test_xattr_wire_roundtrip();
test_xattr_reject_privileged_namespace();
test_xattr_reject_oversized_value();
test_xattr_count_bound();
test_xattr_capture_and_appliable();
test_link_copy_fallback_preserves_xattrs();
}
+6
View File
@@ -0,0 +1,6 @@
#ifndef TEST_XATTR_H
#define TEST_XATTR_H
void test_xattr(void);
#endif