Compare commits
38
Commits
72cccaa256
...
v2.21.0
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
919a729206 | ||
|
|
8cd2b550d9 | ||
|
|
99c0fd8016 | ||
|
|
a2200f039a | ||
|
|
1437c6dc6b | ||
|
|
38356ecc1e | ||
|
|
7badac7f97 | ||
|
|
6ccf16b650 | ||
|
|
1174993d6b | ||
|
|
d5fcfa2c5c | ||
|
|
e79d2b47b0 | ||
|
|
7c24a365cf | ||
|
|
5893de4a34 | ||
|
|
825ba69753 | ||
|
|
34abaadb9a | ||
|
|
10c4ffebdf | ||
|
|
dfa2a42028 | ||
|
|
5b0ec5880d | ||
|
|
1a26bde2d4 | ||
|
|
58a28334b8 | ||
|
|
e48f19ee2b | ||
|
|
a2370433b2 | ||
|
|
9da5a0a9ed | ||
|
|
80c1ff321c | ||
|
|
551c187005 | ||
|
|
0d6c1f784f | ||
|
|
f75a69f96a | ||
|
|
df887c73b1 | ||
|
|
5d39619a8a | ||
|
|
6269ae54e5 | ||
|
|
07f7555c1d | ||
|
|
5b8aca5799 | ||
|
|
a1eaa93357 | ||
|
|
99df0a8a6d | ||
|
|
334fc5b3e8 | ||
|
|
88aee6ce94 | ||
|
|
f6e8b6ddc4 | ||
|
|
6f974eff19 |
No files matched your search
@@ -16,7 +16,7 @@ Ask the user or determine from context:
|
||||
- **Minor** (x.Y.0) — new features, backward compatible
|
||||
- **Patch** (x.y.Z) — bug fixes, no protocol changes
|
||||
|
||||
Current version: `PROTOCOL_VERSION "2.20.0"` in `src/shared/config.h`
|
||||
Current version: `PROTOCOL_VERSION "2.21.0"` in `src/shared/config.h`
|
||||
|
||||
### Step 2: Check Protocol Version
|
||||
|
||||
|
||||
+101
-15
@@ -4,24 +4,110 @@ All notable changes to FastSync are documented here. Versions match
|
||||
`PROTOCOL_VERSION` (printed by `fastsync --version`); the client and server must
|
||||
run the same version because the handshake is strict.
|
||||
|
||||
## [Unreleased]
|
||||
## [2.21.0] - 2026-09-14
|
||||
|
||||
### Added
|
||||
|
||||
- Optional server→client rejection detail (protocol 2.21.0). A rejected
|
||||
operation may now carry a bounded human-readable reason via
|
||||
`STATUS_ERROR_DETAIL` instead of a bare `STATUS_ERROR`, so the client can
|
||||
report *why* the server refused (daemon module gate, config validation,
|
||||
receiver-side path/node validation). `receive_status()` transparently maps the
|
||||
new status back to `STATUS_ERROR` for every existing call site and captures
|
||||
the reason into a thread-local buffer exposed by `protocol_last_error()`. The
|
||||
detail body is always consumed, so the stream cannot desynchronize, and
|
||||
messages are sliced to `MAX_ERROR_DETAIL_BYTES` (4096) on send.
|
||||
- **Server-contacting `--dry-run` (protocol 2.21.0).** `--dry-run` now performs
|
||||
a real handshake with a remote/daemon receiver and reports exactly what WOULD
|
||||
change based on receiver state (existing destination files, mtimes, checksums,
|
||||
basis dirs). The wire config carries the dry-run intent (`Config.dry_run`) and
|
||||
the receiver answers each per-file check with `STATUS_DRY_RUN_TRANSFER` (would
|
||||
transfer) or `STATUS_OK` (already up to date); the sender prints the
|
||||
would-transfer set and its trailer without sending any file data. The receiver
|
||||
performs the normal read-only incremental decision but mutates nothing: no temp
|
||||
files, writes, renames, deletes, metadata/xattr/chown, or directory creation.
|
||||
A plain local destination (no explicit `--server-port`/remote) keeps the
|
||||
original client-side dry-run. Would-delete reporting for `--delete*` is
|
||||
deferred to a follow-up; dry-run never deletes.
|
||||
- Daemon `max connections per host` (per-source-IP concurrent cap, default 0 =
|
||||
unlimited), `auth lockout threshold` (default 10; 0 disables) and
|
||||
`auth lockout duration` (default 300 s) config keys.
|
||||
- `fastsync-server --allow-super` opt-in for a privileged standalone TCP server;
|
||||
without it a root standalone receiver forces super-user activities off (device
|
||||
nodes, `--write-devices`, ownership). The `--stdio` SSH argv is client-composed,
|
||||
so super activities always stay off there.
|
||||
|
||||
### Changed
|
||||
|
||||
- Config wire fields are now declared once in an X-macro table
|
||||
(`CONFIG_WIRE_FIELDS` in `src/shared/config.h`) that generates the struct
|
||||
members, defaults, and the send/receive sequence, removing the manual
|
||||
six-site field sync. Wire bytes and `PROTOCOL_VERSION` are unchanged.
|
||||
- `receive_incremental_check()` (the per-file `STATUS_CHECK` fast path) is split
|
||||
into small static helpers with a short linear orchestrator. Pure refactor: the
|
||||
wire byte stream and all cleanup are unchanged.
|
||||
- `authorized_root` state has a single owner (`utils.c`) with read accessors; the
|
||||
duplicated statics in `file.c` and the server were removed.
|
||||
- `Data` records its owning `ProtocolSession` so its memory charge is returned to
|
||||
the session that reserved it, regardless of the destroying thread.
|
||||
- The receiver pipeline moved out of `shared` into `server/receiver_pipeline.[ch]`;
|
||||
the build now uses explicit `fastsync_shared` / `fastsync_client_core` /
|
||||
`fastsync_server_core` targets instead of a GLOB, and the client no longer links
|
||||
server code.
|
||||
- The benchmark tool generates the requested random/compressible data mix
|
||||
accurately, verifies each transfer before recording it, computes correct
|
||||
percentiles, adds a MB/s column, handles `tc`/netem without requiring `sudo`
|
||||
when already root, builds into a dedicated `build-bench/` directory, and adds a
|
||||
`--warm` incremental-transfer mode.
|
||||
- The `nix-shell` dev environment provides the full toolchain (clang-format,
|
||||
cppcheck, pytest-xdist, OpenSSH, rsync, iproute2, valgrind, lcov) and no longer
|
||||
builds on entry.
|
||||
|
||||
### Security
|
||||
|
||||
- Enforce the daemon's per-module `max connections` cap and add a global
|
||||
`max connections per host` cap plus a cross-process `auth lockout`
|
||||
(`auth lockout threshold` / `auth lockout duration`). Because the listener
|
||||
forks one child per connection, the counters live in an anonymous shared
|
||||
mapping created before the accept loop and reclaimed by the parent's
|
||||
`SIGCHLD` handler, so the per-module, per-source and auth-failure state is
|
||||
shared across every child (including after `SIGKILL`). The per-source table
|
||||
now has a bounded lifetime (expired-lockout/idle entries are reclaimed, with a
|
||||
rate-limited warning when it is genuinely full), and the occupancy counters are
|
||||
re-derived from the shared slot table on every child exit so a child killed
|
||||
mid-registration cannot leak a count. Trusted loopback peers are exempt from the
|
||||
per-host cap and the auth lockout (they share one address); clients behind a
|
||||
shared NAT/proxy still share a single per-host budget and lockout, which is
|
||||
documented.
|
||||
- Enforce the daemon's per-module `max connections` cap (0 = unlimited) and add
|
||||
the shared per-source `max connections per host` cap plus a cross-process
|
||||
`auth lockout`. Because the listener forks one child per connection, the
|
||||
counters live in an anonymous shared mapping created before the accept loop and
|
||||
reclaimed by the parent's `SIGCHLD` handler, so the per-module, per-source and
|
||||
auth-failure state is shared across every child (including after `SIGKILL`). The
|
||||
per-source table has a bounded lifetime (expired/idle entries are reclaimed,
|
||||
with a rate-limited warning when genuinely full), and the occupancy counters are
|
||||
re-derived from the shared slot table on every child exit. Trusted loopback
|
||||
peers are exempt (they share one address); clients behind a shared NAT/proxy
|
||||
share a single per-host budget and lockout, which is documented.
|
||||
- Hardening from a full security audit:
|
||||
- Fail a truncated zstd frame instead of spinning forever (remote DoS).
|
||||
- Open receiver destination/basis/hard-link entries `O_NONBLOCK` so a
|
||||
client-planted FIFO cannot block a worker indefinitely.
|
||||
- Require a regular file before `--inplace` writes, closing a FIFO-hang and a
|
||||
raw-device write that bypassed the `--write-devices` gate.
|
||||
- Reject SSH destinations whose user/host begins with `-` and insert `--` before
|
||||
the host token, closing `-o ProxyCommand=…` argument injection (RCE).
|
||||
- Gate client `--force` recursive removal behind the server `--allow-delete`
|
||||
policy.
|
||||
- Reject empty `hosts allow`/`hosts deny`/`auth users` values instead of
|
||||
silently meaning "unrestricted".
|
||||
- Restrict TLS 1.2 to AEAD suites and set server cipher preference; load the
|
||||
private key TOCTOU-safely from an `O_NOFOLLOW` fd; verify IP literals against
|
||||
IP SANs; guard client-cert CN truncation.
|
||||
- Make `--dry-run` content-blind: it neither reads destination files nor
|
||||
hashes basis files, removing a 1-bit content oracle against `read only`
|
||||
modules.
|
||||
- Bound glob matching (iterative DP, no exponential backtracking) and bound
|
||||
line reads for filter/`--files-from`/pattern files.
|
||||
- Gate `system.posix_acl_*` xattrs on `--acls` and charge decompression/chunk
|
||||
allocations against the per-connection memory budget.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Pre-auth NULL dereference in `config_delete()` when an over-long
|
||||
`basis_count` (and the analogous count fields) was received and then failed
|
||||
validation; received counts are now validated before being published.
|
||||
- Leaked inherited `Data` in the forked compression-truncation unit test
|
||||
(valgrind definite leak).
|
||||
- `receive_status()` no longer loses a captured rejection reason when owed
|
||||
keepalives are drained.
|
||||
|
||||
## [2.20.0] - 2026-09-13
|
||||
|
||||
|
||||
+2
-1
@@ -1,6 +1,6 @@
|
||||
cmake_minimum_required(VERSION 3.22)
|
||||
|
||||
project(FastFileTransfer VERSION 2.20.0)
|
||||
project(FastFileTransfer VERSION 2.21.0)
|
||||
|
||||
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
|
||||
set(CMAKE_C_STANDARD 11)
|
||||
@@ -223,6 +223,7 @@ set(TEST_SRCS
|
||||
tests/test_multiprocessing.c
|
||||
tests/test_property.c
|
||||
tests/test_protocol.c
|
||||
tests/test_protocol_error.c
|
||||
tests/test_queue.c
|
||||
tests/test_receiver_timeout.c
|
||||
tests/test_robustness.c
|
||||
|
||||
@@ -178,6 +178,7 @@ transfer is never aborted.
|
||||
| `--ca <path>` | TLS CA certificate file for verification (PEM) |
|
||||
| `--destination-root <path>` | Authorized destination root (default: `.`) |
|
||||
| `--allow-delete` | Permit manifest deletion |
|
||||
| `--allow-super` | Standalone TCP listener only: keep super-user activities enabled for a **root** receiver. Without it a root standalone server forces `SUPER_MODE_OFF`, so client `--devices`/`--write-devices`/`--super` and client-chosen ownership requests are skipped/refused. **Rejected with `--stdio`** (the SSH remote argv is client-composed, so a client could otherwise pass it and defeat the secure default; operators exposing `fastsync-server --stdio` over SSH must use a forced command if the default must hold). No effect when not root. |
|
||||
| `--allow-unauthenticated` | Permit plaintext TCP clients. For an `auth users` module this opts in **loopback plaintext only**; remote auth still requires verified TLS, so the flag never permits remote plaintext auth. |
|
||||
| `-v, --verbose` | Enable debug logging |
|
||||
| `--help` | Show help |
|
||||
@@ -304,11 +305,22 @@ The remote host must have `fastsync-server` available in `PATH`, or use
|
||||
working directory, so use a destination below that directory unless the
|
||||
remote server is otherwise configured with a matching authorized root.
|
||||
|
||||
The remote `--stdio` server argv is composed by the client, so it must never
|
||||
be trusted to opt a root receiver into super-user activities: `--allow-super`
|
||||
is rejected with `--stdio` and super stays off on that path. Operators
|
||||
exposing `fastsync-server --stdio` over SSH must use a forced command (e.g. an
|
||||
`authorized_keys` `command=` entry) if the default must hold.
|
||||
|
||||
```bash
|
||||
ssh user@host 'mkdir -p destination'
|
||||
./build/client /path/to/source user@host:destination
|
||||
```
|
||||
|
||||
FastSync is **push-only**: the source (first argument) is always a local
|
||||
directory and only the destination may be remote. A remote source such as
|
||||
`client user@host:src ./local` (a "pull") is intentionally not supported; see
|
||||
[RSYNC_COMPAT.md](RSYNC_COMPAT.md#direction).
|
||||
|
||||
### TCP transfer
|
||||
|
||||
Start the FastSync server:
|
||||
@@ -498,7 +510,8 @@ link-target transfer remains incomplete. |
|
||||
| `--ca <path>` | CA file for peer verification. |
|
||||
| `--destination-root <path>` | Confine received files to this server-side root;
|
||||
defaults to the current directory. |
|
||||
| `--allow-delete` | Permit client delete manifests. Deletion is refused by default. |
|
||||
| `--allow-delete` | Permit client delete manifests. Deletion is refused by default. This also gates `--force` (which can recursively replace/remove a destination directory tree). |
|
||||
| `--allow-super` | Standalone TCP listener only: keep super-user activities enabled for a **root** receiver. Without it a root standalone server forces `SUPER_MODE_OFF`, so client `--devices`/`--write-devices`/`--super` and client-chosen ownership requests are skipped/refused. Rejected with `--stdio` (the SSH remote argv is client-composed; use a forced command if the default must hold). No effect when not root. Daemon modules opt in per module with `client owner = yes`. |
|
||||
| `-v`, `--verbose` | Enable debug logging. |
|
||||
| `--help` | Print server usage. |
|
||||
|
||||
@@ -582,7 +595,7 @@ before the module list, before authentication, and the connecting peer address
|
||||
|
||||
## Protocol and Security
|
||||
|
||||
FastSync protocol version `2.20.0` is shared by the client and server. The
|
||||
FastSync protocol version `2.21.0` is shared by the client and server. The
|
||||
current protocol is sender-driven and includes configuration negotiation,
|
||||
including the maximum allocation limit, incremental checks, checksums,
|
||||
manifests, keep-alives, abort handling, per-file remove-source results, and
|
||||
|
||||
+10
-9
@@ -93,7 +93,7 @@ This document maps rsync's full feature set to FastSync's current implementation
|
||||
|
||||
| Flag | Rsync Description | FastSync Status | Notes |
|
||||
|------|-------------------|-----------------|-------|
|
||||
| `-n`, `--dry-run` | Trial run with no changes | ✅ Implemented | `dry_run` config field |
|
||||
| `-n`, `--dry-run` | Trial run with no changes | ✅ Implemented | Server-contacting since protocol 2.21.0. The final routing predicate is `dry_run_targets_server()` in `src/client/client_send.c`: any target a real run would reach over the wire selects the server-contacting path — an SSH transport, a daemon `host::module` destination, an explicit `--server-host` or `--server-port`/`--port`, TLS, or a source-bind `--address` — and the client handshakes with the receiver, which runs the normal read-only per-file check and answers `STATUS_DRY_RUN_TRANSFER`/`STATUS_OK` without mutating anything. A plain local destination (none of those) keeps the original client-side manifest that never dials the default `127.0.0.1:8080`. Would-delete reporting for `--delete*` is deferred (dry-run never deletes). |
|
||||
| `-b`, `--backup` | Make backups of overwritten files | ✅ Implemented | Backup before overwrite |
|
||||
| `--backup-dir=DIR` | Backup directory hierarchy | ✅ Implemented | `backup_dir` config field |
|
||||
| `--suffix=SUFFIX` | Backup suffix (default ~) | ✅ Implemented | `suffix` config field |
|
||||
@@ -257,14 +257,14 @@ why plain `--append` works on the normal atomic path, not only with `--inplace`.
|
||||
| `-N`, `--crtimes` | Preserve create times | ⛔ Impossible/Divergence | Birth-times cannot be set by any portable filesystem call (`utimensat`/`futimens` only set atime/mtime), so this row is an explicit **Impossible/Divergence** (Phase 7 Wave B). Capture + transmit stays: `statx(STATX_BTIME)` on Linux records the source birth time as a wire field; the receiver logs a debug note that it cannot be applied and continues — never failing the transfer and never pretending it worked. On platforms without `statx` it parses as a documented no-op (flag accepted; nothing is captured). Implies metadata transmission. Wire: new `crtime` fields + a `preserve_crtimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** (see the Phase-4 metadata-time notes) |
|
||||
| `-O`, `--omit-dir-times` | Omit dirs from --times | ✅ Implemented | Real modifier now that FastSync preserves directory times. With metadata on, the scanner captures every traversed source directory's mtime (and atime under `-U`) and the sender transmits them in trailing `STATUS_DIR_TIMES` frame(s) **after all file data and the optional delete manifest** (chunked at the receiver's `MAX_MANIFEST_ENTRIES` per-frame cap); a dir-time entry only RECORDS metadata and never creates the directory, so empty source directories stay untransferred. The receiver defers applying them until its delete / `--delay-updates` publication phases have committed, so writing or removing a child never clobbers a parent directory's mtime (rsync applies directory times at the end for exactly this reason). When `-O` is set (the boolean crosses the wire) the receiver does not apply any of them; without `-O` an `-a`/`--preserve` transfer now restores directory times (reversing the old "never preserves dir times" divergence). Wire change: the terminal `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** |
|
||||
| `-J`, `--omit-link-times` | Omit symlinks from --times | ✅ Implemented | Real modifier now that FastSync preserves symlink times. Symlink entries already carried their metadata on `STATUS_SYMLINK`; the receiver now applies it with **no-follow primitives only** (`utimensat(..., AT_SYMLINK_NOFOLLOW)`, plus best-effort `fchmodat(..., AT_SYMLINK_NOFOLLOW)` and policy-gated `fchownat(..., AT_SYMLINK_NOFOLLOW)`), so the link itself is stamped without ever dereferencing it, confined fd-relative below the authorized receive root. A symlink has no children, so the times are applied immediately at creation. When `-J` is set (the boolean crosses the wire) the receiver skips the timestamps (mode/ownership are unaffected); without `-J` an `-a`/`-l` transfer restores symlink mtimes. Wire change alongside `-O`: the shared `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** |
|
||||
| `--super` | Receiver attempts super-user activities | ✅ Implemented | Phase 7 Wave E: receiver-side **safe-subset + clear-refusal** privilege model, tri-state `super_mode` (auto/on/off). `--super` **permits** the receiver to attempt super-user activities — ownership application and char/block device-node creation — that are already confined fd-relative below the authorized receive root; `--no-super` **forbids** them even when the receiver is root; the default (`auto`) preserves the pre-existing **best-effort** behavior of *attempting* them (not only when already root: an unprivileged attempt is refused by the kernel and skipped per entry, matching FastSync's history). The server additionally accepts an operator-level `--no-super` veto that forces `OFF` for every connection it accepts (so it also refuses any client `--copy-as`/`--super`); the `--fake-super` owner replay and the `--write-devices` write path are gated by the same policy. **FastSync never elevates**: no `setuid`/`seteuid`/`setgid` is ever called, and `--super` never bypasses the confinement floor (`file_open_secure_parent`, `O_NOFOLLOW`, root checks) — it only permits an attempt that is already confined. `--super` does **not** imply `--numeric-ids` and never enables client-chosen ownership on its own: ownership is applied only when an explicit identity policy (`--usermap`/`--groupmap`/`--chown`/`--numeric-ids`/`--copy-as`) is also given. A non-root receiver given `--super` logs exactly one warning at activation and each confined attempt is then refused by the kernel and skipped per entry (never aborts); `--no-super` suppresses ownership, char/block `mknod`, `--write-devices` and the fake-super owner replay, while unprivileged FIFO creation is unaffected. Wire: one trailing `super_mode` int on the config frame (validated 0..2), sent **before** the `--copy-as` block (fixed order: super int, then copy-as presence int + ids); `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Documented divergence from rsync:** rsync's `--super` runs the receiver with elevated privilege; FastSync only permits a confined attempt and never elevates |
|
||||
| `--super` | Receiver attempts super-user activities | ✅ Implemented | Phase 7 Wave E: receiver-side **safe-subset + clear-refusal** privilege model, tri-state `super_mode` (auto/on/off). `--super` **permits** the receiver to attempt super-user activities — ownership application and char/block device-node creation — that are already confined fd-relative below the authorized receive root; `--no-super` **forbids** them even when the receiver is root; the default (`auto`) preserves the pre-existing **best-effort** behavior of *attempting* them (not only when already root: an unprivileged attempt is refused by the kernel and skipped per entry, matching FastSync's history). The server additionally accepts an operator-level `--no-super` veto that forces `OFF` for every connection it accepts (so it also refuses any client `--copy-as`/`--super`); a **privileged (root) standalone TCP listener now also defaults to `OFF`** unless the operator opts in with the new server-only `--allow-super` flag (the flag is **rejected with `--stdio`**, whose remote argv is composed by the client and must never defeat the secure default; operators exposing `fastsync-server --stdio` over SSH need a forced command if the default must hold. An unprivileged receiver is unchanged, since the kernel refuses the confined attempts anyway; the `--daemon` path keeps its per-module `client owner = yes` opt-in); the `--fake-super` owner replay and the `--write-devices` write path are gated by the same policy. **FastSync never elevates**: no `setuid`/`seteuid`/`setgid` is ever called, and `--super` never bypasses the confinement floor (`file_open_secure_parent`, `O_NOFOLLOW`, root checks) — it only permits an attempt that is already confined. `--super` does **not** imply `--numeric-ids` and never enables client-chosen ownership on its own: ownership is applied only when an explicit identity policy (`--usermap`/`--groupmap`/`--chown`/`--numeric-ids`/`--copy-as`) is also given. A non-root receiver given `--super` logs exactly one warning at activation and each confined attempt is then refused by the kernel and skipped per entry (never aborts); `--no-super` suppresses ownership, char/block `mknod`, `--write-devices` and the fake-super owner replay, while unprivileged FIFO creation is unaffected. Wire: one trailing `super_mode` int on the config frame (validated 0..2), sent **before** the `--copy-as` block (fixed order: super int, then copy-as presence int + ids); `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Documented divergence from rsync:** rsync's `--super` runs the receiver with elevated privilege; FastSync only permits a confined attempt and never elevates |
|
||||
| `--fake-super` | Store/recover privileged attrs via xattrs | ✅ Implemented | Phase 7 Wave B: full record **and replay**. The receiver writes the source `uid:gid:mode:mtime_sec:mtime_nsec` into a reserved `user.fastsync.stat` xattr on each written file (best-effort, fd-relative, format unchanged), then immediately re-applies it via `fake_super_restore_fd`: `fchown` (only where privileged — a non-root EPERM/EACCES is skipped silently, matching FastSync's identity philosophy), `fchmod`, and `futimens`. The OWNER leg is additionally skipped unless an explicit ownership identity policy (`--numeric-ids`/`--usermap`/`--groupmap`/`--chown`/`--copy-as`) is active — `--fake-super` on its own only *records* the source owner and must not act as an un-gated chown primitive — when `--no-super` forbids super-user activities (even for root), or when an active `--copy-as` is authoritative, so the recorded source owner can never override a forced `--copy-as` owner; the xattr record is still stored/replayed for a later privileged restore and mode/mtime still apply, so unprivileged `--fake-super` keeps working. The restored mode goes through the same sanitization as the normal metadata path (group/other write bits are never granted, so a recorded 0666 restores as 0644), so fake-super replay can never grant group/other-write that plain `--preserve` would refuse. Absence or a malformed record is a silent no-op, never fatal. The recording format diverges from rsync's `user.rsync.%stat%`; no cross-tool conversion is attempted. Implies metadata transmission so the source uid/gid/mode/mtime are available. Both it and `-X`/`-A` are incompatible with `-s` (chunk serialization), rejected up front |
|
||||
| `--open-noatime` | Avoid changing access time when opening files | ✅ Implemented | Sender-side policy: the sender opens source files with `O_NOATIME` (Linux) when reading them for transfer, so the open/read does NOT bump the source's on-disk access time. Degrades safely when `O_NOATIME` is unavailable (not defined) or refused (`EPERM`, since it needs `CAP_FOWNER` or file ownership): the code falls back to a normal open, so the data always transfers — only the atime-bump is skipped. It does not itself capture/preserve atime; it only avoids modifying it. **Client-only, never crosses the wire.** Exposed as `file_open_for_read()` and applied to both the buffered data path and the sendfile path |
|
||||
| `--numeric-ids` | Do not map uid/gid by name | ✅ Implemented | Ownership is applied through FastSync's opt-in identity path (see the Phase-4 identity notes below). `--numeric-ids` is a mapping-policy modifier: when applying ownership it uses the transmitted numeric uid/gid directly, skipping the name lookup. Without an ownership-affecting option it is inert (FastSync only applies ownership when the user opts in). It does not need `-M` to be parsed, but ownership is only applied when metadata (hence the source uid/gid) is actually transmitted (see the notes) |
|
||||
| `--usermap=STRING` | Map usernames | ✅ Implemented | Opt-in ownership application. rsync subset implemented: comma-separated `FROM:TO` rules evaluated in order, first match wins; `FROM`/`TO` are group/user names (resolved on the SOURCE machine at parse time), `*` (FROM matches any id / TO = the receiving process's current euid), and an `@N` or bare `N` numeric id. Rules are carried over the wire as resolved numeric id pairs; the receiver applies a matching rule (else falls back to `--chown`, `--numeric-ids`, then a best-effort name lookup) via an fd-relative `fchown`. Malformed/unresolvable specs are rejected with a clear error, never a silent no-op. Implies metadata preservation so the source uid/gid travel. Only effective when the receiver can actually change ownership (root or membership); otherwise it warns and continues |
|
||||
| `--groupmap=STRING` | Map group names | ✅ Implemented | Same rsync subset and semantics as `--usermap` but for the group (gid) side and the group databases. See the Phase-4 identity notes |
|
||||
| `--chown=USER:GROUP` | Map owner and group | ✅ Implemented | Opt-in ownership override applied receiver-side. Forms: `USER:GROUP`, `USER` (owner only), `:GROUP` (group only); a `*` for USER/GROUP means the current/root user or group as appropriate; an `@N`/bare `N` numeric id is accepted. A `:` inside a name may be escaped as `\:`. Equivalent to a trailing `*:*` usermap+groupmap rule (so an explicit `--usermap`/`--groupmap` match wins). Malformed or unresolvable specs are clear parse errors. Implies metadata preservation. Only effective when the receiver has permission to chown; otherwise it warns and continues (rsync parity) |
|
||||
| `--copy-as=USER[:GROUP]` | Perform the copy as another user/group | ✅ Implemented | Safe-subset implementation, an explicit divergence from rsync's **real identity switching**. rsync makes the receiving process actually assume USER/GROUP (setuid/setgid); FastSync's receiver is multithreaded, so a real credential drop would be unsafe and is never attempted — FastSync never calls `setuid`/`seteuid`/`setgid`. Instead the receiver FORCES the ownership of every entry it writes to `copy_as_uid`/`copy_as_gid` through the existing confined, fd-relative identity path (the same `fchown`/`fchownat` mechanism as `--chown`/`--usermap`/`--groupmap`; symlinks use `fchownat(..., AT_SYMLINK_NOFOLLOW)`, and directories — including intermediate parents created implicitly while writing a nested file — and char/block/FIFO nodes are owned no-follow too, so a directory never keeps the receiver's owner while its children get the target owner), with `--copy-as` at the **highest priority** — it beats usermap/groupmap/`--chown`/`--numeric-ids` and the best-effort name lookup. This REQUIRES a privileged (root) receiver: an unprivileged receiver REFUSES the whole transfer up front at the config handshake (`server_module_gate`, running inside `config_receive_with_validate` before the `STATUS_OK` ack) with a clear error and no file data exchanged — never a silent wrong-ownership result. A server running with an operator `--no-super` veto also refuses it, and a **daemon** refuses `--copy-as`, like every other client-chosen-ownership request (`--numeric-ids`/`--chown`/`--usermap`/`--groupmap`/`--fake-super`/explicit `--super`), unless the selected module opts in with `client owner = yes`; without that per-module opt-in a daemon must not honor an arbitrary client-selected owner (the standalone listener and SSH `--stdio` server keep honoring these for their single operator-authorized root). `--fake-super` interaction: `--copy-as` is authoritative, so the recorded source owner is never replayed over the forced target owner. If the ownership apply still fails with EPERM/EACCES (capability-restricted root, root-squash, read-only mount) the failure is logged at ERROR and the **entry is reported as failed** rather than written with the wrong owner, which fails the transfer (fail-fast) so overall success is never reported with the wrong owner. USER is resolved on the client against the user database (a name, an `@N`/bare `N` numeric id, or `*` meaning the client's current euid); when `:GROUP` is present it is resolved against the group database (`*` meaning the client's egid). **Group-default rule:** when the group is omitted FastSync uses the user's primary gid (`getpwuid(uid)->pw_gid`); a numeric id with no local passwd entry has no primary gid to look up, so `gid` falls back to `uid` (documented divergence). Malformed/empty/unresolvable specs are clear parse errors, never a silent no-op. Never elevates privileges and never bypasses the confined receive root. Implies metadata preservation (the source uid/gid must be transmitted). Wire: a new trailing config-frame block **sent after** the `--super` int (presence int, then the two int32 ids, both validated `>= 0` on receive; the ids are also rejected if they do not fit int32 at CLI parse time); `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0** |
|
||||
| `--copy-as=USER[:GROUP]` | Perform the copy as another user/group | ✅ Implemented | Safe-subset implementation, an explicit divergence from rsync's **real identity switching**. rsync makes the receiving process actually assume USER/GROUP (setuid/setgid); FastSync's receiver is multithreaded, so a real credential drop would be unsafe and is never attempted — FastSync never calls `setuid`/`seteuid`/`setgid`. Instead the receiver FORCES the ownership of every entry it writes to `copy_as_uid`/`copy_as_gid` through the existing confined, fd-relative identity path (the same `fchown`/`fchownat` mechanism as `--chown`/`--usermap`/`--groupmap`; symlinks use `fchownat(..., AT_SYMLINK_NOFOLLOW)`, and directories — including intermediate parents created implicitly while writing a nested file — and char/block/FIFO nodes are owned no-follow too, so a directory never keeps the receiver's owner while its children get the target owner), with `--copy-as` at the **highest priority** — it beats usermap/groupmap/`--chown`/`--numeric-ids` and the best-effort name lookup. This REQUIRES a privileged (root) receiver: an unprivileged receiver REFUSES the whole transfer up front at the config handshake (`server_module_gate`, running inside `config_receive_with_validate` before the `STATUS_OK` ack) with a clear error and no file data exchanged — never a silent wrong-ownership result. A server running with an operator `--no-super` veto also refuses it; a privileged (root) standalone TCP listener refuses it by default too and only honors it after the operator passes `--allow-super` (the flag is rejected with `--stdio`, where the client-composed remote argv could otherwise defeat the default; a forced command is required if the default must hold), and a **daemon** refuses `--copy-as`, like every other client-chosen-ownership request (`--numeric-ids`/`--chown`/`--usermap`/`--groupmap`/`--fake-super`/explicit `--super`), unless the selected module opts in with `client owner = yes`; without that per-module opt-in a daemon must not honor an arbitrary client-selected owner (a root standalone listener honors these for its single operator-authorized root only when started with `--allow-super`). `--fake-super` interaction: `--copy-as` is authoritative, so the recorded source owner is never replayed over the forced target owner. If the ownership apply still fails with EPERM/EACCES (capability-restricted root, root-squash, read-only mount) the failure is logged at ERROR and the **entry is reported as failed** rather than written with the wrong owner, which fails the transfer (fail-fast) so overall success is never reported with the wrong owner. USER is resolved on the client against the user database (a name, an `@N`/bare `N` numeric id, or `*` meaning the client's current euid); when `:GROUP` is present it is resolved against the group database (`*` meaning the client's egid). **Group-default rule:** when the group is omitted FastSync uses the user's primary gid (`getpwuid(uid)->pw_gid`); a numeric id with no local passwd entry has no primary gid to look up, so `gid` falls back to `uid` (documented divergence). Malformed/empty/unresolvable specs are clear parse errors, never a silent no-op. Never elevates privileges and never bypasses the confined receive root. Implies metadata preservation (the source uid/gid must be transmitted). Wire: a new trailing config-frame block **sent after** the `--super` int (presence int, then the two int32 ids, both validated `>= 0` on receive; the ids are also rejected if they do not fit int32 at CLI parse time); `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0** |
|
||||
|
||||
**Phase-4 metadata-time notes:** `-U/--atimes`, `-N/--crtimes`,
|
||||
`-O/--omit-dir-times`, `-J/--omit-link-times`, and `--open-noatime` are new.
|
||||
@@ -639,8 +639,9 @@ now transmits targets (the prior behavior was broken/partial); its status moved
|
||||
- **Host access control (`hosts allow`/`hosts deny`):** both keys accept a comma- and/or whitespace-separated list of patterns and may appear globally and/or per module (multiple config-file lines append; a `--dparam` override replaces). Supported patterns are `*` (match all), an IPv4 or IPv6 literal (`10.0.0.1`, `2001:db8::1`), and an IPv4/IPv6 CIDR (`10.0.0.0/8`, `2001:db8::/32`). Hostname patterns are **not** supported: because the peer is always a numeric address and no reverse DNS is performed, a hostname/glob pattern would silently never match, so it is rejected at load time (fail-closed) instead of being accepted as a dead rule. An IPv4 peer on a dual-stack IPv6 listener is normalized from its `::ffff:a.b.c.d` form so IPv4 patterns match it. rsync-like semantics: a matching `hosts deny` rejects; if any `hosts allow` entries exist, a peer matching none of them is rejected; deny takes precedence over allow. The daemon enforces the global list first, then the selected module's list, **before authentication** in `server_module_gate`, with an audit log line naming the peer, the module and the outcome. The numeric peer address is obtained with `getpeername`+`inet_ntop` (`utils_fd_peer_ip`, handling both address families); when it cannot be obtained a module with any ACL fails closed (refused), while an ACL-free module continues and logs at debug. A malformed pattern (e.g. an out-of-range CIDR prefix) is a parse error at load time.
|
||||
- **Connection caps, shared registry and auth lockout:** the global `max connections` key (default 100) is plumbed into the listener (`transport_tcp.c`), which rejects a connection once the accept-loop parent's active-child count reaches it; the IPv4/IPv6 peer is logged for every accepted connection. Because the listener forks one child per connection, the per-module `max connections` cap, the global `max connections per host` cap, and the auth-failure counter live in a fixed-size registry carved from an anonymous shared mapping (`daemon_limits.c`, `mmap(MAP_SHARED|MAP_ANONYMOUS)`) created by the parent before the accept loop, so every forked child shares the same counters (C11 atomics only — never a pthread lock, which can deadlock in a forked child). The parent reserves a registry slot per accepted connection and the child records the selected module and source IP once known; the parent's `SIGCHLD` handler reclaims the slot when the child dies (including `SIGKILL`) and re-derives the per-module and per-source occupancy counts from the surviving REGISTERED slots, so a child killed mid-registration cannot leak a count. The per-source table has a bounded lifetime: an entry with no live connection is reclaimed after its lockout expires or it has been idle (300 s); if the table is genuinely full the per-source cap/lockout fails open for new sources (per-module cap and ACLs still apply) with a rate-limited warning. The per-module cap (0 = unlimited) is enforced after the module lookup and before auth; per-source identity reuses the normalized numeric peer address (`utils_fd_peer_ip`, IPv4-mapped IPv6 collapsed to IPv4), and a trusted loopback peer (127.0.0.0/8 / `::1`, `utils_fd_peer_is_local`) is exempt from the per-source cap and the auth lockout because all local clients share one address (the per-module/global caps still apply). Clients behind a shared NAT/proxy address likewise share one per-source budget and lockout counter. A failed authentication increments the shared per-source failure count and, once `auth lockout threshold` (default 10; 0 disables) is reached, the source is refused for `auth lockout duration` seconds (default 300) before any challenge is sent, even when the next attempt is handled by a different forked child; a successful authentication clears the counter. On a failed authentication the per-connection child still sleeps the global `auth failure delay` (default 500 ms, 0 disables, capped at 5000) via `nanosleep`, rate-limiting online guessing without delaying a success. A missing registry (allocation failure) degrades to the global cap and host ACLs rather than refusing to start.
|
||||
- **Module selection & confinement:** the client requests a module with an rsync-style `host::module[/path]` destination. The module name crosses the wire as a trailing string on the config frame (bumping `PROTOCOL_VERSION` 2.14.0 → 2.15.0; the bump is required because the config-frame layout changed and the strict same-version handshake is what prevents a peer from desynchronizing on the new trailing field). The daemon looks the module up in ITS OWN config and uses the module's `path` as the authorized root through the exact same `configure_authorization` confinement the standalone server applies to `--destination-root` (`file_open_secure_parent`, `has_path_traversal`, `path_is_within`); the client never supplies the root, every client-chosen-ownership/super-user request is refused unless the module declares `client owner = yes` (the daemon's per-module opt-in, see below), and the operator `--no-super` veto forces super-user activities off for every daemon connection. The client's `/path` part is relative inside the module and is rejected if absolute or if it contains `..`. Unknown modules are refused before any data moves (the run fails cleanly at the config handshake). An absolute destination and a module request against a non-daemon server are also refused.
|
||||
- **`client owner` (client-chosen-ownership opt-in):** by default a daemon module refuses every request that would let the client pick an owner or ask for super-user activities — `--numeric-ids`, `--chown`, `--usermap`/`--groupmap`, `--fake-super`, `--copy-as`, and an explicit `--super` — at the config handshake (before `STATUS_OK`), because a daemon has no per-module opt-in for client-chosen ownership and any anonymous client could otherwise force arbitrary owner ids inside the module root. `client owner = yes` opts a single module in, allowing those requests within that module's root (the standalone listener and the SSH `--stdio` server always honor them for their single operator-authorized root). Without the opt-in the daemon also forces super-user **device** activity off for that connection — char/block device-node creation (`--devices`) and `--write-devices` — even under the default `AUTO` mode, so a non-opted module can never be made to `mknod` or write a raw device; those entries are skipped (not refused) so an ordinary `-a` push still succeeds without device nodes. The opt-in does **not** lift the privilege requirement: `--copy-as` still needs a root receiver, and the operator `--no-super` veto still forces super-user activities off for every connection. The daemon logs a prominent startup warning for each `client owner = yes` module so the operator's deliberate choice is visible.
|
||||
- **`client owner` (client-chosen-ownership opt-in):** by default a daemon module refuses every request that would let the client pick an owner or ask for super-user activities — `--numeric-ids`, `--chown`, `--usermap`/`--groupmap`, `--fake-super`, `--copy-as`, and an explicit `--super` — at the config handshake (before `STATUS_OK`), because a daemon has no per-module opt-in for client-chosen ownership and any anonymous client could otherwise force arbitrary owner ids inside the module root. `client owner = yes` opts a single module in, allowing those requests within that module's root (a root standalone TCP listener honors them for its single operator-authorized root only when started with `--allow-super`; the flag is rejected with `--stdio`, whose client-composed remote argv must never opt back into super mode). Without the opt-in the daemon also forces super-user **device** activity off for that connection — char/block device-node creation (`--devices`) and `--write-devices` — even under the default `AUTO` mode, so a non-opted module can never be made to `mknod` or write a raw device; those entries are skipped (not refused) so an ordinary `-a` push still succeeds without device nodes. The opt-in does **not** lift the privilege requirement: `--copy-as` still needs a root receiver, and the operator `--no-super` veto still forces super-user activities off for every connection. The daemon logs a prominent startup warning for each `client owner = yes` module so the operator's deliberate choice is visible.
|
||||
- **`read only` safe default:** every network transfer FastSync currently supports is a push that writes under the module root, so a `read only` module refuses the connection (clear server log "module is read only"; the client exits non-zero, nothing is transferred). A future pull/list operation can be opened up when it exists; the knob is already stored.
|
||||
- **Direction — remote source / pull is intentionally unsupported:** FastSync is push-only. The first positional argument is always a **local** source directory and the second is the destination; only the destination is parsed for remote syntax (`user@host:path` SSH, `host::module[/path]` daemon). A remote source such as `fastsync user@host:src ./local` is deliberately **not** implemented: rsync has no pull flag (direction is positional), so supporting a remote source is an optional feature rather than a compatibility requirement, and it would require a protocol role reversal (server as sender, client as receiver) across both transports. FastSync documents this as an intentional limitation rather than a missing rsync option. <a id="direction"></a>
|
||||
- **`auth users` (A7 SCRAM-SHA-256 authentication):** a module that declares `auth users` requires the client to present credentials. The config frame carries ONLY the username; the daemon answers an auth-required module with `STATUS_AUTH_CHALLENGE` (PBKDF2 iteration count, 16-byte salt, 32-byte server nonce), the client answers with `STATUS_AUTH_RESPONSE` (fresh 32-byte client nonce + a 32-byte ClientProof), and the daemon accepts only when the proof verifies **and** the username is **on the module's `auth users` list** and has a store entry, replying `STATUS_AUTH_OK` with a 32-byte ServerSignature the client verifies before proceeding. Verification is constant-time over fixed 32-byte keys (the compare runs even for a miss), username membership uses a constant-time full-length scan, and an unknown/off-list user still receives a challenge and runs the same math against a dummy verifier: a deterministic per-username salt (`HMAC-SHA256(store dummy key, username)`), the store-wide uniform iteration count and dummy keys. Re-probing the same unknown username therefore yields an identical salt and iteration count while a different username yields a different salt, so there is no user-enumeration or timing oracle. The daemon logs the username but **never the password, proof or keys**. A module WITHOUT `auth users` stays open (legitimate rsync configuration); credentials sent to such a module are ignored. Read-only is orthogonal: even a correctly authenticated push to a `read only` module is still refused (all FastSync network transfers write). Fail-closed policy: a daemon whose config declares `auth users` on any module refuses to start unless a credential store was given (`--password-file` and/or `--early-input`); a missing or empty store is never silently treated as "open". A failed handshake (missing credentials, unknown/off-list user, wrong proof or malformed data) yields a single generic `STATUS_AUTH_FAILED` and the daemon closes before any data moves. The dummy key is persisted in an owner-only `<store_path>.dummykey` sidecar (auto-created on first load, mode 0600) so the dummy salt stays stable across daemon restarts, closing the restart-gated enumeration channel. The sidecar is secret material and must be protected like the credential store (owner-only 0600, included with the store in backups and rotation). It must be preserved across restarts for that guarantee; if it cannot be created (a process-substitution/FIFO store path such as `/dev/fd/N`, a read-only filesystem, a missing directory, or a create/write/fsync/link/fchmod failure), the daemon logs a warning and uses a transient per-run key, so unknown-user challenges change across restarts and the cross-restart guarantee does not hold for that deployment. One residual is accepted: the store iteration count is observable pre-auth by design, since the miss path must match a hit. **Transport policy (hardening A7-3/S1):** an auth-required module accepts credentials only when either (a) the connection is an encrypted, verified TLS connection whose client certificate matches `--client-cn`, or (b) the connection is plaintext from a loopback TCP peer **and** the operator explicitly passed `--allow-unauthenticated`. A remote plaintext peer, and a loopback plaintext peer without that flag, are refused at the config gate before any challenge is sent; `--allow-unauthenticated` never permits remote plaintext auth (remote peers still require verified TLS). Daemon modules are a `--daemon`-only feature — the SSH `--stdio` path never loads a daemon config and is not an auth transport for them. Because the loopback allowance trusts whichever peer the kernel reports as `127.0.0.1`, it assumes nothing relays remote connections to the daemon: a local TCP forwarder or TLS-terminating proxy in front of an auth-module listener makes remote clients appear as loopback and bypasses the mutual-TLS identity check, so do not front an auth-module listener with such a relay.
|
||||
- **Credential store format:** server `--password-file`/`--early-input` files are line-based `user:$fastsync$1$pbkdf2-sha256$<iters>$<salt_b64>$<stored_key_b64>$<server_key_b64>`, one per line (standard base64; 16-byte salt, 32-byte keys; `iters` in `[100000, 10000000]`, default 600000). Every entry in the resulting store must agree on `iters` (a store whose entries disagree, or where a layered `--early-input` disagrees with `--password-file`, is rejected). Generate lines with `fastsync-server --hash-credentials FILE [--iterations N]`; the emitted lines are secret material, so redirect them to an owner-only (mode 0600) file (the tool warns on stderr if stdout is a group/other-accessible regular file). Blank lines and lines starting with `#`/`;` are comments; the parser is strict (a malformed line fails the whole load, so a typo can never let a different set of users in). **The legacy `user:SHA256HEX` form is hard-rejected** with an actionable "legacy" error; there is no auto-upgrade, so a replayable bearer digest can never be loaded by a 2.19.0 daemon. The client `--password-file` holds `user:password` on its first meaningful line (the literal password, used only for the handshake then burned); keep both files readable only by their owner (mode 0600). Per-username wire length is bounded (256 chars) and every decoded salt/key length is validated. Loading the store also maintains an owner-only `<store_path>.dummykey` sidecar (auto-created, mode 0600, exactly 32 bytes) holding the store-wide dummy key that shapes unknown-user challenges; persist it across daemon restarts so those challenges stay stable, and treat a sidecar with the wrong owner, a mode other than exactly 0600, the wrong size or the wrong type as a fatal load error (fail closed). If the sidecar cannot be created (e.g. a process-substitution store path such as `/dev/fd/N`, a read-only filesystem, a missing directory, or a create/write/fsync/link/fchmod failure), the daemon logs a warning and uses a transient per-run key, so the cross-restart stability guarantee does not hold there.
|
||||
- **Plaintext caveat:** an auth-required module is refused, **before any challenge is sent**, unless the connection is encrypted and verified TLS whose client certificate matches the server's `--client-cn`, or it is plaintext from a loopback TCP peer **and** the operator passed `--allow-unauthenticated`. A remote plaintext peer, and a loopback plaintext peer without that flag, never receive a challenge, and `--allow-unauthenticated` never permits remote plaintext auth (remote peers still require verified TLS). On the loopback plaintext transport that remains permitted, a local sniffer could still read the challenge and response and mount an **offline dictionary attack** against a weak password, so use `--tls` for any real deployment. `--client-cn` matches the certificate CN only (not a subjectAltName), which is acceptable for a private CA. Clients sending daemon credentials with `--password-file` to a non-loopback daemon must use `--tls`; the client rejects such a destination before any network I/O. Unlike the old challenge-less exchange there is **no replay**: the proof is bound to the fresh per-connection server nonce, so a captured `STATUS_AUTH_RESPONSE` cannot be reused on another connection (an integration test proxies the daemon and proves this). TLS client-CN (`--client-cn`) is an independent transport identity check and composes with password auth; because `--tls` already mandates `--client-cn`, a TLS auth connection always verifies the client CN, so both checks necessarily apply together on such a connection.
|
||||
@@ -662,7 +663,7 @@ now transmits targets (the prior behavior was broken/partial); its status moved
|
||||
| `--trust-sender` | Trust remote sender's file list | ✅ Implemented | Long-form-only, receiver-local policy that never crosses the wire. The receiver skips its redundant up-front re-validation of the incoming file list (empty/`..` path rejection and the escaping-symlink-target containment), trusting the sender instead of double-checking (fewer checks, faster, potentially unsafe, matching rsync). Off by default. The low-level fd-relative confinement primitives (`file_open_secure_parent`, the O_NOFOLLOW parent walk, leaf/destination confinement) are deliberately KEPT even under `--trust-sender`, so a hostile sender still cannot write or link outside the authorized root (see Phase-5 notes below) |
|
||||
| `--old-args` | Disable modern arg protection | ✅ Implemented | SSH-only; accepted for CLI compatibility but is now a **documented no-op**: FastSync always single-quote-escapes the remote server path and each `--remote-option` value (`ssh_build_remote_command`), so a metacharacter-bearing `--rsync-path` can never be interpreted by the remote shell. The flag no longer disables that quoting (the old raw-construction behavior was an injection foot-gun and is removed); the safety-relevant behavior is identical either way |
|
||||
| `--ignore-missing-args` | Ignore missing source args | ✅ Implemented | FastSync has a single source-root argument (which always exists), so the "explicitly requested source arguments" are the `--files-from` entries and the flags only ever apply there (inert without `--files-from`, like `-R`). Without the flag a listed-but-missing entry stays a hard pre-transfer error (nothing is transferred). With it each missing entry is skipped: nothing is sent for it, it never enters the keep-set, and the run succeeds for the rest — an all-missing non-empty list succeeds transferring nothing, matching rsync. `--dirs` + `--files-from` missing entries are skipped the same way. Every skipped entry is logged and a per-run warning names the count, so the handling is never a silent no-op. Divergences: an EMPTY `--files-from` file stays a hard error in every mode (no argument was requested at all; rsync likewise reports "no source files specified"); missing-arg skipping only applies to the pre-transfer list validation, so an entry that is present at preflight and vanishes mid-transfer still fails (matching rsync, whose flag "does not affect subsequent vanished-file errors"); `--no-ignore-missing-args` is not a supported negation |
|
||||
| `--delete-missing-args` | Delete missing source args | ✅ Implemented | Implies `--ignore-missing-args` (order-independent) and additionally removes each missing entry's destination mirror receiver-side. The mirror is computed exactly like a present sibling's wire path: the bare relative entry under `-R`, otherwise the full source-mirror path below the destination root. rsync parity, verified against the man page: it does **not** imply `--delete` generally and is "independent of any other type of delete processing" — unrelated destination extras are untouched unless `--delete` is also present. Composition with `--delete` + timing: the exact-path deletions commit with the manifest, early for `--delete-before`/`--delete-during`, else only after a fully-successful transfer (delete-after/commit). A non-empty directory mirror is removed only when `--force` or `--delete` is in effect (otherwise it is left with a warning and the run continues, like rsync); an absent mirror is a no-op. An explicitly listed missing arg is a user request, not an excluded file: its deletion is never blocked by the filter-exclusion protection of excluded destination mirrors (a mirror sitting inside a filter-excluded directory is still removed). Safety/policy: gated by the server `--allow-delete` policy like `--delete`; the request paths cross the wire only in the delete-manifest frame and are confined by the same receiver validation as the keep-set (non-empty, relative, traversal-free, bounded by the per-section/per-frame manifest caps); the `--delay-updates` staging directory and basis snapshots are protected exactly as in the extras walker. Divergence: the missing-args deletions are not counted toward `--max-delete` (they are explicit per-path requests, not discovered extras). See the Phase-3 wire note below for the `PROTOCOL_VERSION` bump |
|
||||
| `--delete-missing-args` | Delete missing source args | ✅ Implemented | Implies `--ignore-missing-args` (order-independent) and additionally removes each missing entry's destination mirror receiver-side. The mirror is computed exactly like a present sibling's wire path: the bare relative entry under `-R`, otherwise the full source-mirror path below the destination root. rsync parity, verified against the man page: it does **not** imply `--delete` generally and is "independent of any other type of delete processing" — unrelated destination extras are untouched unless `--delete` is also present. Composition with `--delete` + timing: the exact-path deletions commit with the manifest, early for `--delete-before`/`--delete-during`, else only after a fully-successful transfer (delete-after/commit). A non-empty directory mirror is removed only when `--force` or `--delete` is in effect (otherwise it is left with a warning and the run continues, like rsync); an absent mirror is a no-op. `--force` is deletion authority and is therefore gated by the server `--allow-delete` policy exactly like `--delete`/`--delete-missing-args`: without it the receiver clears the flag, so a client cannot use `--force` to recursively replace or remove a destination directory tree. An explicitly listed missing arg is a user request, not an excluded file: its deletion is never blocked by the filter-exclusion protection of excluded destination mirrors (a mirror sitting inside a filter-excluded directory is still removed). Safety/policy: gated by the server `--allow-delete` policy like `--delete`; the request paths cross the wire only in the delete-manifest frame and are confined by the same receiver validation as the keep-set (non-empty, relative, traversal-free, bounded by the per-section/per-frame manifest caps); the `--delay-updates` staging directory and basis snapshots are protected exactly as in the extras walker. Divergence: the missing-args deletions are not counted toward `--max-delete` (they are explicit per-path requests, not discovered extras). See the Phase-3 wire note below for the `PROTOCOL_VERSION` bump |
|
||||
|
||||
## 16. Batch Operations
|
||||
|
||||
@@ -679,7 +680,7 @@ now transmits targets (the prior behavior was broken/partial); its status moved
|
||||
| `--stop-after=MINS` | Stop after N minutes | ✅ Implemented | Client-only sender stop deadline (Phase 6): computing `--stop-after=MINS` (a positive minute count; 0/negative/garbage rejected) and `--stop-at=TIME` (`HH:MM`, `HH:MM:SS`, or `now+N[smhd]`; a past time stops immediately). The transfer stops ELEGANTLY at the next chunk boundary: everything already fully sent is kept and applied, the run returns 0, and --delete (late/delete-after timing) does NOT wipe the destination — when the scan is cut short the partial keep-set manifest is suppressed with a warning (the delete walk is skipped rather than acting on an incomplete keep-set, so unscanned source mirrors survive). `--delete-before`/`--delete-during` still run their complete pre-scan (which ignores the deadline). Local client-only fields: never serialized into the wire config frame, so no PROTOCOL_VERSION bump. `--stop-after` uses CLOCK_MONOTONIC; `--stop-at` uses the wall clock. Works single-threaded and under `-j`/`--threads` (multithreaded). Divergence: rsync computes `--stop-after` from the run start; FastSync likewise. When both are given, the earlier of the two deadlines wins (checked per iteration). See the Phase-6 stop notes below |
|
||||
| `--stop-at=TIME` | Stop at specified time | ✅ Implemented | Same feature as `--stop-after` (deadline transfer stop), absolute wall-clock form (`HH:MM[:SS]` or `now+N[smhd]`). See the row above and the Phase-6 stop notes |
|
||||
| `--fsync` | Fsync every written file before publication | ✅ Implemented | |
|
||||
| `--protocol=NUM` | Force older protocol version | ✅ Implemented | Forces the wire protocol version for this transfer. FastSync has exactly ONE wire format (`PROTOCOL_VERSION`, currently 2.20.0) with no downgrade/backward-compat code paths, so `--protocol=2.20.0` is accepted (it sets the version claim the client sends, which the server already requires to match exactly) and **every other value is rejected up front** with a clear error before any connection — it does not and cannot speak an older or virtual wire format. Divergence from rsync (which negotiates a range and downgrades to an integer 0..31): FastSync's honest contract is force-to-the-one-supported-value; a genuine downgrade would require a per-version compatibility layer that does not exist. Client-only; the server-side exact-match check is unchanged. `--protocol=2.19.0`/`2.18.0`/`2.18`/`2.17.0`/`2.16.0`/`2.15.0`/`216`/`31`/garbage are all rejected. See the Phase-6 protocol note below |
|
||||
| `--protocol=NUM` | Force older protocol version | ✅ Implemented | Forces the wire protocol version for this transfer. FastSync has exactly ONE wire format (`PROTOCOL_VERSION`, currently 2.21.0) with no downgrade/backward-compat code paths, so `--protocol=2.21.0` is accepted (it sets the version claim the client sends, which the server already requires to match exactly) and **every other value is rejected up front** with a clear error before any connection — it does not and cannot speak an older or virtual wire format. Divergence from rsync (which negotiates a range and downgrades to an integer 0..31): FastSync's honest contract is force-to-the-one-supported-value; a genuine downgrade would require a per-version compatibility layer that does not exist. Client-only; the server-side exact-match check is unchanged. `--protocol=2.20.0`/`2.19.0`/`2.18.0`/`2.18`/`2.17.0`/`2.16.0`/`2.15.0`/`216`/`31`/garbage are all rejected. See the Phase-6 protocol note below |
|
||||
| `--iconv=CONVERT_SPEC` | Charset conversion | ✅ Implemented | Charset conversion of FILE NAMES (not content) at the protocol boundary via iconv(3): `--iconv=LOCAL[,REMOTE]` — the sender converts each local filename LOCAL→REMOTE before transmitting, and the receiver converts each wire filename REMOTE→LOCAL before creating/writing. The full CONVERT_SPEC is serialized into the config frame as a new trailing string field so the peer knows the wire charset; **PROTOCOL_VERSION bumped 2.15.0 → 2.16.0**. `LOCAL[,REMOTE]` parse: single charset ⇒ LOCAL==REMOTE (identity both ways); garbage rejected up front. Validation probes BOTH directions (a spec that only opens one way is refused, as is a NUL-emitting target charset like utf-16/utf-32/ucs-2, since filenames cannot contain NUL). An unrepresentable name (EILSEQ/EINVAL) fails that path cleanly with a logged `--iconv: cannot convert file name ...` and is never written mangled/truncated. Conversion is applied at EVERY wire-path site (regular/MKDIR/hardlink path+target/symlink path+target/SPECIAL, the delete manifest, the incremental-check path, and the `-s`/`chunk_serialize` embedded blob path), on both client and server (`--iconv` is also a server/daemon option). Zero overhead when unset. See the Phase-6 iconv notes below |
|
||||
| `--checksum-seed=NUM` | Set checksum seed | ✅ Implemented | Sets the seed for FastSync's whole-file xxHash64 digest (full 64-bit seed) and for the delta path's per-block xxHash32 strong checksum (low 32 bits of the seed). An explicit seed deterministically changes every computed digest on BOTH endpoints (sender and receiver share the seed via the config frame, protocol 2.10.0), so identical runs with the same seed skip the same files and a changed seed changes the digests — the explicit-seed path that makes xxHash comparisons deterministic. `--checksum-choice=md5` has no seed and ignores it (documented). The value is a strict decimal 0..2⁶⁴-1 (blank, signed, or non-numeric values are rejected). Like rsync, a seed only matters where a digest is actually computed (`--checksum` or a basis-dir run, or a delta transfer); it does not by itself enable `--checksum`/`--delta`. Divergence from rsync: the default is seed 0, and FastSync never randomizes the seed (rsync uses a random per-transfer seed when `--checksum-seed` is unset); FastSync's unset default therefore reproduces its historical byte-for-byte behavior |
|
||||
| `--secluded-args`, `-s` | Use protocol to send args | ⛔ Impossible/Divergence | Accepted for CLI compatibility (including the rsync short `-s`, Phase 7 Wave A) but a documented **no-op / divergence**. rsync's `-s` protects arguments from shell expansion by shipping them over the protocol; FastSync never passes remote arguments through a shell expansion boundary in the first place — its SSH transport builds the remote argv as **single-quote-escaped shell words** (`ssh_build_remote_command`), so the injection/leak that `-s` guards against does not exist and there is nothing to "seclude". Implementing a true arg-send protocol would mean replacing the argv-based SSH launch with an in-band argument channel, a large redesign of the transport that buys no security here. Chunk serialization remains the long-only `--chunk-serialization`. |
|
||||
@@ -795,7 +796,7 @@ These are the hardest compatibility items because they require durable formats o
|
||||
|
||||
**Phase 6, Wave B (iconv) shipping note (PROTOCOL 2.15.0 → 2.16.0):** `--iconv=LOCAL[,REMOTE]` converts file NAMES at the wire boundary (never content). The full CONVERT_SPEC is serialized into the config frame as a new trailing string field (empty→NULL canonicalized), so both ends share the same wire charset interpretation; this required the PROTOCOL bump because the frame is a strict ordered sequence and a peer that does not parse the new trailing field would desynchronize. Each end derives LOCAL (its own charset) and REMOTE (the wire charset): the sender opens LOCAL→REMOTE and converts every transmitted filename; the receiver opens REMOTE→LOCAL and converts every received filename before creating/writing. Conversion is applied at every wire-path site (regular/MKDIR/hardlink path+target/symlink path+target/SPECIAL, the delete manifest keep/protected/missing entries, the incremental-check path, and the embedded `-s`/chunk-blob path). A name it cannot convert (EILSEQ/EINVAL) is failed cleanly with a logged `--iconv: cannot convert file name ...` and is never written truncated/mangled. Validation probes both directions up front (both the sender local→remote and the receiver remote→local, and, for a server/daemon with its own `--iconv`, the client-REMOTE→server-LOCAL pair) so an unusable spec is rejected before the connection rather than mid-transfer, and NUL-emitting target charsets (utf-16/utf-32/ucs-2) are refused because filenames cannot contain NUL. Divergence documented upstream: the receiver does NOT half-swap; the wire charset always comes from the sender's REMOTE half, so a server whose local charset differs from the client's LOCAL must declare it with its own `--iconv`. Conversion is process-global and runs on a single thread per process (sender thread / receiver-loop thread), initialized before worker threads start and freed after they join.
|
||||
|
||||
**Phase 6, Wave C (protocol-version) shipping note (no PROTOCOL_VERSION change):** `--protocol=NUM` lets the client force the wire protocol version for a transfer. FastSync's protocol is a single lockstep format: the config frame is a strict ordered sequence and the server requires the client's version string to equal `PROTOCOL_VERSION` exactly (`config_receive_with_validate`, src/shared/config.c) — there are no older-format code paths and no downgrade/negotiation machinery, so a lower/higher/virtual version can never be spoken. The honest contract is therefore: `--protocol=2.20.0` (the current `PROTOCOL_VERSION`, as of the packed-metadata wave) is accepted and stored into the client's `version` claim (which `config_send` already transmits), and every other value — `2.19.0`, `2.18.0`, `2.18`, `2.17.0`, `2.16.0`, `2.15.0`, `3.0.0`, rsync-integer spellings like `216`/`31`, garbage, empty — is rejected up front in `validate_config()` before any connection, with a clear error that FastSync supports only its current wire protocol and cannot speak an older or virtual one. Implementation is client-only: a server-side `--protocol` is intentionally not added because the server has no negotiation (it only enforces exact match), and it could only ever be the current version. This preserves (and slightly tightens) existing validation: the client now also refuses to launch with a version it cannot actually speak, rather than only the server rejecting it later. A genuine downgrade would require a per-version compatibility layer for every frame/feature added since (append 2.10, preallocate 2.11, hardlinks 2.12, devices/specials/symlink-trust/xattr 2.13, remote-option 2.14, daemon module/auth 2.15, iconv 2.16, dir/symlink times 2.17, privilege flags --super/--copy-as 2.18, SCRAM daemon auth 2.19, packed metadata 2.20) and is intentionally out of scope — documented divergences from rsync's integer-negotiated downgrade remain.
|
||||
**Phase 6, Wave C (protocol-version) shipping note (no PROTOCOL_VERSION change):** `--protocol=NUM` lets the client force the wire protocol version for a transfer. FastSync's protocol is a single lockstep format: the config frame is a strict ordered sequence and the server requires the client's version string to equal `PROTOCOL_VERSION` exactly (`config_receive_with_validate`, src/shared/config.c) — there are no older-format code paths and no downgrade/negotiation machinery, so a lower/higher/virtual version can never be spoken. The honest contract is therefore: `--protocol=2.21.0` (the current `PROTOCOL_VERSION`, as of the combined error-detail + server-contacting dry-run wave) is accepted and stored into the client's `version` claim (which `config_send` already transmits), and every other value — `2.20.0`, `2.19.0`, `2.18.0`, `2.18`, `2.17.0`, `2.16.0`, `2.15.0`, `3.0.0`, rsync-integer spellings like `216`/`31`, garbage, empty — is rejected up front in `validate_config()` before any connection, with a clear error that FastSync supports only its current wire protocol and cannot speak an older or virtual one. Implementation is client-only: a server-side `--protocol` is intentionally not added because the server has no negotiation (it only enforces exact match), and it could only ever be the current version. This preserves (and slightly tightens) existing validation: the client now also refuses to launch with a version it cannot actually speak, rather than only the server rejecting it later. A genuine downgrade would require a per-version compatibility layer for every frame/feature added since (append 2.10, preallocate 2.11, hardlinks 2.12, devices/specials/symlink-trust/xattr 2.13, remote-option 2.14, daemon module/auth 2.15, iconv 2.16, dir/symlink times 2.17, privilege flags --super/--copy-as 2.18, SCRAM daemon auth 2.19, packed metadata 2.20) and is intentionally out of scope — documented divergences from rsync's integer-negotiated downgrade remain.
|
||||
|
||||
**Phase-1/2 selection-and-update status correction (docs):** `-I/--ignore-times`, `--size-only`, `-@/--modify-window`, `--existing`, `--ignore-existing`, `-u/--update`, `-W/--whole-file`, and `--compress-threads` were previously listed as not-implemented in this document but are in fact fully implemented and tested on `dev`. This pass corrects the matrix to match the code. The realistic model of these is that FastSync is a *sender-driven* whole-tree copy, so the size+mtime quick-check and all three receiver-policy skips (`--existing`, `--ignore-existing`, `-u`) are evaluated against the **destination** on the receiver side, and their booleans cross the wire in the config frame. `-I`/`--size-only`/`--modify-window` modify the `--incremental` per-file `STATUS_CHECK` handshake's match predicate (`-I` disables the mtime leg and forces transfer; `--size-only` drops only the mtime leg; `--modify-window` adds tolerance to `metadata_mtime_matches`); they require `--incremental` (or a basis dir) to have a handshake to affect, mirroring how they only matter where a quick-check exists in rsync. `--existing`/`--ignore-existing`/`-u` are receiver write-time policies (skipping the write / newer-destination guard) applied across the regular-file, `--delay-updates`-staged, hardlink-sibling, and special/device paths; `-u` implies `-M` metadata and uses a second-then-nanosecond strict `>` newer check; both correctly influence `--remove-source-files` (a skipped source is not removed). `-W/--whole-file` disables block-level delta (opt-in via `--delta`), folded into the wire `use_delta` so no protocol bump was needed, and makes `--fuzzy` inert; `--append`/`--append-verify` are rejected with `-W`. `--compress-threads=NUM` (1..64, client-only, never crosses the wire) sizes the zstd compression worker pool. No code was changed by this correction; the implementation had landed in earlier merge waves (feat/ignore-times, feat/ignore-existing via the newer `file_to_disk_secure_no_replace`/`linkat EEXIST` path, feat/size-only, feat/modify-window, feat/whole-file, feat/update, compression-threads).
|
||||
|
||||
@@ -832,9 +833,9 @@ These are the last compatibility items and the closing phase toward rsync flag p
|
||||
|
||||
**Wave E (LAST) — Privilege: `--super`/`--no-super` and `--copy-as=USER[:GROUP]` (✅ implemented).** FastSync adopts a **safe-subset + clear-refusal** privilege model: it never blind-elevates and never calls `setuid`/`seteuid`/`setgid`. All privileged operations remain fd-relative and confined below the authorized receive root.
|
||||
|
||||
`--super`/`--no-super` set a receiver-side tri-state `Config->super_mode` (`SUPER_MODE_AUTO`/`ON`/`OFF`). `privilege_super_permitted()` / `privilege_super_mode_permitted()` (src/shared/identity.c) return true for `ON` and `AUTO` (AUTO preserves FastSync's historical best-effort attempt, where the kernel refuses an unprivileged call and the caller skips it) and false only for `OFF`. The gate covers every super-user activity FastSync performs: ownership application (`identity_apply_ownership`/`_link`), char/block device-node creation (`file_save_special_to_disk`), writes into an existing device (`--write-devices`), and the `--fake-super` owner replay. Unprivileged FIFO creation is deliberately unaffected. `--super` does **not** imply `--numeric-ids`: ownership is applied only when an explicit identity policy (`--usermap`/`--groupmap`/`--chown`/`--numeric-ids`/`--copy-as`) is also given. `--no-super` suppresses those activities even for a root receiver. A non-root receiver given `--super` logs one warning at activation (`identity_set_active`); each confined attempt is then refused by the kernel and skipped, never aborting. The confinement floor is unchanged (`file_open_secure_parent`, `O_NOFOLLOW`, root/path checks). Operator control: the server CLI accepts `--no-super`, a veto that forces `OFF` for every connection, refuses any client `--copy-as`, and neutralizes an explicit `--super` (the connection is accepted but no super-user activity is attempted). On a daemon, a module that has not opted in with `client owner = yes` additionally has super-user device activity forced off (see the Daemon Mode notes).
|
||||
`--super`/`--no-super` set a receiver-side tri-state `Config->super_mode` (`SUPER_MODE_AUTO`/`ON`/`OFF`). `privilege_super_permitted()` / `privilege_super_mode_permitted()` (src/shared/identity.c) return true for `ON` and `AUTO` (AUTO preserves FastSync's historical best-effort attempt, where the kernel refuses an unprivileged call and the caller skips it) and false only for `OFF`. The gate covers every super-user activity FastSync performs: ownership application (`identity_apply_ownership`/`_link`), char/block device-node creation (`file_save_special_to_disk`), writes into an existing device (`--write-devices`), and the `--fake-super` owner replay. Unprivileged FIFO creation is deliberately unaffected. `--super` does **not** imply `--numeric-ids`: ownership is applied only when an explicit identity policy (`--usermap`/`--groupmap`/`--chown`/`--numeric-ids`/`--copy-as`) is also given. `--no-super` suppresses those activities even for a root receiver. A non-root receiver given `--super` logs one warning at activation (`identity_set_active`); each confined attempt is then refused by the kernel and skipped, never aborting. The confinement floor is unchanged (`file_open_secure_parent`, `O_NOFOLLOW`, root/path checks). Operator control: the server CLI accepts `--no-super`, a veto that forces `OFF` for every connection, refuses any client `--copy-as`, and neutralizes an explicit `--super` (the connection is accepted but no super-user activity is attempted). A privileged (root) standalone TCP listener instead defaults to `OFF` and requires the server-only `--allow-super` opt-in to attempt any super-user activity (the flag is rejected with `--stdio`, whose client-composed remote argv must never defeat the default; use a forced command if the default must hold); a non-root server is unchanged. On a daemon, a module that has not opted in with `client owner = yes` additionally has super-user device activity forced off (see the Daemon Mode notes).
|
||||
|
||||
`--copy-as=USER[:GROUP]` is the safe subset. FastSync's receiver is multithreaded, so a real credential switch is unsafe; instead the receiver forces the ownership of **every entry it writes** — regular files, symlinks, directories (including implicitly-created parents), and special nodes — to the resolved target ids through the confined fd-relative identity path. USER is resolved on the client (name, `@N`/bare N, or `*` = client euid); when `:GROUP` is omitted the user's primary gid is used (falling back to `gid == uid` for a numeric id with no local passwd entry). It requires a privileged (root) receiver: an unprivileged receiver refuses the whole transfer at the config handshake, before `STATUS_OK`, so no data is ever written with the wrong ownership. A `--copy-as` chown failure on a capability-restricted root is logged at ERROR (never silently downgraded). `--copy-as` implies metadata (`--no-preserve` is rejected) and `--fake-super` cannot override it. Daemon policy: a `--daemon` receiver refuses **every** client-chosen-ownership / super-user request — `--numeric-ids`, `--chown`, `--usermap`/`--groupmap`, `--fake-super`, `--copy-as`, and explicit `--super` — unless the selected module opts in with `client owner = yes`; without that per-module opt-in any client could force arbitrary ownership inside the module root (the standalone listener and the SSH-launched `--stdio` server, which each serve one operator-authorized root, honor these requests). A `--copy-as` chown failure on a capability-restricted root marks the entry as failed rather than reporting success with the wrong owner.
|
||||
`--copy-as=USER[:GROUP]` is the safe subset. FastSync's receiver is multithreaded, so a real credential switch is unsafe; instead the receiver forces the ownership of **every entry it writes** — regular files, symlinks, directories (including implicitly-created parents), and special nodes — to the resolved target ids through the confined fd-relative identity path. USER is resolved on the client (name, `@N`/bare N, or `*` = client euid); when `:GROUP` is omitted the user's primary gid is used (falling back to `gid == uid` for a numeric id with no local passwd entry). It requires a privileged (root) receiver: an unprivileged receiver refuses the whole transfer at the config handshake, before `STATUS_OK`, so no data is ever written with the wrong ownership. A `--copy-as` chown failure on a capability-restricted root is logged at ERROR (never silently downgraded). `--copy-as` implies metadata (`--no-preserve` is rejected) and `--fake-super` cannot override it. Daemon policy: a `--daemon` receiver refuses **every** client-chosen-ownership / super-user request — `--numeric-ids`, `--chown`, `--usermap`/`--groupmap`, `--fake-super`, `--copy-as`, and explicit `--super` — unless the selected module opts in with `client owner = yes`; without that per-module opt-in any client could force arbitrary ownership inside the module root (a root standalone TCP listener, which serves one operator-authorized root, honors these requests only when started with `--allow-super`; the flag is rejected with `--stdio`). A `--copy-as` chown failure on a capability-restricted root marks the entry as failed rather than reporting success with the wrong owner.
|
||||
|
||||
**Wire:** two trailing config-frame blocks after the `--iconv` spec, in fixed order — `send_privilege_options`/`receive_privilege_options` (one `super_mode` int, validated `0..2`), then `send_copy_as_options`/`receive_copy_as_options` (presence int + two int32 ids, validated `>= 0`, with `copy_as_set ⇒ use_metadata`). `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Divergences from rsync:** rsync's `--super` elevates the receiver and `--copy-as` actually switches its credentials; FastSync never elevates and only permits/forwards confined attempts, and `--copy-as` forces ownership rather than switching identity.
|
||||
|
||||
|
||||
+397
-60
@@ -3,19 +3,24 @@
|
||||
|
||||
Compares FastSync configs against rsync (no compression) and rsync+zstd.
|
||||
Data is ~75% random/incompressible and ~25% structured/compressible by default,
|
||||
controllable via --random-ratio.
|
||||
controllable via --random-ratio. Transfers are verified by default (source and
|
||||
destination must match) so a fast-but-broken copy is never counted.
|
||||
|
||||
Usage:
|
||||
python3 benchmark/bench.py
|
||||
python3 benchmark/bench.py --runs 5 --profiles lan wan
|
||||
python3 benchmark/bench.py --random-ratio 0.5 --size-mb 50
|
||||
python3 benchmark/bench.py --delay 50ms --jitter 10ms --throughput 100mbit
|
||||
python3 benchmark/bench.py --warm --runs 3
|
||||
python3 benchmark/bench.py --output json
|
||||
"""
|
||||
import argparse
|
||||
import filecmp
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import random
|
||||
import shlex
|
||||
import shutil
|
||||
import socket
|
||||
import statistics
|
||||
@@ -25,7 +30,10 @@ import tempfile
|
||||
import time
|
||||
|
||||
PROJECT_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
|
||||
BUILD_DIR = os.path.join(PROJECT_ROOT, "build")
|
||||
DEFAULT_BUILD_DIR = "build-bench"
|
||||
# Populated by configure_build_dirs(); default to the dedicated bench dir so
|
||||
# importing this module never depends on the user's existing build/ tree.
|
||||
BUILD_DIR = os.path.join(PROJECT_ROOT, DEFAULT_BUILD_DIR)
|
||||
SERVER_CMD = [os.path.join(BUILD_DIR, "server"), "--allow-unauthenticated"]
|
||||
CLIENT_CMD = [os.path.join(BUILD_DIR, "client")]
|
||||
BENCH_DIR = os.path.join(PROJECT_ROOT, "bench_data")
|
||||
@@ -57,6 +65,7 @@ RSYNC_CONFIGS = [
|
||||
{"name": "rsync -z --zstd", "flags": ["-z", "--zc", "zstd"],"tool": "rsync"},
|
||||
]
|
||||
|
||||
|
||||
class RsyncDaemon:
|
||||
"""Manages an rsync daemon for network-fair benchmarking."""
|
||||
|
||||
@@ -118,6 +127,12 @@ STRUCTURED_FILES = {
|
||||
"nested/another.txt": b"another nested file\n" * 50,
|
||||
}
|
||||
|
||||
# Repeated text used to synthesize genuinely compressible filler of any size.
|
||||
COMPRESSIBLE_TEXT = (
|
||||
b"FastSync benchmark payload: the quick brown fox jumps over the lazy dog. "
|
||||
b"0123456789 ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz\n"
|
||||
)
|
||||
|
||||
|
||||
class Progress:
|
||||
"""Simple progress bar with ETA."""
|
||||
@@ -152,35 +167,127 @@ class Progress:
|
||||
sys.stderr.flush()
|
||||
|
||||
|
||||
def write_compressible(path, nbytes):
|
||||
"""Write exactly nbytes of highly compressible, repeated text content."""
|
||||
if nbytes <= 0:
|
||||
return
|
||||
block = COMPRESSIBLE_TEXT * (max(1, 8192 // len(COMPRESSIBLE_TEXT)) + 1)
|
||||
remaining = nbytes
|
||||
with open(path, "wb") as f:
|
||||
while remaining > 0:
|
||||
piece = block if remaining >= len(block) else block[:remaining]
|
||||
f.write(piece)
|
||||
remaining -= len(piece)
|
||||
|
||||
|
||||
def generate_bench_data(source_dir, size_mb=25, random_ratio=0.75):
|
||||
"""Generate test data. ~random_ratio is incompressible, rest is structured."""
|
||||
"""Generate test data honouring the requested random/compressible split.
|
||||
|
||||
Exactly ``random_ratio * target`` bytes are incompressible random data and
|
||||
the remainder is genuinely compressible structured/repeated content. The
|
||||
measured byte counts are returned so callers can report the real mix.
|
||||
"""
|
||||
if os.path.exists(source_dir):
|
||||
shutil.rmtree(source_dir)
|
||||
os.makedirs(source_dir)
|
||||
|
||||
target = size_mb * 1024 * 1024
|
||||
structured_budget = int(target * (1 - random_ratio))
|
||||
written = 0
|
||||
random_budget = int(target * random_ratio)
|
||||
compressible_budget = target - random_budget
|
||||
compressible_written = 0
|
||||
random_written = 0
|
||||
files = 0
|
||||
|
||||
# A handful of fixed, human-meaningful files (directories, small files, a
|
||||
# binary blob) as long as they fit inside the compressible budget.
|
||||
for rel_path, content in STRUCTURED_FILES.items():
|
||||
if written >= structured_budget:
|
||||
if compressible_written + len(content) > compressible_budget:
|
||||
break
|
||||
full_path = os.path.join(source_dir, rel_path)
|
||||
os.makedirs(os.path.dirname(full_path), exist_ok=True)
|
||||
with open(full_path, "wb") as f:
|
||||
f.write(content)
|
||||
written += len(content)
|
||||
compressible_written += len(content)
|
||||
files += 1
|
||||
|
||||
os.makedirs(os.path.join(source_dir, "bulk"), exist_ok=True)
|
||||
# Fill the rest of the compressible share with generated repeated content.
|
||||
if compressible_written < compressible_budget:
|
||||
os.makedirs(os.path.join(source_dir, "compressible"), exist_ok=True)
|
||||
i = 0
|
||||
while written < target:
|
||||
chunk_size = min(5 * 1024 * 1024, target - written)
|
||||
with open(os.path.join(source_dir, f"bulk/file_{i}.dat"), "wb") as f:
|
||||
f.write(random.randbytes(chunk_size))
|
||||
written += chunk_size
|
||||
while compressible_written < compressible_budget:
|
||||
chunk = min(1024 * 1024, compressible_budget - compressible_written)
|
||||
write_compressible(os.path.join(source_dir, "compressible", f"text_{i}.dat"), chunk)
|
||||
compressible_written += chunk
|
||||
files += 1
|
||||
i += 1
|
||||
|
||||
return written
|
||||
# Incompressible share.
|
||||
if random_written < random_budget:
|
||||
os.makedirs(os.path.join(source_dir, "bulk"), exist_ok=True)
|
||||
i = 0
|
||||
while random_written < random_budget:
|
||||
chunk = min(5 * 1024 * 1024, random_budget - random_written)
|
||||
with open(os.path.join(source_dir, "bulk", f"file_{i}.dat"), "wb") as f:
|
||||
f.write(random.randbytes(chunk))
|
||||
random_written += chunk
|
||||
files += 1
|
||||
i += 1
|
||||
|
||||
return {
|
||||
"total_bytes": compressible_written + random_written,
|
||||
"compressible_bytes": compressible_written,
|
||||
"random_bytes": random_written,
|
||||
"files": files,
|
||||
}
|
||||
|
||||
|
||||
def list_relative_files(root):
|
||||
"""Return the set of file paths (relative to root) under a directory."""
|
||||
found = set()
|
||||
for dirpath, _dirnames, filenames in os.walk(root):
|
||||
for name in filenames:
|
||||
full = os.path.join(dirpath, name)
|
||||
found.add(os.path.relpath(full, root))
|
||||
return found
|
||||
|
||||
|
||||
def verify_transfer(source_dir, dest_dir):
|
||||
"""Recursively check dest matches source (paths, sizes, content).
|
||||
|
||||
Returns (ok, detail). Content is compared byte-for-byte, never hashed, so
|
||||
collisions are impossible. This is intentionally not part of the timing.
|
||||
"""
|
||||
if not os.path.isdir(dest_dir):
|
||||
return False, "destination directory missing"
|
||||
src_files = list_relative_files(source_dir)
|
||||
dst_files = list_relative_files(dest_dir)
|
||||
if src_files != dst_files:
|
||||
missing = src_files - dst_files
|
||||
extra = dst_files - src_files
|
||||
return False, f"path set mismatch (missing {len(missing)}, extra {len(extra)})"
|
||||
for rel in sorted(src_files):
|
||||
src = os.path.join(source_dir, rel)
|
||||
dst = os.path.join(dest_dir, rel)
|
||||
if os.path.getsize(src) != os.path.getsize(dst):
|
||||
return False, f"size mismatch: {rel}"
|
||||
if not filecmp.cmp(src, dst, shallow=False):
|
||||
return False, f"content mismatch: {rel}"
|
||||
return True, ""
|
||||
|
||||
|
||||
def percentile(values, pct):
|
||||
"""Linear-interpolation percentile (matches numpy's default method)."""
|
||||
if not values:
|
||||
return None
|
||||
ordered = sorted(values)
|
||||
if len(ordered) == 1:
|
||||
return ordered[0]
|
||||
rank = (len(ordered) - 1) * (pct / 100.0)
|
||||
low = math.floor(rank)
|
||||
high = math.ceil(rank)
|
||||
if low == high:
|
||||
return ordered[int(rank)]
|
||||
return ordered[low] + (ordered[high] - ordered[low]) * (rank - low)
|
||||
|
||||
|
||||
def find_free_port():
|
||||
@@ -208,18 +315,45 @@ def wait_proc(proc, timeout=5):
|
||||
proc.wait()
|
||||
|
||||
|
||||
def _tc_base_cmd():
|
||||
"""Return the command prefix for tc, honouring root vs sudo."""
|
||||
tc = shutil.which("tc")
|
||||
if not tc:
|
||||
raise RuntimeError(
|
||||
"tc (iproute2) not found in PATH; install iproute2 to use network profiles")
|
||||
if os.geteuid() == 0:
|
||||
return [tc]
|
||||
sudo = shutil.which("sudo")
|
||||
if sudo:
|
||||
return [sudo, tc]
|
||||
raise RuntimeError(
|
||||
"applying network limits requires root or sudo; "
|
||||
"re-run as root or install sudo")
|
||||
|
||||
|
||||
def _run_tc(args, check=True):
|
||||
return subprocess.run(_tc_base_cmd() + args, check=check, capture_output=True)
|
||||
|
||||
|
||||
def netem_apply(delay=None, jitter=None, throughput=None, loss=None):
|
||||
"""Apply tc/netem rules to loopback. Pass None to skip a parameter."""
|
||||
netem_reset()
|
||||
cmd = ["sudo", "tc", "qdisc", "add", "dev", "lo", "root", "netem"]
|
||||
params = []
|
||||
if throughput:
|
||||
cmd += ["rate", throughput]
|
||||
params += ["rate", throughput]
|
||||
if delay:
|
||||
cmd += ["delay", delay, jitter or "0ms"]
|
||||
params += ["delay", delay, jitter or "0ms"]
|
||||
if loss:
|
||||
cmd += ["loss", loss]
|
||||
if len(cmd) > 6:
|
||||
subprocess.run(cmd, check=True, capture_output=True)
|
||||
params += ["loss", loss]
|
||||
if not params:
|
||||
return
|
||||
try:
|
||||
_run_tc(["qdisc", "add", "dev", "lo", "root", "netem"] + params)
|
||||
except subprocess.CalledProcessError as exc:
|
||||
detail = exc.stderr.decode(errors="replace").strip() if exc.stderr else str(exc)
|
||||
raise RuntimeError(f"failed to apply network profile via tc/netem: {detail}") from exc
|
||||
except RuntimeError:
|
||||
raise
|
||||
|
||||
|
||||
def netem_apply_profile(profile_name):
|
||||
@@ -236,7 +370,11 @@ def netem_apply_profile(profile_name):
|
||||
|
||||
|
||||
def netem_reset():
|
||||
subprocess.run("sudo tc qdisc del dev lo root".split(), capture_output=True)
|
||||
"""Best-effort removal of any loopback qdisc. Always safe to call."""
|
||||
try:
|
||||
_run_tc(["qdisc", "del", "dev", "lo", "root"], check=False)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def run_fastsync(source_dir, dest_dir, flags, port):
|
||||
@@ -287,7 +425,81 @@ def run_transfer(config, source_dir, dest_dir, port=None, rsync_daemon=None):
|
||||
return run_fastsync(source_dir, dest_dir, config["flags"], port)
|
||||
|
||||
|
||||
def run_benchmark(source_dir, dest_dir, configs, runs, profile_name, progress=None):
|
||||
def apply_incremental_changes(source_dir, target_bytes):
|
||||
"""Add and modify a few files so a warm transfer has real work to do.
|
||||
|
||||
Returns a mutation record (changed byte count plus enough data to revert
|
||||
and re-apply it) so every warm run can start from a pristine source.
|
||||
"""
|
||||
modified_n = 3
|
||||
added_n = 2
|
||||
per_file = max(4096, target_bytes // (modified_n + added_n))
|
||||
modified = {}
|
||||
added = {}
|
||||
changed = 0
|
||||
|
||||
existing = sorted(list_relative_files(source_dir))
|
||||
if existing:
|
||||
step = max(1, len(existing) // modified_n)
|
||||
for rel in existing[::step][:modified_n]:
|
||||
path = os.path.join(source_dir, rel)
|
||||
original_size = os.path.getsize(path)
|
||||
with open(path, "ab") as f:
|
||||
f.write(random.randbytes(per_file))
|
||||
modified[rel] = (original_size, per_file)
|
||||
changed += per_file
|
||||
|
||||
for i in range(added_n):
|
||||
os.makedirs(os.path.join(source_dir, "incremental"), exist_ok=True)
|
||||
rel = os.path.join("incremental", f"new_{i}.dat")
|
||||
write_compressible(os.path.join(source_dir, rel), per_file)
|
||||
added[rel] = per_file
|
||||
changed += per_file
|
||||
|
||||
return {"changed": changed, "modified": modified, "added": added}
|
||||
|
||||
|
||||
def revert_incremental_changes(source_dir, mutation):
|
||||
"""Undo apply_incremental_changes so the source is pristine again."""
|
||||
if not mutation:
|
||||
return
|
||||
for rel, (original_size, _appended) in mutation["modified"].items():
|
||||
path = os.path.join(source_dir, rel)
|
||||
if os.path.exists(path):
|
||||
with open(path, "r+b") as f:
|
||||
f.truncate(original_size)
|
||||
for rel in mutation["added"]:
|
||||
path = os.path.join(source_dir, rel)
|
||||
if os.path.exists(path):
|
||||
os.remove(path)
|
||||
|
||||
|
||||
def reapply_incremental_changes(source_dir, mutation):
|
||||
"""Re-apply a mutation after an untimed pristine seed transfer."""
|
||||
if not mutation:
|
||||
return
|
||||
for rel, (_original_size, appended) in mutation["modified"].items():
|
||||
with open(os.path.join(source_dir, rel), "ab") as f:
|
||||
f.write(random.randbytes(appended))
|
||||
for rel, size in mutation["added"].items():
|
||||
write_compressible(os.path.join(source_dir, rel), size)
|
||||
|
||||
|
||||
def expected_received_root(dest_dir, source_dir, tool):
|
||||
"""Where a tool places transferred files inside dest_dir.
|
||||
|
||||
FastSync mirrors the absolute source path under dest_dir (see the
|
||||
integration suite's get_dest_received_dir); rsync copies the source tree
|
||||
contents directly into dest_dir.
|
||||
"""
|
||||
if tool == "rsync":
|
||||
return dest_dir
|
||||
return os.path.join(dest_dir, os.path.abspath(source_dir).lstrip(os.sep))
|
||||
|
||||
|
||||
def run_benchmark(source_dir, dest_dir, configs, runs, profile_name,
|
||||
measure_bytes, verify=True, warm=False, mutation=None,
|
||||
progress=None):
|
||||
"""Run benchmark for all configs, returns list of results."""
|
||||
is_limited = profile_name != "unlimited"
|
||||
has_rsync = any(c["tool"] == "rsync" for c in configs)
|
||||
@@ -303,7 +515,10 @@ def run_benchmark(source_dir, dest_dir, configs, runs, profile_name, progress=No
|
||||
results = []
|
||||
for config in configs:
|
||||
times = []
|
||||
invalid = 0
|
||||
for run_idx in range(runs):
|
||||
if warm:
|
||||
revert_incremental_changes(source_dir, mutation)
|
||||
if os.path.exists(dest_dir):
|
||||
shutil.rmtree(dest_dir)
|
||||
os.makedirs(dest_dir, exist_ok=True)
|
||||
@@ -311,15 +526,33 @@ def run_benchmark(source_dir, dest_dir, configs, runs, profile_name, progress=No
|
||||
port = find_free_port()
|
||||
server = None
|
||||
try:
|
||||
if config["tool"] == "fastsync":
|
||||
if config["tool"] == "fastsync" or warm:
|
||||
server = subprocess.Popen(
|
||||
SERVER_CMD + ["-p", str(port)],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
|
||||
)
|
||||
wait_for_port(port)
|
||||
|
||||
if warm:
|
||||
seed = run_transfer(config, source_dir, dest_dir, port, rsync_daemon)
|
||||
if seed is None:
|
||||
invalid += 1
|
||||
sys.stderr.write(" warm-mode seeding failed; run not counted\n")
|
||||
continue
|
||||
reapply_incremental_changes(source_dir, mutation)
|
||||
|
||||
t = run_transfer(config, source_dir, dest_dir, port, rsync_daemon)
|
||||
if t is not None:
|
||||
if t is None:
|
||||
invalid += 1
|
||||
elif verify:
|
||||
root = expected_received_root(dest_dir, source_dir, config["tool"])
|
||||
ok, detail = verify_transfer(source_dir, root)
|
||||
if ok:
|
||||
times.append(t)
|
||||
else:
|
||||
invalid += 1
|
||||
sys.stderr.write(f" verification FAILED ({detail}); run not counted\n")
|
||||
else:
|
||||
times.append(t)
|
||||
finally:
|
||||
if server:
|
||||
@@ -332,15 +565,22 @@ def run_benchmark(source_dir, dest_dir, configs, runs, profile_name, progress=No
|
||||
"config": config["name"],
|
||||
"tool": config["tool"],
|
||||
"profile": profile_name,
|
||||
"warm": warm,
|
||||
"runs": len(times),
|
||||
"invalid": invalid,
|
||||
"times": [round(t, 4) for t in times],
|
||||
}
|
||||
if times:
|
||||
entry["p50"] = round(statistics.median(times), 4)
|
||||
entry["p95"] = round(sorted(times)[int(len(times) * 0.95)], 4) if len(times) > 1 else entry["p50"]
|
||||
p50 = percentile(times, 50)
|
||||
p95 = percentile(times, 95)
|
||||
entry["p50"] = round(p50, 4)
|
||||
entry["p95"] = round(p95, 4)
|
||||
entry["min"] = round(min(times), 4)
|
||||
entry["max"] = round(max(times), 4)
|
||||
entry["stdev"] = round(statistics.stdev(times), 4) if len(times) > 1 else 0.0
|
||||
if measure_bytes:
|
||||
entry["throughput_mbps"] = round(
|
||||
(measure_bytes / (1024 * 1024)) / p50, 3)
|
||||
results.append(entry)
|
||||
return results
|
||||
finally:
|
||||
@@ -350,44 +590,59 @@ def run_benchmark(source_dir, dest_dir, configs, runs, profile_name, progress=No
|
||||
netem_reset()
|
||||
|
||||
|
||||
def print_table(results, total_bytes, random_ratio):
|
||||
def print_table(results, measure_bytes, stats, warm):
|
||||
"""Print results as a human-readable table grouped by profile."""
|
||||
profiles = {}
|
||||
for r in results:
|
||||
profiles.setdefault(r["profile"], []).append(r)
|
||||
|
||||
total = stats["total_bytes"]
|
||||
comp_pct = stats["compressible_bytes"] / total * 100 if total else 0
|
||||
rand_pct = stats["random_bytes"] / total * 100 if total else 0
|
||||
|
||||
for profile, entries in profiles.items():
|
||||
params = NETWORK_PROFILES.get(profile, {})
|
||||
print(f"\n{'=' * 85}")
|
||||
print(f"\n{'=' * 95}")
|
||||
print(f" Profile: {profile.upper()}")
|
||||
if params.get("rate"):
|
||||
print(f" Network: {params['rate']}, {params['delay']} +/- {params['jitter']}, loss {params['loss']}")
|
||||
else:
|
||||
print(f" Network: unlimited")
|
||||
print(f" Data: {total_bytes / (1024*1024):.1f} MB ({random_ratio*100:.0f}% random, {(1-random_ratio)*100:.0f}% compressible)")
|
||||
print(f"{'=' * 85}")
|
||||
print(f" Data: {total / (1024*1024):.1f} MB "
|
||||
f"({rand_pct:.0f}% random, {comp_pct:.0f}% compressible actual)")
|
||||
if warm:
|
||||
print(f" Mode: warm (incremental) — measured {measure_bytes / (1024*1024):.2f} MB "
|
||||
f"changed after an untimed full seed")
|
||||
else:
|
||||
print(" Mode: cold (full copy)")
|
||||
print(f"{'=' * 95}")
|
||||
|
||||
fs_entries = [e for e in entries if e.get("tool") == "fastsync"]
|
||||
rsync_entries = [e for e in entries if e.get("tool") == "rsync"]
|
||||
|
||||
header = (f" {'Config':<38} {'p50':>8} {'p95':>8} {'min':>8} {'max':>8} "
|
||||
f"{'stdev':>8} {'MB/s':>9} {'runs':>5} {'bad':>4}")
|
||||
rule = (f" {'-' * 38} {'-' * 8} {'-' * 8} {'-' * 8} {'-' * 8} "
|
||||
f"{'-' * 8} {'-' * 9} {'-' * 5} {'-' * 4}")
|
||||
|
||||
if fs_entries:
|
||||
print(f"\n FastSync:")
|
||||
print(f" {'Config':<38} {'p50':>8} {'p95':>8} {'min':>8} {'max':>8} {'stdev':>8} {'runs':>5}")
|
||||
print(f" {'-' * 38} {'-' * 8} {'-' * 8} {'-' * 8} {'-' * 8} {'-' * 8} {'-' * 5}")
|
||||
print(header)
|
||||
print(rule)
|
||||
for e in sorted(fs_entries, key=lambda x: x.get("p50", 999)):
|
||||
_print_entry(e)
|
||||
|
||||
if rsync_entries:
|
||||
print(f"\n rsync:")
|
||||
print(f" {'Config':<38} {'p50':>8} {'p95':>8} {'min':>8} {'max':>8} {'stdev':>8} {'runs':>5}")
|
||||
print(f" {'-' * 38} {'-' * 8} {'-' * 8} {'-' * 8} {'-' * 8} {'-' * 8} {'-' * 5}")
|
||||
print(header)
|
||||
print(rule)
|
||||
for e in sorted(rsync_entries, key=lambda x: x.get("p50", 999)):
|
||||
_print_entry(e)
|
||||
|
||||
if params.get("rate_bps") and fs_entries and rsync_entries:
|
||||
fs_best = min((e["p50"] for e in fs_entries if "p50" in e), default=None)
|
||||
rsync_best = min((e["p50"] for e in rsync_entries if "p50" in e), default=None)
|
||||
theoretical = total_bytes / params["rate_bps"]
|
||||
theoretical = measure_bytes / params["rate_bps"]
|
||||
if fs_best and rsync_best:
|
||||
print(f"\n Theoretical max (line rate): {theoretical:.4f}s")
|
||||
print(f" FastSync best: {fs_best:.4f}s ({theoretical/fs_best:.2f}x vs line rate)")
|
||||
@@ -397,10 +652,43 @@ def print_table(results, total_bytes, random_ratio):
|
||||
|
||||
def _print_entry(e):
|
||||
if "p50" in e:
|
||||
tp = f"{e['throughput_mbps']:.2f}" if "throughput_mbps" in e else "N/A"
|
||||
print(f" {e['config']:<38} {e['p50']:>7.4f}s {e['p95']:>7.4f}s "
|
||||
f"{e['min']:>7.4f}s {e['max']:>7.4f}s {e['stdev']:>7.4f} {e['runs']:>5}")
|
||||
f"{e['min']:>7.4f}s {e['max']:>7.4f}s {e['stdev']:>7.4f} "
|
||||
f"{tp:>9} {e['runs']:>5} {e.get('invalid', 0):>4}")
|
||||
else:
|
||||
print(f" {e['config']:<38} {'N/A':>8} {'N/A':>8} {'N/A':>8} {'N/A':>8} {'N/A':>8} {e['runs']:>5}")
|
||||
print(f" {e['config']:<38} {'N/A':>8} {'N/A':>8} {'N/A':>8} {'N/A':>8} "
|
||||
f"{'N/A':>8} {'N/A':>9} {e['runs']:>5} {e.get('invalid', 0):>4}")
|
||||
|
||||
|
||||
def configure_build_dirs(build_dir):
|
||||
"""Install the selected build directory and derived binary paths."""
|
||||
global BUILD_DIR, SERVER_CMD, CLIENT_CMD
|
||||
if not os.path.isabs(build_dir):
|
||||
build_dir = os.path.join(PROJECT_ROOT, build_dir)
|
||||
BUILD_DIR = os.path.abspath(build_dir)
|
||||
SERVER_CMD = [os.path.join(BUILD_DIR, "server"), "--allow-unauthenticated"]
|
||||
CLIENT_CMD = [os.path.join(BUILD_DIR, "client")]
|
||||
|
||||
|
||||
def build_project():
|
||||
"""Configure (Release) and build into the dedicated bench build dir."""
|
||||
if shutil.which("cmake") is None:
|
||||
sys.stderr.write("cmake not found in PATH; cannot build\n")
|
||||
sys.exit(1)
|
||||
os.makedirs(BUILD_DIR, exist_ok=True)
|
||||
configure = ["cmake", "-B", BUILD_DIR, "-S", PROJECT_ROOT,
|
||||
"-DCMAKE_BUILD_TYPE=Release"]
|
||||
result = subprocess.run(configure, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
sys.stderr.write("CMake configure failed:\n" + result.stdout + result.stderr + "\n")
|
||||
sys.exit(1)
|
||||
jobs = str(os.cpu_count() or 1)
|
||||
result = subprocess.run(["cmake", "--build", BUILD_DIR, "-j", jobs],
|
||||
capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
sys.stderr.write("Build failed:\n" + result.stdout + result.stderr + "\n")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def main():
|
||||
@@ -417,12 +705,20 @@ Custom network limits (--delay/--jitter/--throughput) override profiles.
|
||||
|
||||
Data mix:
|
||||
Default is ~75%% random/incompressible + ~25%% structured/compressible,
|
||||
reflecting typical real-world file sets.
|
||||
reflecting typical real-world file sets. The actual mix is measured and
|
||||
reported. Transfers are verified (destination must match source) unless
|
||||
--no-verify is given.
|
||||
|
||||
Warm mode:
|
||||
--warm seeds the destination with an untimed full copy of a pristine base,
|
||||
then measures only the incremental transfer after modifying a few files.
|
||||
|
||||
Examples:
|
||||
%(prog)s --profiles wan --runs 5
|
||||
%(prog)s --throughput 50mbit --delay 30ms --jitter 5ms
|
||||
%(prog)s --random-ratio 0.5 --size-mb 100
|
||||
%(prog)s --warm --runs 3 --no-rsync
|
||||
%(prog)s --dry-run --size-mb 4 --random-ratio 0.25
|
||||
""")
|
||||
parser.add_argument("--runs", type=int, default=3,
|
||||
help="Number of runs per config (default: 3)")
|
||||
@@ -430,7 +726,8 @@ Examples:
|
||||
choices=list(NETWORK_PROFILES.keys()),
|
||||
help="Predefined network profiles (default: unlimited)")
|
||||
parser.add_argument("--configs", nargs="+", default=None,
|
||||
help="Custom FastSync config flags")
|
||||
help="Custom FastSync config flags (shell-quoted, e.g. "
|
||||
"\"-j -z --chunk-serialization\")")
|
||||
parser.add_argument("--size-mb", type=int, default=25,
|
||||
help="Test data size in MB (default: 25)")
|
||||
parser.add_argument("--random-ratio", type=float, default=0.75,
|
||||
@@ -445,6 +742,14 @@ Examples:
|
||||
help="Custom packet loss (e.g. 1%%)")
|
||||
parser.add_argument("--no-rsync", action="store_true",
|
||||
help="Skip rsync comparison")
|
||||
parser.add_argument("--no-verify", action="store_true",
|
||||
help="Skip source/destination verification after each run")
|
||||
parser.add_argument("--warm", action="store_true",
|
||||
help="Incremental mode: seed dest first, measure only changes")
|
||||
parser.add_argument("--build-dir", default=DEFAULT_BUILD_DIR,
|
||||
help=f"Build directory (default: {DEFAULT_BUILD_DIR})")
|
||||
parser.add_argument("--dry-run", action="store_true",
|
||||
help="Only generate data and report its composition, then exit")
|
||||
parser.add_argument("--progress", action="store_true",
|
||||
help="Show progress bar with ETA")
|
||||
parser.add_argument("--output", choices=["table", "json"], default="table",
|
||||
@@ -453,14 +758,48 @@ Examples:
|
||||
help="Don't clean up test data")
|
||||
args = parser.parse_args()
|
||||
|
||||
if not 0.0 <= args.random_ratio <= 1.0:
|
||||
parser.error("--random-ratio must be between 0.0 and 1.0")
|
||||
if args.size_mb <= 0:
|
||||
parser.error("--size-mb must be positive")
|
||||
|
||||
configure_build_dirs(args.build_dir)
|
||||
|
||||
# Generate data
|
||||
source_dir = os.path.join(BENCH_DIR, "source")
|
||||
dest_dir = os.path.join(BENCH_DIR, "dest")
|
||||
stats = generate_bench_data(source_dir, args.size_mb, args.random_ratio)
|
||||
total_bytes = stats["total_bytes"]
|
||||
comp_pct = stats["compressible_bytes"] / total_bytes * 100 if total_bytes else 0
|
||||
rand_pct = stats["random_bytes"] / total_bytes * 100 if total_bytes else 0
|
||||
print(f"Generated {total_bytes / (1024*1024):.1f} MB in {stats['files']} files "
|
||||
f"({rand_pct:.0f}% random, {comp_pct:.0f}% compressible actual)",
|
||||
file=sys.stderr)
|
||||
|
||||
if args.dry_run:
|
||||
print(f"size_mb={args.size_mb} random_ratio={args.random_ratio:.4f} "
|
||||
f"total_bytes={stats['total_bytes']} "
|
||||
f"compressible_bytes={stats['compressible_bytes']} "
|
||||
f"random_bytes={stats['random_bytes']} files={stats['files']}")
|
||||
if not args.keep_data:
|
||||
shutil.rmtree(BENCH_DIR, ignore_errors=True)
|
||||
return
|
||||
|
||||
# Warm mode: keep a pristine base copy, then mutate the live source.
|
||||
base_dir = None
|
||||
measure_bytes = total_bytes
|
||||
mutation = None
|
||||
if args.warm:
|
||||
change_target = max(64 * 1024, min(int(total_bytes * 0.01), 4 * 1024 * 1024))
|
||||
mutation = apply_incremental_changes(source_dir, change_target)
|
||||
measure_bytes = mutation["changed"]
|
||||
revert_incremental_changes(source_dir, mutation)
|
||||
print(f"Warm mode: each run seeds a full copy, then measures "
|
||||
f"{measure_bytes / (1024*1024):.3f} MB of add/change deltas", file=sys.stderr)
|
||||
|
||||
# Build (Release: benchmarking a debug build is meaningless)
|
||||
print("Building (Release)...", file=sys.stderr)
|
||||
configure = (f"cmake -B {BUILD_DIR} -S {PROJECT_ROOT} "
|
||||
f"-DCMAKE_BUILD_TYPE=Release > /dev/null 2>&1")
|
||||
if os.system(configure) != 0:
|
||||
print("CMake configure failed", file=sys.stderr); sys.exit(1)
|
||||
if os.system(f"cmake --build {BUILD_DIR} -j$(nproc) > /dev/null 2>&1") != 0:
|
||||
print("Build failed", file=sys.stderr); sys.exit(1)
|
||||
print(f"Building (Release) into {BUILD_DIR}...", file=sys.stderr)
|
||||
build_project()
|
||||
|
||||
# Determine active profile for display
|
||||
has_custom_net = args.delay or args.jitter or args.throughput or args.loss
|
||||
@@ -480,19 +819,10 @@ Examples:
|
||||
else:
|
||||
profiles_to_run = args.profiles or ["unlimited"]
|
||||
|
||||
# Generate data
|
||||
source_dir = os.path.join(BENCH_DIR, "source")
|
||||
dest_dir = os.path.join(BENCH_DIR, "dest")
|
||||
total_bytes = generate_bench_data(source_dir, args.size_mb, args.random_ratio)
|
||||
compressible_pct = (1 - args.random_ratio) * 100
|
||||
random_pct = args.random_ratio * 100
|
||||
print(f"Generated {total_bytes / (1024*1024):.1f} MB "
|
||||
f"({random_pct:.0f}% random, {compressible_pct:.0f}% compressible)",
|
||||
file=sys.stderr)
|
||||
|
||||
# Build config list
|
||||
# Build config list (shlex so quoted/space-separated flags survive)
|
||||
if args.configs:
|
||||
fastsync_configs = [{"name": c, "flags": c.split(), "tool": "fastsync"} for c in args.configs]
|
||||
fastsync_configs = [{"name": c, "flags": shlex.split(c), "tool": "fastsync"}
|
||||
for c in args.configs]
|
||||
else:
|
||||
fastsync_configs = list(FASTSYNC_CONFIGS)
|
||||
|
||||
@@ -509,9 +839,16 @@ Examples:
|
||||
all_results = []
|
||||
try:
|
||||
for profile in profiles_to_run:
|
||||
results = run_benchmark(source_dir, dest_dir, configs, args.runs, profile, progress)
|
||||
results = run_benchmark(source_dir, dest_dir, configs, args.runs, profile,
|
||||
measure_bytes, verify=not args.no_verify,
|
||||
warm=args.warm, mutation=mutation,
|
||||
progress=progress)
|
||||
all_results.extend(results)
|
||||
except RuntimeError as exc:
|
||||
sys.stderr.write(f"error: {exc}\n")
|
||||
sys.exit(1)
|
||||
finally:
|
||||
netem_reset()
|
||||
if not args.keep_data:
|
||||
shutil.rmtree(BENCH_DIR, ignore_errors=True)
|
||||
|
||||
@@ -519,7 +856,7 @@ Examples:
|
||||
if args.output == "json":
|
||||
print(json.dumps(all_results, indent=2))
|
||||
else:
|
||||
print_table(all_results, total_bytes, args.random_ratio)
|
||||
print_table(all_results, measure_bytes, stats, args.warm)
|
||||
print()
|
||||
|
||||
|
||||
|
||||
@@ -3,11 +3,35 @@
|
||||
}:
|
||||
|
||||
pkgs.mkShell {
|
||||
# Development shell for FastSync. Provides the host-side toolchain needed to
|
||||
# build, lint, unit-test, integration-test and benchmark the project.
|
||||
# It deliberately does NOT build on entry: run the CMake commands in README.md
|
||||
# (or use the CI Docker image for exact CI parity).
|
||||
nativeBuildInputs = with pkgs; [
|
||||
# build
|
||||
gcc
|
||||
cmake
|
||||
gnumake
|
||||
pkg-config
|
||||
# lint / static analysis (matches CI)
|
||||
clang-tools # clang-format
|
||||
cppcheck
|
||||
# tests
|
||||
(python3.withPackages (ps: with ps; [ pytest pytest-xdist psutil ]))
|
||||
openssh # SSH transport integration tests
|
||||
# debugging
|
||||
gdb
|
||||
valgrind
|
||||
# coverage
|
||||
lcov
|
||||
# benchmark tooling
|
||||
rsync
|
||||
iproute2 # tc/netem for network shaping
|
||||
# misc
|
||||
git
|
||||
curl
|
||||
nodejs
|
||||
nixpkgs-fmt
|
||||
docker
|
||||
tea
|
||||
];
|
||||
@@ -15,14 +39,21 @@ pkgs.mkShell {
|
||||
buildInputs = with pkgs; [
|
||||
zstd
|
||||
openssl
|
||||
(python3.withPackages (ps: with ps; [ pytest ]))
|
||||
];
|
||||
|
||||
# The CMake configure step fetches xxHash via FetchContent, which needs
|
||||
# network access; NIX_ENFORCE_PURITY must be off so the sandbox does not block.
|
||||
NIX_ENFORCE_PURITY = 0;
|
||||
|
||||
shellHook = ''
|
||||
export NIX_ENFORCE_PURITY=0
|
||||
cmake -B build
|
||||
# Make an existing build tree available on PATH, but never build here.
|
||||
if [ -d "$PWD/build" ]; then
|
||||
export PATH="$PWD/build:$PATH"
|
||||
fi
|
||||
echo "FastSync dev shell ready."
|
||||
echo " Build: cmake -B build -S . && cmake --build build -j\$(nproc)"
|
||||
echo " Unit: ./build/tests"
|
||||
echo " CI parity: docker run --rm --user \"\$(id -u):\$(id -g)\" -v \"\$PWD:/workspace\" -w /workspace gitea.tap-tap.win/taptap/fastsync-ci:v10 ..."
|
||||
'';
|
||||
}
|
||||
+65
-9
@@ -19,6 +19,7 @@
|
||||
#include "usage.h"
|
||||
#include "utils.h"
|
||||
#include <errno.h>
|
||||
#include <fcntl.h>
|
||||
#include <limits.h>
|
||||
#include <time.h>
|
||||
#include <signal.h>
|
||||
@@ -27,6 +28,7 @@
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <unistd.h>
|
||||
|
||||
/* Async-signal-safe abort flag set by the SIGINT/SIGTERM handler. Exposed via
|
||||
* client_send.h so the send loops can poll it. Defined here (not in
|
||||
@@ -429,8 +431,14 @@ static int parse_info_flags(const char* value, Config* config) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* Parse a string as an unsigned long long. Returns 0 on success, -1 on error. */
|
||||
/* Parse a string as an unsigned long long. Returns 0 on success, -1 on error.
|
||||
* A leading '-'/'+' (or whitespace) is rejected outright: strtoull would
|
||||
* otherwise silently wrap a negative value to a huge unsigned one. */
|
||||
static int parse_ull_arg(const char* val, unsigned long long* out, const char* optname) {
|
||||
if (!val || val[0] < '0' || val[0] > '9') {
|
||||
log_message(LOG_LEVEL_ERROR, "%s must be a non-negative integer", optname);
|
||||
return -1;
|
||||
}
|
||||
char* end;
|
||||
errno = 0;
|
||||
unsigned long long v = strtoull(val, &end, 10);
|
||||
@@ -537,7 +545,10 @@ static int config_add_filter(Config* config, const char* rule) {
|
||||
char err[160];
|
||||
FilterRule* parsed = filter_rule_parse(rule, err, sizeof(err));
|
||||
if (!parsed) {
|
||||
log_message(LOG_LEVEL_ERROR, "invalid --filter rule '%s': %s", rule, err);
|
||||
char* escaped = output_escape(rule, log_get_8_bit_output());
|
||||
log_message(LOG_LEVEL_ERROR, "invalid --filter rule '%s': %s",
|
||||
escaped ? escaped : "<allocation failed>", err);
|
||||
free(escaped);
|
||||
return -1;
|
||||
}
|
||||
filter_rule_free(parsed);
|
||||
@@ -1106,6 +1117,11 @@ static bool cli_handle_table_option(CliParseCtx* ctx) {
|
||||
}
|
||||
config->use_metadata = true;
|
||||
}
|
||||
/* Remember that --server-host was explicitly given (the field itself
|
||||
defaults to 127.0.0.1, so a value check cannot distinguish it). Used
|
||||
by --dry-run to route an explicit remote target to the server. */
|
||||
if (entry->offset == offsetof(Config, server_host))
|
||||
config->server_host_set = true;
|
||||
}
|
||||
} else if (apply_table_option(config, entry, NULL) != 0) {
|
||||
ctx->exit_code = -1;
|
||||
@@ -1298,10 +1314,15 @@ static bool cli_handle_ssh_and_pattern_options(CliParseCtx* ctx) {
|
||||
ctx->exit_code = -1;
|
||||
return true;
|
||||
}
|
||||
if (val >= DELTA_MIN_FILE_SIZE)
|
||||
if (val >= DELTA_MIN_FILE_SIZE && val <= DELTA_MAX_FILE_SIZE) {
|
||||
config->delta_max_file_size = val;
|
||||
else
|
||||
} else if (val < DELTA_MIN_FILE_SIZE) {
|
||||
log_message(LOG_LEVEL_WARNING, "--delta-max value %llu too small, using default", val);
|
||||
} else {
|
||||
log_message(LOG_LEVEL_ERROR, "--delta-max must not exceed %llu bytes",
|
||||
(unsigned long long)DELTA_MAX_FILE_SIZE);
|
||||
ctx->exit_code = -1;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
@@ -1389,6 +1410,7 @@ static int set_server_port_option(Config* config, const char* value, const char*
|
||||
return -1;
|
||||
}
|
||||
config->server_port = port;
|
||||
config->server_port_set = true;
|
||||
return 0;
|
||||
}
|
||||
|
||||
@@ -1457,6 +1479,12 @@ static bool cli_handle_io_options(CliParseCtx* ctx) {
|
||||
ctx->exit_code = -1;
|
||||
return true;
|
||||
}
|
||||
if (val > MAX_CHUNK_SIZE) {
|
||||
log_message(LOG_LEVEL_ERROR, "--chunk-size must be between 1 and %llu",
|
||||
(unsigned long long)MAX_CHUNK_SIZE);
|
||||
ctx->exit_code = -1;
|
||||
return true;
|
||||
}
|
||||
config->chunk_size = val;
|
||||
return true;
|
||||
}
|
||||
@@ -1473,11 +1501,20 @@ static bool cli_handle_io_options(CliParseCtx* ctx) {
|
||||
fclose(config->log_file);
|
||||
config->log_file = NULL;
|
||||
}
|
||||
FILE* lf = fopen(ctx->argv[++ctx->i], "a");
|
||||
const char* log_path = ctx->argv[++ctx->i];
|
||||
/* Refuse a symlinked target and never leak the descriptor across exec: an
|
||||
* attacker who can plant a symlink in the working directory must not be
|
||||
* able to redirect (or truncate) an arbitrary file via --log-file. The log
|
||||
* is created with owner-only permissions. */
|
||||
int log_fd = open(log_path, O_WRONLY | O_CREAT | O_APPEND | O_NOFOLLOW | O_CLOEXEC, 0600);
|
||||
FILE* lf = log_fd >= 0 ? fdopen(log_fd, "a") : NULL;
|
||||
if (!lf) {
|
||||
char* escaped = output_escape(ctx->argv[ctx->i], false);
|
||||
int open_errno = errno;
|
||||
if (log_fd >= 0)
|
||||
close(log_fd);
|
||||
char* escaped = output_escape(log_path, false);
|
||||
log_message(LOG_LEVEL_ERROR, "could not open log file '%s': %s",
|
||||
escaped ? escaped : "<allocation failed>", strerror(errno));
|
||||
escaped ? escaped : "<allocation failed>", strerror(open_errno));
|
||||
free(escaped);
|
||||
ctx->exit_code = -1;
|
||||
return true;
|
||||
@@ -2005,8 +2042,27 @@ static int read_patterns_from_file(const char* filepath, char*** patterns, int*
|
||||
}
|
||||
char* line = NULL;
|
||||
size_t line_size = 0;
|
||||
ssize_t n;
|
||||
while ((n = getline(&line, &line_size, fp)) != -1) {
|
||||
while (true) {
|
||||
ssize_t n = utils_getdelim_bounded(fp, &line, &line_size, '\n', UTILS_MAX_LINE_LEN);
|
||||
if (n < 0) {
|
||||
/* output_escape() may allocate (and clobber errno): capture the reader's
|
||||
* errno first so an over-long line is still reported as EFBIG. */
|
||||
int saved_errno = errno;
|
||||
char* escaped = output_escape(filepath, false);
|
||||
if (saved_errno == EFBIG) {
|
||||
log_message(LOG_LEVEL_ERROR, "pattern file '%s' has a line exceeding %d bytes",
|
||||
escaped ? escaped : "<allocation failed>", (int)UTILS_MAX_LINE_LEN);
|
||||
} else {
|
||||
log_message(LOG_LEVEL_ERROR, "could not read pattern file '%s': %s",
|
||||
escaped ? escaped : "<allocation failed>", strerror(saved_errno));
|
||||
}
|
||||
free(escaped);
|
||||
free(line);
|
||||
fclose(fp);
|
||||
return -1;
|
||||
}
|
||||
if (n == 0)
|
||||
break;
|
||||
char* p = line;
|
||||
while (*p == ' ' || *p == '\t')
|
||||
p++;
|
||||
|
||||
+280
-15
@@ -48,6 +48,22 @@
|
||||
Always cast to double when dividing so the output stays fractional. */
|
||||
#define BYTES_PER_MIB (1024ULL * 1024ULL)
|
||||
|
||||
/* Surface a server rejection to the user. When the last status exchange
|
||||
carried a STATUS_ERROR_DETAIL reason (protocol 2.21.0) it is appended to the
|
||||
client-side context; a bare STATUS_ERROR still logs the context alone. */
|
||||
static void log_server_rejection(const char* context) {
|
||||
const char* detail = protocol_last_error();
|
||||
if (detail && detail[0] != '\0') {
|
||||
/* The detail is peer-controlled: escape it so terminal/log-format
|
||||
* metacharacters cannot be injected into the client's output. */
|
||||
char* escaped = output_escape(detail, log_get_8_bit_output());
|
||||
log_message(LOG_LEVEL_ERROR, "%s: %s", context, escaped ? escaped : "<allocation failed>");
|
||||
free(escaped);
|
||||
} else {
|
||||
log_message(LOG_LEVEL_ERROR, "%s", context);
|
||||
}
|
||||
}
|
||||
|
||||
/* Forward declaration for progress-reporting thread used in multithreaded send. */
|
||||
static int progress_thread_fn(void* arg);
|
||||
|
||||
@@ -294,8 +310,11 @@ static bool files_from_list_check(const Config* config, ArrayList* missing_dest,
|
||||
return false;
|
||||
}
|
||||
if (set->count == 0) {
|
||||
char* escaped_list =
|
||||
output_escape(config->files_from ? config->files_from : "", log_get_8_bit_output());
|
||||
log_message(LOG_LEVEL_ERROR, "--files-from file '%s' contains no entries; nothing to transfer",
|
||||
config->files_from ? config->files_from : "");
|
||||
escaped_list ? escaped_list : "<allocation failed>");
|
||||
free(escaped_list);
|
||||
return false;
|
||||
}
|
||||
bool ignore = config->ignore_missing_args || config->delete_missing_args;
|
||||
@@ -313,7 +332,10 @@ static bool files_from_list_check(const Config* config, ArrayList* missing_dest,
|
||||
free(full);
|
||||
if (ignore) {
|
||||
(*skipped_out)++;
|
||||
log_info_message(LOG_INFO_MISC, "skipping missing --files-from entry '%s'", entry);
|
||||
char* escaped_entry = output_escape(entry, log_get_8_bit_output());
|
||||
log_info_message(LOG_INFO_MISC, "skipping missing --files-from entry '%s'",
|
||||
escaped_entry ? escaped_entry : "<allocation failed>");
|
||||
free(escaped_entry);
|
||||
if (config->delete_missing_args && missing_dest) {
|
||||
char* mirror = files_from_missing_dest_path(config, entry);
|
||||
if (!mirror || !array_list_add(missing_dest, mirror)) {
|
||||
@@ -324,8 +346,13 @@ static bool files_from_list_check(const Config* config, ArrayList* missing_dest,
|
||||
}
|
||||
continue;
|
||||
}
|
||||
log_message(LOG_LEVEL_ERROR, "--files-from entry '%s' not found in source '%s'", entry,
|
||||
config->send_directory);
|
||||
char* escaped_entry = output_escape(entry, log_get_8_bit_output());
|
||||
char* escaped_src = output_escape(config->send_directory, log_get_8_bit_output());
|
||||
log_message(LOG_LEVEL_ERROR, "--files-from entry '%s' not found in source '%s'",
|
||||
escaped_entry ? escaped_entry : "<allocation failed>",
|
||||
escaped_src ? escaped_src : "<allocation failed>");
|
||||
free(escaped_entry);
|
||||
free(escaped_src);
|
||||
return false;
|
||||
}
|
||||
free(full);
|
||||
@@ -481,6 +508,28 @@ static void disconnect_transfer_client(Client* client) {
|
||||
client_delete(client);
|
||||
}
|
||||
|
||||
/* True when --dry-run should contact a receiver rather than running the
|
||||
* client-side local manifest. Any target a real run would reach over the wire
|
||||
* selects the server-contacting path: a remote (SSH host:path), a daemon
|
||||
* (host::module/path), an explicit --server-host, --server-port/--port, TLS, or
|
||||
* a source-bind --address. A plain local destination (none of these) keeps the
|
||||
* original client-side behavior, which never dials the default 127.0.0.1:8080. */
|
||||
static bool dry_run_targets_server(const Config* config) {
|
||||
if (!config)
|
||||
return false;
|
||||
if (config->transport == TRANSPORT_SSH)
|
||||
return true;
|
||||
if (config->module && config->module[0] != '\0')
|
||||
return true;
|
||||
if (config->server_host_set || config->server_port_set)
|
||||
return true;
|
||||
if (config->use_tls)
|
||||
return true;
|
||||
if (config->address != NULL)
|
||||
return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
static bool add_chunk_to_manifest(ArrayList* manifest, const Chunk* chunk) {
|
||||
if (!manifest)
|
||||
return true;
|
||||
@@ -550,8 +599,12 @@ static void remove_transferred_sources(const Config* config, ArrayList* paths) {
|
||||
close(dirfd);
|
||||
continue;
|
||||
}
|
||||
if (unlinkat(dirfd, leaf, 0) != 0)
|
||||
log_message(LOG_LEVEL_WARNING, "Could not remove source file %s", source->path);
|
||||
if (unlinkat(dirfd, leaf, 0) != 0) {
|
||||
char* escaped_path = output_escape(source->path, log_get_8_bit_output());
|
||||
log_message(LOG_LEVEL_WARNING, "Could not remove source file %s",
|
||||
escaped_path ? escaped_path : "<allocation failed>");
|
||||
free(escaped_path);
|
||||
}
|
||||
close(dirfd);
|
||||
}
|
||||
}
|
||||
@@ -608,8 +661,10 @@ static bool finalize_transfer(Client* client, const Config* config, ArrayList* r
|
||||
Status per_file;
|
||||
if (!receive_status(client->file_descriptor, &per_file))
|
||||
return false;
|
||||
if (per_file == STATUS_ERROR)
|
||||
if (per_file == STATUS_ERROR) {
|
||||
log_server_rejection("Receiver reported a per-file error");
|
||||
return false;
|
||||
}
|
||||
if (per_file == STATUS_OK) {
|
||||
((SourceFile*)remove_sources->items[i])->skipped = true;
|
||||
} else if (per_file != STATUS_NEXT) {
|
||||
@@ -619,7 +674,13 @@ static bool finalize_transfer(Client* client, const Config* config, ArrayList* r
|
||||
}
|
||||
}
|
||||
Status status;
|
||||
return receive_status(client->file_descriptor, &status) && status == STATUS_OK;
|
||||
if (!receive_status(client->file_descriptor, &status))
|
||||
return false;
|
||||
if (status != STATUS_OK) {
|
||||
log_server_rejection("Receiver reported transfer failure");
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
static void pipeline_cancel(PipelineContextSender* context) {
|
||||
@@ -914,7 +975,7 @@ static bool send_delete_manifest_early(Client* client, ArrayList* manifest,
|
||||
return false;
|
||||
}
|
||||
if (ack != STATUS_OK) {
|
||||
log_message(LOG_LEVEL_ERROR, "Server failed to delete files before the transfer");
|
||||
log_server_rejection("Server failed to delete files before the transfer");
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
@@ -991,7 +1052,7 @@ static int incremental_check(Client* client, File* file, const Config* config,
|
||||
if (!receive_status(client->file_descriptor, &s))
|
||||
return -1;
|
||||
if (s == STATUS_ERROR) {
|
||||
log_message(LOG_LEVEL_ERROR, "Server reported error for file");
|
||||
log_server_rejection("Server reported error for file");
|
||||
return -1;
|
||||
}
|
||||
if (s == STATUS_OK)
|
||||
@@ -1024,8 +1085,21 @@ static int incremental_check(Client* client, File* file, const Config* config,
|
||||
*resume_offset = offset;
|
||||
return 3;
|
||||
}
|
||||
/* Server-contacting --dry-run: the receiver decided the file is not up to
|
||||
date and answered "would transfer" WITHOUT expecting any data. Treat it as
|
||||
the dry-run code ONLY when this session actually requested dry-run. A
|
||||
hostile/buggy peer that emits it outside dry-run is a protocol error: fail
|
||||
closed (and send STATUS_ERROR) rather than fall through to the normal path,
|
||||
which would transmit file data the receiver is not reading and desync. */
|
||||
if (s == STATUS_DRY_RUN_TRANSFER) {
|
||||
if (config->dry_run)
|
||||
return 4;
|
||||
log_message(LOG_LEVEL_ERROR, "Unexpected DRY_RUN_TRANSFER status outside a --dry-run session");
|
||||
send_status(client->file_descriptor, STATUS_ERROR);
|
||||
return -1;
|
||||
}
|
||||
if (s != STATUS_NEXT) {
|
||||
log_message(LOG_LEVEL_ERROR, "Unexpected server status");
|
||||
log_server_rejection("Unexpected server status");
|
||||
send_status(client->file_descriptor, STATUS_ERROR);
|
||||
return -1;
|
||||
}
|
||||
@@ -1119,6 +1193,7 @@ static int send_append(const Client* client, File* file, Config* config,
|
||||
return rc;
|
||||
}
|
||||
if (resp != STATUS_APPEND_OK) {
|
||||
log_server_rejection("Unexpected append-verify response");
|
||||
send_status(fd, STATUS_ERROR);
|
||||
return -1;
|
||||
}
|
||||
@@ -1163,6 +1238,174 @@ static int send_append(const Client* client, File* file, Config* config,
|
||||
return ok ? 0 : -1;
|
||||
}
|
||||
|
||||
/* Server-contacting --dry-run. Connects to the configured remote/daemon and
|
||||
* runs the normal per-file incremental decision WITHOUT transmitting any file
|
||||
* data: the receiver (which also sees dry_run=true on the wire) answers
|
||||
* STATUS_OK for an up-to-date file and STATUS_DRY_RUN_TRANSFER for a file it
|
||||
* would otherwise write, mutating nothing on either side. The would-transfer
|
||||
* set and the same trailer as the local dry-run are printed. A
|
||||
* --compare-dest exact basis hit with no destination copy is reported as a
|
||||
* skip by the receiver.
|
||||
*
|
||||
* Only regular files take the receiver-consulted check; directory / symlink /
|
||||
* special / hard-link-sibling entries have no per-file content check, so they
|
||||
* are reported conservatively as would-transfer and their frames are never
|
||||
* sent (which is what keeps the receiver mutation-free). --delete* is
|
||||
* deliberately NOT transmitted in dry-run, so no deletion can occur; the
|
||||
* would-delete manifest report is a documented follow-up.
|
||||
*
|
||||
* Returns 0 on success, 1 on error. */
|
||||
static int send_dry_run_remote(Config* config) {
|
||||
int from_skipped = 0;
|
||||
ArrayList* missing_args = NULL;
|
||||
if (config->delete_missing_args) {
|
||||
missing_args = array_list_create(free);
|
||||
if (!missing_args)
|
||||
return 1;
|
||||
}
|
||||
if (!files_from_list_check(config, missing_args, &from_skipped)) {
|
||||
if (missing_args)
|
||||
array_list_delete(missing_args);
|
||||
return 1;
|
||||
}
|
||||
if (missing_args)
|
||||
array_list_delete(missing_args);
|
||||
/* Alternate basis dirs force the whole-file per-file check on the real
|
||||
receiver; refuse an oversize source up front exactly as send_files does so
|
||||
dry-run reports the same clear diagnostic instead of aborting mid-stream. */
|
||||
if (config_has_basis(config) && !basis_oversize_preflight(config))
|
||||
return 1;
|
||||
/* Would-delete reporting requires a receiver-side read-only extras walk that
|
||||
is not implemented yet; be explicit that --delete is a no-op in dry-run
|
||||
rather than silently ignoring it. */
|
||||
if ((config->use_delete || config->delete_missing_args) && !config->quiet)
|
||||
log_message(LOG_LEVEL_WARNING,
|
||||
"--dry-run: would-delete reporting is not available in this release; nothing is "
|
||||
"deleted");
|
||||
|
||||
/* A live session may follow, so arm graceful abort handling. */
|
||||
client_set_abort_armed(true);
|
||||
Client* client = connect_transfer_client(config);
|
||||
if (!client) {
|
||||
if (config->transport == TRANSPORT_TCP)
|
||||
log_message(LOG_LEVEL_ERROR, "could not connect to server%s",
|
||||
config->use_tls ? " via TLS" : "");
|
||||
client_set_abort_armed(false);
|
||||
return 1;
|
||||
}
|
||||
ProtocolSession session;
|
||||
protocol_session_init(&session, client->file_descriptor, client->file_descriptor);
|
||||
protocol_session_set_io_timeout(&session, config->timeout);
|
||||
protocol_session_set_ssl(&session, (SSL*)client->ssl);
|
||||
protocol_session_bind(&session);
|
||||
|
||||
int ret = 1;
|
||||
PreparedScanner prepared;
|
||||
memset(&prepared, 0, sizeof(prepared));
|
||||
DirectoryScanner* scanner = NULL;
|
||||
if (!config_send(client->file_descriptor, config))
|
||||
goto dry_fail;
|
||||
receive_daemon_motd(client, config);
|
||||
if (!prepare_scanner(config, 0, &prepared))
|
||||
goto dry_fail;
|
||||
scanner = directory_scanner_create_with_options(config->send_directory, &prepared.options);
|
||||
if (!scanner)
|
||||
goto dry_fail;
|
||||
|
||||
int file_count = 0;
|
||||
unsigned long long total_bytes = 0;
|
||||
char size_buffer[32];
|
||||
if (!config->quiet)
|
||||
printf("Dry run: files to be transferred\n");
|
||||
Chunk* chunk;
|
||||
while ((chunk = directory_scanner_next(scanner)) != NULL) {
|
||||
for (int i = 0; i < chunk->element_count; i++) {
|
||||
File* f = chunk->items[i];
|
||||
if (!f)
|
||||
continue;
|
||||
unsigned long long fsize = f->data ? f->data->size : 0;
|
||||
bool would;
|
||||
if (f->is_dir || f->is_symlink || f->is_special ||
|
||||
(f->link_group != 0 && !f->link_first && f->hardlink_target != NULL)) {
|
||||
/* No receiver-side content check exists for these frame types; a real
|
||||
run would (re)create them, so report would-transfer and send no
|
||||
frame (the receiver must stay mutation-free). */
|
||||
would = true;
|
||||
} else if (fsize > MAX_RECEIVE_WHOLE_FILE_SIZE && !config->use_incremental &&
|
||||
!config_has_basis(config)) {
|
||||
/* A non-incremental run streams a >whole-file-limit source without the
|
||||
STATUS_CHECK handshake, so no read-only receiver decision is possible
|
||||
(and none is needed: a real run would transfer it). */
|
||||
would = true;
|
||||
} else {
|
||||
DeltaSignature* sig = NULL;
|
||||
unsigned long long resume_offset = 0;
|
||||
int rc = incremental_check(client, f, config, &sig, &resume_offset);
|
||||
delta_signature_destroy(sig);
|
||||
if (rc < 0) {
|
||||
chunk_destroy(chunk);
|
||||
goto dry_fail;
|
||||
}
|
||||
if (rc == 1)
|
||||
continue; /* up to date; nothing to report */
|
||||
if (rc != 4) {
|
||||
log_message(LOG_LEVEL_ERROR, "Unexpected receiver reply during dry-run");
|
||||
chunk_destroy(chunk);
|
||||
goto dry_fail;
|
||||
}
|
||||
would = true;
|
||||
}
|
||||
if (would) {
|
||||
if (!config->quiet) {
|
||||
char* escaped_path = output_escape(file_wire_path(f), config->eight_bit_output);
|
||||
if (!escaped_path) {
|
||||
chunk_destroy(chunk);
|
||||
goto dry_fail;
|
||||
}
|
||||
if (config->human_readable)
|
||||
printf(" %s (%s)\n", escaped_path,
|
||||
display_bytes(fsize, true, size_buffer, sizeof(size_buffer)));
|
||||
else
|
||||
printf(" %s (%llu bytes)\n", escaped_path, fsize);
|
||||
free(escaped_path);
|
||||
}
|
||||
total_bytes += fsize;
|
||||
file_count++;
|
||||
}
|
||||
}
|
||||
chunk_destroy(chunk);
|
||||
}
|
||||
bool io_error = directory_scanner_had_io_error(scanner);
|
||||
if (directory_scanner_failed(scanner))
|
||||
goto dry_fail;
|
||||
if (io_error)
|
||||
log_message(LOG_LEVEL_WARNING, "source scan hit an unreadable directory");
|
||||
/* Terminate the stream so the receiver emits its success frame; no data
|
||||
frame and no delete manifest are ever sent in dry-run. */
|
||||
if (!send_status(client->file_descriptor, STATUS_FINISHED))
|
||||
goto dry_fail;
|
||||
Status status;
|
||||
if (!receive_status(client->file_descriptor, &status) || status != STATUS_OK)
|
||||
goto dry_fail;
|
||||
if (!config->quiet) {
|
||||
if (config->human_readable)
|
||||
printf("Total: %d files, %s\n", file_count,
|
||||
display_bytes(total_bytes, true, size_buffer, sizeof(size_buffer)));
|
||||
else
|
||||
printf("Total: %d files, %.1f MB\n", file_count, (double)total_bytes / (double)BYTES_PER_MIB);
|
||||
}
|
||||
ret = io_error ? 1 : 0;
|
||||
|
||||
dry_fail:
|
||||
if (scanner)
|
||||
directory_scanner_destroy(scanner);
|
||||
prepared_scanner_destroy(&prepared);
|
||||
disconnect_transfer_client(client);
|
||||
protocol_session_unbind();
|
||||
client_set_abort_armed(false);
|
||||
return ret;
|
||||
}
|
||||
|
||||
// Send a single file directly (non-incremental path).
|
||||
static bool send_file_direct(File* file, int fd, bool use_metadata, int compression_level,
|
||||
const Config* config) {
|
||||
@@ -1289,6 +1532,16 @@ static int send_single_file(Client* client, File* file, Config* config, bool use
|
||||
}
|
||||
return arc == 0 ? 0 : -1;
|
||||
}
|
||||
// rc == 4: the receiver answered DRY_RUN_TRANSFER, which is only valid in
|
||||
// incremental_check's dedicated dry-run consumer. send_single_file never
|
||||
// runs a dry-run session, so this is a protocol error: abort instead of
|
||||
// falling through and sending data the receiver is not reading.
|
||||
if (rc == 4) {
|
||||
log_message(LOG_LEVEL_ERROR, "Receiver answered DRY_RUN_TRANSFER in a non-dry-run transfer");
|
||||
delta_signature_destroy(sig);
|
||||
send_status(client->file_descriptor, STATUS_ERROR);
|
||||
return -1;
|
||||
}
|
||||
// rc == 0: unchanged file, skip
|
||||
// rc == 2: server sent delta signature but sendfile doesn't support delta
|
||||
delta_signature_destroy(sig);
|
||||
@@ -1330,6 +1583,14 @@ static int send_single_file(Client* client, File* file, Config* config, bool use
|
||||
}
|
||||
return arc == 0 ? 0 : -1;
|
||||
}
|
||||
if (rc == 4) {
|
||||
/* See the sendfile branch above: DRY_RUN_TRANSFER is only valid in the
|
||||
dedicated dry-run consumer, never in the normal per-file send path. */
|
||||
log_message(LOG_LEVEL_ERROR, "Receiver answered DRY_RUN_TRANSFER in a non-dry-run transfer");
|
||||
delta_signature_destroy(sig);
|
||||
send_status(client->file_descriptor, STATUS_ERROR);
|
||||
return -1;
|
||||
}
|
||||
if (rc == 2 && config->use_delta && !config->whole_file) {
|
||||
int drc = send_delta(client, file, sig, config);
|
||||
delta_signature_destroy(sig);
|
||||
@@ -1874,7 +2135,7 @@ int write_batch_from_source(const Config* config, const char* batch_path) {
|
||||
prepared_scanner_destroy(&prepared);
|
||||
return 1;
|
||||
}
|
||||
int fd = open(batch_path, O_WRONLY | O_CREAT | O_TRUNC, 0644);
|
||||
int fd = open(batch_path, O_WRONLY | O_CREAT | O_TRUNC | O_NOFOLLOW | O_CLOEXEC, 0600);
|
||||
if (fd < 0) {
|
||||
log_perror("could not create batch file");
|
||||
directory_scanner_destroy(scanner);
|
||||
@@ -1889,8 +2150,10 @@ int write_batch_from_source(const Config* config, const char* batch_path) {
|
||||
if (f == NULL || f->data == NULL)
|
||||
continue;
|
||||
if (f->data->size > 0 && f->data->data == NULL && !file_load_data(f)) {
|
||||
char* escaped_path = output_escape(f->path ? f->path : "", log_get_8_bit_output());
|
||||
log_message(LOG_LEVEL_ERROR, "batch: failed to load data for %s",
|
||||
f->path ? f->path : "<no path>");
|
||||
escaped_path ? escaped_path : "<allocation failed>");
|
||||
free(escaped_path);
|
||||
ok = false;
|
||||
break;
|
||||
}
|
||||
@@ -1931,7 +2194,8 @@ int send_files(Config* config) {
|
||||
if (config->list_only)
|
||||
return send_list_only(config);
|
||||
if (config->dry_run)
|
||||
return send_dry_run_manifest(config);
|
||||
return dry_run_targets_server(config) ? send_dry_run_remote(config)
|
||||
: send_dry_run_manifest(config);
|
||||
ArrayList* missing_args = NULL;
|
||||
int skipped = 0;
|
||||
if (config->delete_missing_args) {
|
||||
@@ -2243,7 +2507,8 @@ int send_files_multithreaded(Config** config_ptr) {
|
||||
if (config->list_only)
|
||||
return send_list_only(config);
|
||||
if (config->dry_run)
|
||||
return send_dry_run_manifest(config);
|
||||
return dry_run_targets_server(config) ? send_dry_run_remote(config)
|
||||
: send_dry_run_manifest(config);
|
||||
ArrayList* missing_args = NULL;
|
||||
int skipped = 0;
|
||||
if (config->delete_missing_args) {
|
||||
|
||||
@@ -22,6 +22,19 @@ bool validate_config(const Config* config) {
|
||||
"--write-batch, --only-write-batch, and --read-batch are mutually exclusive");
|
||||
return false;
|
||||
}
|
||||
/* A dry-run of a local batch apply is not meaningful: --read-batch bypasses
|
||||
the client-side scan/server decision entirely, so dry-run would have no
|
||||
wire state to report (and must not be used as a mutation escape hatch).
|
||||
--only-write-batch likewise never contacts a receiver. --write-batch DOES
|
||||
run a live transfer but additionally mutates the filesystem by emitting the
|
||||
batch file, so a dry-run must not write it either. Reject all three up
|
||||
front instead of silently ignoring --dry-run. */
|
||||
if (config->dry_run && (read_batch || only_write_batch || write_batch)) {
|
||||
log_message(LOG_LEVEL_ERROR,
|
||||
"--dry-run cannot be combined with --read-batch, --only-write-batch, or "
|
||||
"--write-batch; a dry-run must not mutate anything, including batch files");
|
||||
return false;
|
||||
}
|
||||
if (read_batch) {
|
||||
if (!config->receive_root_directory) {
|
||||
log_message(LOG_LEVEL_ERROR, "--read-batch requires a destination directory");
|
||||
|
||||
+16
-5
@@ -179,7 +179,7 @@ static bool entry_passes_selection(const FileListSet* file_list, const FilterRul
|
||||
static void scanner_capture_xattrs(const DirectoryScanner* scanner, File* file) {
|
||||
if (!scanner || !file || !(scanner->options.preserve_xattrs || scanner->options.preserve_acls))
|
||||
return;
|
||||
file->xattrs = xattr_capture_path(file->path);
|
||||
file->xattrs = xattr_capture_path(file->path, scanner->options.preserve_acls);
|
||||
}
|
||||
|
||||
/* Apply --hard-links (-H) detection to one regular File. On a sibling (a
|
||||
@@ -284,7 +284,10 @@ static int open_directory_filter_context(DirectoryScanner* scanner, const Filter
|
||||
filter_file_read(scanner->current_path, scanner->current_rel ? scanner->current_rel : "",
|
||||
&exists, err, sizeof(err));
|
||||
if (!own) {
|
||||
log_message(LOG_LEVEL_ERROR, "invalid .rsync-filter in %s: %s", scanner->current_path, err);
|
||||
char* escaped_path = output_escape(scanner->current_path, log_get_8_bit_output());
|
||||
log_message(LOG_LEVEL_ERROR, "invalid .rsync-filter in %s: %s",
|
||||
escaped_path ? escaped_path : "<allocation failed>", err);
|
||||
free(escaped_path);
|
||||
scanner->failed = true;
|
||||
return -1;
|
||||
}
|
||||
@@ -777,11 +780,19 @@ static File* dirs_file_for_entry(DirectoryScanner* scanner, const char* entry) {
|
||||
nothing (missing entries never appear there). Without the flags it stays
|
||||
a hard pre-transfer error. */
|
||||
if (scanner->options.ignore_missing_args) {
|
||||
log_info_message(LOG_INFO_MISC, "skipping missing --files-from entry '%s'", entry);
|
||||
char* escaped_entry = output_escape(entry, log_get_8_bit_output());
|
||||
log_info_message(LOG_INFO_MISC, "skipping missing --files-from entry '%s'",
|
||||
escaped_entry ? escaped_entry : "<allocation failed>");
|
||||
free(escaped_entry);
|
||||
free(abs_path);
|
||||
return NULL;
|
||||
}
|
||||
log_message(LOG_LEVEL_ERROR, "--dirs listed entry is not present under the source: %s", entry);
|
||||
{
|
||||
char* escaped_entry = output_escape(entry, log_get_8_bit_output());
|
||||
log_message(LOG_LEVEL_ERROR, "--dirs listed entry is not present under the source: %s",
|
||||
escaped_entry ? escaped_entry : "<allocation failed>");
|
||||
free(escaped_entry);
|
||||
}
|
||||
free(abs_path);
|
||||
scanner->failed = true;
|
||||
return NULL;
|
||||
@@ -1434,7 +1445,7 @@ static void scan_root_entry(const ScannerOptions* options, const FilterNode* roo
|
||||
}
|
||||
if ((options->preserve_xattrs || options->preserve_acls) &&
|
||||
!(file->link_group != 0 && !file->link_first))
|
||||
file->xattrs = xattr_capture_path(file->path);
|
||||
file->xattrs = xattr_capture_path(file->path, options->preserve_acls);
|
||||
if (!array_list_add(root_files, file)) {
|
||||
free(rel);
|
||||
file_destroy(file);
|
||||
|
||||
+36
-6
@@ -288,10 +288,19 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver
|
||||
goto fail;
|
||||
}
|
||||
if (status == STATUS_CHECK) {
|
||||
bool skipped;
|
||||
File* file = receive_incremental_check(file_descriptor, config, &skipped);
|
||||
if (!skipped && (!file || !sink->store_file(file, sink->context)))
|
||||
bool skipped = false;
|
||||
bool would_transfer = false;
|
||||
File* file = receive_incremental_check_ex(file_descriptor, config, &skipped, &would_transfer);
|
||||
if (config->dry_run) {
|
||||
/* Server-contacting --dry-run: the reply has already been sent
|
||||
(STATUS_OK = up to date, STATUS_DRY_RUN_TRANSFER = would transfer) and
|
||||
nothing may be stored. Both flags false means a genuine protocol
|
||||
error (STATUS_ERROR already sent or sent by receive_error below). */
|
||||
if (!skipped && !would_transfer)
|
||||
goto receive_error;
|
||||
} else if (!skipped && (!file || !sink->store_file(file, sink->context))) {
|
||||
goto receive_error;
|
||||
}
|
||||
} else if (status == STATUS_CHUNK) {
|
||||
Chunk* chunk = receive_chunk_data(file_descriptor, config);
|
||||
if (!chunk || !receiver_process_chunk(chunk, sink))
|
||||
@@ -323,6 +332,15 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver
|
||||
DeleteManifest* manifest = receive_manifest_entries(file_descriptor);
|
||||
if (!manifest)
|
||||
goto fail; /* receive_manifest_entries already sent STATUS_ERROR */
|
||||
if (config->dry_run) {
|
||||
/* Server-contacting --dry-run mutates nothing, so a keep-set manifest
|
||||
is consumed and discarded. The early-delete mode still needs its ACK
|
||||
so a sender blocked on the delete handshake is not left hanging. */
|
||||
delete_manifest_free(manifest);
|
||||
if (early_delete && !send_status(file_descriptor, STATUS_OK))
|
||||
goto fail;
|
||||
goto next_status;
|
||||
}
|
||||
if (early_delete) {
|
||||
/* --delete-before / --delete-during: the manifest is authoritative the
|
||||
moment it arrives, before any file data. Delete now and acknowledge
|
||||
@@ -441,7 +459,11 @@ typedef struct {
|
||||
static bool receiver_save_file(File* file, void* context_pointer) {
|
||||
ReceiverSaveContext* context = context_pointer;
|
||||
FileSaveResult result = FILE_SAVE_ERROR;
|
||||
if (!context->config->save_to_disk) {
|
||||
if (context->config->dry_run) {
|
||||
/* Defense in depth: a dry-run receiver mutates nothing even if a data
|
||||
frame reaches the sink (the sender is not supposed to send one). */
|
||||
result = FILE_SAVE_SKIPPED;
|
||||
} else if (!context->config->save_to_disk) {
|
||||
/* Nothing is stored; report the file as not-written so a
|
||||
--remove-source-files sender keeps its source. */
|
||||
result = FILE_SAVE_SKIPPED;
|
||||
@@ -457,8 +479,12 @@ static bool receiver_save_file(File* file, void* context_pointer) {
|
||||
file_destroy(file);
|
||||
return false;
|
||||
}
|
||||
if (result != FILE_SAVE_ERROR && context->config->remove_source_files && !file->is_dir &&
|
||||
!file->is_special && !file->skip &&
|
||||
/* A dry-run receiver mutates nothing AND records no per-file outcomes: a
|
||||
hostile dry-run client that streamed data frames anyway must not be able to
|
||||
grow `outcomes` without bound (receiver_outcomes_append reallocs uncharged)
|
||||
or force a per-frame ack. */
|
||||
if (!context->config->dry_run && result != FILE_SAVE_ERROR &&
|
||||
context->config->remove_source_files && !file->is_dir && !file->is_special && !file->skip &&
|
||||
!receiver_outcomes_append(&context->outcomes, (unsigned char)result)) {
|
||||
file_destroy(file);
|
||||
return false;
|
||||
@@ -469,6 +495,10 @@ static bool receiver_save_file(File* file, void* context_pointer) {
|
||||
|
||||
static bool receiver_send_success_frame(int fd, void* context_pointer) {
|
||||
ReceiverSaveContext* context = context_pointer;
|
||||
/* Server-contacting --dry-run: nothing was staged or written, so there is
|
||||
nothing to publish and no directory times to stamp. */
|
||||
if (context->config->dry_run)
|
||||
return receiver_send_final_success(fd, context->config, &context->outcomes);
|
||||
/* --delay-updates: the whole protocol stream (including manifest/delete
|
||||
handling, which ran inside receiver_process) has succeeded and every
|
||||
staged file was fully written. Publish them atomically now, before the
|
||||
|
||||
@@ -196,7 +196,11 @@ int write_thread(void* pipeline_context) {
|
||||
}
|
||||
size_t file_bytes = file->data ? file->data->size : 0;
|
||||
FileSaveResult result = FILE_SAVE_SKIPPED;
|
||||
if (save_to_disk) {
|
||||
/* Server-contacting --dry-run: never write. The receiver thread does not
|
||||
enqueue anything on the dry-run path, but this keeps the writer thread
|
||||
provably mutation-free if a data frame ever reached it. */
|
||||
bool dry_run = context->config->dry_run;
|
||||
if (save_to_disk && !dry_run) {
|
||||
result = file_save_to_disk_full(root_directory, file, context->config);
|
||||
if (result == FILE_SAVE_ERROR) {
|
||||
file_destroy(file);
|
||||
@@ -215,7 +219,7 @@ int write_thread(void* pipeline_context) {
|
||||
/* P7 Wave D: a directory's times are never applied inline (a later child
|
||||
write would clobber them); accumulate the metadata here and let the
|
||||
caller apply it once every writer has drained. */
|
||||
if (result != FILE_SAVE_ERROR && file->is_dir && file->metadata &&
|
||||
if (!dry_run && result != FILE_SAVE_ERROR && file->is_dir && file->metadata &&
|
||||
dir_times_should_capture(context->config) &&
|
||||
!dir_time_list_add(&context->dir_times, file->path, file->metadata)) {
|
||||
file_destroy(file);
|
||||
@@ -234,8 +238,8 @@ int write_thread(void* pipeline_context) {
|
||||
which sources were actually written versus skipped on the receiver.
|
||||
Explicit directory entries and recreated device/special nodes have no
|
||||
source and are never acknowledged (mirrors receiver.c). */
|
||||
if (context->config->remove_source_files && !file->is_dir && !file->is_special && !file->skip &&
|
||||
!receiver_outcomes_append(&context->outcomes, (unsigned char)result)) {
|
||||
if (!dry_run && context->config->remove_source_files && !file->is_dir && !file->is_special &&
|
||||
!file->skip && !receiver_outcomes_append(&context->outcomes, (unsigned char)result)) {
|
||||
file_destroy(file);
|
||||
pipeline_context_receiver_note_bytes_released(context, file_bytes);
|
||||
mtx_lock(&context->mutex);
|
||||
|
||||
+99
-23
@@ -37,6 +37,15 @@ static bool allow_unauthenticated;
|
||||
* root), so no super-user activity is attempted and any client --copy-as is
|
||||
* refused. Set once in main before the accept loop / stdio handler. */
|
||||
static bool server_no_super;
|
||||
/* --allow-super: locally-launched standalone TCP opt-in that preserves the
|
||||
* historical permissive super mode for a root receiver. When false, a
|
||||
* privileged standalone receiver forces SUPER_MODE_OFF for every connection
|
||||
* (C3), so a client cannot make it create device nodes / write raw devices /
|
||||
* apply client-chosen ownership. It is REJECTED for --stdio (the SSH remote
|
||||
* argv is composed by the client, so it must never be able to opt a root
|
||||
* receiver back into super mode); the --stdio path always keeps the secure
|
||||
* default. */
|
||||
static bool server_allow_super;
|
||||
static const char* required_client_cn;
|
||||
/* --iconv CONVERT_SPEC the server was itself started with (borrowed argv
|
||||
* pointer). Its LOCAL half may override the local charset the client assumed;
|
||||
@@ -187,13 +196,25 @@ static bool tls_client_identity_allowed(SSL* ssl) {
|
||||
X509* certificate = SSL_get1_peer_certificate(ssl);
|
||||
if (!certificate)
|
||||
return false;
|
||||
char common_name[256];
|
||||
int length = X509_NAME_get_text_by_NID(X509_get_subject_name(certificate), NID_commonName,
|
||||
common_name, sizeof(common_name));
|
||||
size_t required_length = strlen(required_client_cn);
|
||||
bool allowed = length >= 0 && (size_t)length == required_length &&
|
||||
required_length < sizeof(common_name) &&
|
||||
credentials_secure_equal(common_name, required_client_cn, required_length);
|
||||
bool allowed = false;
|
||||
X509_NAME* subject = X509_get_subject_name(certificate);
|
||||
int index = subject ? X509_NAME_get_index_by_NID(subject, NID_commonName, -1) : -1;
|
||||
if (index >= 0) {
|
||||
X509_NAME_ENTRY* entry = X509_NAME_get_entry(subject, index);
|
||||
ASN1_STRING* data = entry ? X509_NAME_ENTRY_get_data(entry) : NULL;
|
||||
/* Convert the CN to UTF-8 to get its FULL byte length: unlike
|
||||
* X509_NAME_get_text_by_NID (which truncates an over-long CN to the buffer
|
||||
* and reports the truncated length), ASN1_STRING_to_UTF8 never truncates, so
|
||||
* an exactly-required-length CN is accepted while an over-long one cannot be
|
||||
* prefix-matched by a shorter required name. */
|
||||
unsigned char* utf8 = NULL;
|
||||
int cn_length = data ? ASN1_STRING_to_UTF8(&utf8, data) : -1;
|
||||
if (cn_length >= 0 && (size_t)cn_length == required_length)
|
||||
allowed = credentials_secure_equal((const char*)utf8, required_client_cn, required_length);
|
||||
if (utf8)
|
||||
OPENSSL_free(utf8);
|
||||
}
|
||||
X509_free(certificate);
|
||||
return allowed;
|
||||
}
|
||||
@@ -217,12 +238,24 @@ static bool path_is_within(const char* root, const char* path) {
|
||||
root is rejected up front instead of being silently invented by a later
|
||||
write. Both paths are confined to the authorized root by the secure file
|
||||
helpers. */
|
||||
/* Existence-only half of the precondition: the destination root must already
|
||||
resolve to a directory below the authorized root. Never creates anything, so
|
||||
a server-contacting --dry-run can apply the exact same fail-closed check a
|
||||
real run would without mutating the tree. */
|
||||
static bool receive_root_exists(const Config* config) {
|
||||
if (!config || !config->receive_root_directory)
|
||||
return false;
|
||||
return file_directory_exists_secure(config->receive_root_directory);
|
||||
}
|
||||
|
||||
/* Full precondition for a real run: --mkpath creates the root (and missing
|
||||
leading components), otherwise it must already exist as a directory. */
|
||||
static bool ensure_receive_root(const Config* config) {
|
||||
if (!config || !config->receive_root_directory)
|
||||
return false;
|
||||
if (config->mkpath)
|
||||
return file_ensure_directory_secure(config->receive_root_directory);
|
||||
return file_directory_exists_secure(config->receive_root_directory);
|
||||
return receive_root_exists(config);
|
||||
}
|
||||
|
||||
static bool configure_authorization(const char* root) {
|
||||
@@ -263,9 +296,11 @@ typedef enum {
|
||||
} ModuleAuthResult;
|
||||
|
||||
/* Looks up the daemon module selected by the client's config frame and rejects
|
||||
* a `read only` one (every FastSync network transfer writes; there is no
|
||||
* read-only wire operation yet). Returns the module, or NULL with *error set
|
||||
* to the caller-facing rejection message. */
|
||||
* a `read only` one for a real write transfer. A server-contacting --dry-run
|
||||
* IS a read-only wire operation (it reports what would transfer/skip and
|
||||
* mutates nothing), so a `read only` module is the safest possible dry-run
|
||||
* target and is accepted. Returns the module, or NULL with *error set to the
|
||||
* caller-facing rejection message. */
|
||||
static const DaemonModule* module_gate_lookup_module(const Config* config, const char** error) {
|
||||
const DaemonModule* module = daemon_conf_find_module(g_daemon_conf, config->module);
|
||||
if (module == NULL) {
|
||||
@@ -276,7 +311,7 @@ static const DaemonModule* module_gate_lookup_module(const Config* config, const
|
||||
*error = "requested daemon module does not exist";
|
||||
return NULL;
|
||||
}
|
||||
if (module->read_only) {
|
||||
if (module->read_only && !config->dry_run) {
|
||||
log_message(LOG_LEVEL_ERROR, "daemon module '%s' is read only; refusing write transfer",
|
||||
config->module);
|
||||
*error = "requested daemon module is read only";
|
||||
@@ -556,9 +591,10 @@ static const char* module_gate_install_root(const Config* config, const DaemonMo
|
||||
* becomes the authorized root via configure_authorization -- exactly the same
|
||||
* root confinement the standalone server applies to its single
|
||||
* --destination-root, but per-module and NEVER client-chosen. The module is
|
||||
* refused (with a clear log) when it is unknown, when it is `read only` (every
|
||||
* FastSync network transfer writes; there is no read-only wire operation yet),
|
||||
* when it requests client-chosen ownership without the module's
|
||||
* refused (with a clear log) when it is unknown, when it is `read only` for a
|
||||
* real write transfer (a server-contacting --dry-run is a read-only wire
|
||||
* operation and may target a `read only` module), when it requests
|
||||
* client-chosen ownership without the module's
|
||||
* `client owner = yes` opt-in (P7 Wave E hardening), or when the presented
|
||||
* daemon credentials fail for a module that declares `auth users`. Wave A
|
||||
* refused every auth-required module (auth was not yet implemented); Wave B
|
||||
@@ -577,6 +613,22 @@ static const char* server_module_gate(const Config* config, void* context) {
|
||||
if (gate_ctx)
|
||||
gate_ctx->super_mode_override = SUPER_MODE_OFF;
|
||||
}
|
||||
/* C3: a privileged (root) STANDALONE receiver defaults to SUPER_MODE_OFF.
|
||||
* Without this a client --devices/--write-devices/--super would let a root
|
||||
* server create arbitrary device nodes and write raw devices, and
|
||||
* client-chosen ownership (--numeric-ids/--chown/--usermap/--groupmap) would
|
||||
* be applied, with no operator opt-in. The operator must pass --allow-super
|
||||
* to restore the historical permissive behavior; the flag is rejected for
|
||||
* --stdio, whose client-composed argv must never defeat this default (an
|
||||
* operator exposing `fastsync-server --stdio` over SSH needs a forced command
|
||||
* to keep the permissive behavior). An unprivileged receiver is unaffected
|
||||
* (the kernel refuses the confined attempts) and the daemon path keeps its
|
||||
* per-module `client owner = yes` gate. */
|
||||
if (g_daemon_conf == NULL && geteuid() == 0 && !server_allow_super) {
|
||||
effective.super_mode = SUPER_MODE_OFF;
|
||||
if (gate_ctx)
|
||||
gate_ctx->super_mode_override = SUPER_MODE_OFF;
|
||||
}
|
||||
/* --copy-as (P7 Wave E, protocol 2.18.0): FastSync's safe subset forces the
|
||||
ownership of every written entry to the requested ids, which needs a
|
||||
privileged (root) receiver. An unprivileged receiver REFUSES the whole
|
||||
@@ -736,6 +788,13 @@ void handler(int file_descriptor) {
|
||||
goto done;
|
||||
}
|
||||
config->use_delete = config->use_delete && allow_delete;
|
||||
/* --force (receiver-side) is deletion authority too: it lets an incoming
|
||||
* regular file recursively remove a non-empty destination directory tree, and
|
||||
* lets --delete-missing-args remove a non-empty directory mirror. Without
|
||||
* the operator's --allow-delete it must be inert, exactly like --delete and
|
||||
* --delete-missing-args, so a client cannot use --force to bypass the delete
|
||||
* policy. */
|
||||
config->force_delete = config->force_delete && allow_delete;
|
||||
/* --iconv (protocol 2.16.0): install the receiver-side wire->local conversion
|
||||
now that the client's full CONVERT_SPEC has been received and validated,
|
||||
before any received file name is decoded. The server's own --iconv (if
|
||||
@@ -756,9 +815,14 @@ void handler(int file_descriptor) {
|
||||
skipped via its implied --ignore-missing-args, but nothing is deleted). */
|
||||
config->delete_missing_args = config->delete_missing_args && allow_delete;
|
||||
/* --mkpath: create the destination root (and its missing leading components)
|
||||
before anything else; without it the root must pre-exist. A failure here
|
||||
aborts the connection cleanly before any file data is exchanged. */
|
||||
if (!ensure_receive_root(config)) {
|
||||
* before anything else; without it the root must pre-exist. The precondition
|
||||
* is UNCONDITIONAL: a server-contacting --dry-run must reject exactly the
|
||||
* root a real session would reject, so a client cannot set the wire dry_run
|
||||
* bit to relax it. Dry-run only runs the existence/directory check (never
|
||||
* --mkpath) so it creates nothing while still failing closed. A failure here
|
||||
* aborts the connection cleanly before any file data is exchanged. */
|
||||
bool root_ok = config->dry_run ? receive_root_exists(config) : ensure_receive_root(config);
|
||||
if (!root_ok) {
|
||||
char* escaped_root = output_escape(config->receive_root_directory, log_get_8_bit_output());
|
||||
log_message(LOG_LEVEL_ERROR, "destination root is not available: %s",
|
||||
escaped_root ? escaped_root : "<allocation failed>");
|
||||
@@ -767,8 +831,9 @@ void handler(int file_descriptor) {
|
||||
}
|
||||
/* A --delay-updates transfer stages under a private 0700 directory inside
|
||||
the receive root. Create it up front (wiping leftovers of any previously
|
||||
interrupted delayed transfer) so a fully-skipped run also starts clean. */
|
||||
if (config->delay_updates) {
|
||||
interrupted delayed transfer) so a fully-skipped run also starts clean.
|
||||
A dry-run stages nothing, so the staging tree is never created. */
|
||||
if (config->delay_updates && !config->dry_run) {
|
||||
config->delay_context = delay_updates_context_create(config->receive_root_directory);
|
||||
if (!config->delay_context || !delay_updates_prepare(config->delay_context)) {
|
||||
log_message(LOG_LEVEL_ERROR, "Failed to initialize --delay-updates staging area");
|
||||
@@ -864,12 +929,13 @@ void handler(int file_descriptor) {
|
||||
thrd_join(receiver, &receiver_result);
|
||||
thrd_join(writer, &writer_result);
|
||||
bool transfer_ok = receiver_result == thrd_success && writer_result == thrd_success;
|
||||
if (transfer_ok) {
|
||||
if (transfer_ok && !config->dry_run) {
|
||||
/* Commit-style (late) deletion: receive_thread handed the keep-set
|
||||
manifest here instead of deleting while write_thread might still be
|
||||
draining, so by now every file is on disk and the whole transfer is
|
||||
known to have succeeded. Remove the extras before publishing a
|
||||
--delay-updates run; the walker skips the staging directory. */
|
||||
--delay-updates run; the walker skips the staging directory. A
|
||||
server-contacting --dry-run deletes nothing (no manifest is sent). */
|
||||
if (context->deferred_manifest) {
|
||||
if (!manifest_delete_all(config, context->deferred_manifest)) {
|
||||
transfer_ok = false;
|
||||
@@ -878,7 +944,7 @@ void handler(int file_descriptor) {
|
||||
context->deferred_manifest = NULL;
|
||||
}
|
||||
}
|
||||
if (transfer_ok) {
|
||||
if (transfer_ok && !config->dry_run) {
|
||||
/* --delay-updates: receive_thread has finished the whole protocol stream
|
||||
(including manifest/delete handling) and write_thread has drained its
|
||||
queue, so every staged file is complete. Publish atomically before the
|
||||
@@ -899,7 +965,7 @@ void handler(int file_descriptor) {
|
||||
if (!receiver_send_final_success(file_descriptor, config, &context->outcomes))
|
||||
transfer_ok = false;
|
||||
} else {
|
||||
send_status(file_descriptor, STATUS_ERROR);
|
||||
send_error_detail(file_descriptor, "transfer failed on receiver");
|
||||
}
|
||||
if (!transfer_ok)
|
||||
log_message(LOG_LEVEL_ERROR, "Transfer failed");
|
||||
@@ -988,6 +1054,13 @@ static void print_server_usage(void) {
|
||||
printf(" --no-super Operator veto: never attempt super-user activities\n");
|
||||
printf(" (ownership, device nodes) even as root, and refuse\n");
|
||||
printf(" any client --copy-as/--super request\n");
|
||||
printf(" --allow-super Standalone TCP listener only: keep super-user\n");
|
||||
printf(" activities enabled for a root receiver. Without it a\n");
|
||||
printf(" root standalone server forces SUPER_MODE_OFF, so client\n");
|
||||
printf(" --devices/--write-devices/--super and ownership\n");
|
||||
printf(" requests are refused/skipped. Never honored with\n");
|
||||
printf(" --stdio (the SSH remote argv is client-composed, so\n");
|
||||
printf(" super stays off there); no effect when not root\n");
|
||||
printf(" --iconv=LOCAL[,REMOTE] Declare this server's LOCAL charset for file-name\n");
|
||||
printf(" conversion: received names are translated to this\n");
|
||||
printf(" charset (the wire charset still comes from the\n");
|
||||
@@ -1112,6 +1185,9 @@ int main(int argc, char* argv[]) {
|
||||
trust_sender = opts.trust_sender;
|
||||
allow_unauthenticated = opts.allow_unauthenticated;
|
||||
server_no_super = opts.no_super;
|
||||
/* --stdio rejects --allow-super at parse time; force it off here as well so
|
||||
* this process-global policy cannot be re-enabled by a future caller. */
|
||||
server_allow_super = opts.allow_super && !opts.stdio_mode;
|
||||
server_iconv_spec = opts.iconv_spec;
|
||||
signal(SIGINT, cleanup);
|
||||
signal(SIGTERM, cleanup);
|
||||
|
||||
@@ -179,6 +179,8 @@ int server_cli_parse(int argc, char* argv[], ServerCliOptions* opts, char* err,
|
||||
opts->trust_sender = true;
|
||||
} else if (arg_is(argv[i], "--no-super")) {
|
||||
opts->no_super = true;
|
||||
} else if (arg_is(argv[i], "--allow-super")) {
|
||||
opts->allow_super = true;
|
||||
} else if (arg_is(argv[i], "--allow-unauthenticated")) {
|
||||
opts->allow_unauthenticated = true;
|
||||
} else if (arg_has_value(argv[i], "--iconv", &inline_value)) {
|
||||
@@ -259,6 +261,28 @@ int server_cli_parse(int argc, char* argv[], ServerCliOptions* opts, char* err,
|
||||
set_error(err, err_size, "--hash-credentials cannot be combined with --daemon or --stdio");
|
||||
return -1;
|
||||
}
|
||||
if (opts->allow_super && opts->no_super) {
|
||||
set_error(err, err_size, "--allow-super and --no-super are mutually exclusive");
|
||||
return -1;
|
||||
}
|
||||
if (opts->allow_super && opts->daemon_mode) {
|
||||
set_error(err, err_size,
|
||||
"--allow-super is for a locally-launched standalone TCP server; daemon modules opt "
|
||||
"in per module with 'client owner = yes'");
|
||||
return -1;
|
||||
}
|
||||
/* --stdio is the SSH transport: the remote server argv is composed by the
|
||||
* CLIENT (directly and via --remote-option), so a client could otherwise pass
|
||||
* --allow-super to a root --stdio receiver and defeat the C3 secure default.
|
||||
* Never honor it there; the super mode stays forced OFF. An operator who
|
||||
* must keep the historical permissive behavior over SSH has to launch the
|
||||
* receiver through a forced command, not via client-composed argv. */
|
||||
if (opts->allow_super && opts->stdio_mode) {
|
||||
set_error(err, err_size,
|
||||
"--allow-super is not accepted with --stdio (the remote argv is client-composed; "
|
||||
"use a forced command if the default must hold)");
|
||||
return -1;
|
||||
}
|
||||
if (opts->hash_iterations_set && opts->hash_credentials_file == NULL) {
|
||||
set_error(err, err_size, "--iterations requires --hash-credentials");
|
||||
return -1;
|
||||
|
||||
@@ -45,6 +45,17 @@ typedef struct ServerCliOptions {
|
||||
* device-node creation) even when running as root. Applies to --stdio and
|
||||
* --daemon alike; also makes the server refuse any client --copy-as. */
|
||||
bool no_super; /* --no-super */
|
||||
/* --allow-super: locally-launched standalone TCP listener opt-in that keeps
|
||||
* the historical permissive behavior for a PRIVILEGED (root) receiver.
|
||||
* Without it a root standalone server forces SUPER_MODE_OFF, so a client
|
||||
* --devices / --write-devices / --super / ownership request cannot make it
|
||||
* create device nodes, write raw devices, or apply client-chosen ownership.
|
||||
* It is rejected for --stdio: that path's remote argv is composed by the
|
||||
* client (directly and via --remote-option), so it must never opt a root
|
||||
* receiver back into super mode. Non-root receivers are unaffected (the
|
||||
* kernel refuses the confined attempts). The daemon path instead uses the
|
||||
* per-module `client owner = yes` opt-in. */
|
||||
bool allow_super; /* --allow-super */
|
||||
/* --iconv=CONVERT_SPEC: the server's own LOCAL charset declaration. The
|
||||
* client's full spec rides the wire config frame anyway; when the server is
|
||||
* started with its own --iconv, its LOCAL half overrides the local charset
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
#include <limits.h>
|
||||
#include <stdatomic.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
@@ -20,6 +21,33 @@
|
||||
#define MAX_FILE_DATA_SIZE (64ULL * 1024 * 1024)
|
||||
#define MAX_FILES_PER_CHUNK 65536U
|
||||
|
||||
/* Reserve `charge` against `session`'s connection budget. This mirrors the
|
||||
static protocol_reserve_memory() in protocol.c: the receive-side call sites
|
||||
only have the Data.owner pointer (a ProtocolSession*), and protocol.c is out
|
||||
of scope for this fix, so the same atomic CAS accounting is reproduced here.
|
||||
The matching release always goes through data_destroy()'s Data.owner path. */
|
||||
static bool chunk_session_reserve(ProtocolSession* session, size_t charge) {
|
||||
unsigned long long allocated = atomic_load(&session->total_allocated_bytes);
|
||||
while (true) {
|
||||
if (allocated > MAX_CONNECTION_MEMORY ||
|
||||
(unsigned long long)charge > MAX_CONNECTION_MEMORY - allocated)
|
||||
return false;
|
||||
if (atomic_compare_exchange_weak(&session->total_allocated_bytes, &allocated,
|
||||
allocated + (unsigned long long)charge))
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
bool data_charge_session(Data* data, ProtocolSession* session, size_t charge) {
|
||||
if (!data || charge == 0 || session == NULL)
|
||||
return true;
|
||||
if (!chunk_session_reserve(session, charge))
|
||||
return false;
|
||||
data->owner = session;
|
||||
data->protocol_charge = charge;
|
||||
return true;
|
||||
}
|
||||
|
||||
Chunk* chunk_create(File** items, int element_count) {
|
||||
if (element_count < 0 || (element_count > 0 && items == NULL))
|
||||
return NULL;
|
||||
@@ -370,6 +398,15 @@ Chunk* chunk_deserialize(Data* data, bool use_metadata) {
|
||||
Data* replacement = data_create(file_data, file_data_size);
|
||||
if (replacement == NULL)
|
||||
goto error;
|
||||
/* Charge the retained per-file copy to the connection budget (when the
|
||||
inbound chunk carries an owning session) so the queued copies are not
|
||||
held outside MAX_CONNECTION_MEMORY (B6). A NULL owner (e.g. a local
|
||||
batch apply) leaves the copy uncharged. */
|
||||
if (!data_charge_session(replacement, data->owner, allocation_size)) {
|
||||
log_message(LOG_LEVEL_ERROR, "Per-connection memory limit exceeded for chunk file data");
|
||||
data_destroy(replacement);
|
||||
goto error;
|
||||
}
|
||||
data_destroy(file->data);
|
||||
file->data = replacement;
|
||||
data_pointer += file_data_size;
|
||||
@@ -466,12 +503,21 @@ Chunk* receive_chunk_data(int fd, const Config* config) {
|
||||
}
|
||||
Data* data_to_process = chunk_data;
|
||||
if (config->use_compression) {
|
||||
/* Preserve the inbound session across decompression so the (larger)
|
||||
decompressed chunk is charged to the same connection budget; the
|
||||
compressed buffer's own charge is released by data_destroy below. */
|
||||
ProtocolSession* owner = chunk_data->owner;
|
||||
data_to_process = data_decompress_limited(chunk_data, MAX_CHUNK_SIZE);
|
||||
data_destroy(chunk_data);
|
||||
if (data_to_process == NULL) {
|
||||
log_message(LOG_LEVEL_ERROR, "Failed to decompress chunk");
|
||||
return NULL;
|
||||
}
|
||||
if (!data_charge_session(data_to_process, owner, data_to_process->size)) {
|
||||
log_message(LOG_LEVEL_ERROR, "Per-connection memory limit exceeded for decompressed chunk");
|
||||
data_destroy(data_to_process);
|
||||
return NULL;
|
||||
}
|
||||
}
|
||||
|
||||
// Reject chunks larger than the maximum allowed size to prevent OOM.
|
||||
|
||||
@@ -23,4 +23,15 @@ Data* chunk_compress_with_threads(Chunk* chunk, int compression_level, bool use_
|
||||
int compression_threads);
|
||||
Chunk* receive_chunk_data(int fd, const Config* config);
|
||||
|
||||
/* Charge `charge` retained bytes of `data` against `session`'s per-connection
|
||||
* budget (MAX_CONNECTION_MEMORY), mirroring the protocol layer's accounting, and
|
||||
* record them on `data` so data_destroy() returns the charge through the
|
||||
* Data.owner path. Returns false (leaving `data` uncharged) when the ceiling
|
||||
* would be exceeded. A NULL/zero-size charge or a NULL session is a no-op
|
||||
* success. The receive-side decompression and chunk-copy paths know the owning
|
||||
* session only through the Data.owner of the buffer they are processing, so
|
||||
* this is the entry point that lets them participate in the connection budget
|
||||
* without a session handle (B6). */
|
||||
bool data_charge_session(Data* data, ProtocolSession* session, size_t charge);
|
||||
|
||||
#endif
|
||||
@@ -243,9 +243,13 @@ Data* data_decompress_limited(Data* compressed_data, size_t maximum_size) {
|
||||
log_debug_message(LOG_DEBUG_UTIL, "Start to decompress data");
|
||||
unsigned long long dst_size =
|
||||
ZSTD_getFrameContentSize(compressed_data->data, compressed_data->size);
|
||||
if (ZSTD_isError(dst_size)) {
|
||||
log_message(LOG_LEVEL_ERROR, "Failed to get decompressed size: %s",
|
||||
ZSTD_getErrorName(dst_size));
|
||||
/* ZSTD_isError() is also true for ZSTD_CONTENTSIZE_ERROR and
|
||||
* ZSTD_CONTENTSIZE_UNKNOWN (both are encoded near (size_t)-1), so test the
|
||||
* sentinels explicitly instead of blanket-rejecting every error-ish value:
|
||||
* only CONTENTSIZE_ERROR means an unreadable header, while CONTENTSIZE_UNKNOWN
|
||||
* must reach the estimate fallback below. */
|
||||
if (dst_size == ZSTD_CONTENTSIZE_ERROR) {
|
||||
log_message(LOG_LEVEL_ERROR, "Failed to get decompressed size: invalid zstd frame");
|
||||
return NULL;
|
||||
}
|
||||
|
||||
@@ -324,6 +328,20 @@ Data* data_decompress_limited(Data* compressed_data, size_t maximum_size) {
|
||||
uncompressed_data->data = new_data;
|
||||
output.dst = new_data;
|
||||
output.size = buf_size;
|
||||
/* Re-attempt with the larger output buffer; the truncated-frame check
|
||||
* below must not reject a complete frame that merely filled the previous
|
||||
* buffer exactly. */
|
||||
continue;
|
||||
}
|
||||
/* A positive hint with all input consumed means the frame is incomplete: a
|
||||
* truncated stream would otherwise spin here forever (ZSTD_decompressStream
|
||||
* keeps returning the same hint). Fail instead of burning CPU. */
|
||||
if (ret != 0 && input.pos == input.size) {
|
||||
log_message(LOG_LEVEL_ERROR,
|
||||
"Truncated zstd frame: input exhausted with %zu bytes still expected", ret);
|
||||
data_destroy(uncompressed_data);
|
||||
uncompressed_data = NULL;
|
||||
goto cleanup;
|
||||
}
|
||||
} while (ret > 0);
|
||||
|
||||
|
||||
+50
-17
@@ -21,7 +21,6 @@ static void config_set_defaults(Config* config) {
|
||||
config->scanner_threads = 0;
|
||||
config->metadata_explicitly_disabled = false;
|
||||
config->show_progress = false;
|
||||
config->dry_run = false;
|
||||
config->compression_threads = 0;
|
||||
config->ssh_port = 22;
|
||||
config->transport = TRANSPORT_TCP;
|
||||
@@ -42,6 +41,8 @@ static void config_set_defaults(Config* config) {
|
||||
config->tls_ca = NULL;
|
||||
config->server_host = str_dup("127.0.0.1");
|
||||
config->server_port = 8080;
|
||||
config->server_port_set = false;
|
||||
config->server_host_set = false;
|
||||
/* 0 means "--timeout not given": the transport keeps its own built-in 30 s
|
||||
* socket timeout (tcp_set_timeouts ignores non-positive values) and the
|
||||
* protocol layer keeps its built-in 60 s per-message deadline. A positive
|
||||
@@ -190,11 +191,11 @@ static bool validate_received_config(const Config* config) {
|
||||
valid_wire_bool(config->delay_updates) && valid_wire_bool(config->mkpath) &&
|
||||
valid_wire_bool(config->partial) && valid_wire_bool(config->delete_before) &&
|
||||
valid_wire_bool(config->checksum) && valid_wire_bool(config->eight_bit_output) &&
|
||||
checksum_algo_valid(config->checksum_algo) && identity_wire_valid(config) &&
|
||||
valid_wire_bool(config->preserve_atimes) && valid_wire_bool(config->preserve_crtimes) &&
|
||||
valid_wire_bool(config->omit_dir_times) && valid_wire_bool(config->omit_link_times) &&
|
||||
valid_wire_bool(config->munge_links) && valid_wire_bool(config->keep_dirlinks) &&
|
||||
valid_wire_bool(config->fake_super) &&
|
||||
valid_wire_bool(config->dry_run) && checksum_algo_valid(config->checksum_algo) &&
|
||||
identity_wire_valid(config) && valid_wire_bool(config->preserve_atimes) &&
|
||||
valid_wire_bool(config->preserve_crtimes) && valid_wire_bool(config->omit_dir_times) &&
|
||||
valid_wire_bool(config->omit_link_times) && valid_wire_bool(config->munge_links) &&
|
||||
valid_wire_bool(config->keep_dirlinks) && valid_wire_bool(config->fake_super) &&
|
||||
(!config->copy_as_set || (config->copy_as_uid >= 0 && config->copy_as_gid >= 0)) &&
|
||||
(!config->use_compression ||
|
||||
(config->compression_level >= 1 && config->compression_level <= 22)) &&
|
||||
@@ -612,19 +613,36 @@ int config_parse_transport_dest(Config* config) {
|
||||
int daemon_ret = config_parse_daemon_dest(config);
|
||||
if (daemon_ret != 0)
|
||||
return daemon_ret;
|
||||
config_parse_ssh_dest(config);
|
||||
return 0;
|
||||
/* 0 for a local destination (nothing parsed) or a valid SSH destination;
|
||||
* -1 (already logged) for an injection-shaped user@host. */
|
||||
return config_parse_ssh_dest(config);
|
||||
}
|
||||
|
||||
void config_parse_ssh_dest(Config* config) {
|
||||
int config_parse_ssh_dest(Config* config) {
|
||||
if (!config || !config->receive_root_directory)
|
||||
return 0;
|
||||
if (!config_is_remote_dest(config->receive_root_directory))
|
||||
return;
|
||||
return 0;
|
||||
const char* dest = config->receive_root_directory;
|
||||
const char* colon = strchr(dest, ':');
|
||||
/* The user@host token is passed to ssh in option position, so a user or host
|
||||
* beginning with '-' would be consumed by ssh as an option (argument
|
||||
* injection: e.g. "-oProxyCommand=..."). An empty host is likewise not a
|
||||
* valid destination. Validate before any wire/argv construction. */
|
||||
const char* at = memchr(dest, '@', (size_t)(colon - dest));
|
||||
const char* host = at ? at + 1 : dest;
|
||||
size_t host_len = (size_t)(colon - host);
|
||||
size_t user_len = at ? (size_t)(at - dest) : 0;
|
||||
if (host_len == 0 || host[0] == '-' || (user_len > 0 && dest[0] == '-'))
|
||||
return daemon_dest_parse_error("invalid remote destination user@host (must not be empty or "
|
||||
"start with '-')",
|
||||
dest);
|
||||
config->transport = TRANSPORT_SSH;
|
||||
config->ssh_destination = str_dup(config->receive_root_directory);
|
||||
const char* colon = strchr(config->receive_root_directory, ':');
|
||||
config->ssh_destination = str_dup(dest);
|
||||
char* path = str_dup(colon + 1);
|
||||
free(config->receive_root_directory);
|
||||
config->receive_root_directory = path;
|
||||
return 0;
|
||||
}
|
||||
|
||||
void config_burn_auth(Config* config) {
|
||||
@@ -773,8 +791,8 @@ static bool config_receive_module(int fd, Config* c, ConfigStringBudget* budget)
|
||||
return false;
|
||||
if (*module != '\0' && !daemon_module_name_valid(module)) {
|
||||
log_message(LOG_LEVEL_WARNING, "Daemon client sent an invalid or over-long module name");
|
||||
send_error_detail(fd, "invalid or over-long daemon module name");
|
||||
free(module);
|
||||
send_status(fd, STATUS_ERROR);
|
||||
return false;
|
||||
}
|
||||
if (*module != '\0') {
|
||||
@@ -1238,7 +1256,16 @@ bool config_send(int file_descriptor, const Config* config) {
|
||||
return false;
|
||||
}
|
||||
if (status != STATUS_OK) {
|
||||
const char* detail = protocol_last_error();
|
||||
if (detail && detail[0] != '\0') {
|
||||
/* The detail is peer-controlled: escape it before logging. */
|
||||
char* escaped = output_escape(detail, log_get_8_bit_output());
|
||||
log_message(LOG_LEVEL_ERROR, "Error transmitting config: %s",
|
||||
escaped ? escaped : "<allocation failed>");
|
||||
free(escaped);
|
||||
} else {
|
||||
log_message(LOG_LEVEL_ERROR, "Error transmitting config");
|
||||
}
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
@@ -1258,8 +1285,11 @@ Config* config_receive_with_validate(int file_descriptor, ConfigValidateFunc val
|
||||
char* escaped_version = output_escape(config->version, false);
|
||||
log_message(LOG_LEVEL_ERROR, "Protocol version mismatch: client=%s, server=%s",
|
||||
escaped_version ? escaped_version : "<allocation failed>", PROTOCOL_VERSION);
|
||||
char detail[160];
|
||||
snprintf(detail, sizeof(detail), "protocol version mismatch (client=%s, server=%s)",
|
||||
escaped_version ? escaped_version : "<allocation failed>", PROTOCOL_VERSION);
|
||||
send_error_detail(file_descriptor, detail);
|
||||
free(escaped_version);
|
||||
send_status(file_descriptor, STATUS_ERROR);
|
||||
goto error;
|
||||
}
|
||||
if (!receive_core_fields(file_descriptor, config, &budget) ||
|
||||
@@ -1285,13 +1315,16 @@ Config* config_receive_with_validate(int file_descriptor, ConfigValidateFunc val
|
||||
char* escaped_choice = output_escape(config->compress_choice, config->eight_bit_output);
|
||||
log_message(LOG_LEVEL_ERROR, "Unsupported compression choice: %s",
|
||||
escaped_choice ? escaped_choice : "<allocation failed>");
|
||||
char detail[128];
|
||||
snprintf(detail, sizeof(detail), "unsupported compression choice: %s",
|
||||
escaped_choice ? escaped_choice : "<allocation failed>");
|
||||
send_error_detail(file_descriptor, detail);
|
||||
free(escaped_choice);
|
||||
send_status(file_descriptor, STATUS_ERROR);
|
||||
goto error;
|
||||
}
|
||||
if (!validate_received_config(config)) {
|
||||
log_message(LOG_LEVEL_ERROR, "Invalid configuration received from client");
|
||||
send_status(file_descriptor, STATUS_ERROR);
|
||||
send_error_detail(file_descriptor, "invalid configuration received from client");
|
||||
goto error;
|
||||
}
|
||||
if (validate) {
|
||||
@@ -1305,7 +1338,7 @@ Config* config_receive_with_validate(int file_descriptor, ConfigValidateFunc val
|
||||
* written. */
|
||||
if (rejection != CONFIG_VALIDATE_ALREADY_TERMINATED) {
|
||||
log_message(LOG_LEVEL_ERROR, "%s", rejection);
|
||||
send_status(file_descriptor, STATUS_ERROR);
|
||||
send_error_detail(file_descriptor, rejection);
|
||||
}
|
||||
goto error;
|
||||
}
|
||||
|
||||
+56
-6
@@ -76,7 +76,7 @@ typedef struct {
|
||||
typedef enum SuperMode { SUPER_MODE_AUTO = 0, SUPER_MODE_ON = 1, SUPER_MODE_OFF = 2 } SuperMode;
|
||||
|
||||
/* ===========================================================================
|
||||
* Config wire-field table (single source of truth for protocol 2.20.0).
|
||||
* Config wire-field table (single source of truth for protocol 2.21.0).
|
||||
*
|
||||
* Every field below crosses the wire. The table is the ONLY place a
|
||||
* serialized field is named: config.h expands CONFIG_WIRE_FIELDS() to declare
|
||||
@@ -109,6 +109,11 @@ typedef enum SuperMode { SUPER_MODE_AUTO = 0, SUPER_MODE_ON = 1, SUPER_MODE_OFF
|
||||
* =========================================================================== */
|
||||
#define CONFIG_WIRE_HEADER_FIELDS(X) X(version, char*, str_dup(PROTOCOL_VERSION), STR)
|
||||
|
||||
/* dry_run (--dry-run) is CLIENT-INTENT that now CROSSES the wire (protocol
|
||||
* 2.21.0): the receiver needs it to answer what WOULD transfer/skip without
|
||||
* touching disk. The client-only launch behavior (no server contact for a
|
||||
* local destination) is decided separately in client_send.c before the frame
|
||||
* is ever sent. */
|
||||
#define CONFIG_WIRE_CORE_FIELDS(X) \
|
||||
X(eight_bit_output, bool, false, BOOL_8BIT) \
|
||||
X(max_alloc, unsigned long long, DEFAULT_MAX_ALLOC, RAW_MAXALLOC) \
|
||||
@@ -122,7 +127,8 @@ typedef enum SuperMode { SUPER_MODE_AUTO = 0, SUPER_MODE_ON = 1, SUPER_MODE_OFF
|
||||
X(use_executability, bool, false, BOOL) \
|
||||
X(compression_level, int, 5, INT) \
|
||||
X(chunk_size, unsigned long long, DEFAULT_CHUNK_SIZE, RAW) \
|
||||
X(use_sendfile, bool, false, BOOL)
|
||||
X(use_sendfile, bool, false, BOOL) \
|
||||
X(dry_run, bool, false, BOOL)
|
||||
|
||||
#define CONFIG_WIRE_DELTA_FIELDS(X) \
|
||||
X(use_delete, bool, false, BOOL) \
|
||||
@@ -261,7 +267,6 @@ typedef struct Config {
|
||||
int scanner_threads;
|
||||
bool metadata_explicitly_disabled;
|
||||
bool show_progress;
|
||||
bool dry_run;
|
||||
int compression_threads;
|
||||
int ssh_port;
|
||||
TransportType transport;
|
||||
@@ -281,6 +286,18 @@ typedef struct Config {
|
||||
bool use_tls;
|
||||
char* server_host;
|
||||
int server_port;
|
||||
/* True when --server-port/--port was explicitly given. CLIENT-ONLY (never
|
||||
* serialized): --dry-run uses it to decide whether a real server handshake
|
||||
* was requested, so a plain local destination (no explicit port) keeps the
|
||||
* existing client-side dry-run behavior instead of dialing the default
|
||||
* 127.0.0.1:8080. */
|
||||
bool server_port_set;
|
||||
/* True when --server-host was explicitly given. CLIENT-ONLY (never
|
||||
* serialized), and distinct from the "127.0.0.1" default: --dry-run uses it
|
||||
* to route an explicit remote target to the server so it reports receiver
|
||||
* state exactly like a real run, instead of silently running the client-side
|
||||
* manifest. */
|
||||
bool server_host_set;
|
||||
char* tls_cert;
|
||||
char* tls_key;
|
||||
char* tls_ca;
|
||||
@@ -756,8 +773,38 @@ typedef struct Config {
|
||||
* The bump is therefore a deliberate lockstep-release marker, not a
|
||||
* desynchronization fix — the strict same-version handshake still rejects a
|
||||
* mixed 2.19/2.20 deployment. The chunk codec, which already used the packed
|
||||
* metadata_to_buf()/metadata_from_buf() form, is unchanged. */
|
||||
#define PROTOCOL_VERSION "2.20.0"
|
||||
* metadata_to_buf()/metadata_from_buf() form, is unchanged.
|
||||
*
|
||||
* Error-Detail + Server-contacting Dry-run Wave: 2.20.0 -> 2.21.0.
|
||||
*
|
||||
* WHY the bump, grounded in the wire: this release combines two changes on the
|
||||
* same lockstep version.
|
||||
*
|
||||
* (1) Error detail: a server may now answer a rejected operation with
|
||||
* STATUS_ERROR_DETAIL followed by a bounded (<= MAX_ERROR_DETAIL_BYTES)
|
||||
* length-prefixed string instead of a bare STATUS_ERROR (see protocol.h). The
|
||||
* config-frame LAYOUT is unchanged, but the FRAME STREAM gains a new framed
|
||||
* body after a status, so a 2.20 peer that does not consume it would
|
||||
* desynchronize on the following exchange. receive_status() transparently maps
|
||||
* STATUS_ERROR_DETAIL back to STATUS_ERROR for every existing call site and
|
||||
* captures the reason into a thread-local buffer consulted via
|
||||
* protocol_last_error().
|
||||
*
|
||||
* (2) --dry-run: --dry-run now contacts the receiver and reports exactly what
|
||||
* WOULD change. The binary config frame gains one serialized bool
|
||||
* (Config->dry_run) appended to CONFIG_WIRE_CORE_FIELDS after use_sendfile, and
|
||||
* the frame stream gains one terminal status (STATUS_DRY_RUN_TRANSFER) sent in
|
||||
* reply to a per-file STATUS_CHECK when the file is not already up to date.
|
||||
* The receiver performs the normal read-only incremental decision but no
|
||||
* mutation; the sender then skips the data.
|
||||
*
|
||||
* Any config-frame layout or frame-sequence change must bump the protocol
|
||||
* version: a 2.20 peer would desynchronize on the extra trailing byte, the
|
||||
* unknown status, or the unconsumed detail body, and the strict same-version
|
||||
* handshake (config_receive rejects a mismatched version before parsing
|
||||
* anything else) is what keeps a 2.21 client and a 2.20 server from ever
|
||||
* reaching that state. */
|
||||
#define PROTOCOL_VERSION "2.21.0"
|
||||
#define DEFAULT_CHUNK_SIZE (10 * 1024 * 1024)
|
||||
/* Upper bound on total basis-dir entries (rsync caps --link-dest at 20). */
|
||||
#define MAX_BASIS_DIRS 64
|
||||
@@ -804,7 +851,10 @@ bool config_send(int file_descriptor, const Config* config);
|
||||
bool config_send_wire_block(int file_descriptor, const Config* config);
|
||||
Config* config_receive(int file_descriptor);
|
||||
bool config_is_remote_dest(const char* s);
|
||||
void config_parse_ssh_dest(Config* config);
|
||||
/* Parse a single-colon host:path SSH destination (0 = not an SSH destination or
|
||||
* parsed successfully, -1 = rejected, e.g. a user@host beginning with '-'; the
|
||||
* reason is logged). */
|
||||
int config_parse_ssh_dest(Config* config);
|
||||
|
||||
/* A ConfigValidateFunc may return this sentinel to tell
|
||||
* config_receive_with_validate that the callback ALREADY sent a terminal status
|
||||
|
||||
@@ -158,13 +158,13 @@ static int hex_value(char c) {
|
||||
* digits. Such a line is refused loudly (and never accepted) so an operator
|
||||
* cannot keep a replayable bearer digest in place after the protocol bump. */
|
||||
static bool secret_is_legacy_hex(const char* s) {
|
||||
if (!s)
|
||||
if (!s || strlen(s) != 64)
|
||||
return false;
|
||||
for (int i = 0; i < 64; i++) {
|
||||
if (hex_value(s[i]) < 0)
|
||||
return false;
|
||||
}
|
||||
return s[64] == '\0';
|
||||
return true;
|
||||
}
|
||||
|
||||
bool credentials_b64_encode(const uint8_t* in, size_t n, char* out, size_t out_sz) {
|
||||
|
||||
@@ -133,6 +133,7 @@ static bool store_host_list(char*** list, int* count, const char* value, const c
|
||||
return false;
|
||||
}
|
||||
char* save = NULL;
|
||||
int added = 0;
|
||||
for (char* token = strtok_r(copy, ", \t", &save); token; token = strtok_r(NULL, ", \t", &save)) {
|
||||
if (!host_pattern_valid(token)) {
|
||||
if (module_name)
|
||||
@@ -163,8 +164,20 @@ static bool store_host_list(char*** list, int* count, const char* value, const c
|
||||
return false;
|
||||
}
|
||||
(*list)[(*count)++] = dup;
|
||||
added++;
|
||||
}
|
||||
free(copy);
|
||||
/* A present key with an empty (or separator-only) value would otherwise
|
||||
* install a zero-length list, i.e. no ACL at all: a strict-parse config must
|
||||
* never silently turn a restrictive directive into "allow everyone". */
|
||||
if (added == 0) {
|
||||
if (module_name)
|
||||
set_error(err, err_size, "module '%s': '%s' must list at least one host pattern", module_name,
|
||||
key);
|
||||
else
|
||||
set_error(err, err_size, "'%s' must list at least one host pattern", key);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -403,6 +416,7 @@ static bool apply_module_key(DaemonModule* module, char* key, char* value, char*
|
||||
return false;
|
||||
}
|
||||
char* save = NULL;
|
||||
int added = 0;
|
||||
for (char* token = strtok_r(list, ",", &save); token; token = strtok_r(NULL, ",", &save)) {
|
||||
const char* user = trim_ws(token);
|
||||
if (*user == '\0')
|
||||
@@ -430,8 +444,16 @@ static bool apply_module_key(DaemonModule* module, char* key, char* value, char*
|
||||
return false;
|
||||
}
|
||||
module->auth_users[module->auth_user_count++] = dup;
|
||||
added++;
|
||||
}
|
||||
free(list);
|
||||
/* An empty/separator-only value must not silently disable authentication:
|
||||
* the key's presence is an explicit request for an allow-list. */
|
||||
if (added == 0) {
|
||||
set_error(err, err_size, "module '%s': 'auth users' must list at least one user",
|
||||
module->name);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
if (key_equals(key, "max connections"))
|
||||
@@ -439,10 +461,10 @@ static bool apply_module_key(DaemonModule* module, char* key, char* value, char*
|
||||
"max connections", module->name, err, err_size);
|
||||
if (key_equals(key, "hosts allow"))
|
||||
return store_host_list(&module->hosts_allow, &module->hosts_allow_count, value, "hosts allow",
|
||||
false, module->name, err, err_size);
|
||||
module->name, false, err, err_size);
|
||||
if (key_equals(key, "hosts deny"))
|
||||
return store_host_list(&module->hosts_deny, &module->hosts_deny_count, value, "hosts deny",
|
||||
false, module->name, err, err_size);
|
||||
module->name, false, err, err_size);
|
||||
set_error(err, err_size, "unknown key '%s' in module '%s'", key, module->name);
|
||||
return false;
|
||||
}
|
||||
|
||||
+37
-7
@@ -890,13 +890,35 @@ static bool file_to_disk_secure_impl(const char* path, const void* data,
|
||||
if (inplace) {
|
||||
/* --inplace writes directly into the destination; a scratch --temp-dir
|
||||
does not apply and must never redirect these writes. */
|
||||
fd = openat(dirfd, leaf, O_WRONLY | O_CREAT | O_CLOEXEC | O_NOFOLLOW, 0644);
|
||||
/* Type gate BEFORE opening: an existing destination entry that is not a
|
||||
regular file (FIFO, socket, char/block device, directory) must never be
|
||||
opened for writing. Opening a FIFO would block the receive thread
|
||||
forever and writing into a device would bypass the --write-devices /
|
||||
super-mode gate (a client-controlled device write). fstatat with
|
||||
AT_SYMLINK_NOFOLLOW does not follow a symlink and does not block. */
|
||||
struct stat pre_stat;
|
||||
if (fstatat(dirfd, leaf, &pre_stat, AT_SYMLINK_NOFOLLOW) == 0 && !S_ISREG(pre_stat.st_mode)) {
|
||||
close(dirfd);
|
||||
free(leaf);
|
||||
return false;
|
||||
}
|
||||
/* O_NONBLOCK: a no-op for a regular file, but a raced-in FIFO cannot block
|
||||
the open before the post-open S_ISREG re-check rejects it. */
|
||||
fd = openat(dirfd, leaf, O_WRONLY | O_CREAT | O_CLOEXEC | O_NOFOLLOW | O_NONBLOCK, 0644);
|
||||
if (fd >= 0) {
|
||||
struct stat destination_stat;
|
||||
/* Re-check the opened descriptor: a concurrent replacement between the
|
||||
fstatat probe and the open (or a device/FIFO raced in) must never be
|
||||
written through. */
|
||||
if (fstat(fd, &destination_stat) != 0 || !S_ISREG(destination_stat.st_mode)) {
|
||||
close(fd);
|
||||
close(dirfd);
|
||||
free(leaf);
|
||||
return false;
|
||||
}
|
||||
bool newer = false;
|
||||
if (update && metadata && fstat(fd, &destination_stat) == 0 &&
|
||||
S_ISREG(destination_stat.st_mode)) {
|
||||
newer = stat_is_newer(&destination_stat, metadata);
|
||||
if (update && metadata && stat_is_newer(&destination_stat, metadata)) {
|
||||
newer = true;
|
||||
}
|
||||
if (newer) {
|
||||
ok = true;
|
||||
@@ -1226,10 +1248,18 @@ static bool file_to_disk_secure_link_impl(const char* path, const char* basis_pa
|
||||
if (linked) {
|
||||
int target_dirfd = scratch_dirfd >= 0 ? scratch_dirfd : dirfd;
|
||||
if (use_fsync) {
|
||||
int tfd = openat(target_dirfd, tmp, O_RDONLY | O_NOFOLLOW | O_CLOEXEC);
|
||||
if (tfd < 0 || fsync(tfd) != 0) {
|
||||
/* O_NONBLOCK: the freshly linked temp is normally the basis's regular
|
||||
file, but a raced-in FIFO at the name must not block this reopen
|
||||
forever. With O_NONBLOCK such an open fails with ENXIO instead of
|
||||
blocking, which is treated as a benign fsync-skip (the link itself
|
||||
is still installed); any other open/fsync failure falls back to the
|
||||
byte-copy path as before. */
|
||||
int tfd = openat(target_dirfd, tmp, O_RDONLY | O_NOFOLLOW | O_CLOEXEC | O_NONBLOCK);
|
||||
if (tfd < 0) {
|
||||
if (errno != ENXIO)
|
||||
linked = false;
|
||||
} else if (fsync(tfd) != 0) {
|
||||
linked = false;
|
||||
if (tfd >= 0)
|
||||
close(tfd);
|
||||
} else {
|
||||
close(tfd);
|
||||
|
||||
+22
-4
@@ -23,6 +23,8 @@ static void string_list_destroy(StringList* list) {
|
||||
|
||||
static bool string_list_add(StringList* list, const char* text) {
|
||||
if (list->count == list->capacity) {
|
||||
if (list->capacity > INT_MAX / 2)
|
||||
return false;
|
||||
int new_cap = list->capacity > 0 ? list->capacity * 2 : 16;
|
||||
char** grown = realloc(list->items, (size_t)new_cap * sizeof(char*));
|
||||
if (!grown)
|
||||
@@ -56,8 +58,14 @@ static int normalize_entry(const char* raw, size_t len, bool strip_line_endings,
|
||||
snprintf(err, err_size, "absolute path entries are not allowed: '%.*s'", print_len, raw);
|
||||
return -1;
|
||||
}
|
||||
/* Reject NUL bytes inside a token defensively (NUL-delimited mode splits on
|
||||
* them, so this only guards against embedded garbage). */
|
||||
/* Reject NUL bytes inside a token defensively. In NUL-delimited mode the
|
||||
* delimiter itself is the final byte and is expected; in line mode any NUL is
|
||||
* embedded garbage (strlen-based parsing would otherwise silently truncate). */
|
||||
size_t scan_len = strip_line_endings ? len : len - 1;
|
||||
if (memchr(raw, '\0', scan_len)) {
|
||||
snprintf(err, err_size, "entry contains an embedded NUL byte");
|
||||
return -1;
|
||||
}
|
||||
char* dup = malloc(len + 1);
|
||||
if (!dup) {
|
||||
snprintf(err, err_size, "memory allocation failed");
|
||||
@@ -158,10 +166,20 @@ FileListSet* file_list_load(const char* path, bool null_separated, char* err, si
|
||||
StringList raw = {0};
|
||||
char* line = NULL;
|
||||
size_t line_cap = 0;
|
||||
ssize_t n;
|
||||
bool ok = true;
|
||||
char delim = null_separated ? '\0' : '\n';
|
||||
while (ok && (n = getdelim(&line, &line_cap, delim, fp)) != -1) {
|
||||
while (ok) {
|
||||
ssize_t n = utils_getdelim_bounded(fp, &line, &line_cap, delim, UTILS_MAX_LINE_LEN);
|
||||
if (n < 0) {
|
||||
if (errno == EFBIG)
|
||||
snprintf(err, err_size, "entry in file list exceeds %d bytes", (int)UTILS_MAX_LINE_LEN);
|
||||
else
|
||||
snprintf(err, err_size, "error reading file list: %s", strerror(errno));
|
||||
ok = false;
|
||||
break;
|
||||
}
|
||||
if (n == 0)
|
||||
break;
|
||||
int r = normalize_entry(line, (size_t)n, !null_separated, &raw, err, err_size);
|
||||
if (r < 0) {
|
||||
ok = false;
|
||||
|
||||
+509
-335
File diff suppressed because it is too large.
Load diff
@@ -24,6 +24,13 @@ File* file_receive_symlink(int file_descriptor, const Config* config);
|
||||
File* file_receive_special(int file_descriptor);
|
||||
bool file_special_rdev_valid(int32_t major, int32_t minor, mode_t mode);
|
||||
File* receive_incremental_check(int fd, const Config* config, bool* skipped);
|
||||
/* Extended variant used by the receiver. `would_transfer` (may be NULL) is set
|
||||
* true only on the server-contacting --dry-run path when the file is not up to
|
||||
* date: the receiver has already sent STATUS_DRY_RUN_TRANSFER and returns NULL
|
||||
* without storing anything. On that path `*skipped` is true for an up-to-date
|
||||
* (STATUS_OK) file and both flags are false for a genuine error. */
|
||||
File* receive_incremental_check_ex(int fd, const Config* config, bool* skipped,
|
||||
bool* would_transfer);
|
||||
|
||||
/* P7 Wave D directory-time accumulator. The receiver collects the metadata of
|
||||
* every directory it creates/receives (STATUS_MKDIR with metadata and/or the
|
||||
|
||||
+20
-4
@@ -2,6 +2,7 @@
|
||||
#include "log.h"
|
||||
#include "utils.h"
|
||||
#include <errno.h>
|
||||
#include <limits.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
@@ -176,6 +177,8 @@ bool filter_rule_list_add(FilterRuleList* list, FilterRule* rule) {
|
||||
if (!list || !rule)
|
||||
return false;
|
||||
if (list->count == list->capacity) {
|
||||
if (list->capacity > INT_MAX / 2)
|
||||
return false;
|
||||
int new_cap = list->capacity > 0 ? list->capacity * 2 : 8;
|
||||
FilterRule** grown = realloc(list->items, (size_t)new_cap * sizeof(FilterRule*));
|
||||
if (!grown)
|
||||
@@ -322,8 +325,10 @@ FilterRuleList* filter_file_read(const char* dir_path, const char* owner_rel, bo
|
||||
if (!fp) {
|
||||
if (errno == ENOENT || errno == ENOTDIR)
|
||||
return filter_rule_list_create();
|
||||
log_message(LOG_LEVEL_WARNING, "Could not read .rsync-filter in %s: %s", dir_path,
|
||||
strerror(errno));
|
||||
char* escaped_dir = output_escape(dir_path, log_get_8_bit_output());
|
||||
log_message(LOG_LEVEL_WARNING, "Could not read .rsync-filter in %s: %s",
|
||||
escaped_dir ? escaped_dir : "<allocation failed>", strerror(errno));
|
||||
free(escaped_dir);
|
||||
return filter_rule_list_create();
|
||||
}
|
||||
if (exists)
|
||||
@@ -336,9 +341,20 @@ FilterRuleList* filter_file_read(const char* dir_path, const char* owner_rel, bo
|
||||
}
|
||||
char* line = NULL;
|
||||
size_t line_cap = 0;
|
||||
ssize_t n;
|
||||
bool ok = true;
|
||||
while ((n = getline(&line, &line_cap, fp)) != -1) {
|
||||
while (true) {
|
||||
ssize_t n = utils_getdelim_bounded(fp, &line, &line_cap, '\n', UTILS_MAX_LINE_LEN);
|
||||
if (n < 0) {
|
||||
if (errno == EFBIG) {
|
||||
snprintf(err, err_size, "line in .rsync-filter exceeds %d bytes", (int)UTILS_MAX_LINE_LEN);
|
||||
} else {
|
||||
snprintf(err, err_size, "error reading .rsync-filter: %s", strerror(errno));
|
||||
}
|
||||
ok = false;
|
||||
break;
|
||||
}
|
||||
if (n == 0)
|
||||
break;
|
||||
const char* p = line;
|
||||
while (*p == ' ' || *p == '\t')
|
||||
p++;
|
||||
|
||||
+159
-17
@@ -22,6 +22,10 @@ static __thread ProtocolSession* bound_session;
|
||||
static __thread ProtocolSession legacy_io_session = {
|
||||
.read_fd = -1, .write_fd = -1, .max_alloc = DEFAULT_MAX_ALLOC};
|
||||
|
||||
/* Last STATUS_ERROR_DETAIL reason received on this thread (protocol 2.21.0).
|
||||
* Empty when the last status read carried no detail. */
|
||||
static __thread char io_error_detail[MAX_ERROR_DETAIL_BYTES + 1];
|
||||
|
||||
static unsigned long long io_bwlimit = 0;
|
||||
static mtx_t bw_mutex;
|
||||
static once_flag bw_mutex_once = ONCE_FLAG_INIT;
|
||||
@@ -59,6 +63,10 @@ void io_set_fds(int read_fd, int write_fd) {
|
||||
bound_session = NULL;
|
||||
io_read_fd = read_fd;
|
||||
io_write_fd = write_fd;
|
||||
/* A descriptor switch starts a new connection on this thread: a stale
|
||||
rejection detail captured from the previous transport must not leak into
|
||||
the new one. */
|
||||
io_error_detail[0] = '\0';
|
||||
/* A descriptor switch starts a new transport; never reuse a TLS object
|
||||
belonging to a previous connection or test pipe. */
|
||||
io_ssl = NULL;
|
||||
@@ -294,10 +302,14 @@ bool protocol_send_n_data(ProtocolSession* session, const void* data, size_t dat
|
||||
if (pfd.revents & (POLLERR | POLLNVAL))
|
||||
return false;
|
||||
ssize_t bytes_send;
|
||||
if (session->ssl)
|
||||
bytes_send = SSL_write(session->ssl, (const char*)data + total_bytes_send, chunk);
|
||||
else
|
||||
if (session->ssl) {
|
||||
/* SSL_write takes an int length; clamp a >INT_MAX request into chunks so
|
||||
* the size_t downcast can never truncate into a negative/partial write. */
|
||||
size_t ssl_chunk = chunk > (size_t)INT_MAX ? (size_t)INT_MAX : chunk;
|
||||
bytes_send = SSL_write(session->ssl, (const char*)data + total_bytes_send, (int)ssl_chunk);
|
||||
} else {
|
||||
bytes_send = write(fd, (const char*)data + total_bytes_send, chunk);
|
||||
}
|
||||
if (bytes_send <= 0) {
|
||||
if (session->ssl) {
|
||||
int ssl_err = SSL_get_error(session->ssl, (int)bytes_send);
|
||||
@@ -334,27 +346,25 @@ bool protocol_receive_n_data(ProtocolSession* session, void* data, size_t data_s
|
||||
return protocol_receive_n_data_timed(session, data, data_size, timeout_sec);
|
||||
}
|
||||
|
||||
bool protocol_receive_n_data_timed(ProtocolSession* session, void* data, size_t data_size,
|
||||
int timeout_sec) {
|
||||
/* Read exactly `data_size` bytes from `session` before `deadline` elapses
|
||||
* (CLOCK_MONOTONIC). Shared by the ordinary timed primitive and the error-detail
|
||||
* body reader so the latter can clamp itself to whatever deadline its caller
|
||||
* already established instead of always applying the session's 60 s window. */
|
||||
static bool protocol_receive_n_data_until(ProtocolSession* session, void* data, size_t data_size,
|
||||
const struct timespec* deadline) {
|
||||
log_debug_message(LOG_DEBUG_IO, " Receiving n Data: %zu", data_size);
|
||||
if (!session)
|
||||
if (!session || !deadline)
|
||||
return false;
|
||||
int fd = session->read_fd;
|
||||
if (timeout_sec <= 0)
|
||||
timeout_sec = RECEIVE_TIMEOUT_SEC;
|
||||
|
||||
struct timespec deadline;
|
||||
clock_gettime(CLOCK_MONOTONIC, &deadline);
|
||||
deadline.tv_sec += timeout_sec;
|
||||
|
||||
size_t total_bytes_received = 0;
|
||||
short wait_events = POLLIN;
|
||||
while (total_bytes_received < data_size) {
|
||||
if (!session->ssl || SSL_pending(session->ssl) == 0) {
|
||||
struct pollfd pfd = {.fd = fd, .events = wait_events};
|
||||
int poll_result = poll(&pfd, 1, deadline_remaining_ms(&deadline));
|
||||
int poll_result = poll(&pfd, 1, deadline_remaining_ms(deadline));
|
||||
if (poll_result == 0) {
|
||||
log_message(LOG_LEVEL_ERROR, "Receive timeout after %ds", timeout_sec);
|
||||
log_message(LOG_LEVEL_ERROR, "Receive timeout");
|
||||
return false;
|
||||
}
|
||||
if (poll_result < 0) {
|
||||
@@ -381,6 +391,13 @@ bool protocol_receive_n_data_timed(ProtocolSession* session, void* data, size_t
|
||||
wait_events = ssl_err == SSL_ERROR_WANT_WRITE ? POLLOUT : POLLIN;
|
||||
continue;
|
||||
}
|
||||
/* A signal interrupts the blocking TLS read: retry (mirrors the send
|
||||
path and protocol_read_status_until) so the loop reaches its next
|
||||
abort/deadline checkpoint instead of failing spuriously. */
|
||||
if (ssl_err == SSL_ERROR_SYSCALL && errno == EINTR)
|
||||
continue;
|
||||
} else if (errno == EINTR) {
|
||||
continue;
|
||||
}
|
||||
if (bytes_received == 0)
|
||||
log_message(LOG_LEVEL_ERROR, "Connection closed while receiving data");
|
||||
@@ -396,6 +413,18 @@ bool protocol_receive_n_data_timed(ProtocolSession* session, void* data, size_t
|
||||
return true;
|
||||
}
|
||||
|
||||
bool protocol_receive_n_data_timed(ProtocolSession* session, void* data, size_t data_size,
|
||||
int timeout_sec) {
|
||||
if (!session)
|
||||
return false;
|
||||
if (timeout_sec <= 0)
|
||||
timeout_sec = RECEIVE_TIMEOUT_SEC;
|
||||
struct timespec deadline;
|
||||
clock_gettime(CLOCK_MONOTONIC, &deadline);
|
||||
deadline.tv_sec += timeout_sec;
|
||||
return protocol_receive_n_data_until(session, data, data_size, &deadline);
|
||||
}
|
||||
|
||||
static const char* status_to_string(Status status) {
|
||||
switch (status) {
|
||||
case STATUS_OK:
|
||||
@@ -446,6 +475,10 @@ static const char* status_to_string(Status status) {
|
||||
return "AUTH_OK";
|
||||
case STATUS_AUTH_FAILED:
|
||||
return "AUTH_FAILED";
|
||||
case STATUS_ERROR_DETAIL:
|
||||
return "ERROR_DETAIL";
|
||||
case STATUS_DRY_RUN_TRANSFER:
|
||||
return "DRY_RUN_TRANSFER";
|
||||
default:
|
||||
return "UNKNOWN";
|
||||
}
|
||||
@@ -600,8 +633,82 @@ bool protocol_send_status(ProtocolSession* session, Status status) {
|
||||
return true;
|
||||
}
|
||||
|
||||
/* Read the bounded, length-prefixed body of a STATUS_ERROR_DETAIL frame within
|
||||
* `deadline` (CLOCK_MONOTONIC), polling `abort_check` (may be NULL) between
|
||||
* drain chunks. The declared length is validated BEFORE any allocation:
|
||||
*
|
||||
* - `size > MAX_STRING_SIZE`: an absurd framing error. Reading/draining that
|
||||
* many bytes could never finish, so it is fatal (the caller tears the
|
||||
* connection down) rather than drained.
|
||||
* - `MAX_ERROR_DETAIL_BYTES < size <= MAX_STRING_SIZE`: drain exactly `size`
|
||||
* bytes through a small fixed scratch buffer so the stream stays in sync,
|
||||
* leaving the captured detail empty. No allocation happens.
|
||||
* - `size <= MAX_ERROR_DETAIL_BYTES`: read straight into the thread-local
|
||||
* `io_error_detail` buffer (size+1 capacity, already reserved), so the
|
||||
* session's --max-alloc / MAX_CONNECTION_MEMORY budgets are never touched.
|
||||
*
|
||||
* Returns false on a fatal framing problem or any I/O failure; the terminal
|
||||
* detail is then empty. The body is consumed on every non-fatal path even when
|
||||
* the caller ignores protocol_last_error(), so the stream never desyncs. */
|
||||
static bool protocol_receive_error_detail_until(ProtocolSession* session,
|
||||
const struct timespec* deadline,
|
||||
ProtocolWaitAbort abort_check) {
|
||||
io_error_detail[0] = '\0';
|
||||
size_t size = 0;
|
||||
if (!protocol_receive_n_data_until(session, &size, sizeof(size), deadline))
|
||||
return false;
|
||||
if (size > MAX_STRING_SIZE) {
|
||||
log_message(LOG_LEVEL_ERROR, "Error detail length %zu exceeds maximum %llu", size,
|
||||
(unsigned long long)MAX_STRING_SIZE);
|
||||
return false;
|
||||
}
|
||||
if (size > MAX_ERROR_DETAIL_BYTES) {
|
||||
char scratch[256];
|
||||
size_t remaining = size;
|
||||
while (remaining > 0) {
|
||||
if (abort_check && abort_check())
|
||||
return false;
|
||||
size_t chunk = remaining < sizeof(scratch) ? remaining : sizeof(scratch);
|
||||
if (!protocol_receive_n_data_until(session, scratch, chunk, deadline))
|
||||
return false;
|
||||
remaining -= chunk;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
if (!protocol_receive_n_data_until(session, io_error_detail, size, deadline))
|
||||
return false;
|
||||
io_error_detail[size] = '\0';
|
||||
return true;
|
||||
}
|
||||
|
||||
/* Consume the optional detail body of a STATUS_ERROR_DETAIL frame and map the
|
||||
* status back to STATUS_ERROR for existing callers. Invoked for EVERY status
|
||||
* read so a stale detail from an earlier exchange is never reported for a later
|
||||
* one -- except for STATUS_KEEPALIVE, which carries no body and whose drain
|
||||
* (protocol_receive_status_keepalive) must NOT erase the terminal detail that
|
||||
* arrived just before it. Returns false on a fatal framing error. */
|
||||
static bool protocol_capture_error_detail(ProtocolSession* session, Status* status,
|
||||
const struct timespec* deadline,
|
||||
ProtocolWaitAbort abort_check) {
|
||||
if (*status == STATUS_KEEPALIVE)
|
||||
return true;
|
||||
io_error_detail[0] = '\0';
|
||||
if (*status != STATUS_ERROR_DETAIL)
|
||||
return true;
|
||||
*status = STATUS_ERROR;
|
||||
return protocol_receive_error_detail_until(session, deadline, abort_check);
|
||||
}
|
||||
|
||||
bool protocol_receive_status(ProtocolSession* session, Status* status) {
|
||||
if (!protocol_receive_n_data(session, status, sizeof(Status)))
|
||||
if (!session || !status)
|
||||
return false;
|
||||
int timeout_sec = session->io_timeout_sec > 0 ? session->io_timeout_sec : RECEIVE_TIMEOUT_SEC;
|
||||
struct timespec deadline;
|
||||
clock_gettime(CLOCK_MONOTONIC, &deadline);
|
||||
deadline.tv_sec += timeout_sec;
|
||||
if (!protocol_receive_n_data_until(session, status, sizeof(Status), &deadline))
|
||||
return false;
|
||||
if (!protocol_capture_error_detail(session, status, &deadline, NULL))
|
||||
return false;
|
||||
log_debug_message(LOG_DEBUG_PROTO, "Received Status: %s", status_to_string(*status));
|
||||
return true;
|
||||
@@ -610,9 +717,19 @@ bool protocol_receive_status(ProtocolSession* session, Status* status) {
|
||||
/* protocol_receive_status with an explicit per-message deadline (seconds).
|
||||
Used where a single reply may legitimately take far longer than the default
|
||||
60 s receive window - e.g. the sender waiting for the early-delete ACK after
|
||||
the receiver committed a large (up to MAX_SERVER_DELETE_COUNT) deletion. */
|
||||
the receiver committed a large (up to MAX_SERVER_DELETE_COUNT) deletion. The
|
||||
error-detail body shares the same deadline as the status header. */
|
||||
bool protocol_receive_status_timed(ProtocolSession* session, Status* status, int timeout_sec) {
|
||||
if (!protocol_receive_n_data_timed(session, status, sizeof(Status), timeout_sec))
|
||||
if (!session || !status)
|
||||
return false;
|
||||
if (timeout_sec <= 0)
|
||||
timeout_sec = RECEIVE_TIMEOUT_SEC;
|
||||
struct timespec deadline;
|
||||
clock_gettime(CLOCK_MONOTONIC, &deadline);
|
||||
deadline.tv_sec += timeout_sec;
|
||||
if (!protocol_receive_n_data_until(session, status, sizeof(Status), &deadline))
|
||||
return false;
|
||||
if (!protocol_capture_error_detail(session, status, &deadline, NULL))
|
||||
return false;
|
||||
log_debug_message(LOG_DEBUG_PROTO, "Received Status: %s", status_to_string(*status));
|
||||
return true;
|
||||
@@ -725,6 +842,8 @@ bool protocol_receive_status_keepalive(ProtocolSession* session, Status* status,
|
||||
Status received;
|
||||
if (!protocol_read_status_until(session, &received, &deadline))
|
||||
return false;
|
||||
if (!protocol_capture_error_detail(session, &received, &deadline, abort_check))
|
||||
return false;
|
||||
if (received == STATUS_KEEPALIVE) {
|
||||
/* The receiver's answer to one of our keepalives. */
|
||||
replies_seen++;
|
||||
@@ -751,6 +870,8 @@ bool protocol_receive_status_keepalive(ProtocolSession* session, Status* status,
|
||||
keepalives_sent - replies_seen);
|
||||
break;
|
||||
}
|
||||
if (!protocol_capture_error_detail(session, &drained, &drain_deadline, abort_check))
|
||||
return false;
|
||||
if (drained != STATUS_KEEPALIVE) {
|
||||
log_message(LOG_LEVEL_ERROR, "Unexpected status while draining keepalive replies");
|
||||
return false;
|
||||
@@ -807,3 +928,24 @@ bool receive_status_keepalive(int fd, Status* status, int timeout_sec, int keepa
|
||||
return protocol_receive_status_keepalive(legacy_session(fd, -1), status, timeout_sec,
|
||||
keepalive_interval_sec, abort_check);
|
||||
}
|
||||
|
||||
bool send_error_detail(int fd, const char* message) {
|
||||
if (!message)
|
||||
message = "";
|
||||
char bounded[MAX_ERROR_DETAIL_BYTES + 1];
|
||||
size_t len = strlen(message);
|
||||
if (len > MAX_ERROR_DETAIL_BYTES) {
|
||||
memcpy(bounded, message, MAX_ERROR_DETAIL_BYTES);
|
||||
bounded[MAX_ERROR_DETAIL_BYTES] = '\0';
|
||||
message = bounded;
|
||||
}
|
||||
return send_status(fd, STATUS_ERROR_DETAIL) && send_str(fd, message);
|
||||
}
|
||||
|
||||
const char* protocol_last_error(void) {
|
||||
return io_error_detail;
|
||||
}
|
||||
|
||||
void protocol_clear_last_error(void) {
|
||||
io_error_detail[0] = '\0';
|
||||
}
|
||||
+38
-1
@@ -9,6 +9,12 @@
|
||||
/* Maximum allowed string size for receive_str (64 KB) */
|
||||
#define MAX_STRING_SIZE (64 * 1024)
|
||||
|
||||
/* Hard cap on the optional server->client rejection detail carried by
|
||||
* STATUS_ERROR_DETAIL (protocol 2.21.0). A longer message is sliced to this
|
||||
* many bytes before it is sent, so a peer can never be made to retain more than
|
||||
* this for a rejection and the detail frame stays a small, fixed bound. */
|
||||
#define MAX_ERROR_DETAIL_BYTES 4096
|
||||
|
||||
/* Maximum uncompressed file payload accepted by the receiver's whole-file
|
||||
* paths. A single whole file is charged against the per-connection memory
|
||||
* reservation (MAX_CONNECTION_MEMORY) and against the server allocation
|
||||
@@ -132,7 +138,24 @@ enum NET_STATUS {
|
||||
STATUS_AUTH_CHALLENGE,
|
||||
STATUS_AUTH_RESPONSE,
|
||||
STATUS_AUTH_OK,
|
||||
STATUS_AUTH_FAILED
|
||||
STATUS_AUTH_FAILED,
|
||||
/* Optional server->client rejection detail (protocol 2.21.0). When the
|
||||
* server refuses a transfer for a concrete reason it may send
|
||||
* STATUS_ERROR_DETAIL followed by a length-prefixed, bounded string instead
|
||||
* of a bare STATUS_ERROR. receive_status() consumes the string and maps the
|
||||
* status back to STATUS_ERROR, so every pre-2.21 call site keeps working;
|
||||
* callers that want the human-readable reason consult protocol_last_error().
|
||||
* Appended immediately after STATUS_AUTH_FAILED so the existing wire values
|
||||
* never move. */
|
||||
STATUS_ERROR_DETAIL,
|
||||
/* Server-contacting --dry-run (protocol 2.21.0). Sent by the receiver in
|
||||
* response to a per-file STATUS_CHECK when the wire config carries
|
||||
* dry_run=true and the file is NOT already up to date: it tells the sender
|
||||
* the file WOULD be transferred, and the sender must NOT transmit any data
|
||||
* (the receiver reads none in dry-run). STATUS_OK keeps its meaning in this
|
||||
* path ("already up to date / nothing to do"). Appended after
|
||||
* STATUS_ERROR_DETAIL so no existing status is renumbered. */
|
||||
STATUS_DRY_RUN_TRANSFER
|
||||
};
|
||||
|
||||
void io_set_fds(int read_fd, int write_fd);
|
||||
@@ -192,6 +215,20 @@ bool send_int(int file_descriptor, int data);
|
||||
bool receive_int(int file_descriptor, int* data);
|
||||
bool send_status(int file_descriptor, Status status);
|
||||
bool receive_status(int file_descriptor, Status* status);
|
||||
/* Send STATUS_ERROR_DETAIL followed by a bounded (<= MAX_ERROR_DETAIL_BYTES)
|
||||
* length-prefixed string. Over-long messages are sliced and NULL is treated
|
||||
* as "". Returns false if the status or the string could not be sent. */
|
||||
bool send_error_detail(int file_descriptor, const char* message);
|
||||
/* Human-readable reason captured from the most recent STATUS_ERROR_DETAIL
|
||||
* received on this thread, or "" when the last status was a bare STATUS_ERROR
|
||||
* (or no detail was seen). Thread-local, and valid until the next non-keepalive
|
||||
* status read on the same thread; a later STATUS_KEEPALIVE does NOT clear it.
|
||||
* The detail body is bounded by MAX_ERROR_DETAIL_BYTES: an over-cap declared
|
||||
* length is drained and yields "" (so the stream never desyncs), while an
|
||||
* absurd length is a fatal framing error that fails the status read. */
|
||||
const char* protocol_last_error(void);
|
||||
/* Clear the thread-local last-error buffer. */
|
||||
void protocol_clear_last_error(void);
|
||||
/* receive_status with an explicit per-message deadline in seconds, instead of
|
||||
the default RECEIVE_TIMEOUT_SEC. A reply that may legitimately take longer
|
||||
(e.g. the early-delete ACK after a large receiver-side deletion) must use
|
||||
|
||||
@@ -75,6 +75,15 @@ static int parse_remote_dest(const char* dest, RemoteDest* r) {
|
||||
memcpy(r->host, dest, host_len);
|
||||
r->host[host_len] = '\0';
|
||||
}
|
||||
/* The user@host token is handed to ssh in option position. Reject anything
|
||||
* that ssh would consume as an option (a leading '-') or an empty host, so a
|
||||
* crafted destination can never inject an ssh option such as
|
||||
* -oProxyCommand=... . This mirrors config_parse_ssh_dest's validation and
|
||||
* is defense-in-depth for callers that bypass it. */
|
||||
if (r->host[0] == '\0' || r->host[0] == '-' || r->user[0] == '-') {
|
||||
remote_dest_destroy(r);
|
||||
return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
@@ -216,10 +225,10 @@ char** ssh_build_client_argv(const char* rsh_command, int port, const char* user
|
||||
nwords = 1;
|
||||
}
|
||||
|
||||
/* Fixed tail: three -o pairs (6) + optional -p/value (2) + user@host +
|
||||
* remote command + terminating NULL. */
|
||||
/* Fixed tail: three -o pairs (6) + optional -p/value (2) + the "--" end of
|
||||
* options marker + user@host + remote command + terminating NULL. */
|
||||
int port_extra = (port > 0 && port != 22) ? 2 : 0;
|
||||
size_t total = (size_t)nwords + 6 + (size_t)port_extra + 3;
|
||||
size_t total = (size_t)nwords + 6 + (size_t)port_extra + 4;
|
||||
char** argv = calloc(total, sizeof(char*));
|
||||
if (!argv) {
|
||||
for (int i = 0; i < nwords; i++)
|
||||
@@ -253,6 +262,13 @@ char** ssh_build_client_argv(const char* rsh_command, int port, const char* user
|
||||
goto fail_argv;
|
||||
ac++;
|
||||
}
|
||||
/* End of options: guarantees the user@host token that follows is treated as
|
||||
* the destination and never re-interpreted as an ssh option, even if every
|
||||
* caller-side validation were bypassed. */
|
||||
argv[ac] = str_dup("--");
|
||||
if (!argv[ac])
|
||||
goto fail_argv;
|
||||
ac++;
|
||||
argv[ac] = str_dup(userhost);
|
||||
if (!argv[ac])
|
||||
goto fail_argv;
|
||||
|
||||
+76
-15
@@ -4,7 +4,9 @@
|
||||
#include "transport_tcp.h"
|
||||
#include "utils.h"
|
||||
#include <arpa/inet.h>
|
||||
#include <fcntl.h>
|
||||
#include <openssl/err.h>
|
||||
#include <openssl/pem.h>
|
||||
#include <openssl/ssl.h>
|
||||
#include <signal.h>
|
||||
#include <stdio.h>
|
||||
@@ -34,6 +36,59 @@ static void log_ssl_errors(void) {
|
||||
}
|
||||
}
|
||||
|
||||
/* Load the TLS private key through an already-opened, no-follow descriptor so
|
||||
* the owner/mode policy is checked on the SAME file object that is loaded: an
|
||||
* attacker cannot swap the path between a stat() and a later open() (TOCTOU).
|
||||
* The exact-owner / 0600 policy is preserved and group/other execute bits are
|
||||
* rejected as well. Ownership of the descriptor passes to the BIO and is
|
||||
* released exactly once by BIO_free() (BIO_CLOSE). */
|
||||
static bool load_private_key_secure(SSL_CTX* ctx, const char* key) {
|
||||
int fd = open(key, O_RDONLY | O_NOFOLLOW | O_CLOEXEC);
|
||||
if (fd < 0) {
|
||||
char* escaped = output_escape(key, false);
|
||||
log_message(LOG_LEVEL_ERROR, "Failed to open private key: %s",
|
||||
escaped ? escaped : "<allocation failed>");
|
||||
free(escaped);
|
||||
return false;
|
||||
}
|
||||
struct stat key_stat;
|
||||
if (fstat(fd, &key_stat) != 0 || !S_ISREG(key_stat.st_mode) || key_stat.st_uid != geteuid() ||
|
||||
(key_stat.st_mode & (S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH | S_IXGRP | S_IXOTH))) {
|
||||
log_message(LOG_LEVEL_ERROR,
|
||||
"TLS private key must be a regular file owned by the current user and private "
|
||||
"(mode 0600)");
|
||||
close(fd);
|
||||
return false;
|
||||
}
|
||||
BIO* bio = BIO_new_fd(fd, BIO_CLOSE);
|
||||
if (!bio) {
|
||||
close(fd);
|
||||
log_message(LOG_LEVEL_ERROR, "Failed to read private key");
|
||||
return false;
|
||||
}
|
||||
EVP_PKEY* pkey = PEM_read_bio_PrivateKey(bio, NULL, NULL, NULL);
|
||||
BIO_free(bio); /* releases fd via BIO_CLOSE */
|
||||
if (!pkey) {
|
||||
char* escaped = output_escape(key, false);
|
||||
log_message(LOG_LEVEL_ERROR, "Failed to load private key: %s",
|
||||
escaped ? escaped : "<allocation failed>");
|
||||
free(escaped);
|
||||
log_ssl_errors();
|
||||
return false;
|
||||
}
|
||||
int use_ok = SSL_CTX_use_PrivateKey(ctx, pkey);
|
||||
EVP_PKEY_free(pkey);
|
||||
if (use_ok != 1) {
|
||||
char* escaped = output_escape(key, false);
|
||||
log_message(LOG_LEVEL_ERROR, "Failed to use private key: %s",
|
||||
escaped ? escaped : "<allocation failed>");
|
||||
free(escaped);
|
||||
log_ssl_errors();
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
static SSL_CTX* create_ssl_ctx(bool is_server, const char* cert, const char* key,
|
||||
const char* ca_path) {
|
||||
if (!is_server && !ca_path) {
|
||||
@@ -56,12 +111,21 @@ static SSL_CTX* create_ssl_ctx(bool is_server, const char* cert, const char* key
|
||||
#ifdef SSL_OP_NO_RENEGOTIATION
|
||||
SSL_CTX_set_options(ctx, SSL_OP_NO_RENEGOTIATION);
|
||||
#endif
|
||||
/* Let the server's own preference order decide the negotiated cipher rather
|
||||
* than the client's, so a client cannot steer both peers into a weaker (but
|
||||
* still offered) suite. */
|
||||
SSL_CTX_set_options(ctx, SSL_OP_CIPHER_SERVER_PREFERENCE);
|
||||
|
||||
if (SSL_CTX_set_min_proto_version(ctx, TLS1_2_VERSION) != 1) {
|
||||
SSL_CTX_free(ctx);
|
||||
return NULL;
|
||||
}
|
||||
if (SSL_CTX_set_cipher_list(ctx, "HIGH:!aNULL:!eNULL:!MD5:!RC4:!3DES") != 1) {
|
||||
/* TLS 1.2 and below: an AEAD-only suite list. "HIGH" still includes CBC
|
||||
* suites (Lucky13/POODLE-adjacent MAC-then-encrypt constructions), so restrict
|
||||
* the list to ECDHE key agreement with an AEAD record cipher (AES-GCM or
|
||||
* ChaCha20-Poly1305). A NULL/weak/3DES cipher is never selectable. */
|
||||
if (SSL_CTX_set_cipher_list(ctx, "ECDHE+AESGCM:ECDHE+CHACHA20:!aNULL:!eNULL:!MD5:!RC4:!3DES") !=
|
||||
1) {
|
||||
SSL_CTX_free(ctx);
|
||||
return NULL;
|
||||
}
|
||||
@@ -79,13 +143,6 @@ static SSL_CTX* create_ssl_ctx(bool is_server, const char* cert, const char* key
|
||||
#endif
|
||||
|
||||
if (cert && key) {
|
||||
struct stat key_stat;
|
||||
if (stat(key, &key_stat) != 0 || !S_ISREG(key_stat.st_mode) || key_stat.st_uid != geteuid() ||
|
||||
(key_stat.st_mode & (S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH))) {
|
||||
log_message(LOG_LEVEL_ERROR, "TLS private key must be owned by the current user and private");
|
||||
SSL_CTX_free(ctx);
|
||||
return NULL;
|
||||
}
|
||||
if (SSL_CTX_use_certificate_file(ctx, cert, SSL_FILETYPE_PEM) <= 0) {
|
||||
char* escaped = output_escape(cert, false);
|
||||
log_message(LOG_LEVEL_ERROR, "Failed to load certificate: %s",
|
||||
@@ -95,12 +152,7 @@ static SSL_CTX* create_ssl_ctx(bool is_server, const char* cert, const char* key
|
||||
SSL_CTX_free(ctx);
|
||||
return NULL;
|
||||
}
|
||||
if (SSL_CTX_use_PrivateKey_file(ctx, key, SSL_FILETYPE_PEM) <= 0) {
|
||||
char* escaped = output_escape(key, false);
|
||||
log_message(LOG_LEVEL_ERROR, "Failed to load private key: %s",
|
||||
escaped ? escaped : "<allocation failed>");
|
||||
free(escaped);
|
||||
log_ssl_errors();
|
||||
if (!load_private_key_secure(ctx, key)) {
|
||||
SSL_CTX_free(ctx);
|
||||
return NULL;
|
||||
}
|
||||
@@ -144,7 +196,16 @@ static SSL* wrap_fd_with_ssl(int fd, SSL_CTX* ctx, bool is_server, const char* h
|
||||
// Enable hostname verification for client connections when a hostname is provided.
|
||||
// Must be done before SSL_connect to take effect during the handshake.
|
||||
if (!is_server && hostname) {
|
||||
if (SSL_set1_host(ssl, hostname) != 1) {
|
||||
/* An IP-literal host must be verified against the certificate's IP SAN
|
||||
* (X509_check_ip_asc), not as a DNS name: SSL_set1_host would look for a
|
||||
* DNS SAN that a legitimate IP-SAN certificate never carries. */
|
||||
struct in_addr ipv4;
|
||||
struct in6_addr ipv6;
|
||||
bool is_ip_literal =
|
||||
inet_pton(AF_INET, hostname, &ipv4) == 1 || inet_pton(AF_INET6, hostname, &ipv6) == 1;
|
||||
int set_ok = is_ip_literal ? X509_VERIFY_PARAM_set1_ip_asc(SSL_get0_param(ssl), hostname)
|
||||
: SSL_set1_host(ssl, hostname);
|
||||
if (set_ok != 1) {
|
||||
SSL_free(ssl);
|
||||
return NULL;
|
||||
}
|
||||
|
||||
+137
-41
@@ -378,59 +378,155 @@ char* output_escape(const char* string, bool eight_bit_output) {
|
||||
return escaped;
|
||||
}
|
||||
|
||||
ssize_t utils_getdelim_bounded(FILE* stream, char** line, size_t* cap, int delim, size_t max_len) {
|
||||
if (!stream || !line || !cap || max_len == 0) {
|
||||
errno = EINVAL;
|
||||
return -1;
|
||||
}
|
||||
size_t limit = max_len + 1; /* content bytes plus the terminating NUL */
|
||||
if (*line == NULL || *cap < 2) {
|
||||
size_t initial = limit < 256 ? limit : 256;
|
||||
char* buf = malloc(initial);
|
||||
if (!buf)
|
||||
return -1;
|
||||
free(*line);
|
||||
*line = buf;
|
||||
*cap = initial;
|
||||
}
|
||||
size_t len = 0;
|
||||
int c;
|
||||
while ((c = getc_unlocked(stream)) != EOF) {
|
||||
if (len >= max_len) {
|
||||
errno = EFBIG;
|
||||
return -1;
|
||||
}
|
||||
if (len + 2 > *cap) {
|
||||
size_t new_cap = *cap * 2;
|
||||
if (new_cap < len + 2)
|
||||
new_cap = len + 2;
|
||||
if (new_cap > limit)
|
||||
new_cap = limit;
|
||||
char* grown = realloc(*line, new_cap);
|
||||
if (!grown)
|
||||
return -1;
|
||||
*line = grown;
|
||||
*cap = new_cap;
|
||||
}
|
||||
(*line)[len++] = (char)c;
|
||||
if (c == delim)
|
||||
break;
|
||||
}
|
||||
if (c == EOF && len == 0)
|
||||
return 0;
|
||||
(*line)[len] = '\0';
|
||||
return (ssize_t)len;
|
||||
}
|
||||
|
||||
/* Match a glob pattern against a string. Supported wildcards:
|
||||
* ? matches any single character except '/'.
|
||||
* * matches any sequence of characters within one path component (no '/').
|
||||
* ** matches any sequence of characters, including '/' (cross-directory).
|
||||
* slash-star-star-slash is treated as a cross-directory wildcard when it appears between
|
||||
* literals.
|
||||
*/
|
||||
*
|
||||
* The matcher is an iterative O(pattern * string) dynamic program rather than the
|
||||
* original backtracking recursion: overlapping `*`/`**` wildcards made a pattern
|
||||
* like `*a*a*a*...*b` run in exponential time against a long run of `a`, a CPU
|
||||
* denial-of-service vector reachable from a hostile --exclude/--include pattern
|
||||
* or `.rsync-filter`. The DP reasons over (pattern position, string position)
|
||||
* so every state is visited once; the transitions below mirror the original
|
||||
* recursion exactly. */
|
||||
bool glob_match(const char* pattern, const char* str) {
|
||||
while (*pattern) {
|
||||
if (*pattern == '*') {
|
||||
if (*(pattern + 1) == '*') {
|
||||
/* globstar: match across directories */
|
||||
pattern += 2;
|
||||
if (*pattern == '\0')
|
||||
return true;
|
||||
if (*pattern == '/')
|
||||
pattern++;
|
||||
while (*str) {
|
||||
if (glob_match(pattern, str))
|
||||
return true;
|
||||
str++;
|
||||
}
|
||||
return glob_match(pattern, str);
|
||||
}
|
||||
/* single *: match within one path component */
|
||||
pattern++;
|
||||
while (*str && *str != '/') {
|
||||
if (glob_match(pattern, str))
|
||||
return true;
|
||||
str++;
|
||||
}
|
||||
return glob_match(pattern, str);
|
||||
} else if (*pattern == '?') {
|
||||
if (!*str || *str == '/')
|
||||
if (!pattern || !str)
|
||||
return false;
|
||||
pattern++;
|
||||
str++;
|
||||
} else {
|
||||
if (*pattern != *str) {
|
||||
/* allow literal / ** / rest to match any number of directories */
|
||||
if (*pattern == '/' && *(pattern + 1) == '*' && *(pattern + 2) == '*') {
|
||||
const char* rest = pattern + 3;
|
||||
if (*rest == '/')
|
||||
size_t pattern_len = strlen(pattern);
|
||||
size_t str_len = strlen(str);
|
||||
if (pattern_len == 0)
|
||||
return str_len == 0;
|
||||
/* Defensive work cap: the DP is bounded by pattern*string states, but a
|
||||
* 64 KiB pattern against a 64 KiB path would still cost billions of steps.
|
||||
* Treat the pattern as non-matching above the cap instead of burning CPU. */
|
||||
if (str_len > (SIZE_MAX / (pattern_len + 1)) - 1)
|
||||
return false;
|
||||
if ((pattern_len + 1) * (str_len + 1) > 64u * 1024u * 1024u)
|
||||
return false;
|
||||
|
||||
size_t row_bytes = str_len + 1;
|
||||
/* Rows for pattern positions i, i+1, i+2 and i+3 are live at once (the
|
||||
* globstar transition can skip up to three pattern bytes). Four rotating
|
||||
* rows keep memory at O(string length); a stack buffer avoids an allocation
|
||||
* for the common short-leaf case. */
|
||||
enum { STACK_ROW = 257 };
|
||||
uint8_t stack_rows[4 * STACK_ROW];
|
||||
uint8_t* rows = stack_rows;
|
||||
if (row_bytes > STACK_ROW) {
|
||||
rows = malloc(4 * row_bytes);
|
||||
if (!rows)
|
||||
return false;
|
||||
}
|
||||
|
||||
#define GLOB_ROW(i) (rows + ((pattern_len - (i)) & 3) * row_bytes)
|
||||
|
||||
/* Base row: pattern position `pattern_len` matches only the string's end. */
|
||||
for (size_t j = 0; j <= str_len; j++)
|
||||
GLOB_ROW(pattern_len)[j] = (j == str_len) ? 1 : 0;
|
||||
|
||||
for (size_t i = pattern_len; i-- > 0;) {
|
||||
const char pc = pattern[i];
|
||||
uint8_t* cur = GLOB_ROW(i);
|
||||
const uint8_t* next = GLOB_ROW(i + 1);
|
||||
if (pc == '*') {
|
||||
if (i + 1 < pattern_len && pattern[i + 1] == '*') {
|
||||
/* Globstar: skip `**` and an optional following '/', then consume any
|
||||
* (possibly empty) run of characters -- including '/'. */
|
||||
size_t rest = i + 2;
|
||||
if (rest < pattern_len && pattern[rest] == '/')
|
||||
rest++;
|
||||
return glob_match(rest, str);
|
||||
const uint8_t* rest_row = GLOB_ROW(rest);
|
||||
for (size_t j = str_len + 1; j-- > 0;) {
|
||||
bool v = rest_row[j] != 0;
|
||||
if (!v && j < str_len)
|
||||
v = cur[j + 1] != 0;
|
||||
cur[j] = v ? 1 : 0;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
pattern++;
|
||||
str++;
|
||||
} else {
|
||||
/* Single `*`: zero characters, or one non-'/' character. */
|
||||
for (size_t j = str_len + 1; j-- > 0;) {
|
||||
bool v = next[j] != 0;
|
||||
if (!v && j < str_len && str[j] != '/')
|
||||
v = cur[j + 1] != 0;
|
||||
cur[j] = v ? 1 : 0;
|
||||
}
|
||||
}
|
||||
return *str == '\0';
|
||||
} else if (pc == '?') {
|
||||
for (size_t j = str_len + 1; j-- > 0;) {
|
||||
bool v = j < str_len && str[j] != '/' && next[j + 1] != 0;
|
||||
cur[j] = v ? 1 : 0;
|
||||
}
|
||||
} else {
|
||||
/* Literal: consume an equal character, or -- for a '/' immediately before
|
||||
* a globstar -- let the '/' match zero directories and continue at `**`. */
|
||||
for (size_t j = str_len + 1; j-- > 0;) {
|
||||
bool v = false;
|
||||
if (j < str_len && str[j] == pc) {
|
||||
v = next[j + 1] != 0;
|
||||
} else if (pc == '/' && i + 2 < pattern_len && pattern[i + 1] == '*' &&
|
||||
pattern[i + 2] == '*') {
|
||||
size_t rest = i + 3;
|
||||
if (rest < pattern_len && pattern[rest] == '/')
|
||||
rest++;
|
||||
v = GLOB_ROW(rest)[j] != 0;
|
||||
}
|
||||
cur[j] = v ? 1 : 0;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
bool matched = GLOB_ROW(0)[0] != 0;
|
||||
#undef GLOB_ROW
|
||||
if (rows != stack_rows)
|
||||
free(rows);
|
||||
return matched;
|
||||
}
|
||||
|
||||
bool format_human_bytes(unsigned long long bytes, char* buffer, size_t buffer_size) {
|
||||
|
||||
@@ -4,7 +4,9 @@
|
||||
#include "array_list.h"
|
||||
#include <stddef.h>
|
||||
#include <stdbool.h>
|
||||
#include <stdio.h>
|
||||
#include <sys/socket.h>
|
||||
#include <sys/types.h>
|
||||
|
||||
/* Small open-addressing string hash set used to turn quadratic membership
|
||||
* scans into O(path length) exact-match lookups (the --delete keep-set and the
|
||||
@@ -79,6 +81,17 @@ bool path_index_has_descendant(const PathIndex* index, const char* path);
|
||||
|
||||
char* str_dup(const char* string);
|
||||
char* output_escape(const char* string, bool eight_bit_output);
|
||||
/* Upper bound on one line/token read from a local list file (--files-from,
|
||||
* --exclude-from/--include-from, .rsync-filter). Mirrors MAX_STRING_SIZE and
|
||||
* stops a hostile multi-gigabyte line from forcing unbounded allocation. */
|
||||
#define UTILS_MAX_LINE_LEN (64 * 1024)
|
||||
/* Read one `delim`-terminated record from `stream` into *line (grown as needed
|
||||
* and NUL-terminated), refusing to consume/allocate more than `max_len` bytes
|
||||
* of content. Returns the number of bytes stored (delimiter included, matching
|
||||
* getdelim), 0 at end of file, or -1 on error (errno is EFBIG when the record
|
||||
* exceeds `max_len`, ENOMEM on allocation failure). *line and *cap are updated
|
||||
* as the buffer grows and the caller owns *line. */
|
||||
ssize_t utils_getdelim_bounded(FILE* stream, char** line, size_t* cap, int delim, size_t max_len);
|
||||
char* path_cat(const char* path1, const char* path2);
|
||||
bool glob_match(const char* pattern, const char* str);
|
||||
/* Result of a bounded extra-file deletion run. */
|
||||
@@ -148,6 +161,12 @@ const char* utils_get_authorized_root_path(void);
|
||||
* callers guarantee this); this is containment by string, not by resolved
|
||||
* symlinks. Shared by the utils and file secure-walk root confinement. */
|
||||
bool path_is_within_root(const char* root, const char* path);
|
||||
/* True when `path` contains a ".." component. This is a purely lexical
|
||||
* dot-dot check: an absolute path is NOT rejected here, because default
|
||||
* (non-relative) transfers legitimately put the sender's absolute source path
|
||||
* on the wire and the receiver re-roots it under the destination with
|
||||
* path_cat(). Callers that accept a strictly relative path (e.g. batch paths)
|
||||
* must reject a leading '/' themselves (see utils_valid_batch_path). */
|
||||
bool has_path_traversal(const char* path);
|
||||
bool utils_valid_batch_path(const char* path);
|
||||
bool format_human_bytes(unsigned long long bytes, char* buffer, size_t buffer_size);
|
||||
|
||||
+43
-13
@@ -73,13 +73,17 @@ bool xattr_list_append(FileXattrList* list, const char* name, const void* value,
|
||||
|
||||
/* A Linux xattr name is "namespace.name" with an optional leading "trusted.",
|
||||
* "system.", "security.", "user.", or "trusted." prefix. We only ever touch
|
||||
* the unprivileged "user.*" namespace and the two POSIX ACL xattrs carried in
|
||||
* the "system." namespace. Everything else -- especially "security.*" (ACLs,
|
||||
* capabilities, SELinux labels) and "trusted.*" -- is refused so a client can
|
||||
* never compel the receiver to apply a privileged attribute it would not
|
||||
* otherwise be able to set (and which would be a local privilege escalation if
|
||||
* it could). */
|
||||
bool xattr_name_appliable(const char* name) {
|
||||
* the unprivileged "user.*" namespace and, only when --acls/-A was negotiated,
|
||||
* the two POSIX ACL xattrs carried in the "system." namespace. Everything else
|
||||
* -- especially "security.*" (ACLs, capabilities, SELinux labels) and
|
||||
* "trusted.*" -- is refused so a client can never compel the receiver to apply a
|
||||
* privileged attribute it would not otherwise be able to set (and which would be
|
||||
* a local privilege escalation if it could).
|
||||
*
|
||||
* The ACL gate is deliberate: --xattrs/-X alone derives use_xattrs but must NOT
|
||||
* authorize the ACL names, otherwise a -X client could plant an ACL the
|
||||
* receiver never opted into (B4). */
|
||||
bool xattr_name_appliable(const char* name, bool preserve_acls) {
|
||||
if (!name || name[0] == '\0')
|
||||
return false;
|
||||
size_t len = strlen(name);
|
||||
@@ -95,15 +99,24 @@ bool xattr_name_appliable(const char* name) {
|
||||
if (strncmp(name, "user.", 5) == 0)
|
||||
return name[5] != '\0';
|
||||
if (strcmp(name, "system.posix_acl_access") == 0)
|
||||
return true;
|
||||
return preserve_acls;
|
||||
if (strcmp(name, "system.posix_acl_default") == 0)
|
||||
return true;
|
||||
return preserve_acls;
|
||||
return false;
|
||||
}
|
||||
|
||||
/* The two POSIX ACL xattr names: the only names whose applicablity is
|
||||
* conditional (they require --acls). Used by the receiver to distinguish "not
|
||||
* negotiated" (drop the entry, keep user.* working for -X) from a genuinely
|
||||
* disallowed namespace (hard reject). */
|
||||
static bool xattr_name_is_posix_acl(const char* name) {
|
||||
return name != NULL && (strcmp(name, "system.posix_acl_access") == 0 ||
|
||||
strcmp(name, "system.posix_acl_default") == 0);
|
||||
}
|
||||
|
||||
/* ---- SENDER: capture ---- */
|
||||
|
||||
FileXattrList* xattr_capture_path(const char* path) {
|
||||
FileXattrList* xattr_capture_path(const char* path, bool preserve_acls) {
|
||||
if (!path)
|
||||
return NULL;
|
||||
ssize_t list_size = listxattr(path, NULL, 0);
|
||||
@@ -130,7 +143,10 @@ FileXattrList* xattr_capture_path(const char* path) {
|
||||
if (name_len == 0)
|
||||
break; /* trailing double NUL not expected; stop */
|
||||
offset += (ssize_t)name_len + 1;
|
||||
if (!xattr_name_appliable(name))
|
||||
/* Capture is sender-side: the scanner has already gated on -X/-A, so the
|
||||
per-name whitelist here allows the ACL names only when --acls was
|
||||
negotiated. Without it a plain -X capture never carries an ACL. */
|
||||
if (!xattr_name_appliable(name, preserve_acls))
|
||||
continue;
|
||||
ssize_t value_size = getxattr(path, name, NULL, 0);
|
||||
if (value_size < 0)
|
||||
@@ -190,7 +206,7 @@ bool xattr_send(int fd, const FileXattrList* list) {
|
||||
return true;
|
||||
}
|
||||
|
||||
FileXattrList* xattr_receive(int fd, int* ok) {
|
||||
FileXattrList* xattr_receive(int fd, int* ok, bool preserve_acls) {
|
||||
if (ok)
|
||||
*ok = 0;
|
||||
int count;
|
||||
@@ -232,12 +248,20 @@ FileXattrList* xattr_receive(int fd, int* ok) {
|
||||
xattr_list_free(list);
|
||||
return NULL;
|
||||
}
|
||||
if (!xattr_name_appliable(name)) {
|
||||
bool skip = false;
|
||||
if (!xattr_name_appliable(name, preserve_acls)) {
|
||||
if (!preserve_acls && xattr_name_is_posix_acl(name)) {
|
||||
/* -X without -A: the sender may still carry ACLs, but the receiver must
|
||||
never apply an ACL it was not asked to preserve. Consume and drop the
|
||||
entry (keeping -X compatibility) rather than failing the transfer. */
|
||||
skip = true;
|
||||
} else {
|
||||
log_message(LOG_LEVEL_ERROR, "rejected xattr block: disallowed namespace for '%s'", name);
|
||||
free(name);
|
||||
xattr_list_free(list);
|
||||
return NULL;
|
||||
}
|
||||
}
|
||||
int32_t value_len32;
|
||||
if (!receive_n_data(fd, &value_len32, sizeof(value_len32))) {
|
||||
free(name);
|
||||
@@ -273,6 +297,12 @@ FileXattrList* xattr_receive(int fd, int* ok) {
|
||||
return NULL;
|
||||
}
|
||||
}
|
||||
if (skip) {
|
||||
free(value);
|
||||
free(name);
|
||||
budget += (size_t)name_len32 + (size_t)value_len32;
|
||||
continue;
|
||||
}
|
||||
if (!xattr_list_append(list, name, value, (size_t)value_len32)) {
|
||||
free(value);
|
||||
free(name);
|
||||
|
||||
+17
-9
@@ -59,20 +59,28 @@ void xattr_list_free(FileXattrList* list);
|
||||
bool xattr_list_append(FileXattrList* list, const char* name, const void* value, size_t value_len);
|
||||
|
||||
/* True when `name` is a well-formed xattr name AND belongs to a namespace this
|
||||
* build is authorized to apply (user.* or the two POSIX ACL xattrs). Used for
|
||||
* both capture and receiver-side validation. */
|
||||
bool xattr_name_appliable(const char* name);
|
||||
* build is authorized to apply. `user.*` is always accepted for -X; the two
|
||||
* POSIX ACL xattrs are accepted only when `preserve_acls` (--acls/-A) is set, so
|
||||
* a plain -X run can never carry or apply an ACL the receiver did not ask for.
|
||||
* Used for both capture and receiver-side validation. */
|
||||
bool xattr_name_appliable(const char* name, bool preserve_acls);
|
||||
|
||||
/* Sender: read the whitelisted xattrs of `path` into a new list. Returns NULL
|
||||
* when the path has no appliable xattrs (or the filesystem has no xattr
|
||||
* support); an empty-but-valid list is never returned distinct from NULL. */
|
||||
FileXattrList* xattr_capture_path(const char* path);
|
||||
/* Sender: read the whitelisted xattrs of `path` into a new list. The POSIX ACL
|
||||
* names are captured only when `preserve_acls` (--acls/-A) is set, so a plain
|
||||
* -X run never carries an ACL it was not asked to preserve; `user.*` is
|
||||
* unaffected. Returns NULL when the path has no appliable xattrs (or the
|
||||
* filesystem has no xattr support); an empty-but-valid list is never returned
|
||||
* distinct from NULL. */
|
||||
FileXattrList* xattr_capture_path(const char* path, bool preserve_acls);
|
||||
|
||||
/* Wire: bounded serialization. xattr_send returns false on write failure; an
|
||||
* empty/NULL list transmits a zero-count block. xattr_receive returns NULL and
|
||||
* sets *ok = 0 on any malformed / oversized / non-whitelisted entry. */
|
||||
* sets *ok = 0 on any malformed / oversized / non-whitelisted entry. When
|
||||
* `preserve_acls` is false, any POSIX ACL entries are consumed and DROPPED (so
|
||||
* a -X transfer still succeeds and never applies an ACL it did not negotiate);
|
||||
* a genuinely disallowed namespace is still rejected. */
|
||||
bool xattr_send(int fd, const FileXattrList* list);
|
||||
FileXattrList* xattr_receive(int fd, int* ok);
|
||||
FileXattrList* xattr_receive(int fd, int* ok, bool preserve_acls);
|
||||
|
||||
/* Receiver: apply every entry fd-relative (fsetxattr) to the just-written file
|
||||
* descriptor. A per-attribute failure (e.g. ACL set refused for non-root on a
|
||||
|
||||
+7
-1
@@ -16,7 +16,13 @@ def shared_server():
|
||||
Under pytest-xdist this session fixture is instantiated once per worker
|
||||
process, so each worker gets its own server on an ephemeral port."""
|
||||
server = ServerManager()
|
||||
server.start()
|
||||
# --allow-super keeps the historical permissive super mode for a root
|
||||
# receiver: the integration suite's root-only ownership/device/copy-as tests
|
||||
# exercise that opted-in configuration. The secure default (a root
|
||||
# standalone server without --allow-super forces SUPER_MODE_OFF) is covered
|
||||
# explicitly by TestStandaloneSuperDefault in test_features.py. Non-root
|
||||
# runs are unaffected by the flag.
|
||||
server.start(extra_args=["--allow-super"])
|
||||
yield server
|
||||
server.stop()
|
||||
|
||||
|
||||
@@ -75,7 +75,7 @@ static void write_best_effort(int fd, const void* data, size_t size) {
|
||||
}
|
||||
|
||||
static void receive_stream(const unsigned char* prefix, size_t prefix_len, const uint8_t* data,
|
||||
size_t size) {
|
||||
size_t size, bool preserve_acls) {
|
||||
int sv[2];
|
||||
if (socketpair(AF_UNIX, SOCK_STREAM, 0, sv) != 0)
|
||||
return;
|
||||
@@ -91,7 +91,7 @@ static void receive_stream(const unsigned char* prefix, size_t prefix_len, const
|
||||
shutdown(sv[0], SHUT_WR);
|
||||
|
||||
int ok = 0;
|
||||
FileXattrList* list = xattr_receive(sv[1], &ok);
|
||||
FileXattrList* list = xattr_receive(sv[1], &ok, preserve_acls);
|
||||
xattr_list_free(list);
|
||||
|
||||
close(sv[0]);
|
||||
@@ -102,14 +102,18 @@ int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) {
|
||||
if (!g_block_ready)
|
||||
build_canonical_block();
|
||||
|
||||
/* Raw bytes as the whole block. */
|
||||
receive_stream(NULL, 0, data, size);
|
||||
/* Raw bytes as the whole block. Exercise both the -X-only (no ACLs) and the
|
||||
* -A (ACL names accepted) receiver gates. */
|
||||
for (int acls = 0; acls < 2; acls++) {
|
||||
bool preserve_acls = acls != 0;
|
||||
receive_stream(NULL, 0, data, size, preserve_acls);
|
||||
|
||||
/* Valid framing so the fuzzer mutates the entry list, the first value and
|
||||
* the second entry respectively instead of stopping at the count. */
|
||||
receive_stream(g_block, g_off_after_entry0, data, size);
|
||||
receive_stream(g_block, g_off_value0, data, size);
|
||||
receive_stream(g_block, g_off_after_count, data, size);
|
||||
receive_stream(g_block, g_off_after_entry0, data, size, preserve_acls);
|
||||
receive_stream(g_block, g_off_value0, data, size, preserve_acls);
|
||||
receive_stream(g_block, g_off_after_count, data, size, preserve_acls);
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
@@ -118,3 +118,15 @@ def test_batch_modes_conflict():
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, timeout=180)
|
||||
assert result.returncode != 0, \
|
||||
f"expected conflict failure for {flags}: {result.stderr}"
|
||||
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_dry_run_rejects_write_batch():
|
||||
"""--dry-run must not emit a batch file (it must not mutate anything)."""
|
||||
if os.path.exists(BATCH_FILE):
|
||||
os.unlink(BATCH_FILE)
|
||||
cmd = _run(["--source-dir", SOURCE_DIR, "--dest-dir", DEST1,
|
||||
"--dry-run", "--write-batch", BATCH_FILE])
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, timeout=180)
|
||||
assert result.returncode != 0, result.stderr
|
||||
assert not os.path.exists(BATCH_FILE), "dry-run must not create a batch file"
|
||||
@@ -43,6 +43,9 @@ from common import (
|
||||
_find_free_port,
|
||||
_wait_for_port,
|
||||
)
|
||||
# The dry-run no-mutation contract is asserted with the same structural snapshot
|
||||
# (mode/inode/mtime/xattr/content) the feature suite uses.
|
||||
from test_features import _snapshot_tree
|
||||
|
||||
SOURCE_DIR = os.path.join(TEST_DATA_DIR, "daemon_source")
|
||||
MODULE_ROOT = os.path.join(TEST_DATA_DIR, "daemon_modules")
|
||||
@@ -355,6 +358,33 @@ class TestDaemonRejection:
|
||||
assert result.returncode != 0
|
||||
assert self._tree_files() == before, "read-only rejection wrote under the module root"
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_read_only_module_allows_dry_run(self, daemon):
|
||||
"""A server-contacting --dry-run IS a read-only wire operation, so a
|
||||
`read only = yes` module is the safest dry-run target and must accept it
|
||||
while writing nothing."""
|
||||
result, _ = run_client(SOURCE_DIR, "127.0.0.1::readonly", flags=["--dry-run"],
|
||||
port=daemon.port)
|
||||
assert result.returncode == 0, (result.stderr or result.stdout)[:300]
|
||||
assert "Dry run:" in result.stdout, result.stdout[:200]
|
||||
assert _tree_file_count(READONLY_MODULE) == 0, "read-only dry-run wrote a file"
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_module_dry_run_mutates_nothing(self, daemon):
|
||||
"""A daemon-module dry-run reports would-transfer entries but leaves the
|
||||
module tree structurally identical (mode/inode/mtime/xattr/content)."""
|
||||
result = _push("127.0.0.1::files", daemon.port)
|
||||
assert result.returncode == 0, result.stderr or result.stdout
|
||||
before = _snapshot_tree(FILES_MODULE)
|
||||
# --ignore-times forces every regular file to be reported as
|
||||
# would-transfer, so the dry-run exercises the receiver decision rather
|
||||
# than an all-skip shortcut -- while still mutating nothing.
|
||||
result, _ = run_client(SOURCE_DIR, "127.0.0.1::files",
|
||||
flags=["--dry-run", "--ignore-times"], port=daemon.port)
|
||||
assert result.returncode == 0, (result.stderr or result.stdout)[:300]
|
||||
assert "Dry run:" in result.stdout, result.stdout[:200]
|
||||
assert _snapshot_tree(FILES_MODULE) == before, "daemon dry-run mutated the module root"
|
||||
|
||||
def test_unknown_module_rejected(self, daemon):
|
||||
result = _push("127.0.0.1::no-such-module", daemon.port)
|
||||
assert result.returncode != 0
|
||||
|
||||
@@ -36,7 +36,7 @@ from common import ( # noqa: E402
|
||||
verify_transfer,
|
||||
)
|
||||
|
||||
PROTOCOL_VERSION = b"2.20.0"
|
||||
PROTOCOL_VERSION = b"2.21.0"
|
||||
STATUS_MANIFEST = 5
|
||||
STATUS_OK = 0
|
||||
|
||||
|
||||
@@ -370,6 +370,384 @@ class TestDryRun:
|
||||
assert not mismatches, f"Mismatch: {mismatches}"
|
||||
|
||||
|
||||
def _snapshot_xattrs(path):
|
||||
"""Return a stable, comparable tuple of (name, value) xattr pairs.
|
||||
|
||||
Returns None when the platform/filesystem does not expose xattrs so both
|
||||
snapshots agree on "unavailable" instead of one being treated as changed."""
|
||||
try:
|
||||
names = os.listxattr(path, follow_symlinks=False)
|
||||
except (AttributeError, OSError):
|
||||
return None
|
||||
if not names:
|
||||
return ()
|
||||
pairs = []
|
||||
for name in sorted(names):
|
||||
try:
|
||||
value = os.getxattr(path, name, follow_symlinks=False)
|
||||
except OSError:
|
||||
value = None
|
||||
pairs.append((name, value))
|
||||
return tuple(pairs)
|
||||
|
||||
|
||||
def _snapshot_tree(root):
|
||||
"""Return a structural snapshot of a directory tree.
|
||||
|
||||
Every entry (including directories) is recorded as
|
||||
(inode, mtime_ns, mode, xattrs, kind-specific payload) so a dry-run that
|
||||
touched a mode, inode, mtime, xattr, or content is observable. Regular
|
||||
files carry their size+bytes, symlinks their target, and special entries
|
||||
(FIFO/socket/device) their size only -- opening a special file could block.
|
||||
Returns an empty dict for a missing root so "nothing was created" is also
|
||||
observable."""
|
||||
snapshot = {}
|
||||
if not os.path.exists(root):
|
||||
return snapshot
|
||||
for dirpath, dirnames, filenames in os.walk(root):
|
||||
for name in list(dirnames) + filenames:
|
||||
path = os.path.join(dirpath, name)
|
||||
rel = os.path.relpath(path, root)
|
||||
st = os.lstat(path)
|
||||
entry = [st.st_ino, st.st_mtime_ns, stat.S_IMODE(st.st_mode), _snapshot_xattrs(path)]
|
||||
if stat.S_ISLNK(st.st_mode):
|
||||
entry.append(("symlink", os.readlink(path)))
|
||||
elif stat.S_ISREG(st.st_mode):
|
||||
with open(path, "rb") as fh:
|
||||
data = fh.read()
|
||||
entry += [st.st_size, data]
|
||||
else:
|
||||
entry.append(st.st_size)
|
||||
snapshot[rel] = tuple(entry)
|
||||
return snapshot
|
||||
|
||||
|
||||
class TestRemoteDryRun:
|
||||
"""Server-contacting --dry-run (protocol 2.21.0): contacts the receiver,
|
||||
reports what WOULD transfer/skip based on receiver state, and mutates
|
||||
nothing on either side."""
|
||||
|
||||
def _seed(self, source):
|
||||
clean_dir(source)
|
||||
os.makedirs(os.path.join(source, "nested"), exist_ok=True)
|
||||
with open(os.path.join(source, "keep.txt"), "wb") as f:
|
||||
f.write(b"unchanged content\n")
|
||||
with open(os.path.join(source, "changed.txt"), "wb") as f:
|
||||
f.write(b"original content\n")
|
||||
with open(os.path.join(source, "nested", "deep.txt"), "wb") as f:
|
||||
f.write(b"deep file\n")
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_remote_dry_run_reports_changes_and_mutates_nothing(self, shared_server):
|
||||
source = os.path.join(TEST_DATA_DIR, "remote_dry_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "remote_dry_dst")
|
||||
self._seed(source)
|
||||
clean_dir(dest)
|
||||
|
||||
# Populate the destination with a real transfer that preserves mtimes
|
||||
# (--preserve), then make exactly one file differ (content+size) and add
|
||||
# a brand-new file.
|
||||
result, _ = run_client(source, dest, flags=["--preserve"], port=shared_server.port)
|
||||
assert result.returncode == 0, f"seed transfer failed: {result.stderr[:200]}"
|
||||
received = get_dest_received_dir(dest, source)
|
||||
|
||||
with open(os.path.join(source, "changed.txt"), "wb") as f:
|
||||
f.write(b"a much longer replacement payload\n")
|
||||
with open(os.path.join(source, "added.txt"), "wb") as f:
|
||||
f.write(b"newly added\n")
|
||||
|
||||
before = _snapshot_tree(received)
|
||||
# --checksum must NOT read destination contents in a dry-run (B3), so
|
||||
# the up-to-date decision is metadata-only. The --preserve seed made
|
||||
# keep.txt and deep.txt size+mtime-identical; the dry-run must also
|
||||
# transmit metadata (--preserve) for that metadata to be comparable.
|
||||
result, _ = run_client(source, dest, flags=["--dry-run", "--checksum", "--preserve"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode == 0, f"remote dry-run failed: {result.stderr[:300]}"
|
||||
assert "Dry run:" in result.stdout, result.stdout[:200]
|
||||
assert "changed.txt" in result.stdout, result.stdout
|
||||
assert "added.txt" in result.stdout, result.stdout
|
||||
assert "keep.txt" not in result.stdout, (
|
||||
f"up-to-date file must not be reported as would-transfer: {result.stdout}"
|
||||
)
|
||||
assert "deep.txt" not in result.stdout, result.stdout
|
||||
assert _snapshot_tree(received) == before, "remote dry-run mutated the destination"
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_remote_dry_run_checksum_does_not_read_destination(self, shared_server):
|
||||
"""B3: --dry-run --checksum against a read-only module must not read the
|
||||
destination file's content (a 1-bit hash oracle). A same-size/same-content
|
||||
file whose mtime differs is therefore reported as would-transfer because
|
||||
the metadata-only decision is inconclusive, instead of being hashed and
|
||||
silently skipped."""
|
||||
source = os.path.join(TEST_DATA_DIR, "remote_dry_oracle_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "remote_dry_oracle_dst")
|
||||
self._seed(source)
|
||||
clean_dir(dest)
|
||||
result, _ = run_client(source, dest, flags=["--preserve"], port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:200]
|
||||
received = get_dest_received_dir(dest, source)
|
||||
|
||||
target = os.path.join(received, "keep.txt")
|
||||
# Identical size and content, but a deliberately different mtime.
|
||||
os.utime(target, (1000000000, 1000000000))
|
||||
before = _snapshot_tree(received)
|
||||
|
||||
result, _ = run_client(source, dest, flags=["--dry-run", "--checksum", "--preserve"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:300]
|
||||
assert "keep.txt" in result.stdout, (
|
||||
f"dry-run --checksum must not read the destination to prove equality: {result.stdout}"
|
||||
)
|
||||
assert _snapshot_tree(received) == before, "dry-run mutated the destination"
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_remote_dry_run_into_empty_dest_creates_nothing(self, shared_server):
|
||||
source = os.path.join(TEST_DATA_DIR, "remote_dry_empty_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "remote_dry_empty_dst")
|
||||
self._seed(source)
|
||||
clean_dir(dest)
|
||||
received = get_dest_received_dir(dest, source)
|
||||
assert not os.path.exists(received)
|
||||
|
||||
result, _ = run_client(source, dest, flags=["--dry-run"], port=shared_server.port)
|
||||
assert result.returncode == 0, f"exit {result.returncode}: {result.stderr[:300]}"
|
||||
assert "keep.txt" in result.stdout
|
||||
assert "changed.txt" in result.stdout
|
||||
assert "deep.txt" in result.stdout
|
||||
# Nowhere may the receiver have created the destination mirror.
|
||||
assert not os.path.exists(received), "dry-run created directories on the receiver"
|
||||
assert _snapshot_tree(received) == {}
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_remote_dry_run_mkpath_does_not_create_root(self, shared_server):
|
||||
"""A wire dry_run cannot make --mkpath create anything, and it cannot
|
||||
relax the precondition either: a nonexistent root is rejected (a real
|
||||
run without the created root is impossible in dry-run) while nothing is
|
||||
created."""
|
||||
source = os.path.join(TEST_DATA_DIR, "remote_dry_mk_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "remote_dry_mk_dst")
|
||||
self._seed(source)
|
||||
shutil.rmtree(dest, ignore_errors=True)
|
||||
assert not os.path.exists(dest)
|
||||
|
||||
result, _ = run_client(source, dest, flags=["--dry-run", "--mkpath"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode != 0, "dry-run --mkpath accepted a nonexistent receive root"
|
||||
assert not os.path.exists(dest), "dry-run --mkpath created the destination root"
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_remote_dry_run_with_delete_does_not_delete(self, shared_server):
|
||||
source = os.path.join(TEST_DATA_DIR, "remote_dry_del_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "remote_dry_del_dst")
|
||||
self._seed(source)
|
||||
clean_dir(dest)
|
||||
result, _ = run_client(source, dest, port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:200]
|
||||
received = get_dest_received_dir(dest, source)
|
||||
extra = os.path.join(received, "extra.txt")
|
||||
with open(extra, "wb") as f:
|
||||
f.write(b"must survive a dry-run delete\n")
|
||||
before = _snapshot_tree(received)
|
||||
|
||||
for flags in (["--dry-run", "--delete"], ["--dry-run", "--delete-after"]):
|
||||
result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
|
||||
assert result.returncode == 0, f"{flags}: {result.stderr[:300]}"
|
||||
assert os.path.exists(extra), f"{flags} deleted an extra in dry-run"
|
||||
assert _snapshot_tree(received) == before, f"{flags} mutated the destination"
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_remote_dry_run_quiet_is_silent(self, shared_server):
|
||||
source = os.path.join(TEST_DATA_DIR, "remote_dry_quiet_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "remote_dry_quiet_dst")
|
||||
self._seed(source)
|
||||
clean_dir(dest)
|
||||
result, _ = run_client(source, dest, flags=["-q", "--dry-run"], port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:300]
|
||||
assert result.stdout == ""
|
||||
assert result.stderr == ""
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_remote_dry_run_threaded_routes_to_server(self, shared_server):
|
||||
source = os.path.join(TEST_DATA_DIR, "remote_dry_mt_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "remote_dry_mt_dst")
|
||||
self._seed(source)
|
||||
clean_dir(dest)
|
||||
result, _ = run_client(source, dest, flags=["--dry-run", "--threads"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:300]
|
||||
assert "changed.txt" in result.stdout
|
||||
assert _snapshot_tree(get_dest_received_dir(dest, source)) == {}
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_normal_transfer_unaffected_by_dry_run(self, shared_server):
|
||||
"""A real transfer after dry-run still installs the changes."""
|
||||
source = os.path.join(TEST_DATA_DIR, "remote_dry_normal_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "remote_dry_normal_dst")
|
||||
self._seed(source)
|
||||
clean_dir(dest)
|
||||
run_client(source, dest, port=shared_server.port)
|
||||
received = get_dest_received_dir(dest, source)
|
||||
with open(os.path.join(source, "changed.txt"), "wb") as f:
|
||||
f.write(b"updated payload for the real transfer\n")
|
||||
run_client(source, dest, flags=["--dry-run"], port=shared_server.port)
|
||||
|
||||
result, _ = run_client(source, dest, port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:200]
|
||||
with open(os.path.join(received, "changed.txt"), "rb") as f:
|
||||
assert f.read() == b"updated payload for the real transfer\n"
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_remote_dry_run_delay_updates_mutates_nothing(self, shared_server):
|
||||
"""--delay-updates stages under the receive root; a dry-run must neither
|
||||
create that staging tree nor publish anything (mode/inode/mtime intact)."""
|
||||
source = os.path.join(TEST_DATA_DIR, "remote_dry_delay_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "remote_dry_delay_dst")
|
||||
self._seed(source)
|
||||
clean_dir(dest)
|
||||
result, _ = run_client(source, dest, flags=["--delay-updates"], port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:200]
|
||||
|
||||
with open(os.path.join(source, "changed.txt"), "wb") as f:
|
||||
f.write(b"changed for delay-updates dry-run\n")
|
||||
before = _snapshot_tree(dest)
|
||||
result, _ = run_client(source, dest, flags=["--dry-run", "--delay-updates"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:300]
|
||||
assert "changed.txt" in result.stdout, result.stdout
|
||||
assert _snapshot_tree(dest) == before, "delay-updates dry-run mutated the destination"
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_remote_dry_run_backup_mutates_nothing(self, shared_server):
|
||||
"""--backup would rename the old file aside; a dry-run must not."""
|
||||
source = os.path.join(TEST_DATA_DIR, "remote_dry_backup_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "remote_dry_backup_dst")
|
||||
self._seed(source)
|
||||
clean_dir(dest)
|
||||
result, _ = run_client(source, dest, port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:200]
|
||||
|
||||
with open(os.path.join(source, "changed.txt"), "wb") as f:
|
||||
f.write(b"changed for backup dry-run\n")
|
||||
before = _snapshot_tree(dest)
|
||||
result, _ = run_client(source, dest, flags=["--dry-run", "--backup"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:300]
|
||||
assert "changed.txt" in result.stdout, result.stdout
|
||||
assert _snapshot_tree(dest) == before, "--backup dry-run mutated the destination"
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_remote_dry_run_symlink_mutates_nothing(self, shared_server):
|
||||
source = os.path.join(TEST_DATA_DIR, "remote_dry_symlink_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "remote_dry_symlink_dst")
|
||||
self._seed(source)
|
||||
os.symlink("changed.txt", os.path.join(source, "link"))
|
||||
clean_dir(dest)
|
||||
result, _ = run_client(source, dest, flags=["-a"], port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:200]
|
||||
received = get_dest_received_dir(dest, source)
|
||||
assert os.path.islink(os.path.join(received, "link"))
|
||||
|
||||
# Re-point the source link so the entry is genuinely stale, then prove a
|
||||
# dry-run leaves the destination link target, inode, and mtime untouched.
|
||||
os.unlink(os.path.join(source, "link"))
|
||||
os.symlink("keep.txt", os.path.join(source, "link"))
|
||||
before = _snapshot_tree(dest)
|
||||
result, _ = run_client(source, dest, flags=["-a", "--dry-run"], port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:300]
|
||||
assert _snapshot_tree(dest) == before, "symlink dry-run mutated the destination"
|
||||
assert os.readlink(os.path.join(received, "link")) == "changed.txt"
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_remote_dry_run_hardlink_mutates_nothing(self, shared_server):
|
||||
source = os.path.join(TEST_DATA_DIR, "remote_dry_hardlink_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "remote_dry_hardlink_dst")
|
||||
clean_dir(source)
|
||||
clean_dir(dest)
|
||||
with open(os.path.join(source, "h1.txt"), "wb") as f:
|
||||
f.write(b"hardlinked payload\n")
|
||||
os.link(os.path.join(source, "h1.txt"), os.path.join(source, "h2.txt"))
|
||||
result, _ = run_client(source, dest, flags=["-H"], port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:200]
|
||||
received = get_dest_received_dir(dest, source)
|
||||
assert os.stat(os.path.join(received, "h1.txt")).st_ino == \
|
||||
os.stat(os.path.join(received, "h2.txt")).st_ino
|
||||
|
||||
# Change the shared inode; both names are now stale in the destination.
|
||||
with open(os.path.join(source, "h1.txt"), "wb") as f:
|
||||
f.write(b"changed hardlinked payload\n")
|
||||
before = _snapshot_tree(dest)
|
||||
result, _ = run_client(source, dest, flags=["-H", "--dry-run"], port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:300]
|
||||
assert _snapshot_tree(dest) == before, "hardlink dry-run mutated the destination"
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_remote_dry_run_fifo_special_mutates_nothing(self, shared_server):
|
||||
source = os.path.join(TEST_DATA_DIR, "remote_dry_fifo_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "remote_dry_fifo_dst")
|
||||
clean_dir(source)
|
||||
clean_dir(dest)
|
||||
with open(os.path.join(source, "plain.txt"), "wb") as f:
|
||||
f.write(b"plain\n")
|
||||
os.mkfifo(os.path.join(source, "existing.fifo"))
|
||||
result, _ = run_client(source, dest, flags=["--specials"], port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:200]
|
||||
received = get_dest_received_dir(dest, source)
|
||||
assert stat.S_ISFIFO(os.lstat(os.path.join(received, "existing.fifo")).st_mode)
|
||||
|
||||
os.mkfifo(os.path.join(source, "new.fifo"))
|
||||
before = _snapshot_tree(dest)
|
||||
result, _ = run_client(source, dest, flags=["--specials", "--dry-run"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:300]
|
||||
assert not os.path.exists(os.path.join(received, "new.fifo")), \
|
||||
"dry-run created a FIFO on the receiver"
|
||||
assert _snapshot_tree(dest) == before, "special-node dry-run mutated the destination"
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_read_batch_with_dry_run_is_refused(self, shared_server):
|
||||
"""A dry-run of a local batch apply is meaningless (and must not become a
|
||||
mutation escape hatch): the CLI rejects the combination up front."""
|
||||
source = os.path.join(TEST_DATA_DIR, "remote_dry_batch_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "remote_dry_batch_dst")
|
||||
self._seed(source)
|
||||
clean_dir(dest)
|
||||
result, _ = run_client(source, dest, flags=["--read-batch=/nonexistent.batch", "--dry-run"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode != 0, "read-batch + dry-run was accepted"
|
||||
combined = (result.stderr or "") + (result.stdout or "")
|
||||
assert "cannot be combined" in combined or "--dry-run" in combined, combined[:300]
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_remote_dry_run_bad_root_fails_like_real_run(self, shared_server):
|
||||
"""A wire dry_run must not relax the destination-root precondition: a
|
||||
missing or non-directory root that fails a real run fails a dry-run too,
|
||||
and the dry-run must not create/replace anything."""
|
||||
source = os.path.join(TEST_DATA_DIR, "remote_dry_badroot_src")
|
||||
self._seed(source)
|
||||
|
||||
missing = os.path.join(TEST_DATA_DIR, "remote_dry_badroot_missing")
|
||||
shutil.rmtree(missing, ignore_errors=True)
|
||||
real, _ = run_client(source, missing, port=shared_server.port)
|
||||
assert real.returncode != 0, "real run accepted a missing receive root"
|
||||
assert not os.path.exists(missing), "real run created the missing root"
|
||||
dry, _ = run_client(source, missing, flags=["--dry-run"], port=shared_server.port)
|
||||
assert dry.returncode != 0, "dry-run accepted a missing receive root a real run rejects"
|
||||
assert not os.path.exists(missing), "dry-run created the missing receive root"
|
||||
|
||||
fileroot = os.path.join(TEST_DATA_DIR, "remote_dry_badroot_file")
|
||||
shutil.rmtree(fileroot, ignore_errors=True)
|
||||
with open(fileroot, "wb") as f:
|
||||
f.write(b"i am a regular file, not a directory\n")
|
||||
real, _ = run_client(source, fileroot, port=shared_server.port)
|
||||
assert real.returncode != 0, "real run accepted a regular-file receive root"
|
||||
dry, _ = run_client(source, fileroot, flags=["--dry-run"], port=shared_server.port)
|
||||
assert dry.returncode != 0, "dry-run accepted a regular-file receive root a real run rejects"
|
||||
with open(fileroot, "rb") as f:
|
||||
assert f.read() == b"i am a regular file, not a directory\n", \
|
||||
"dry-run clobbered a regular-file receive root"
|
||||
|
||||
|
||||
class TestRemoveSourceFiles:
|
||||
def test_removes_only_transferred_regular_files(self, shared_server):
|
||||
source = os.path.join(TEST_DATA_DIR, "remove_source")
|
||||
@@ -1214,8 +1592,33 @@ class TestDelete:
|
||||
assert not missing, f"Missing: {missing}"
|
||||
assert not mismatches, f"Mismatch: {mismatches}"
|
||||
|
||||
|
||||
class TestProgress:
|
||||
@pytest.mark.ci
|
||||
def test_force_cannot_replace_directory_without_allow_delete(self):
|
||||
"""C2: --force is deletion authority (an incoming file may recursively
|
||||
remove a non-empty destination directory tree). A server started without
|
||||
--allow-delete must clear it, so the operator's delete policy cannot be
|
||||
bypassed with --force."""
|
||||
source = os.path.join(TEST_DATA_DIR, "force_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "force_dst")
|
||||
clean_dir(source)
|
||||
clean_dir(dest)
|
||||
with open(os.path.join(source, "blocker"), "wb") as f:
|
||||
f.write(b"incoming file\n")
|
||||
received = get_dest_received_dir(dest, source)
|
||||
blocker = os.path.join(received, "blocker")
|
||||
os.makedirs(blocker)
|
||||
nested = os.path.join(blocker, "nested.txt")
|
||||
with open(nested, "w") as f:
|
||||
f.write("survivor")
|
||||
# Deliberately NO --allow-delete.
|
||||
server = ServerManager()
|
||||
server.start()
|
||||
try:
|
||||
run_client(source, dest, flags=["--force"], port=server.port)
|
||||
finally:
|
||||
server.stop()
|
||||
assert os.path.isdir(blocker), "unauthorized --force removed a destination directory"
|
||||
assert os.path.exists(nested), "unauthorized --force removed a nested file"
|
||||
def test_progress_output(self, shared_server):
|
||||
clean_dir(DEST_DIR)
|
||||
result, dur = run_client(
|
||||
@@ -3435,6 +3838,27 @@ class TestBasisDestDirs:
|
||||
assert _read_file(os.path.join(received, self.ADDED)) == \
|
||||
self._source_tree("c")[self.ADDED], "added file not transferred"
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_dry_run_compare_dest_does_not_read_basis(self, shared_server):
|
||||
# A dry-run --compare-dest must never read/hash the basis file: doing so
|
||||
# is a 1-bit content oracle against the client-supplied digest. Even a
|
||||
# byte-identical basis with a matching size+mtime is therefore reported
|
||||
# as would-transfer, and nothing is created.
|
||||
source = self._make_source("basis_dry_src", {self.UNCHANGED: b"stable content v1\n"})
|
||||
dest = os.path.join(TEST_DATA_DIR, "basis_dry_dst")
|
||||
clean_dir(dest)
|
||||
self._seed_basis(dest, source, "drybasis", {self.UNCHANGED: b"stable content v1\n"})
|
||||
before = _snapshot_tree(dest)
|
||||
result, _ = run_client(source, dest,
|
||||
flags=["--compare-dest=drybasis", "--dry-run"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode == 0, \
|
||||
f"dry-run compare-dest failed: {result.stderr[:300]}"
|
||||
assert self.UNCHANGED in result.stdout, (
|
||||
"dry-run compare-dest silently skipped: receiver read the basis content"
|
||||
)
|
||||
assert _snapshot_tree(dest) == before, "dry-run compare-dest mutated the destination"
|
||||
|
||||
def test_compare_dest_content_mismatch_forces_transfer(self, shared_server):
|
||||
# The basis holds a file with a DIFFERENT body: even though it shares
|
||||
# the mtime pin, the xxHash check fails and the data must be sent.
|
||||
@@ -4231,6 +4655,61 @@ class TestSuperPrivilege:
|
||||
f"--no-super must suppress fake-super's owner replay: uid={st.st_uid} gid={st.st_gid}"
|
||||
|
||||
|
||||
class TestStandaloneSuperDefault:
|
||||
"""C3: a privileged (root) STANDALONE server without --allow-super forces
|
||||
SUPER_MODE_OFF, so a client cannot make it create device nodes, write raw
|
||||
devices, apply ownership, or use --copy-as. The shared_server fixture opts in
|
||||
with --allow-super to keep the historical behavior available to the existing
|
||||
root-only tests; these tests start their own un-opted server."""
|
||||
|
||||
@pytest.mark.ci
|
||||
def test_copy_as_refused_without_allow_super(self):
|
||||
"""--copy-as is a client-chosen-ownership request and must be refused by
|
||||
a standalone server that did not opt in with --allow-super (on a non-root
|
||||
receiver it is refused for lack of privilege either way)."""
|
||||
source = os.path.join(TEST_DATA_DIR, "super_default_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "super_default_dst")
|
||||
clean_dir(source)
|
||||
clean_dir(dest)
|
||||
with open(os.path.join(source, "f.txt"), "wb") as f:
|
||||
f.write(b"no copy-as\n")
|
||||
server = ServerManager()
|
||||
server.start() # deliberately no --allow-super
|
||||
try:
|
||||
result, _ = run_client(source, dest,
|
||||
flags=["--preserve", "--copy-as=@65534:@65534"],
|
||||
port=server.port)
|
||||
finally:
|
||||
server.stop()
|
||||
assert result.returncode != 0, (
|
||||
"standalone server accepted --copy-as without --allow-super"
|
||||
)
|
||||
|
||||
@pytest.mark.skipif(os.geteuid() != 0, reason="root can create the source device node")
|
||||
def test_devices_skipped_without_allow_super(self):
|
||||
"""Root standalone server without --allow-super must skip device-node
|
||||
creation even for a client --devices request (the run still succeeds and
|
||||
the regular file transfers)."""
|
||||
source = os.path.join(TEST_DATA_DIR, "super_default_dev_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "super_default_dev_dst")
|
||||
clean_dir(source)
|
||||
clean_dir(dest)
|
||||
with open(os.path.join(source, "plain.txt"), "wb") as f:
|
||||
f.write(b"regular\n")
|
||||
os.mknod(os.path.join(source, "null"), stat.S_IFCHR | 0o666, os.makedev(1, 3))
|
||||
server = ServerManager()
|
||||
server.start() # deliberately no --allow-super
|
||||
try:
|
||||
result, _ = run_client(source, dest, flags=["--devices"], port=server.port)
|
||||
finally:
|
||||
server.stop()
|
||||
assert result.returncode == 0, f"exit {result.returncode}: {(result.stderr or '')[:200]}"
|
||||
received = get_dest_received_dir(dest, source)
|
||||
assert not os.path.lexists(os.path.join(received, "null")), (
|
||||
"root standalone server created a device node without --allow-super"
|
||||
)
|
||||
|
||||
|
||||
class TestHardLinks:
|
||||
"""-H/--hard-links: source files sharing an inode are re-created as hard
|
||||
links to one another on the destination (dedup preserved, first copy
|
||||
|
||||
@@ -94,14 +94,14 @@ def _seed_protocol_source(source):
|
||||
class TestProtocol:
|
||||
@pytest.mark.ci
|
||||
def test_protocol_current_version_accepted(self, shared_server):
|
||||
"""--protocol=2.20.0 (the current PROTOCOL_VERSION) is accepted and the
|
||||
"""--protocol=2.21.0 (the current PROTOCOL_VERSION) is accepted and the
|
||||
transfer completes normally."""
|
||||
source = os.path.join(TEST_DATA_DIR, "proto_ok_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, "proto_ok_dst")
|
||||
shutil.rmtree(dest, ignore_errors=True)
|
||||
os.makedirs(dest)
|
||||
_seed_protocol_source(source)
|
||||
result, _ = run_client(source, dest, flags=["--protocol=2.20.0"],
|
||||
result, _ = run_client(source, dest, flags=["--protocol=2.21.0"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode == 0, \
|
||||
f"--protocol current run failed: {(result.stderr or result.stdout)[:400]}"
|
||||
@@ -118,7 +118,7 @@ class TestProtocol:
|
||||
shutil.rmtree(dest, ignore_errors=True)
|
||||
os.makedirs(dest)
|
||||
_seed_protocol_source(source)
|
||||
for bad in ("2.19.0", "2.18.0", "2.17.0", "2.15.0", "2.16.0", "216", "31"):
|
||||
for bad in ("2.20.0", "2.19.0", "2.18.0", "2.17.0", "2.15.0", "2.16.0", "216", "31"):
|
||||
result, _ = run_client(source, dest, flags=[f"--protocol={bad}"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode != 0, f"--protocol={bad} should be rejected"
|
||||
|
||||
@@ -25,6 +25,7 @@
|
||||
#include "test_multiprocessing.h"
|
||||
#include "test_property.h"
|
||||
#include "test_protocol.h"
|
||||
#include "test_protocol_error.h"
|
||||
#include "test_queue.h"
|
||||
#include "test_receiver_timeout.h"
|
||||
#include "test_robustness.h"
|
||||
@@ -65,6 +66,7 @@ int main() {
|
||||
RUN_TEST(test_delta);
|
||||
RUN_TEST(test_data);
|
||||
RUN_TEST(test_protocol);
|
||||
RUN_TEST(test_protocol_error);
|
||||
RUN_TEST(test_receiver_timeout);
|
||||
RUN_TEST(test_metadata);
|
||||
RUN_TEST(test_glob);
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
|
||||
#include "chunk.h"
|
||||
#include "protocol.h"
|
||||
#include "test_utils.h"
|
||||
#include "utils.h"
|
||||
#include <string.h>
|
||||
#include <sys/socket.h>
|
||||
#include <sys/stat.h>
|
||||
#include <unistd.h>
|
||||
|
||||
@@ -279,6 +281,51 @@ static void test_chunk_special_rdev_out_of_range_rejected() {
|
||||
chunk_destroy(chunk);
|
||||
}
|
||||
|
||||
/* B6: chunk_deserialize() charges each retained per-file copy to the owning
|
||||
* session's connection budget (MAX_CONNECTION_MEMORY) so queued chunk payloads
|
||||
* are not held outside the per-connection ceiling; destroying the chunk returns
|
||||
* the charge through the Data.owner path. */
|
||||
static void test_chunk_deserialize_charges_session_budget() {
|
||||
const char* path = "temp_chunk_charge.txt";
|
||||
const char* content = "charge me to the connection budget";
|
||||
unlink(path);
|
||||
file_write_to_disk(path, content, strlen(content), false, false);
|
||||
struct stat st;
|
||||
EXPECT_EQ_INT(stat(path, &st), 0);
|
||||
|
||||
int p[2];
|
||||
EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0);
|
||||
ProtocolSession session;
|
||||
protocol_session_init(&session, p[0], p[1]);
|
||||
protocol_session_set_max_alloc(&session, 4ULL * 1024 * 1024);
|
||||
|
||||
File* f = file_create(path);
|
||||
EXPECT_NOT_NULL(f);
|
||||
f->data->size = (unsigned long long)st.st_size;
|
||||
EXPECT_TRUE(file_load_data(f));
|
||||
File* files[1] = {f};
|
||||
Chunk* chunk = chunk_create(files, 1);
|
||||
EXPECT_NOT_NULL(chunk);
|
||||
Data* serialized = chunk_serialize(chunk, false);
|
||||
EXPECT_NOT_NULL(serialized);
|
||||
/* Simulate a received buffer carrying its owning session. */
|
||||
serialized->owner = &session;
|
||||
|
||||
Chunk* deserialized = chunk_deserialize(serialized, false);
|
||||
EXPECT_NOT_NULL(deserialized);
|
||||
unsigned long long charged = atomic_load(&session.total_allocated_bytes);
|
||||
EXPECT_EQ_INT((int)charged, (int)strlen(content));
|
||||
chunk_destroy(deserialized);
|
||||
/* The copy's charge is released with the File/Data on destroy. */
|
||||
EXPECT_EQ_INT((int)atomic_load(&session.total_allocated_bytes), 0);
|
||||
|
||||
data_destroy(serialized);
|
||||
chunk_destroy(chunk);
|
||||
close(p[0]);
|
||||
close(p[1]);
|
||||
unlink(path);
|
||||
}
|
||||
|
||||
void test_chunk() {
|
||||
test_file_operations();
|
||||
test_chunk_operations();
|
||||
@@ -286,4 +333,5 @@ void test_chunk() {
|
||||
test_chunk_symlink_roundtrip();
|
||||
test_chunk_special_rdev_roundtrip();
|
||||
test_chunk_special_rdev_out_of_range_rejected();
|
||||
test_chunk_deserialize_charges_session_budget();
|
||||
}
|
||||
+88
-4
@@ -306,7 +306,7 @@ static void test_parse_args_protocol_accept_current() {
|
||||
Config* cfg = valid_client_config();
|
||||
EXPECT_NOT_NULL(cfg);
|
||||
char* argv_equals[] = {"fastsync", "--source-dir", "/src",
|
||||
"--dest-dir", "/dst", "--protocol=2.20.0"};
|
||||
"--dest-dir", "/dst", "--protocol=2.21.0"};
|
||||
int positional_args[2];
|
||||
int positional_count = 0;
|
||||
EXPECT_EQ_INT(parse_args(cfg, 6, argv_equals, positional_args, &positional_count), 0);
|
||||
@@ -316,7 +316,7 @@ static void test_parse_args_protocol_accept_current() {
|
||||
cfg = valid_client_config();
|
||||
EXPECT_NOT_NULL(cfg);
|
||||
char* argv_space[] = {"fastsync", "--source-dir", "/src", "--dest-dir",
|
||||
"/dst", "--protocol", "2.20.0"};
|
||||
"/dst", "--protocol", "2.21.0"};
|
||||
positional_count = 0;
|
||||
EXPECT_EQ_INT(parse_args(cfg, 7, argv_space, positional_args, &positional_count), 0);
|
||||
EXPECT_EQ_STR(cfg->version, PROTOCOL_VERSION);
|
||||
@@ -326,8 +326,9 @@ static void test_parse_args_protocol_accept_current() {
|
||||
/* Any --protocol value other than the current PROTOCOL_VERSION must end in
|
||||
* failure (parse_args simply stores it; validate_config rejects it up front). */
|
||||
static void test_parse_args_protocol_rejects_other_versions() {
|
||||
static const char* const bad_versions[] = {
|
||||
"2.17", "2.16", "2.15.0", "2.16.0", "2.17.0", "2.18.0", "2.19.0", "216", "31", "abc", ""};
|
||||
static const char* const bad_versions[] = {"2.17", "2.16", "2.15.0", "2.16.0",
|
||||
"2.17.0", "2.18.0", "2.19.0", "2.20.0",
|
||||
"216", "31", "abc", ""};
|
||||
for (size_t i = 0; i < sizeof(bad_versions) / sizeof(bad_versions[0]); i++) {
|
||||
Config* cfg = valid_client_config();
|
||||
EXPECT_NOT_NULL(cfg);
|
||||
@@ -589,6 +590,9 @@ static void test_parse_args_port_alias() {
|
||||
int positional_count = 0;
|
||||
EXPECT_EQ_INT(parse_args(cfg, 5, argv_space, positional_args, &positional_count), 0);
|
||||
EXPECT_EQ_INT(cfg->server_port, 9000);
|
||||
/* The default port is 8080; the explicit bit is what lets --dry-run tell an
|
||||
explicit remote target from the default and route to the server. */
|
||||
EXPECT_TRUE(cfg->server_port_set);
|
||||
config_delete(cfg);
|
||||
|
||||
cfg = config_create();
|
||||
@@ -596,6 +600,7 @@ static void test_parse_args_port_alias() {
|
||||
positional_count = 0;
|
||||
EXPECT_EQ_INT(parse_args(cfg, 4, argv_inline, positional_args, &positional_count), 0);
|
||||
EXPECT_EQ_INT(cfg->server_port, 9001);
|
||||
EXPECT_TRUE(cfg->server_port_set);
|
||||
config_delete(cfg);
|
||||
|
||||
cfg = config_create();
|
||||
@@ -603,6 +608,29 @@ static void test_parse_args_port_alias() {
|
||||
positional_count = 0;
|
||||
EXPECT_EQ_INT(parse_args(cfg, 4, argv_long, positional_args, &positional_count), 0);
|
||||
EXPECT_EQ_INT(cfg->server_port, 9002);
|
||||
EXPECT_TRUE(cfg->server_port_set);
|
||||
config_delete(cfg);
|
||||
}
|
||||
|
||||
/* An explicit --server-host must set its own routing bit (the field itself
|
||||
* defaults to 127.0.0.1, so a value check cannot distinguish an explicit host
|
||||
* from the default); --dry-run uses it to route to the server. */
|
||||
static void test_parse_args_server_host_sets_routing_bit() {
|
||||
Config* cfg = config_create();
|
||||
int positional_args[2];
|
||||
int positional_count = 0;
|
||||
EXPECT_FALSE(cfg->server_host_set);
|
||||
char* argv_space[] = {"fastsync", "--server-host", "example.test", "/src", "/dst"};
|
||||
EXPECT_EQ_INT(parse_args(cfg, 5, argv_space, positional_args, &positional_count), 0);
|
||||
EXPECT_EQ_STR(cfg->server_host, "example.test");
|
||||
EXPECT_TRUE(cfg->server_host_set);
|
||||
config_delete(cfg);
|
||||
|
||||
cfg = config_create();
|
||||
char* argv_inline[] = {"fastsync", "--server-host=example.test", "/src", "/dst"};
|
||||
positional_count = 0;
|
||||
EXPECT_EQ_INT(parse_args(cfg, 4, argv_inline, positional_args, &positional_count), 0);
|
||||
EXPECT_TRUE(cfg->server_host_set);
|
||||
config_delete(cfg);
|
||||
}
|
||||
|
||||
@@ -3247,6 +3275,58 @@ static void test_parse_args_block_size() {
|
||||
config_delete(cfg);
|
||||
}
|
||||
|
||||
/* An over-long --exclude-from/--include-from line is rejected at parse time
|
||||
* rather than being read without a bound. */
|
||||
static void test_parse_args_pattern_file_oversized_rejected() {
|
||||
const char* list_path = "cli_pattern_oversized.txt";
|
||||
size_t len = UTILS_MAX_LINE_LEN + 4096;
|
||||
char* big = malloc(len);
|
||||
EXPECT_NOT_NULL(big);
|
||||
memset(big, 'a', len);
|
||||
write_file_bytes(list_path, big, len);
|
||||
free(big);
|
||||
|
||||
Config* cfg = config_create();
|
||||
int positional_args[2];
|
||||
int positional_count = 0;
|
||||
char* argv[] = {"fastsync", "--exclude-from", (char*)list_path, "/src", "/dst"};
|
||||
EXPECT_EQ_INT(parse_args(cfg, 5, argv, positional_args, &positional_count), -1);
|
||||
config_delete(cfg);
|
||||
remove(list_path);
|
||||
}
|
||||
|
||||
/* A leading '-'/'+' must be rejected for every unsigned numeric option so
|
||||
* strtoull can never silently wrap (e.g. -1 -> ULLONG_MAX). */
|
||||
static void test_parse_args_unsigned_options_reject_sign() {
|
||||
static const char* const opts[] = {"--chunk-size", "--bwlimit", "--delta-max"};
|
||||
for (size_t i = 0; i < sizeof(opts) / sizeof(opts[0]); i++) {
|
||||
Config* cfg = config_create();
|
||||
int positional_args[2];
|
||||
int positional_count = 0;
|
||||
char* argv[] = {"fastsync", (char*)opts[i], "-1", "/src", "/dst"};
|
||||
EXPECT_EQ_INT(parse_args(cfg, 5, argv, positional_args, &positional_count), -1);
|
||||
config_delete(cfg);
|
||||
}
|
||||
|
||||
/* An over-cap --chunk-size is rejected at parse time (max 64 MiB). */
|
||||
Config* cfg = config_create();
|
||||
int positional_args[2];
|
||||
int positional_count = 0;
|
||||
char* big_argv[] = {"fastsync", "--chunk-size", "67108865", "/src", "/dst"};
|
||||
EXPECT_EQ_INT(parse_args(cfg, 5, big_argv, positional_args, &positional_count), -1);
|
||||
config_delete(cfg);
|
||||
}
|
||||
|
||||
/* --dry-run must not emit a batch file, so it is rejected alongside
|
||||
* --read-batch/--only-write-batch. */
|
||||
static void test_validate_config_dry_run_rejects_write_batch() {
|
||||
Config* cfg = valid_client_config();
|
||||
cfg->dry_run = true;
|
||||
cfg->write_batch = str_dup("batch.dat");
|
||||
EXPECT_FALSE(validate_config(cfg));
|
||||
config_delete(cfg);
|
||||
}
|
||||
|
||||
void test_client_cli() {
|
||||
test_validate_config_required_paths();
|
||||
test_parse_args_numeric_ids();
|
||||
@@ -3300,6 +3380,7 @@ void test_client_cli() {
|
||||
test_parse_args_non_numeric_port();
|
||||
test_parse_args_invalid_server_port();
|
||||
test_parse_args_port_alias();
|
||||
test_parse_args_server_host_sets_routing_bit();
|
||||
test_parse_args_threads();
|
||||
test_client_abort_flag();
|
||||
test_parse_args_invalid_compression_level();
|
||||
@@ -3403,4 +3484,7 @@ void test_client_cli() {
|
||||
test_parse_args_remote_option_short_M();
|
||||
test_parse_args_no_motd();
|
||||
test_parse_args_password_file();
|
||||
test_parse_args_pattern_file_oversized_rejected();
|
||||
test_parse_args_unsigned_options_reject_sign();
|
||||
test_validate_config_dry_run_rejects_write_batch();
|
||||
}
|
||||
@@ -6,8 +6,10 @@
|
||||
#include "utils.h"
|
||||
#include <string.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/wait.h>
|
||||
#include <threads.h>
|
||||
#include <unistd.h>
|
||||
#include <zstd.h>
|
||||
|
||||
static void test_data_compress_decompress_roundtrip() {
|
||||
const char original[] = "Hello, World! This is test data for compression round-trip!";
|
||||
@@ -138,6 +140,63 @@ static void test_chunk_compress_decompress_roundtrip() {
|
||||
unlink(path2);
|
||||
}
|
||||
|
||||
/* Build a zstd frame whose header omits the content size (the content size
|
||||
* flag is cleared), which ZSTD_getFrameContentSize reports as
|
||||
* ZSTD_CONTENTSIZE_UNKNOWN. */
|
||||
static Data* make_unknown_size_frame(const void* src, size_t len) {
|
||||
ZSTD_CCtx* cctx = ZSTD_createCCtx();
|
||||
if (!cctx)
|
||||
return NULL;
|
||||
ZSTD_CCtx_setParameter(cctx, ZSTD_c_contentSizeFlag, 0);
|
||||
size_t cap = ZSTD_compressBound(len);
|
||||
Data* out = data_create_empty(cap);
|
||||
if (!out) {
|
||||
ZSTD_freeCCtx(cctx);
|
||||
return NULL;
|
||||
}
|
||||
ZSTD_inBuffer in = {src, len, 0};
|
||||
ZSTD_outBuffer ob = {out->data, cap, 0};
|
||||
size_t ret;
|
||||
do {
|
||||
ret = ZSTD_compressStream2(cctx, &ob, &in, ZSTD_e_end);
|
||||
if (ZSTD_isError(ret)) {
|
||||
data_destroy(out);
|
||||
ZSTD_freeCCtx(cctx);
|
||||
return NULL;
|
||||
}
|
||||
} while (ret > 0);
|
||||
out->size = ob.pos;
|
||||
ZSTD_freeCCtx(cctx);
|
||||
return out;
|
||||
}
|
||||
|
||||
/* ZSTD_CONTENTSIZE_UNKNOWN is flagged by ZSTD_isError(), so a naive
|
||||
* ZSTD_isError() check rejects every unknown-size frame. Such a frame must
|
||||
* instead reach the 3x estimate fallback and decompress correctly. */
|
||||
static void test_data_decompress_unknown_size_frame() {
|
||||
const char original[] = "unknown-content-size frame: the decompressor must use the 3x estimate, "
|
||||
"not reject the frame as an error.";
|
||||
size_t len = strlen(original);
|
||||
char* buf = malloc(len);
|
||||
EXPECT_NOT_NULL(buf);
|
||||
memcpy(buf, original, len);
|
||||
|
||||
Data* frame = make_unknown_size_frame(buf, len);
|
||||
free(buf);
|
||||
EXPECT_NOT_NULL(frame);
|
||||
/* Guard the premise of the test: the frame really has no stored size. */
|
||||
EXPECT_EQ_INT((int)ZSTD_getFrameContentSize(frame->data, frame->size),
|
||||
(int)ZSTD_CONTENTSIZE_UNKNOWN);
|
||||
|
||||
Data* decompressed = data_decompress(frame);
|
||||
EXPECT_NOT_NULL(decompressed);
|
||||
EXPECT_EQ_INT((int)decompressed->size, (int)len);
|
||||
EXPECT_EQ_INT(memcmp(decompressed->data, original, len), 0);
|
||||
|
||||
data_destroy(decompressed);
|
||||
data_destroy(frame);
|
||||
}
|
||||
|
||||
typedef struct {
|
||||
int id;
|
||||
int iterations;
|
||||
@@ -211,9 +270,52 @@ static void test_data_compress_reused_contexts_multithreaded() {
|
||||
compression_free_thread_contexts();
|
||||
}
|
||||
|
||||
/* A truncated zstd frame used to make the decompressor spin forever: the
|
||||
* stream call keeps returning a positive hint with all input consumed. Run the
|
||||
* decompression in a child with an alarm so a regression (infinite loop) is
|
||||
* caught as a timeout failure instead of hanging the whole unit suite. */
|
||||
static void test_data_decompress_truncated_frame_fails() {
|
||||
const char* original =
|
||||
"The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog.";
|
||||
size_t len = strlen(original);
|
||||
char* buf = malloc(len);
|
||||
EXPECT_NOT_NULL(buf);
|
||||
memcpy(buf, original, len);
|
||||
Data* input = data_create(buf, len);
|
||||
EXPECT_NOT_NULL(input);
|
||||
|
||||
pid_t pid = fork();
|
||||
EXPECT_TRUE(pid >= 0);
|
||||
if (pid == 0) {
|
||||
alarm(10); /* kills the child if the decompressor hangs */
|
||||
Data* compressed = data_compress(input, 3);
|
||||
if (compressed && compressed->size > 1) {
|
||||
compressed->size -= 1; /* drop the final byte: frame is now incomplete */
|
||||
Data* out = data_decompress(compressed);
|
||||
bool failed_cleanly = (out == NULL);
|
||||
data_destroy(out);
|
||||
data_destroy(compressed);
|
||||
/* The child inherited `input` across fork(); free it before _exit so the
|
||||
* valgrind CI job (which instruments forked children too) sees no leak. */
|
||||
data_destroy(input);
|
||||
_exit(failed_cleanly ? 0 : 1);
|
||||
}
|
||||
data_destroy(compressed);
|
||||
data_destroy(input);
|
||||
_exit(2);
|
||||
}
|
||||
int status;
|
||||
waitpid(pid, &status, 0);
|
||||
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
|
||||
|
||||
data_destroy(input);
|
||||
}
|
||||
|
||||
void test_compression() {
|
||||
test_data_compress_decompress_roundtrip();
|
||||
test_data_compress_decompress_large();
|
||||
test_data_decompress_unknown_size_frame();
|
||||
test_data_decompress_truncated_frame_fails();
|
||||
test_skip_compress_suffix_matching();
|
||||
test_data_compress_with_threads_roundtrip();
|
||||
test_data_compress_reused_contexts_multithreaded();
|
||||
|
||||
+38
-6
@@ -87,6 +87,34 @@ static void test_config_ssh_dest_no_user() {
|
||||
config_delete(cfg);
|
||||
}
|
||||
|
||||
/* C1: the user@host token is passed to ssh in option position, so a host or user
|
||||
* beginning with '-' (e.g. "-oProxyCommand=...") must be rejected before any
|
||||
* argv is built, and an empty host must be rejected too. */
|
||||
static void test_config_ssh_dest_rejects_option_injection() {
|
||||
Config* cfg = make_config("1.0", "/src", "-oProxyCommand=id:/dst", true, false, false, false,
|
||||
false, 1, false, 0);
|
||||
EXPECT_EQ_INT(config_parse_ssh_dest(cfg), -1);
|
||||
EXPECT_EQ_INT(cfg->transport, TRANSPORT_TCP);
|
||||
EXPECT_NULL(cfg->ssh_destination);
|
||||
config_delete(cfg);
|
||||
|
||||
cfg =
|
||||
make_config("1.0", "/src", "-user@host:/dst", true, false, false, false, false, 1, false, 0);
|
||||
EXPECT_EQ_INT(config_parse_ssh_dest(cfg), -1);
|
||||
config_delete(cfg);
|
||||
|
||||
cfg = make_config("1.0", "/src", "user@:/dst", true, false, false, false, false, 1, false, 0);
|
||||
EXPECT_EQ_INT(config_parse_ssh_dest(cfg), -1);
|
||||
config_delete(cfg);
|
||||
|
||||
/* config_parse_transport_dest propagates the rejection (and still returns 1
|
||||
* for daemon syntax first). */
|
||||
cfg = make_config("1.0", "/src", "-oProxyCommand=id:/dst", true, false, false, false, false, 1,
|
||||
false, 0);
|
||||
EXPECT_EQ_INT(config_parse_transport_dest(cfg), -1);
|
||||
config_delete(cfg);
|
||||
}
|
||||
|
||||
static void test_config_daemon_dest_parse() {
|
||||
Config* cfg = make_config("1.0", "/src", "dahost::files/sub/dir", true, false, false, false,
|
||||
false, 1, false, 0);
|
||||
@@ -2347,6 +2375,7 @@ static void golden_config_populate(Config* c) {
|
||||
c->compression_level = 7;
|
||||
c->chunk_size = 65536;
|
||||
c->use_sendfile = false;
|
||||
c->dry_run = true;
|
||||
c->use_delete = true;
|
||||
c->use_incremental = true;
|
||||
c->size_only = false;
|
||||
@@ -2398,7 +2427,7 @@ static void golden_config_populate(Config* c) {
|
||||
c->modify_window = 3;
|
||||
c->compress_choice = str_dup("zstd");
|
||||
/* "u=rwx,go=rx" is the same 11 bytes as the original "u=rwX,go=rX" (so the
|
||||
* frame stays 633 bytes) but X is not in FastSync's chmod grammar, and the
|
||||
* frame stays 637 bytes) but X is not in FastSync's chmod grammar, and the
|
||||
* receive-side golden validates the frame. */
|
||||
c->chmod_spec = str_dup("u=rwx,go=rx");
|
||||
c->skip_compress_set = true;
|
||||
@@ -2443,11 +2472,13 @@ static void golden_config_populate(Config* c) {
|
||||
c->copy_as_gid = 222;
|
||||
}
|
||||
|
||||
/* The pinned golden frame (protocol 2.20.0). The values below are the only
|
||||
/* The pinned golden frame (protocol 2.21.0). The values below are the only
|
||||
* thing that ties the generated table to the historical wire format; update
|
||||
* them ONLY with a PROTOCOL_VERSION bump and a documented reason. */
|
||||
#define GOLDEN_WIRE_LEN 633
|
||||
#define GOLDEN_WIRE_HASH 9160991280011164139ULL
|
||||
* them ONLY with a PROTOCOL_VERSION bump and a documented reason. The combined
|
||||
* 2.21.0 wave appends the serialized dry_run bool to CONFIG_WIRE_CORE_FIELDS
|
||||
* and keeps the protocol version string at 2.21.0. */
|
||||
#define GOLDEN_WIRE_LEN 637
|
||||
#define GOLDEN_WIRE_HASH 13228626061067899189ULL
|
||||
|
||||
static unsigned long long fnv1a_64(const unsigned char* buf, size_t len) {
|
||||
unsigned long long h = 1469598103934665603ULL;
|
||||
@@ -2529,7 +2560,7 @@ static unsigned long long capture_wire_hash(const Config* cfg, size_t* out_len)
|
||||
return h;
|
||||
}
|
||||
|
||||
/* Byte-for-byte wire compatibility guard (protocol 2.20.0). The expected hash
|
||||
/* Byte-for-byte wire compatibility guard (protocol 2.21.0). The expected hash
|
||||
* pins the pre-X-macro byte stream; the refactor MUST NOT change it. */
|
||||
static void test_config_wire_golden() {
|
||||
if (is_running_under_valgrind())
|
||||
@@ -2786,6 +2817,7 @@ void test_config() {
|
||||
test_config_ssh_dest();
|
||||
test_config_ssh_dest_local_path();
|
||||
test_config_ssh_dest_no_user();
|
||||
test_config_ssh_dest_rejects_option_injection();
|
||||
test_config_daemon_dest_parse();
|
||||
test_config_daemon_dest_no_path();
|
||||
test_config_daemon_dest_double_slash_normalized();
|
||||
|
||||
@@ -473,6 +473,34 @@ static void test_credentials_store_rejects_legacy_hex() {
|
||||
free(path);
|
||||
}
|
||||
|
||||
/* C9: the legacy-hex detector must check the length before indexing 64 bytes, so
|
||||
* a short secret is never read out of bounds. Such a line is rejected for the
|
||||
* ordinary "expected verifier" reason, never as legacy. */
|
||||
static void test_credentials_store_rejects_short_secret() {
|
||||
char contents[CREDENTIAL_MAX_LINE];
|
||||
snprintf(contents, sizeof(contents), "alice:%s\n", "abc");
|
||||
char* path = make_tmp_file(contents);
|
||||
EXPECT_NOT_NULL(path);
|
||||
char err[512];
|
||||
const CredentialStore* store = credentials_load(path, NULL, err, sizeof(err));
|
||||
EXPECT_NULL(store);
|
||||
EXPECT_TRUE(strstr(err, "legacy unsalted") == NULL);
|
||||
rm_temp(path);
|
||||
free(path);
|
||||
|
||||
char short_hex[64];
|
||||
memset(short_hex, 'a', 63);
|
||||
short_hex[63] = '\0';
|
||||
snprintf(contents, sizeof(contents), "alice:%s\n", short_hex);
|
||||
path = make_tmp_file(contents);
|
||||
EXPECT_NOT_NULL(path);
|
||||
store = credentials_load(path, NULL, err, sizeof(err));
|
||||
EXPECT_NULL(store);
|
||||
EXPECT_TRUE(strstr(err, "legacy unsalted") == NULL);
|
||||
rm_temp(path);
|
||||
free(path);
|
||||
}
|
||||
|
||||
static void test_credentials_store_duplicate_rejected() {
|
||||
char line[CREDENTIAL_MAX_LINE];
|
||||
EXPECT_TRUE(make_store_line("alice", KAT_PASSWORD, CREDENTIAL_MIN_ITERS, line, sizeof(line)));
|
||||
@@ -1066,6 +1094,7 @@ void test_credentials(void) {
|
||||
test_credentials_store_parse_valid();
|
||||
test_credentials_store_parse_rejects_malformed();
|
||||
test_credentials_store_rejects_legacy_hex();
|
||||
test_credentials_store_rejects_short_secret();
|
||||
test_credentials_store_duplicate_rejected();
|
||||
test_credentials_store_rejects_nonuniform_iters();
|
||||
test_credentials_store_parse_missing_file();
|
||||
|
||||
@@ -391,6 +391,22 @@ static void test_daemon_conf_auth_users_validated() {
|
||||
EXPECT_EQ_STR(ok_conf->modules[0].auth_users[0], "alice");
|
||||
EXPECT_EQ_STR(ok_conf->modules[0].auth_users[1], "bob");
|
||||
daemon_conf_free(ok_conf);
|
||||
|
||||
/* C4: an empty or separator-only `auth users` value is a parse error. It
|
||||
* would otherwise leave the module with a zero-length allow-list, silently
|
||||
* disabling the authentication the operator asked for. */
|
||||
const char* empty_auth[] = {
|
||||
"[m]\npath = /x\nauth users = \n",
|
||||
"[m]\npath = /x\nauth users = , ,\n",
|
||||
"[m]\npath = /x\nauth users = \t\n",
|
||||
};
|
||||
for (size_t i = 0; i < sizeof(empty_auth) / sizeof(empty_auth[0]); i++) {
|
||||
EXPECT_EQ_INT(write_conf(empty_auth[i], &path), 0);
|
||||
const DaemonConf* rejected = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NULL(rejected);
|
||||
EXPECT_TRUE(strstr(err, "'auth users' must list at least one user") != NULL);
|
||||
}
|
||||
}
|
||||
|
||||
/* Wave 3 daemon hardening: configurable global/per-module connection caps,
|
||||
@@ -475,12 +491,55 @@ static void test_daemon_conf_limits_and_hosts_parse() {
|
||||
EXPECT_EQ_INT(conf->modules[0].max_connections, 0);
|
||||
daemon_conf_free(conf);
|
||||
|
||||
/* An empty hosts list is not an error (no patterns are added). */
|
||||
EXPECT_EQ_INT(write_conf("hosts allow = \n[m]\npath = /x\n", &path), 0);
|
||||
/* C4: a present hosts key with an empty/separator-only value must not silently
|
||||
* install a zero-length (allow-everyone) list. */
|
||||
const char* empty_hosts[] = {
|
||||
"hosts allow = \n[m]\npath = /x\n",
|
||||
"hosts deny = \n[m]\npath = /x\n",
|
||||
"hosts allow = , ,\n[m]\npath = /x\n",
|
||||
"hosts deny = \t\n[m]\npath = /x\n",
|
||||
};
|
||||
for (size_t i = 0; i < sizeof(empty_hosts) / sizeof(empty_hosts[0]); i++) {
|
||||
EXPECT_EQ_INT(write_conf(empty_hosts[i], &path), 0);
|
||||
const DaemonConf* rejected = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NULL(rejected);
|
||||
EXPECT_TRUE(strstr(err, "must list at least one host pattern") != NULL);
|
||||
}
|
||||
|
||||
const char* empty_module_hosts[] = {
|
||||
"[m]\npath = /x\nhosts allow = \n",
|
||||
"[m]\npath = /x\nhosts deny = ,\n",
|
||||
};
|
||||
for (size_t i = 0; i < sizeof(empty_module_hosts) / sizeof(empty_module_hosts[0]); i++) {
|
||||
EXPECT_EQ_INT(write_conf(empty_module_hosts[i], &path), 0);
|
||||
const DaemonConf* rejected = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NULL(rejected);
|
||||
EXPECT_TRUE(strstr(err, "must list at least one host pattern") != NULL);
|
||||
/* The diagnostic must name the offending module. */
|
||||
EXPECT_TRUE(strstr(err, "module 'm'") != NULL);
|
||||
}
|
||||
|
||||
/* Per-module host lists APPEND across lines like the global ones. (A swapped
|
||||
* store_host_list call passed the module name as `replace`, so each line
|
||||
* silently replaced the previous one and only the last survived.) */
|
||||
EXPECT_EQ_INT(write_conf("[m]\npath = /x\n"
|
||||
"hosts allow = 127.0.0.1\n"
|
||||
"hosts allow = 10.0.0.0/8\n"
|
||||
"hosts deny = 192.168.0.1\n"
|
||||
"hosts deny = 2001:db8::/32\n",
|
||||
&path),
|
||||
0);
|
||||
conf = daemon_conf_load(path, err, sizeof(err));
|
||||
free(path);
|
||||
EXPECT_NOT_NULL(conf);
|
||||
EXPECT_EQ_INT(conf->global.hosts_allow_count, 0);
|
||||
EXPECT_EQ_INT(conf->modules[0].hosts_allow_count, 2);
|
||||
EXPECT_EQ_STR(conf->modules[0].hosts_allow[0], "127.0.0.1");
|
||||
EXPECT_EQ_STR(conf->modules[0].hosts_allow[1], "10.0.0.0/8");
|
||||
EXPECT_EQ_INT(conf->modules[0].hosts_deny_count, 2);
|
||||
EXPECT_EQ_STR(conf->modules[0].hosts_deny[0], "192.168.0.1");
|
||||
EXPECT_EQ_STR(conf->modules[0].hosts_deny[1], "2001:db8::/32");
|
||||
daemon_conf_free(conf);
|
||||
}
|
||||
|
||||
|
||||
@@ -14,6 +14,7 @@
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/sysmacros.h>
|
||||
#include <sys/wait.h>
|
||||
#include <time.h>
|
||||
#include <unistd.h>
|
||||
@@ -994,6 +995,94 @@ static void test_inplace_overwrite_truncates_shorter_payload() {
|
||||
rmdir(root);
|
||||
}
|
||||
|
||||
/* B2: --inplace must refuse an existing non-regular destination entry. A FIFO
|
||||
would block open(O_WRONLY) forever and a device node would be written
|
||||
directly, bypassing the --write-devices/super gate. Forked with an alarm so
|
||||
a regression is a prompt failure instead of a hung suite. */
|
||||
static void test_inplace_refuses_fifo_destination() {
|
||||
const char* root = "test_inplace_fifo_tmp";
|
||||
const char* path = "test_inplace_fifo_tmp/fifo";
|
||||
unlink(path);
|
||||
rmdir(root);
|
||||
EXPECT_EQ_INT(mkdir(root, 0700), 0);
|
||||
EXPECT_EQ_INT(mkfifo(path, 0600), 0);
|
||||
|
||||
pid_t pid = fork();
|
||||
if (pid == 0) {
|
||||
alarm(10);
|
||||
File* f = file_create("fifo");
|
||||
if (!f)
|
||||
_exit(1);
|
||||
const char* content = "payload";
|
||||
f->data->data = malloc(strlen(content));
|
||||
if (!f->data->data)
|
||||
_exit(1);
|
||||
memcpy(f->data->data, content, strlen(content));
|
||||
f->data->size = strlen(content);
|
||||
Config* cfg = config_create();
|
||||
if (!cfg)
|
||||
_exit(1);
|
||||
cfg->inplace = true;
|
||||
bool written = file_save_to_disk(root, f, cfg);
|
||||
file_destroy(f);
|
||||
config_delete(cfg);
|
||||
_exit(written ? 1 : 0); /* must be refused */
|
||||
}
|
||||
int status;
|
||||
waitpid(pid, &status, 0);
|
||||
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
|
||||
struct stat st;
|
||||
EXPECT_EQ_INT(lstat(path, &st), 0);
|
||||
EXPECT_TRUE(S_ISFIFO(st.st_mode)); /* left untouched */
|
||||
unlink(path);
|
||||
rmdir(root);
|
||||
}
|
||||
|
||||
/* B2: an existing char device must not be written by --inplace. mknod needs
|
||||
privilege, so a non-root run skips gracefully. /dev/null's (1:3) rdev makes
|
||||
the negative case harmless if it ever regresses. */
|
||||
static void test_inplace_refuses_device_destination() {
|
||||
const char* root = "test_inplace_dev_tmp";
|
||||
const char* path = "test_inplace_dev_tmp/dev";
|
||||
unlink(path);
|
||||
rmdir(root);
|
||||
EXPECT_EQ_INT(mkdir(root, 0700), 0);
|
||||
if (mknod(path, S_IFCHR | 0600, makedev(1, 3)) != 0) {
|
||||
rmdir(root);
|
||||
return; /* no privilege to create a device node: skip */
|
||||
}
|
||||
|
||||
pid_t pid = fork();
|
||||
if (pid == 0) {
|
||||
alarm(10);
|
||||
File* f = file_create("dev");
|
||||
if (!f)
|
||||
_exit(1);
|
||||
const char* content = "payload";
|
||||
f->data->data = malloc(strlen(content));
|
||||
if (!f->data->data)
|
||||
_exit(1);
|
||||
memcpy(f->data->data, content, strlen(content));
|
||||
f->data->size = strlen(content);
|
||||
Config* cfg = config_create();
|
||||
if (!cfg)
|
||||
_exit(1);
|
||||
cfg->inplace = true;
|
||||
bool written = file_save_to_disk(root, f, cfg);
|
||||
file_destroy(f);
|
||||
config_delete(cfg);
|
||||
_exit(written ? 1 : 0); /* must be refused */
|
||||
}
|
||||
int status;
|
||||
waitpid(pid, &status, 0);
|
||||
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
|
||||
struct stat st;
|
||||
EXPECT_EQ_INT(lstat(path, &st), 0);
|
||||
EXPECT_TRUE(S_ISCHR(st.st_mode)); /* still a device, not replaced */
|
||||
unlink(path);
|
||||
rmdir(root);
|
||||
}
|
||||
|
||||
/* Explicit directory entries (--dirs) create the directory under the receive
|
||||
root through the same save funnel, creating parents as needed, and reject
|
||||
traversal the same way a file path does. */
|
||||
@@ -1606,4 +1695,6 @@ void test_file() {
|
||||
test_inplace_overwrite_clears_special_mode_bits();
|
||||
test_inplace_overwrite_metadata_strips_special_bits();
|
||||
test_inplace_overwrite_truncates_shorter_payload();
|
||||
test_inplace_refuses_fifo_destination();
|
||||
test_inplace_refuses_device_destination();
|
||||
}
|
||||
@@ -1,5 +1,6 @@
|
||||
#include "test_file_list.h"
|
||||
#include "file_list.h"
|
||||
#include "utils.h"
|
||||
#include "test_utils.h"
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
@@ -189,8 +190,35 @@ static void test_deep_paths_are_bounded() {
|
||||
free(entry);
|
||||
}
|
||||
|
||||
/* An over-long list entry must be rejected cleanly instead of being read
|
||||
without a bound (the reader never allocates beyond UTILS_MAX_LINE_LEN). */
|
||||
static void test_oversized_entry_rejected() {
|
||||
const char* path = "test_file_list_oversized.txt";
|
||||
FILE* fp = fopen(path, "wb");
|
||||
EXPECT_NOT_NULL(fp);
|
||||
char chunk[4096];
|
||||
memset(chunk, 'a', sizeof(chunk));
|
||||
size_t total = 0;
|
||||
while (total <= UTILS_MAX_LINE_LEN) {
|
||||
EXPECT_EQ_INT((int)fwrite(chunk, 1, sizeof(chunk), fp), (int)sizeof(chunk));
|
||||
total += sizeof(chunk);
|
||||
}
|
||||
EXPECT_EQ_INT(fputc('\n', fp), '\n');
|
||||
fclose(fp);
|
||||
|
||||
char err[160];
|
||||
FileListSet* set = file_list_load(path, false, err, sizeof(err));
|
||||
if (set) {
|
||||
file_list_destroy(set);
|
||||
EXPECT_FAIL("over-long entry was accepted");
|
||||
}
|
||||
EXPECT_TRUE(strstr(err, "exceeds") != NULL);
|
||||
remove(path);
|
||||
}
|
||||
|
||||
void test_file_list() {
|
||||
test_membership_matches_reference();
|
||||
test_ancestor_and_descendant_queries();
|
||||
test_deep_paths_are_bounded();
|
||||
test_oversized_entry_rejected();
|
||||
}
|
||||
@@ -1,7 +1,9 @@
|
||||
#include "test_glob.h"
|
||||
#include "utils.h"
|
||||
#include "test_utils.h"
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <time.h>
|
||||
|
||||
static void test_glob_exact_match() {
|
||||
EXPECT_TRUE(glob_match("foo", "foo"));
|
||||
@@ -76,6 +78,34 @@ static void test_glob_doublestar_mid() {
|
||||
EXPECT_FALSE(glob_match("a/**/b", "a/x/bad"));
|
||||
}
|
||||
|
||||
/* The old backtracking matcher explored an exponential number of paths for a
|
||||
* pattern with many `*` wildcards against a long run that never matches the
|
||||
* trailing literal. The iterative matcher must stay bounded: 30 `*a` groups
|
||||
* followed by `b` against ten thousand `a`s is a few hundred thousand states,
|
||||
* not 2^30 recursion nodes. */
|
||||
static void test_glob_pathological_is_bounded() {
|
||||
char pattern[128];
|
||||
size_t pos = 0;
|
||||
for (int i = 0; i < 30; i++) {
|
||||
pattern[pos++] = '*';
|
||||
pattern[pos++] = 'a';
|
||||
}
|
||||
pattern[pos++] = 'b';
|
||||
pattern[pos] = '\0';
|
||||
|
||||
char* text = malloc(10001);
|
||||
EXPECT_NOT_NULL(text);
|
||||
memset(text, 'a', 10000);
|
||||
text[10000] = '\0';
|
||||
|
||||
clock_t start = clock();
|
||||
EXPECT_FALSE(glob_match(pattern, text));
|
||||
double elapsed = (double)(clock() - start) / CLOCKS_PER_SEC;
|
||||
EXPECT_TRUE(elapsed < 5.0);
|
||||
|
||||
free(text);
|
||||
}
|
||||
|
||||
void test_glob() {
|
||||
test_glob_exact_match();
|
||||
test_glob_question_mark();
|
||||
@@ -91,4 +121,5 @@ void test_glob() {
|
||||
test_glob_doublestar_prefix();
|
||||
test_glob_doublestar_suffix();
|
||||
test_glob_doublestar_mid();
|
||||
test_glob_pathological_is_bounded();
|
||||
}
|
||||
@@ -0,0 +1,297 @@
|
||||
/* Unit tests for the protocol 2.21.0 STATUS_ERROR_DETAIL frame API:
|
||||
* send_error_detail() / receive_status() mapping / protocol_last_error(). */
|
||||
#include "protocol.h"
|
||||
#include "test_utils.h"
|
||||
#include <string.h>
|
||||
#include <threads.h>
|
||||
#include <time.h>
|
||||
#include <unistd.h>
|
||||
|
||||
/* A detail frame maps back to STATUS_ERROR for the caller and its body is
|
||||
* captured verbatim. */
|
||||
static void test_error_detail_maps_and_captures(void) {
|
||||
int p[2];
|
||||
EXPECT_EQ_INT(pipe(p), 0);
|
||||
io_set_fds(p[0], p[1]);
|
||||
io_set_bwlimit(0);
|
||||
protocol_clear_last_error();
|
||||
|
||||
EXPECT_TRUE(send_error_detail(0, "module is read-only"));
|
||||
Status received = STATUS_OK;
|
||||
EXPECT_TRUE(receive_status(0, &received));
|
||||
EXPECT_EQ_INT((int)received, (int)STATUS_ERROR);
|
||||
EXPECT_EQ_STR(protocol_last_error(), "module is read-only");
|
||||
|
||||
close(p[0]);
|
||||
close(p[1]);
|
||||
}
|
||||
|
||||
/* An over-long message is sliced to the hard cap before it goes on the wire, so
|
||||
* the receiver never retains more than MAX_ERROR_DETAIL_BYTES. */
|
||||
static void test_error_detail_over_long_is_bounded(void) {
|
||||
int p[2];
|
||||
EXPECT_EQ_INT(pipe(p), 0);
|
||||
io_set_fds(p[0], p[1]);
|
||||
io_set_bwlimit(0);
|
||||
|
||||
char big[MAX_ERROR_DETAIL_BYTES + 512];
|
||||
memset(big, 'x', sizeof(big) - 1);
|
||||
big[sizeof(big) - 1] = '\0';
|
||||
EXPECT_TRUE(send_error_detail(0, big));
|
||||
Status received = STATUS_OK;
|
||||
EXPECT_TRUE(receive_status(0, &received));
|
||||
EXPECT_EQ_INT((int)received, (int)STATUS_ERROR);
|
||||
EXPECT_EQ_INT((int)strlen(protocol_last_error()), (int)MAX_ERROR_DETAIL_BYTES);
|
||||
|
||||
close(p[0]);
|
||||
close(p[1]);
|
||||
}
|
||||
|
||||
/* A bare STATUS_ERROR (no detail body) must not leave a stale reason visible. */
|
||||
static void test_bare_error_clears_last_error(void) {
|
||||
int p[2];
|
||||
EXPECT_EQ_INT(pipe(p), 0);
|
||||
io_set_fds(p[0], p[1]);
|
||||
io_set_bwlimit(0);
|
||||
|
||||
EXPECT_TRUE(send_error_detail(0, "stale reason"));
|
||||
Status received = STATUS_OK;
|
||||
EXPECT_TRUE(receive_status(0, &received));
|
||||
EXPECT_EQ_STR(protocol_last_error(), "stale reason");
|
||||
|
||||
EXPECT_TRUE(send_status(0, STATUS_ERROR));
|
||||
EXPECT_TRUE(receive_status(0, &received));
|
||||
EXPECT_EQ_INT((int)received, (int)STATUS_ERROR);
|
||||
EXPECT_EQ_STR(protocol_last_error(), "");
|
||||
|
||||
close(p[0]);
|
||||
close(p[1]);
|
||||
}
|
||||
|
||||
/* A tiny --max-alloc must not prevent the bounded detail body from being
|
||||
* drained: the status still maps to STATUS_ERROR with the full reason, and the
|
||||
* following frame is read intact (no desync). */
|
||||
static void test_error_detail_drains_despite_tiny_max_alloc(void) {
|
||||
int p[2];
|
||||
EXPECT_EQ_INT(pipe(p), 0);
|
||||
ProtocolSession receiver;
|
||||
protocol_session_init(&receiver, p[0], p[1]);
|
||||
protocol_session_set_max_alloc(&receiver, 4);
|
||||
ProtocolSession sender;
|
||||
protocol_session_init(&sender, -1, p[1]);
|
||||
|
||||
EXPECT_TRUE(protocol_send_status(&sender, STATUS_ERROR_DETAIL));
|
||||
EXPECT_TRUE(protocol_send_str(&sender, "reason"));
|
||||
Status status = STATUS_OK;
|
||||
EXPECT_TRUE(protocol_receive_status(&receiver, &status));
|
||||
EXPECT_EQ_INT((int)status, (int)STATUS_ERROR);
|
||||
EXPECT_EQ_STR(protocol_last_error(), "reason");
|
||||
/* The bounded detail reader must never touch the session allocation ceiling. */
|
||||
EXPECT_EQ_INT((int)receiver.max_alloc, 4);
|
||||
|
||||
EXPECT_TRUE(protocol_send_status(&sender, STATUS_NEXT));
|
||||
EXPECT_TRUE(protocol_receive_status(&receiver, &status));
|
||||
EXPECT_EQ_INT((int)status, (int)STATUS_NEXT);
|
||||
|
||||
close(p[0]);
|
||||
close(p[1]);
|
||||
}
|
||||
|
||||
/* An over-cap (but not absurd) declared length is drained through a fixed
|
||||
* scratch buffer so the stream stays in sync, and yields an empty detail. The
|
||||
* session's tiny --max-alloc must remain untouched. */
|
||||
static void test_error_detail_over_cap_is_drained(void) {
|
||||
int p[2];
|
||||
EXPECT_EQ_INT(pipe(p), 0);
|
||||
ProtocolSession receiver;
|
||||
protocol_session_init(&receiver, p[0], p[1]);
|
||||
protocol_session_set_max_alloc(&receiver, 8);
|
||||
ProtocolSession sender;
|
||||
protocol_session_init(&sender, -1, p[1]);
|
||||
|
||||
size_t size = MAX_ERROR_DETAIL_BYTES + 128;
|
||||
EXPECT_TRUE(protocol_send_status(&sender, STATUS_ERROR_DETAIL));
|
||||
EXPECT_TRUE(protocol_send_n_data(&sender, &size, sizeof(size)));
|
||||
char chunk[512];
|
||||
memset(chunk, 'z', sizeof(chunk));
|
||||
size_t written = 0;
|
||||
while (written < size) {
|
||||
size_t n = size - written < sizeof(chunk) ? size - written : sizeof(chunk);
|
||||
EXPECT_TRUE(protocol_send_n_data(&sender, chunk, n));
|
||||
written += n;
|
||||
}
|
||||
EXPECT_TRUE(protocol_send_status(&sender, STATUS_NEXT));
|
||||
|
||||
Status status = STATUS_OK;
|
||||
EXPECT_TRUE(protocol_receive_status(&receiver, &status));
|
||||
EXPECT_EQ_INT((int)status, (int)STATUS_ERROR);
|
||||
EXPECT_EQ_STR(protocol_last_error(), "");
|
||||
EXPECT_EQ_INT((int)receiver.max_alloc, 8);
|
||||
|
||||
/* Stream is still framed: the following status is read intact. */
|
||||
EXPECT_TRUE(protocol_receive_status(&receiver, &status));
|
||||
EXPECT_EQ_INT((int)status, (int)STATUS_NEXT);
|
||||
|
||||
close(p[0]);
|
||||
close(p[1]);
|
||||
}
|
||||
|
||||
/* A declared length beyond even the absolute string bound can never be drained
|
||||
* sensibly, so it is a fatal framing error and the status read fails. */
|
||||
static void test_error_detail_absurd_length_is_fatal(void) {
|
||||
int p[2];
|
||||
EXPECT_EQ_INT(pipe(p), 0);
|
||||
ProtocolSession receiver;
|
||||
protocol_session_init(&receiver, p[0], p[1]);
|
||||
ProtocolSession sender;
|
||||
protocol_session_init(&sender, -1, p[1]);
|
||||
|
||||
size_t size = (size_t)MAX_STRING_SIZE + 1;
|
||||
EXPECT_TRUE(protocol_send_status(&sender, STATUS_ERROR_DETAIL));
|
||||
EXPECT_TRUE(protocol_send_n_data(&sender, &size, sizeof(size)));
|
||||
|
||||
Status status = STATUS_OK;
|
||||
EXPECT_FALSE(protocol_receive_status(&receiver, &status));
|
||||
|
||||
close(p[0]);
|
||||
close(p[1]);
|
||||
}
|
||||
|
||||
/* The detail body must share the caller's deadline: with the session window at
|
||||
* the 60 s default, a withheld body under a 1 s receive_status_timed deadline
|
||||
* must fail in about a second, not fall back to the session timeout. */
|
||||
static void test_error_detail_body_honors_deadline(void) {
|
||||
int p[2];
|
||||
EXPECT_EQ_INT(pipe(p), 0);
|
||||
io_set_fds(p[0], p[1]);
|
||||
io_set_bwlimit(0);
|
||||
|
||||
/* Only the status header, body withheld. */
|
||||
EXPECT_TRUE(send_status(0, STATUS_ERROR_DETAIL));
|
||||
|
||||
struct timespec start, end;
|
||||
clock_gettime(CLOCK_MONOTONIC, &start);
|
||||
Status status = STATUS_OK;
|
||||
EXPECT_FALSE(receive_status_timed(0, &status, 1));
|
||||
clock_gettime(CLOCK_MONOTONIC, &end);
|
||||
long long elapsed_ms =
|
||||
(end.tv_sec - start.tv_sec) * 1000LL + (end.tv_nsec - start.tv_nsec) / 1000000LL;
|
||||
EXPECT_TRUE(elapsed_ms < 10000);
|
||||
|
||||
close(p[0]);
|
||||
close(p[1]);
|
||||
}
|
||||
|
||||
typedef struct {
|
||||
int peer_read_fd;
|
||||
int peer_write_fd;
|
||||
bool replied;
|
||||
} DetailKeepalivePeerArg;
|
||||
|
||||
static int detail_keepalive_peer(void* arg) {
|
||||
DetailKeepalivePeerArg* peer = arg;
|
||||
ProtocolSession session;
|
||||
protocol_session_init(&session, peer->peer_read_fd, peer->peer_write_fd);
|
||||
Status status = STATUS_ERROR;
|
||||
if (protocol_receive_status(&session, &status) && status == STATUS_KEEPALIVE) {
|
||||
/* The busy receiver answers the real status (with its detail) first, then the
|
||||
keepalive reply it owes -- which the client then drains. */
|
||||
peer->replied = protocol_send_status(&session, STATUS_ERROR_DETAIL) &&
|
||||
protocol_send_str(&session, "boom") &&
|
||||
protocol_send_status(&session, STATUS_KEEPALIVE);
|
||||
}
|
||||
return thrd_success;
|
||||
}
|
||||
|
||||
/* Draining the keepalive replies the peer still owes must not erase the terminal
|
||||
* detail that arrived just before them. */
|
||||
static void test_error_detail_survives_keepalive_drain(void) {
|
||||
int to_client[2];
|
||||
int to_peer[2];
|
||||
EXPECT_EQ_INT(pipe(to_client), 0);
|
||||
EXPECT_EQ_INT(pipe(to_peer), 0);
|
||||
|
||||
ProtocolSession session;
|
||||
protocol_session_init(&session, to_client[0], to_peer[1]);
|
||||
|
||||
DetailKeepalivePeerArg peer = {.peer_read_fd = to_peer[0], .peer_write_fd = to_client[1]};
|
||||
thrd_t thread;
|
||||
EXPECT_EQ_INT(thrd_create(&thread, detail_keepalive_peer, &peer), thrd_success);
|
||||
|
||||
Status received = STATUS_OK;
|
||||
EXPECT_TRUE(protocol_receive_status_keepalive(&session, &received, 10, 1, NULL));
|
||||
EXPECT_EQ_INT((int)received, (int)STATUS_ERROR);
|
||||
EXPECT_EQ_STR(protocol_last_error(), "boom");
|
||||
|
||||
int result = 0;
|
||||
EXPECT_EQ_INT(thrd_join(thread, &result), thrd_success);
|
||||
EXPECT_EQ_INT(result, thrd_success);
|
||||
EXPECT_TRUE(peer.replied);
|
||||
|
||||
close(to_client[0]);
|
||||
close(to_client[1]);
|
||||
close(to_peer[0]);
|
||||
close(to_peer[1]);
|
||||
}
|
||||
|
||||
typedef struct {
|
||||
ProtocolSession* receiver;
|
||||
} DetailWorkerArg;
|
||||
|
||||
static int detail_worker(void* arg) {
|
||||
DetailWorkerArg* worker = arg;
|
||||
Status status = STATUS_OK;
|
||||
if (!protocol_receive_status(worker->receiver, &status) || status != STATUS_ERROR)
|
||||
return thrd_error;
|
||||
return strcmp(protocol_last_error(), "worker reason") == 0 ? thrd_success : thrd_error;
|
||||
}
|
||||
|
||||
/* Each thread keeps its own last-error buffer: a detail captured on a worker
|
||||
* must not overwrite the one captured on the main thread. */
|
||||
static void test_last_error_is_thread_local(void) {
|
||||
int main_pipe[2];
|
||||
int worker_pipe[2];
|
||||
EXPECT_EQ_INT(pipe(main_pipe), 0);
|
||||
EXPECT_EQ_INT(pipe(worker_pipe), 0);
|
||||
|
||||
io_set_fds(main_pipe[0], main_pipe[1]);
|
||||
io_set_bwlimit(0);
|
||||
EXPECT_TRUE(send_error_detail(0, "main reason"));
|
||||
Status received = STATUS_OK;
|
||||
EXPECT_TRUE(receive_status(0, &received));
|
||||
EXPECT_EQ_STR(protocol_last_error(), "main reason");
|
||||
|
||||
ProtocolSession receiver;
|
||||
protocol_session_init(&receiver, worker_pipe[0], -1);
|
||||
ProtocolSession sender;
|
||||
protocol_session_init(&sender, -1, worker_pipe[1]);
|
||||
EXPECT_TRUE(protocol_send_status(&sender, STATUS_ERROR_DETAIL));
|
||||
EXPECT_TRUE(protocol_send_str(&sender, "worker reason"));
|
||||
|
||||
DetailWorkerArg arg = {.receiver = &receiver};
|
||||
thrd_t thread;
|
||||
EXPECT_EQ_INT(thrd_create(&thread, detail_worker, &arg), thrd_success);
|
||||
int result = 0;
|
||||
EXPECT_EQ_INT(thrd_join(thread, &result), thrd_success);
|
||||
EXPECT_EQ_INT(result, thrd_success);
|
||||
|
||||
/* The worker's capture must not have disturbed this thread's buffer. */
|
||||
EXPECT_EQ_STR(protocol_last_error(), "main reason");
|
||||
|
||||
close(main_pipe[0]);
|
||||
close(main_pipe[1]);
|
||||
close(worker_pipe[0]);
|
||||
close(worker_pipe[1]);
|
||||
}
|
||||
|
||||
void test_protocol_error(void) {
|
||||
test_error_detail_maps_and_captures();
|
||||
test_error_detail_over_long_is_bounded();
|
||||
test_bare_error_clears_last_error();
|
||||
test_error_detail_drains_despite_tiny_max_alloc();
|
||||
test_error_detail_over_cap_is_drained();
|
||||
test_error_detail_absurd_length_is_fatal();
|
||||
test_error_detail_body_honors_deadline();
|
||||
test_error_detail_survives_keepalive_drain();
|
||||
test_last_error_is_thread_local();
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
#ifndef TEST_PROTOCOL_ERROR_H
|
||||
#define TEST_PROTOCOL_ERROR_H
|
||||
|
||||
void test_protocol_error(void);
|
||||
|
||||
#endif
|
||||
@@ -1,4 +1,5 @@
|
||||
#include "test_server.h"
|
||||
#include "checksum.h"
|
||||
#include "config.h"
|
||||
#include "delta.h"
|
||||
#include "file.h"
|
||||
@@ -6,6 +7,7 @@
|
||||
#include "protocol.h"
|
||||
#include "test_utils.h"
|
||||
#include "utils.h"
|
||||
#include <dirent.h>
|
||||
#include <fcntl.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
@@ -376,6 +378,88 @@ static void test_incremental_check_size_mismatch_full_transfer() {
|
||||
}
|
||||
}
|
||||
|
||||
/* Server-contacting --dry-run: with the wire config's dry_run set, a file that
|
||||
is NOT up to date makes the receiver answer STATUS_DRY_RUN_TRANSFER and
|
||||
return immediately; no data body is read and the destination file is left
|
||||
byte-for-byte unchanged (no temp file, no write, no rename). */
|
||||
static void test_incremental_check_dry_run_reports_transfer_without_writing() {
|
||||
Config* cfg = config_create();
|
||||
EXPECT_NOT_NULL(cfg);
|
||||
cfg->dry_run = true;
|
||||
char* root = make_check_root("dryw");
|
||||
EXPECT_NOT_NULL(root);
|
||||
cfg->receive_root_directory = str_dup(root);
|
||||
write_check_file(root, "file.txt", "0123456789abcdef");
|
||||
|
||||
char path[1024];
|
||||
snprintf(path, sizeof(path), "%s/file.txt", root);
|
||||
struct stat st;
|
||||
EXPECT_EQ_INT(stat(path, &st), 0);
|
||||
|
||||
int p[2];
|
||||
EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0);
|
||||
io_set_fds(p[0], p[1]);
|
||||
io_set_bwlimit(0);
|
||||
|
||||
pid_t pid = fork();
|
||||
if (pid == 0) {
|
||||
alarm(30);
|
||||
close(p[1]);
|
||||
io_set_fds(p[0], p[0]);
|
||||
bool skipped = false;
|
||||
bool would_transfer = false;
|
||||
File* file = receive_incremental_check_ex(p[0], cfg, &skipped, &would_transfer);
|
||||
bool ok = file == NULL && !skipped && would_transfer;
|
||||
file_destroy(file);
|
||||
config_delete(cfg);
|
||||
close(p[0]);
|
||||
_exit(ok ? 0 : 1);
|
||||
} else {
|
||||
close(p[0]);
|
||||
io_set_fds(p[1], p[1]);
|
||||
EXPECT_TRUE(send_str(p[1], "file.txt"));
|
||||
unsigned long long size = (unsigned long long)st.st_size + 1;
|
||||
long long mtime = (long long)st.st_mtime;
|
||||
long long mtime_nsec = 0;
|
||||
#ifdef __linux__
|
||||
mtime_nsec = (long long)st.st_mtim.tv_nsec;
|
||||
#endif
|
||||
EXPECT_TRUE(send_n_data(p[1], &size, sizeof(size)));
|
||||
EXPECT_TRUE(send_n_data(p[1], &mtime, sizeof(mtime)));
|
||||
EXPECT_TRUE(send_n_data(p[1], &mtime_nsec, sizeof(mtime_nsec)));
|
||||
Status s;
|
||||
EXPECT_TRUE(receive_status(p[1], &s));
|
||||
EXPECT_EQ_INT(s, STATUS_DRY_RUN_TRANSFER);
|
||||
|
||||
int status;
|
||||
waitpid(pid, &status, 0);
|
||||
close(p[1]);
|
||||
config_delete(cfg);
|
||||
/* The destination file must be untouched and no temp sibling may appear. */
|
||||
char buf[32] = {0};
|
||||
int fd = open(path, O_RDONLY);
|
||||
EXPECT_TRUE(fd >= 0);
|
||||
ssize_t got = read(fd, buf, sizeof(buf) - 1);
|
||||
EXPECT_EQ_INT((int)got, 16);
|
||||
EXPECT_EQ_STR(buf, "0123456789abcdef");
|
||||
close(fd);
|
||||
DIR* d = opendir(root);
|
||||
EXPECT_NOT_NULL(d);
|
||||
int entries = 0;
|
||||
const struct dirent* e;
|
||||
while ((e = readdir(d)) != NULL) {
|
||||
if (strcmp(e->d_name, ".") != 0 && strcmp(e->d_name, "..") != 0)
|
||||
entries++;
|
||||
}
|
||||
closedir(d);
|
||||
EXPECT_EQ_INT(entries, 1);
|
||||
unlink(path);
|
||||
rmdir(root);
|
||||
free(root);
|
||||
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
|
||||
}
|
||||
}
|
||||
|
||||
/* Issue #256: when a received delta claims a result above the whole-file cap,
|
||||
receive_delta_file must mark the operation failed so the caller aborts with
|
||||
STATUS_ERROR instead of emitting STATUS_NEXT and waiting for a body that
|
||||
@@ -761,6 +845,258 @@ static void test_special_socket_path_log_escaped() {
|
||||
EXPECT_NOT_NULL(strstr(output, "socket not recreated: evil\\#012path"));
|
||||
}
|
||||
|
||||
/* B1: a client-planted FIFO at the destination must not block the receiver's
|
||||
* incremental-check open. With the O_NONBLOCK open plus the post-open S_ISREG
|
||||
* gate the FIFO is simply "no existing regular file", so the receiver proceeds
|
||||
* to a full transfer; without O_NONBLOCK the child blocks in openat() and the
|
||||
* alarm(30) kills it. */
|
||||
static void test_incremental_check_fifo_destination_does_not_hang() {
|
||||
Config* cfg = config_create();
|
||||
EXPECT_NOT_NULL(cfg);
|
||||
char* root = make_check_root("qffo");
|
||||
EXPECT_NOT_NULL(root);
|
||||
cfg->receive_root_directory = str_dup(root);
|
||||
char path[1024];
|
||||
snprintf(path, sizeof(path), "%s/file.txt", root);
|
||||
EXPECT_EQ_INT(mkfifo(path, 0600), 0);
|
||||
|
||||
int p[2];
|
||||
EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0);
|
||||
io_set_fds(p[0], p[1]);
|
||||
io_set_bwlimit(0);
|
||||
|
||||
pid_t pid = fork();
|
||||
if (pid == 0) {
|
||||
alarm(30);
|
||||
close(p[1]);
|
||||
io_set_fds(p[0], p[0]);
|
||||
bool skipped = false;
|
||||
File* file = receive_incremental_check(p[0], cfg, &skipped);
|
||||
bool ok = file != NULL && !skipped;
|
||||
file_destroy(file);
|
||||
config_delete(cfg);
|
||||
close(p[0]);
|
||||
_exit(ok ? 0 : 1);
|
||||
} else {
|
||||
close(p[0]);
|
||||
io_set_fds(p[1], p[1]);
|
||||
EXPECT_TRUE(send_str(p[1], "file.txt"));
|
||||
unsigned long long size = 4;
|
||||
long long mtime = 42;
|
||||
long long mtime_nsec = 0;
|
||||
EXPECT_TRUE(send_n_data(p[1], &size, sizeof(size)));
|
||||
EXPECT_TRUE(send_n_data(p[1], &mtime, sizeof(mtime)));
|
||||
EXPECT_TRUE(send_n_data(p[1], &mtime_nsec, sizeof(mtime_nsec)));
|
||||
Status s;
|
||||
EXPECT_TRUE(receive_status(p[1], &s));
|
||||
EXPECT_EQ_INT(s, STATUS_NEXT);
|
||||
|
||||
Data* body = data_create_reserve(4);
|
||||
EXPECT_NOT_NULL(body);
|
||||
body->data = malloc(4);
|
||||
EXPECT_NOT_NULL(body->data);
|
||||
memcpy(body->data, "data", 4);
|
||||
body->size = 4;
|
||||
EXPECT_TRUE(send_data(p[1], body));
|
||||
data_destroy(body);
|
||||
|
||||
int status;
|
||||
waitpid(pid, &status, 0);
|
||||
close(p[1]);
|
||||
config_delete(cfg);
|
||||
unlink(path);
|
||||
rmdir(root);
|
||||
free(root);
|
||||
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
|
||||
}
|
||||
}
|
||||
|
||||
/* A server-contacting --dry-run with an alternate basis dir must never read or
|
||||
hash the basis file. An exact (size+mtime+content) basis match would
|
||||
otherwise let a client probe the basis bytes against its own supplied digest
|
||||
(a 1-bit content oracle). The dry-run decision is metadata-only, so even a
|
||||
byte-identical basis is reported as would-transfer, not a compare-dest skip. */
|
||||
static void test_incremental_check_dry_run_basis_does_not_read_content() {
|
||||
Config* cfg = config_create();
|
||||
EXPECT_NOT_NULL(cfg);
|
||||
cfg->dry_run = true;
|
||||
char* root = make_check_root("dryb");
|
||||
EXPECT_NOT_NULL(root);
|
||||
cfg->receive_root_directory = str_dup(root);
|
||||
|
||||
char basis_dir[1024];
|
||||
char basis_path[2048];
|
||||
snprintf(basis_dir, sizeof(basis_dir), "%s/basis", root);
|
||||
EXPECT_EQ_INT(mkdir(basis_dir, 0700), 0);
|
||||
const char* content = "basis content that matches\n";
|
||||
write_check_file(basis_dir, "file.txt", content);
|
||||
snprintf(basis_path, sizeof(basis_path), "%s/file.txt", basis_dir);
|
||||
struct stat bst;
|
||||
EXPECT_EQ_INT(stat(basis_path, &bst), 0);
|
||||
EXPECT_EQ_INT(config_basis_append(cfg, BASIS_DEST_COMPARE, "basis"), 0);
|
||||
|
||||
/* The (correct) source digest for the basis bytes: an unfixed dry-run would
|
||||
read+hash the basis and treat this as an exact compare-dest hit. */
|
||||
uint8_t digest[CHECKSUM_MAX_DIGEST_LEN];
|
||||
size_t digest_len = 0;
|
||||
EXPECT_TRUE(checksum_digest((ChecksumAlgo)cfg->checksum_algo, cfg->checksum_seed, content,
|
||||
strlen(content), digest, sizeof(digest), &digest_len));
|
||||
|
||||
int p[2];
|
||||
EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0);
|
||||
io_set_fds(p[0], p[1]);
|
||||
io_set_bwlimit(0);
|
||||
|
||||
pid_t pid = fork();
|
||||
if (pid == 0) {
|
||||
alarm(30);
|
||||
close(p[1]);
|
||||
io_set_fds(p[0], p[0]);
|
||||
bool skipped = false;
|
||||
bool would_transfer = false;
|
||||
File* file = receive_incremental_check_ex(p[0], cfg, &skipped, &would_transfer);
|
||||
bool ok = file == NULL && !skipped && would_transfer;
|
||||
file_destroy(file);
|
||||
config_delete(cfg);
|
||||
close(p[0]);
|
||||
_exit(ok ? 0 : 1);
|
||||
} else {
|
||||
close(p[0]);
|
||||
io_set_fds(p[1], p[1]);
|
||||
EXPECT_TRUE(send_str(p[1], "file.txt"));
|
||||
unsigned long long size = (unsigned long long)bst.st_size;
|
||||
long long mtime = (long long)bst.st_mtime;
|
||||
long long mtime_nsec = 0;
|
||||
#ifdef __linux__
|
||||
mtime_nsec = (long long)bst.st_mtim.tv_nsec;
|
||||
#endif
|
||||
EXPECT_TRUE(send_n_data(p[1], &size, sizeof(size)));
|
||||
EXPECT_TRUE(send_n_data(p[1], &mtime, sizeof(mtime)));
|
||||
EXPECT_TRUE(send_n_data(p[1], &mtime_nsec, sizeof(mtime_nsec)));
|
||||
uint8_t wire_len = (uint8_t)digest_len;
|
||||
EXPECT_TRUE(send_n_data(p[1], &wire_len, sizeof(wire_len)));
|
||||
EXPECT_TRUE(send_n_data(p[1], digest, digest_len));
|
||||
Status s;
|
||||
EXPECT_TRUE(receive_status(p[1], &s));
|
||||
/* A skip here would mean the receiver read+hashed the basis file. */
|
||||
EXPECT_EQ_INT(s, STATUS_DRY_RUN_TRANSFER);
|
||||
|
||||
int status;
|
||||
waitpid(pid, &status, 0);
|
||||
close(p[1]);
|
||||
config_delete(cfg);
|
||||
/* The dry-run must not have materialized anything in the receive root. */
|
||||
char dest_path[2048];
|
||||
snprintf(dest_path, sizeof(dest_path), "%s/file.txt", root);
|
||||
EXPECT_FALSE(file_path_exists_secure(dest_path));
|
||||
unlink(basis_path);
|
||||
rmdir(basis_dir);
|
||||
rmdir(root);
|
||||
free(root);
|
||||
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
|
||||
}
|
||||
}
|
||||
|
||||
/* B1: a FIFO planted in a --link-dest basis directory must not block
|
||||
* basis_open_regular() either; the basis match is simply declined. */
|
||||
static void test_incremental_check_basis_fifo_does_not_hang() {
|
||||
Config* cfg = config_create();
|
||||
EXPECT_NOT_NULL(cfg);
|
||||
char* root = make_check_root("qbfi");
|
||||
EXPECT_NOT_NULL(root);
|
||||
cfg->receive_root_directory = str_dup(root);
|
||||
char basis_dir[1024];
|
||||
char basis_path[2048];
|
||||
snprintf(basis_dir, sizeof(basis_dir), "%s/basis", root);
|
||||
EXPECT_EQ_INT(mkdir(basis_dir, 0700), 0);
|
||||
snprintf(basis_path, sizeof(basis_path), "%s/file.txt", basis_dir);
|
||||
EXPECT_EQ_INT(mkfifo(basis_path, 0600), 0);
|
||||
EXPECT_EQ_INT(config_basis_append(cfg, BASIS_DEST_LINK, "basis"), 0);
|
||||
|
||||
int p[2];
|
||||
EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0);
|
||||
io_set_fds(p[0], p[1]);
|
||||
io_set_bwlimit(0);
|
||||
|
||||
pid_t pid = fork();
|
||||
if (pid == 0) {
|
||||
alarm(30);
|
||||
close(p[1]);
|
||||
io_set_fds(p[0], p[0]);
|
||||
bool skipped = false;
|
||||
File* file = receive_incremental_check(p[0], cfg, &skipped);
|
||||
bool ok = file != NULL && !skipped;
|
||||
file_destroy(file);
|
||||
config_delete(cfg);
|
||||
close(p[0]);
|
||||
_exit(ok ? 0 : 1);
|
||||
} else {
|
||||
close(p[0]);
|
||||
io_set_fds(p[1], p[1]);
|
||||
EXPECT_TRUE(send_str(p[1], "file.txt"));
|
||||
unsigned long long size = 4;
|
||||
long long mtime = 42;
|
||||
long long mtime_nsec = 0;
|
||||
EXPECT_TRUE(send_n_data(p[1], &size, sizeof(size)));
|
||||
EXPECT_TRUE(send_n_data(p[1], &mtime, sizeof(mtime)));
|
||||
EXPECT_TRUE(send_n_data(p[1], &mtime_nsec, sizeof(mtime_nsec)));
|
||||
/* config_has_basis() makes the request carry the source digest. */
|
||||
uint8_t wire_len = 8;
|
||||
uint8_t digest[8] = {0};
|
||||
EXPECT_TRUE(send_n_data(p[1], &wire_len, sizeof(wire_len)));
|
||||
EXPECT_TRUE(send_n_data(p[1], digest, sizeof(digest)));
|
||||
Status s;
|
||||
EXPECT_TRUE(receive_status(p[1], &s));
|
||||
EXPECT_EQ_INT(s, STATUS_NEXT);
|
||||
|
||||
Data* body = data_create_reserve(4);
|
||||
EXPECT_NOT_NULL(body);
|
||||
body->data = malloc(4);
|
||||
EXPECT_NOT_NULL(body->data);
|
||||
memcpy(body->data, "data", 4);
|
||||
body->size = 4;
|
||||
EXPECT_TRUE(send_data(p[1], body));
|
||||
data_destroy(body);
|
||||
|
||||
int status;
|
||||
waitpid(pid, &status, 0);
|
||||
close(p[1]);
|
||||
config_delete(cfg);
|
||||
unlink(basis_path);
|
||||
rmdir(basis_dir);
|
||||
rmdir(root);
|
||||
free(root);
|
||||
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
|
||||
}
|
||||
}
|
||||
|
||||
/* B5: the aggregate entry count across the three manifest sections is capped at
|
||||
* MAX_MANIFEST_ENTRIES, and a section that would push the total over the cap is
|
||||
* rejected before its entries are read (so a tiny first section followed by a
|
||||
* huge claimed second section fails fast). */
|
||||
static void test_receive_manifest_total_entry_cap() {
|
||||
Config* cfg = config_create();
|
||||
EXPECT_NOT_NULL(cfg);
|
||||
cfg->receive_root_directory = str_dup("/tmp/dst");
|
||||
int p[2];
|
||||
EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0);
|
||||
io_set_fds(p[0], p[1]);
|
||||
|
||||
EXPECT_TRUE(send_int(p[1], 1));
|
||||
EXPECT_TRUE(send_str(p[1], "keep.txt"));
|
||||
/* The second section alone is within its per-section cap, but 1 + it exceeds
|
||||
the cross-section cap; the receiver must reject at the count. */
|
||||
EXPECT_TRUE(send_int(p[1], MAX_MANIFEST_ENTRIES));
|
||||
EXPECT_NULL(receive_manifest_entries(p[0]));
|
||||
Status status;
|
||||
EXPECT_TRUE(receive_status(p[1], &status));
|
||||
EXPECT_EQ_INT(status, STATUS_ERROR);
|
||||
|
||||
close(p[0]);
|
||||
close(p[1]);
|
||||
config_delete(cfg);
|
||||
}
|
||||
|
||||
void test_server() {
|
||||
test_special_socket_path_log_escaped();
|
||||
if (!is_running_under_valgrind()) {
|
||||
@@ -771,7 +1107,12 @@ void test_server() {
|
||||
test_receive_incremental_check_rejects_invalid_nanoseconds();
|
||||
test_incremental_check_quick_skip_by_mtime();
|
||||
test_incremental_check_size_mismatch_full_transfer();
|
||||
test_incremental_check_dry_run_reports_transfer_without_writing();
|
||||
test_incremental_check_delta_oversize_reports_failure();
|
||||
test_incremental_check_fifo_destination_does_not_hang();
|
||||
test_incremental_check_basis_fifo_does_not_hang();
|
||||
test_incremental_check_dry_run_basis_does_not_read_content();
|
||||
test_receive_manifest_total_entry_cap();
|
||||
test_late_manifest_abort_frees_keepset();
|
||||
test_late_manifest_eof_frees_keepset();
|
||||
test_late_second_manifest_frees_both();
|
||||
|
||||
@@ -32,6 +32,35 @@ static void test_server_cli_defaults() {
|
||||
EXPECT_FALSE(opts.allow_delete);
|
||||
EXPECT_FALSE(opts.allow_unauthenticated);
|
||||
EXPECT_FALSE(opts.no_super);
|
||||
EXPECT_FALSE(opts.allow_super);
|
||||
server_cli_options_free(&opts);
|
||||
}
|
||||
|
||||
/* C3: --allow-super is the locally-launched standalone TCP opt-in for a
|
||||
* privileged receiver; it never combines with --no-super, is refused with
|
||||
* --stdio (whose client-composed remote argv must not defeat the default), and
|
||||
* daemon modules use their own per-module `client owner = yes` opt-in instead. */
|
||||
static void test_server_cli_allow_super() {
|
||||
const char* args[] = {"fastsync-server", "--allow-super", "--destination-root", "/srv"};
|
||||
ServerCliOptions opts;
|
||||
EXPECT_EQ_INT(parse_ok(args, 4, &opts), 0);
|
||||
EXPECT_TRUE(opts.allow_super);
|
||||
server_cli_options_free(&opts);
|
||||
|
||||
char err[256];
|
||||
const char* a1[] = {"s", "--allow-super", "--no-super"};
|
||||
EXPECT_EQ_INT(server_cli_parse(3, (char**)a1, &opts, err, sizeof(err)), -1);
|
||||
EXPECT_TRUE(strstr(err, "mutually exclusive") != NULL);
|
||||
|
||||
const char* a2[] = {"s", "--daemon", "--config=/tmp/x.conf", "--allow-super"};
|
||||
EXPECT_EQ_INT(server_cli_parse(4, (char**)a2, &opts, err, sizeof(err)), -1);
|
||||
EXPECT_TRUE(strstr(err, "client owner") != NULL);
|
||||
|
||||
/* The SSH/--stdio receiver argv is composed by the client, so --allow-super
|
||||
* must be rejected there and the C3 secure default stays in force. */
|
||||
const char* a3[] = {"s", "--stdio", "--allow-super", "--destination-root", "/srv"};
|
||||
EXPECT_EQ_INT(server_cli_parse(5, (char**)a3, &opts, err, sizeof(err)), -1);
|
||||
EXPECT_TRUE(strstr(err, "--stdio") != NULL);
|
||||
server_cli_options_free(&opts);
|
||||
}
|
||||
|
||||
@@ -224,5 +253,6 @@ void test_server_cli() {
|
||||
test_server_cli_password_and_early_input();
|
||||
test_server_cli_password_requires_daemon();
|
||||
test_server_cli_no_super();
|
||||
test_server_cli_allow_super();
|
||||
test_server_cli_help();
|
||||
}
|
||||
@@ -519,9 +519,38 @@ static void test_path_index_semantics() {
|
||||
path_index_free(&empty);
|
||||
}
|
||||
|
||||
/* utils_getdelim_bounded must return normal short lines unchanged and refuse an
|
||||
* over-long record with EFBIG rather than allocating without bound. */
|
||||
static void test_getdelim_bounded() {
|
||||
FILE* fp = tmpfile();
|
||||
EXPECT_NOT_NULL(fp);
|
||||
const char* short_line = "short\n";
|
||||
EXPECT_EQ_INT((int)fwrite(short_line, 1, strlen(short_line), fp), (int)strlen(short_line));
|
||||
char big[32];
|
||||
memset(big, 'x', 20);
|
||||
big[20] = '\n';
|
||||
EXPECT_EQ_INT((int)fwrite(big, 1, 21, fp), 21);
|
||||
rewind(fp);
|
||||
|
||||
char* line = NULL;
|
||||
size_t cap = 0;
|
||||
ssize_t n = utils_getdelim_bounded(fp, &line, &cap, '\n', 64);
|
||||
EXPECT_EQ_INT((int)n, 6);
|
||||
EXPECT_EQ_STR(line, "short\n");
|
||||
|
||||
errno = 0;
|
||||
n = utils_getdelim_bounded(fp, &line, &cap, '\n', 10);
|
||||
EXPECT_EQ_INT((int)n, -1);
|
||||
EXPECT_EQ_INT(errno, EFBIG);
|
||||
|
||||
free(line);
|
||||
fclose(fp);
|
||||
}
|
||||
|
||||
void test_shared_utils() {
|
||||
test_path_index_bounded();
|
||||
test_path_index_semantics();
|
||||
test_getdelim_bounded();
|
||||
test_walker_removes_extras_keeps_manifest_and_protected();
|
||||
test_walker_keeps_nested_manifest_dirs();
|
||||
test_walker_max_delete_exceeded_deletes_nothing();
|
||||
|
||||
+29
-10
@@ -66,16 +66,18 @@ static void test_ssh_remote_command_argument_modes() {
|
||||
free(command);
|
||||
}
|
||||
|
||||
/* The build for a single-word argv is [prog, six -o args, user, command]. */
|
||||
/* The build for a single-word argv is [prog, six -o args, "--", user, command]. */
|
||||
|
||||
static void test_ssh_build_client_argv_default_is_ssh() {
|
||||
char** argv = ssh_build_client_argv(NULL, 0, "u@h", "'srv' --stdio");
|
||||
EXPECT_NOT_NULL(argv);
|
||||
EXPECT_EQ_STR(argv[0], "ssh");
|
||||
EXPECT_EQ_STR(argv[1], "-o");
|
||||
EXPECT_EQ_STR(argv[7], "u@h");
|
||||
EXPECT_EQ_STR(argv[8], "'srv' --stdio");
|
||||
EXPECT_NULL(argv[9]);
|
||||
/* The "--" end-of-options marker precedes the destination token. */
|
||||
EXPECT_EQ_STR(argv[7], "--");
|
||||
EXPECT_EQ_STR(argv[8], "u@h");
|
||||
EXPECT_EQ_STR(argv[9], "'srv' --stdio");
|
||||
EXPECT_NULL(argv[10]);
|
||||
ssh_free_client_argv(argv);
|
||||
}
|
||||
|
||||
@@ -84,7 +86,7 @@ static void test_ssh_build_client_argv_uses_custom_rsh() {
|
||||
char** argv = ssh_build_client_argv("myrsh", 0, "u@h", "rc");
|
||||
EXPECT_NOT_NULL(argv);
|
||||
EXPECT_EQ_STR(argv[0], "myrsh");
|
||||
EXPECT_NULL(argv[9]);
|
||||
EXPECT_NULL(argv[10]);
|
||||
ssh_free_client_argv(argv);
|
||||
}
|
||||
|
||||
@@ -96,21 +98,37 @@ static void test_ssh_build_client_argv_whitespace_command_and_port() {
|
||||
EXPECT_EQ_STR(argv[0], "ssh");
|
||||
EXPECT_EQ_STR(argv[1], "-p");
|
||||
EXPECT_EQ_STR(argv[2], "2222");
|
||||
EXPECT_NULL(argv[11]);
|
||||
EXPECT_NULL(argv[12]);
|
||||
ssh_free_client_argv(argv);
|
||||
|
||||
argv = ssh_build_client_argv("ssh", 2222, "u@h", "rc");
|
||||
EXPECT_NOT_NULL(argv);
|
||||
EXPECT_EQ_STR(argv[0], "ssh");
|
||||
/* Flat [prog, -o x6, -p, port, user, command]. */
|
||||
/* Flat [prog, -o x6, -p, port, "--", user, command]. */
|
||||
EXPECT_EQ_STR(argv[7], "-p");
|
||||
EXPECT_EQ_STR(argv[8], "2222");
|
||||
EXPECT_EQ_STR(argv[9], "u@h");
|
||||
EXPECT_EQ_STR(argv[10], "rc");
|
||||
EXPECT_NULL(argv[11]);
|
||||
EXPECT_EQ_STR(argv[9], "--");
|
||||
EXPECT_EQ_STR(argv[10], "u@h");
|
||||
EXPECT_EQ_STR(argv[11], "rc");
|
||||
EXPECT_NULL(argv[12]);
|
||||
ssh_free_client_argv(argv);
|
||||
}
|
||||
|
||||
/* C1: a destination host/user beginning with '-' would be parsed by ssh as an
|
||||
* option (argument injection: -oProxyCommand=...), and an empty host is never
|
||||
* valid. These are refused before any child is forked, so no Client is
|
||||
* returned and no command can run. */
|
||||
static void test_ssh_connect_rejects_option_host() {
|
||||
/* cppcheck-suppress constVariablePointer */
|
||||
Client* client = client_connect_ssh("-oProxyCommand=touch /tmp/pwned:/remote", 22, NULL, false,
|
||||
NULL, false, NULL, 0);
|
||||
EXPECT_NULL(client);
|
||||
client = client_connect_ssh("-evil:/remote", 22, NULL, false, NULL, false, NULL, 0);
|
||||
EXPECT_NULL(client);
|
||||
client = client_connect_ssh("user@:/remote", 22, NULL, false, NULL, false, NULL, 0);
|
||||
EXPECT_NULL(client);
|
||||
}
|
||||
|
||||
/* --remote-option=OPT appends OPT to the remote command line after " --stdio",
|
||||
* each escaped as its own single-quoted shell word. Metacharacters that could
|
||||
* break out of the quoting are neutralized (never injected), matching the
|
||||
@@ -162,6 +180,7 @@ void test_transport_ssh() {
|
||||
test_ssh_connect_invalid_dest_empty();
|
||||
test_ssh_connect_malformed();
|
||||
test_ssh_connect_unreachable();
|
||||
test_ssh_connect_rejects_option_host();
|
||||
test_ssh_remote_command_argument_modes();
|
||||
test_ssh_build_client_argv_default_is_ssh();
|
||||
test_ssh_build_client_argv_uses_custom_rsh();
|
||||
|
||||
@@ -24,6 +24,17 @@ static void test_server_create_tls_without_certs() {
|
||||
#ifdef SSL_OP_NO_RENEGOTIATION
|
||||
EXPECT_TRUE((SSL_CTX_get_options(ctx) & SSL_OP_NO_RENEGOTIATION) != 0);
|
||||
#endif
|
||||
/* C5: the server's preference order decides the cipher and the TLS 1.2 list is
|
||||
* AEAD-only (no CBC/RC4/3DES legacy suites). */
|
||||
EXPECT_TRUE((SSL_CTX_get_options(ctx) & SSL_OP_CIPHER_SERVER_PREFERENCE) != 0);
|
||||
STACK_OF(SSL_CIPHER)* ciphers = SSL_CTX_get_ciphers(ctx);
|
||||
EXPECT_NOT_NULL(ciphers);
|
||||
for (int i = 0; i < sk_SSL_CIPHER_num(ciphers); i++) {
|
||||
const char* name = SSL_CIPHER_get_name(sk_SSL_CIPHER_value(ciphers, i));
|
||||
EXPECT_TRUE(name != NULL && strstr(name, "CBC") == NULL);
|
||||
EXPECT_TRUE(name != NULL && strstr(name, "RC4") == NULL);
|
||||
EXPECT_TRUE(name != NULL && strstr(name, "3DES") == NULL);
|
||||
}
|
||||
server_delete(&s);
|
||||
EXPECT_NULL(s);
|
||||
}
|
||||
|
||||
+135
-10
@@ -17,7 +17,7 @@
|
||||
|
||||
static void run_recv_helper(int fd) {
|
||||
int ok = 0;
|
||||
FileXattrList* list = xattr_receive(fd, &ok);
|
||||
FileXattrList* list = xattr_receive(fd, &ok, false);
|
||||
if (!ok)
|
||||
_exit(1);
|
||||
if (!list) {
|
||||
@@ -64,7 +64,7 @@ static void test_xattr_wire_roundtrip() {
|
||||
|
||||
static void run_recv_must_fail(int fd) {
|
||||
int ok = 0;
|
||||
FileXattrList* list = xattr_receive(fd, &ok);
|
||||
FileXattrList* list = xattr_receive(fd, &ok, false);
|
||||
/* A NULL list with ok==0 is the expected rejection. */
|
||||
if (ok == 0 && list == NULL)
|
||||
_exit(0);
|
||||
@@ -148,15 +148,64 @@ static void test_xattr_count_bound() {
|
||||
/* The captured list on a plain file reflects only whitelisted namespaces
|
||||
* (Linux only; skipped when the filesystem has no xattr support). */
|
||||
static void test_xattr_capture_and_appliable() {
|
||||
EXPECT_FALSE(xattr_name_appliable(NULL));
|
||||
EXPECT_FALSE(xattr_name_appliable(""));
|
||||
EXPECT_FALSE(xattr_name_appliable("security.selinux"));
|
||||
EXPECT_FALSE(xattr_name_appliable("trusted.blob"));
|
||||
EXPECT_TRUE(xattr_name_appliable("user.foo"));
|
||||
EXPECT_FALSE(xattr_name_appliable(NULL, false));
|
||||
EXPECT_FALSE(xattr_name_appliable("", false));
|
||||
EXPECT_FALSE(xattr_name_appliable("security.selinux", false));
|
||||
EXPECT_FALSE(xattr_name_appliable("trusted.blob", false));
|
||||
EXPECT_TRUE(xattr_name_appliable("user.foo", false));
|
||||
EXPECT_TRUE(xattr_name_appliable("user.foo", true));
|
||||
/* The reserved fake-super key is receiver-only and never forwarded/applied. */
|
||||
EXPECT_FALSE(xattr_name_appliable("user.fastsync.stat"));
|
||||
EXPECT_TRUE(xattr_name_appliable("system.posix_acl_access"));
|
||||
EXPECT_TRUE(xattr_name_appliable("system.posix_acl_default"));
|
||||
EXPECT_FALSE(xattr_name_appliable("user.fastsync.stat", false));
|
||||
EXPECT_FALSE(xattr_name_appliable("user.fastsync.stat", true));
|
||||
/* B4: the ACL names require --acls; -X alone must not authorize them. */
|
||||
EXPECT_FALSE(xattr_name_appliable("system.posix_acl_access", false));
|
||||
EXPECT_FALSE(xattr_name_appliable("system.posix_acl_default", false));
|
||||
EXPECT_TRUE(xattr_name_appliable("system.posix_acl_access", true));
|
||||
EXPECT_TRUE(xattr_name_appliable("system.posix_acl_default", true));
|
||||
}
|
||||
|
||||
/* B4: a `-X`-only receiver (preserve_acls false) must NOT apply an incoming
|
||||
* ACL xattr, while a user.* attribute in the same block still survives. The
|
||||
* ACL entry is dropped, not applied (and the -X transfer is not failed). */
|
||||
static void run_recv_drops_acl_keeps_user(int fd) {
|
||||
int ok = 0;
|
||||
FileXattrList* list = xattr_receive(fd, &ok, false);
|
||||
if (!ok || list == NULL)
|
||||
_exit(1);
|
||||
bool saw_user = false;
|
||||
for (int i = 0; i < list->count; i++) {
|
||||
if (strcmp(list->items[i].name, "system.posix_acl_access") == 0)
|
||||
_exit(1); /* ACL must have been dropped */
|
||||
if (strcmp(list->items[i].name, "user.keep") == 0)
|
||||
saw_user = true;
|
||||
}
|
||||
xattr_list_free(list);
|
||||
_exit(saw_user ? 0 : 1);
|
||||
}
|
||||
|
||||
static void test_xattr_receive_drops_acl_without_preserve_acls() {
|
||||
int p[2];
|
||||
EXPECT_EQ_INT(pipe(p), 0);
|
||||
io_set_fds(p[0], p[1]);
|
||||
io_set_bwlimit(0);
|
||||
pid_t pid = fork();
|
||||
if (pid == 0) {
|
||||
close(p[1]);
|
||||
io_set_fds(p[0], p[0]);
|
||||
run_recv_drops_acl_keeps_user(p[0]);
|
||||
}
|
||||
close(p[0]);
|
||||
io_set_fds(p[1], p[1]);
|
||||
FileXattrList* list = xattr_list_new();
|
||||
EXPECT_NOT_NULL(list);
|
||||
EXPECT_TRUE(xattr_list_append(list, "system.posix_acl_access", "\x02\x00\x00\x00", 4));
|
||||
EXPECT_TRUE(xattr_list_append(list, "user.keep", "yes", 3));
|
||||
xattr_send(p[1], list);
|
||||
xattr_list_free(list);
|
||||
int status;
|
||||
waitpid(pid, &status, 0);
|
||||
close(p[1]);
|
||||
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
|
||||
}
|
||||
|
||||
/* MINOR-2: a --link-dest / -H copy fallback (linkat refused) must still apply
|
||||
@@ -224,6 +273,80 @@ static void test_link_copy_fallback_preserves_xattrs() {
|
||||
rmdir(basis_dir);
|
||||
}
|
||||
|
||||
/* Capture must honor --acls: xattr_capture_path(path, false) (plain -X) must
|
||||
* never return the POSIX ACL names, while xattr_capture_path(path, true) (-A)
|
||||
* does; user.* is captured either way. This is the capture-side counterpart of
|
||||
* the receiver's --acls gate and must not depend on the caller having checked
|
||||
* the flag. Guarded on filesystem/ACL support. */
|
||||
static void test_xattr_capture_filters_acls() {
|
||||
const char* path = "test_xattr_capture_acls.txt";
|
||||
unlink(path);
|
||||
int fd = open(path, O_WRONLY | O_CREAT | O_TRUNC, 0600);
|
||||
if (fd < 0)
|
||||
return;
|
||||
bool has_xattr = setxattr(path, "user.fastsync.xprobe", "p", 1, 0) == 0;
|
||||
if (has_xattr)
|
||||
removexattr(path, "user.fastsync.xprobe");
|
||||
if (!has_xattr) {
|
||||
close(fd);
|
||||
unlink(path);
|
||||
return; /* filesystem without xattr support */
|
||||
}
|
||||
if (setxattr(path, "user.keep", "yes", 3, 0) != 0) {
|
||||
close(fd);
|
||||
unlink(path);
|
||||
return;
|
||||
}
|
||||
|
||||
/* Synthesize a valid non-trivial POSIX access ACL blob (little-endian):
|
||||
version 2 followed by USER_OBJ/USER/GROUP_OBJ/MASK/OTHER entries. */
|
||||
uint32_t acl_uid = geteuid() == 0 ? 65534u : (uint32_t)geteuid();
|
||||
unsigned char blob[4 + 5 * 8];
|
||||
uint32_t version = 2;
|
||||
memcpy(blob, &version, 4);
|
||||
const uint16_t tags[5] = {0x01, 0x02, 0x04, 0x10, 0x20}; /* OBJ/USER/GROUP/MASK/OTHER */
|
||||
const uint16_t perms[5] = {0x04, 0x04, 0x04, 0x04, 0x00};
|
||||
const uint32_t ids[5] = {0xFFFFFFFFu, acl_uid, 0xFFFFFFFFu, 0xFFFFFFFFu, 0xFFFFFFFFu};
|
||||
size_t off = 4;
|
||||
for (int i = 0; i < 5; i++) {
|
||||
memcpy(blob + off, &tags[i], sizeof(tags[i]));
|
||||
off += sizeof(tags[i]);
|
||||
memcpy(blob + off, &perms[i], sizeof(perms[i]));
|
||||
off += sizeof(perms[i]);
|
||||
memcpy(blob + off, &ids[i], sizeof(ids[i]));
|
||||
off += sizeof(ids[i]);
|
||||
}
|
||||
if (setxattr(path, "system.posix_acl_access", blob, off, 0) != 0) {
|
||||
close(fd);
|
||||
unlink(path);
|
||||
return; /* no unprivileged ACL support: skip silently */
|
||||
}
|
||||
close(fd);
|
||||
|
||||
FileXattrList* plain = xattr_capture_path(path, false);
|
||||
FileXattrList* with_acls = xattr_capture_path(path, true);
|
||||
bool plain_user = false, plain_acl = false, acl_user = false, acl_acl = false;
|
||||
for (int i = 0; plain && i < plain->count; i++) {
|
||||
if (strcmp(plain->items[i].name, "user.keep") == 0)
|
||||
plain_user = true;
|
||||
if (strcmp(plain->items[i].name, "system.posix_acl_access") == 0)
|
||||
plain_acl = true;
|
||||
}
|
||||
for (int i = 0; with_acls && i < with_acls->count; i++) {
|
||||
if (strcmp(with_acls->items[i].name, "user.keep") == 0)
|
||||
acl_user = true;
|
||||
if (strcmp(with_acls->items[i].name, "system.posix_acl_access") == 0)
|
||||
acl_acl = true;
|
||||
}
|
||||
EXPECT_TRUE(plain_user);
|
||||
EXPECT_FALSE(plain_acl);
|
||||
EXPECT_TRUE(acl_user);
|
||||
EXPECT_TRUE(acl_acl);
|
||||
xattr_list_free(plain);
|
||||
xattr_list_free(with_acls);
|
||||
unlink(path);
|
||||
}
|
||||
|
||||
/* --fake-super replay: fake_super_store_fd records the source stat into the
|
||||
* reserved xattr, and fake_super_restore_fd re-applies mode/mtime (and owner,
|
||||
* when the process may) fd-relative. Restore must also be a safe no-op with no
|
||||
@@ -360,6 +483,8 @@ void test_xattr() {
|
||||
test_xattr_reject_oversized_value();
|
||||
test_xattr_count_bound();
|
||||
test_xattr_capture_and_appliable();
|
||||
test_xattr_capture_filters_acls();
|
||||
test_xattr_receive_drops_acl_without_preserve_acls();
|
||||
test_link_copy_fallback_preserves_xattrs();
|
||||
test_fake_super_restore();
|
||||
test_fake_super_owner_gate();
|
||||
|
||||
Reference in new issue
Block a user