11 Commits
Author SHA1 Message Date
TapTap c062a0762e Merge branch 'fix/audit-docs3' into fix/audit-cycle
CI / lint (pull_request) Successful in 2m46s
CI / parity-full (pull_request) Skipped
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
CI / parity-fast (pull_request) Successful in 31s
CI / build-and-test (pull_request) Successful in 54s
2026-09-21 22:11:53 +02:00
TapTap 283f9f0823 docs: reflect filter modifiers, inplace+partial-dir, credentials hardening
Update the docs for the audit follow-up fixes:
- --filter merge modifiers e/n/w/- are now accepted-and-consumed on
  merge/dir-merge rules (rejected on non-merge, x rejected everywhere);
  their semantics stay unimplemented, so the --filter row moves to Caveat
  and the tally becomes 119/11/27 = 157.
- --inplace + --partial-dir is rejected with rsync's message.
- secret_file_open() O_NOFOLLOW (symlinked credential paths fail closed;
  fd-backed paths exempt) and ~3 s bound-wait on FIFO reads.
- AGENTS setpriv wording corrected to the collected instance count.
- CHANGELOG [Unreleased] audit section extended with the follow-ups.
2026-09-21 22:11:19 +02:00
TapTap 5754b9a952 Merge branch 'fix/audit-misc2' into fix/audit-cycle 2026-09-21 22:02:05 +02:00
TapTap b8a0efef7b Merge branch 'fix/audit-filter2' into fix/audit-cycle 2026-09-21 22:02:05 +02:00
TapTap 2e77c09447 Merge branch 'fix/audit-creds2' into fix/audit-cycle 2026-09-21 22:02:05 +02:00
TapTap 06c4026b74 fix(filter): accept e/n/w/- merge modifiers on merge/dir-merge rules
The earlier modifier-rejection change rejected e/n/w on all rules, but rsync
3.4.1 accepts them (plus the '-' merge-only modifier) on merge and dir-merge
rules.  Restrict the rejection to non-merge rules and consume the merge-file
modifiers (e/n/w/-) so they no longer leak into the merge filename.

- is_merge_rule()/is_merge_modifier_char() gate the merge-only modifiers.
- scan vs consume sets: e/n/w still count as modifier-run chars on every rule
  (pure tokens like -new/-press stay rejected), but are only consumed on merge
  rules, preserving mixed-token parsing such as H,!secret -> ecret.
- '-' is accepted/consumed only on merge/dir-merge (e.g. dir-merge,- .rules).
- x remains rejected everywhere with its dedicated message.
- e/n/w/- semantics remain unimplemented and are documented as accepted-but-
  ignored in filter.h.

Tests: split the merge forms out of the rejection test into a new acceptance
test asserting the merge file is read and dir_merge_names keeps the modifier-
free basename; non-merge pure-modifier forms still rejected.
2026-09-21 22:01:44 +02:00
TapTap 9f47b13712 fix(credentials): bound-wait on FIFO reads so slow process substitution works
secret_file_open() opened secret files with O_NONBLOCK and only cleared it
for S_ISREG, so on a FIFO/process-substitution source (--password-file
<(...), --early-input <(...)) fgets() failed immediately with EAGAIN when
the writer had not yet produced data, breaking slow producers.

Keep O_NONBLOCK at open() (a writer-less FIFO must not block the open) and
route all three readers through a new secret_read_line() helper.  It
accumulates a line across reads and, on EAGAIN/EWOULDBLOCK (or a partial
line) with no newline and no EOF, clearerr()s and polls for readability
against one overall CLOCK_MONOTONIC deadline of
CREDENTIAL_FIFO_READ_TIMEOUT_MS (3000 ms); on timeout or a real read error
it fails with a clear message.  EOF finishes normally.  Regular files are
left blocking and read exactly as before.

Handles a line split across several write()s and keeps the owner/mode
fstat gate, O_NOFOLLOW and the /dev/fd/N exception unchanged.
2026-09-21 22:01:18 +02:00
TapTap b1eddf0133 fix: config leak, compression log, inplace+partial-dir rejection, umask/root test fixes 2026-09-21 21:59:58 +02:00
TapTap 338c27db73 fix: resolve cppcheck shadow/always-true findings 2026-09-21 21:28:36 +02:00
TapTap 8d46a26c04 Merge branch 'fix/audit-docs2' into fix/audit-cycle 2026-09-21 21:18:48 +02:00
TapTap cee9b7647c docs: fix parity tally, compat rows, changelog, handoff for audit cycle 2026-09-21 21:14:37 +02:00
17 changed files with 738 additions and 131 deletions
+1 -1
View File
@@ -43,7 +43,7 @@ cmake -B build -S . -DSANITIZER=undefined # UndefinedBehaviorSanitizer (
cmake -B build -S . -DSANITIZER=thread # ThreadSanitizer (TSan); local-only, NOT in CI
```
The CI workflow (`.gitea/workflows/ci.yaml`) runs lint (clang-format, cppcheck), then a **fast PR gate** — build + unit + a representative subset of integration tests marked `@pytest.mark.ci`, parallelized with pytest-xdist (`-n 4 --dist=load`). The full coverage jobs (full integration suite as `-m "not setpriv"`, the `address`+`undefined` sanitizer matrix, fuzz, coverage, valgrind) run **only on push to `dev`/`main`**; pull requests skip them to keep PR CI under ~3 minutes. TSan is not part of the CI matrix and is a local-only configuration. The four `setpriv` privilege tests are excluded from CI via a marker because their result depends on the runner/container uid and host mount permissions.
The CI workflow (`.gitea/workflows/ci.yaml`) runs lint (clang-format, cppcheck), then a **fast PR gate** — build + unit + a representative subset of integration tests marked `@pytest.mark.ci`, parallelized with pytest-xdist (`-n 4 --dist=load`). The full coverage jobs (full integration suite as `-m "not setpriv"`, the `address`+`undefined` sanitizer matrix, fuzz, coverage, valgrind) run **only on push to `dev`/`main`**; pull requests skip them to keep PR CI under ~3 minutes. TSan is not part of the CI matrix and is a local-only configuration. The `setpriv`-marked privilege tests (four decorated functions, collecting to eight instances because two are parametrized) are excluded from CI via a marker because their result depends on the runner/container uid and host mount permissions.
## Build
+88
View File
@@ -10,6 +10,19 @@ The rsync-parity cycle 2.29 (no wire change; `PROTOCOL_VERSION` stays 2.28.0).
`RSYNC_COMPAT.md` moves from **116 ✅ / 14 ⚠️ / 27 ❌** to
**120 ✅ / 10 ⚠️ / 27 ❌** of 157 rows.
An audit cycle follows on the same wire version (`PROTOCOL_VERSION` stays
2.28.0): a security-and-correctness pass over the parity-2.29 baseline, plus a
set of audit follow-ups (filter merge modifiers, the `--inplace`/`--partial-dir`
conflict, credential-file hardening, and small leak/log/test fixes). It fixes
a `--temp-dir` symlink escape, gates client-controlled special permission bits,
corrects `--partial-dir`/`--bwlimit`/`-z` behavior, handles unsupported filter
modifiers, and tightens client and wire validation. The only parity
reclassification is `--filter=RULE` moving ✅ → ⚠️, because its merge-only
`e`/`n`/`w`/`-` modifiers are now accepted and consumed but their semantics
remain unimplemented (accepted-but-ignored); the matrix is therefore **119 ✅ /
11 ⚠️ / 27 ❌** of 157 rows. The affected rows' notes and the summary tally in
`RSYNC_COMPAT.md` were updated.
### Changed
- **rsync-exact traversal order.** The sequential scanner now walks each
@@ -47,6 +60,81 @@ The rsync-parity cycle 2.29 (no wire change; `PROTOCOL_VERSION` stays 2.28.0).
(a general whole-file limit, not basis-specific).
- `--stats` byte totals and `--msgs2stderr` stay documented divergences.
### Security
- **`--temp-dir` symlink escape fixed.** The receiver's scratch directory was
opened with a bare `open()`, so a symlink planted under the receive root could
redirect receiver scratch files outside the authorized root. The opened
directory is now judged by the real path of its fd (`/proc/self/fd` via
`realpath`) and an escaping target is refused (`EACCES`, logged); an in-root
link to another filesystem (the `EXDEV` fallback case) still works.
- **Client-controlled special bits masked when super-user activities are not
permitted.** Setuid/setgid/sticky bits (`--perms`, `--chmod`, the symlink and
special-node paths, and deferred directory modes) are now stripped when the
connection forbids super activities (`--no-super`, a non-opted daemon module,
a privileged listener without `--allow-super`); exact rsync semantics are
preserved wherever super activities are permitted.
- **Daemon umask no longer forced to `0`.** `daemonize()` now sets the
conventional `022`, so implied parent directories created without `-p` are no
longer world-writable `0777`.
- **Credentials and signal handling hardened.** Secret files are opened with
`O_NOFOLLOW|O_NONBLOCK` (while allowing fd-backed store paths and bound-waiting
a FIFO read for ~3 s so a slow process substitution works but a connected-but-
silent FIFO cannot hang), and signal handlers use `sigaction` with
async-signal-safe bodies.
### Fixed
- **`-z` on 100–256 MiB files.** The decompressor's internal ceiling was 100 MiB
while the receiver advertises and the sender compresses whole files up to
`MAX_RECEIVE_WHOLE_FILE_SIZE` (256 MiB), so `-z` on a 100–256 MiB regular file
failed with `Declared decompressed size exceeds 104857600 bytes`. The ceiling
is now defined in terms of the protocol whole-file bound (still an
allocation-clamped bomb guard).
- **`--bwlimit` now paces `--sendfile`.** The plaintext-TCP `--sendfile` fast
path bypassed the protocol's token bucket, so the limit was ignored there. It
now throttles through the same per-session leaky bucket as the TLS path.
- **`--partial-dir` implies `--partial`.** Matching rsync 3.4.1 (which sets
`keep_partial` after option parsing), `--partial-dir=DIR` alone retains an
interrupted transfer's partial and wins over an explicit `--no-partial`;
`--inplace` still bypasses the partial machinery, and combining `--inplace`
with `--partial-dir` is now rejected up front with rsync's message
(`--inplace cannot be used with --partial-dir`).
- **Filter modifiers handled.** The `x` xattr-name modifier is rejected with a
clear error everywhere. The merge-only `e`/`n`/`w` and `-` modifiers are now
accepted and consumed on `merge`/`dir-merge` rules (so they no longer leak
into the merge filename) while still being rejected on non-merge rules,
matching rsync; their semantics remain unimplemented (accepted-but-ignored).
Glued patterns (`-newfile`, `-e2e`) and mixed tokens (`H,!secret`) keep their
historical parsing.
- **Credential-file reads hardened.** Secret files (`--password-file`/
`--early-input`/`--hash-credentials` input) are opened with `O_NOFOLLOW`, so a
symlinked credential path now fails closed (`ELOOP`) instead of being followed
before the owner/mode gate; literal fd-backed paths (`/dev/fd/<digits>`,
`/proc/self/fd/<digits>`) are exempt so process substitution still works. A
FIFO/process-substitution read now waits under a bounded ~3 s deadline for its
writer, so a slow producer works while a connected-but-silent FIFO fails
instead of hanging.
- **Miscellaneous correctness fixes:** `--filter` rule count is checked
client-side against `MAX_FILTER_RULES` before any network I/O (the receiver
still re-checks the expanded count); unknown wire `Status` values are rejected
as protocol errors; a mutex leak on an init-failure path, an `errno` read
after `free()` in deferred delete application, `log_perror` misuse for
non-`errno` conditions, and a `NULL` `server_host`/`ssh_destination`
allocation path were fixed (the `config_create` failure now releases through
`config_delete`); the decompression-limit log now prints the effective bound
rather than the compile-time ceiling; the daemon umask and root test fixtures
were hardened; `SSL_read` length is clamped and `sendfile` `poll()` retries on
`EINTR`.
### Refactored / Docs
- Dropped dead `filter_rules_apply` and dead `--old-args` plumbing, unified
`set_error`, deduplicated `path_is_within` and shared constants, and added
printf format attributes (fixing format mismatches). `RSYNC_COMPAT.md`,
`CHANGELOG.md` and `HANDOFF.md` were updated for the audit cycle; the
`RSYNC_COMPAT.md` summary tally was corrected to match the rows.
## [2.28.0] - 2026-09-20
The rsync-parity cycle. `PROTOCOL_VERSION` moves `2.26.0 → 2.27.0 → 2.28.0`;
+61 -26
View File
@@ -1,22 +1,30 @@
# FastSync — Session Handoff (2026-09-20)
# FastSync — Session Handoff (2026-09-21)
## Current status
- **Release `v2.28.0`** is tagged and merged to `main` (PR #304, `b4d54504`).
`dev` is at `558782d` (the incremental-check flake fix).
- **Release `v2.28.0`** is tagged and merged to `main`: tag `v2.28.0` points at
`ee6523a`, and the PR #304 merge commit `b4d54504` is on `main`.
- **`dev` is at `0fbb9de`** — the merge of parity cycle 2.29 (PR #305). The old
`558782d` (incremental-check flake fix) is an ancestor.
- **`PROTOCOL_VERSION` = `"2.28.0"`** (`src/shared/config.h`); CMake
`project(FastFileTransfer VERSION 2.28.0)`.
- **Parity cycle 2.29 on branch `feat/parity-2.29`** (from `dev` @ `558782d`),
no wire change. It closes the scanner-order, delete-timing, relative-basis and
fuzzy-eligibility residuals and improves the `--info`/`--stats`/`--debug`
partials. Parity matrix: **120 ✅ / 10 ⚠️ / 27 ❌ = 157** (was 116/14/27).
Remaining ⚠️ rows: `--info`, `--debug`, `--msgs2stderr`, `--stats`,
`--progress`, `--delete-before`, `--compare-dest`/`--copy-dest`/`--link-dest`
(over-256-MiB basis MISS), `-y`/`--fuzzy` (256 MiB buffer cap).
- **Deferred (needs a wire bump):** the `--progress`/`--info` receiver→sender
event channel (root `./` line, ancestor suppression, `skip`/`backup` echo,
symlink/empty-dir quick-check); `--delete-before` phase-0 keep-set; and the
general >256 MiB single-file streaming limit (B4).
- Feature branch `feat/parity-2.29`; integration PR to `dev` pending.
- **Parity cycle 2.29 is merged to `dev`** (PR #305), no wire change. It closed
the scanner-order, delete-timing, relative-basis and fuzzy-eligibility
residuals and improved the `--info`/`--stats`/`--debug` partials. Parity
matrix: **120 ✅ / 10 ⚠️ / 27 ❌ = 157**. Remaining ⚠️ rows: `--info`,
`--debug`, `--msgs2stderr`, `--stats`, `--progress`, `--delete-before`, the
three basis-dir options, and `-y`/`--fuzzy`.
- **Audit cycle complete on branch `fix/audit-cycle`** (branched from `dev` @
`0fbb9de`), integration PR to `dev` pending. No wire change
(`PROTOCOL_VERSION` stays 2.28.0). It lands the receiver/client security and
correctness fixes — `--temp-dir` symlink-escape confinement, special-bit
masking under a super-off policy, daemon `umask(022)`, the `-z` decompression
ceiling raised to the 256 MiB whole-file bound, `--bwlimit` pacing the
plaintext `--sendfile` path, `--partial-dir` implying `--partial`, rejection
of unsupported filter modifiers (`x`/`e`/`n`/`w`), client-side
`MAX_FILTER_RULES` enforcement, unknown wire `Status` rejection, and the
accompanying refactors/docs. The parity matrix is unchanged at
**120 ✅ / 10 ⚠️ / 27 ❌ = 157**; this docs pass (worktree `fix/audit-docs2`)
corrects the `RSYNC_COMPAT.md` summary tally to match the rows.
## What landed this session
@@ -102,7 +110,8 @@
uptodate` plus the leading `./` root name line for `--info=name` (only the
root-line trigger condition and receiver-side `skip` wording remain). Matrix
now **111 ✅ / 14 ⚠️ / 32 ❌ = 157**; differential + unit tests added in
`test_features.py`, `test_option_parity.py`, `test_delete_plan.c`,
`test_features.py`, `test_option_parity.py`, the unit test
`tests/test_delete_plan.c`,
`test_delete_delay_budget_parity.py`, `test_delete_timing_parity.py`.
12. **No-wire parity track 2b** on `feat/parity-2.28` (no protocol change):
`--progress`/`-P`/`--info=progress` (when not `--quiet`) now run an opt-in
@@ -199,17 +208,43 @@
**116 ✅ / 14 ⚠️ / 27 ❌ = 157** (the `--delete`/`--delete-during` rows stay
⚠️ for the abort boundary; `--delete-after` stays ✅).
17. **Audit cycle** on `fix/audit-cycle` (from `dev` @ `0fbb9de`;
`PROTOCOL_VERSION` stays `2.28.0`): a security/correctness pass over the
parity-2.29 baseline. It raises the decompression ceiling to the 256 MiB
protocol whole-file bound (`-z` on 100–256 MiB files now works), paces the
plaintext-TCP `--sendfile` path with `--bwlimit`, confines the `--temp-dir`
scratch dir by the fd's real path (symlink escape refused), masks
client-controlled setuid/setgid/sticky bits when super activities are not
permitted, sets the daemon umask to `022`, makes `--partial-dir` imply
`--partial`, rejects the unsupported filter modifiers (`x`/`e`/`n`/`w`),
enforces `MAX_FILTER_RULES` client-side, rejects unknown wire `Status`
values, and hardens credentials/signal handling (with the accompanying
refactors and docs). No row changes classification, so the matrix stays
**120 ✅ / 10 ⚠️ / 27 ❌ = 157**. This docs pass is on `fix/audit-docs2`.
## Next steps
1. **Merge PR #284** (`dev` -> `main`) once reviewed (protected branch).
2. **Deferred security items** (documented, not implemented):
- Pre-auth config/daemon-auth handshake has no aggregate wall-clock deadline
(per-message timeout only) — slowloris holds connection slots.
- Per-source registry fails open when the shared table is full (per-module/global
caps and host ACLs still apply); consider fail-closed or larger/evicting table.
- SCRAM-like daemon auth has no TLS channel binding (and is not RFC 5802).
- `cleanup()` signal handler calls non-async-signal-safe teardown; daemon `umask(0)`.
- Wire protocol assumes homogeneous word size/endianness (lengths are native
`size_t`) — document or move to fixed-width framing.
1. **Open and merge the audit-cycle PR** (`fix/audit-cycle`, including this
`fix/audit-docs2` docs pass) into `dev` once reviewed. `dev` is the default
branch; all PRs target `dev`, never `main` directly.
2. **Remaining deferred items:**
- **Large structural refactors:** delete-engine consolidation
(`delete_extras_fd`/`manifest_delete_extras`/the delete-plan path),
god-function splits, and translation-unit splits.
- **`--progress`/`--info` receiver→sender event channel:** the root `./`
line, ancestor-directory suppression, receiver-side `skip`/`backup` echo,
and symlink/empty-dir quick-check feedback.
- **`--delete-before` phase-0 keep-set** (rsync fixes the file list before
the data pass; FastSync keeps its pre-scan snapshot race).
- **>256 MiB single-file streaming** (B4, the general whole-file limit).
- **Wire native-size framing:** lengths are native `size_t` and the protocol
assumes homogeneous word size/endianness — document or move to fixed-width
framing.
- **SCRAM-like daemon auth channel binding:** no TLS channel binding today
(and it is not RFC 5802).
- Still-open security nits: the pre-auth config/daemon-auth handshake has no
aggregate wall-clock deadline (per-message timeout only — slowloris holds
connection slots); the per-source registry fails open when the shared table
is full (per-module/global caps and host ACLs still apply).
3. **Out of scope / intentional:** pull (remote source) mode is **not** planned —
FastSync is push-only; see `RSYNC_COMPAT.md#direction`.
+2 -2
View File
@@ -610,8 +610,8 @@ remote SSH argv is already built injection-safe.
| `--backup-dir <dir>` | Store backups under a separate directory (requires `--backup`). |
| `--suffix <suffix>` | Set the backup filename suffix (default: `~`). |
| `--partial` | Select partial-transfer handling. On failed/interrupted writes the already-written temp file is retained (best-effort) for resumption. With `--partial --partial-dir <dir>`, completed files are written under the partial directory and installed atomically. |
| `--partial-dir <dir>` | Set a relative partial-transfer directory below the server destination root. Implies `--partial` (unless `--inplace`, which bypasses the partial/temp staging). |
| `--inplace` | Write directly to the destination instead of using a temporary file. |
| `--partial-dir <dir>` | Set a relative partial-transfer directory below the server destination root. Implies `--partial`. Rejected together with `--inplace` (`--inplace cannot be used with --partial-dir`, matching rsync), because the inplace path bypasses partial/temp staging. |
| `--inplace` | Write directly to the destination instead of using a temporary file. Cannot be combined with `--partial-dir`. |
| `--fsync` | Fsync every written file before publication. |
| `--write-batch=FILE` | Run the normal live transfer and also emit a self-contained batch file of the source tree. |
| `--only-write-batch=FILE` | Emit the batch file only (no destination, no server). |
+39 -17
View File
@@ -6,8 +6,8 @@ This document maps rsync's full feature set to FastSync's current implementation
| Status | Count | Description |
|--------|-------|-------------|
| ✅ Parity | 116 | Reproduces rsync's semantics for this option's scope |
| ⚠️ Caveat | 14 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
| ✅ Parity | 119 | Reproduces rsync's semantics for this option's scope |
| ⚠️ Caveat | 11 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
| ❌ Divergent | 27 | Rejected, an accepted no-op, deliberately non-rsync (native config/auth/batch, privileged namespaces, safe-subset privilege), or impossible on any portable filesystem call |
| **Total** | **157** | One row per rsync option/feature group; a row may name several spellings |
@@ -62,9 +62,23 @@ matrix is **111 ✅ / 13 ⚠️ / 33 ❌ = 157**.
- **Fuzzy eligibility.** The `-y/--fuzzy` candidate search no longer inherits the ordinary delta engine's 16 KiB minimum or 10× ratio bound, so an oversized or sub-16-KiB sibling is reused as rsync reuses it (`test_parity_basis_fuzzy.py`).
- **Output partials.** `--info=mount`/`--info=stats`, the `--stats` `dir:` breakdown under `-r`, and real `--debug` output for `flist`/`del`/`hash`/`deltasum`/`recv`/`filter`/`send` were added (`test_parity_info_mount_stats.py`, `test_output_parity.py`, `test_parity_debug.py`); those rows stay ⚠️ for their remaining documented residuals. `--delete-before`'s phase-0 late-file divergence and the `--progress` root/ancestor/symlink feedback remain open (they need a receiver→sender event channel), and the >256 MiB single-file streaming limit (B4) was not addressed. The matrix is now **120 ✅ / 10 ⚠️ / 27 ❌ = 157**.
**Audit cycle (no wire change; `PROTOCOL_VERSION` stays 2.28.0).** A security-and-correctness audit pass ran against the parity-2.29 baseline, followed by a set of audit follow-ups (filter merge modifiers, the `--inplace`/`--partial-dir` conflict, credential-file hardening, and small leak/log/test fixes). The only classification change is `--filter=RULE` moving ✅ → ⚠️, because its merge-only `e`/`n`/`w`/`-` modifiers are now accepted and consumed but their semantics remain unimplemented (accepted-but-ignored); the matrix is therefore **119 ✅ / 11 ⚠️ / 27 ❌ = 157**. The affected rows (`-z`/`--compress`, `--bwlimit`, `-T`/`--temp-dir`, `-p`/`--chmod`, `--partial-dir`, `--filter`) had their notes updated in place:
- **Decompression ceiling.** `MAX_DECOMPRESSED_SIZE` was 100 MiB while the receiver advertises and the sender compresses whole files up to `MAX_RECEIVE_WHOLE_FILE_SIZE` (256 MiB), so `-z` on a 100–256 MiB regular file failed with `Declared decompressed size exceeds 104857600 bytes`. The ceiling is now defined in terms of the protocol whole-file bound (still a real allocation-clamped bomb guard), so the two cannot drift; `-z` on 100–256 MiB files now works.
- **`--bwlimit` with `--sendfile`.** The plaintext-TCP `--sendfile` fast path wrote through `sendfile(2)` without passing through the protocol's token bucket, so `--bwlimit` was ignored on that path. It is now paced through the same per-session leaky bucket, so TLS and plaintext transports share identical `--bwlimit` semantics.
- **`--temp-dir` confinement.** The receiver's scratch dir was opened with a bare `open()`, so a client-planted symlink under the receive root could redirect receiver scratch files outside the authorized root. The opened directory is now judged by the real path of its fd (`/proc/self/fd` via `realpath`), and an escaping target is refused (`EACCES`, logged); an in-root link to another filesystem (the `EXDEV` fallback case) still works.
- **Special-bit masking and daemon umask.** Setuid/setgid/sticky bits from the client (`--perms`, `--chmod`, symlink and special-node paths, deferred directory modes) were applied even when the connection forbade super-user activities. They are now stripped when the super policy is off (`FileAttrPolicy.super_permitted`), and exact rsync semantics are preserved when permitted. The daemon's forced `umask(0)` is now `umask(022)`, so implied parent directories are no longer world-writable `0777`.
- **`--partial-dir` implies `--partial`.** Matching rsync 3.4.1 (which sets `keep_partial` after option parsing), `--partial-dir=DIR` alone now retains an interrupted transfer's partial and wins over an explicit `--no-partial`; `--inplace` still bypasses the partial machinery, and combining `--inplace` with `--partial-dir` is now rejected up front with rsync's message (`--inplace cannot be used with --partial-dir`) instead of silently ignoring the partial dir.
- **Filter modifiers.** The `x` xattr-name modifier is rejected everywhere with a clear error. The merge-only `e`/`n`/`w` and `-` modifiers are now accepted and consumed on `merge`/`dir-merge` rules (so they no longer leak into the merge filename) while still being rejected on non-merge rules, matching rsync; their semantics remain unimplemented (accepted-but-ignored). Glued patterns (`-newfile`, `-e2e`) and mixed tokens (`H,!secret`) keep their historical parsing.
- **Bounds and wire validation.** `--filter` rule count is now checked client-side against `MAX_FILTER_RULES` (with an actionable message before any network I/O) rather than surfacing as an opaque receiver protocol error; `send_protect_entries()` still re-checks the expanded count. Unknown wire `Status` values are rejected as protocol errors (`status_is_valid()`), and the audit also fixed a mutex leak on an init-failure path, an `errno`-after-`free()` in deferred delete application, `log_perror` misuse for non-`errno` conditions, `SSL_read` length clamping, `sendfile` `poll` `EINTR` retry, and printf-format/attribute issues.
- **Credential-file hardening follow-up.** `secret_file_open()` now opens `--password-file`/`--early-input`/`--hash-credentials` inputs with `O_NOFOLLOW`, so a symlinked credential path fails closed (`ELOOP`) instead of being followed before the owner/mode gate; literal fd-backed paths (`/dev/fd/<digits>`, `/proc/self/fd/<digits>`, which is what a bash process substitution passes) are exempt, so process substitution still works. A FIFO/process-substitution read now waits under a bounded ~3 s deadline for its writer, so a slow producer works while a connected-but-silent FIFO fails instead of hanging. The follow-up also fixed a `config_create` allocation leak on its `server_host` failure path (`config_delete` now releases it), corrected the decompression-limit log message to print the effective bound rather than the compile-time ceiling, and hardened the daemon umask/root test fixtures.
**Parity completion wave (protocol 2.23.0 → 2.26.0).** This wave closed the
remaining gaps the rsync-parity wave left open (delete timing, wire counters and
output, codec breadth, general `-R`/`-d`, the full filter grammar, receiver-side
output, codec breadth, general `-R`/`-d`, the filter grammar (the unsupported
`x` xattr-name modifier is explicitly rejected everywhere, while the merge-only
`e`/`n`/`w`/`-` modifiers are accepted and consumed on merge/dir-merge rules and
rejected elsewhere — see the audit-cycle follow-up note above), receiver-side
name resolution, absolute basis dirs, and the remaining client quick wins) and
reclassified the inherently non-rsync rows as **divergent** (native daemon
config/auth, the non-interoperable batch container, `--fake-super`'s xattr
@@ -122,7 +136,7 @@ Every one of those has an entry below with its remaining caveats.
|------|-------------------|-----------------|-------|
| `--exclude-from=FILE` | Read exclude patterns from file | ✅ Parity | Reads patterns from file |
| `--include-from=FILE` | Read include patterns from file | ✅ Parity | Reads patterns from file |
| `--filter=RULE` | Add file-filtering rule | ✅ Parity | The short `-f` **is** bound to `--filter` (the old FastSync sendfile conflict is gone; sendfile is long-only `--sendfile`), and `-f RULE`, `-f=RULE`, `--filter=RULE` and the two-argument form all parse. Protocol 2.26.0 implements rsync's filter grammar: `+`/`-`, `include`/`exclude`, a leading `/` anchor (to the transfer root or a `.rsync-filter` file's directory), a trailing `/` dir-only rule, and the `merge`/`.`, `dir-merge`/`:`, `hide`/`H`, `show`/`S`, `protect`/`P`, `risk`/`R` and `clear`/`!` words, including the `:`/`.` modifiers. First match wins; the filter layer is independent of `--exclude`/`--include`. **Track 4a (protocol 2.28.0) adds the receiver filter engine:** the sender compiles its root-level rules exactly as the scanner does (`filter_base_build`) and streams them as one bounded, self-describing config-frame block; the receiver reconstructs them and re-applies first-match-wins to every extraneous destination path during deletion, so a `P *.log` rule protects a destination-only `extra.log` (differential `filter_protect`/`filter_protect_during`/`filter_protect_delay` vs rsync 3.4.1, plus the `-n` would-delete enumeration) — matching rsync's dual-sided engine for the command-line rule set. **Remaining residual:** per-directory merge (`:`/`.`, and therefore `-F`) is not yet re-derived on the receiver; a destination-only entry that matches ONLY a per-directory merge rule is still protected only through the sender-derived source-mirror prefixes, not by the received base rule list |
| `--filter=RULE` | Add file-filtering rule | ⚠️ Caveat | The short `-f` **is** bound to `--filter` (the old FastSync sendfile conflict is gone; sendfile is long-only `--sendfile`), and `-f RULE`, `-f=RULE`, `--filter=RULE` and the two-argument form all parse. Protocol 2.26.0 implements rsync's filter grammar: `+`/`-`, `include`/`exclude`, a leading `/` anchor (to the transfer root or a `.rsync-filter` file's directory), a trailing `/` dir-only rule, and the `merge`/`.`, `dir-merge`/`:`, `hide`/`H`, `show`/`S`, `protect`/`P`, `risk`/`R` and `clear`/`!` words, including the `:`/`.` modifiers. The xattr-name `x` modifier is **explicitly rejected everywhere with a clear error**. The merge-only `e`/`n`/`w` and `-` modifiers are **accepted and consumed on `merge`/`dir-merge` rules** (so they no longer leak into the merge filename) while still being **rejected on non-merge rules**, matching rsync; their semantics remain unimplemented, so they are accepted-but-ignored (the reason this row is a caveat rather than parity). A token made up solely of modifier characters that names an unsupported modifier is rejected on non-merge rules, while glued patterns (`-newfile`, `-e2e`) and mixed tokens (`H,!secret`) keep their historical parsing. First match wins; the filter layer is independent of `--exclude`/`--include`. **Track 4a (protocol 2.28.0) adds the receiver filter engine:** the sender compiles its root-level rules exactly as the scanner does (`filter_base_build`) and streams them as one bounded, self-describing config-frame block; the receiver reconstructs them and re-applies first-match-wins to every extraneous destination path during deletion, so a `P *.log` rule protects a destination-only `extra.log` (differential `filter_protect`/`filter_protect_during`/`filter_protect_delay` vs rsync 3.4.1, plus the `-n` would-delete enumeration) — matching rsync's dual-sided engine for the command-line rule set. **Remaining residual:** per-directory merge (`:`/`.`, and therefore `-F`) is not yet re-derived on the receiver; a destination-only entry that matches ONLY a per-directory merge rule is still protected only through the sender-derived source-mirror prefixes, not by the received base rule list |
| `--files-from=FILE` | Read source file list from file | ✅ Parity | Entries are paths relative to the source root (leading `./` stripped, `..`/absolute rejected at parse time, blank lines ignored; NUL-delimited with `-0`). A listed regular file is transferred; a listed directory transfers its whole subtree (FastSync recursion is always on). Non-listed paths are pruned by the scanner; the delete manifest is scoped to the listed directory subtrees. A listed entry that does not exist is a hard error unless `--ignore-missing-args`/`--delete-missing-args` is given. **An empty list is a zero-transfer success (exit 0), matching rsync 3.4.1** — the earlier claim that rsync reports "no source files specified" was wrong. Scalability note: `file_list_affects` is O(list size) per scanned entry, so a very large list against a huge tree is quadratic (the documented bound) |
| `-0`, `--from0` | Delimit *-from files with NULs | ✅ Parity | `--files-from` entries become NUL-delimited; the flag may appear before or after `--files-from` on the command line. NUL mode preserves entry bytes exactly (trailing CR/LF are part of the name; only newline mode trims them) |
| `--max-size=SIZE` | Skip files larger than SIZE | ✅ Parity | `max_size` in scanner |
@@ -168,9 +182,9 @@ Every one of those has an entry below with its remaining caveats.
| `--backup-dir=DIR` | Backup directory hierarchy | ✅ Parity | `backup_dir` config field |
| `--suffix=SUFFIX` | Backup suffix (default ~) | ✅ Parity | `suffix` config field |
| `--delay-updates` | Put updated files in place at end | ❌ Divergent | Successfully received files are staged under a private 0700 `.fastsync-stage` dir inside the receive root and atomically renamed into their final destinations only after the whole transfer (manifest/delete handling included) succeeds, just before the success/outcome frame is sent. The delete walker deliberately skips the staging dir at the receive root, so `--delete` removes genuine extras but never the staged files (deletion runs before publication; rsync's delete-after ordering is not implemented). `--existing`/`--ignore-existing`/`--update` decide against the final destination path at stage time; `--backup` moves the old file aside at publication, and **`--force` is honored at publication** (protocol 2.23.0): a staged regular file or symlink may replace a destination directory that blocks it. Incompatible with `--inplace` and with `--backup-dir=.fastsync-stage` (the internal staging name is reserved; both are rejected). The staging dir name is fixed, so two simultaneous delayed transfers to the same destination root are serialized with an exclusive advisory lock held for the whole transfer: the second session fails cleanly instead of corrupting the first. Aborting or failing before publication installs nothing and removes the staging tree; a crash between stage and publish leaves staged leftovers that the next delayed run wipes at start (process death releases the lock). A stage→publish failure aborts the transfer (best-effort cleanup of the not-yet-published staged files; already-published files are not rolled back). **Reclassified Divergent (differential evidence):** the staging name is fixed and a delayed run wipes a pre-existing destination tree of that name at start even without `--delete`, whereas rsync uses its own internal temp name and leaves a genuine destination entry named `.fastsync-stage` untouched (`test_delay_updates_staging_name_collision_residual`); deletion also runs before publication while rsync's `--delay-updates` implies `--delete-after`. Works in single-threaded and `-j`/`--threads` modes |
| `-T`, `--temp-dir=DIR` | Create temporary files in DIR | ❌ Divergent | `--temp-dir` with the rsync short `-T` (the timeout alias moved to long-only `--timeout`). A **relative** dir matches rsync exactly: it is resolved below the receive/destination root and must already exist (differentially verified: `rsync -a --temp-dir=scratch src/ dst/` and FastSync produce identical trees and an empty scratch dir). **Reclassified as a deliberate divergence because an absolute `--temp-dir` is rejected by the receiver** — it is resolved verbatim by rsync standalone (which will use `/tmp` or any other absolute directory, including one outside the destination), but FastSync's security-reviewed receiver confines the scratch dir to the authorized receive root and rejects any absolute path or one containing `..`. A differential test confirms rsync exits 0 using an absolute scratch dir while FastSync refuses before writing anything into it (the scratch dir stays empty). Its daemon mode also confines relative to the module, but standalone rsync's absolute-temp-dir behavior is not reproduced because it would let a client place receiver scratch files outside the sandbox. Temp copies use a unique name in the scratch dir and are atomically renamed into place; **on `EXDEV` (scratch dir and destination on different filesystems, reachable via a confined relative symlink) the receiver falls back to a non-atomic copy instead of aborting**, matching rsync. `--inplace` and `--partial-dir` writes bypass the scratch dir |
| `-T`, `--temp-dir=DIR` | Create temporary files in DIR | ❌ Divergent | `--temp-dir` with the rsync short `-T` (the timeout alias moved to long-only `--timeout`). A **relative** dir matches rsync exactly: it is resolved below the receive/destination root and must already exist (differentially verified: `rsync -a --temp-dir=scratch src/ dst/` and FastSync produce identical trees and an empty scratch dir). **Reclassified as a deliberate divergence because an absolute `--temp-dir` is rejected by the receiver** — it is resolved verbatim by rsync standalone (which will use `/tmp` or any other absolute directory, including one outside the destination), but FastSync's security-reviewed receiver confines the scratch dir to the authorized receive root and rejects any absolute path or one containing `..`. **Audit-cycle hardening:** the opened dir is additionally judged by the real path of its fd (`/proc/self/fd`), so a client-planted symlink under the receive root cannot redirect receiver scratch files outside the authorized root (an escaping target is refused with `EACCES`), while an in-root symlink to another filesystem — the `EXDEV` fallback case — still works. A differential test confirms rsync exits 0 using an absolute scratch dir while FastSync refuses before writing anything into it (the scratch dir stays empty). Its daemon mode also confines relative to the module, but standalone rsync's absolute-temp-dir behavior is not reproduced because it would let a client place receiver scratch files outside the sandbox. Temp copies use a unique name in the scratch dir and are atomically renamed into place; **on `EXDEV` (scratch dir and destination on different filesystems, reachable via a confined relative symlink) the receiver falls back to a non-atomic copy instead of aborting**, matching rsync. `--inplace` and `--partial-dir` writes bypass the scratch dir |
| `--partial` | Keep partially transferred files | ✅ Parity | On a failed/interrupted write the already-written temp file is retained at the destination path (best-effort rename instead of unlink) so a later `--append`/`--append-verify` run can resume it. Retention never runs when no data was actually written or under `--ignore-existing`/`--existing` (the destination is not ours to overwrite), and it only ever renames the already-written temp. A failed rename falls back to the normal unlink |
| `--partial-dir=DIR` | Keep partial files in DIR | ✅ Parity | With `--partial`, the working file is written under the confined partial directory (a relative dir below the receive root) and atomically renamed into place once complete, so an interrupted transfer leaves a resumable copy there and completed transfers do not linger under it. `--inplace` bypasses the partial dir (rsync parity). Requires `--partial` |
| `--partial-dir=DIR` | Keep partial files in DIR | ✅ Parity | The working file is written under the confined partial directory (a relative dir below the receive root) and atomically renamed into place once complete, so an interrupted transfer leaves a resumable copy there and completed transfers do not linger under it. `--inplace` bypasses the partial dir (rsync parity), and combining `--inplace` with `--partial-dir` is now **rejected up front** with rsync's message (`--inplace cannot be used with --partial-dir`) instead of silently ignoring the partial dir. **Implies `--partial`** (audit-cycle fix, matching rsync 3.4.1, which sets `keep_partial` after option parsing): `--partial-dir=DIR` alone retains an interrupted transfer's partial, and the implication wins over an explicit `--no-partial` regardless of order |
## 7. Deletion
@@ -316,12 +330,12 @@ why plain `--append` works on the normal atomic path, not only with `--inplace`.
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `--preserve` | (FastSync alias, not an rsync flag) | ✅ Parity | **FastSync-only alias** for `-p` + `-t` (mode + mtime), long-form only. It is not rsync's `--preserve` (rsync has no such option); the short `-M` that used to spell it is now rsync's `--remote-option`. The wire metadata also carries uid/gid for `-o`/`-g`/`-a`, and ownership is applied via `-o`/`-g`, `-a`, or an explicit identity flag (`--numeric-ids`/`--usermap`/`--groupmap`/`--chown`/`--copy-as`) |
| `-p`, `--perms` | Preserve permissions | ✅ Parity | Real per-attribute flag (protocol 2.22.0): `preserve_perms` applies the source mode independently of times/owner/group. **Strict rsync parity (protocol 2.23.0): the source mode is copied exactly, including setuid/setgid/sticky and group/other-write bits — there is no masking.** Without `-p`, a new file gets `source_mode & ~umask` when metadata is present (else the historical fixed `0644`); new directories without `-p` still use FastSync's `0755` creation default, because directory metadata is only applied when a directory attribute is requested. `-A/--acls` implies `-p`; `--chmod` does **not** imply `-p` (rsync parity) and applies its own unsanitized changes to the new mode. `-X/--xattrs` does not imply `-p`. The SSH port moved to `--ssh-port`. rsync-parity short form |
| `-p`, `--perms` | Preserve permissions | ✅ Parity | Real per-attribute flag (protocol 2.22.0): `preserve_perms` applies the source mode independently of times/owner/group. **Strict rsync parity when super-user activities are permitted (protocol 2.23.0): the source mode is copied exactly, including setuid/setgid/sticky and group/other-write bits.** **Audit-cycle fix:** when the connection forbids super-user activities (`--no-super`, a non-opted daemon module, or a privileged standalone listener without `--allow-super`), the setuid/setgid/sticky bits are masked from the applied mode (the other bits are unaffected); exact rsync semantics are preserved wherever super activities are permitted. Without `-p`, a new file gets `source_mode & ~umask` when metadata is present (else the historical fixed `0644`); new directories without `-p` still use FastSync's `0755` creation default, because directory metadata is only applied when a directory attribute is requested. **Audit-cycle fix:** the daemon no longer forces `umask(0)` (which made implied parent directories world-writable `0777`); it uses the conventional `022`, and `-p`/`-a` still restore the exact source mode via `fchmod`. `-A/--acls` implies `-p`; `--chmod` does **not** imply `-p` (rsync parity) and applies its own unsanitized changes to the new mode. `-X/--xattrs` does not imply `-p`. The SSH port moved to `--ssh-port`. rsync-parity short form |
| `-o`, `--owner` | Preserve owner | ✅ Parity | Real per-attribute flag (`preserve_owner`): preserve the source uid, resolved on the receiver by name against its own user database with a raw-numeric fallback (only numeric ids cross the wire). `--usermap`/`--chown=USER` imply it. Application follows the `--super`/`--no-super` policy; a non-opted daemon module applies no ownership (see the Daemon Mode notes) |
| `-g`, `--group` | Preserve group | ✅ Parity | Real per-attribute flag (`preserve_group`): preserve the source gid, resolved by name on the receiver with a raw-numeric fallback. `--groupmap`/`--chown=:GROUP` imply it. Same privilege/super-policy gating as `-o` |
| `-t`, `--times` | Preserve modification times | ✅ Parity | Real per-attribute flag (`preserve_times`): apply the source mtime independently of the other attributes. `-O/--omit-dir-times` suppresses directories only and `-J/--omit-link-times` suppresses symlinks only; `-U`/`-N` do not imply it. `--preserve`/`-a` imply it, and `--incremental`/`--delta` auto-enable it unless `--no-times`/`--no-preserve` |
| `-E`, `--executability` | Preserve executability | ✅ Parity | Preserves executable permission bits (implies metadata preservation) |
| `--chmod=CHMOD` | Affect file permissions | ✅ Parity | Faithful port of rsync 3.4.1's `parse_chmod`/`tweak_mode`: numeric octal and symbolic `ugo`/`rwx` changes, `D`/`F` directory/file selectors, `X` (execute only on directories or already-executable files), `s`/`t` setuid/setgid/sticky, and append semantics — repeated clauses and repeated `--chmod` options accumulate in order (joined with commas). The changes are applied to the new mode **without sanitization** (matching rsync) and `--chmod` does **not** imply `-p` (rsync parity). Applied to files and directories on the receiver |
| `--chmod=CHMOD` | Affect file permissions | ✅ Parity | Faithful port of rsync 3.4.1's `parse_chmod`/`tweak_mode`: numeric octal and symbolic `ugo`/`rwx` changes, `D`/`F` directory/file selectors, `X` (execute only on directories or already-executable files), `s`/`t` setuid/setgid/sticky, and append semantics — repeated clauses and repeated `--chmod` options accumulate in order (joined with commas). The changes are applied to the new mode **without sanitization** (matching rsync), except that setuid/setgid/sticky are masked when the connection forbids super-user activities (audit-cycle fix, see `-p`), and `--chmod` does **not** imply `-p` (rsync parity). Applied to files and directories on the receiver |
| `-A`, `--acls` | Preserve ACLs | ✅ Parity | Implemented on Linux via the POSIX-ACL xattr representation: the sender captures the `system.posix_acl_access` / `system.posix_acl_default` xattrs and the receiver re-applies them fd-relative. A differential test with `setfacl` confirms the complete access and default ACL sets (including `mask`) are identical to rsync's on a directory. libacl is not required; a `fsetxattr` an unprivileged receiver may not perform is logged and skipped, never fatal. Only the `system.posix_acl_*` namespaces plus `user.*` are ever applied; privileged namespaces are never applied. Implies metadata transmission |
| `-X`, `--xattrs` | Preserve extended attributes | ❌ Divergent | Deliberately restricted to unprivileged `user.*` extended attributes plus the two POSIX ACL xattrs; `security.*` (SELinux, capabilities, ...) and `trusted.*` are **never** captured or applied — a client can never force a privileged attribute onto the destination, and the receiver independently re-validates every incoming name against the whitelist. This is a security-policy divergence from rsync, which can preserve the privileged namespaces with the needed privilege; implementing them would defeat FastSync's privilege-escalation guard. `user.*` capture/apply matches rsync in a differential test. Payloads are bounded on both ends. Incompatible with `-s` |
| `-H`, `--hard-links` | Preserve hard links | ✅ Parity | Files on the source that share an inode (`st_dev`+`st_ino`, e.g. a `cp -al` tree) are re-created as hard links to one another on the destination, so duplicate links stay deduplicated and only the first member's data is sent (later members are transmitted as payload-less `STATUS_HARDLINK` frames). The receiver links each sibling to the first member's installed file with an atomic link + rename; on `link()` failure it falls back to a byte-identical local copy of the first member, never a partial/corrupt file. Requires the sequential scan for ordering (the first member is always emitted and installed before any sibling is linked). Works single-threaded and under `-j`/`--threads`, `--inplace`, `--delay-updates` (links staged and published by rename) and `--partial`. Crosses the wire (`preserve_hard_links` bool; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0**, peers must match). Incompatible with `-s` (chunk serialization) and `--append`/`--append-verify`, rejected up front with a distinct error. See the Phase-4 hard-links notes below |
@@ -683,7 +697,7 @@ targets verbatim, matching rsync.
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-z`, `--compress` | Compress file data | ✅ Parity | Streaming compression. **Protocol 2.26.0 implements rsync 3.4.1's codec set** (`zstd` default, `lz4`, `zlib`, `zlibx`, `none`), selectable via `--compress-choice`/`--zc` and negotiated with `auto`. `-z` is the compression short form; `-c` is rsync's `--checksum`. `--skip-compress` applies rsync 3.4.1's default suffix list when no list is given. **Track 3a closes the codec caveats:** `zlibx` is no longer a divergence — FastSync's zlib stream already carries only the delta/token (literal) bytes, which is exactly rsync's zlibx semantics, so `--zc=zlib` and `--zc=zlibx` land the same tree/stdout/exit (differential `test_compress_codec_matches_rsync_bytes`) and the zlib/zlibx aliasing is only an implementation detail. Each codec now uses rsync's own default `--compress-level` (zstd 3, zlib/zlibx 6, lz4 ignored) and `auto` consults `RSYNC_COMPRESS_LIST` before the compiled-in order; the deterministic same-build resolution needs no peer probe |
| `-z`, `--compress` | Compress file data | ✅ Parity | Streaming compression. **Protocol 2.26.0 implements rsync 3.4.1's codec set** (`zstd` default, `lz4`, `zlib`, `zlibx`, `none`), selectable via `--compress-choice`/`--zc` and negotiated with `auto`. `-z` is the compression short form; `-c` is rsync's `--checksum`. `--skip-compress` applies rsync 3.4.1's default suffix list when no list is given. **Track 3a closes the codec caveats:** `zlibx` is no longer a divergence — FastSync's zlib stream already carries only the delta/token (literal) bytes, which is exactly rsync's zlibx semantics, so `--zc=zlib` and `--zc=zlibx` land the same tree/stdout/exit (differential `test_compress_codec_matches_rsync_bytes`) and the zlib/zlibx aliasing is only an implementation detail. Each codec now uses rsync's own default `--compress-level` (zstd 3, zlib/zlibx 6, lz4 ignored) and `auto` consults `RSYNC_COMPRESS_LIST` before the compiled-in order; the deterministic same-build resolution needs no peer probe. **Audit-cycle fix:** the decompressor's internal ceiling is now defined by the protocol whole-file bound (`MAX_RECEIVE_WHOLE_FILE_SIZE`, 256 MiB) instead of a separate 100 MiB constant, so `-z` on a 100–256 MiB regular file no longer fails with `Declared decompressed size exceeds 104857600 bytes` |
| `--compress-choice=STR`, `--zc=STR` | Choose compression algorithm | ✅ Parity | Protocol 2.26.0 accepts rsync 3.4.1's compiled-in choices — `zstd` (default), `lz4`, `zlib`, `zlibx`, `none`, `auto` — and rejects an unknown name with exit 4 like rsync. The negotiated codec id crosses the wire (`compression_algo`), so the receiver decodes with the sender's codec. `--zc` is the alias. `auto` now resolves through `RSYNC_COMPRESS_LIST` (whitespace-separated; unknown names skipped, first supported wins, all-unknown is exit 4) and then the compiled-in order, and an explicit `--zc` wins; the deterministic same-build resolution needs no peer probe. `zlib`/`zlibx` share FastSync's literal-only zlib path, which is rsync's zlibx behavior and is observably identical for both, so `zlibx` is not a divergence (the aliasing is an implementation detail) |
| `--compress-level=NUM`, `--zl=NUM` | Set compression level | ✅ Parity | Accepted range 1-22. When omitted, rsync 3.4.1's **per-codec default** applies: zstd 3 (`ZSTD_CLEVEL_DEFAULT`), zlib/zlibx 6 (`Z_DEFAULT_COMPRESSION` resolved), lz4 ignored (no tunable level; FastSync keeps a positive gate value and `lz4_compress` ignores it, so the bytes match rsync). An explicit level is clamped per codec like rsync's `init_compression_level()`: zstd 1-22, zlib/zlibx 1-9, lz4 ignored. Verified against `rsync --debug=NSTR1`, which reports the same effective level per codec |
| `--compress-threads=NUM` | Set compression threads | ✅ Parity | `compression_threads` config field (client-only; does not cross the wire). Sets the number of worker threads used by the zstd compression pool to NUM (1..64; 0/garbage/oversized rejected up front). Accepted in both `--compress-threads=NUM` and two-argument `--compress-threads NUM` forms. Composes with `-z`/compression; under the `-j`/`--threads` multithreaded pipeline it parallelizes compressed chunk encoding. See test_tcp.py `-z --compress-threads=2` and test_client_cli.c |
@@ -704,7 +718,7 @@ targets verbatim, matching rsync.
| `-4`, `--ipv4` | Prefer IPv4 | ✅ Parity | Forces `AF_INET` in the `getaddrinfo` hints for client destination/source resolution and the server bind (see the Phase 5, Wave B note). Mutually exclusive with `-6` |
| `-6`, `--ipv6` | Prefer IPv6 | ✅ Parity | Forces `AF_INET6` in the `getaddrinfo` hints for client destination/source resolution and the server bind. Mutually exclusive with `-4` |
| `--remote-option=OPT`, `-M` | Send an option only to the remote side | ❌ Divergent | Each value is appended to the remote server invocation over SSH as an individually single-quote-escaped shell word in `ssh_build_remote_command()`. Values are validated (non-empty, no control characters) and shell metacharacters cannot break out of the quoting (`;`, `&`, `\|`, <code>`</code>, `$`, `(`, `)`, quotes are neutralized), so a value cannot inject an arbitrary remote command and a subsequent `--` on the client line cannot be turned into one. The short `-M` form (`-M OPT`, `-M=OPT`, and rsync-style attached `-MOPT`) is available, matching rsync; metadata mode moved to long-only `--preserve`. **Reclassified because the daemon/TCP case cannot be reproduced:** `-M` is only meaningful for the SSH transport (`user@host:path`); a daemon (`host::module/path`) or local TCP destination **rejects** it, whereas rsync forwards it to its own remote process on every transport. A differential test starts a real rsync daemon and shows `-M--totally-bogus` reaching the remote parser (`unknown option`) while a valid `-M--safe-links` is accepted. FastSync's daemon handshake is a fixed binary config frame with no per-connection argv channel; adding one would let a client set arbitrary server-side options (the same class of divergence as the native daemon config/auth), so the safe subset stays SSH-only |
| `--bwlimit=RATE` | Limit I/O bandwidth | ✅ Parity | A faithful port of rsync 3.4.1's `parse_size_arg(bwlimit_arg, 'K', "bwlimit", 512, -1, True)`: a bare value is KiB/s, `K`/`M`/`G`/`T`/`P` are binary suffixes, `KB`/`MB` are decimal, `KiB`/`MiB` are binary, decimals are accepted and quantized to whole KiB exactly like rsync's `(size + 512) / 1024`, `0` (or an empty value) means "no limit", and any other value below the 512-byte floor is rejected. The token bucket's burst capacity is ~100 ms of bandwidth, matching the point at which rsync's leaky bucket starts sleeping, so a throttled transfer paces like rsync (4 MiB at `--bwlimit=1024`/`2048` matches rsync within ~4%). Differential-tested: the accept/reject matrix and the wall-clock rate both match rsync 3.4.1. The limit is a local I/O concern and is not negotiated on the wire |
| `--bwlimit=RATE` | Limit I/O bandwidth | ✅ Parity | A faithful port of rsync 3.4.1's `parse_size_arg(bwlimit_arg, 'K', "bwlimit", 512, -1, True)`: a bare value is KiB/s, `K`/`M`/`G`/`T`/`P` are binary suffixes, `KB`/`MB` are decimal, `KiB`/`MiB` are binary, decimals are accepted and quantized to whole KiB exactly like rsync's `(size + 512) / 1024`, `0` (or an empty value) means "no limit", and any other value below the 512-byte floor is rejected. The token bucket's burst capacity is ~100 ms of bandwidth, matching the point at which rsync's leaky bucket starts sleeping, so a throttled transfer paces like rsync (4 MiB at `--bwlimit=1024`/`2048` matches rsync within ~4%). Differential-tested: the accept/reject matrix and the wall-clock rate both match rsync 3.4.1. **Audit-cycle fix:** the plaintext-TCP `--sendfile` fast path now passes its writes through the same token bucket, so `--bwlimit` also paces it (previously the `sendfile(2)` path bypassed the limiter entirely); the TLS and plaintext transports therefore share identical throttling. The limit is a local I/O concern and is not negotiated on the wire |
## 14. Daemon Mode
@@ -714,8 +728,8 @@ targets verbatim, matching rsync.
| `--config=FILE` | Alternate rsyncd.conf file | ❌ Divergent | Wave A: selects the daemon config file. Default when omitted (in `--daemon` mode): `~/.config/fastsync/fastsyncd.conf` if it exists, else `/etc/fastsyncd.conf`. The grammar is FastSync-native (documented in the Daemon Mode notes below) and strictly rejects unknown keys so a typo can never silently change what a module serves; requires `--daemon` |
| `--dparam=OVERRIDE` | Override global daemon config | ❌ Divergent | Wave A: overrides one global scalar from the command line (`--dparam port=8734` and `--dparam=KEY=VALUE` both work). Limited to the global keys the grammar defines (`port`, `motd file`, `address`, `max connections`, `max connections per host`, `auth failure delay`, `auth lockout threshold`, `auth lockout duration`, `hosts allow`, `hosts deny`); keys are case-insensitive and unknown keys/invalid values are rejected. Requires `--daemon` |
| `--no-detach` | Don't detach from parent | ✅ Parity | Wave A: with `--daemon`, keeps the listener in the foreground (what integration tests use). Without it the daemonizes (fork/setsid, stdio redirected to /dev/null) after the listening socket is bound. Requires `--daemon` |
| `--password-file=FILE` | Read daemon password from file | ❌ Divergent | A7 daemon auth. Client: `--password-file` supplies `user:password` for a `host::module/path` destination (the username is taken from this file, so `user@host::module` stays rejected); the literal password is held client-side only for the SCRAM handshake and wiped at teardown. Server (`fastsync-server --daemon --password-file FILE`): the salted-PBKDF2 verifier store that modules with `auth users` are verified against. **Neither the password nor any replayable bearer value crosses the wire or is stored server-side** — the store holds a per-user salt plus derived keys, and the daemon proves the secret with a per-connection nonce challenge. The file must be private to its owner: both the client and server verify the exact inode they read (open-then-`fstat`, so the check cannot be raced) and refuse a `--password-file`/`--early-input` that is not owned by the current user or grants any group/other permission bit (mode 0600), mirroring the TLS private-key check. A process-substitution pipe (`--early-input <(vault ...)`) is still accepted when it satisfies those checks. See the Daemon Mode notes below for the file formats and the plaintext/TLS caveat |
| `--early-input=FILE` | Use FILE for daemon early exec | ❌ Divergent | Server-only (requires `--daemon`): a second credential-store file, same new-format grammar as `--password-file`, read before the listener accepts connections (a secrets-manager / process-substitution source). Its entries layer over `--password-file`: byte-identical verifiers dedupe, a conflicting verifier for the same user is a startup error. A daemon whose modules declare `auth users` must be given at least one of the two, or it refuses to start (fail closed) |
| `--password-file=FILE` | Read daemon password from file | ❌ Divergent | A7 daemon auth. Client: `--password-file` supplies `user:password` for a `host::module/path` destination (the username is taken from this file, so `user@host::module` stays rejected); the literal password is held client-side only for the SCRAM handshake and wiped at teardown. Server (`fastsync-server --daemon --password-file FILE`): the salted-PBKDF2 verifier store that modules with `auth users` are verified against. **Neither the password nor any replayable bearer value crosses the wire or is stored server-side** — the store holds a per-user salt plus derived keys, and the daemon proves the secret with a per-connection nonce challenge. The file must be private to its owner: both the client and server verify the exact inode they read (open-then-`fstat`, so the check cannot be raced) and refuse a `--password-file`/`--early-input` that is not owned by the current user or grants any group/other permission bit (mode 0600), mirroring the TLS private-key check. A process-substitution pipe (`--early-input <(vault ...)`) is still accepted when it satisfies those checks. **Hardening follow-up:** the file is opened with `O_NOFOLLOW`, so a symlinked credential path fails closed (`ELOOP`) instead of being followed before the owner/mode gate; literal fd-backed paths (`/dev/fd/<digits>`, `/proc/self/fd/<digits>`, which is what a bash process substitution passes) are exempt, so process substitution still works. A FIFO/process-substitution read now waits under a bounded ~3 s deadline for its writer, so a slow producer works while a connected-but-silent FIFO fails instead of hanging. See the Daemon Mode notes below for the file formats and the plaintext/TLS caveat |
| `--early-input=FILE` | Use FILE for daemon early exec | ❌ Divergent | Server-only (requires `--daemon`): a second credential-store file, same new-format grammar as `--password-file`, read before the listener accepts connections (a secrets-manager / process-substitution source). Its entries layer over `--password-file`: byte-identical verifiers dedupe, a conflicting verifier for the same user is a startup error. Opened with the same `O_NOFOLLOW` hardening as `--password-file` (a symlinked path fails closed with `ELOOP`; fd-backed `/dev/fd/N`/`/proc/self/fd/N` process-substitution paths are exempt) and a FIFO read is bound-waited (~3 s) so a slow producer works while a writer-less FIFO cannot hang. A daemon whose modules declare `auth users` must be given at least one of the two, or it refuses to start (fail closed) |
| `--hash-credentials=FILE`, `--iterations N` | Hash a plaintext credential file | ❌ Divergent | Server-only offline tool (A7): reads the `user:password` lines of FILE (same owner-only 0600 check) and prints one new-format store line per entry to stdout, then exits. `--iterations` sets the PBKDF2 work factor (default 600000, range 100000–10000000). Dependency-free and does not run a listener. Use its output as `--password-file` for `--daemon`. There is no auto-upgrade: a legacy store line is hard-rejected by the loader and must be regenerated |
**Daemon Mode notes (Wave A protocol 2.15.0; A7 auth protocol 2.19.0; MOTD no bump):** FastSync daemon mode is supported in FastSync's own protocol/config grammar, not rsync's SMB/daemon option encoding.
@@ -740,7 +754,7 @@ targets verbatim, matching rsync.
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| Path escape detection | Ensure files stay within root | ✅ Parity | `has_path_traversal()` + realpath |
| Symlink-safe delete | Skip symlinks in delete walk | ✅ Parity | `delete_extras_walk()` |
| Symlink-safe delete | Skip symlinks in delete walk | ✅ Parity | `delete_extras_fd()` (`src/shared/utils.c`) and `manifest_delete_extras()` (`src/shared/file_receive.c`) |
| Protocol version check | Verify compatible versions | ✅ Parity | `config_receive()` |
| Max data/string/chunk sizes | Prevent OOM attacks | ✅ Parity | Per-message limits |
| Per-connection memory limit | Cap memory per connection | ✅ Parity | `MAX_CONNECTION_MEMORY` is **256 MiB per connection** (256 * 1024 * 1024 bytes), charged across protocol reservations and decompression/chunk allocations. This is a FastSync-internal bound with no direct rsync analogue |
@@ -937,10 +951,12 @@ These are the last compatibility items and the closing phase toward rsync flag p
**Wire:** two trailing config-frame blocks after the `--iconv` spec, in fixed order — `send_privilege_options`/`receive_privilege_options` (one `super_mode` int, validated `0..2`), then `send_copy_as_options`/`receive_copy_as_options` (presence int + two int32 ids, validated `>= 0`, with `copy_as_set ⇒ use_metadata`). `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Divergences from rsync:** rsync's `--super` elevates the receiver and `--copy-as` actually switches its credentials; FastSync never elevates and only permits/forwards confined attempts, and `--copy-as` forces ownership rather than switching identity.
**Honest status after the parity 2.29 cycle (protocol 2.28.0, no wire change), updated by the parity cycle 2.29 pass.** ✅ Parity 120 / ⚠️ Caveat 10 / ❌ Divergent 27 = 157 rows. The 2.29 cycle closed the scanner-order, delete-timing, relative-basis, and fuzzy-eligibility residuals (moving `-n`/`--delete`/`--del`/`--delete-delay` to ✅) and improved the `--info`/`--stats`/`--debug` partial rows; the remaining ⚠️ rows are `--info`, `--debug`, `--msgs2stderr`, `--stats`, `--progress`, `--delete-before`, the three basis-dir options, and `-y/--fuzzy`. Earlier: **Honest status after the parity 2.28.0 cycle (protocol 2.28.0), updated by the rsync-parity-stats, rsync-parity-options, rsync-parity-fs, parity-review, no-wire parity-track-1/2b and wire parity-track-4a/5a passes.** ✅ Parity 116 / ⚠️ Caveat 14 / ❌ Divergent 27 = 157 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. This reclassification makes every difference explicit. The completion wave closed 23 previously-caveated rows (9 that triage showed were already parity, plus 14 genuine fixes) and turned the 17 inherently non-rsync rows — native daemon config/auth, the FastSync batch container, the safe-subset device/privilege flags, `-X`'s privileged namespaces, `--fake-super`'s native xattr format, and the `--old-args` no-op — into explicit ❌ divergences. The stats pass flipped `--delete-delay` to ✅ (actual-removal accounting), but the parity-review pass moved it back to ⚠️ because FastSync charged the `--max-delete` budget at plan/snapshot time and left a refilled snapshotted directory in place, whereas rsync charges on actual removals and recursively removes a queued directory (including content created after its plan). The no-wire parity-track-1 pass fixed both (actual-removal charging plus recursive deferred removal with an independent deferred-list cap), narrowing the caveat to the partial-delete ordering. The stats pass also reclassified `--out-format` to ❌ (protocol-specific `%b`/delta-`%c`), and sharpened the `--stats`/`--progress`/`--checksum-choice` residuals. The options pass flipped `--bwlimit` and `--ignore-errors` to ✅ (rsync-exact size parsing and ~100 ms leaky-bucket throttling, and rsync's skip-unreadable-subdir plus IO-error-suppressed deletion with exit 23) and emits rsync-format `--info=name/flist/del/remove/nonreg/progress` lines (real-run `deleting`/`*deleting` carried over a new trailing `report_deletes` wire bool, `PROTOCOL_VERSION` 2.26.0 → 2.27.0), while reclassifying `-M` over daemon/TCP
**Honest status after the parity 2.29 cycle (protocol 2.28.0, no wire change), updated by the parity cycle 2.29 pass and the audit-cycle follow-ups.** ✅ Parity 119 / ⚠️ Caveat 11 / ❌ Divergent 27 = 157 rows. The 2.29 cycle closed the scanner-order, delete-timing, relative-basis, and fuzzy-eligibility residuals (moving `-n`/`--delete`/`--del`/`--delete-delay` to ✅) and improved the `--info`/`--stats`/`--debug` partial rows; the remaining ⚠️ rows are `--info`, `--debug`, `--msgs2stderr`, `--stats`, `--progress`, `--delete-before`, `--filter`, the three basis-dir options, and `-y/--fuzzy`. Earlier: **Honest status after the parity 2.28.0 cycle (protocol 2.28.0), updated by the rsync-parity-stats, rsync-parity-options, rsync-parity-fs, parity-review, no-wire parity-track-1/2b and wire parity-track-4a/5a passes.** ✅ Parity 116 / ⚠️ Caveat 14 / ❌ Divergent 27 = 157 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. This reclassification makes every difference explicit. The completion wave closed 23 previously-caveated rows (9 that triage showed were already parity, plus 14 genuine fixes) and turned the 17 inherently non-rsync rows — native daemon config/auth, the FastSync batch container, the safe-subset device/privilege flags, `-X`'s privileged namespaces, `--fake-super`'s native xattr format, and the `--old-args` no-op — into explicit ❌ divergences. The stats pass flipped `--delete-delay` to ✅ (actual-removal accounting), but the parity-review pass moved it back to ⚠️ because FastSync charged the `--max-delete` budget at plan/snapshot time and left a refilled snapshotted directory in place, whereas rsync charges on actual removals and recursively removes a queued directory (including content created after its plan). The no-wire parity-track-1 pass fixed both (actual-removal charging plus recursive deferred removal with an independent deferred-list cap), narrowing the caveat to the partial-delete ordering. The stats pass also reclassified `--out-format` to ❌ (protocol-specific `%b`/delta-`%c`), and sharpened the `--stats`/`--progress`/`--checksum-choice` residuals. The options pass flipped `--bwlimit` and `--ignore-errors` to ✅ (rsync-exact size parsing and ~100 ms leaky-bucket throttling, and rsync's skip-unreadable-subdir plus IO-error-suppressed deletion with exit 23) and emits rsync-format `--info=name/flist/del/remove/nonreg/progress` lines (real-run `deleting`/`*deleting` carried over a new trailing `report_deletes` wire bool, `PROTOCOL_VERSION` 2.26.0 → 2.27.0), while reclassifying `-M` over daemon/TCP
and receiver-side `protect`/`risk` re-derivation to ❌ (no argv channel /
receiver filter engine); the wire parity-track-4a pass later added that
receiver filter engine, flipping `--filter=RULE` back to ✅ (see above). The fs pass flips `-d/--dirs` and `--iconv` to ✅ — recursive transfers now recreate empty source directories (and replace a blocking destination non-directory with an incoming directory); `-R --no-implied-dirs --files-from` places a listed file under a missing implied parent with default attributes instead of refusing; and `--iconv` now reproduces rsync's push direction (destination charset = the spec's REMOTE half) — and reclassified six rows to ❌ after reproducing their exact residual with differential tests: `--temp-dir` (the receiver confines the scratch dir to the receive root, so an absolute temp dir is deliberately rejected although standalone rsync follows it), the three basis-dir options (FastSync xxHash-verifies a basis hit while rsync's `--size-only` quick check installs the wrong basis content), `--delay-updates` (fixed staging name wipes an unrelated destination entry of that name), and `--dry-run` (would-delete report over-reports). `--fuzzy` was also reclassified to ❌ (deterministic heuristic with a 10× size window, not rsync's matcher), but its residual is the candidate-selection heuristic itself: the final tree is byte-exact by design, so no destination differential can expose it and the row is pinned by the `TestFuzzy` threshold suite rather than a byte-level rsync differential. (Track 5b later showed the name heuristic is in fact rsync's own and moved the row ❌ → ⚠️, leaving only the narrower delta size window as the residual; see the track 5b paragraph above.) The remaining ⚠️ rows are the ones with a documented residual (see the row notes and the **Parity Completion Wave (protocol 2.26.0)** section below).
receiver filter engine, flipping `--filter=RULE` back to ✅ (see above; the
audit-cycle follow-ups later moved it to ⚠️ for the accepted-but-ignored merge
modifiers, see the audit-cycle note). The fs pass flips `-d/--dirs` and `--iconv` to ✅ — recursive transfers now recreate empty source directories (and replace a blocking destination non-directory with an incoming directory); `-R --no-implied-dirs --files-from` places a listed file under a missing implied parent with default attributes instead of refusing; and `--iconv` now reproduces rsync's push direction (destination charset = the spec's REMOTE half) — and reclassified six rows to ❌ after reproducing their exact residual with differential tests: `--temp-dir` (the receiver confines the scratch dir to the receive root, so an absolute temp dir is deliberately rejected although standalone rsync follows it), the three basis-dir options (FastSync xxHash-verifies a basis hit while rsync's `--size-only` quick check installs the wrong basis content), `--delay-updates` (fixed staging name wipes an unrelated destination entry of that name), and `--dry-run` (would-delete report over-reports). `--fuzzy` was also reclassified to ❌ (deterministic heuristic with a 10× size window, not rsync's matcher), but its residual is the candidate-selection heuristic itself: the final tree is byte-exact by design, so no destination differential can expose it and the row is pinned by the `TestFuzzy` threshold suite rather than a byte-level rsync differential. (Track 5b later showed the name heuristic is in fact rsync's own and moved the row ❌ → ⚠️, leaving only the narrower delta size window as the residual; see the track 5b paragraph above.) The remaining ⚠️ rows are the ones with a documented residual (see the row notes and the **Parity Completion Wave (protocol 2.26.0)** section below).
**Preserve-attribute split (protocol 2.21.0 → 2.22.0) — ✅ implemented.** FastSync splits the former single metadata bundle into four independent, rsync-compatible per-attribute flags — `-p/--perms`, `-t/--times`, `-o/--owner`, `-g/--group` — each with a negation (`--no-perms`/`--no-times`/`--no-owner`/`--no-group`, short `--no-p`/`--no-t`/`--no-o`/`--no-g`), plus `--no-preserve` clearing all four. `-a/--archive` is now full rsync `-rlptgoD` (owner and group included, though their application stays privilege-gated), `-A/--acls` implies `-p`, `-X/--xattrs` does not, `-E/--executability` sets only executability, and `-U`/`-N` do not imply `-t`. `--incremental`/`--delta` still auto-preserve perms+times unless the user explicitly negated them. Wire: the binary config frame gains four appended booleans (`preserve_perms`/`preserve_times`/`preserve_owner`/`preserve_group`) after `omit_link_times`, so `PROTOCOL_VERSION` is bumped **2.21.0 → 2.22.0**; the fixed-width `FileMetadata` layout is unchanged and the receiver gates the metadata frame on a derived `use_metadata`. Receiver behavior: each attribute is applied independently, directory modes are applied under `-p` (at the end of the transfer, alongside dir times), symlink mode under `-p`, and `-O/--omit-dir-times` suppresses directory times only. Documented divergences as of 2.22.0, **all but (d)/(e) removed by the rsync-parity wave (protocol 2.23.0)**: (a) the mode-masking divergence is **gone** — under `-p` the source mode is now copied exactly, including `S_IWGRP`/`S_IWOTH` and setuid/setgid/sticky; (b) a brand-new file without `-p` still gets `source_mode & ~umask` when metadata is present (else the historical fixed `0644`), and a new *directory* without `-p` still uses FastSync's `0755` default; (c) the `--chmod`-implies-`-p` divergence is **gone** — `--chmod` no longer implies `-p` (rsync parity); (d) `-o`/`-g` map by name on the receiver with a raw-numeric fallback (only numeric ids cross the wire); (e) a daemon module without `client owner = yes` does not refuse a plain `-a`/`-o`/`-g` — it forces super off, applies no ownership, and logs a warning, while explicit `--chown`/`--usermap`/`--groupmap`/`--numeric-ids`/`--copy-as`/`--super` are still refused.
@@ -1073,7 +1089,9 @@ These remain after the wave; they are the reasons a row above is ⚠️.
count) are reported as 0; `--progress` is an aggregate line, not per-file.
- **`--password-file`/`--early-input`/`--hash-credentials`/`--iterations` are
FastSync-native** (SCRAM/PBKDF2), not rsync semantics; the batch format is not
rsync-interoperable.
rsync-interoperable. Credential files are opened with `O_NOFOLLOW` (a symlinked
path fails closed; fd-backed process-substitution paths are exempt) and a FIFO
read is bound-waited (~3 s).
- **xattr/ACL namespace policy** permits only `user.*` and
`system.posix_acl_*` when `-A` is negotiated (stricter than rsync).
- **`--stop-at` remains a FastSync-flexible parser** (client-only, not
@@ -1153,7 +1171,11 @@ wire protocol three times (full rationale in `src/shared/config.h`):
- **Filter grammar:** `merge`/`.`, `dir-merge`/`:`, `hide`/`H`, `show`/`S`,
`protect`/`P`, `risk`/`R`, `clear`/`!`, include/exclude and the `:`/`.`
modifiers; `-f` is bound to `--filter`; a single `-F` transfers
`.rsync-filter` and `-FF` excludes it.
`.rsync-filter` and `-FF` excludes it. The xattr-name `x` modifier is **not
implemented** and is rejected with a clear error everywhere. The merge-only
`e`/`n`/`w` and `-` modifiers are accepted and consumed on `merge`/`dir-merge`
rules (rejected elsewhere, matching rsync), but their semantics are **not
implemented** (accepted-but-ignored).
- **Absolute basis directories** are used verbatim (rsync semantics) and
**`--link-dest`** relinks an already up-to-date destination.
+7
View File
@@ -75,6 +75,13 @@ bool validate_config(const Config* config) {
log_message(LOG_LEVEL_ERROR, "-4/--ipv4 and -6/--ipv6 are mutually exclusive");
return false;
}
/* rsync 3.4.1 rejects --inplace together with --partial-dir (exit 1): the
inplace write path bypasses partial staging, so a partial-dir name would be
silently ignored. Match rsync's message and refuse before any I/O. */
if (config->inplace && config->partial_dir) {
log_message(LOG_LEVEL_ERROR, "--inplace cannot be used with --partial-dir");
return false;
}
if (config->log_file_format && !config->log_file) {
log_message(LOG_LEVEL_ERROR, "--log-file-format requires --log-file");
return false;
+1 -2
View File
@@ -644,8 +644,7 @@ static Data* zstd_decompress(Data* compressed_data, size_t maximum_size) {
}
if (ret > 0 && output.pos == output.size) {
if (buf_size >= hard_limit || buf_size > SIZE_MAX / 2) {
log_message(LOG_LEVEL_ERROR, "Decompressed data exceeds %llu bytes",
(unsigned long long)MAX_DECOMPRESSED_SIZE);
log_message(LOG_LEVEL_ERROR, "Decompressed data exceeds %llu bytes", hard_limit);
data_destroy(uncompressed_data);
uncompressed_data = NULL;
goto cleanup;
+1 -1
View File
@@ -234,7 +234,7 @@ Config* config_create(void) {
* server_host NULL and would crash later consumers, so fail the whole create
* (every caller already handles a NULL return). */
if (!config->server_host) {
free(config);
config_delete(config);
return NULL;
}
return config;
+160 -14
View File
@@ -8,11 +8,13 @@
#include <openssl/evp.h>
#include <openssl/params.h>
#include <openssl/rand.h>
#include <poll.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <time.h>
#include <unistd.h>
/* One store entry: a username and its salted PBKDF2 verifier. The plaintext
@@ -105,11 +107,16 @@ static bool is_fd_backed_path(const char* path) {
* O_NOFOLLOW guards against, and requiring O_NOFOLLOW would break the
* documented process-substitution/FIFO usage. For them only, O_NOFOLLOW is
* omitted; the same fstat owner/mode gate still applies to the resolved inode.
* O_NONBLOCK keeps a FIFO
* from blocking the open/read forever: an empty or writer-less FIFO yields
* EOF/EAGAIN rather than hanging in fgets. Only regular files and FIFOs pass
* the ownership/mode checks; O_NONBLOCK is cleared for regular files, where it
* is a no-op anyway, so their stdio read path is byte-for-byte unchanged.
* O_NONBLOCK keeps the OPEN itself from
* blocking forever on a writer-less FIFO (a blocking O_RDONLY open would wait
* for a writer). The fd is left nonblocking for FIFOs so a read never blocks
* either; the read loop (secret_read_line) absorbs the resulting EAGAIN by
* waiting, under a bounded deadline, for the writer -- this is what makes a
* slow process substitution (`--password-file <(sleep 1; ...)`) work while a
* writer-less FIFO still fails after the deadline instead of hanging. Only
* regular files and FIFOs pass the ownership/mode checks; O_NONBLOCK is
* cleared for regular files, where it is a no-op anyway and no EAGAIN can
* occur, so their stdio read path is byte-for-byte unchanged.
*
* Returns a FILE* the caller must fclose, or NULL with `err` filled. */
static FILE* secret_file_open(const char* path, char* err, size_t err_size) {
@@ -141,9 +148,9 @@ static FILE* secret_file_open(const char* path, char* err, size_t err_size) {
* most systems, but explicit. Failures here are ignored: O_NONBLOCK on a
* regular file does not affect reads either way. */
if (S_ISREG(st.st_mode)) {
int flags = fcntl(fd, F_GETFL);
if (flags >= 0)
(void)fcntl(fd, F_SETFL, flags & ~O_NONBLOCK);
int status_flags = fcntl(fd, F_GETFL);
if (status_flags >= 0)
(void)fcntl(fd, F_SETFL, status_flags & ~O_NONBLOCK);
}
FILE* fp = fdopen(fd, "r");
if (!fp) {
@@ -154,6 +161,123 @@ static FILE* secret_file_open(const char* path, char* err, size_t err_size) {
return fp;
}
/* Overall bound on how long the reader waits for a process-substitution/FIFO
* writer to produce data before giving up. It must comfortably exceed a
* producer's startup delay (e.g. `--password-file <(sleep 1; ...)`) while still
* bounding a writer-less FIFO, so a stray or hostile FIFO cannot stall the
* daemon or client indefinitely. */
#define CREDENTIAL_FIFO_READ_TIMEOUT_MS 3000
/* Monotonic milliseconds, used only for the read deadline (wall-clock changes
* must not extend or shorten the wait). */
static int64_t credential_monotonic_ms(void) {
struct timespec ts;
if (clock_gettime(CLOCK_MONOTONIC, &ts) != 0)
return 0;
return (int64_t)ts.tv_sec * 1000 + (int64_t)(ts.tv_nsec / 1000000);
}
/* Wait until `fd` is readable or the deadline passes. Returns true when it is
* readable, false on timeout or a poll error (err filled). EINTR is retried
* against the same deadline, so signals cannot extend the wait. */
static bool credential_wait_readable(int fd, int64_t deadline, const char* label, const char* path,
char* err, size_t err_size) {
for (;;) {
int64_t remaining = deadline - credential_monotonic_ms();
if (remaining <= 0)
break;
if (remaining > INT_MAX)
remaining = INT_MAX;
struct pollfd pfd = {.fd = fd, .events = POLLIN, .revents = 0};
int rc = poll(&pfd, 1, (int)remaining);
if (rc > 0)
return true;
if (rc == 0)
break;
if (errno != EINTR) {
set_error(err, err_size, "error waiting for %s '%s': %s", label, path, strerror(errno));
return false;
}
}
set_error(err, err_size, "timed out after %d ms waiting for %s '%s'",
CREDENTIAL_FIFO_READ_TIMEOUT_MS, label, path);
return false;
}
typedef enum {
SECRET_READ_LINE,
SECRET_READ_EOF,
SECRET_READ_ERROR,
} SecretReadResult;
/* Read one complete line from `fp` into `line` (capacity `cap`), including the
* trailing newline when present and always NUL-terminating. `*out_len`
* receives strlen(line).
*
* A regular file is read exactly as before: secret_file_open leaves it
* blocking, so fgets never sees EAGAIN. A FIFO stays nonblocking, so fgets
* returns NULL (or a partial line) with EAGAIN while the writer is still
* starting up; instead of treating that as a fatal error the loop clearerr()s
* and polls for readability against one overall deadline. The `used`
* accumulator reassembles a line that arrived in several write()s into a single
* line, so a split write is not misparsed as two entries.
*
* Returns SECRET_READ_LINE, SECRET_READ_EOF, or SECRET_READ_ERROR (err filled)
* on timeout or a genuine read error. */
static SecretReadResult secret_read_line(char* line, size_t cap, FILE* fp, const char* label,
const char* path, size_t* out_len, char* err,
size_t err_size) {
int fd = fileno(fp);
int64_t deadline = credential_monotonic_ms() + CREDENTIAL_FIFO_READ_TIMEOUT_MS;
size_t used = 0;
line[0] = '\0';
for (;;) {
errno = 0;
if (fgets(line + used, (int)(cap - used), fp)) {
used += strlen(line + used);
if (used > 0 && line[used - 1] == '\n') {
*out_len = used;
return SECRET_READ_LINE;
}
if (feof(fp)) {
*out_len = used; /* final unterminated line */
return SECRET_READ_LINE;
}
/* No newline and not EOF. A full buffer is the caller's over-long-line
* case; otherwise the line is only partially available (a nonblocking
* FIFO under a slow writer), so any genuine read error fails and anything
* else waits for the rest. */
if (used >= cap - 1) {
*out_len = used;
return SECRET_READ_LINE;
}
int e = ferror(fp) ? errno : 0;
if (e != 0 && e != EAGAIN && e != EWOULDBLOCK) {
set_error(err, err_size, "error reading %s '%s': %s", label, path, strerror(e));
return SECRET_READ_ERROR;
}
clearerr(fp);
if (!credential_wait_readable(fd, deadline, label, path, err, err_size))
return SECRET_READ_ERROR;
continue;
}
/* fgets returned NULL: EOF, a not-yet-readable FIFO, or a real error. */
if (feof(fp)) {
*out_len = used;
return used > 0 ? SECRET_READ_LINE : SECRET_READ_EOF;
}
if (errno == EAGAIN || errno == EWOULDBLOCK) {
clearerr(fp);
if (!credential_wait_readable(fd, deadline, label, path, err, err_size))
return SECRET_READ_ERROR;
continue;
}
set_error(err, err_size, "error reading %s '%s': %s", label, path,
errno != 0 ? strerror(errno) : "read failed");
return SECRET_READ_ERROR;
}
}
/* Trim leading/trailing ASCII space and tab in place; returns the new start. */
static char* trim_space(char* s) {
while (*s == ' ' || *s == '\t')
@@ -549,9 +673,17 @@ static CredentialStore* load_store_file(const char* path, char* err, size_t err_
char line[CREDENTIAL_MAX_LINE + 2];
bool ok = true;
while (fgets(line, sizeof(line), fp)) {
for (;;) {
size_t len = 0;
SecretReadResult rr =
secret_read_line(line, sizeof(line), fp, "credential file", path, &len, err, err_size);
if (rr == SECRET_READ_EOF)
break;
if (rr == SECRET_READ_ERROR) {
ok = false;
break;
}
line_no++;
size_t len = strlen(line);
if (len == CREDENTIAL_MAX_LINE + 1 && line[len - 1] != '\n' && !feof(fp)) {
set_error(err, err_size, "credential file '%s' line %d exceeds the %d-byte limit", path,
line_no, CREDENTIAL_MAX_LINE);
@@ -1188,9 +1320,17 @@ int credentials_hash_file(const char* path, uint32_t iters, FILE* out, char* err
int line_no = 0;
int result = 0;
char line[CREDENTIAL_MAX_LINE + 2];
while (fgets(line, sizeof(line), fp)) {
for (;;) {
size_t len = 0;
SecretReadResult rr =
secret_read_line(line, sizeof(line), fp, "plaintext file", path, &len, err, err_size);
if (rr == SECRET_READ_EOF)
break;
if (rr == SECRET_READ_ERROR) {
result = -1;
break;
}
line_no++;
size_t len = strlen(line);
if (len == CREDENTIAL_MAX_LINE + 1 && line[len - 1] != '\n' && !feof(fp)) {
set_error(err, err_size, "plaintext file '%s' line %d exceeds the %d-byte limit", path,
line_no, CREDENTIAL_MAX_LINE);
@@ -1268,9 +1408,15 @@ int credentials_read_secret_file(const char* path, char** user_out, char** passw
char line[CREDENTIAL_MAX_LINE + 2];
int result = -1;
while (fgets(line, sizeof(line), fp)) {
for (;;) {
size_t len = 0;
SecretReadResult rr =
secret_read_line(line, sizeof(line), fp, "password file", path, &len, err, err_size);
if (rr == SECRET_READ_EOF)
break;
if (rr == SECRET_READ_ERROR)
goto done;
line_no++;
size_t len = strlen(line);
if (len == CREDENTIAL_MAX_LINE + 1 && line[len - 1] != '\n' && !feof(fp)) {
set_error(err, err_size, "password file '%s' line %d exceeds the %d-byte limit", path,
line_no, CREDENTIAL_MAX_LINE);
+39 -15
View File
@@ -162,33 +162,57 @@ static bool is_modifier_char(char c) {
return c == 's' || c == 'r' || c == 'p' || c == 'x' || c == '/' || c == '!' || c == 'C';
}
/* Modifiers rsync defines but FastSync does not implement. They must still be
* consumed as part of the modifier run so they are rejected explicitly instead
* of leaking into the pattern (which produced misleading failures such as
* "could not read merge file 'n file'"). */
/* merge/dir-merge rules are the only rules rsync accepts the merge-file
* modifiers on. */
static bool is_merge_rule(RuleKind kind) {
return kind == RULE_KIND_MERGE || kind == RULE_KIND_DIR_MERGE;
}
/* Merge-file modifiers rsync defines but FastSync does not implement:
* 'e' exclude the merge file itself, 'n' do not inherit the merge file, 'w'
* word-split the merge file. They are recognized as part of a modifier run on
* every rule (so a pure e/n/w token is rejected rather than folded into the
* pattern), but are accepted (and ignored) only on merge/dir-merge rules. */
static bool is_unsupported_modifier_char(char c) {
return c == 'e' || c == 'n' || c == 'w';
}
/* Characters that are part of a modifier run, whether supported or not. */
static bool is_modifier_scan_char(char c) {
return is_modifier_char(c) || is_unsupported_modifier_char(c);
/* Merge-file modifiers rsync accepts on merge/dir-merge rules: 'e', 'n', 'w'
* and '-' (do not transfer the merge file). */
static bool is_merge_modifier_char(char c) {
return c == 'e' || c == 'n' || c == 'w' || c == '-';
}
/* Characters that count as part of a modifier run for `kind` when deciding
* whether a token is a pure modifier run. e/n/w count on every rule so that a
* pure e/n/w token is rejected on non-merge rules; '-' only on merge rules. */
static bool is_modifier_scan_char(char c, RuleKind kind) {
return is_modifier_char(c) || is_unsupported_modifier_char(c) ||
(is_merge_rule(kind) && is_merge_modifier_char(c));
}
/* Characters actually consumed as modifiers for `kind`. The merge-file
* modifiers are consumed only on merge/dir-merge rules; elsewhere e/n/w fall
* through to the pattern (so mixed tokens such as "H,!secret" keep their
* historical "ecret" pattern). */
static bool is_consumed_modifier_char(char c, RuleKind kind) {
return is_modifier_char(c) || (is_merge_rule(kind) && is_merge_modifier_char(c));
}
/* Inspect the token that follows a rule name (up to the first space/underscore
* or the end). If the token is composed *solely* of modifier characters and
* includes one FastSync does not implement, it is unambiguously a modifier run:
* includes one that is invalid for `kind`, it is unambiguously a modifier run:
* return that character so the caller can reject it. A token that contains any
* non-modifier character is a pattern (e.g. "-newfile") and returns '\0', which
* keeps the historical parsing of mixed tokens such as "H,!secret" intact. */
static char unsupported_modifier_in_token(const char* tok) {
static char unsupported_modifier_in_token(const char* tok, RuleKind kind) {
if (*tok == '\0' || *tok == ' ' || *tok == '_')
return '\0';
char bad = '\0';
for (const char* q = tok; *q != '\0' && *q != ' ' && *q != '_'; q++) {
if (!is_modifier_scan_char(*q))
if (!is_modifier_scan_char(*q, kind))
return '\0';
if (is_unsupported_modifier_char(*q))
if (!is_merge_rule(kind) && is_unsupported_modifier_char(*q))
bad = *q;
}
return bad;
@@ -238,9 +262,9 @@ static bool parse_rule_syntax(const char* text, RuleKind* kind, unsigned* sides,
Only commit a modifier run that terminates at a separator or the end, so a
pattern such as "*.tmp" written as "-*.tmp" is not mistaken for modifiers. */
if (*p == ',') {
*bad_mod = unsupported_modifier_in_token(p + 1);
*bad_mod = unsupported_modifier_in_token(p + 1, *kind);
} else if (is_short) {
*bad_mod = unsupported_modifier_in_token(p);
*bad_mod = unsupported_modifier_in_token(p, *kind);
}
if (*bad_mod != '\0')
return false;
@@ -250,12 +274,12 @@ static bool parse_rule_syntax(const char* text, RuleKind* kind, unsigned* sides,
if (*p == ',') {
p++;
mod_start = p;
while (is_modifier_char(*p))
while (is_consumed_modifier_char(*p, *kind))
p++;
mod_end = p;
} else if (is_short) {
const char* scan = p;
while (is_modifier_char(*scan))
while (is_consumed_modifier_char(*scan, *kind))
scan++;
if (*scan == '\0' || *scan == ' ' || *scan == '_') {
mod_start = p;
+6 -2
View File
@@ -24,8 +24,12 @@
* clear/! clear the current rule list (takes no argument)
* Modifiers: '/' absolute anchor, '!' negate match, 'C' inject CVS defaults,
* 's' sender side, 'r' receiver side, 'p' perishable. The rsync 'x'
* (xattr-name) modifier and the merge-only 'e'/'n'/'w' modifiers are not
* implemented and are rejected explicitly.
* (xattr-name) modifier is not implemented and is rejected explicitly
* everywhere. The merge-file modifiers 'e' (exclude the merge file itself),
* 'n' (do not inherit the merge file), 'w' (word-split the merge file) and '-'
* (do not transfer the merge file) are accepted and consumed only on merge/
* dir-merge rules (rejected on every other rule, matching rsync); their
* semantics are not implemented and they are otherwise ignored.
* A trailing '/' makes a pattern match directories only. A leading '/' anchors
* the pattern to its owner directory.
*/
+37 -10
View File
@@ -63,6 +63,10 @@ DETACH_MODULE = os.path.join(MODULE_ROOT, "detach")
DETACH_CONF = os.path.join(TEST_DATA_DIR, "fastsyncd_detach.conf")
DETACH_PORT = None
# A dedicated config for the umask test: the daemon must be launched in the real
# (double-fork) detach path, whose daemonize() applies umask(022).
UMASK_CONF = os.path.join(TEST_DATA_DIR, "fastsyncd_umask.conf")
# Passwords are never sent as plaintext and never logged; these literals are
# only hashed into the server credential file / client password file.
ALICE_PASS = "alice-s3cret"
@@ -332,20 +336,43 @@ class TestDaemonModuleSelection:
assert not missing, f"missing: {missing[:5]}"
assert not mismatches, f"mismatch: {mismatches[:5]}"
def test_daemon_new_dirs_not_world_writable(self, daemon):
def test_daemon_new_dirs_not_world_writable(self):
"""The daemon must not force umask 0: implied parent directories created
without -p are the source default (0755 under a 022 umask), never
world-writable 0777."""
without -p are the source default (0755 under the daemon's 022 umask),
never world-writable 0777.
This drives the real double-fork detach path, where the umask(022) fix
lives (daemonize()); the --no-detach path never calls it. The launcher
is run with umask 0, so without the fix the daemon would inherit 0 and
create a 0777 directory; with the fix the assertion below fails only if
the fix regresses."""
port = _find_free_port()
with open(UMASK_CONF, "w") as f:
f.write("port = %d\n\n[files]\npath = %s\n" % (port, FILES_MODULE))
sub = os.path.join(FILES_MODULE, "umask_check")
shutil.rmtree(sub, ignore_errors=True)
os.makedirs(sub, exist_ok=True)
result = _push("127.0.0.1::files/umask_check", daemon.port)
assert result.returncode == 0, result.stderr or result.stdout
received = get_dest_received_dir(sub, SOURCE_DIR)
nested = os.path.join(received, "nested")
assert os.path.isdir(nested), f"nested dir missing under {received}"
mode = stat.S_IMODE(os.stat(nested).st_mode)
assert (mode & 0o022) == 0, f"implied directory is group/other writable: {oct(mode)}"
log_path = os.path.join(TEST_DATA_DIR, "fastsyncd_umask.log")
log = open(log_path, "w")
cmd = SERVER_CMD + ["--daemon", "--config", UMASK_CONF, "--allow-unauthenticated"]
proc = subprocess.Popen(cmd, stdout=log, stderr=log, stdin=subprocess.DEVNULL,
preexec_fn=lambda: os.umask(0))
try:
_wait_for_port(port, timeout=15)
result = _push("127.0.0.1::files/umask_check", port)
assert result.returncode == 0, result.stderr or result.stdout
received = get_dest_received_dir(sub, SOURCE_DIR)
nested = os.path.join(received, "nested")
assert os.path.isdir(nested), f"nested dir missing under {received}"
mode = stat.S_IMODE(os.stat(nested).st_mode)
assert (mode & 0o022) == 0, f"implied directory is group/other writable: {oct(mode)}"
finally:
_kill_by_cmdline_marker(UMASK_CONF)
log.close()
try:
proc.wait(timeout=5)
except subprocess.TimeoutExpired:
proc.kill()
class TestDaemonRejection:
+6 -1
View File
@@ -578,13 +578,18 @@ static void test_parse_args_partial_dir_implies_partial() {
config_delete(cfg);
}
{
/* --inplace bypasses partial staging: the implication must not fire. */
/* --inplace bypasses partial staging, so parse_args must not set the
implied --partial; the combination itself is invalid (rsync parity:
"--inplace cannot be used with --partial-dir"), so validation rejects. */
Config* cfg = config_create();
char* argv[] = {"fastsync", "--inplace", "--partial-dir=.partial", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, argv, positional_args, &positional_count), 0);
EXPECT_FALSE(cfg->partial);
cfg->send_directory = str_dup("/src");
cfg->receive_root_directory = str_dup("/dst");
EXPECT_FALSE(validate_config(cfg));
config_delete(cfg);
}
{
+158
View File
@@ -10,6 +10,7 @@
#include <string.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>
/* Known-answer vector, independently recomputed with Python
@@ -764,6 +765,160 @@ static void test_credentials_read_secret_file_fifo_no_hang() {
unlink(fifo);
}
/* Write `s` fully to `fd`, retrying EINTR. */
static void write_all_fd(int fd, const char* s) {
size_t total = strlen(s);
size_t off = 0;
while (off < total) {
ssize_t w = write(fd, s + off, total - off);
if (w < 0) {
if (errno == EINTR)
continue;
return;
}
off += (size_t)w;
}
}
/* Deterministically model a slow process substitution (`--password-file
* <(sleep N; ...)`): attach a writer to the FIFO (so the reader sees EAGAIN --
* the empty/no-writer FIFO instead yields an immediate EOF), have it sleep
* `delay_ms`, then write `first` and, after another `delay_ms`, `second` (NULL
* for a single write). Splitting across the delay exercises reassembly of a
* line delivered by several write()s.
*
* The parent keeps a spare read end open for the lifetime of the test so the
* writer always has a reader; the caller must close(*hold_out), waitpid() the
* returned pid and unlink the FIFO. Returns the child pid, or -1 on setup
* failure. */
static pid_t fifo_writer_sleep_then_write(const char* fifo, const char* first, unsigned delay_ms,
const char* second, int* hold_out) {
int sync[2];
if (pipe(sync) != 0)
return -1;
pid_t pid = fork();
if (pid < 0) {
close(sync[0]);
close(sync[1]);
return -1;
}
if (pid == 0) {
close(sync[0]);
int wfd = open(fifo, O_WRONLY | O_CLOEXEC);
char ready = wfd >= 0 ? 1 : 0;
if (write(sync[1], &ready, 1) != 1)
_exit(1);
close(sync[1]);
if (wfd >= 0) {
usleep(delay_ms * 1000);
write_all_fd(wfd, first);
if (second) {
usleep(delay_ms * 1000);
write_all_fd(wfd, second);
}
close(wfd);
}
_exit(0);
}
close(sync[1]);
int hold = open(fifo, O_RDONLY | O_NONBLOCK | O_CLOEXEC);
char ready = 0;
ssize_t got = read(sync[0], &ready, 1);
close(sync[0]);
if (got != 1 || ready != 1) {
if (hold >= 0)
close(hold);
return -1;
}
*hold_out = hold;
return pid;
}
/* A FIFO writer that produces its data after a short delay must be read
* successfully (the regression: O_NONBLOCK made fgets fail with EAGAIN before
* the writer ran). */
static void test_credentials_read_secret_file_fifo_delayed_writer() {
char err[512];
char fifo[256];
snprintf(fifo, sizeof(fifo), "/tmp/fs_cred_pwfifo_slow_%d_%d", (int)getpid(), g_file_counter++);
unlink(fifo);
EXPECT_EQ_INT(mkfifo(fifo, 0600), 0);
int hold = -1;
pid_t writer =
fifo_writer_sleep_then_write(fifo, "alice:correct horse battery staple\n", 250, NULL, &hold);
EXPECT_TRUE(writer > 0);
char* user = NULL;
char* password = NULL;
EXPECT_EQ_INT(credentials_read_secret_file(fifo, &user, &password, err, sizeof(err)), 0);
EXPECT_EQ_STR(user, "alice");
EXPECT_EQ_STR(password, "correct horse battery staple");
free(user);
free(password);
int status = 0;
waitpid(writer, &status, 0);
if (hold >= 0)
close(hold);
unlink(fifo);
}
/* The same, but the line is written in two chunks separated by the delay: the
* reader must reassemble one line rather than parse the first chunk as an
* empty-password entry. */
static void test_credentials_read_secret_file_fifo_split_write() {
char err[512];
char fifo[256];
snprintf(fifo, sizeof(fifo), "/tmp/fs_cred_pwfifo_split_%d_%d", (int)getpid(), g_file_counter++);
unlink(fifo);
EXPECT_EQ_INT(mkfifo(fifo, 0600), 0);
int hold = -1;
pid_t writer =
fifo_writer_sleep_then_write(fifo, "alice:correct horse", 200, " battery staple\n", &hold);
EXPECT_TRUE(writer > 0);
char* user = NULL;
char* password = NULL;
EXPECT_EQ_INT(credentials_read_secret_file(fifo, &user, &password, err, sizeof(err)), 0);
EXPECT_EQ_STR(user, "alice");
EXPECT_EQ_STR(password, "correct horse battery staple");
free(user);
free(password);
int status = 0;
waitpid(writer, &status, 0);
if (hold >= 0)
close(hold);
unlink(fifo);
}
/* The server-side store loader (--password-file / --early-input) must also
* accept a FIFO whose writer appears after a delay. */
static void test_credentials_store_fifo_delayed_writer() {
char err[512];
char fifo[256];
snprintf(fifo, sizeof(fifo), "/tmp/fs_cred_storefifo_%d_%d", (int)getpid(), g_file_counter++);
unlink(fifo);
EXPECT_EQ_INT(mkfifo(fifo, 0600), 0);
int hold = -1;
pid_t writer = fifo_writer_sleep_then_write(fifo, KAT_STORE_LINE "\n", 250, NULL, &hold);
EXPECT_TRUE(writer > 0);
CredentialStore* store = credentials_load(fifo, NULL, err, sizeof(err));
EXPECT_NOT_NULL(store);
EXPECT_EQ_INT(credentials_store_size(store), 1);
credentials_free(store);
int status = 0;
waitpid(writer, &status, 0);
if (hold >= 0)
close(hold);
rm_temp(fifo);
}
/* fd-backed store paths (bash process substitution `<(...)`, i.e. /dev/fd/N and
* /proc/self/fd/N) are symlinks, so the ordinary O_NOFOLLOW rule would reject
* them with ELOOP. They name the calling process's own descriptors, so they
@@ -1184,6 +1339,9 @@ void test_credentials(void) {
test_credentials_read_secret_file_bad();
test_credentials_read_secret_file_symlink_rejected();
test_credentials_read_secret_file_fifo_no_hang();
test_credentials_read_secret_file_fifo_delayed_writer();
test_credentials_read_secret_file_fifo_split_write();
test_credentials_store_fifo_delayed_writer();
test_credentials_read_secret_file_fd_backed_accepted();
test_credentials_hash_file();
test_credentials_rejects_group_or_other_accessible();
+66 -38
View File
@@ -391,54 +391,82 @@ static void test_file_open_temp_dir_symlink_confinement() {
rmdir("test_tempdir_link_root/scratch");
rmdir(root);
rmdir(outside);
EXPECT_EQ_INT(mkdir(root, 0755), 0);
EXPECT_EQ_INT(mkdir(outside, 0755), 0);
EXPECT_NOT_NULL(realpath(root, root_abs));
EXPECT_NOT_NULL(realpath(outside, outside_abs));
int root_fd = open(root_abs, O_RDONLY | O_DIRECTORY | O_CLOEXEC);
EXPECT_TRUE(root_fd >= 0);
// cppcheck-suppress knownConditionTrueFalse
if (root_fd < 0) {
rmdir(root);
rmdir(outside);
return;
int mkdir_root_ret = mkdir(root, 0755);
int mkdir_outside_ret = mkdir(outside, 0755);
bool root_resolved = realpath(root, root_abs) != NULL;
bool outside_resolved = realpath(outside, outside_abs) != NULL;
int root_fd = root_resolved ? open(root_abs, O_RDONLY | O_DIRECTORY | O_CLOEXEC) : -1;
bool root_set = false;
bool scratch_ok = false;
int scratch_fd = -1;
bool escape_staged = false;
int escape_fd = 0;
bool inside_staged = false;
int inside_fd = -1;
char* scratch = NULL;
char* escape = NULL;
char* inside_link = NULL;
/* Only touch the global authorized root and the scratch fixtures once the
setup succeeded; the teardown below always runs regardless. */
if (root_fd >= 0 && outside_resolved) {
root_set = utils_set_authorized_root(root_fd, root_abs);
/* An existing in-root scratch dir opens normally. */
scratch = path_cat(root_abs, "scratch");
if (scratch && mkdir(scratch, 0755) == 0) {
scratch_ok = true;
scratch_fd = file_open_temp_dir(scratch);
if (scratch_fd >= 0)
close(scratch_fd);
}
/* A symlink whose target is outside the root is refused. */
escape = path_cat(root_abs, "escape");
if (escape && symlink(outside_abs, escape) == 0) {
escape_staged = true;
escape_fd = file_open_temp_dir(escape);
}
/* A symlink that stays inside the root is accepted (EXDEV fallback). */
inside_link = path_cat(root_abs, "inside_link");
if (inside_link && scratch && symlink(scratch, inside_link) == 0) {
inside_staged = true;
inside_fd = file_open_temp_dir(inside_link);
if (inside_fd >= 0)
close(inside_fd);
}
}
EXPECT_TRUE(utils_set_authorized_root(root_fd, root_abs));
/* An existing in-root scratch dir opens normally. */
char* scratch = path_cat(root_abs, "scratch");
EXPECT_NOT_NULL(scratch);
EXPECT_EQ_INT(mkdir(scratch, 0755), 0);
int scratch_fd = file_open_temp_dir(scratch);
EXPECT_TRUE(scratch_fd >= 0);
if (scratch_fd >= 0)
close(scratch_fd);
/* A symlink whose target is outside the root is refused. */
char* escape = path_cat(root_abs, "escape");
EXPECT_NOT_NULL(escape);
EXPECT_EQ_INT(symlink(outside_abs, escape), 0);
EXPECT_EQ_INT(file_open_temp_dir(escape), -1);
/* A symlink that stays inside the root is accepted (EXDEV fallback path). */
char* inside_link = path_cat(root_abs, "inside_link");
EXPECT_NOT_NULL(inside_link);
EXPECT_EQ_INT(symlink(scratch, inside_link), 0);
int link_fd = file_open_temp_dir(inside_link);
EXPECT_TRUE(link_fd >= 0);
if (link_fd >= 0)
close(link_fd);
/* Release the global authorized root and all fixtures BEFORE asserting:
EXPECT_* returns early on failure, so a failed assertion must not be able
to leave the process state poisoned or leak root_fd. */
utils_set_authorized_root(-1, NULL);
if (root_fd >= 0)
close(root_fd);
free(inside_link);
free(escape);
free(scratch);
utils_set_authorized_root(-1, NULL);
close(root_fd);
unlink("test_tempdir_link_root/escape");
unlink("test_tempdir_link_root/inside_link");
rmdir("test_tempdir_link_root/scratch");
rmdir(root);
rmdir(outside);
EXPECT_EQ_INT(mkdir_root_ret, 0);
EXPECT_EQ_INT(mkdir_outside_ret, 0);
EXPECT_TRUE(root_resolved);
EXPECT_TRUE(outside_resolved);
EXPECT_TRUE(root_fd >= 0);
EXPECT_TRUE(root_set);
EXPECT_TRUE(scratch_ok);
EXPECT_TRUE(scratch_fd >= 0);
EXPECT_TRUE(escape_staged);
EXPECT_EQ_INT(escape_fd, -1);
EXPECT_TRUE(inside_staged);
EXPECT_TRUE(inside_fd >= 0);
}
/* Issue #251: file_save_to_disk_full must distinguish receiver-side skips
+65 -1
View File
@@ -40,11 +40,17 @@ static void test_filter_list_rejects_xattr_modifier() {
}
static void test_filter_list_rejects_unsupported_modifiers() {
/* The merge-file modifiers e/n/w/- are invalid on every non-merge rule; a
token made up solely of modifier characters is a modifier run, so it must
be rejected rather than folded into the pattern. */
static const char* const rules[] = {
"-e foo", /* e: merge-only in rsync */
"-n foo", /* n: merge-only in rsync */
"-w foo", /* w: merge-only in rsync */
"merge,n /tmp/x", ".e /tmp/x", "dir-merge,e .rules", "exclude,w foo",
"-new", /* pure modifier letters (n/e/w) */
"-press", /* pure modifier letters (p/r/e/s) */
"exclude,w foo", "exclude,e foo", "exclude,n foo",
"hide,w foo", "protect,n foo", "risk,e foo",
};
for (size_t i = 0; i < sizeof(rules) / sizeof(rules[0]); i++) {
FilterRuleList* list = filter_rule_list_create();
@@ -57,6 +63,63 @@ static void test_filter_list_rejects_unsupported_modifiers() {
}
}
/* rsync accepts the merge-file modifiers e/n/w/- on merge and dir-merge rules.
* They must be consumed so they never leak into the merge filename. */
static void test_filter_list_accepts_merge_modifiers() {
char tmpl[] = "/tmp/fastsync_filter_mmod_XXXXXX";
EXPECT_TRUE(mkdtemp(tmpl) != NULL);
char path[512];
snprintf(path, sizeof(path), "%s/rules", tmpl);
FILE* fp = fopen(path, "w");
EXPECT_NOT_NULL(fp);
fputs("- *.tmp\n", fp);
fclose(fp);
/* merge with e/n/w/- consumes the modifiers and reads the right file. */
static const char* const fmts[] = {
"merge,e %s", "merge,n %s", "merge,w %s", "merge,- %s", ".e %s", ".- %s",
};
for (size_t i = 0; i < sizeof(fmts) / sizeof(fmts[0]); i++) {
FilterRuleList* list = filter_rule_list_create();
EXPECT_NOT_NULL(list);
char rule[600];
char err[256] = "";
snprintf(rule, sizeof(rule), fmts[i], path);
bool ok = filter_rule_list_parse_append(list, rule, NULL, NULL, err, sizeof(err));
if (!ok)
printf(" merge rule '%s' errored: %s\n", rule, err);
EXPECT_TRUE(ok);
EXPECT_EQ_INT(list->count, 1);
EXPECT_EQ_STR(list->items[0]->pattern, "*.tmp");
filter_rule_list_free(list);
}
/* dir-merge with e/n/w/- registers the basename without the modifiers. */
static const struct {
const char* rule;
const char* want;
} drules[] = {
{"dir-merge,e .rules", ".rules"}, {"dir-merge,n .rules", ".rules"},
{"dir-merge,w .rules", ".rules"}, {"dir-merge,- .rules", ".rules"},
{":e .rules", ".rules"}, {":- .rules", ".rules"},
};
for (size_t i = 0; i < sizeof(drules) / sizeof(drules[0]); i++) {
FilterRuleList* list = filter_rule_list_create();
EXPECT_NOT_NULL(list);
char err[256] = "";
bool ok = filter_rule_list_parse_append(list, drules[i].rule, NULL, NULL, err, sizeof(err));
if (!ok)
printf(" dir-merge rule '%s' errored: %s\n", drules[i].rule, err);
EXPECT_TRUE(ok);
EXPECT_EQ_INT(list->dir_merge_count, 1);
EXPECT_EQ_STR(list->dir_merge_names[0], drules[i].want);
filter_rule_list_free(list);
}
unlink(path);
rmdir(tmpl);
}
static void test_filter_list_accepts_supported_rules_and_modifiers() {
static const char* const rules[] = {
"- *.tmp", "+ /a.txt", "-s foo", "-r foo", "-p foo",
@@ -223,6 +286,7 @@ static void test_filter_rules_apply_supported_modifiers() {
void test_filter() {
test_filter_list_rejects_xattr_modifier();
test_filter_list_rejects_unsupported_modifiers();
test_filter_list_accepts_merge_modifiers();
test_filter_list_accepts_supported_rules_and_modifiers();
test_filter_list_merge_file_still_supported();
test_filter_rule_parse_rejects_unsupported_and_keeps_supported();
+1 -1
View File
@@ -741,7 +741,7 @@ static void test_protocol_throttle_bytes_unlimited() {
clock_gettime(CLOCK_MONOTONIC, &now);
long long elapsed_ms =
(now.tv_sec - start.tv_sec) * 1000LL + (now.tv_nsec - start.tv_nsec) / 1000000LL;
EXPECT_TRUE(elapsed_ms < 50);
EXPECT_TRUE(elapsed_ms < 2000);
protocol_session_unbind();
}