12 Commits
Author SHA1 Message Date
TapTap f64d252faf docs: recount RSYNC_COMPAT to 141/0/4/0/0/2 after Phase 7 Waves B-D; refresh README short-option note
CI / lint (push) Successful in 1m42s
CI / sanitizers (undefined) (push) Successful in 1m12s
CI / sanitizers (address) (push) Successful in 1m14s
CI / fuzz-build (push) Successful in 23s
CI / coverage (push) Successful in 47s
CI / valgrind (push) Successful in 39s
CI / build-and-test (push) Successful in 4m44s
2026-09-12 11:30:06 +02:00
TapTap 003a5e8f2f Merge feat/p7-times: Phase 7 Wave D (real dir/symlink time preservation making -O/-J meaningful; secluded-args -> Impossible/Divergence; PROTOCOL 2.17.0) 2026-09-12 11:22:23 +02:00
TapTap 9d97e1d3c0 Merge feat/p7-devices: Phase 7 Wave C (device/special statuses; --copy-devices --sendfile non-regular fallback) 2026-09-12 11:21:09 +02:00
TapTap 402e829fef docs(p7-times): re-review nits (chunked STATUS_DIR_TIMES comment; -M/--metadata -> --preserve; -m sink -> -j/--threads) 2026-09-12 11:20:40 +02:00
TapTap 9749c7878c fix(p7-times): dir-time entries only record (never create dirs); bound/chunk dir-time frames; harden list add; docs+tests
Review fixes for Phase 7 Wave D.

#1 (HIGH): STATUS_DIR_TIMES entries no longer create directories. A new
receiver-only File.dir_time_only flag marks dir-time entries; file_save_to_disk_full
short-circuits them as FILE_SAVE_SKIPPED before any device/dir branch, so the sink
still accumulates metadata into the deferred DirTimeList but creates nothing. Empty
source dirs stay untransferred (-a), -m/--prune-empty-dirs semantics are preserved,
and a pre-existing regular file/symlink at an empty-dir mirror path no longer aborts
the transfer. dir_time_list_apply fstatat()s the leaf (AT_SYMLINK_NOFOLLOW) and skips
absent/non-directory paths QUIETLY; only a real existing directory is stamped.
Also initialize File.dir_time_only in file_create() (uninitialised garbage otherwise).

#2 (MED): send_dir_times() chunks entries into repeated STATUS_DIR_TIMES frames of at
most MAX_MANIFEST_ENTRIES, matching the receiver's per-frame bound; the tautological
> INT_MAX check is gone.

#3 (LOW): dir_time_list_add() assigns each grown array right after its realloc (no
dangling) and advances capacity only after both succeed.

#4 (LOW): RSYNC_COMPAT.md -- STATUS_MKDIR carries metadata, dir times are transmitted
via STATUS_DIR_TIMES and applied at the end, empty dirs are still never created; -m
rationale, -O row and Wave D notes updated. Summary counts untouched.

#5 (LOW): integration tests for the three #1 scenarios (empty-dir non-creation under
-a and -a -m, collision non-abort), scanner test now covers empty-dir capture, and
test_file_restore_symlink_metadata asserts the positive apply path when supported.

PROTOCOL_VERSION stays 2.17.0; config-frame layout unchanged.
2026-09-12 11:06:05 +02:00
TapTap 8ca2b74e79 fix(p7-devices): sendfile falls back to buffered read for non-regular sources (--copy-devices no longer hangs); test/doc hardening 2026-09-12 11:00:26 +02:00
TapTap 04514c5b70 Merge feat/p7-output-fs: Phase 7 Wave B (sparse hole preservation, partial retention, fake-super replay, block-size; crtimes/stderr -> Impossible/Divergence) 2026-09-12 10:32:03 +02:00
TapTap a539d8b2ba docs(p7-output-fs): align fake-super comments with silent EPERM/EACCES skip (re-review nit) 2026-09-12 10:31:56 +02:00
TapTap f1a447bb4a feat(p7-times): real directory/symlink time preservation; -O/-J meaningful
Wave D of Phase 7. Make -O/--omit-dir-times and -J/--omit-link-times real by
preserving directory and symlink times, and mark --secluded-args as an explicit
Impossible/Divergence no-op.

Wire: PROTOCOL_VERSION 2.16.0 -> 2.17.0. Adds a terminal STATUS_DIR_TIMES frame
(int count + (wire path, metadata) pairs) sent after all file data and the
optional delete manifest. STATUS_MKDIR also carries metadata for --dirs entries.
Config-frame layout is unchanged.

Sender: the recursive scanner captures every traversed source directory (both
DirectoryScanner and the parallel scanner root + workers, appends mutex-guarded)
into a shared list; the single-threaded and -m paths transmit it last.

Receiver: a DirTimeList accumulates received directory metadata and applies it
with fd-relative no-follow utimensat only at the very end -- after all children,
after the commit-style --delete, and after --delay-updates publication -- in the
single-threaded success frame and in server.c after the -m threads join. -O skips
the application. Symlink metadata is applied at link creation with
utimensat/fchownat/fchmodat AT_SYMLINK_NOFOLLOW; -J suppresses only link times.
identity_apply_ownership_link shares the identity resolver with the fd path.

Docs: -O/-J rows -> Implemented; --secluded-args -> Impossible/Divergence;
--protocol accepted/rejected values and Phase-6/7 notes updated.

Tests: unit (scanner dir capture, DirTimeList apply, symlink metadata, protocol
version values) and integration (dir mtime round-trip + -O, symlink mtime
round-trip + -J, independent suppression), parameterized over single/multithread.
2026-09-12 10:31:20 +02:00
TapTap 227d001092 feat(p7-devices): finalize device/special-file statuses (devices/copy/write -> implemented, specials -> Impossible/Divergence for sockets) + coverage 2026-09-12 10:10:21 +02:00
TapTap f2ba8211ce fix(p7-output-fs): address c-review (fake-super mode sanitization HIGH; sparse/preallocate precedence; partial no_replace guard; stronger tests)
- fake_super_restore_fd now sanitizes mode like metadata_mode (never grants
  S_IWGRP|S_IWOTH; 0666 -> 0644), fixing a privilege regression
- --sparse takes precedence over --preallocate (skip posix_fallocate when
  sparse) so holes are not re-allocated; docs corrected
- --partial retention disabled under --no_replace (ignore/existing) and only
  marks write_attempted after the write begins (no empty-temp retention)
- accept --block-size=SIZE / --delta-block=SIZE inline forms; neutral messages
- fake-super EPERM/EACCES skipped silently (docs aligned); EINVAL still logged
- sparse unit test now memcmp's the full buffer; TestBlockSize integration keeps
  the destination basis so delta is genuinely exercised
- unit 37/37, cppcheck 0, clang-format 0
2026-09-12 09:55:40 +02:00
TapTap 47de05d215 feat(p7-output-fs): sparse hole preservation (-S), partial retention (-P), fake-super replay; block-size verified; crtimes/stderr -> Impossible/Divergence (Wave B)
- write_all_sparse: skips all-zero runs >= 4096 bytes via lseek(SEEK_CUR) and
  ftruncates the final size, wired into the atomic temp+rename and --inplace
  paths with no wire change (full image already in memory).
- --partial retention: on a save failure after the temp held data, rename the
  already-written temp to the destination path (best-effort; falls through to
  unlink; never retains when --partial is off) so --append/--append-verify can
  resume; tested by forcing futimens EINVAL with an out-of-range nsec.
- --block-size aliases --delta-block; verified config->delta_block_size is
  honored by the delta engine end-to-end (unit + integration tests).
- fake_super_restore_fd: parses and re-applies user.fastsync.stat fd-relative
  (fchown best-effort/non-root skipped, fchmod, futimens); a save under
  --fake-super now both records and re-applies.
- -N/--crtimes and --stderr=client promoted to a new 'Impossible/Divergence'
  status bucket (Summary: 136/2/4/3/2 = 147).
- cppcheck/clang-format clean; unit 37/37; integration 407 passed.
2026-09-11 15:58:20 +02:00
33 changed files with 1757 additions and 153 deletions

No files matched your search

+27 -17
View File
@@ -63,17 +63,26 @@ replacement for every rsync feature or protocol mode.
- Archive mode does not yet provide all of rsync's `-rlptgoD` behavior.
- Symlink transfer is incomplete; link targets are not yet recreated in all
modes.
- Owner/group, ACL, xattr, hard-link, device, and special-file handling is
incomplete or unavailable.
- Sparse-file handling does not yet preserve all holes correctly.
- `--partial`, `--partial-dir`, `-P`, `--append`, and `--append-verify` are not
yet full rsync-style resumable transfers. Interrupted files are not retained
for resumption.
- Owner/group, ACL, xattr, and hard-link handling is incomplete or
unavailable.
- Device and special-file preservation is implemented with documented
divergences: recreated device nodes require `CAP_MKNOD` on the receiver (a
non-root receiver skips the entry), and sockets cannot be recreated (FIFOs
are).
- Sparse-file hole preservation (`-S`, `--sparse`) is implemented receiver-side:
long all-zero runs are written as holes (no wire change; the full file image
is already in memory).
- `--partial`, `--partial-dir`, `-P`, `--append`, and `--append-verify` keep
the write atomic (temp + rename). With `--partial`, a failed/interrupted write
now retains the already-written temp at the destination path (best-effort) so
a later `--append`/`--append-verify` run can resume it.
- `--dirs` is not implemented. Its compatibility aliases `--old-dirs` and
`--old-d` are recognized but rejected explicitly rather than silently using
FastSync's recursive directory behavior.
- Several rsync short options currently have FastSync-specific meanings. Do
not assume every short option is interchangeable yet.
- Short-option names are now rsync-parity (Phase 7 Wave A): FastSync's former
collisions were renamed (`-j`/`--threads`, `--preserve`, `--sendfile`,
`--chunk-serialization`, `--timeout`, `--ssh-port`), so `-m`, `-M`, `-f`,
`-s`, `-T`, `-p`, `-c`, `-a`, and `-z` follow rsync. See `RSYNC_COMPAT.md`.
The detailed flag matrix is maintained in
[`RSYNC_COMPAT.md`](RSYNC_COMPAT.md). It distinguishes implemented,
@@ -103,7 +112,7 @@ partial, alternate, and planned behavior.
| `-v, --verbose` | Enable debug logging |
| `-q, --quiet` | Suppress non-error output |
| `--progress` | Show real-time transfer speed |
| `-P` | Enables partial-transfer mode and progress output (partial retention is incomplete) |
| `-P` | Enables partial-transfer mode + progress output; interrupted writes retain the already-written temp for resumption |
| `--delete` | Delete files on receiver not present in source (default timing: delete-after, i.e. only after the whole transfer succeeded) |
| `--delete-before` | Delete extras before the transfer starts (implies `--delete`) |
| `--delete-during`, `--del` | Delete extras once the keep-set is known, before data is applied (implies `--delete`) |
@@ -363,7 +372,7 @@ features without changing the meaning of ordinary compatibility options.
| `--chunk-serialization` | Enable FastSync chunk serialization (long form only; `-s` is rsync's `--secluded-args`). |
| `--sendfile` | Use TCP `sendfile()` zero-copy transfer. Incompatible with compression and chunk serialization. Long form only. |
| `--delta` | Use FastSync-native block delta transfer. Requires `--incremental`. |
| `--delta-block <bytes>` | Set the FastSync delta block size. |
| `--delta-block <bytes>` | Set the FastSync delta block size (`--block-size` is an alias). |
| `--delta-max <bytes>` | Limit files eligible for FastSync delta transfer. |
| `--server-host <host>` | Select the TCP server host. |
| `--server-port <port>` | Select the TCP server port. |
@@ -410,12 +419,13 @@ remote SSH argv is already built injection-safe.
zero means unlimited.| | `--incremental` | Skip files matching destination size and mtime.|
| `--checksum` | Include xxHash64 content checks in incremental comparisons.| | `--backup` |
Back up overwritten files.| | `--backup - dir<dir>` | Store backups under a separate directory.|
| `--suffix<suffix>` | Set the backup filename suffix.| | `--partial` |
Select partial - transfer handling.With `--partial - dir`,
completed files are written there;
resumable transfers are not implemented.| | `--partial - dir<dir>` |
Set a relative partial - transfer directory below the server destination root;
use with `--partial`. |
| `--suffix<suffix>` | Set the backup filename suffix.| | `--partial` |
Select partial - transfer handling. On failed/interrupted writes the
already-written temp file is retained (best-effort) for resumption.|
With `--partial --partial-dir <dir>`, completed files are written under the
partial directory and installed atomically. | | `--partial - dir<dir>` |
Set a relative partial - transfer directory below the server destination root.
Use with `--partial`. |
| `--inplace` | Write directly to the destination instead of using a temporary file. |
### Metadata and links
@@ -428,7 +438,7 @@ link-target transfer remains incomplete. |
| `--copy-links` | Copy symlink referents. |
| `--safe-links` | Skip symlinks that point outside the transfer tree. |
| `--copy-unsafe-links` | Copy unsafe symlink referents. |
| `-S`, `--sparse` | Request sparse-file handling; full hole preservation is planned. |
| `-S`, `--sparse` | Sparse-file handling: receiver preserves holes (zero runs are written as holes; no wire change). |
### Output and logging
+45 -36
View File
@@ -6,10 +6,11 @@ This document maps rsync's full feature set to FastSync's current implementation
| Status | Count | Description |
|--------|-------|-------------|
| ✅ Implemented | 132 | Feature works end-to-end |
| ✅ Implemented | 141 | Feature works end-to-end |
| 🔀 Alt Arg | 0 | Functionality exists but under different flag/semantics |
| ⚠️ Partial | 10 | Flag parsed/stored but behavior incomplete |
| 🔄 Compatibility No-op | 3 | Flag is accepted for CLI compatibility but has no effect |
| ⛔ Impossible/Divergence | 4 | Flag is a documented divergence or cannot be implemented on any portable filesystem call |
| ⚠️ Partial | 0 | Flag parsed/stored but behavior incomplete |
| 🔄 Compatibility No-op | 0 | Flag is accepted for CLI compatibility but has no effect |
| ❌ Not Implemented | 2 | Flag not recognized or no behavior |
| **Total** | **147** | |
@@ -26,7 +27,7 @@ This document maps rsync's full feature set to FastSync's current implementation
| `-V`, `--version` | Print version | ✅ Implemented | |
| `--info=FLAGS` | Fine-grained info verbosity | ✅ Implemented | Supports `copy`, `misc`, `skip`, `stats`, `all`, and `none`; explicit flags override `--verbose`, and `none` suppresses info output; unsupported names are rejected |
| `--debug=FLAGS` | Fine-grained debug verbosity | ✅ Implemented | `io`, `proto`, `pack`, and `util` are supported; `--debug=help` lists flags; other rsync categories are rejected |
| `--stderr=MODE` | Change stderr output mode | ⚠️ Partial | `errors` (default) and `all` are supported; `client` is rejected because FastSync has no rsync message channel |
| `--stderr=MODE` | Change stderr output mode | ⛔ Impossible/Divergence | `errors` (default) and `all` are supported; `client` is rejected with a clear error (`--stderr=client is not supported`) because FastSync has no rsync client-message channel — the rejection itself is the documented behavior (Phase 7 Wave B decision). The modes that exist work; the missing rsync channel cannot be emulated without a wire change |
| `--no-motd` | Suppress daemon MOTD | ✅ Implemented | Client-only display switch (Wave C): the daemon still sends the configured `motd file` on a `host::module/path` connection; the client reads and discards the frame without showing it. Without the flag the MOTD is printed to stdout after the config/auth handshake and escaped so control bytes cannot inject terminal sequences |
| `--exclude=PATTERN` | Exclude files matching pattern | ✅ Implemented | Glob matching in scanner |
| `--include=PATTERN` | Include files matching pattern | ✅ Implemented | Glob matching in scanner |
@@ -40,7 +41,7 @@ This document maps rsync's full feature set to FastSync's current implementation
| `-h`, `--human-readable` | Human-readable numbers | ✅ Implemented | Formats transfer byte sizes using binary units |
| `-i`, `--itemize-changes` | Per-file change summary | ✅ Implemented | Prints rsync-style `>f+++++++++` lines to stdout only for files actually sent (also under `-j`/`--threads`); unchanged files print nothing, matching single-`-i` behavior |
| `--progress` | Show progress | ✅ Implemented | Progress callback in sender |
| `-P` | Same as --partial --progress | ⚠️ Partial | Parses and enables progress, but interrupted files are not retained for resumable transfers |
| `-P` | Same as --partial --progress | ✅ Implemented | Phase 7 Wave B: `-P` parses to `--partial` + `--progress`. On a failed/interrupted write the receiver now retains the already-written temp file at the destination path (best-effort rename instead of unlink when configured), so a later `--append`/`--append-verify` run can resume it; `--partial-dir` still stages completed files under the confined partial dir and installs them atomically. The retention never runs when `--partial` is off, when no data was actually written, or under `--ignore-existing`/`--existing` (the destination is not ours to overwrite), and it only ever renames the already-written temp (never a corrupt blend; a failed rename falls back to the normal unlink). See the `-S`/`--sparse` interplay note (a retained sparse temp has full logical size) |
| `--out-format=FORMAT` | Custom output format | ✅ Implemented | Per-transfer template on stdout; tokens `%f` `%n` `%l` `%b` `%M` `%%` (`%b` is the source length, always `== %l`; post-compression/delta wire bytes are not counted); unknown escapes preserved |
| `--log-file=FILE` | Log to file | ✅ Implemented | `log_file` config field |
| `--log-file-format=FMT` | Log format | ✅ Implemented | Requires `--log-file`; writes one template line per transferred file using the same token set as `--out-format` (including `%b` `==` source length) |
@@ -74,7 +75,7 @@ This document maps rsync's full feature set to FastSync's current implementation
| `-r`, `--recursive` | Recurse into directories | ✅ Implemented | Default behavior |
| `-R`, `--relative` | Use relative path names | ✅ Implemented | Meaningful together with `--files-from` (FastSync's default full-tree scan always mirrors the full source argument path below the destination root, so -R does not change it). With `-R` + `--files-from` each listed entry is transmitted under its bare relative destination path: an entry `sub/x.txt` lands at `<dest>/sub/x.txt` (its leading components preserved) instead of under the `<dest>/<full source path>` mirror. Only the path sent on the wire changes; the client still reads the absolute source path, and the delete manifest derives from the sent (relative) paths so `--delete` and `--remove-source-files` stay consistent in both layouts. Works single-threaded and under `-j`/`--threads` (including chunk serialization) |
| `--no-implied-dirs` | Don't send implied dirs with -R | ✅ Implemented | Client-side, meaningful only with `-R` + `--files-from`. rsync would normally create the ancestor directories implied by a listed file so it can be written; with `--no-implied-dirs` a listed file whose parent directory is not itself (or via an ancestor) explicitly listed cannot be placed, and FastSync fails the whole run up front with a clear error (`--no-implied-dirs: cannot place file '...': parent directory '...' is not explicitly listed`). Listing the directory (or an ancestor of it, or the whole tree `.`) permits the file. In every other mode the option has no effect. FastSync has no per-entry skip channel, so the rsync "omit the file" case is surfaced as a hard pre-transfer error |
| `-d`, `--dirs`, `--old-dirs`, `--old-d` | Transfer dirs without recursing | ✅ Implemented | `-d <dir>` transmits an explicit directory entry for the source-root directory, so the destination mirror is created empty and nothing is descended into. With `--files-from` exactly the listed items are transferred: a listed directory is created empty (no descent) and a listed file is transferred with its content; the dest layout follows the same -R rules as plain files. A new wire frame (`STATUS_MKDIR`) carries each directory entry (path only); the receiver creates it with the same confined mkdir-parent semantics as regular writes, in single-threaded and `-j`/`--threads` receivers (chunk serialization carries a per-entry type marker). Directory entries appear in the delete manifest so `--delete` prunes correctly. FastSync divergences: directory mtimes/modes are not transmitted, filter/`--exclude` rules are not re-applied to the listed dirs mode (there is no descent during which they would apply), and `-d` never creates the intermediate directories between the destination root and a listed file beyond the usual on-demand parent creation. Under `--delay-updates` only regular files are staged: directory entries are created immediately, so a delayed run that fails part way can leave the already-created empty directories behind (matching rsync, which also creates directories as it processes the file list and only delays regular-file data) |
| `-d`, `--dirs`, `--old-dirs`, `--old-d` | Transfer dirs without recursing | ✅ Implemented | `-d <dir>` transmits an explicit directory entry for the source-root directory, so the destination mirror is created empty and nothing is descended into. With `--files-from` exactly the listed items are transferred: a listed directory is created empty (no descent) and a listed file is transferred with its content; the dest layout follows the same -R rules as plain files. A new wire frame (`STATUS_MKDIR`) carries each directory entry — the path and, when `--preserve`/`-a` (metadata mode) is negotiated, the directory's metadata; the receiver creates it with the same confined mkdir-parent semantics as regular writes, in single-threaded and `-j`/`--threads` receivers (chunk serialization carries a per-entry type marker). Directory entries appear in the delete manifest so `--delete` prunes correctly. Directory TIMES are transmitted (the `STATUS_DIR_TIMES` frame carries every traversed source directory's captured times, including `--dirs` entries) and applied by the receiver at the END of the transfer, after all children and the delete/publication phases, so a later child write cannot clobber a directory's mtime (`-O`/`--omit-dir-times` skips this application). FastSync divergences: directory modes/ownership are still not applied (only times are), and empty directories are still never created (a `STATUS_DIR_TIMES` entry is record-only), filter/`--exclude` rules are not re-applied to the listed dirs mode (there is no descent during which they would apply), and `-d` never creates the intermediate directories between the destination root and a listed file beyond the usual on-demand parent creation. Under `--delay-updates` only regular files are staged: directory entries are created immediately, so a delayed run that fails part way can leave the already-created empty directories behind (matching rsync, which also creates directories as it processes the file list and only delays regular-file data) |
| `--mkpath` | Create missing path components | ✅ Implemented | Wire option (client → server). At connection start the server creates the client's destination root directory (and any missing leading components below its own authorized root) when `--mkpath` is set, failing the connection cleanly if it cannot. Without `--mkpath` a destination root that does not exist yet is rejected up front (rsync semantics), so the flag is the only way to transfer into a not-yet-created destination directory. Creation is confined by the same secure mkdir walk as file writes (`O_NOFOLLOW`, no `..`) |
## 5. Transfer Modifications
@@ -86,7 +87,7 @@ This document maps rsync's full feature set to FastSync's current implementation
| `--append` | Append data to shorter files | ✅ Implemented | Tail-only resume. When an existing destination file is SHORTER than the source, the receiver negotiates a resume offset with the sender and only the tail is transferred; the receiver rebuilds the full file (retained prefix + tail) and installs it through the normal atomic store path, so the result is byte-identical to the source whenever the retained prefix matches. Plain `--append` does NOT content-verify that prefix (rsync parity): a destination whose prefix differs from the source is resumed anyway, so the result (wrong prefix + correct tail) is NOT byte-identical and the file is effectively left corrupt — the documented rsync-parity risk (use `--append-verify` when the prefix cannot be trusted). Non-content attributes (permissions/ownership/mtime, via `-M`) are still applied. Requires the per-file `STATUS_CHECK` handshake, so it implies `--incremental`; it takes precedence over block delta for a growing file and falls back to delta/full when the destination is not shorter. Incompatible with `-s` (chunk serialization) and `--whole-file` (both rejected up front so the mode never silently degrades to a full transfer). Combines with `--inplace`, `--partial`/`--partial-dir`, and `--delay-updates` (the reconstructed full file flows through those paths unchanged). Divergence: rsync appends in place; FastSync reconstructs and atomically installs, so an interrupted or failed resume never leaves a half-written file at the destination (no corruption window), and `--append` is thus safe to use with the normal atomic path — not only with in-place writes |
| `--append-verify` | Append with old-data checksum | ✅ Implemented | Like `--append`, but the retained prefix IS verified before resuming: the sender transmits the source prefix checksum and the receiver compares it to the xxHash64 of the retained destination prefix; on a match only the tail is transferred, on a MISMATCH the run falls back to a clean full transfer so the result is always a byte-identical source copy (never a corrupt prefix+tail blend). Wire/protocol: the append handshake adds `STATUS_APPEND` / `STATUS_APPEND_SIG` / `STATUS_APPEND_OK` / `STATUS_APPEND_DATA` frames and `PROTOCOL_VERSION` was bumped **2.9.0 → 2.10.0** (peers must match, and both must be 2.10.0 or the run fails the version check). Same implications/incompatibilities as `--append`; when both spellings are given `--append-verify` wins (the safer semantics). See the Phase-3 append notes below |
| `-W`, `--whole-file` | Copy whole file (no delta) | ✅ Implemented | `whole_file` config field. Forces a full (whole-file) copy, disabling the block-level delta machinery: the sender only sends `STATUS_NEXT` + full data (client_send.c) and the receiver never requests a delta signature/reconstruction — the receiver's `try_delta = use_delta && !whole_file && ...` short-circuits. `whole_file` crosses the wire folded into `use_delta` (the wire carries `use_delta && !whole_file`), so no separate field/bump is needed. Delta is opt-in (`--delta` needs `--incremental`); `-W` additionally makes `--fuzzy` inert (no similar-file delta basis). `--append`/`--append-verify` are incompatible with `-W` and rejected up front (both sides). See the delta/append notes below |
| `--block-size=SIZE` | Force checksum block-size | ⚠️ Partial | Parsed as `--delta-block`; controls delta transfer block size |
| `--block-size=SIZE` | Force checksum block-size | ✅ Implemented | Phase 7 Wave B: `--block-size` is an alias for `--delta-block`; both set `config->delta_block_size` (default `DELTA_BLOCK_SIZE_DEFAULT`, bounds `DELTA_BLOCK_SIZE_MIN..MAX`, out-of-range values are rejected with the default kept). The value is genuinely honored by the delta engine end-to-end: `delta_signature_create_seeded(old, size, config->delta_block_size, seed)` on the sender and receiver, `delta_apply(old, ...)` with the same size, so a non-default block size changes the block count of every signature the harnesses exchange (verified by unit + integration tests) |
## 6. Destination Handling
@@ -112,7 +113,7 @@ This document maps rsync's full feature set to FastSync's current implementation
| `--max-delete=NUM` | Max files to delete | ✅ Implemented | `max_delete` config field (default -1 = no client limit; 0 = delete nothing). NUM bounds a `--delete` run with rsync's all-or-nothing semantics: the receiver rehearses the deletion first and, if the destination holds more than NUM extras, deletes NOTHING and fails the transfer with a distinct `--max-delete` error. A run at or below NUM deletes exactly the extras. NUM only applies together with `--delete` (it is inert otherwise, matching rsync). The hard server bound `MAX_SERVER_DELETE_COUNT` (100000) still caps the walk; a NUM above it never raises that cap, and exceeding the server bound is its own all-or-nothing error. Directories count toward the limit (each removed empty directory is one deletion), like rsync |
| `--ignore-errors` | Delete even with I/O errors | ✅ Implemented | Sender-side, client-only config field. rsync suppresses `--delete` when the transfer had I/O errors; FastSync's equivalent is a source-scan I/O error (an unreadable directory, e.g. EACCES): by default the scan aborts the run so no deletion happens. With `--ignore-errors` the scan continues past the unreadable directory, the readable tree is transferred and the deletion still runs (the mirror of the unreadable directory is treated as an extra). The run still exits non-zero (the error is reported, matching rsync's error status). Divergence: without the flag FastSync aborts the whole run on the scan error, whereas rsync transfers the rest of the tree and merely skips the deletion; both leave the deletion undone |
| `--force` | Force deletion of non-empty dirs | ✅ Implemented | `force_delete` receiver config field (crosses the wire). rsync's `--force` lets an incoming non-directory replace a destination directory; FastSync implements exactly that: when a regular file is written to a path that is currently a (possibly non-empty) destination directory, `--force` removes that directory tree first — confined to the receive root and symlink-safe (O_NOFOLLOW fd walk, symlinks removed by name, never followed) — so the atomic install can place the file. Without `--force` such a write fails and the run aborts. Divergence: `--force` acts on the immediate-install path only; under `--delay-updates` a blocking directory is not cleared (publication renames over regular files) |
| `-m`, `--prune-empty-dirs` | Prune empty dir chains | ✅ Implemented | `-m`/`--prune-empty-dirs` (Phase 7 Wave A freed the rsync short `-m`; FastSync multithreading is now `-j`/`--threads`). FastSync's recursive transfer never emits directory entries, so empty directories are inherently never transferred (which is rsync's `-m` behavior) and truly-empty destination directory chains are removed by `--delete` regardless of this flag. The flag's additional real effect is on the `--dirs` explicit directory-entry generator: a plain `-d <empty-dir>` run omits the empty source directory's entry, so nothing is created at the destination (no `STATUS_MKDIR`, no `-i`/`--out-format` change line, and an existing empty mirror becomes an extra that `--delete` prunes). Explicitly `--files-from`-listed directories always pass through (documented `--files-from` behavior). A directory that still holds an excluded-but-protected file survives, matching the `--delete-excluded` default |
| `-m`, `--prune-empty-dirs` | Prune empty dir chains | ✅ Implemented | `-m`/`--prune-empty-dirs` (Phase 7 Wave A freed the rsync short `-m`; FastSync multithreading is now `-j`/`--threads`). FastSync's recursive transfer records directory times but never CREATES an empty directory (a `STATUS_DIR_TIMES` entry is record-only, and `--dirs` empty entries are pruned by this flag), so empty directories are inherently never transferred (which is rsync's `-m` behavior) and truly-empty destination directory chains are removed by `--delete` regardless of this flag. The flag's additional real effect is on the `--dirs` explicit directory-entry generator: a plain `-d <empty-dir>` run omits the empty source directory's entry, so nothing is created at the destination (no `STATUS_MKDIR`, no `-i`/`--out-format` change line, and an existing empty mirror becomes an extra that `--delete` prunes). Explicitly `--files-from`-listed directories always pass through (documented `--files-from` behavior). A directory that still holds an excluded-but-protected file survives, matching the `--delete-excluded` default |
**Deletion-timing implementation notes (Phase 3):** the delete flags above are
real. Two new config booleans (`delete_during`, `delete_delay`) join the already
@@ -248,16 +249,16 @@ why plain `--append` works on the normal atomic path, not only with `--inplace`.
| `-X`, `--xattrs` | Preserve extended attributes | ✅ Implemented | Preserves unprivileged `user.*` extended attributes (Linux `listxattr`/`getxattr` on capture, `fsetxattr` on the written destination fd). Both capture (sender) and application (receiver) are restricted to the `user.*` namespace and the two POSIX ACL xattrs, so a client can **never** force a `security.*`/`trusted.*`/privileged attribute onto the destination; the receiver independently re-validates every incoming name against this whitelist and rejects anything else. Payloads are bounded (per-name ≤255B, per-value ≤1MiB, per-file count ≤256 total bytes ≤4MiB) on both ends, and an oversized/malformed frame is a clean protocol rejection (no OOM). Applied fd-relative to the exact written file. Implies metadata transmission. Incompatible with `-s` (chunk serialization), rejected up front (see the notes); a `--link-dest`/`-H` hard-link copy fallback re-applies the attributes so they are not dropped when a link is refused |
| `-H`, `--hard-links` | Preserve hard links | ✅ Implemented | Files on the source that share an inode (`st_dev`+`st_ino`, e.g. a `cp -al` tree) are re-created as hard links to one another on the destination, so duplicate links stay deduplicated and only the first member's data is sent (later members are transmitted as payload-less `STATUS_HARDLINK` frames). The receiver links each sibling to the first member's installed file with an atomic link + rename; on `link()` failure it falls back to a byte-identical local copy of the first member, never a partial/corrupt file. Requires the sequential scan for ordering (the first member is always emitted and installed before any sibling is linked). Works single-threaded and under `-j`/`--threads`, `--inplace`, `--delay-updates` (links staged and published by rename) and `--partial`. Crosses the wire (`preserve_hard_links` bool; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0**, peers must match). Incompatible with `-s` (chunk serialization) and `--append`/`--append-verify`, rejected up front with a distinct error. See the Phase-4 hard-links notes below |
| `-D` | Same as --devices --specials | ✅ Implemented | Implies `--devices --specials`. `-D` was unassigned in FastSync (verified: no collision), so it is free to imply both device-node and special-file preservation. See the `--devices`/`--specials` rows and the Phase-4 devices notes below |
| `--devices` | Preserve device files | ⚠️ Partial | Recreates char/block device nodes on the destination via `mknod` instead of transferring content. Type + rdev are validated strictly (S_IFMT from the transmitted mode; major/minor range-checked, non-negative), and creation is **privilege-gated**: `mknod` needs `CAP_MKNOD`, so a non-root receiver (CI runs via setpriv as non-root) logs a warning and **skips the device entry safely** — the whole transfer never aborts just because the node could not be made. The node is created fd-relative below the receive root (`mknodat` on the confined secure parent), so it can never be placed outside the authorized root, never follows a symlink, and never replaces an existing directory. Only a char/block mode is honored. Crosses the wire (a new `STATUS_SPECIAL` frame carries the path + metadata mode + rdev; `PROTOCOL_VERSION` bumped **2.12.0 → 2.13.0**). Divergence: per-entry skip (not a hard error) when the receiver lacks `CAP_MKNOD`, documented in the Phase-4 devices notes |
| `--specials` | Preserve special files | ⚠️ Partial | Recreates **FIFOs** on the destination via `mkfifo` (unprivileged, so this is a real, assertable behavior under CI). Sockets cannot be recreated by any standard filesystem call and are skipped with an explicit note (best-effort / unsupported, matching the plan). FIFO creation is privileged-gated only in the sense of graceful skip on any permission failure. Node creation is confined below the receive root (`mkfifoat` on the secure fd-relative parent; no `..`, no symlink follow). Crosses the wire like `--devices` (the `STATUS_SPECIAL` frame; `PROTOCOL_VERSION` bumped **2.12.0 → 2.13.0**). See the Phase-4 devices notes |
| `--copy-devices` | Copy device contents as file | ⚠️ Partial | Copy a device's CONTENT into an ordinary regular file on the destination instead of recreating the node — non-privileged and safe. FastSync scans a device/FIFO as a regular file: its reported size (`st_size`, typically 0 for char devices and FIFOs) is copied, so a FIFO or a non-readable device becomes an empty (or size-bounded) regular file without ever blocking or reading unbounded pseudo-device streams. The run always succeeds and never crashes on such input. **Deliberate, safe divergence from rsync's dd-like unbounded device read.** See the Phase-4 devices notes |
| `--write-devices` | Write to devices as files | ⚠️ Partial | Write the received data directly into an **existing** device node on the destination instead of creating a regular file. Restricted and best-effort: the destination must already exist and be a char/block device (opened only under the confined receive root, with `O_NOFOLLOW` + `O_NONBLOCK`); a missing, symlinked, FIFO-with-no-reader (`ENXIO`), non-device destination, or any write failure is **skipped with a warning** rather than allowed, so a run can never clobber the system, never blocks on a special-file target, and never aborts on an unusable target. See the Phase-4 devices notes |
| `--devices` | Preserve device files | ✅ Implemented | Recreates char/block device nodes on the destination via `mknod` instead of transferring content. Type + rdev are validated strictly (S_IFMT from the transmitted mode; major/minor range-checked, non-negative), and creation is **privilege-gated**: `mknod` needs `CAP_MKNOD`, so a non-root receiver (CI runs via setpriv as non-root) logs a warning and **skips the device entry safely** — the whole transfer never aborts just because the node could not be made. The node is created fd-relative below the receive root (`mknodat` on the confined secure parent), so it can never be placed outside the authorized root, never follows a symlink, and never replaces an existing directory. Only a char/block mode is honored. Crosses the wire (a new `STATUS_SPECIAL` frame carries the path + metadata mode + rdev; `PROTOCOL_VERSION` bumped **2.12.0 → 2.13.0**). Divergence: per-entry skip (not a hard error) when the receiver lacks `CAP_MKNOD`, documented in the Phase-4 devices notes |
| `--specials` | Preserve special files | ⛔ Impossible/Divergence | **FIFO recreation works**: FIFOs are recreated on the destination via `mkfifo` (unprivileged, so this is a real, assertable behavior under CI). **Only socket recreation is impossible**: a socket entry can be created only by `bind(2)` on a live socket, not by any filesystem call, so a source socket is skipped with an explicit note. That one unsupported node kind is why the flag is classified Impossible/Divergence even though FIFO recreation itself works; its normal path is otherwise complete. FIFO creation is privileged-gated only in the sense of graceful skip on any permission failure. Node creation is confined below the receive root (`mkfifoat` on the secure fd-relative parent; no `..`, no symlink follow). Crosses the wire like `--devices` (the `STATUS_SPECIAL` frame; `PROTOCOL_VERSION` bumped **2.12.0 → 2.13.0**). See the Phase-4 devices notes |
| `--copy-devices` | Copy device contents as file | ✅ Implemented | Copy a device's CONTENT into an ordinary regular file on the destination instead of recreating the node — non-privileged and safe. FastSync scans a device/FIFO as a regular file: its reported size (`st_size`, typically 0 for char devices and FIFOs) is copied, so a FIFO or a non-readable device becomes an empty (or size-bounded) regular file. The default data path is size-bounded and never blocks (it sends exactly `st_size` bytes, never an unbounded pseudo-device stream); with `--sendfile`, a non-regular source (FIFO/device) is detected from its `stat` mode and falls back to that same buffered read, so `--copy-devices --sendfile` cannot hang either. The run always succeeds and never crashes on such input. **Deliberate, safe divergence from rsync's dd-like unbounded device read.** See the Phase-4 devices notes |
| `--write-devices` | Write to devices as files | ✅ Implemented | Write the received data directly into an **existing** device node on the destination instead of creating a regular file. Restricted and best-effort: the destination must already exist and be a char/block device (opened only under the confined receive root, with `O_NOFOLLOW` + `O_NONBLOCK`); a missing, symlinked, FIFO-with-no-reader (`ENXIO`), non-device destination, or any write failure is **skipped with a warning** rather than allowed, so a run can never clobber the system, never blocks on a special-file target, and never aborts on an unusable target. See the Phase-4 devices notes |
| `-U`, `--atimes` | Preserve access times | ✅ Implemented | Captures the source access time (from the scanner's pre-read stat, so it is not clobbered by reading the file for transfer) and transmits it over the wire; the receiver restores it together with the mtime via `futimens`/`utimensat`. Implies metadata transmission (the times travel inside the `-M` metadata payload), but does not enable ownership application (that stays opt-in via the identity flags). Wire: new `atime` fields on the metadata frame + a `preserve_atimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** |
| `-N`, `--crtimes` | Preserve create times | ⚠️ Partial | Captures the source birth time via `statx(STATX_BTIME)` on Linux and transmits it (recorded as a wire field), but there is **no portable way to set a birth time** (`utimensat` can only set atime/mtime), so the receiver explicitly does NOT apply it: it logs a debug note and continues — never failing the transfer and never pretending it worked. On platforms without `statx` it parses as a documented no-op (flag accepted; nothing is captured). Implies metadata transmission. Wire: new `crtime` fields + a `preserve_crtimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** (see the Phase-4 metadata-time notes) |
| `-O`, `--omit-dir-times` | Omit dirs from --times | 🔄 Compatibility No-op | Accepted and parsed for CLI compatibility, and the config boolean crosses the wire, but it has **no effect**: FastSync never preserves directory mtimes in the first place (directories are created via `mkdir` with no metadata, a documented divergence under `-d`/recursive), so there is nothing for an "omit" to suppress. It never breaks a normal run |
| `-J`, `--omit-link-times` | Omit symlinks from --times | 🔄 Compatibility No-op | Accepted and parsed for CLI compatibility, and the config boolean crosses the wire, but it has **no effect**: FastSync never sets symlink times (`-l`/`--links` copies symlinks as symlinks but the receiver does not apply timestamps/owner to symlink entries), so there is nothing for an "omit" to suppress. It never breaks a normal run |
| `-N`, `--crtimes` | Preserve create times | ⛔ Impossible/Divergence | Birth-times cannot be set by any portable filesystem call (`utimensat`/`futimens` only set atime/mtime), so this row is an explicit **Impossible/Divergence** (Phase 7 Wave B). Capture + transmit stays: `statx(STATX_BTIME)` on Linux records the source birth time as a wire field; the receiver logs a debug note that it cannot be applied and continues — never failing the transfer and never pretending it worked. On platforms without `statx` it parses as a documented no-op (flag accepted; nothing is captured). Implies metadata transmission. Wire: new `crtime` fields + a `preserve_crtimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** (see the Phase-4 metadata-time notes) |
| `-O`, `--omit-dir-times` | Omit dirs from --times | ✅ Implemented | Real modifier now that FastSync preserves directory times. With metadata on, the scanner captures every traversed source directory's mtime (and atime under `-U`) and the sender transmits them in trailing `STATUS_DIR_TIMES` frame(s) **after all file data and the optional delete manifest** (chunked at the receiver's `MAX_MANIFEST_ENTRIES` per-frame cap); a dir-time entry only RECORDS metadata and never creates the directory, so empty source directories stay untransferred. The receiver defers applying them until its delete / `--delay-updates` publication phases have committed, so writing or removing a child never clobbers a parent directory's mtime (rsync applies directory times at the end for exactly this reason). When `-O` is set (the boolean crosses the wire) the receiver does not apply any of them; without `-O` an `-a`/`--preserve` transfer now restores directory times (reversing the old "never preserves dir times" divergence). Wire change: the terminal `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** |
| `-J`, `--omit-link-times` | Omit symlinks from --times | ✅ Implemented | Real modifier now that FastSync preserves symlink times. Symlink entries already carried their metadata on `STATUS_SYMLINK`; the receiver now applies it with **no-follow primitives only** (`utimensat(..., AT_SYMLINK_NOFOLLOW)`, plus best-effort `fchmodat(..., AT_SYMLINK_NOFOLLOW)` and policy-gated `fchownat(..., AT_SYMLINK_NOFOLLOW)`), so the link itself is stamped without ever dereferencing it, confined fd-relative below the authorized receive root. A symlink has no children, so the times are applied immediately at creation. When `-J` is set (the boolean crosses the wire) the receiver skips the timestamps (mode/ownership are unaffected); without `-J` an `-a`/`-l` transfer restores symlink mtimes. Wire change alongside `-O`: the shared `STATUS_DIR_TIMES` frame; `PROTOCOL_VERSION` bumped **2.16.0 → 2.17.0** |
| `--super` | Receiver attempts super-user activities | ❌ Not Implemented | |
| `--fake-super` | Store/recover privileged attrs via xattrs | ⚠️ Partial | Honest, limited subset. The receiver records the source `uid:gid:mode:mtime_sec:mtime_nsec` into a reserved `user.fastsync.stat` xattr on each written file (best-effort, fd-relative), so a later privileged restore could re-apply them — without attempting the (typically failing as non-root) `chown`. Full rsync fake-super **replay** (parsing that xattr to actually re-apply ownership on a later privileged run) is out of scope and is **divergent** from rsync, which uses its own `user.rsync.%stat%` format; no cross-tool conversion is attempted. Implies metadata transmission so the source uid/gid/mode/mtime are available. Both it and `-X`/`-A` are incompatible with `-s` (chunk serialization), rejected up front |
| `--fake-super` | Store/recover privileged attrs via xattrs | ✅ Implemented | Phase 7 Wave B: full record **and replay**. The receiver writes the source `uid:gid:mode:mtime_sec:mtime_nsec` into a reserved `user.fastsync.stat` xattr on each written file (best-effort, fd-relative, format unchanged), then immediately re-applies it via `fake_super_restore_fd`: `fchown` (only where privileged — a non-root EPERM/EACCES is skipped silently, matching FastSync's identity philosophy), `fchmod`, and `futimens`. The restored mode goes through the same sanitization as the normal metadata path (group/other write bits are never granted, so a recorded 0666 restores as 0644), so fake-super replay can never grant group/other-write that plain `--preserve` would refuse. Absence or a malformed record is a silent no-op, never fatal. The recording format diverges from rsync's `user.rsync.%stat%`; no cross-tool conversion is attempted. Implies metadata transmission so the source uid/gid/mode/mtime are available. Both it and `-X`/`-A` are incompatible with `-s` (chunk serialization), rejected up front |
| `--open-noatime` | Avoid changing access time when opening files | ✅ Implemented | Sender-side policy: the sender opens source files with `O_NOATIME` (Linux) when reading them for transfer, so the open/read does NOT bump the source's on-disk access time. Degrades safely when `O_NOATIME` is unavailable (not defined) or refused (`EPERM`, since it needs `CAP_FOWNER` or file ownership): the code falls back to a normal open, so the data always transfers — only the atime-bump is skipped. It does not itself capture/preserve atime; it only avoids modifying it. **Client-only, never crosses the wire.** Exposed as `file_open_for_read()` and applied to both the buffered data path and the sendfile path |
| `--numeric-ids` | Do not map uid/gid by name | ✅ Implemented | Ownership is applied through FastSync's opt-in identity path (see the Phase-4 identity notes below). `--numeric-ids` is a mapping-policy modifier: when applying ownership it uses the transmitted numeric uid/gid directly, skipping the name lookup. Without an ownership-affecting option it is inert (FastSync only applies ownership when the user opts in). It does not need `-M` to be parsed, but ownership is only applied when metadata (hence the source uid/gid) is actually transmitted (see the notes) |
| `--usermap=STRING` | Map usernames | ✅ Implemented | Opt-in ownership application. rsync subset implemented: comma-separated `FROM:TO` rules evaluated in order, first match wins; `FROM`/`TO` are group/user names (resolved on the SOURCE machine at parse time), `*` (FROM matches any id / TO = the receiving process's current euid), and an `@N` or bare `N` numeric id. Rules are carried over the wire as resolved numeric id pairs; the receiver applies a matching rule (else falls back to `--chown`, `--numeric-ids`, then a best-effort name lookup) via an fd-relative `fchown`. Malformed/unresolvable specs are rejected with a clear error, never a silent no-op. Implies metadata preservation so the source uid/gid travel. Only effective when the receiver can actually change ownership (root or membership); otherwise it warns and continues |
@@ -348,14 +349,15 @@ fails the transfer and never pretends the crtime was applied. This is the
explicit, documented unsupported-attribute handling. On platforms without
`statx` the flag is accepted but nothing is captured (a documented no-op).
**omit-dir-times / omit-link-times:** `-O` and `-J` are **accepted and parsed
for CLI compatibility** and their config booleans cross the wire, but they are
genuine **no-ops**: FastSync does not apply directory or symlink times at all
(directories are made via `mkdir` with no metadata; symlinks are dereferenced
or skipped, never written with a target), so there is nothing for an "omit" to
suppress. They never break a normal run. This is documented as a
divergence — the flags recognize the rsync interface but have no filtering
effect in FastSync.
**omit-dir-times / omit-link-times:** `-O` and `-J` are **real modifiers** as of
P7 Wave D (`🔄 → ✅ Implemented`). FastSync now preserves directory mtimes
(captured by the scanner, transmitted in trailing `STATUS_DIR_TIMES` frame(s),
applied only after all children and the delete/publication phases) and symlink
mtime/owner/mode (no-follow `utimensat`/`fchownat`/`fchmodat` at link creation).
`-O` makes the receiver skip the directory-time set; `-J` makes it skip the
symlink timestamps (ownership/mode application is unaffected and stays governed
by the identity opt-in). Both config booleans already crossed the wire. See the
`-O`/`-J` rows and the Wave D note below.
**-U/-N and -M interaction:** because FastSync carries all metadata (mode, uid,
gid, mtime, and now atime/crtime) in one bounded payload that is only sent when
@@ -523,9 +525,10 @@ was bumped **2.12.0 → 2.13.0** (peers must match, exactly as prior phases did)
predicate unconditionally, a plain `-l` sync **refuses to round-trip a
legitimate absolute symlink target** (it is dropped, never created pointing
outside the root — see the `--munge-links` note for the symmetric trust
boundary); a relative in-root target is copied as-is. FastSync also does not
set timestamps/owner on symlinks (no symlink-mode metadata application),
matching its existing no-op `--omit-link-times`.
boundary); a relative in-root target is copied as-is. As of P7 Wave D FastSync
also applies the symlink's own metadata with no-follow primitives
(`utimensat`/`fchownat`/`fchmodat` with `AT_SYMLINK_NOFOLLOW`), so `-J` is a
real omit switch rather than a no-op.
- **`-k/--copy-dirlinks`** (sender): a symlink whose referent is a directory is
dereferenced and recursed into as a real directory; a symlink to a regular
file (or any non-directory) is kept as a symlink. This is rsync's `-k`. When
@@ -576,8 +579,8 @@ now transmits targets (the prior behavior was broken/partial); its status moved
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-S`, `--sparse` | Sparse block handling | ⚠️ Partial | Flag is accepted, but full hole preservation is not implemented |
| `--preallocate` | Allocate dest files before writing | ✅ Implemented | The receiver preallocates the destination file's full expected space before any data is written, so a transfer that would overflow disk fails fast at allocation time (a clean error, not a half-written file) and the file is laid out contiguously, avoiding fragmentation. Crosses the wire (the config frame carries a `preallocate` boolean; `PROTOCOL_VERSION` bumped **2.10.0 → 2.11.0**, peers must match) so the sender knows the receiver will preallocate and the receiver performs it. **Allocation approach:** `posix_fallocate()` is preferred because it reserves *real* disk blocks (true fail-fast on ENOSPC), falling back to plain `ftruncate()` only when the filesystem reports the allocation is unsupported (`EOPNOTSUPP`/`ENOSYS`); `ftruncate` still extends the logical size so the intent degrades gracefully. **Fallback/error semantics:** `EOPNOTSUPP`/`ENOSYS` → clean fallback to `ftruncate` (best-effort, preallocates the logical size and never fails a transfer on filesystems that lack `posix_fallocate`); a genuine allocation failure (`ENOSPC`/`EDQUOT`/`EFBIG`/…) aborts the file/receive with a distinct `preallocate failed ... transfer aborted` error — it does **not** fall back to a normal non-preallocated write, preserving the fail-fast purpose. **Size-known requirement:** preallocation only runs when the final size is already known up front (the normal regular-file case); unknown-length data is skipped (never failed). **Orthogonality:** applies uniformly across the atomic temp+rename store path, `--inplace`, `--partial`/`--partial-dir`, `--delay-updates` (the staged temp file is preallocated before data flows) and the `--link-dest` copy fallback; it neither implies nor conflicts with `-s`, `--append`, or delta. rsync-divergence: rsync signals that `--preallocate` is ignored with `--sparse`; FastSync simply preallocates first and still honours `--sparse`'s `ftruncate` sizing/trim, so the two combine rather than one being silently ignored. See the Phase-4 preallocate notes below |
| `-S`, `--sparse` | Sparse block handling | ✅ Implemented | Phase 7 Wave B: real hole preservation with no wire change. The receiver's sparse-aware writer (`write_all_sparse`, next to `write_all` in `src/shared/file.c` and `src/shared/file_store.c`) walks the in-memory file image and emits any all-zero run ≥ 4096 bytes as a hole via `lseek(SEEK_CUR)` (the pre-size `ftruncate` guarantees the offset bookkeeping and logical size), `ftruncate(size)` after the last run pins the final size even with a hole tail. Wired into both the atomic temp+rename store and `--inplace` when `sparse` is set; the non-sparse path is byte-identical to before. **Sparse wins over `--preallocate`** (posix_fallocate is skipped when sparse is set, so the holes are not re-allocated). Interplay note: under `--partial` a retained sparse temp already has the full logical size (trailing content is holes), so `--append`'s "shorter destination" resume does not re-run; the retained file is still valid and a normal re-transfer (or `-W`/delta) repairs it — documented so the combination is never surprising |
| `--preallocate` | Allocate dest files before writing | ✅ Implemented | The receiver preallocates the destination file's full expected space before any data is written, so a transfer that would overflow disk fails fast at allocation time (a clean error, not a half-written file) and the file is laid out contiguously, avoiding fragmentation. Crosses the wire (the config frame carries a `preallocate` boolean; `PROTOCOL_VERSION` bumped **2.10.0 → 2.11.0**, peers must match) so the sender knows the receiver will preallocate and the receiver performs it. **Allocation approach:** `posix_fallocate()` is preferred because it reserves *real* disk blocks (true fail-fast on ENOSPC), falling back to plain `ftruncate()` only when the filesystem reports the allocation is unsupported (`EOPNOTSUPP`/`ENOSYS`); `ftruncate` still extends the logical size so the intent degrades gracefully. **Fallback/error semantics:** `EOPNOTSUPP`/`ENOSYS` → clean fallback to `ftruncate` (best-effort, preallocates the logical size and never fails a transfer on filesystems that lack `posix_fallocate`); a genuine allocation failure (`ENOSPC`/`EDQUOT`/`EFBIG`/…) aborts the file/receive with a distinct `preallocate failed ... transfer aborted` error — it does **not** fall back to a normal non-preallocated write, preserving the fail-fast purpose. **Size-known requirement:** preallocation only runs when the final size is already known up front (the normal regular-file case); unknown-length data is skipped (never failed). **Orthogonality:** applies uniformly across the atomic temp+rename store path, `--inplace`, `--partial`/`--partial-dir`, `--delay-updates` (the staged temp file is preallocated before data flows) and the `--link-dest` copy fallback; it neither implies nor conflicts with `-s`, `--append`, or delta. rsync-divergence: rsync signals that `--preallocate` is ignored with `--sparse`; FastSync gives **sparse precedence** — when both are set, `posix_fallocate` is skipped so the holes the sparse writer creates are not re-allocated (the `ftruncate` presize sizing stays), matching the intent of "sparse wins". See the Phase-4 preallocate notes below |
**Preallocate notes (Phase 4, preallocate wave):** `--preallocate` is implemented as a real receiver-side allocation of the destination file's space before data is written. It is a plain boolean config flag that crosses the wire (serialized in the config frame's selection-options block, mirroring `--inplace`/`--append`/`--force`), so the run requires matching ends: `PROTOCOL_VERSION` was bumped **2.10.0 → 2.11.0** (peers must match or the version check fails). The allocation is performed on the exact destination fd, immediately after it is opened, before any bytes are streamed; `posix_fallocate` (and the `ftruncate` fallback) leave the fd's file offset untouched, so the subsequent data write at offset 0 is unaffected and complete. Because FastSync writes each file's byte payload in one in-memory batch, the "full expected size" is exactly the known `data_size`, which is what gets preallocated. Unknown-length/streamed payloads are skipped rather than failed. A failed allocation logs a distinct `preallocate failed` error and aborts the file (the atomic temp is unlinked, the inplace target is left untrimmed) so the run fails cleanly and never silently degrades to a non-preallocated write — preserving rsync's fail-fast intent on a full disk.
@@ -672,10 +675,10 @@ now transmits targets (the prior behavior was broken/partial); its status moved
| `--stop-after=MINS` | Stop after N minutes | ✅ Implemented | Client-only sender stop deadline (Phase 6): computing `--stop-after=MINS` (a positive minute count; 0/negative/garbage rejected) and `--stop-at=TIME` (`HH:MM`, `HH:MM:SS`, or `now+N[smhd]`; a past time stops immediately). The transfer stops ELEGANTLY at the next chunk boundary: everything already fully sent is kept and applied, the run returns 0, and --delete (late/delete-after timing) does NOT wipe the destination — when the scan is cut short the partial keep-set manifest is suppressed with a warning (the delete walk is skipped rather than acting on an incomplete keep-set, so unscanned source mirrors survive). `--delete-before`/`--delete-during` still run their complete pre-scan (which ignores the deadline). Local client-only fields: never serialized into the wire config frame, so no PROTOCOL_VERSION bump. `--stop-after` uses CLOCK_MONOTONIC; `--stop-at` uses the wall clock. Works single-threaded and under `-j`/`--threads` (multithreaded). Divergence: rsync computes `--stop-after` from the run start; FastSync likewise. When both are given, the earlier of the two deadlines wins (checked per iteration). See the Phase-6 stop notes below |
| `--stop-at=TIME` | Stop at specified time | ✅ Implemented | Same feature as `--stop-after` (deadline transfer stop), absolute wall-clock form (`HH:MM[:SS]` or `now+N[smhd]`). See the row above and the Phase-6 stop notes |
| `--fsync` | Fsync every written file before publication | ✅ Implemented | |
| `--protocol=NUM` | Force older protocol version | ✅ Implemented | Forces the wire protocol version for this transfer. FastSync has exactly ONE wire format (`PROTOCOL_VERSION`, currently 2.16.0) with no downgrade/backward-compat code paths, so `--protocol=2.16.0` is accepted (it sets the version claim the client sends, which the server already requires to match exactly) and **every other value is rejected up front** with a clear error before any connection — it does not and cannot speak an older or virtual wire format. Divergence from rsync (which negotiates a range and downgrades to an integer 0..31): FastSync's honest contract is force-to-the-one-supported-value; a genuine downgrade would require a per-version compatibility layer that does not exist. Client-only; the server-side exact-match check is unchanged. `--protocol=2.16`/`2.15.0`/`2.17.0`/`216`/`31`/garbage are all rejected. See the Phase-6 protocol note below |
| `--protocol=NUM` | Force older protocol version | ✅ Implemented | Forces the wire protocol version for this transfer. FastSync has exactly ONE wire format (`PROTOCOL_VERSION`, currently 2.17.0) with no downgrade/backward-compat code paths, so `--protocol=2.17.0` is accepted (it sets the version claim the client sends, which the server already requires to match exactly) and **every other value is rejected up front** with a clear error before any connection — it does not and cannot speak an older or virtual wire format. Divergence from rsync (which negotiates a range and downgrades to an integer 0..31): FastSync's honest contract is force-to-the-one-supported-value; a genuine downgrade would require a per-version compatibility layer that does not exist. Client-only; the server-side exact-match check is unchanged. `--protocol=2.17`/`2.16.0`/`2.15.0`/`216`/`31`/garbage are all rejected. See the Phase-6 protocol note below |
| `--iconv=CONVERT_SPEC` | Charset conversion | ✅ Implemented | Charset conversion of FILE NAMES (not content) at the protocol boundary via iconv(3): `--iconv=LOCAL[,REMOTE]` — the sender converts each local filename LOCAL→REMOTE before transmitting, and the receiver converts each wire filename REMOTE→LOCAL before creating/writing. The full CONVERT_SPEC is serialized into the config frame as a new trailing string field so the peer knows the wire charset; **PROTOCOL_VERSION bumped 2.15.0 → 2.16.0**. `LOCAL[,REMOTE]` parse: single charset ⇒ LOCAL==REMOTE (identity both ways); garbage rejected up front. Validation probes BOTH directions (a spec that only opens one way is refused, as is a NUL-emitting target charset like utf-16/utf-32/ucs-2, since filenames cannot contain NUL). An unrepresentable name (EILSEQ/EINVAL) fails that path cleanly with a logged `--iconv: cannot convert file name ...` and is never written mangled/truncated. Conversion is applied at EVERY wire-path site (regular/MKDIR/hardlink path+target/symlink path+target/SPECIAL, the delete manifest, the incremental-check path, and the `-s`/`chunk_serialize` embedded blob path), on both client and server (`--iconv` is also a server/daemon option). Zero overhead when unset. See the Phase-6 iconv notes below |
| `--checksum-seed=NUM` | Set checksum seed | ✅ Implemented | Sets the seed for FastSync's whole-file xxHash64 digest (full 64-bit seed) and for the delta path's per-block xxHash32 strong checksum (low 32 bits of the seed). An explicit seed deterministically changes every computed digest on BOTH endpoints (sender and receiver share the seed via the config frame, protocol 2.10.0), so identical runs with the same seed skip the same files and a changed seed changes the digests — the explicit-seed path that makes xxHash comparisons deterministic. `--checksum-choice=md5` has no seed and ignores it (documented). The value is a strict decimal 0..2⁶⁴-1 (blank, signed, or non-numeric values are rejected). Like rsync, a seed only matters where a digest is actually computed (`--checksum` or a basis-dir run, or a delta transfer); it does not by itself enable `--checksum`/`--delta`. Divergence from rsync: the default is seed 0, and FastSync never randomizes the seed (rsync uses a random per-transfer seed when `--checksum-seed` is unset); FastSync's unset default therefore reproduces its historical byte-for-byte behavior |
| `--secluded-args`, `-s` | Use protocol to send args | 🔄 Compatibility No-op | Accepted for CLI compatibility, including the rsync short `-s` (Phase 7 Wave A); it does not change FastSync transport or protocol behavior, because remote SSH argv is already built injection-safe (single-quote-escaped). Chunk serialization is the long-only `--chunk-serialization`. |
| `--secluded-args`, `-s` | Use protocol to send args | ⛔ Impossible/Divergence | Accepted for CLI compatibility (including the rsync short `-s`, Phase 7 Wave A) but a documented **no-op / divergence**. rsync's `-s` protects arguments from shell expansion by shipping them over the protocol; FastSync never passes remote arguments through a shell expansion boundary in the first place — its SSH transport builds the remote argv as **single-quote-escaped shell words** (`ssh_build_remote_command`), so the injection/leak that `-s` guards against does not exist and there is nothing to "seclude". Implementing a true arg-send protocol would mean replacing the argv-based SSH launch with an in-band argument channel, a large redesign of the transport that buys no security here. Chunk serialization remains the long-only `--chunk-serialization`. |
| `--no-OPTION` | Turn off implied option | ✅ Supported | Supported boolean FastSync options and archive-implied options; unsafe or value-taking options are rejected. |
---
@@ -788,7 +791,7 @@ These are the hardest compatibility items because they require durable formats o
**Phase 6, Wave B (iconv) shipping note (PROTOCOL 2.15.0 → 2.16.0):** `--iconv=LOCAL[,REMOTE]` converts file NAMES at the wire boundary (never content). The full CONVERT_SPEC is serialized into the config frame as a new trailing string field (empty→NULL canonicalized), so both ends share the same wire charset interpretation; this required the PROTOCOL bump because the frame is a strict ordered sequence and a peer that does not parse the new trailing field would desynchronize. Each end derives LOCAL (its own charset) and REMOTE (the wire charset): the sender opens LOCAL→REMOTE and converts every transmitted filename; the receiver opens REMOTE→LOCAL and converts every received filename before creating/writing. Conversion is applied at every wire-path site (regular/MKDIR/hardlink path+target/symlink path+target/SPECIAL, the delete manifest keep/protected/missing entries, the incremental-check path, and the embedded `-s`/chunk-blob path). A name it cannot convert (EILSEQ/EINVAL) is failed cleanly with a logged `--iconv: cannot convert file name ...` and is never written truncated/mangled. Validation probes both directions up front (both the sender local→remote and the receiver remote→local, and, for a server/daemon with its own `--iconv`, the client-REMOTE→server-LOCAL pair) so an unusable spec is rejected before the connection rather than mid-transfer, and NUL-emitting target charsets (utf-16/utf-32/ucs-2) are refused because filenames cannot contain NUL. Divergence documented upstream: the receiver does NOT half-swap; the wire charset always comes from the sender's REMOTE half, so a server whose local charset differs from the client's LOCAL must declare it with its own `--iconv`. Conversion is process-global and runs on a single thread per process (sender thread / receiver-loop thread), initialized before worker threads start and freed after they join.
**Phase 6, Wave C (protocol-version) shipping note (no PROTOCOL_VERSION change):** `--protocol=NUM` lets the client force the wire protocol version for a transfer. FastSync's protocol is a single lockstep format: the config frame is a strict ordered sequence and the server requires the client's version string to equal `PROTOCOL_VERSION` exactly (`config_receive_with_validate`, src/shared/config.c) — there are no older-format code paths and no downgrade/negotiation machinery, so a lower/higher/virtual version can never be spoken. The honest contract is therefore: `--protocol=2.16.0` (the current `PROTOCOL_VERSION`) is accepted and stored into the client's `version` claim (which `config_send` already transmits), and every other value — `2.16`, `2.15.0`, `2.17.0`, `3.0.0`, rsync-integer spellings like `216`/`31`, garbage, empty — is rejected up front in `validate_config()` before any connection, with a clear error that FastSync supports only its current wire protocol and cannot speak an older or virtual one. Implementation is client-only: a server-side `--protocol` is intentionally not added because the server has no negotiation (it only enforces exact match), and it could only ever be the current version. This preserves (and slightly tightens) existing validation: the client now also refuses to launch with a version it cannot actually speak, rather than only the server rejecting it later. A genuine downgrade would require a per-version compatibility layer for every frame/feature added since (append 2.10, preallocate 2.11, hardlinks 2.12, devices/specials/symlink-trust/xattr 2.13, remote-option 2.14, daemon module/auth 2.15, iconv 2.16) and is intentionally out of scope — documented divergences from rsync's integer-negotiated downgrade remain.
**Phase 6, Wave C (protocol-version) shipping note (no PROTOCOL_VERSION change):** `--protocol=NUM` lets the client force the wire protocol version for a transfer. FastSync's protocol is a single lockstep format: the config frame is a strict ordered sequence and the server requires the client's version string to equal `PROTOCOL_VERSION` exactly (`config_receive_with_validate`, src/shared/config.c) — there are no older-format code paths and no downgrade/negotiation machinery, so a lower/higher/virtual version can never be spoken. The honest contract is therefore: `--protocol=2.17.0` (the current `PROTOCOL_VERSION`, as of the P7 Wave D times bump) is accepted and stored into the client's `version` claim (which `config_send` already transmits), and every other value — `2.17`, `2.16.0`, `2.15.0`, `3.0.0`, rsync-integer spellings like `216`/`31`, garbage, empty — is rejected up front in `validate_config()` before any connection, with a clear error that FastSync supports only its current wire protocol and cannot speak an older or virtual one. Implementation is client-only: a server-side `--protocol` is intentionally not added because the server has no negotiation (it only enforces exact match), and it could only ever be the current version. This preserves (and slightly tightens) existing validation: the client now also refuses to launch with a version it cannot actually speak, rather than only the server rejecting it later. A genuine downgrade would require a per-version compatibility layer for every frame/feature added since (append 2.10, preallocate 2.11, hardlinks 2.12, devices/specials/symlink-trust/xattr 2.13, remote-option 2.14, daemon module/auth 2.15, iconv 2.16, dir/symlink times 2.17) and is intentionally out of scope — documented divergences from rsync's integer-negotiated downgrade remain.
**Phase-1/2 selection-and-update status correction (docs):** `-I/--ignore-times`, `--size-only`, `-@/--modify-window`, `--existing`, `--ignore-existing`, `-u/--update`, `-W/--whole-file`, and `--compress-threads` were previously listed as not-implemented in this document but are in fact fully implemented and tested on `dev`. This pass corrects the matrix to match the code. The realistic model of these is that FastSync is a *sender-driven* whole-tree copy, so the size+mtime quick-check and all three receiver-policy skips (`--existing`, `--ignore-existing`, `-u`) are evaluated against the **destination** on the receiver side, and their booleans cross the wire in the config frame. `-I`/`--size-only`/`--modify-window` modify the `--incremental` per-file `STATUS_CHECK` handshake's match predicate (`-I` disables the mtime leg and forces transfer; `--size-only` drops only the mtime leg; `--modify-window` adds tolerance to `metadata_mtime_matches`); they require `--incremental` (or a basis dir) to have a handshake to affect, mirroring how they only matter where a quick-check exists in rsync. `--existing`/`--ignore-existing`/`-u` are receiver write-time policies (skipping the write / newer-destination guard) applied across the regular-file, `--delay-updates`-staged, hardlink-sibling, and special/device paths; `-u` implies `-M` metadata and uses a second-then-nanosecond strict `>` newer check; both correctly influence `--remove-source-files` (a skipped source is not removed). `-W/--whole-file` disables block-level delta (opt-in via `--delta`), folded into the wire `use_delta` so no protocol bump was needed, and makes `--fuzzy` inert; `--append`/`--append-verify` are rejected with `-W`. `--compress-threads=NUM` (1..64, client-only, never crosses the wire) sizes the zstd compression worker pool. No code was changed by this correction; the implementation had landed in earlier merge waves (feat/ignore-times, feat/ignore-existing via the newer `file_to_disk_secure_no_replace`/`linkat EEXIST` path, feat/size-only, feat/modify-window, feat/whole-file, feat/update, compression-threads).
@@ -811,15 +814,21 @@ These are the last compatibility items and the closing phase toward rsync flag p
| `-T` / `--timeout` | `-T` = `--temp-dir` | → `--timeout` (long-only) |
| `-a` / `--archive` (= `-c -m -M`) | `-a` = `-rlptgoD` | → becomes **real rsync `-a`** after the renames |
**Wave B — Output & filesystem completion.** `-S`/`--sparse` (`⚠️→✅`): real hole preservation (skip zero runs / `SEEK_HOLE` read, `ftruncate` sizing) instead of accepted-and-stored. `-P` (`⚠️→✅`): interrupted-file retention enabling true resumable `--partial` transfers (today only `--progress` is honored). `--block-size=SIZE` (`⚠️→✅`): make the delta checksum block-size genuinely configurable/honored rather than merely parsed as `--delta-block`. `--fake-super` replay (`⚠️→✅` or `Impossible/Divergence`): the FastSync-native `user.fastsync.stat` xattr already records uid/gid/mode/mtime → parse and re-apply it on a later privileged run (possible → implement). `--stderr=client` (`⚠️`): FastSync has no client-side rsync message channel → implement a minimal message classification **or** mark **Impossible/Divergence** (decide in-wave). `-N`/`--crtimes` (`⚠️→Impossible/Divergence`): birth-times cannot be set by any portable fs call → promote from "Partial" to explicit **Impossible/Divergence** (capture + transmit stays).
**Wave B — Output & filesystem completion (✅ implemented).** `-S`/`--sparse` (`⚠️→✅`): real hole preservation — a sparse-aware writer (`write_all_sparse`) skips all-zero runs ≥ 4096 bytes with `lseek(SEEK_CUR)` and `ftruncate`s the final size, wired into both the atomic temp+rename store and `--inplace` receiver-side with **no wire change** (the full file image is already in memory; the ftruncate presize is kept). `-P` (`⚠️→✅`): interrupted-write retention — on a save failure after data reached the temp fd, `--partial` now renames the already-written temp to the destination path (best-effort; falls through to the normal unlink on failure, never retains when `--partial` is off) so a later `--append`/`--append-verify` run can resume. `--block-size=SIZE` (`⚠️→✅`): promoted after verification — `--block-size` is now an alias for `--delta-block`, both set `config->delta_block_size`, which the delta engine already honored end-to-end (`delta_signature_create_seeded` + `delta_apply`); out-of-range values keep the default. `--fake-super` (`⚠️→✅`): added `fake_super_restore_fd` to parse and re-apply the recorded `user.fastsync.stat` record fd-relative (fchown best-effort/non-root skipped, fchmod, futimens); a save under `--fake-super` now re-applies the recorded attrs instead of only recording them, with the recording format unchanged. `--stderr=client` (`⚠️→⛔ Impossible/Divergence`): FastSync has no rsync client-message channel, and `client` is rejected at CLI parse — the rejection is the documented behavior (unit-tested). `-N`/`--crtimes` (`⚠️→⛔ Impossible/Divergence`): birth-times cannot be set by any portable fs call (`utimensat` sets only atime/mtime); capture/transmit stays, setting is impossible, the flag is accepted and safely inert. Review-hardening (post-eval): fake-super replay applies the mode through the same sanitization as the normal metadata path (group/other write bits are never granted); `--sparse` takes precedence over `--preallocate` (posix_fallocate skipped so holes survive); `--partial` retention is disabled for `--no_replace` (ignore/existing) and only marks a write-attempt after the actual write begins; `--block-size=SIZE`/`--delta-block=SIZE` inline forms are accepted.
**Wave C — Devices & special files (finalize statuses + tests).** `--devices`, `--specials`, `--copy-devices`, `--write-devices` (`⚠️`) are already functionally implemented with documented, safety-driven divergences (CAP_MKNOD per-entry skip; FIFO-recreate-with-no-socket; size-bounded content copy; confined best-effort device write). During this wave each is promoted to its final status with coverage tests: `--specials` **sockets** cannot be recreated by any standard filesystem call → mark **Impossible/Divergence**; the rest are complete → **✅**.
**Wave C — Devices & special files (finalize statuses + tests) (✅ implemented).** The four special-file rows are finalized with coverage tests. `--devices`, `--copy-devices`, and `--write-devices` are **✅ Implemented**, each with a documented, safety-driven divergence: device-node creation is privilege-gated, so a receiver without `CAP_MKNOD` skips that entry with a warning (a per-entry skip, never a transfer failure); `--copy-devices` copies a device/FIFO's reported size into an ordinary regular file (a size-bounded safe divergence from rsync's unbounded dd-like read); `--write-devices` writes only into an existing char/block node under the confined receive root and skips every unusable target rather than clobbering or aborting. `--specials` is classified **⛔ Impossible/Divergence** for one reason only: **FIFO recreation works** (unprivileged `mkfifo`, asserted under CI), but **sockets cannot be recreated by any standard filesystem call**, so a source socket is skipped with an explicit note. Tests assert FIFO recreation, the safe socket skip, the regular-file result of `--copy-devices`, the skipped/missing and non-device `--write-devices` targets, and (root-gated) real device-node creation; a root runner additionally drops the receiver to an unprivileged user to assert the `CAP_MKNOD` skip is graceful.
**Wave D — Times superstructure & arg-protection no-ops (`🔄`).** `-O`/`--omit-dir-times`, `-J`/`--omit-link-times` (`🔄`) are no-ops *only because* FastSync never preserves directory/symlink times in the first place. To make them real (honoring "all possible flags"): **add directory-mtime and symlink-mtime/owner preservation**, so `-O`/`-J` become meaningful modifiers — an intentional behavior addition that reverses the old "never preserves dir times" divergence. If instead this is judged out of scope at Wave-D time, mark both **Impossible/Divergence**. `--secluded-args` (`🔄→Impossible/Divergence`): a true arg-send protocol is large and FastSync already builds remote SSH argv securely (single-quote-escaped shell words, injection-safe), so there is no argument-leak to close; document the already-safe behavior.
**Wave D — Times superstructure & arg-protection no-ops (✅ implemented, `--secluded-args` ⛔).** `-O`/`--omit-dir-times` and `-J`/`--omit-link-times` are now **real modifiers** (both `🔄 → ✅ Implemented`), reversing the old "never preserves directory/symlink times" divergence:
- **Directory times.** The recursive scanner captures every traversed source directory's metadata (mtime, plus atime under `-U`) into a per-transfer list — two paths are covered: the sequential `DirectoryScanner` captures each opened directory (including the transfer root), and the parallel scanner captures both the root in `parallel_scanner_create_with_options` and each worker's subdirectories in `open_next_directory` (appends are guarded by a mutex shared with the sender's pipeline context). The sender transmits them in trailing `STATUS_DIR_TIMES` frames (each: int count + count × (wire path, metadata) pairs) sent **after all file data and after the optional delete manifest**, just before `STATUS_FINISHED`. A tree larger than `MAX_MANIFEST_ENTRIES` (1 048 576) directories is chunked into repeated frames, each within the receiver's per-frame bound. A dir-time entry is RECORD-ONLY (`file->dir_time_only`): `file_save_to_disk_full` returns `FILE_SAVE_SKIPPED` without creating anything, so a source directory that was empty (or pruned by `-m/--prune-empty-dirs`) is never resurrected. The receiver accumulates received directory metadata in a `DirTimeList` and applies it only at the very end — after the entire stream, after the commit-style `--delete` deletion, and after `--delay-updates` publication — because creating or removing a child bumps the parent's mtime. Application is fd-relative/walk-confined (`file_open_secure_parent` + `utimensat(..., AT_SYMLINK_NOFOLLOW)`) and best-effort per entry: an absent path (an intentionally uncreated empty dir) is skipped QUIETLY and only a real existing directory is stamped. `-O` (config boolean, already on the wire) makes the receiver skip the whole set. The single-threaded sink applies in `receiver_send_success_frame`; the `-j`/`--threads` sink accumulates in `write_thread` and server.c applies after both threads join and the deletion commits.
- **Symlink times/owner/mode.** `STATUS_SYMLINK` already carried metadata; the receiver now applies it with no-follow primitives only: `utimensat(..., AT_SYMLINK_NOFOLLOW)`, best-effort `fchmodat(..., AT_SYMLINK_NOFOLLOW)` (honest no-op where unsupported, e.g. Linux), and policy-gated `fchownat(..., AT_SYMLINK_NOFOLLOW)` via a new `identity_apply_ownership_link` that shares the identity resolver with the fd path. `-J` suppresses only the timestamps; ownership stays governed by the identity opt-in (`--numeric-ids`/`--usermap`/`--groupmap`/`--chown`) exactly like regular files. A symlink has no children, so this is applied immediately at creation.
- **Wire:** the shared `STATUS_DIR_TIMES` frame (and metadata on `STATUS_MKDIR` for `--dirs` entries) is a frame-sequence change, so `PROTOCOL_VERSION` was bumped **2.16.0 → 2.17.0**; every version-sensitive test (`--protocol` accepted/rejected values) was updated. The config-frame layout itself is unchanged (the omit booleans already crossed). Non-metadata and `--no-preserve` transfers send no `STATUS_DIR_TIMES` frame and no directory metadata, keeping them byte-identical.
`--secluded-args` (`🔄 → ⛔ Impossible/Divergence`): a true arg-send protocol would replace the argv-based SSH launch with an in-band channel, and FastSync already builds the remote SSH argv injection-safe (single-quote-escaped shell words), so there is no argument-leak to close; the already-safe behavior is documented in the row and no transport change is made.
**Wave E (LAST) — Privilege (deferred decision, `❌`).** `--super`, `--copy-as=USER[:GROUP]`: **deferred by explicit project decision — the privilege model must be decided when this wave starts.** Candidate directions to fix then: a **safe** receiver model — `--copy-as` performs a drop-to-uid/group only when the process is privileged (and a clear refusal otherwise, never blind elevation); `--super` lifts only within the confined receive root — versus a **full setuid/elevation** model (higher security-review burden). Recommended: the safe-subset + clear-refusal direction, consistent with FastSync's confinement philosophy. These are the only remaining `❌` rows.
**Post-Phase-7 Summary targets.** The 3 `🔀 Alt Arg` rows (`-a`, `-p`, `-z`) → **✅** (real rsync semantics; `-z` divergence shrinks to "zstd-only", matching `--checksum-choice`). `⚠️ Partial` (10) → real `✅` or explicit **Impossible/Divergence**. `🔄 Compatibility No-op` (3) → real `✅` (dir/symlink times) or **Impossible/Divergence** (`--secluded-args`). `❌ Not Implemented` (2) → still deferred to Wave E. A new **Impossible/Divergence** status bucket is added to the Summary table; everything else lands at `✅`.
**Post-Phase-7 Summary (after Waves A–D).** ✅141 / 🔀0 / ⛔4 / ⚠️0 / 🔄0 / ❌2 = 147. The 3 `🔀 Alt Arg` rows (`-a`, `-p`, `-z`) are ✅ (Wave A). All 10 prior `⚠️ Partial` rows are resolved to ✅ (`-S`, `-P`, `--block-size`, `--fake-super`, `--devices`, `--copy-devices`, `--write-devices`) or ⛔ (`--stderr=client`, `-N/--crtimes`, `--specials` for the impossible socket case). The 3 `🔄 Compatibility No-op` rows are resolved: `-O`/`-J` are now real ✅ (Wave D), `--secluded-args` is ⛔. The **Impossible/Divergence** bucket holds the 4 physically-impossible/divergent flags: `--stderr=client`, `-N/--crtimes`, `--specials` (sockets), `--secluded-args`. The only remaining `❌ Not Implemented` rows are `--super` and `--copy-as=USER[:GROUP]`, deferred to **Wave E** pending an explicit privilege-model decision (see that paragraph).
### Recommended Delivery Order
+19 -3
View File
@@ -1079,18 +1079,34 @@ int parse_args(Config* config, int argc, char* argv[], int* positional_args,
if (config_add_pattern(&config->include_patterns, &config->include_count, argv[++i],
"--include") != 0)
return -1;
} else if (opt_is(argv[i], "--delta-block", NULL)) {
} else if (strncmp(argv[i], "--delta-block=", 14) == 0) {
unsigned long long val;
if (parse_ull_arg(argv[i] + 14, &val, "--block-size/--delta-block") != 0)
return -1;
if (val >= DELTA_BLOCK_SIZE_MIN && val <= DELTA_BLOCK_SIZE_MAX)
config->delta_block_size = (uint32_t)val;
else
log_message(LOG_LEVEL_WARNING, "block size value %llu out of range, using default", val);
} else if (strncmp(argv[i], "--block-size=", 13) == 0) {
unsigned long long val;
if (parse_ull_arg(argv[i] + 13, &val, "--block-size/--delta-block") != 0)
return -1;
if (val >= DELTA_BLOCK_SIZE_MIN && val <= DELTA_BLOCK_SIZE_MAX)
config->delta_block_size = (uint32_t)val;
else
log_message(LOG_LEVEL_WARNING, "block size value %llu out of range, using default", val);
} else if (opt_is(argv[i], "--delta-block", "--block-size")) {
if (i + 1 >= argc) {
log_message(LOG_LEVEL_ERROR, "missing argument for %s", argv[i]);
return -1;
}
unsigned long long val;
if (parse_ull_arg(argv[++i], &val, "--delta-block") != 0)
if (parse_ull_arg(argv[++i], &val, "--block-size/--delta-block") != 0)
return -1;
if (val >= DELTA_BLOCK_SIZE_MIN && val <= DELTA_BLOCK_SIZE_MAX)
config->delta_block_size = (uint32_t)val;
else
log_message(LOG_LEVEL_WARNING, "--delta-block value %llu out of range, using default", val);
log_message(LOG_LEVEL_WARNING, "block size value %llu out of range, using default", val);
} else if (opt_is(argv[i], "--delta-max", NULL)) {
if (i + 1 >= argc) {
log_message(LOG_LEVEL_ERROR, "missing argument for %s", argv[i]);
+97 -11
View File
@@ -127,6 +127,11 @@ static bool prepare_scanner(const Config* config, int num_threads, PreparedScann
options->excluded_paths = NULL;
options->excluded_mutex = NULL;
options->hardlinks = NULL;
/* P7 Wave D: capture source directory times whenever metadata rides the
wire. Whether they are APPLIED is decided receiver-side (-O skips). */
options->capture_dir_times = config->use_metadata;
options->dir_entries = NULL;
options->dir_entries_mutex = NULL;
if (config->preserve_hard_links) {
out->hardlinks = hardlink_table_create();
if (!out->hardlinks) {
@@ -1115,14 +1120,51 @@ static bool send_file_direct(File* file, int fd, bool use_metadata, int compress
}
/* Transmit one explicit directory entry (--dirs): a STATUS_MKDIR frame whose
payload is only the destination path. The receiver validates the path and
creates the directory under the receive root. */
static bool send_directory_entry(Client* client, File* file) {
payload is the destination path and, when metadata is negotiated, the
directory's metadata frame. The receiver validates the path, creates the
directory under the receive root, and (metadata case) defers applying its
times to the end of the transfer so -O/--omit-dir-times is honored. */
static bool send_directory_entry(const Client* client, File* file, const Config* config) {
if (!file || !file_wire_path(file))
return false;
if (!send_status(client->file_descriptor, STATUS_MKDIR))
if (!send_status(client->file_descriptor, STATUS_MKDIR) ||
!send_wire_str(client->file_descriptor, file_wire_path(file)))
return false;
return send_wire_str(client->file_descriptor, file_wire_path(file));
return !config->use_metadata || metadata_send(client->file_descriptor, file->metadata);
}
/* P7 Wave D: transmit every captured source directory's metadata in terminal
STATUS_DIR_TIMES frames (count, then (path, metadata) pairs) after all file
data and the optional delete manifest. The receiver applies them at the END
of its own transfer (after deletion and --delay-updates publication) so a
directory's mtime is not clobbered by writing its children. A non-metadata
transfer (or an empty set) sends nothing, keeping the stream byte-identical.
The receiver rejects a frame whose count exceeds MAX_MANIFEST_ENTRIES, so a
huge tree is CHUNKED into repeated frames of at most that many entries each
(the receiver's loop handles repeated STATUS_DIR_TIMES frames). Every frame
stays within the receiver's bound, and a frame that would exceed it is never
emitted. */
static bool send_dir_times(const Client* client, const Config* config, ArrayList* dir_entries) {
if (!client || !config || !config->use_metadata || !dir_entries || dir_entries->size == 0)
return true;
int fd = client->file_descriptor;
int index = 0;
while (index < dir_entries->size) {
int remaining = dir_entries->size - index;
int chunk = remaining > MAX_MANIFEST_ENTRIES ? MAX_MANIFEST_ENTRIES : remaining;
if (!send_status(fd, STATUS_DIR_TIMES) || !send_int(fd, chunk))
return false;
for (int i = 0; i < chunk; i++) {
File* file = (File*)dir_entries->items[index + i];
if (!file || !file_wire_path(file))
return false;
if (!send_wire_str(fd, file_wire_path(file)) || !metadata_send(fd, file->metadata))
return false;
}
index += chunk;
}
return true;
}
/* Transmit one symlink entry: a STATUS_SYMLINK frame carrying the destination
@@ -1259,6 +1301,19 @@ static int send_single_file(Client* client, File* file, Config* config, bool use
return 0;
}
/* Sendfile calls a blocking open() on the source (file_send_sendfile_with_skip
* -> file_open_for_read), which never returns for a FIFO/device with no writer.
* Only a regular file may take the zero-copy sendfile path; a non-regular source
* (FIFO/device copied by --copy-devices) must use the buffered, size-bounded
* read path instead. `stat` follows symlinks, so a dereferenced symlink to a
* regular file keeps the sendfile fast path. */
static bool source_is_regular_file(const File* file) {
if (!file || !file->path)
return false;
struct stat st;
return stat(file->path, &st) == 0 && S_ISREG(st.st_mode);
}
static int send_chunk_with_removal(Client* client, Chunk* chunk, Config* config,
ArrayList* remove_sources) {
if (config->use_chunk_serialization) {
@@ -1300,10 +1355,10 @@ static int send_chunk_with_removal(Client* client, Chunk* chunk, Config* config,
if (f == NULL)
continue;
if (f->is_dir) {
/* Explicit directory entry (--dirs): a MKDIR frame carrying only the
destination path. Directories have no source to remove and no
incremental check. */
if (!send_directory_entry(client, f))
/* Explicit directory entry (--dirs): a MKDIR frame carrying the
destination path (and metadata when negotiated). Directories have no
source to remove and no incremental check. */
if (!send_directory_entry(client, f, config))
return -1;
change_emit_dir_sent(config, f);
continue;
@@ -1337,8 +1392,9 @@ static int send_chunk_with_removal(Client* client, Chunk* chunk, Config* config,
continue;
}
bool stream = f->data->data == NULL && f->data->size > 0;
bool use_sendfile =
(config->use_sendfile && !config->use_compression) || (stream && !config->use_compression);
bool use_sendfile = ((config->use_sendfile && !config->use_compression) ||
(stream && !config->use_compression)) &&
source_is_regular_file(f);
SourceFile* source = remove_sources ? source_file_create(f) : NULL;
int rc = send_single_file(client, f, config, config->use_incremental, use_sendfile);
if (rc == 1) {
@@ -1487,6 +1543,13 @@ static int send_chunks_multithreaded(void* pipeline_context) {
if (send_delete_manifest(client->file_descriptor, NULL, NULL, context->missing_args) != 0)
goto send_fail;
}
/* P7 Wave D: transmit the captured directory times last. The scanner thread
(and all parallel workers) has been joined before scanner_done was set, so
the list is complete and race-free; on an early stop the list may be
incomplete and is deliberately not sent. */
if (!context->scan_stopped_early &&
!send_dir_times(client, context->config, context->dir_entries))
goto send_fail;
bool ok = finalize_transfer(client, context->config, context->remove_source_files);
if (!ok && context->config->use_delete)
log_message(LOG_LEVEL_ERROR,
@@ -1528,6 +1591,10 @@ static int scan_directory_multithreaded(void* pipeline_context) {
return thrd_error;
}
prepared.options.stop_condition = &context->stop_condition;
/* P7 Wave D: the recursive scan feeds the shared directory-time list; the
parallel workers append under the context's dedicated mutex. */
prepared.options.dir_entries = context->dir_entries;
prepared.options.dir_entries_mutex = &context->dir_entries_mutex;
/* The keep-set manifest for the late modes is built from this data pass, so
the parallel scanner records the protected excluded prefixes here. The
early modes already transmitted the pre-scan keep-set and its protected
@@ -1829,6 +1896,9 @@ int send_files(Config* config) {
DirectoryScanner* scanner = NULL;
ArrayList* manifest = NULL;
ArrayList* remove_sources = NULL;
/* P7 Wave D: captured source directory times, transmitted in trailing
STATUS_DIR_TIMES frame(s) (only when metadata rides the wire). */
ArrayList* dir_entries = NULL;
/* Protected excluded prefixes (delete-excluded default protection). */
ArrayList* excluded = NULL;
bool delete_early = config->use_delete && config_delete_timing_early(config);
@@ -1841,6 +1911,11 @@ int send_files(Config* config) {
receive_daemon_motd(client, config);
if (!prepare_scanner(config, 0, &prepared))
goto send_fail;
if (config->use_metadata) {
dir_entries = array_list_create(file_destroy);
if (!dir_entries)
goto send_fail;
}
if (config->remove_source_files)
remove_sources = array_list_create(source_file_destroy);
if (config->remove_source_files && !remove_sources)
@@ -1905,6 +1980,10 @@ int send_files(Config* config) {
StopCondition stop = stop_condition_make(config->stop_after_mins > 0, config->stop_after_mins,
config->stop_at_set, config->stop_at, now_mono);
prepared.options.stop_condition = &stop;
/* The early-delete pre-scan above already ran; only the data pass should feed
the directory-time list (otherwise every directory would be captured
twice). */
prepared.options.dir_entries = dir_entries;
scanner = directory_scanner_create_with_options(config->send_directory, &prepared.options);
if (!scanner)
goto send_fail;
@@ -2023,6 +2102,11 @@ int send_files(Config* config) {
}
}
}
/* P7 Wave D: every directory has now been traversed (or the scan stopped
early), so transmit the captured directory times last. The receiver defers
applying them until after its own deletion/publication phase. */
if (!send_dir_times(client, config, dir_entries))
goto send_fail;
bool ok = finalize_transfer(client, config, remove_sources);
if (!ok && config->use_delete)
log_message(LOG_LEVEL_ERROR,
@@ -2064,6 +2148,8 @@ send_fail:
array_list_delete(missing_args);
if (remove_sources)
array_list_delete(remove_sources);
if (dir_entries)
array_list_delete(dir_entries);
if (scanner)
directory_scanner_destroy(scanner);
prepared_scanner_destroy(&prepared);
+83
View File
@@ -488,6 +488,9 @@ DirectoryScanner* directory_scanner_create_with_options(const char* root_directo
scanner->hardlinks = options->hardlinks;
scanner->prune_empty_dirs = options->prune_empty_dirs;
scanner->stop_condition = options->stop_condition;
scanner->capture_dir_times = options->capture_dir_times;
scanner->dir_entries = options->dir_entries;
scanner->dir_entries_mutex = options->dir_entries_mutex;
scanner->dirs_root_emitted = false;
scanner->list_index = 0;
scanner->dirs_batch = NULL;
@@ -595,6 +598,63 @@ static Chunk* chunk_data_to_chunk(ArrayList* chunk_data) {
return chunk;
}
/* P7 Wave D: append one traversed source directory's captured metadata to the
* shared pending-directory-time list. The File carries no payload; only the
* wire path (absolute fs path normally, the bare relative path under
* -R + --files-from) and its metadata are used, and the sender transmits them
* in trailing STATUS_DIR_TIMES frame(s). `mutex` (optional) serializes the
* append for the parallel scanner's shared workers. An unstattable or
* non-directory path is silently skipped (the transfer is unaffected); an
* allocation failure is fatal and reported to the caller. */
static bool scanner_capture_dir_time(ArrayList* dir_entries, mtx_t* mutex, const char* root_path,
const char* fs_path, bool relative_mode, bool preserve_atimes,
bool preserve_crtimes) {
if (!dir_entries || !root_path || !fs_path)
return true;
struct stat st;
if (stat(fs_path, &st) != 0 || !S_ISDIR(st.st_mode))
return true;
char* rel = scanner_path_relative(root_path, fs_path);
if (!rel)
return true;
if (relative_mode && rel[0] == '\0') {
/* -R + --files-from: the transfer root itself has no bare relative wire
path (matches the -R scan, which never emits the root). */
free(rel);
return true;
}
File* file = file_create(fs_path);
if (!file) {
free(rel);
return false;
}
file->is_dir = true;
file->metadata = file_metadata_create(fs_path, &st, preserve_atimes, preserve_crtimes);
if (!file->metadata) {
free(rel);
file_destroy(file);
return false;
}
if (relative_mode) {
file->send_path = rel;
rel = NULL;
}
free(rel);
bool added;
if (mutex) {
mtx_lock(mutex);
added = array_list_add(dir_entries, file);
mtx_unlock(mutex);
} else {
added = array_list_add(dir_entries, file);
}
if (!added) {
file_destroy(file);
return false;
}
return true;
}
/* Open the next queued directory and set up its filter context. Returns 1 when
a directory is open, 0 when the queue is exhausted, and -1 on a fatal error.
A directory that cannot be opened is an I/O error: it is recorded on the
@@ -661,6 +721,17 @@ static int open_next_directory(DirectoryScanner* scanner) {
scanner->current_path = NULL;
return -1;
}
if (scanner->capture_dir_times &&
!scanner_capture_dir_time(scanner->dir_entries, scanner->dir_entries_mutex,
scanner->root_path, scanner->current_path, scanner->relative_mode,
scanner->preserve_atimes, scanner->preserve_crtimes)) {
closedir(scanner->current_dir);
scanner->current_dir = NULL;
free(scanner->current_path);
scanner->current_path = NULL;
scanner->failed = true;
return -1;
}
return 1;
}
return 0;
@@ -1584,6 +1655,18 @@ ParallelScanner* parallel_scanner_create_with_options(const char* root_directory
parallel_scanner_destroy(ps);
return NULL;
}
/* P7 Wave D: the parallel scanner never runs a DirectoryScanner over the
transfer root itself (it hands the root's immediate subdirectories to
workers), so capture the root's directory time here. */
if (options->capture_dir_times &&
!scanner_capture_dir_time(options->dir_entries, options->dir_entries_mutex, root_directory,
root_directory, options->relative && options->file_list != NULL,
options->preserve_atimes, options->preserve_crtimes)) {
array_list_delete(root_files);
array_list_delete(subdirs);
parallel_scanner_destroy(ps);
return NULL;
}
unsigned long long cs = options->chunk_size > 0 ? options->chunk_size : DESIRED_CHUNK_SIZE;
ps->initial_chunk = batch_files(root_files, cs, ps->result_queue, &ps->failed);
+16
View File
@@ -98,6 +98,18 @@ typedef struct {
* (without marking the scan as failed), so a busy scan itself stops early.
* Client-only, never serialized to the wire. */
const StopCondition* stop_condition;
/* P7 Wave D (protocol 2.17.0): directory-time capture sink. When
* `capture_dir_times` is true the recursive scan appends one is_dir File
* (with metadata, no payload) per source directory it traverses to
* `dir_entries`, so the sender can transmit trailing STATUS_DIR_TIMES
* frame(s) and the receiver can apply directory mtimes AFTER all children
* are written. `dir_entries_mutex` (optional) guards the list
* for the parallel scanner's shared worker threads; the caller owns both.
* The --dirs generator does not use this (its directory entries carry their
* metadata inline through STATUS_MKDIR). */
bool capture_dir_times;
ArrayList* dir_entries;
mtx_t* dir_entries_mutex;
} ScannerOptions;
/* Internal per-scanner filter state. FilterNode chains represent the ordered
@@ -173,6 +185,10 @@ typedef struct {
HardLinkTable* hardlinks;
/* Phase 6: sender stop deadline (from ScannerOptions). */
const StopCondition* stop_condition;
/* P7 Wave D directory-time capture (see ScannerOptions). */
bool capture_dir_times;
ArrayList* dir_entries;
mtx_t* dir_entries_mutex;
} DirectoryScanner;
typedef struct {
+5 -5
View File
@@ -140,8 +140,8 @@ void print_usage(void) {
printf(" --incremental and --delta; inert with --whole-file,\n");
printf(" --no-delta, or --no-incremental)\n");
printf(" --no-fuzzy Disable --fuzzy\n");
printf(" --delta-block <n> Delta block size in bytes (default: %d)\n",
DELTA_BLOCK_SIZE_DEFAULT);
printf(" --delta-block <n>, --block-size <n>\n");
printf(" Delta block size in bytes (default: %d)\n", DELTA_BLOCK_SIZE_DEFAULT);
printf(" --delta-max <n> Max file size for delta transfer (default: %llu)\n",
DELTA_MAX_FILE_SIZE);
printf(" -j, --threads Enable multithreading\n");
@@ -165,9 +165,9 @@ void print_usage(void) {
printf(" setting an ACL the receiver is not permitted to\n");
printf(" set is warned and skipped, never fatal)\n");
printf(" --fake-super Store the source uid/gid/mode/mtime in a reserved\n");
printf(" user.fastsync.stat xattr on each written file instead\n");
printf(" of applying ownership (for a later privileged restore);\n");
printf(" partial: full rsync fake-super replay is out of scope\n");
printf(" user.fastsync.stat xattr on each written file and\n");
printf(" re-apply it (fd-relative) on a privileged run; the\n");
printf(" recording format diverges from rsync's user.rsync.%%stat%%\n");
printf(" --chmod <changes> Modify transferred permissions (rsync syntax)\n");
printf(" --numeric-ids Do not map uid/gid by name: use the source numeric\n");
printf(" ids directly when applying ownership\n");
+44 -2
View File
@@ -74,6 +74,25 @@ static bool receiver_process_chunk(Chunk* chunk, const ReceiverSink* sink) {
return true;
}
/* P7 Wave D: read one STATUS_DIR_TIMES frame (a count followed by that many
* (path, metadata) directory entries) and route every entry through the regular
* store_file sink. A dir-time entry is RECORD-ONLY (file->dir_time_only): the
* sink accumulates its metadata for end-of-transfer application but creates
* nothing, so an empty/pruned source directory is never resurrected. A large
* tree arrives as repeated frames, each bounded by MAX_MANIFEST_ENTRIES; a
* malformed count or entry is a hard error. */
static bool receiver_process_dir_times(int fd, const Config* config, const ReceiverSink* sink) {
int count;
if (!receive_int(fd, &count) || count < 0 || count > MAX_MANIFEST_ENTRIES)
return false;
for (int i = 0; i < count; i++) {
File* dir = file_receive_dir_time(fd, config);
if (!dir || !sink->store_file(dir, sink->context))
return false;
}
return true;
}
static bool receiver_process_batch(Config* config, int file_descriptor) {
int count;
if (config->checksum || !receive_int(file_descriptor, &count) || count < 0 ||
@@ -161,7 +180,7 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver
while (status == STATUS_NEXT || status == STATUS_CHUNK || status == STATUS_CHECK ||
status == STATUS_KEEPALIVE || status == STATUS_ABORT || status == STATUS_CHECK_BATCH ||
status == STATUS_MKDIR || status == STATUS_MANIFEST || status == STATUS_HARDLINK ||
status == STATUS_SYMLINK || status == STATUS_SPECIAL) {
status == STATUS_SYMLINK || status == STATUS_SPECIAL || status == STATUS_DIR_TIMES) {
if (status == STATUS_KEEPALIVE) {
if (!send_status(file_descriptor, STATUS_KEEPALIVE))
goto fail;
@@ -185,9 +204,12 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver
goto fail;
goto next_status;
} else if (status == STATUS_MKDIR) {
File* dir = file_receive_directory(file_descriptor);
File* dir = file_receive_directory(file_descriptor, config);
if (!dir || !sink->store_file(dir, sink->context))
goto receive_error;
} else if (status == STATUS_DIR_TIMES) {
if (!receiver_process_dir_times(file_descriptor, config, sink))
goto receive_error;
} else if (status == STATUS_HARDLINK) {
File* file = file_receive_hardlink(file_descriptor);
if (!file || !sink->store_file(file, sink->context))
@@ -311,6 +333,10 @@ receive_error:
typedef struct {
Config* config;
ReceiverOutcomes outcomes;
/* P7 Wave D: directory metadata accumulated during the stream, applied only
after the whole transfer (and its delete/publication phases) has run so a
child write never clobbers a directory mtime. */
DirTimeList dir_times;
} ReceiverSaveContext;
static bool receiver_save_file(File* file, void* context_pointer) {
@@ -323,6 +349,15 @@ static bool receiver_save_file(File* file, void* context_pointer) {
} else {
result = file_save_to_disk_full(context->config->receive_root_directory, file, context->config);
}
/* A directory's times are deferred, never applied inline: collect the
metadata now and apply it at the end. -O/--omit-dir-times is honored by
dir_time_list_apply's caller (see receiver_send_success_frame). */
if (result != FILE_SAVE_ERROR && file->is_dir && file->metadata &&
context->config->use_metadata && !context->config->omit_dir_times &&
!dir_time_list_add(&context->dir_times, file->path, file->metadata)) {
file_destroy(file);
return false;
}
if (result != FILE_SAVE_ERROR && context->config->remove_source_files && !file->is_dir &&
!file->is_special && !file->skip &&
!receiver_outcomes_append(&context->outcomes, (unsigned char)result)) {
@@ -346,15 +381,22 @@ static bool receiver_send_success_frame(int fd, void* context_pointer) {
return false;
}
}
/* P7 Wave D: every child is now written and the delete / --delay-updates
phases have committed, so it is finally safe to stamp directory times.
This runs after the deferred deletion because receiver_process commits it
before calling this success frame. */
dir_time_list_apply(&context->dir_times, context->config->receive_root_directory);
return receiver_send_final_success(fd, context->config, &context->outcomes);
}
int receiver_receive_files(Config* config, int file_descriptor) {
ReceiverSaveContext context = {.config = config, .outcomes = {0}};
dir_time_list_init(&context.dir_times);
ReceiverSink sink = {receiver_save_file, &context, true, true, receiver_send_success_frame};
int ret = receiver_process(config, file_descriptor, &sink);
if (ret != 0 && config->delay_updates && config->delay_context)
delay_updates_cleanup(config->delay_context);
receiver_outcomes_destroy(&context.outcomes);
dir_time_list_free(&context.dir_times);
return ret;
}
+6
View File
@@ -493,6 +493,12 @@ void handler(int file_descriptor) {
!delay_updates_publish(config->delay_context, config)) {
transfer_ok = false;
}
/* P7 Wave D: all writers have joined and the late deletion (and
--delay-updates publication) has committed above, so it is finally safe
to stamp directory times; a directory's mtime must not be clobbered by
its children or by an extra removal. */
if (transfer_ok)
dir_time_list_apply(&context->dir_times, config->receive_root_directory);
}
if (transfer_ok) {
if (!receiver_send_final_success(file_descriptor, config, &context->outcomes))
+22 -2
View File
@@ -543,8 +543,28 @@ typedef struct Config {
* new trailing bytes would desynchronize on the frame boundary, and the strict
* same-version handshake (config_receive rejects a mismatched version before
* parsing anything else) is what keeps a 2.16 client and a 2.15 server from
* ever reaching that state. */
#define PROTOCOL_VERSION "2.16.0"
* ever reaching that state.
*
* Times Wave (P7 Wave D): 2.16.0 -> 2.17.0.
*
* WHY the bump, grounded in the wire: this wave makes -O/--omit-dir-times and
* -J/--omit-link-times REAL by adding directory and symlink time preservation.
* The config-frame LAYOUT is unchanged (the omit flags already crossed the
* wire), but the FRAME STREAM gains a new terminal frame: after all file data
* and the optional delete manifest, the sender transmits STATUS_DIR_TIMES
* frame(s) (each a count followed by (path, metadata) pairs, chunked so no
* frame exceeds the receiver's MAX_MANIFEST_ENTRIES bound) carrying every
* source directory's captured times, so the receiver can apply them AFTER all of a
* directory's children have been written (writing a child bumps the parent's
* mtime). Symlink entries already carry their metadata on the STATUS_SYMLINK
* frame; the receiver now applies it (utimensat/lchown with
* AT_SYMLINK_NOFOLLOW) unless -J is set. Any change to the frame sequence must
* bump the protocol version: a 2.16 peer that does not know STATUS_DIR_TIMES
* would desynchronize on the unknown frame, and the strict same-version
* handshake (config_receive rejects a mismatched version before parsing
* anything else) is what keeps a 2.17 client and a 2.16 server from ever
* reaching that state. */
#define PROTOCOL_VERSION "2.17.0"
#define DEFAULT_CHUNK_SIZE (10 * 1024 * 1024)
/* Upper bound on total basis-dir entries (rsync caps --link-dest at 20). */
#define MAX_BASIS_DIRS 64
+93 -17
View File
@@ -36,6 +36,44 @@ static bool write_all(int fd, const void* data, unsigned long long size) {
return true;
}
/* A run of NUL bytes at least this long is emitted as a hole (lseek) rather
* than written, so the resulting file is genuinely sparse on the filesystem. */
#define SPARSE_HOLE_MIN 4096U
/* Sparse-aware writer (--sparse/-S). Walks `data`; any all-zero run of at
* least SPARSE_HOLE_MIN bytes is skipped with lseek(SEEK_CUR) so the block is
* never allocated (a real hole on the destination); every other byte is written
* normally. The file is pre-sized with ftruncate by the callers before this
* runs, so holes are guaranteed and the offset bookkeeping stays correct
* (each lseek advances the fd offset exactly as a write of that many bytes
* would). After the final run, ftruncate(size) guarantees the logical size is
* exactly `size` even when the tail was a hole. The full file image is in
* memory, so no wire change is needed. Returns false on I/O error. */
static bool write_all_sparse(int fd, const unsigned char* data, unsigned long long size) {
unsigned long long i = 0;
while (i < size) {
if (data[i] == 0) {
unsigned long long run_start = i;
while (i < size && data[i] == 0)
i++;
unsigned long long run_len = i - run_start;
if (run_len >= SPARSE_HOLE_MIN) {
if (lseek(fd, (off_t)run_len, SEEK_CUR) < 0)
return false;
} else if (!write_all(fd, data + run_start, run_len)) {
return false;
}
} else {
unsigned long long run_start = i;
while (i < size && data[i] != 0)
i++;
if (!write_all(fd, data + run_start, i - run_start))
return false;
}
}
return ftruncate(fd, (off_t)size) == 0;
}
/* Preallocate `size` bytes on `fd` before any data is written (--preallocate).
* posix_fallocate reserves real disk blocks, so an out-of-space condition
* (ENOSPC/EDQUOT) surfaces up front instead of partway through a transfer;
@@ -111,6 +149,7 @@ File* file_create(const char* path) {
file->metadata = NULL;
file->skip = false;
file->is_dir = false;
file->dir_time_only = false;
file->basis_link = NULL;
file->link_group = 0;
file->link_first = false;
@@ -805,9 +844,15 @@ int file_open_private_dir(const char* dir_path) {
static void restore_extra_fd(int fd, const FileMetadata* metadata, const FileXattrList* xattrs,
bool fake_super) {
xattr_apply_fd(fd, xattrs);
if (fake_super && metadata)
if (fake_super && metadata) {
fake_super_store_fd(fd, (uint32_t)metadata->uid, (uint32_t)metadata->gid,
(uint32_t)metadata->mode, metadata->mtime_sec, metadata->mtime_nsec);
/* Replay: re-apply the recorded uid/gid/mode/mtime fd-relative so a save
under --fake-super restores the attrs (when privileged) instead of only
recording them. Best-effort; fake_super_restore_fd silently skips a
non-root fchown EPERM/EACCES and never fatal. */
fake_super_restore_fd(fd);
}
}
static bool file_to_disk_secure_impl(const char* path, const void* data,
@@ -815,7 +860,8 @@ static bool file_to_disk_secure_impl(const char* path, const void* data,
bool preallocate, const FileMetadata* metadata,
bool preserve_executability, bool update, bool no_replace,
bool use_fsync, const char* temp_dir,
const FileXattrList* xattrs, bool fake_super) {
const FileXattrList* xattrs, bool fake_super,
bool keep_partial) {
char* leaf = NULL;
int dirfd = file_open_secure_parent(path, &leaf, true);
if (dirfd < 0)
@@ -837,9 +883,12 @@ static bool file_to_disk_secure_impl(const char* path, const void* data,
ok = true;
} else {
/* Preallocate the expected payload size before writing so an
out-of-space condition fails cleanly up front (--preallocate). */
out-of-space condition fails cleanly up front (--preallocate).
--sparse takes precedence: posix_fallocate would allocate every
block, defeating the holes the sparse writer would create, so the
two never combine here (the ftruncate presize below stays). */
int prealloc_rc = 0;
if (preallocate && data_size > 0) {
if (preallocate && !sparse && data_size > 0) {
prealloc_rc = preallocate_fd(fd, data_size);
if (prealloc_rc != 0)
log_message(LOG_LEVEL_ERROR, "preallocate failed for '%s' (%s); transfer aborted", path,
@@ -852,7 +901,9 @@ static bool file_to_disk_secure_impl(const char* path, const void* data,
if (sparse && data_size > 0)
ok = ftruncate(fd, (off_t)data_size) == 0;
if (ok || !sparse || data_size == 0)
ok = write_all(fd, data, data_size);
ok = sparse && data_size > 0
? write_all_sparse(fd, (const unsigned char*)data, data_size)
: write_all(fd, data, data_size);
if (ok)
ok = ftruncate(fd, (off_t)data_size) == 0;
/* Normalize the mode: apply the metadata-derived safe mode when the
@@ -875,6 +926,10 @@ static bool file_to_disk_secure_impl(const char* path, const void* data,
} else {
/* The --update newer-destination check runs first so a skipped file never
creates an empty scratch directory behind it. */
/* True once the temp is being written: distinguishes a mid-write/metadata/
install failure (partial data may exist, --partial may retain it) from a
pre-write validation failure (nothing to retain). */
bool write_attempted = false;
if (update && metadata) {
/* This check protects the normal atomic path as far as possible. A
concurrent replacement can still occur before the final rename. */
@@ -940,7 +995,7 @@ static bool file_to_disk_secure_impl(const char* path, const void* data,
if (fd < 0)
continue; /* EEXIST (or a transient open error): try a fresh name. */
int prealloc_rc = 0;
if (preallocate && data_size > 0) {
if (preallocate && !sparse && data_size > 0) {
prealloc_rc = preallocate_fd(fd, data_size);
if (prealloc_rc != 0)
log_message(LOG_LEVEL_ERROR, "preallocate failed for '%s' (%s); transfer aborted", path,
@@ -950,8 +1005,14 @@ static bool file_to_disk_secure_impl(const char* path, const void* data,
lseek(fd, 0, SEEK_SET);
if (sparse && data_size > 0)
ok = ftruncate(fd, (off_t)data_size) == 0;
if (ok || (!sparse || data_size == 0))
ok = write_all(fd, data, data_size);
/* A real write attempt begins here (the ftruncate presize succeeded or
no presize applies): a later mid-write / metadata / fsync / install
failure may leave partial data that --partial retention can rename. */
if (ok || (!sparse || data_size == 0)) {
write_attempted = true;
ok = sparse && data_size > 0 ? write_all_sparse(fd, (const unsigned char*)data, data_size)
: write_all(fd, data, data_size);
}
if (ok && metadata)
ok = file_restore_metadata_fd(fd, metadata, preserve_executability);
if (ok)
@@ -986,8 +1047,21 @@ static bool file_to_disk_secure_impl(const char* path, const void* data,
ok = false;
}
}
if (!ok)
if (!ok) {
/* --partial retention (best-effort): on a failure that happened after
the temp held data (mid-write / metadata / fsync / install error),
keep the already-written temp at the final destination path instead
of unlinking it, so a later --append / --append-verify run can resume.
This only ever renames the already-written temp (never a corrupt
blend); the rename can fail (cross-device, permissions) and we then
fall through to the normal unlink cleanup. Never retains when
keep_partial is off, when nothing was actually written, or under
--ignore-existing/--existing (no_replace), where the destination is
not ours to overwrite. */
if (!keep_partial || !write_attempted || no_replace ||
renameat(scratch_dirfd >= 0 ? scratch_dirfd : dirfd, tmp, dirfd, leaf) != 0)
unlinkat(scratch_dirfd >= 0 ? scratch_dirfd : dirfd, tmp, 0);
}
/* Once the temp fd was created the outcome is permanent: a write,
metadata, fsync, close, linkat or renameat failure will not be fixed
by retrying under a fresh name, so stop here. Only the open-failure
@@ -1010,7 +1084,7 @@ bool file_to_disk_secure(const char* path, const void* data, unsigned long long
bool preserve_executability, const char* temp_dir) {
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
preserve_executability, false, false, false, temp_dir, NULL,
false);
false, false);
}
bool file_to_disk_secure_update(const char* path, const void* data, unsigned long long data_size,
@@ -1018,7 +1092,7 @@ bool file_to_disk_secure_update(const char* path, const void* data, unsigned lon
const FileMetadata* metadata, bool preserve_executability,
const char* temp_dir) {
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
preserve_executability, true, false, false, temp_dir, NULL,
preserve_executability, true, false, false, temp_dir, NULL, false,
false);
}
@@ -1029,7 +1103,7 @@ bool file_to_disk_secure_with_fsync(const char* path, const void* data,
const char* temp_dir) {
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
preserve_executability, false, false, use_fsync, temp_dir, NULL,
false);
false, false);
}
bool file_to_disk_secure_no_replace(const char* path, const void* data,
@@ -1037,22 +1111,24 @@ bool file_to_disk_secure_no_replace(const char* path, const void* data,
const FileMetadata* metadata, bool preserve_executability,
const char* temp_dir) {
return file_to_disk_secure_impl(path, data, data_size, false, sparse, preallocate, metadata,
preserve_executability, false, true, false, temp_dir, NULL,
preserve_executability, false, true, false, temp_dir, NULL, false,
false);
}
/* Receiver write-path variant that also applies the per-file xattrs (-X/-A)
* and, under --fake-super, parks the source stat in the reserved xattr, on the
* just-written file descriptor before the final rename. `no_replace` / `update`
* mirror the plain wrappers; see file_to_disk_secure_impl for the semantics. */
* mirror the plain wrappers; `keep_partial` enables --partial retention of a
* failed write's temp. See file_to_disk_secure_impl for the semantics. */
bool file_to_disk_secure_attrs(const char* path, const void* data, unsigned long long data_size,
bool inplace, bool sparse, bool preallocate,
const FileMetadata* metadata, bool preserve_executability,
bool update, bool no_replace, bool use_fsync,
const FileXattrList* xattrs, bool fake_super, const char* temp_dir) {
const FileXattrList* xattrs, bool fake_super, bool keep_partial,
const char* temp_dir) {
return file_to_disk_secure_impl(path, data, data_size, inplace, sparse, preallocate, metadata,
preserve_executability, update, no_replace, use_fsync, temp_dir,
xattrs, fake_super);
xattrs, fake_super, keep_partial);
}
/* Atomic --link-dest install. The destination is replaced (via a temporary
@@ -1155,7 +1231,7 @@ static bool file_to_disk_secure_link_impl(const char* path, const char* basis_pa
by the filesystem). Write a byte-identical local copy instead. */
return file_to_disk_secure_attrs(path, data, data_size, false, false, preallocate, metadata,
preserve_executability, false, false, use_fsync, xattrs,
fake_super, temp_dir);
fake_super, false, temp_dir);
}
if (scratch_dirfd >= 0)
+4 -2
View File
@@ -113,12 +113,14 @@ bool file_to_disk_secure_no_replace(const char* path, const void* data,
const char* temp_dir);
/* Receiver write-path variant that also applies per-file xattrs (-X/-A) and the
* --fake-super stat xattr fd-relative before the final rename. `update` /
* `no_replace` / `use_fsync` mirror the plain wrappers above. */
* `no_replace` / `use_fsync` mirror the plain wrappers above; `keep_partial`
* enables --partial best-effort retention of a failed write's temp. */
bool file_to_disk_secure_attrs(const char* path, const void* data, unsigned long long data_size,
bool inplace, bool sparse, bool preallocate,
const FileMetadata* metadata, bool preserve_executability,
bool update, bool no_replace, bool use_fsync,
const FileXattrList* xattrs, bool fake_super, const char* temp_dir);
const FileXattrList* xattrs, bool fake_super, bool keep_partial,
const char* temp_dir);
/* Atomic --link-dest install: replace `path` with a hard link to `basis_path`
(via a temp name + rename); fall back to a byte-identical local copy from
`data` when the link is impossible (EXDEV/EPERM/unsupported filesystem).
+182 -13
View File
@@ -87,10 +87,10 @@ static FileSaveResult file_stage_delayed_update(const char* root_directory,
config->preallocate, metadata, preserve_executability,
config->use_fsync, NULL);
} else {
ok =
file_to_disk_secure_attrs(staged_path, file->data->data, file->data->size, false, sparse,
ok = file_to_disk_secure_attrs(staged_path, file->data->data, file->data->size, false, sparse,
config->preallocate, metadata, preserve_executability, false,
false, config->use_fsync, file->xattrs, config->fake_super, NULL);
false, config->use_fsync, file->xattrs, config->fake_super,
false, NULL);
}
if (!ok) {
free(staged_path);
@@ -551,6 +551,18 @@ FileSaveResult file_save_to_disk_full(const char* root_directory, const File* fi
return FILE_SAVE_ERROR;
}
/* P7 Wave D #1: a STATUS_DIR_TIMES entry is RECORD-ONLY. The scanner
captures every traversed directory -- including empty ones whose parents
were never created by a child write and directories pruned by
-m/--prune-empty-dirs. Creating them here would resurrect empty
directories (an -a behavior change) and could abort the whole transfer on a
pre-existing regular file/symlink at the mirror path. Short-circuit before
any device/write-devices/directory branch and report it as skipped so the
sink still accumulates its metadata for the deferred DirTimeList
application, but create nothing. */
if (file->dir_time_only)
return FILE_SAVE_SKIPPED;
/* Device/special node (--devices/--specials): recreate the node instead of
writing content (privilege-gated, confined, rdev-validated). */
if (file->is_special)
@@ -621,6 +633,12 @@ FileSaveResult file_save_to_disk_full(const char* root_directory, const File* fi
}
ok = file_symlink_at_secure(link_path, target);
free(target);
/* P7 Wave D: apply the symlink's own metadata with no-follow primitives
(utimensat/lchown/fchmodat AT_SYMLINK_NOFOLLOW). -J/--omit-link-times
suppresses the timestamps; ownership stays gated by the identity policy.
A symlink has no children, so this can be applied immediately. */
if (ok && config && config->use_metadata)
file_restore_symlink_metadata(link_path, file->metadata, config->omit_link_times);
free(link_path);
return ok ? FILE_SAVE_WRITTEN : FILE_SAVE_ERROR;
}
@@ -796,11 +814,11 @@ FileSaveResult file_save_to_disk_full(const char* root_directory, const File* fi
} else {
/* The plain no-replace / update / with-fsync engines, plus per-file xattr
(-X/-A) and --fake-super application on the written fd. */
ok = file_to_disk_secure_attrs(disk_path, file->data->data, file->data->size, inplace, sparse,
config && config->preallocate, metadata, preserve_executability,
config && config->update, config && config->ignore_existing,
config && config->use_fsync, file->xattrs,
config ? config->fake_super : false, confined_temp);
ok = file_to_disk_secure_attrs(
disk_path, file->data->data, file->data->size, inplace, sparse,
config && config->preallocate, metadata, preserve_executability, config && config->update,
config && config->ignore_existing, config && config->use_fsync, file->xattrs,
config ? config->fake_super : false, config ? config->partial : false, confined_temp);
}
free(confined_temp);
confined_temp = NULL;
@@ -2137,12 +2155,119 @@ File* file_receive(const Config* config, int file_descriptor) {
return file;
}
/* ---- P7 Wave D: deferred directory times ---- */
void dir_time_list_init(DirTimeList* list) {
if (!list)
return;
list->paths = NULL;
list->entries = NULL;
list->count = 0;
list->capacity = 0;
}
void dir_time_list_free(DirTimeList* list) {
if (!list)
return;
for (size_t i = 0; i < list->count; i++)
free(list->paths[i]);
free(list->paths);
free(list->entries);
list->paths = NULL;
list->entries = NULL;
list->count = 0;
list->capacity = 0;
}
bool dir_time_list_add(DirTimeList* list, const char* wire_path, const FileMetadata* metadata) {
if (!list || !wire_path || !metadata)
return true; /* nothing to remember; never a hard error */
if (list->count == list->capacity) {
size_t new_capacity = list->capacity == 0 ? 16 : list->capacity * 2;
if (new_capacity < list->capacity)
return false;
/* Assign each grown array as soon as its realloc succeeds: the old block is
already freed by then, so discarding the pointer would dangle. capacity
is advanced only after BOTH reallocs succeed, so a partial failure leaves
capacity no larger than the entries allocation (the paths array may be
over-allocated, which is harmless) -- never a mismatched list the next
add could write past. */
char** grown_paths = realloc(list->paths, new_capacity * sizeof(char*));
if (!grown_paths)
return false;
list->paths = grown_paths;
FileMetadata* grown_entries = realloc(list->entries, new_capacity * sizeof(FileMetadata));
if (!grown_entries)
return false;
list->entries = grown_entries;
list->capacity = new_capacity;
}
char* copy = str_dup(wire_path);
if (!copy)
return false;
list->paths[list->count] = copy;
list->entries[list->count] = *metadata;
list->count++;
return true;
}
void dir_time_list_apply(const DirTimeList* list, const char* root_directory) {
if (!list || !root_directory)
return;
for (size_t i = 0; i < list->count; i++) {
char* dir_path = path_cat(root_directory, list->paths[i]);
if (!dir_path)
continue;
char* leaf = NULL;
/* The parent walk is fd-relative and O_NOFOLLOW, so a symlink planted in a
parent component can never redirect the utimensat outside the root. */
int parent_fd = file_open_secure_parent(dir_path, &leaf, false);
if (parent_fd < 0) {
free(dir_path);
continue;
}
/* A dir-time entry only records metadata: the directory is (deliberately)
not created from it, so an empty source directory (or one pruned by
-m/--prune-empty-dirs) may well not exist here. Skip absent paths
QUIETLY rather than warning for every one, and apply the times only to a
real directory that does exist. AT_SYMLINK_NOFOLLOW keeps a same-named
symlink from being followed; a pre-existing regular file/symlink is not a
directory, so it is left completely untouched. */
struct stat st;
if (fstatat(parent_fd, leaf, &st, AT_SYMLINK_NOFOLLOW) != 0 || !S_ISDIR(st.st_mode)) {
close(parent_fd);
free(leaf);
free(dir_path);
continue;
}
struct timespec times[2] = {
{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
{.tv_sec = list->entries[i].mtime_sec, .tv_nsec = list->entries[i].mtime_nsec}};
if (list->entries[i].atime_valid) {
times[0].tv_sec = list->entries[i].atime_sec;
times[0].tv_nsec = list->entries[i].atime_nsec;
}
if (utimensat(parent_fd, leaf, times, AT_SYMLINK_NOFOLLOW) != 0) {
char* escaped_path = output_escape(dir_path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "Failed to set directory timestamps on %s: %s",
escaped_path ? escaped_path : "<allocation failed>", strerror(errno));
free(escaped_path);
}
close(parent_fd);
free(leaf);
free(dir_path);
}
}
/* Receive an explicit directory entry (--dirs): a STATUS_MKDIR frame carries
only the destination path; the entry carries no payload. The same path
validation as a regular file applies (non-empty, relative-or-mirrored, no
traversal), and the created File is routed through the regular store_file
sink so single-threaded and -m receivers handle directories identically. */
File* file_receive_directory(int file_descriptor) {
the destination path and, when metadata is negotiated, the directory's
metadata frame. The same path validation as a regular file applies
(non-empty, relative-or-mirrored, no traversal), and the created File is
routed through the regular store_file sink so single-threaded and -m
receivers handle directories identically. The metadata is NOT applied here:
the sink accumulates it into a DirTimeList that is applied only after the
whole transfer (children would otherwise clobber the directory mtime). */
File* file_receive_directory(int file_descriptor, const Config* config) {
char* path = receive_wire_str(file_descriptor);
if (path == NULL)
return NULL;
@@ -2159,6 +2284,50 @@ File* file_receive_directory(int file_descriptor) {
if (file == NULL)
return NULL;
file->is_dir = true;
if (config && config->use_metadata) {
int meta_ok = 1;
file->metadata = metadata_receive(file_descriptor, &meta_ok);
if (!meta_ok) {
file_destroy(file);
return NULL;
}
}
return file;
}
/* Receive one directory-time entry from a STATUS_DIR_TIMES frame: the
* destination-relative wire path and (when metadata is negotiated) the
* directory's metadata frame. The created File is an is_dir, dir_time_only
* entry routed through the regular store_file sink: the sink records its
* metadata into the deferred DirTimeList but never creates the directory (the
* scanner captures every traversed directory, including empty ones). Unlike a
* STATUS_MKDIR entry, this one must not create anything. */
File* file_receive_dir_time(int file_descriptor, const Config* config) {
char* path = receive_wire_str(file_descriptor);
if (path == NULL)
return NULL;
if (path[0] == '\0' || (!file_get_trust_sender() && has_path_traversal(path))) {
char* escaped_path = output_escape(path, log_get_8_bit_output());
log_message(LOG_LEVEL_ERROR, "Invalid received directory-time path: %s",
escaped_path ? escaped_path : "<allocation failed>");
free(escaped_path);
free(path);
return NULL;
}
File* file = file_create(path);
free(path);
if (!file)
return NULL;
file->is_dir = true;
file->dir_time_only = true;
if (config && config->use_metadata) {
int meta_ok = 1;
file->metadata = metadata_receive(file_descriptor, &meta_ok);
if (!meta_ok) {
file_destroy(file);
return NULL;
}
}
return file;
}
+28 -1
View File
@@ -8,13 +8,40 @@
/* Server-side file receive/save path. */
File* file_receive(const Config* config, int file_descriptor);
File* file_receive_directory(int file_descriptor);
File* file_receive_directory(int file_descriptor, const Config* config);
File* file_receive_dir_time(int file_descriptor, const Config* config);
File* file_receive_hardlink(int file_descriptor);
File* file_receive_symlink(int file_descriptor, const Config* config);
File* file_receive_special(int file_descriptor);
bool file_special_rdev_valid(int32_t major, int32_t minor, mode_t mode);
File* receive_incremental_check(int fd, const Config* config, bool* skipped);
/* P7 Wave D directory-time accumulator. The receiver collects the metadata of
* every directory it creates/receives (STATUS_MKDIR with metadata and/or the
* trailing STATUS_DIR_TIMES frame(s)) and applies the times only at the END of the
* transfer, after all children have been written and after the delete /
* --delay-updates phases have committed (writing or removing a child bumps the
* parent's mtime). -O/--omit-dir-times skips the application entirely. The
* list owns deep copies of the paths and metadata; freed on every path. */
typedef struct {
char** paths; /* owned, destination-relative wire paths */
FileMetadata* entries; /* owned, parallel to paths */
size_t count;
size_t capacity;
} DirTimeList;
void dir_time_list_init(DirTimeList* list);
void dir_time_list_free(DirTimeList* list);
/* Deep-copy one directory's path + metadata into the list. Returns false on
* allocation failure (the caller fails the transfer). */
bool dir_time_list_add(DirTimeList* list, const char* wire_path, const FileMetadata* metadata);
/* Apply every accumulated directory's mtime (and atime when captured) beneath
* `root_directory`, confined fd-relative. Best-effort per entry: an absent
* directory (an empty/pruned source dir that was deliberately not created) or a
* non-directory at the path is skipped QUIETLY, an unreachable one with a
* warning, and never fatal. */
void dir_time_list_apply(const DirTimeList* list, const char* root_directory);
/* A received delete-manifest frame: the keep-set (`keeps`, destination-relative
paths the sender transferred/keeps) plus `protected`, destination-relative
prefixes the sender asks the receiver never to delete (paths excluded on the
+45 -2
View File
@@ -139,6 +139,44 @@ static bool write_all(int fd, const void* data, unsigned long long size) {
return true;
}
/* A run of NUL bytes at least this long is emitted as a hole (lseek) rather
* than written, so the resulting file is genuinely sparse on the filesystem. */
#define SPARSE_HOLE_MIN 4096U
/* Sparse-aware writer (--sparse/-S). Walks `data`; any all-zero run of at
* least SPARSE_HOLE_MIN bytes is skipped with lseek(SEEK_CUR) so the block is
* never allocated (a real hole on the destination); every other byte is written
* normally. The file is pre-sized with ftruncate by the callers before this
* runs, so holes are guaranteed and the offset bookkeeping stays correct
* (each lseek advances the fd offset exactly as a write of that many bytes
* would). After the final run, ftruncate(size) guarantees the logical size is
* exactly `size` even when the tail was a hole. The full file image is in
* memory, so no wire change is needed. Returns false on I/O error. */
static bool write_all_sparse(int fd, const unsigned char* data, unsigned long long size) {
unsigned long long i = 0;
while (i < size) {
if (data[i] == 0) {
unsigned long long run_start = i;
while (i < size && data[i] == 0)
i++;
unsigned long long run_len = i - run_start;
if (run_len >= SPARSE_HOLE_MIN) {
if (lseek(fd, (off_t)run_len, SEEK_CUR) < 0)
return false;
} else if (!write_all(fd, data + run_start, run_len)) {
return false;
}
} else {
unsigned long long run_start = i;
while (i < size && data[i] != 0)
i++;
if (!write_all(fd, data + run_start, i - run_start))
return false;
}
}
return ftruncate(fd, (off_t)size) == 0;
}
bool file_store_write_secure(const char* path, const void* data, unsigned long long data_size,
bool inplace, bool sparse, const FileMetadata* metadata,
bool preserve_executability) {
@@ -151,8 +189,12 @@ bool file_store_write_secure(const char* path, const void* data, unsigned long l
if (inplace) {
fd = openat(dirfd, leaf, O_WRONLY | O_CREAT | O_TRUNC | O_CLOEXEC | O_NOFOLLOW, 0644);
if (fd >= 0) {
if (!sparse || data_size == 0 || ftruncate(fd, (off_t)data_size) == 0)
if (sparse && data_size > 0) {
if (ftruncate(fd, (off_t)data_size) == 0)
ok = write_all_sparse(fd, data, data_size);
} else {
ok = write_all(fd, data, data_size);
}
if (ok && metadata)
ok = file_restore_metadata_fd(fd, metadata, preserve_executability);
}
@@ -177,7 +219,8 @@ bool file_store_write_secure(const char* path, const void* data, unsigned long l
if (sparse && data_size > 0)
ok = ftruncate(fd, (off_t)data_size) == 0;
if (ok || (!sparse || data_size == 0))
ok = write_all(fd, data, data_size);
ok = (sparse && data_size > 0) ? write_all_sparse(fd, (const unsigned char*)data, data_size)
: write_all(fd, data, data_size);
if (ok && metadata)
ok = file_restore_metadata_fd(fd, metadata, preserve_executability);
if (close(fd) != 0)
+8
View File
@@ -42,6 +42,14 @@ typedef struct {
/* True when this entry is an explicit directory entry (--dirs mode): the
* receiver creates the directory instead of writing a regular file. */
bool is_dir;
/* Receiver-only (P7 Wave D): this is a STATUS_DIR_TIMES entry. It carries a
* traversed source directory's metadata for DEFERRED application, but must
* NEVER create the directory: the scanner captures every traversed directory
* (including empty ones whose parents no child write created), so creation
* would resurrect the empty dirs that FastSync deliberately never transfers.
* file_save_to_disk_full short-circuits such an entry as FILE_SAVE_SKIPPED,
* and the sink still accumulates the metadata into its DirTimeList. */
bool dir_time_only;
/* Receiver-only, --link-dest: when set, install the destination entry as a
* hard link to this absolute (root-confined) path instead of writing
* `data`. The matching code has already verified the link target's content
+58 -29
View File
@@ -2,6 +2,7 @@
#include "log.h"
#include "utils.h"
#include <errno.h>
#include <fcntl.h>
#include <grp.h>
#include <limits.h>
#include <pwd.h>
@@ -370,16 +371,11 @@ static bool identity_map_lookup(const IdentityMap* map, int count, int32_t sourc
return false;
}
void identity_apply_ownership(int fd, int32_t source_uid, int32_t source_gid) {
/* Ownership application is OFF unless the client requested an identity flag.
* This is the controlled gate: a default (or plain -M) transfer never changes
* ownership, byte-for-byte preserving FastSync's existing behavior. */
if (!identity_active_enabled() || fd < 0)
return;
struct stat st;
if (fstat(fd, &st) != 0)
return;
/* Resolve the target ownership from the negotiated policy against the entry's
* current stat. Shared by the fd (regular file) and no-follow (symlink) apply
* paths. Returns false when no side is to be changed. */
static bool identity_resolve_targets(const struct stat* st, int32_t source_uid, int32_t source_gid,
uid_t* out_uid, gid_t* out_gid) {
bool set_uid = false;
bool set_gid = false;
uid_t uid = 0;
@@ -431,29 +427,62 @@ void identity_apply_ownership(int fd, int32_t source_uid, int32_t source_gid) {
}
if (!set_uid && !set_gid)
return;
return false;
/* An unset side keeps the file's current id so the other side can change. */
if (!set_uid)
uid = st.st_uid;
uid = st->st_uid;
if (!set_gid)
gid = st.st_gid;
gid = st->st_gid;
/* Only change ownership when the target differs (avoid needless syscalls and
* any chance of clearing setuid/setgid on an already-correct entry). */
if (st->st_uid == uid && st->st_gid == gid)
return false;
*out_uid = uid;
*out_gid = gid;
return true;
}
/* Only call fchown when the target differs (avoid needless syscalls and any
* chance of clearing setuid/setgid on an already-correct file). */
if (st.st_uid == uid && st.st_gid == gid)
static void identity_log_chown_failure(const char* what, uid_t uid, gid_t gid) {
/* EPERM/EACCES are expected when the receiver is not privileged (e.g. the CI
* `nobody` user): warn and continue, never abort the transfer. Any other
* error (EIO/EROFS/ENOSPC/...) is a real failure and must not be silently
* downgraded to a warning. */
if (errno == EPERM || errno == EACCES)
log_message(LOG_LEVEL_WARNING, "could not apply ownership (uid=%ld gid=%ld): %s; leaving as-is",
(long)uid, (long)gid, strerror(errno));
else
log_message(LOG_LEVEL_ERROR, "failed to apply ownership on %s (uid=%ld gid=%ld): %s", what,
(long)uid, (long)gid, strerror(errno));
}
void identity_apply_ownership(int fd, int32_t source_uid, int32_t source_gid) {
/* Ownership application is OFF unless the client requested an identity flag.
* This is the controlled gate: a default (or plain -M) transfer never changes
* ownership, byte-for-byte preserving FastSync's existing behavior. */
if (!identity_active_enabled() || fd < 0)
return;
struct stat st;
if (fstat(fd, &st) != 0)
return;
uid_t uid;
gid_t gid;
if (!identity_resolve_targets(&st, source_uid, source_gid, &uid, &gid))
return;
if (fchown(fd, uid, gid) != 0)
identity_log_chown_failure("file", uid, gid);
}
if (fchown(fd, uid, gid) != 0) {
/* EPERM/EACCES are expected when the receiver is not privileged (e.g. the
* CI `nobody` user): warn and continue, never abort the transfer. Any
* other error (EIO/EROFS/ENOSPC/...) is a real failure and must not be
* silently downgraded to a warning. */
if (errno == EPERM || errno == EACCES)
log_message(LOG_LEVEL_WARNING,
"could not apply ownership (uid=%ld gid=%ld): %s; leaving as-is", (long)uid,
(long)gid, strerror(errno));
else
log_message(LOG_LEVEL_ERROR, "failed to apply ownership (uid=%ld gid=%ld): %s", (long)uid,
(long)gid, strerror(errno));
}
void identity_apply_ownership_link(int parent_fd, const char* leaf, int32_t source_uid,
int32_t source_gid) {
if (!identity_active_enabled() || parent_fd < 0 || !leaf)
return;
struct stat st;
if (fstatat(parent_fd, leaf, &st, AT_SYMLINK_NOFOLLOW) != 0)
return;
uid_t uid;
gid_t gid;
if (!identity_resolve_targets(&st, source_uid, source_gid, &uid, &gid))
return;
if (fchownat(parent_fd, leaf, uid, gid, AT_SYMLINK_NOFOLLOW) != 0)
identity_log_chown_failure("symlink", uid, gid);
}
+7
View File
@@ -55,6 +55,13 @@ bool identity_active_enabled(void);
* never fatal (rsync parity: the transfer must not abort). */
void identity_apply_ownership(int fd, int32_t source_uid, int32_t source_gid);
/* P7 Wave D: the no-follow (symlink) counterpart. Resolves the same
* usermap/groupmap/chown/numeric-ids policy but applies it with
* fchownat(..., AT_SYMLINK_NOFOLLOW) so a symlink's own ownership is changed
* without ever dereferencing it. A no-op unless an identity flag is active. */
void identity_apply_ownership_link(int parent_fd, const char* leaf, int32_t source_uid,
int32_t source_gid);
/* Receiver-side wire validation of the resolved identity fields. */
bool identity_wire_valid(const Config* config);
+37
View File
@@ -357,6 +357,43 @@ void file_restore_metadata(const char* path, const FileMetadata* metadata,
}
}
void file_restore_symlink_metadata(const char* path, const FileMetadata* metadata,
bool omit_link_times) {
if (path == NULL || metadata == NULL)
return;
char* leaf = NULL;
int parent_fd = file_open_secure_parent(path, &leaf, false);
if (parent_fd < 0)
return;
/* Ownership (only when the identity policy is active) via lchown semantics:
fchownat with AT_SYMLINK_NOFOLLOW never dereferences the link. */
identity_apply_ownership_link(parent_fd, leaf, (int32_t)metadata->uid, (int32_t)metadata->gid);
/* Symlink mode: not settable on Linux (fchmodat AT_SYMLINK_NOFOLLOW returns
EOPNOTSUPP/ENOTSUP); attempt it for platforms that support it and quietly
ignore the unsupported case so the transfer never fails over it. */
mode_t link_mode = metadata->mode & 0777;
if (fchmodat(parent_fd, leaf, link_mode, AT_SYMLINK_NOFOLLOW) != 0 && errno != EOPNOTSUPP &&
errno != ENOTSUP && errno != ENOSYS) {
log_message(LOG_LEVEL_DEBUG, "Could not set symlink mode on %s: %s", path, strerror(errno));
}
if (!omit_link_times) {
struct timespec times[2] = {{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
{.tv_sec = metadata->mtime_sec, .tv_nsec = metadata->mtime_nsec}};
if (metadata->atime_valid) {
times[0].tv_sec = metadata->atime_sec;
times[0].tv_nsec = metadata->atime_nsec;
}
if (utimensat(parent_fd, leaf, times, AT_SYMLINK_NOFOLLOW) != 0) {
char* escaped_path = output_escape(path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "Failed to set symlink timestamps on %s: %s",
escaped_path ? escaped_path : "<allocation failed>", strerror(errno));
free(escaped_path);
}
}
close(parent_fd);
free(leaf);
}
bool file_restore_metadata_fd(int fd, const FileMetadata* metadata, bool preserve_executability) {
if (fd < 0 || metadata == NULL)
return metadata == NULL;
+8
View File
@@ -39,6 +39,14 @@ FileMetadata* metadata_receive(int file_descriptor, int* ok);
void file_restore_metadata(const char* path, const FileMetadata* metadata,
bool preserve_executability);
bool file_restore_metadata_fd(int fd, const FileMetadata* metadata, bool preserve_executability);
/* P7 Wave D: apply a SYMLINK's own metadata using no-follow primitives only
* (utimensat/lchown/fchmodat with AT_SYMLINK_NOFOLLOW), confined fd-relative
* under the authorized root. `omit_link_times` (-J/--omit-link-times)
* suppresses the timestamps; the link's mode/ownership are still attempted
* (ownership stays gated by the identity policy and by default is not applied).
* A null metadata or an unfollowable parent is a harmless no-op. */
void file_restore_symlink_metadata(const char* path, const FileMetadata* metadata,
bool omit_link_times);
/* Compare timestamps using rsync's whole-second modification window. */
bool metadata_mtime_matches(time_t left_sec, long left_nsec, time_t right_sec, long right_nsec,
+38
View File
@@ -39,7 +39,14 @@ PipelineContextSender* pipeline_context_sender_create(Config* config, Queue* que
atomic_init(&context->cancelled, false);
protocol_session_init(&context->allocation_session, -1, -1);
protocol_session_set_max_alloc(&context->allocation_session, config->max_alloc);
context->dir_entries = NULL;
context->dir_entries_mutex_init = false;
int init = 0;
if (config->use_metadata) {
context->dir_entries = array_list_create(file_destroy);
if (!context->dir_entries)
goto fail;
}
if (mtx_init(&context->mutex_scanner, mtx_plain) != thrd_success)
goto fail;
init++;
@@ -62,10 +69,17 @@ PipelineContextSender* pipeline_context_sender_create(Config* config, Queue* que
goto fail;
// cppcheck-suppress unreadVariable
init++;
if (mtx_init(&context->dir_entries_mutex, mtx_plain) != thrd_success)
goto fail;
context->dir_entries_mutex_init = true;
return context;
fail:
log_perror("Error initializing synchronization objects");
if (context->dir_entries_mutex_init)
mtx_destroy(&context->dir_entries_mutex);
if (context->dir_entries)
array_list_delete(context->dir_entries);
if (init >= 6)
cnd_destroy(&context->condition_not_empty_loader);
if (init >= 5)
@@ -92,6 +106,10 @@ void pipeline_context_sender_destroy(PipelineContextSender* context) {
array_list_delete(context->missing_args);
if (context->remove_source_files)
array_list_delete(context->remove_source_files);
if (context->dir_entries)
array_list_delete(context->dir_entries);
if (context->dir_entries_mutex_init)
mtx_destroy(&context->dir_entries_mutex);
config_delete(context->config);
queue_destroy(context->queue_scanner);
queue_destroy(context->queue_loader);
@@ -117,6 +135,7 @@ PipelineContextReceiver* pipeline_context_receiver_create(Config* config, Queue*
context->outcomes.entries = NULL;
context->outcomes.count = 0;
context->outcomes.capacity = 0;
dir_time_list_init(&context->dir_times);
protocol_session_init(&context->session, file_descriptor, file_descriptor);
protocol_session_set_ssl(&context->session, ssl);
context->receiver_done = false;
@@ -155,6 +174,7 @@ void pipeline_context_receiver_destroy(PipelineContextReceiver* context) {
delete_manifest_free(context->deferred_manifest);
queue_destroy(context->queue);
receiver_outcomes_destroy(&context->outcomes);
dir_time_list_free(&context->dir_times);
mtx_destroy(&context->mutex);
cnd_destroy(&context->condition_not_full);
cnd_destroy(&context->condition_not_empty);
@@ -307,6 +327,24 @@ int write_thread(void* pipeline_context) {
return thrd_error;
}
}
/* P7 Wave D: a directory's times are never applied inline (a later child
write would clobber them); accumulate the metadata here and let the
caller apply it once every writer has drained. */
if (result != FILE_SAVE_ERROR && file->is_dir && file->metadata &&
context->config->use_metadata && !context->config->omit_dir_times &&
!dir_time_list_add(&context->dir_times, file->path, file->metadata)) {
file_destroy(file);
pipeline_context_receiver_note_bytes_released(context, file_bytes);
mtx_lock(&context->mutex);
atomic_store(&context->cancelled, true);
context->receiver_done = true;
cnd_broadcast(&context->condition_not_full);
cnd_broadcast(&context->condition_not_empty);
mtx_unlock(&context->mutex);
free(root_directory);
protocol_session_unbind();
return thrd_error;
}
/* Record the per-file outcome so a --remove-source-files sender learns
which sources were actually written versus skipped on the receiver.
Explicit directory entries and recreated device/special nodes have no
+11
View File
@@ -67,6 +67,13 @@ typedef struct {
* before it reads the manifest, so no additional synchronization is needed
* to suppress the manifest. */
bool scan_stopped_early;
/* P7 Wave D: captured source directory times, filled by the scanner thread
* (and its parallel workers, guarded by dir_entries_mutex) and drained by the
* sender thread in trailing STATUS_DIR_TIMES frame(s). Owned by the
* context; NULL for non-metadata transfers. */
ArrayList* dir_entries;
mtx_t dir_entries_mutex;
bool dir_entries_mutex_init;
} PipelineContextSender;
typedef struct PipelineContextReceiver {
@@ -97,6 +104,10 @@ typedef struct PipelineContextReceiver {
transfer truly succeeded. NULL in the early delete modes (which delete at
the manifest). */
DeleteManifest* deferred_manifest;
/* P7 Wave D: directory metadata collected by write_thread from received
directory entries. Only write_thread mutates it (before it joins); the
caller (server.c) applies it after the delete/delay-updates phase. */
DirTimeList dir_times;
} PipelineContextReceiver;
PipelineContextSender* pipeline_context_sender_create(Config* config, Queue* queue_scanner,
+6
View File
@@ -407,6 +407,12 @@ static const char* status_to_string(Status status) {
return "APPEND_DATA";
case STATUS_HARDLINK:
return "HARDLINK";
case STATUS_SYMLINK:
return "SYMLINK";
case STATUS_SPECIAL:
return "SPECIAL";
case STATUS_DIR_TIMES:
return "DIR_TIMES";
default:
return "UNKNOWN";
}
+11 -1
View File
@@ -103,7 +103,17 @@ enum NET_STATUS {
* int32 rdev major/minor fields. The receiver validates the kind and rdev,
* confines the node below the receive root, and recreates it (mknod/mkfifo),
* privilege-gating the mknod. Protocol 2.13.0. */
STATUS_SPECIAL
STATUS_SPECIAL,
/* Directory-time superstructure (P7 Wave D, protocol 2.17.0): one or more
* trailing frames sent after all file data (and after the optional delete
* manifest) carrying the source directories' captured metadata so the
* receiver can apply directory mtimes/atimes AFTER all of a directory's
* children have been written. Payload per frame: an int count, then count
* repetitions of (wire path string, metadata frame); an entry count larger
* than MAX_MANIFEST_ENTRIES is split across repeated frames. The receiver
* defers the actual utimensat until its own delete/publish phase has
* committed, then skips the whole set when -O/--omit-dir-times is set. */
STATUS_DIR_TIMES
};
void io_set_fds(int read_fd, int write_fd);
+46
View File
@@ -9,7 +9,10 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/xattr.h>
#include <time.h>
#include <unistd.h>
/* ---- lifecycle ---- */
@@ -329,3 +332,46 @@ void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, int6
FAKESUPER_XATTR, strerror(errno));
}
}
/* --fake-super replay: read the freshly-stored record and re-apply the source
* stat fd-relative. A privileged (root) run can actually change the owner;
* a non-root run silently skips the fchown on EPERM/EACCES (never fatal,
* mirroring the normal metadata identity path; other errors are logged) and
* still applies mode/mtime where permitted. */
bool fake_super_restore_fd(int fd) {
if (fd < 0)
return false;
char record[128];
ssize_t len = fgetxattr(fd, FAKESUPER_XATTR, record, sizeof(record) - 1);
if (len < 0)
return false; /* absent or filesystem without xattrs: silent no-op */
record[len] = '\0';
unsigned long ul_uid, ul_gid, ul_mode;
long long mtime_sec;
long mtime_nsec;
if (sscanf(record, "%lu:%lu:%lo:%lld:%ld", &ul_uid, &ul_gid, &ul_mode, &mtime_sec, &mtime_nsec) !=
5)
return false; /* malformed record: skip, never fatal */
/* Owner is applied best-effort only: a non-root process cannot chown and
must not abort the transfer for that reason (FastSync identity philosophy).
EPERM/EACCES (expected for a non-root receiver) are skipped silently; a
genuine EINVAL (an impossible stored id) is logged so the corruption is
not hidden. */
if (fchown(fd, (uid_t)ul_uid, (gid_t)ul_gid) != 0 && errno != EPERM && errno != EACCES)
log_message(LOG_LEVEL_WARNING, "--fake-super: could not restore owner on destination file: %s",
strerror(errno));
/* Mode is applied through the same sanitization the normal metadata path
uses (metadata_mode): group/other write bits are never granted, so a
recorded source mode of 0666 restores as 0644 — identical to a non-fake-
super --preserve run, never a privilege-granting regression. */
if (fchmod(fd, (mode_t)(ul_mode & 0777U & ~(S_IWGRP | S_IWOTH))) != 0)
log_message(LOG_LEVEL_WARNING, "--fake-super: could not restore mode on destination file: %s",
strerror(errno));
struct timespec times[2] = {{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
{.tv_sec = (time_t)mtime_sec, .tv_nsec = mtime_nsec}};
if (futimens(fd, times) != 0)
log_message(LOG_LEVEL_WARNING, "--fake-super: could not restore mtime on destination file: %s",
strerror(errno));
return true;
}
+10
View File
@@ -86,4 +86,14 @@ bool xattr_apply_fd(int fd, const FileXattrList* list);
void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, int64_t mtime_sec,
int64_t mtime_nsec);
/* --fake-super replay: parse the FAKESUPER_XATTR record previously written on
* `fd` by fake_super_store_fd and re-apply uid/gid/mode/mtime fd-relative.
* Best-effort: absence of the xattr or a malformed record is a silent no-op
* that never fails the transfer; fchown is applied only when permitted (a
* non-root EPERM/EACCES is skipped silently, matching FastSync's identity
* philosophy), and the mode is sanitized exactly like the normal metadata path
* (group/other write bits never granted). Returns true when the xattr was
* present and parsed. */
bool fake_super_restore_fd(int fd);
#endif
+432 -6
View File
@@ -3,6 +3,7 @@ import filecmp
import os
import random
import shutil
import socket
import stat
import subprocess
import sys
@@ -14,7 +15,8 @@ from common import (
PROJECT_ROOT, BUILD_DIR, TEST_DATA_DIR,
run_client, CountingProxy,
generate_test_files, verify_transfer, clean_dir, make_result,
get_dest_received_dir, CLIENT_CMD, ServerManager,
get_dest_received_dir, CLIENT_CMD, SERVER_CMD, ServerManager,
_find_free_port, _wait_for_port, _wait_proc,
)
SOURCE_DIR = os.path.join(TEST_DATA_DIR, "feature_source")
@@ -23,6 +25,31 @@ DEVICE_SOURCE = os.path.join(TEST_DATA_DIR, "device_source")
DEVICE_DEST = os.path.join(TEST_DATA_DIR, "device_dest")
def _start_captured_server(prefix=None, extra_args=None):
"""Start a plain-TCP server with captured stdout/stderr for one test.
Returns (proc, port). The caller owns `proc` and must terminate it via
`_wait_proc` so a server that ignores SIGTERM is killed instead of leaving
a zombie or raising TimeoutExpired. The shared session server discards its
output, so tests that lock in a receiver-side warning need their own. The
server's SIGTERM handler exits via `_exit`, which does not flush stdio, so
`stdbuf -oL` keeps stdout line-buffered and the warning observable."""
port = _find_free_port()
cmd = ["stdbuf", "-oL"] + (prefix or []) + SERVER_CMD + ["-p", str(port), "--allow-unauthenticated"]
if extra_args:
cmd += extra_args
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
_wait_for_port(port)
return proc, port
def _stop_captured_server(proc):
"""Terminate a captured server and return its (stdout, stderr) text."""
proc.terminate()
_wait_proc(proc)
return proc.communicate()
class TestDeviceSpecial:
"""Phase 4: --devices / --specials / -D / --copy-devices / --write-devices.
@@ -64,14 +91,56 @@ class TestDeviceSpecial:
received = get_dest_received_dir(DEVICE_DEST, DEVICE_SOURCE)
assert stat.S_ISFIFO(os.stat(os.path.join(received, "pipe.fifo")).st_mode)
def test_copy_devices_non_crash(self, shared_server):
"""--copy-devices treats a special/device source as a regular-file copy;
a FIFO (st_size 0) must transfer without hanging or crashing."""
@pytest.mark.ci
def test_specials_socket_source_skipped_safely(self):
"""A socket cannot be recreated by any standard filesystem call, so
--specials must skip it with a note and still complete the run (the
adjacent regular file transfers normally; no socket node appears)."""
self._setup()
sock_path = os.path.join(DEVICE_SOURCE, "source.sock")
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
server, port = _start_captured_server()
try:
s.bind(sock_path)
result, _ = run_client(DEVICE_SOURCE, DEVICE_DEST,
flags=["--specials"], port=port)
finally:
s.close()
out, err = _stop_captured_server(server)
assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr[:200]}"
received = get_dest_received_dir(DEVICE_DEST, DEVICE_SOURCE)
with open(os.path.join(received, "plain.txt")) as f:
assert f.read() == "regular content\n"
assert not os.path.lexists(os.path.join(received, "source.sock")), (
"socket source must be skipped, not materialized"
)
assert "socket not recreated" in (out + err), (
f"receiver did not log the documented socket skip: out={out!r} err={err!r}"
)
@pytest.mark.ci
@pytest.mark.parametrize("flags", [["--copy-devices"], ["--copy-devices", "--sendfile"]])
def test_copy_devices_fifo_becomes_regular_file(self, shared_server, flags):
"""--copy-devices treats a special source as an ordinary regular-file
copy: a FIFO (st_size 0) becomes a zero-length REGULAR file on the
destination (never a FIFO, never a hang), and the run succeeds. The
--sendfile variant previously blocked forever in the sendfile open();
the non-regular source now falls back to the buffered read path, so it
must complete within the bounded-time assertion below."""
self._setup()
os.mkfifo(os.path.join(DEVICE_SOURCE, "device_copy.fifo"))
result, _ = run_client(DEVICE_SOURCE, DEVICE_DEST,
flags=["--copy-devices"], port=shared_server.port)
result, dur = run_client(DEVICE_SOURCE, DEVICE_DEST,
flags=flags, port=shared_server.port)
assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr[:200]}"
received = get_dest_received_dir(DEVICE_DEST, DEVICE_SOURCE)
copied = os.path.join(received, "device_copy.fifo")
assert os.path.lexists(copied), "copy-devices source was not transferred"
st = os.lstat(copied)
assert stat.S_ISREG(st.st_mode), (
f"copy-devices must produce a regular file, got mode {oct(st.st_mode)}"
)
assert st.st_size == 0, f"expected a size-bounded 0-byte copy, got {st.st_size}"
assert dur < 60, f"{' '.join(flags)} hung on a FIFO source"
def test_write_devices_non_crash(self, shared_server):
"""--write-devices writes into an existing device only; when the
@@ -85,6 +154,56 @@ class TestDeviceSpecial:
flags=["--write-devices"], port=shared_server.port)
assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr[:200]}"
@pytest.mark.ci
def test_write_devices_regular_file_target_skipped(self, shared_server):
"""--write-devices only ever writes into an existing char/block node: a
pre-existing REGULAR file at the destination path is left byte-identical
(not clobbered) and the run still succeeds."""
self._setup()
received = get_dest_received_dir(DEVICE_DEST, DEVICE_SOURCE)
os.makedirs(received, exist_ok=True)
target = os.path.join(received, "plain.txt")
with open(target, "wb") as f:
f.write(b"pre-existing local content\n")
result, _ = run_client(DEVICE_SOURCE, DEVICE_DEST,
flags=["--write-devices"], port=shared_server.port)
assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr[:200]}"
with open(target, "rb") as f:
assert f.read() == b"pre-existing local content\n", (
"write-devices clobbered a non-device destination"
)
@pytest.mark.setpriv
def test_devices_nonroot_receiver_skips_safely(self):
"""A receiver without CAP_MKNOD must skip a device entry with a warning
and never abort. A root runner drops the receiver (server) to nobody
via setpriv; on a non-root runner (or without setpriv) the test skips."""
if os.geteuid() != 0 or shutil.which("setpriv") is None:
pytest.skip("requires root + setpriv to run the receiver unprivileged")
self._setup()
os.mknod(os.path.join(DEVICE_SOURCE, "chardev"), stat.S_IFCHR | 0o666,
os.makedev(1, 3))
# The unprivileged receiver must be able to create the destination tree.
os.makedirs(DEVICE_DEST, exist_ok=True)
os.chmod(DEVICE_DEST, 0o777)
server, port = _start_captured_server(
prefix=["setpriv", "--reuid=65534", "--regid=65534", "--clear-groups"])
try:
result, _ = run_client(DEVICE_SOURCE, DEVICE_DEST,
flags=["--devices"], port=port)
finally:
out, err = _stop_captured_server(server)
assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr[:300]}"
received = get_dest_received_dir(DEVICE_DEST, DEVICE_SOURCE)
with open(os.path.join(received, "plain.txt")) as f:
assert f.read() == "regular content\n"
assert not os.path.lexists(os.path.join(received, "chardev")), (
"a receiver without CAP_MKNOD must skip the device node, not create it"
)
assert "cannot create device node" in (out + err), (
f"receiver did not log the documented CAP_MKNOD skip: out={out!r} err={err!r}"
)
@pytest.mark.skipif(os.geteuid() != 0, reason="requires root to create device nodes")
def test_devices_recreates_real_char_device(self, shared_server):
"""Root-only: a source char device node is recreated on the destination
@@ -4255,6 +4374,119 @@ class TestCrtimes:
f"combined -U -N dest atime {dst_st.st_atime} != {atime}"
class TestSparse:
"""-S/--sparse: the receiver preserves holes by skipping long zero runs with
lseek (no wire change; the full image is in memory). The destination file
must round-trip its logical size and content byte-for-byte; on filesystems
that report holes (SEEK_HOLE/SEEK_DATA) we additionally assert the file is
genuinely sparse via st_blocks, but that check is tolerant (CI filesystems
may report no holes)."""
def _make_sparse_source(self, name, total, zero_start, zero_len):
source = os.path.join(TEST_DATA_DIR, name)
clean_dir(source)
sfile = os.path.join(source, "blob.bin")
with open(sfile, "wb") as f:
head = os.urandom(zero_start)
tail = os.urandom(total - zero_start - zero_len)
f.write(head)
f.write(b"\x00" * zero_len)
f.write(tail)
assert f.tell() == total
return source, sfile
@pytest.mark.parametrize("flag", ["-S", "--sparse"])
@pytest.mark.parametrize("mt", [False, True])
def test_sparse_transfer_round_trips(self, shared_server, flag, mt):
total = 4 * 1024 * 1024
source = os.path.join(TEST_DATA_DIR, f"sparse_mt{mt}_{flag.lstrip('-')}_src")
dest = os.path.join(TEST_DATA_DIR, f"sparse_mt{mt}_{flag.lstrip('-')}_dst")
clean_dir(source)
clean_dir(dest)
zero_start = 1 * 1024 * 1024
zero_len = 2 * 1024 * 1024
_, sfile = self._make_sparse_source(os.path.basename(source), total, zero_start, zero_len)
with open(sfile, "rb") as f:
src_bytes = f.read()
flags = [flag] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, \
f"{flag} transfer failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
dfile = os.path.join(received, "blob.bin")
assert os.path.getsize(dfile) == total, "logical size must match data_size"
with open(dfile, "rb") as f:
assert f.read() == src_bytes, "sparse destination content must round-trip exactly"
# Tolerant sparseness assert: if the filesystem reports holes, the file
# must actually be sparse (fewer allocated blocks than its size).
with open(dfile, "rb") as f:
off = os.lseek(f.fileno(), zero_start, os.SEEK_DATA)
if off >= 0:
hole = os.lseek(f.fileno(), off, os.SEEK_HOLE)
else:
hole = -1
if hole > zero_start:
st = os.stat(dfile)
assert st.st_blocks * 512 < total, \
f"-S file not sparse: {st.st_blocks} blocks for {total} bytes"
def test_sparse_inplace(self, shared_server):
"""--sparse must also preserve holes in the --inplace write path."""
total = 2 * 1024 * 1024
source = os.path.join(TEST_DATA_DIR, "sparse_inplace_src")
dest = os.path.join(TEST_DATA_DIR, "sparse_inplace_dst")
clean_dir(source)
clean_dir(dest)
_, sfile = self._make_sparse_source(os.path.basename(source), total, total // 2,
total // 4)
with open(sfile, "rb") as f:
src_bytes = f.read()
result, _ = run_client(source, dest, flags=["-S", "--inplace"], port=shared_server.port)
assert result.returncode == 0, \
f"-S --inplace failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
dfile = os.path.join(received, "blob.bin")
assert os.path.getsize(dfile) == total
with open(dfile, "rb") as f:
assert f.read() == src_bytes
class TestBlockSize:
"""--block-size / --delta-block: the checksum block size is genuinely honored
by the delta engine (both spellings parse to config->delta_block_size). An
end-to-end delta transfer with a non-default block size must still be
byte-exact."""
@pytest.mark.parametrize("flag", ["--block-size", "--delta-block"])
def test_non_default_block_size_delta_transfer(self, shared_server, flag):
source = os.path.join(TEST_DATA_DIR, "blocksize_delta_src")
dest = os.path.join(TEST_DATA_DIR, "blocksize_delta_dst")
clean_dir(source)
clean_dir(dest)
payload = os.urandom(300 * 1024) # enough for several 1 KiB blocks
with open(os.path.join(source, "big.bin"), "wb") as f:
f.write(payload)
# First run installs the file as the destination basis (do NOT wipe it
# afterwards: the second run's delta must be computed against it).
result, _ = run_client(source, dest, port=shared_server.port)
assert result.returncode == 0
received = get_dest_received_dir(dest, source)
# Extend the source so it differs from the installed basis: the second
# run with --delta must compute a real delta against that basis.
with open(os.path.join(source, "big.bin"), "ab") as f:
f.write(os.urandom(4096))
result, _ = run_client(source, dest,
flags=["--incremental", "--delta", flag, "1024"],
port=shared_server.port)
assert result.returncode == 0, \
f"{flag} 1024 delta transfer failed: {(result.stderr or result.stdout)[:300]}"
with open(os.path.join(received, "big.bin"), "rb") as f:
with open(os.path.join(source, "big.bin"), "rb") as expect:
assert f.read() == expect.read()
class TestOmitTimes:
"""-O/--omit-dir-times and -J/--omit-link-times are recognized and cross the
wire as receiver-side preferences. FastSync does not currently apply dir or
@@ -4672,3 +4904,197 @@ class TestConnectivityClientOptions:
f"--blocking-io -c failed: {(result.stderr or result.stdout)[:300]}"
mismatches, missing = verify_transfer(source, get_dest_received_dir(dest, source))
assert not mismatches and not missing
DISTINCT_MTIME = 1_000_000_000 # 2001-09-09T01:46:40Z; a whole second
class TestDirectoryAndSymlinkTimes:
"""P7 Wave D: -O/--omit-dir-times and -J/--omit-link-times are real.
FastSync now captures and applies directory mtimes (deferred to the end of
the transfer, after children) and symlink mtimes (immediate, via no-follow
primitives). -O/-J suppress exactly their own class of times.
"""
def _tree(self, name):
source = os.path.join(TEST_DATA_DIR, name + "_src")
dest = os.path.join(TEST_DATA_DIR, name + "_dst")
clean_dir(source)
clean_dir(dest)
os.makedirs(os.path.join(source, "sub", "deep"), exist_ok=True)
with open(os.path.join(source, "sub", "file.txt"), "wb") as fh:
fh.write(b"content\n")
with open(os.path.join(source, "sub", "deep", "deep.txt"), "wb") as fh:
fh.write(b"deeper\n")
dirs = (source, os.path.join(source, "sub"), os.path.join(source, "sub", "deep"))
for d in dirs:
os.utime(d, (DISTINCT_MTIME, DISTINCT_MTIME))
if abs(os.stat(source).st_mtime - DISTINCT_MTIME) > 2:
pytest.skip("filesystem does not preserve directory mtimes")
return source, dest, ("", "sub", os.path.join("sub", "deep"))
def _link_tree(self, name):
source = os.path.join(TEST_DATA_DIR, name + "_src")
dest = os.path.join(TEST_DATA_DIR, name + "_dst")
clean_dir(source)
clean_dir(dest)
os.makedirs(os.path.join(source, "sub"), exist_ok=True)
with open(os.path.join(source, "sub", "file.txt"), "wb") as fh:
fh.write(b"target\n")
link = os.path.join(source, "sub", "link")
# A same-directory relative target (no ".."): FastSync refuses an
# escaping/ambiguous symlink target, and ".." is a deliberate divergence.
os.symlink("file.txt", link)
os.utime(link, (DISTINCT_MTIME, DISTINCT_MTIME), follow_symlinks=False)
if abs(os.lstat(link).st_mtime - DISTINCT_MTIME) > 2:
pytest.skip("filesystem does not preserve symlink mtimes")
return source, dest, os.path.join("sub", "link")
def _run(self, source, dest, flags, shared_server):
result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, \
f"{flags} failed: {(result.stderr or result.stdout)[:400]}"
return get_dest_received_dir(dest, source)
@pytest.mark.ci
@pytest.mark.parametrize("mt", [False, True])
def test_directory_mtime_round_trip(self, shared_server, mt):
source, dest, rels = self._tree("dirtime")
flags = ["-a"] + (["--threads"] if mt else [])
received = self._run(source, dest, flags, shared_server)
for rel in rels:
src_m = os.stat(os.path.join(source, rel)).st_mtime
dst_m = os.stat(os.path.join(received, rel)).st_mtime
assert abs(dst_m - src_m) < 2, \
f"dir '{rel}': source={src_m} dest={dst_m} (flags={flags})"
@pytest.mark.ci
@pytest.mark.parametrize("mt", [False, True])
def test_omit_dir_times_suppresses_only_dirs(self, shared_server, mt):
source, dest, rels = self._tree("omitdir")
flags = ["-a", "-O"] + (["--threads"] if mt else [])
received = self._run(source, dest, flags, shared_server)
for rel in rels:
dst_m = os.stat(os.path.join(received, rel)).st_mtime
assert abs(dst_m - DISTINCT_MTIME) > 5, \
f"-O must not apply directory times ('{rel}' got {dst_m})"
@pytest.mark.ci
@pytest.mark.parametrize("mt", [False, True])
def test_symlink_mtime_round_trip(self, shared_server, mt):
source, dest, rel = self._link_tree("linktime")
flags = ["-a"] + (["--threads"] if mt else [])
received = self._run(source, dest, flags, shared_server)
src_link = os.path.join(source, rel)
dst_link = os.path.join(received, rel)
assert os.path.islink(dst_link), f"{dst_link} is not a symlink"
src_m = os.lstat(src_link).st_mtime
dst_m = os.lstat(dst_link).st_mtime
assert abs(dst_m - src_m) < 2, f"symlink times: source={src_m} dest={dst_m}"
@pytest.mark.ci
@pytest.mark.parametrize("mt", [False, True])
def test_omit_link_times_suppresses_only_links(self, shared_server, mt):
source, dest, rel = self._link_tree("omitlink")
flags = ["-a", "-J"] + (["--threads"] if mt else [])
received = self._run(source, dest, flags, shared_server)
dst_link = os.path.join(received, rel)
assert os.path.islink(dst_link), f"{dst_link} is not a symlink"
dst_m = os.lstat(dst_link).st_mtime
assert abs(dst_m - DISTINCT_MTIME) > 5, \
f"-J must not apply symlink times (got {dst_m})"
@pytest.mark.ci
def test_omit_flags_are_independent(self, shared_server):
"""-O suppresses only directory times and -J only symlink times: with
-O the symlink time is still preserved, and with -J the dir times are."""
source, dest, rel = self._link_tree("omitindep")
# Add a subdirectory mtime to check alongside the symlink.
sub = os.path.join(source, "sub")
os.utime(sub, (DISTINCT_MTIME, DISTINCT_MTIME))
# -O => dir times omitted, symlink time preserved.
clean_dir(dest + "_o")
recv_o = self._run(source, dest + "_o", ["-a", "-O"], shared_server)
assert abs(os.lstat(os.path.join(recv_o, rel)).st_mtime - DISTINCT_MTIME) < 2, \
"-O must not suppress symlink times"
assert abs(os.stat(os.path.join(recv_o, "sub")).st_mtime - DISTINCT_MTIME) > 5, \
"-O must suppress directory times"
# -J => symlink times omitted, dir times preserved.
clean_dir(dest + "_j")
recv_j = self._run(source, dest + "_j", ["-a", "-J"], shared_server)
assert abs(os.lstat(os.path.join(recv_j, rel)).st_mtime - DISTINCT_MTIME) > 5, \
"-J must suppress symlink times"
assert abs(os.stat(os.path.join(recv_j, "sub")).st_mtime - DISTINCT_MTIME) < 2, \
"-J must not suppress directory times"
@pytest.mark.ci
@pytest.mark.parametrize("mt", [False, True])
def test_preserve_does_not_create_empty_source_dir(self, shared_server, mt):
"""P7 Wave D #1: a captured-but-EMPTY source directory is never created
at the destination. The scanner records its time (it is transmitted via
STATUS_DIR_TIMES), but the receiver treats that entry as record-only, so
`-a` keeps the documented "empty dirs are never transferred" behavior."""
source = os.path.join(TEST_DATA_DIR, f"empty_dir_{'m' if mt else 's'}_src")
dest = os.path.join(TEST_DATA_DIR, f"empty_dir_{'m' if mt else 's'}_dst")
clean_dir(source)
clean_dir(dest)
with open(os.path.join(source, "keep.txt"), "wb") as fh:
fh.write(b"regular file\n")
os.makedirs(os.path.join(source, "empty_sub"))
flags = ["-a"] + (["--threads"] if mt else [])
received = self._run(source, dest, flags, shared_server)
assert os.path.isfile(os.path.join(received, "keep.txt")), "regular file missing"
assert not os.path.lexists(os.path.join(received, "empty_sub")), \
f"-a created an empty source directory at {received}/empty_sub"
@pytest.mark.ci
@pytest.mark.parametrize("mt", [False, True])
def test_prune_empty_dirs_still_does_not_create_empty_dir(self, shared_server, mt):
"""P7 Wave D #1: `-a -m` (--prune-empty-dirs) keeps its semantics -- a
captured empty directory is never created even though its time is
recorded."""
source = os.path.join(TEST_DATA_DIR, f"prune_empty_{'m' if mt else 's'}_src")
dest = os.path.join(TEST_DATA_DIR, f"prune_empty_{'m' if mt else 's'}_dst")
clean_dir(source)
clean_dir(dest)
with open(os.path.join(source, "keep.txt"), "wb") as fh:
fh.write(b"regular file\n")
os.makedirs(os.path.join(source, "empty_sub"))
flags = ["-a", "-m"] + (["--threads"] if mt else [])
received = self._run(source, dest, flags, shared_server)
assert os.path.isfile(os.path.join(received, "keep.txt")), "regular file missing"
assert not os.path.lexists(os.path.join(received, "empty_sub")), \
f"-a -m created an empty source directory at {received}/empty_sub"
@pytest.mark.ci
@pytest.mark.parametrize("mt", [False, True])
def test_collision_at_dir_time_path_does_not_abort(self, shared_server, mt):
"""P7 Wave D #1: a pre-existing regular file at a source-empty-dir's
mirror path must not abort the transfer (the old mkdir failed and failed
the run) and must not be clobbered."""
source = os.path.join(TEST_DATA_DIR, f"dirtime_collide_{'m' if mt else 's'}_src")
dest = os.path.join(TEST_DATA_DIR, f"dirtime_collide_{'m' if mt else 's'}_dst")
clean_dir(source)
clean_dir(dest)
with open(os.path.join(source, "keep.txt"), "wb") as fh:
fh.write(b"regular file\n")
os.makedirs(os.path.join(source, "collide"))
# Plant a regular file at exactly the mirror path of source/collide.
received = get_dest_received_dir(dest, source)
os.makedirs(received, exist_ok=True)
blocker = os.path.join(received, "collide")
with open(blocker, "wb") as fh:
fh.write(b"pre-existing blocker\n")
flags = ["-a"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, \
f"-a aborted on a pre-existing file at an empty-dir path: " \
f"{(result.stderr or result.stdout)[:400]}"
assert os.path.isfile(blocker) and not os.path.islink(blocker), \
"the pre-existing blocker was replaced by a directory"
with open(blocker, "rb") as fh:
assert fh.read() == b"pre-existing blocker\n", "the blocker file was clobbered"
assert os.path.isfile(os.path.join(received, "keep.txt")), "regular file missing"
+3 -3
View File
@@ -94,14 +94,14 @@ def _seed_protocol_source(source):
class TestProtocol:
@pytest.mark.ci
def test_protocol_current_version_accepted(self, shared_server):
"""--protocol=2.16.0 (the current PROTOCOL_VERSION) is accepted and the
"""--protocol=2.17.0 (the current PROTOCOL_VERSION) is accepted and the
transfer completes normally."""
source = os.path.join(TEST_DATA_DIR, "proto_ok_src")
dest = os.path.join(TEST_DATA_DIR, "proto_ok_dst")
shutil.rmtree(dest, ignore_errors=True)
os.makedirs(dest)
_seed_protocol_source(source)
result, _ = run_client(source, dest, flags=["--protocol=2.16.0"],
result, _ = run_client(source, dest, flags=["--protocol=2.17.0"],
port=shared_server.port)
assert result.returncode == 0, \
f"--protocol current run failed: {(result.stderr or result.stdout)[:400]}"
@@ -118,7 +118,7 @@ class TestProtocol:
shutil.rmtree(dest, ignore_errors=True)
os.makedirs(dest)
_seed_protocol_source(source)
for bad in ("2.15.0", "2.17.0", "216", "31"):
for bad in ("2.15.0", "2.16.0", "216", "31"):
result, _ = run_client(source, dest, flags=[f"--protocol={bad}"],
port=shared_server.port)
assert result.returncode != 0, f"--protocol={bad} should be rejected"
+61 -3
View File
@@ -3,6 +3,7 @@
#include "client_validation.h"
#include "chmod.h"
#include "config.h"
#include "delta.h"
#include "file_list.h"
#include "log.h"
#include "test_utils.h"
@@ -222,7 +223,7 @@ static void test_parse_args_protocol_accept_current() {
Config* cfg = valid_client_config();
EXPECT_NOT_NULL(cfg);
char* argv_equals[] = {"fastsync", "--source-dir", "/src",
"--dest-dir", "/dst", "--protocol=2.16.0"};
"--dest-dir", "/dst", "--protocol=2.17.0"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 6, argv_equals, positional_args, &positional_count), 0);
@@ -232,7 +233,7 @@ static void test_parse_args_protocol_accept_current() {
cfg = valid_client_config();
EXPECT_NOT_NULL(cfg);
char* argv_space[] = {"fastsync", "--source-dir", "/src", "--dest-dir",
"/dst", "--protocol", "2.16.0"};
"/dst", "--protocol", "2.17.0"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 7, argv_space, positional_args, &positional_count), 0);
EXPECT_EQ_STR(cfg->version, PROTOCOL_VERSION);
@@ -242,7 +243,7 @@ static void test_parse_args_protocol_accept_current() {
/* Any --protocol value other than the current PROTOCOL_VERSION must end in
* failure (parse_args simply stores it; validate_config rejects it up front). */
static void test_parse_args_protocol_rejects_other_versions() {
static const char* const bad_versions[] = {"2.16", "2.15.0", "2.17.0", "216", "31", "abc", ""};
static const char* const bad_versions[] = {"2.16", "2.15.0", "2.16.0", "216", "31", "abc", ""};
for (size_t i = 0; i < sizeof(bad_versions) / sizeof(bad_versions[0]); i++) {
Config* cfg = valid_client_config();
EXPECT_NOT_NULL(cfg);
@@ -2908,6 +2909,62 @@ static void test_parse_args_password_file() {
config_delete(cfg);
}
/* --block-size (Delta block size): --block-size/--delta-block set
* config->delta_block_size, out-of-range values are rejected with the default
* kept, and the configured size genuinely reaches the delta engine (a larger
* block yields fewer signature blocks for identical data). */
static void test_parse_args_block_size() {
Config* cfg = config_create();
cfg->send_directory = str_dup("/src");
cfg->receive_root_directory = str_dup("/dst");
int positional_args[2];
int positional_count = 0;
char* argv_long[] = {"fastsync", "--block-size", "4096", "/src", "/dst"};
EXPECT_EQ_INT(parse_args(cfg, 5, argv_long, positional_args, &positional_count), 0);
EXPECT_EQ_INT((int)cfg->delta_block_size, 4096);
cfg->delta_block_size = DELTA_BLOCK_SIZE_DEFAULT;
char* argv_delta[] = {"fastsync", "--delta-block", "2048", "/src", "/dst"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, argv_delta, positional_args, &positional_count), 0);
EXPECT_EQ_INT((int)cfg->delta_block_size, 2048);
/* Inline =SIZE forms (the documented rsync spelling) are accepted too. */
cfg->delta_block_size = DELTA_BLOCK_SIZE_DEFAULT;
char* argv_eq[] = {"fastsync", "--block-size=8192", "/src", "/dst"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv_eq, positional_args, &positional_count), 0);
EXPECT_EQ_INT((int)cfg->delta_block_size, 8192);
/* Out of range: parsed, warned, and the default is kept. */
cfg->delta_block_size = DELTA_BLOCK_SIZE_DEFAULT;
char* argv_bad[] = {"fastsync", "--block-size", "1", "/src", "/dst"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, argv_bad, positional_args, &positional_count), 0);
EXPECT_EQ_INT((int)cfg->delta_block_size, (int)DELTA_BLOCK_SIZE_DEFAULT);
/* A non-default block size changes the number of signature blocks for
identical data: block_count = ceil(size / block_size). */
const char data[10000] = {0};
DeltaSignature* small = delta_signature_create_seeded(data, sizeof(data), 1024, 0);
DeltaSignature* large = delta_signature_create_seeded(data, sizeof(data), 8192, 0);
EXPECT_NOT_NULL(small);
EXPECT_NOT_NULL(large);
/* cppcheck-suppress knownConditionTrueFalse -- EXPECT_NOT_NULL above asserts,
but cppcheck cannot see through the macro; the guard is defensive. */
if (small && large) {
EXPECT_TRUE(large->block_size == 8192 && small->block_size == 1024);
EXPECT_TRUE(large->block_count < small->block_count);
EXPECT_EQ_INT((int)small->block_count, 10); /* ceil(10000/1024) */
EXPECT_EQ_INT((int)large->block_count, 2); /* ceil(10000/8192) */
}
delta_signature_destroy(small);
delta_signature_destroy(large);
config_delete(cfg);
}
void test_client_cli() {
test_validate_config_required_paths();
test_parse_args_numeric_ids();
@@ -2918,6 +2975,7 @@ void test_client_cli() {
test_parse_args_rejects_malformed_identity();
test_parse_args_preallocate();
test_parse_args_metadata_times();
test_parse_args_block_size();
test_parse_args_devices_specials();
test_parse_args_atimes_long_and_short();
test_parse_args_omit_link_times_long();
+157
View File
@@ -1,5 +1,10 @@
#ifndef _GNU_SOURCE
#define _GNU_SOURCE /* SEEK_HOLE/SEEK_DATA for the sparse-hole sparseness check */
#endif
#include "test_file.h"
#include "file.h"
#include "file_store.h"
#include "file_receive.h"
#include "data.h"
#include "config.h"
#include "utils.h"
@@ -33,6 +38,7 @@ static void test_file_special_rdev_valid() {
mode_t fake_char = S_IFCHR | 0600;
mode_t fake_blk = S_IFBLK | 0600;
mode_t fake_fifo = S_IFIFO | 0600;
mode_t fake_sock = S_IFSOCK | 0600;
/* char/block devices: accept a legal pair, reject negative / oversized. */
EXPECT_TRUE(file_special_rdev_valid(1, 3, fake_char));
EXPECT_TRUE(file_special_rdev_valid(0xffff, 0x00ffffff, fake_blk));
@@ -43,6 +49,8 @@ static void test_file_special_rdev_valid() {
/* FIFOs/sockets must carry an empty rdev. */
EXPECT_TRUE(file_special_rdev_valid(0, 0, fake_fifo));
EXPECT_FALSE(file_special_rdev_valid(1, 0, fake_fifo));
EXPECT_TRUE(file_special_rdev_valid(0, 0, fake_sock));
EXPECT_FALSE(file_special_rdev_valid(0, 1, fake_sock));
EXPECT_FALSE(file_special_rdev_valid(0, 0, (mode_t)(S_IFREG | 0600)));
}
@@ -1216,6 +1224,152 @@ void test_trust_sender() {
file_set_authorized_root(-1, NULL);
}
/* --sparse/-S hole preservation: a buffer with a long zero run written via
* file_store_write_secure(sparse=true) must round-trip its content exactly and
* have the right logical size, and should additionally be genuinely sparse on
* filesystems that support holes. The sparseness assertion is tolerant: if the
* filesystem reports no holes (SEEK_HOLE/SEEK_DATA -> ENXIO) we skip the strict
* block-count check, but content and size always hold. */
static void test_file_write_to_disk_sparse_preserves_holes() {
const char* path = "test_sparse_file.bin";
unlink(path);
/* 256 KiB with a 128 KiB zero run in the middle, bracketed by headers/tails. */
const unsigned long long size = 256u * 1024u;
unsigned char* buf = malloc(size);
EXPECT_NOT_NULL(buf);
/* cppcheck-suppress knownConditionTrueFalse -- EXPECT_NOT_NULL above asserts,
but cppcheck cannot see through the macro; the guard is defensive. */
if (!buf)
return;
memset(buf, 0, size);
for (unsigned long long i = 0; i < 4096; i++) {
buf[i] = (unsigned char)(i % 251);
buf[size - 1 - i] = (unsigned char)((i * 7) % 253);
}
EXPECT_TRUE(file_store_write_secure(path, buf, size, false, true, NULL, false));
/* Logical size must equal data_size exactly. */
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
EXPECT_EQ_INT((int)st.st_size, (int)size);
/* Content must round-trip exactly: the full readback must equal the original
buffer byte-for-byte (header, the hole region staying zero, and tail) —
a writer bug in the lseek-offset bookkeeping would show up here. */
int fd = open(path, O_RDONLY);
EXPECT_TRUE(fd >= 0);
/* cppcheck-suppress knownConditionTrueFalse -- EXPECT_TRUE above asserts,
but cppcheck cannot see through the macro; the guard is defensive. */
if (fd >= 0) {
unsigned char* readback = malloc(size);
if (readback) {
unsigned long long got = 0;
while (got < size) {
ssize_t n = read(fd, readback + got, (size_t)(size - got));
if (n <= 0)
break;
got += (unsigned long long)n;
}
EXPECT_EQ_INT((int)got, (int)size);
if (got == size)
EXPECT_EQ_INT(memcmp(readback, buf, size), 0);
free(readback);
}
/* Tolerant sparseness check: seek for holes; skip if unsupported. */
off_t hole_off = lseek(fd, (off_t)4096, SEEK_HOLE);
if (hole_off >= 0 && hole_off < (off_t)size) {
off_t next_data = lseek(fd, hole_off, SEEK_DATA);
fstat(fd, &st);
int blocks = (int)(st.st_blocks * 512);
if (next_data > hole_off)
EXPECT_TRUE(blocks < (int)size);
}
close(fd);
}
free(buf);
unlink(path);
}
/* --partial retention is hard to provoke end-to-end mid-transfer (the whole
* image is in one in-memory write), so this drives the failure path directly:
* a metadata whose mtime_nsec is out of the legal [0,999999999] range makes
* futimens (in file_restore_metadata_fd) fail with EINVAL AFTER the temp has
* been fully written. With keep_partial=true the written temp must be renamed
* to the destination path (a resumable partial); with keep_partial=false the
* same failure must leave NOTHING behind. The retention is always best-effort
* (never a corrupt blend), and this asserts the both-on/off behavior. */
static void test_file_write_to_disk_partial_retention() {
const char* path = "test_partial_retention.bin";
unlink(path);
const char content[] = "partial-retention payload";
FileMetadata m;
memset(&m, 0, sizeof(m));
m.mode = 0644;
m.uid = (uid_t)geteuid();
m.gid = (gid_t)getegid();
m.mtime_sec = 1700000000;
m.mtime_nsec = 2000000000; /* invalid: forces futimens EINVAL after the write */
m.atime_valid = false;
m.crtime_valid = false;
bool ok = file_to_disk_secure_attrs(path, content, strlen(content), false, false, true, &m, false,
false, false, false, NULL, false, true, NULL);
EXPECT_FALSE(ok); /* the write itself succeeded, but metadata restore failed */
/* Retained: the already-written temp now sits at the destination path. */
int fd = open(path, O_RDONLY);
EXPECT_TRUE(fd >= 0);
/* cppcheck-suppress knownConditionTrueFalse -- EXPECT_TRUE above asserts,
but cppcheck cannot see through the macro; the guard is defensive. */
if (fd >= 0) {
char buf[64];
ssize_t n = read(fd, buf, sizeof(buf));
close(fd);
EXPECT_EQ_INT((int)strlen(content), (int)n);
if (n == (ssize_t)strlen(content))
EXPECT_TRUE(memcmp(buf, content, strlen(content)) == 0);
}
unlink(path);
/* Same failure with keep_partial=false: temp is unlinked, nothing retained. */
ok = file_to_disk_secure_attrs(path, content, strlen(content), false, false, true, &m, false,
false, false, false, NULL, false, false, NULL);
EXPECT_FALSE(ok);
EXPECT_TRUE(access(path, F_OK) == -1);
}
/* P7 Wave D: the deferred directory-time list deep-copies entries and applies
* them (fd-relative, no-follow) to an existing directory, then frees cleanly. */
static void test_dir_time_list() {
const char* root = "test_dir_time_root";
const char* sub = "test_dir_time_root/sub";
file_set_authorized_root(-1, NULL);
rmdir(sub);
rmdir(root);
EXPECT_EQ_INT(mkdir(root, 0755), 0);
EXPECT_EQ_INT(mkdir(sub, 0755), 0);
DirTimeList list;
dir_time_list_init(&list);
EXPECT_EQ_INT((int)list.count, 0);
FileMetadata metadata = {.mtime_sec = 1000000000, .mtime_nsec = 0};
EXPECT_TRUE(dir_time_list_add(&list, "sub", &metadata));
EXPECT_TRUE(dir_time_list_add(&list, "sub", &metadata));
EXPECT_EQ_INT((int)list.count, 2);
dir_time_list_apply(&list, root);
struct stat st;
EXPECT_EQ_INT(stat(sub, &st), 0);
EXPECT_EQ_INT((int)st.st_mtime, 1000000000);
dir_time_list_free(&list);
EXPECT_EQ_INT((int)list.count, 0);
EXPECT_NULL(list.paths);
EXPECT_NULL(list.entries);
rmdir(sub);
rmdir(root);
}
void test_file() {
test_file_create();
test_file_special_rdev_valid();
@@ -1230,6 +1384,8 @@ void test_file() {
test_file_save_to_disk_ignore_existing_entry_types();
test_file_save_to_disk_partial_install();
test_file_save_to_disk_reports_skips();
test_file_write_to_disk_sparse_preserves_holes();
test_file_write_to_disk_partial_retention();
test_file_write_to_disk_basic();
test_file_write_to_disk_with_fsync();
test_file_write_to_disk_preallocate_atomic();
@@ -1254,6 +1410,7 @@ void test_file() {
test_file_send_single_calls_metadata_and_path();
}
test_file_metadata_create();
test_dir_time_list();
test_inplace_overwrite_clears_special_mode_bits();
test_inplace_overwrite_metadata_strips_special_bits();
test_inplace_overwrite_truncates_shorter_payload();
+40
View File
@@ -302,6 +302,45 @@ static void test_chmod_changes() {
EXPECT_FALSE(chmod_apply(0777, "a+r,", &result));
}
/* P7 Wave D: symlink metadata is applied with no-follow primitives, and -J
* (omit_link_times) suppresses the timestamp. The positive apply path is
* asserted when the filesystem actually stores symlink timestamps; a filesystem
* that silently ignores them (or a platform where utimensat AT_SYMLINK_NOFOLLOW
* is unsupported) is tolerated, in which case only the omit-path invariant is
* checked. */
static void test_file_restore_symlink_metadata() {
const char* dir = "temp_symlink_md_test";
const char* target = "temp_symlink_md_test/target";
const char* link = "temp_symlink_md_test/link";
EXPECT_EQ_INT(mkdir(dir, 0755), 0);
FILE* f = fopen(target, "w");
EXPECT_NOT_NULL(f);
fputs("t", f);
fclose(f);
EXPECT_EQ_INT(symlink("target", link), 0);
/* Positive path: a non-omitted apply stamps the link's own mtime. */
FileMetadata applied = {.mtime_sec = 1000000000, .mtime_nsec = 0};
file_restore_symlink_metadata(link, &applied, false);
struct stat st;
EXPECT_EQ_INT(lstat(link, &st), 0);
EXPECT_TRUE(S_ISLNK(st.st_mode));
bool symlink_times_supported = ((int)st.st_mtime == 1000000000);
time_t t1 = st.st_mtime;
/* -J: a different time must be left untouched. */
FileMetadata newer = {.mtime_sec = 1234567890, .mtime_nsec = 0};
file_restore_symlink_metadata(link, &newer, true);
EXPECT_EQ_INT(lstat(link, &st), 0);
EXPECT_EQ_INT((int)st.st_mtime, (int)t1);
if (symlink_times_supported)
EXPECT_EQ_INT((int)st.st_mtime, 1000000000);
unlink(link);
unlink(target);
rmdir(dir);
}
void test_metadata() {
test_metadata_to_from_buf_roundtrip();
test_metadata_to_buf_null();
@@ -315,5 +354,6 @@ void test_metadata() {
test_file_restore_metadata_applies_atime();
test_file_restore_executability_only();
test_directory_restore_executability_only();
test_file_restore_symlink_metadata();
test_chmod_changes();
}
+56
View File
@@ -1,5 +1,6 @@
#include "test_utils.h"
#include "scanner.h"
#include "array_list.h"
#include "file.h"
#include "file_list.h"
#include "filter.h"
@@ -1262,6 +1263,60 @@ static void test_files_from_relative_send_path() {
rmdir(root);
}
/* P7 Wave D: the recursive scan captures every traversed source directory as an
* is_dir File (metadata, no payload) in the shared dir_entries list, including
* the transfer root and an EMPTY directory. The empty dir is captured even
* though the receiver deliberately never creates it, so its time can still be
* applied when the destination already holds that directory. */
static void test_scanner_captures_directory_times() {
const char* root = "test_scan_dirtime";
const char* sub = "test_scan_dirtime/sub";
const char* empty = "test_scan_dirtime/empty";
const char* file1 = "test_scan_dirtime/sub/a.txt";
EXPECT_EQ_INT(mkdir(root, 0755), 0);
EXPECT_EQ_INT(mkdir(sub, 0755), 0);
EXPECT_EQ_INT(mkdir(empty, 0755), 0);
create_test_file(file1, "x");
ArrayList* dirs = array_list_create(file_destroy);
EXPECT_NOT_NULL(dirs);
ScannerOptions options = {0};
options.use_metadata = true;
options.capture_dir_times = true;
options.dir_entries = dirs;
DirectoryScanner* scanner = directory_scanner_create_with_options(root, &options);
EXPECT_NOT_NULL(scanner);
Chunk* chunk;
while ((chunk = directory_scanner_next(scanner)) != NULL)
chunk_destroy(chunk);
EXPECT_FALSE(directory_scanner_failed(scanner));
int found_root = 0;
int found_sub = 0;
int found_empty = 0;
for (int i = 0; i < dirs->size; i++) {
const File* file = (const File*)dirs->items[i];
EXPECT_TRUE(file->is_dir);
EXPECT_NOT_NULL(file->metadata);
if (strcmp(file->path, root) == 0)
found_root = 1;
if (strcmp(file->path, sub) == 0)
found_sub = 1;
if (strcmp(file->path, empty) == 0)
found_empty = 1;
}
EXPECT_TRUE(found_root);
EXPECT_TRUE(found_sub);
EXPECT_TRUE(found_empty);
directory_scanner_destroy(scanner);
array_list_delete(dirs);
unlink(file1);
rmdir(empty);
rmdir(sub);
rmdir(root);
}
void test_scanner() {
test_scanner_single_file();
test_scanner_multiple_files();
@@ -1297,4 +1352,5 @@ void test_scanner() {
test_dirs_no_descent();
test_dirs_files_from();
test_files_from_relative_send_path();
test_scanner_captures_directory_times();
}
+52
View File
@@ -222,6 +222,57 @@ static void test_link_copy_fallback_preserves_xattrs() {
rmdir(basis_dir);
}
/* --fake-super replay: fake_super_store_fd records the source stat into the
* reserved xattr, and fake_super_restore_fd re-applies mode/mtime (and owner,
* when the process may) fd-relative. Restore must also be a safe no-op with no
* xattr present. Guarded on filesystem xattr support. */
static void test_fake_super_restore() {
const char* path = "test_fake_super_restore.txt";
unlink(path);
int fd = open(path, O_WRONLY | O_CREAT | O_TRUNC, 0600);
if (fd < 0)
return;
bool has_xattr = setxattr(path, "user.fastsync.xprobe", "p", 1, 0) == 0;
if (has_xattr)
removexattr(path, "user.fastsync.xprobe");
if (!has_xattr) {
close(fd);
unlink(path);
return; /* skip silently when the filesystem has no xattr support */
}
/* No xattr present yet: restore is a silent no-op (returns false, no crash). */
EXPECT_FALSE(fake_super_restore_fd(fd));
fake_super_store_fd(fd, 1001, 1002, 0751, 1700000000, 123456789);
EXPECT_TRUE(fake_super_restore_fd(fd));
struct stat st;
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)(st.st_mode & 07777), 0751);
/* Mode sanitization: the normal metadata path never grants group/other write
bits, and fake-super replay must not re-add them (a recorded 0666 restores
as 0644, never as world-writable). */
fake_super_store_fd(fd, 1001, 1002, 0666, 1700000000, 0);
EXPECT_TRUE(fake_super_restore_fd(fd));
EXPECT_EQ_INT(fstat(fd, &st), 0);
EXPECT_EQ_INT((int)(st.st_mode & 0777), 0644);
/* Restore with a malformed record must skip without failing. */
time_t before = st.st_mtime;
int wfd = open(path, O_RDONLY);
if (wfd >= 0) {
EXPECT_EQ_INT((int)fsetxattr(wfd, FAKESUPER_XATTR, "not-a-valid-record", 19, 0), 0);
close(wfd);
}
EXPECT_FALSE(fake_super_restore_fd(fd));
fstat(fd, &st);
EXPECT_EQ_INT((int)st.st_mtime, (int)before);
close(fd);
unlink(path);
}
void test_xattr() {
test_xattr_wire_roundtrip();
test_xattr_reject_privileged_namespace();
@@ -229,4 +280,5 @@ void test_xattr() {
test_xattr_count_bound();
test_xattr_capture_and_appliable();
test_link_copy_fallback_preserves_xattrs();
test_fake_super_restore();
}