Merge PR #329: quality cycle

CHANGELOG.md

@@ -112,8 +112,9 @@ layout is unchanged (golden length still 886).
   wording, and symlink/empty-directory quick-checks.
 - `--delete-before`'s phase-0 late-file divergence remains (rsync's pre-scan
   fixes the file list before the data pass).
-- A single file larger than 256 MiB cannot be streamed in the default path
+- A whole-file sender that cannot stream its codec (lz4's one-shot block
-  (a general whole-file limit, not basis-specific).
+  format) or a `--append`/delta source above the bound still buffers; the
+  default zstd/zlib and the uncompressed paths stream (see #318).
 - `--stats` byte totals and `--msgs2stderr` stay documented divergences.
 
 ### Security

HANDOFF.md

@@ -235,7 +235,9 @@
      and symlink/empty-dir quick-check feedback.
    - **`--delete-before` phase-0 keep-set** (rsync fixes the file list before
      the data pass; FastSync keeps its pre-scan snapshot race).
-   - **>256 MiB single-file streaming** (B4, the general whole-file limit).
+   - ~~**>256 MiB single-file streaming** (B4, the general whole-file limit).~~
+     Closed by #318: the whole-file payload, the basis read/verify and the fuzzy
+     basis are streamed through bounded buffers (lz4/append remain buffered).
    - **Wire native-size framing:** lengths are native `size_t` and the protocol
      assumes homogeneous word size/endianness — document or move to fixed-width
      framing.

README.md

@@ -206,9 +206,9 @@ This produces `./build/client` and `./build/server`. `compile_commands.json` is
 | `--incremental` | Skip files unchanged since last transfer (size + mtime). Auto-enables `--preserve`. Incompatible with `--chunk-serialization`. |
 | `--existing` | Skip files not already present at the destination; update existing files normally. |
 | `--ignore-existing` | Skip files that already exist on the receiver; like rsync it does not apply to directories or symlinks. |
-| `--compare-dest <dir>` | Extra comparison basis: unchanged files are not transferred (requires/implies `--incremental`; a basis MISS above the 256 MiB whole-file payload bound is refused — see [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md)) |
+| `--compare-dest <dir>` | Extra comparison basis: unchanged files are not transferred (requires/implies `--incremental`; a basis of any size is supported, streamed in bounded chunks — see [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md)) |
-| `--copy-dest <dir>` | Like `--compare-dest`, but copies the unchanged file from DIR into the destination (same basis-size caveat; see [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md)) |
+| `--copy-dest <dir>` | Like `--compare-dest`, but copies the unchanged file from DIR into the destination (the copy is streamed, so a basis of any size works; see [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md)) |
-| `--link-dest <dir>` | Like `--copy-dest`, but hard-links the unchanged file from DIR (repeatable; earlier DIRs win; same basis-size caveat; see [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md)) |
+| `--link-dest <dir>` | Like `--copy-dest`, but hard-links the unchanged file from DIR (repeatable; earlier DIRs win; a basis of any size works; see [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md)) |
 | `--verify-basis` | FastSync-only: require a basis hit (`--compare-dest`/`--copy-dest`/`--link-dest`) to match the source by whole-file digest instead of trusting the size+mtime quick-check (default matches rsync) |
 | `--delete` | Delete files on receiver not present in source (default timing: delete-during, matching rsync, so destination space is freed progressively). Scoped to the synchronized directories, so `--files-from` subsets are safe |
 | `--delete-before` | Delete extras before the transfer starts (implies `--delete`) |
@@ -300,6 +300,7 @@ transfer is never aborted.
 | `FASTSYNC_SOURCE_DIR` | — | Source directory fallback |
 | `FASTSYNC_DEST_DIR` | — | Destination directory fallback |
 | `FASTSYNC_SAVE_TO_DISK` | `false` | Disk persistence fallback |
+| `FASTSYNC_MAX_WHOLE_FILE_SIZE` | `268435456` | Receiver-only test hook: a byte count that lowers the whole-file streaming bound. Payloads above it are streamed through a bounded buffer. Values are clamped to the 256 MiB protocol ceiling, so it can only lower, never raise, the bound. |
 
 ## Implementation Details
 
@@ -580,9 +581,9 @@ remote SSH argv is already built injection-safe.
 | `--files-from <file>` | Read the source file list from FILE (paths relative to the source root). |
 | `-0, --from0` | Treat entries in `--files-from` files as NUL-delimited instead of newline-delimited. |
 | `--delay-updates` | Put updated files into place only at the end of the transfer (the fixed `.fastsync-stage` staging name diverges from rsync; see [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md)). |
-| `--compare-dest <dir>` | Extra comparison basis: unchanged files are not transferred (requires/implies `--incremental`; a basis MISS above the 256 MiB whole-file payload bound is refused — see [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md)). |
+| `--compare-dest <dir>` | Extra comparison basis: unchanged files are not transferred (requires/implies `--incremental`; a basis of any size is supported, streamed in bounded chunks — see [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md)). |
-| `--copy-dest <dir>` | Like `--compare-dest`, but copies the unchanged file from DIR into the destination (same basis-size caveat; see [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md)). |
+| `--copy-dest <dir>` | Like `--compare-dest`, but copies the unchanged file from DIR into the destination (the copy is streamed, so a basis of any size works; see [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md)). |
-| `--link-dest <dir>` | Like `--copy-dest`, but hard-links the unchanged file from DIR (repeatable; earlier DIRs win; same basis-size caveat; see [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md)). |
+| `--link-dest <dir>` | Like `--copy-dest`, but hard-links the unchanged file from DIR (repeatable; earlier DIRs win; a basis of any size works; see [`RSYNC_COMPAT.md`](RSYNC_COMPAT.md)). |
 | `--verify-basis` | FastSync-only: require a basis hit to match the source by whole-file digest instead of trusting the size+mtime quick-check (default matches rsync). |
 | `--preallocate` | Allocate destination file space up front (fail-fast on a full disk). |
 | `--append` | Resume a shorter destination by appending only its tail (prefix not verified; requires `--incremental`). |

rc/a.txt

@@ -0,0 +1 @@
+top

rc/link

@@ -0,0 +1 @@
+b.txt

rc/sub/b.txt

@@ -0,0 +1 @@
+nested

rd/a.txt

@@ -0,0 +1 @@
+top

rd/link

@@ -0,0 +1 @@
+b.txt

rd/sub/b.txt

@@ -0,0 +1 @@
+nested

src/client/client_cli.c

@@ -2626,6 +2626,17 @@ static int cli_finalize_config(Config* config, bool verbose, bool no_delta, bool
   bool debug_enabled = verbose || config->debug_level != 0;
   set_log_level(config->quiet ? LOG_LEVEL_ERROR
                               : (debug_enabled ? LOG_LEVEL_DEBUG : LOG_LEVEL_WARNING));
+  /* rsync parity: --delay-updates implies --delete-after.  Every staged file is
+     published first and only then are extras removed.  Normalize onto the
+     existing delete_after wire bool (no new wire field), overriding any other
+     explicit timing exactly as rsync does; without --delete there is no
+     deletion, so no timing is set (and the wire config stays valid). */
+  if (config->delay_updates && config->use_delete) {
+    config->delete_before = false;
+    config->delete_during = false;
+    config->delete_delay = false;
+    config->delete_after = true;
+  }
   /* rsync's plain --delete defaults to delete-during (--del): each directory's
      extras are removed as that directory is processed, so space is freed
      progressively and a tight destination never has to hold the whole old+new

src/client/client_manifest.c

@@ -64,16 +64,8 @@ bool add_chunk_to_manifest(ArrayList* manifest, const Chunk* chunk) {
 int send_dry_run_manifest(const Config* config) {
   int skipped = 0;
   ArrayList* missing_dest = NULL;
-  if (config->delete_missing_args) {
+  if (!client_prepare_files_from(config, &missing_dest, &skipped))
-    missing_dest = array_list_create(free);
-    if (!missing_dest)
     return -1;
-  }
-  if (!files_from_list_check(config, missing_dest, &skipped)) {
-    if (missing_dest)
-      array_list_delete(missing_dest);
-    return -1;
-  }
   PreparedScanner prepared;
   if (!prepare_scanner(config, 0, &prepared)) {
     if (missing_dest)
@@ -466,33 +458,18 @@ bool send_delete_manifest_early(Client* client, ArrayList* manifest, ArrayList*
 int send_dry_run_remote(Config* config) {
   int from_skipped = 0;
   ArrayList* missing_args = NULL;
-  if (config->delete_missing_args) {
+  if (!client_prepare_files_from(config, &missing_args, &from_skipped))
-    missing_args = array_list_create(free);
-    if (!missing_args)
     return 1;
-  }
-  if (!files_from_list_check(config, missing_args, &from_skipped)) {
-    if (missing_args)
-      array_list_delete(missing_args);
-    return 1;
-  }
   if (missing_args)
     array_list_delete(missing_args);
   /* A live session may follow, so arm graceful abort handling. */
   client_set_abort_armed(true);
-  Client* client = connect_transfer_client(config);
+  ProtocolSession session;
+  Client* client = client_connect_and_bind_session(config, &session);
   if (!client) {
-    if (config->transport == TRANSPORT_TCP)
-      log_message(LOG_LEVEL_ERROR, "could not connect to server%s",
-                  config->use_tls ? " via TLS" : "");
     client_set_abort_armed(false);
     return 1;
   }
-  ProtocolSession session;
-  protocol_session_init(&session, client->file_descriptor, client->file_descriptor);
-  protocol_session_set_io_timeout(&session, config->timeout);
-  protocol_session_set_ssl(&session, (SSL*)client->ssl);
-  protocol_session_bind(&session);
 
   int ret = 1;
   bool partial = false;

src/client/client_send.c

@@ -136,6 +136,50 @@ void disconnect_transfer_client(Client* client) {
   client_delete(client);
 }
 
+/* Connect the configured transport and install the per-thread protocol session
+ * on it: init with the socket fd pair, apply the I/O timeout and (when
+ * negotiated) the TLS object, then bind it to this thread.  Returns the
+ * connected client, or NULL (after logging the connect failure) when the
+ * transport could not connect. */
+Client* client_connect_and_bind_session(const Config* config, ProtocolSession* session) {
+  Client* client = connect_transfer_client(config);
+  if (!client) {
+    if (config->transport == TRANSPORT_TCP)
+      log_message(LOG_LEVEL_ERROR, "could not connect to server%s",
+                  config->use_tls ? " via TLS" : "");
+    return NULL;
+  }
+  protocol_session_init(session, client->file_descriptor, client->file_descriptor);
+  protocol_session_set_io_timeout(session, config->timeout);
+  protocol_session_set_ssl(session, (SSL*)client->ssl);
+  protocol_session_bind(session);
+  return client;
+}
+
+/* Shared --files-from/--delete-missing-args preamble: allocate the missing-args
+ * destination list when the option is set, then validate the --files-from list
+ * (collecting the destination mirrors of missing entries for the receiver's
+ * exact-deletion request).  On success the caller owns *missing_args_out (NULL
+ * when the option is off); on failure the list is freed and false is returned. */
+bool client_prepare_files_from(const Config* config, ArrayList** missing_args_out,
+                               int* skipped_out) {
+  ArrayList* missing_args = NULL;
+  if (config->delete_missing_args) {
+    missing_args = array_list_create(free);
+    if (!missing_args)
+      return false;
+  }
+  int skipped = 0;
+  if (!files_from_list_check(config, missing_args, &skipped)) {
+    if (missing_args)
+      array_list_delete(missing_args);
+    return false;
+  }
+  *missing_args_out = missing_args;
+  *skipped_out = skipped;
+  return true;
+}
+
 /* (finalize_transfer is defined after the SourceFile helpers below.) */
 
 typedef struct SourceFile {
@@ -877,6 +921,23 @@ static bool source_is_regular_file(const File* file) {
   return stat(file->path, &st) == 0 && S_ISREG(st.st_mode);
 }
 
+/* True when an over-threshold source will be sent by STREAMING from disk rather
+ * than loaded into memory: either the zero-copy sendfile path (no compression)
+ * or the sender-side streaming compressor (zstd/zlib, when this file is not on
+ * the --skip-compress list).  Otherwise the loader must materialize it. */
+static bool loader_can_stream(const Config* config, const File* file) {
+  if (!config || !file || !file->data || file->data->size <= STREAM_THRESHOLD)
+    return false;
+  if (!config->use_compression)
+    return true;
+  if (!compression_stream_compress_supported(compression_get_algo()) ||
+      config->compression_level <= 0)
+    return false;
+  int skip_count = config->skip_compress_set ? config->skip_compress_count : -1;
+  return !compression_should_skip_with_suffixes(file->path, config->skip_compress_suffixes,
+                                                skip_count);
+}
+
 static int send_chunk_with_removal(Client* client, Chunk* chunk, Config* config,
                                    ArrayList* remove_sources, TransferStats* stats) {
   /* --stderr=client: this is a frame boundary, so forward any diagnostics the
@@ -1022,20 +1083,13 @@ static int send_chunk_with_removal(Client* client, Chunk* chunk, Config* config,
 static int send_chunks_multithreaded(void* pipeline_context) {
   PipelineContextSender* context = (PipelineContextSender*)pipeline_context;
   time_t start = time(NULL);
-  Client* client = connect_transfer_client(context->config);
+  ProtocolSession session;
+  Client* client = client_connect_and_bind_session(context->config, &session);
   if (!client) {
-    if (context->config->transport == TRANSPORT_TCP)
-      log_message(LOG_LEVEL_ERROR, "could not connect to server%s",
-                  context->config->use_tls ? " via TLS" : "");
     pipeline_cancel(context);
     mark_sender_done(context);
     return thrd_error;
   }
-  ProtocolSession session;
-  protocol_session_init(&session, client->file_descriptor, client->file_descriptor);
-  protocol_session_set_io_timeout(&session, context->config->timeout);
-  protocol_session_set_ssl(&session, (SSL*)client->ssl);
-  protocol_session_bind(&session);
   client_messages_activate(true);
   if (!config_send(client->file_descriptor, context->config)) {
     pipeline_cancel(context);
@@ -1423,7 +1477,7 @@ static int load_files_multithreaded(void* pipeline_context) {
     if (!context->config->use_sendfile) {
       for (int i = 0; i < chunk->element_count; i++) {
         File* f = chunk->items[i];
-        if (f->data->size > STREAM_THRESHOLD && !context->config->use_compression)
+        if (loader_can_stream(context->config, f))
           continue;
         if (!file_load_data(f)) {
           log_message(LOG_LEVEL_ERROR, "Failed to load file data");
@@ -1812,7 +1866,7 @@ static bool send_files_run(Config* config, SendFilesState* state) {
       bool load_ok = true;
       for (int i = 0; i < current_chunk->element_count; i++) {
         File* f = current_chunk->items[i];
-        if (f->data->size > STREAM_THRESHOLD && !config->use_compression)
+        if (loader_can_stream(config, f))
           continue;
         if (!file_load_data(f)) {
           log_message(LOG_LEVEL_ERROR, "Failed to load file data");
@@ -2018,35 +2072,20 @@ static int send_files_impl(Config* config) {
      shielded -- rsync's `-d DIR/ --delete`. */
   state.delete_per_dir = config->use_delete && config_delete_timing_per_dir(config);
   int skipped = 0;
-  if (config->delete_missing_args) {
+  if (!client_prepare_files_from(config, &state.missing_args, &skipped))
-    state.missing_args = array_list_create(free);
-    if (!state.missing_args)
     return 1;
-  }
-  if (!files_from_list_check(config, state.missing_args, &skipped)) {
-    if (state.missing_args)
-      array_list_delete(state.missing_args);
-    return 1;
-  }
 
   /* From here on a server session may be live, so Ctrl-C/SIGTERM should set the
      abort flag (and be forwarded as STATUS_ABORT) instead of terminating. */
   client_set_abort_armed(true);
-  Client* client = connect_transfer_client(config);
+  ProtocolSession session;
+  Client* client = client_connect_and_bind_session(config, &session);
   if (!client) {
-    if (config->transport == TRANSPORT_TCP)
-      log_message(LOG_LEVEL_ERROR, "could not connect to server%s",
-                  config->use_tls ? " via TLS" : "");
     if (state.missing_args)
       array_list_delete(state.missing_args);
     return 1;
   }
   state.client = client;
-  ProtocolSession session;
-  protocol_session_init(&session, client->file_descriptor, client->file_descriptor);
-  protocol_session_set_io_timeout(&session, config->timeout);
-  protocol_session_set_ssl(&session, (SSL*)client->ssl);
-  protocol_session_bind(&session);
   client_messages_activate(true);
 
   int ret = 1;
@@ -2082,16 +2121,8 @@ static int send_files_multithreaded_impl(Config* config) {
                                           : send_dry_run_manifest(config);
   ArrayList* missing_args = NULL;
   int skipped = 0;
-  if (config->delete_missing_args) {
+  if (!client_prepare_files_from(config, &missing_args, &skipped))
-    missing_args = array_list_create(free);
-    if (!missing_args)
     return 1;
-  }
-  if (!files_from_list_check(config, missing_args, &skipped)) {
-    if (missing_args)
-      array_list_delete(missing_args);
-    return 1;
-  }
 
   /* Armed only once a session may go live (see send_files). */
   client_set_abort_armed(true);
@@ -2120,6 +2151,8 @@ static int send_files_multithreaded_impl(Config* config) {
       queue_destroy(q1);
     if (q2)
       queue_destroy(q2);
+    if (missing_args)
+      array_list_delete(missing_args);
     return 1;
   }
   PipelineContextSender* context = pipeline_context_sender_create(config, q1, q2);

src/client/client_send_internal.h

@@ -13,6 +13,7 @@
 #include "delta.h"
 #include "format.h"
 #include "log.h"
+#include "protocol.h"
 #include "scanner.h"
 #include <stdatomic.h>
 #include <stdbool.h>
@@ -91,6 +92,20 @@ void client_messages_end(void);
 /* client_send.c */
 void receive_daemon_motd(Client* client, const Config* config);
 Client* connect_transfer_client(const Config* config);
+/* Connect the configured transport and install `session` on it: init with the
+ * socket fd pair, apply the I/O timeout and (when negotiated) the TLS object,
+ * then bind the session to this thread.  Returns the connected client, or NULL
+ * after logging the connect failure.  The caller owns the client and must keep
+ * `session` alive until it calls protocol_session_unbind(). */
+Client* client_connect_and_bind_session(const Config* config, ProtocolSession* session);
+/* Shared --files-from/--delete-missing-args preamble for the send entry points:
+ * when --delete-missing-args is set, allocate the list that
+ * files_from_list_check fills with the destination mirrors of missing entries;
+ * then validate the --files-from list.  On success returns true and stores the
+ * (possibly NULL) owned list in *missing_args_out plus the skipped count; on
+ * failure returns false after freeing the list. */
+bool client_prepare_files_from(const Config* config, ArrayList** missing_args_out,
+                               int* skipped_out);
 void disconnect_transfer_client(Client* client);
 int incremental_check(Client* client, File* file, const Config* config, DeltaSignature** out_sig,
                       unsigned long long* resume_offset);

src/client/scanner.c

@@ -149,7 +149,7 @@ DirectoryScanner* directory_scanner_create_with_options(const char* root_directo
   scanner->options = *options;
   if (scanner->options.chunk_size == 0)
     scanner->options.chunk_size = DESIRED_CHUNK_SIZE;
-  scanner->directories = queue_create(100, dir_entry_destroy);
+  scanner->directories = queue_create(SCANNER_RESULT_QUEUE_CAP, dir_entry_destroy);
   if (!scanner->directories) {
     free(scanner);
     return NULL;
@@ -1026,7 +1026,7 @@ static ScannerAction scanner_process_entry(DirectoryScanner* scanner, ArrayList*
                                            Chunk** out_chunk) {
   const char* name = sorted->name;
   ScannerEntry* inspected = &sorted->entry;
-  char* cur_path = inspected->path;
+  const char* cur_path = inspected->path;
   struct stat stats = inspected->stats;
 
   /* --files-from allow-set and the filter layer apply to files and to
@@ -1101,61 +1101,23 @@ static ScannerAction scanner_process_entry(DirectoryScanner* scanner, ArrayList*
     free(rel_copy);
     return SCANNER_ACTION_CONTINUE;
   }
-  File* file = file_create(cur_path);
+  /* Entry construction (data size, -R wire path, special/devices, hardlink
-  if (file == NULL) {
+     group, metadata, xattrs) is shared with the parallel scanner. */
-    free(rel_copy);
+  File* file = NULL;
-    free(inspected->link_target);
+  bool build_failed = false;
-    inspected->link_target = NULL;
+  ScannerBuildStatus status =
-    scanner->failed = true;
+      scanner_build_file_entry(&scanner->options, inspected, rel_copy, &file, &build_failed);
-    return SCANNER_ACTION_CONTINUE;
-  }
-  if (inspected->is_symlink) {
-    file->is_symlink = true;
-    file->symlink_target = inspected->link_target;
-    inspected->link_target = NULL;
-  } else {
-    file->data->size = stats.st_size;
-  }
-  if (scanner->relative_mode) {
-    file->send_path = rel_copy;
-    rel_copy = NULL;
-  } else if (scanner->options.relative_prefix) {
-    file->send_path = scanner_prefix_send_path(scanner->options.relative_prefix, rel_copy);
   free(rel_copy);
   rel_copy = NULL;
-    if (!file->send_path) {
+  if (build_failed)
-      file_destroy(file);
     scanner->failed = true;
-      return SCANNER_ACTION_BREAK;
+  if (status == SCANNER_BUILD_SKIP)
-    }
-  }
-  /* --devices/--specials: a device/FIFO/socket entry marked for preservation
-     becomes a node to recreate (is_special, no data, rdev captured); an
-     unrequested non-regular entry is skipped (rsync default). */
-  ScannerSpecial special =
-      scanner_prepare_special(scanner->options.preserve_devices, scanner->options.preserve_specials,
-                              scanner->options.copy_devices, file, &stats);
-  if (special == SCANNER_SPECIAL_SKIP) {
-    scanner_note_nonreg(&scanner->options, file->path);
-    free(rel_copy);
-    file_destroy(file);
     return SCANNER_ACTION_CONTINUE;
-  }
+  if (status != SCANNER_BUILD_OK) {
-  if (scanner->options.hardlinks && S_ISREG(stats.st_mode))
-    scanner_assign_hardlink(scanner, scanner->options.hardlinks, file, &stats);
-  if (scanner->options.use_metadata)
-    file->metadata = file_metadata_create(file->path, &stats, scanner->options.preserve_atimes,
-                                          scanner->options.preserve_crtimes);
-  if (scanner->options.use_metadata && !file->metadata) {
-    free(rel_copy);
-    file_destroy(file);
     scanner->failed = true;
-    return SCANNER_ACTION_BREAK;
+    return status == SCANNER_BUILD_FAIL_CONTINUE ? SCANNER_ACTION_CONTINUE : SCANNER_ACTION_BREAK;
   }
-  if (!(file->link_group != 0 && !file->link_first))
-    scanner_capture_xattrs(scanner, file);
   if (!array_list_add(chunk_data, file)) {
-    free(rel_copy);
     file_destroy(file);
     scanner->failed = true;
     return SCANNER_ACTION_BREAK;
@@ -1163,14 +1125,12 @@ static ScannerAction scanner_process_entry(DirectoryScanner* scanner, ArrayList*
   scanner->current_dir_produced = true;
   *chunk_data_size += file->data->size;
   if (*chunk_data_size > scanner->options.chunk_size) {
-    free(rel_copy);
     Chunk* result = chunk_data_to_chunk(chunk_data);
     if (!result)
       scanner->failed = true;
     *out_chunk = result;
     return SCANNER_ACTION_CHUNK;
   }
-  free(rel_copy);
   return SCANNER_ACTION_CONTINUE;
 }
 

src/client/scanner.h

@@ -19,6 +19,11 @@
  * keeps one transfer from spawning an unbounded pool on a very large machine. */
 #define MAX_SCANNER_THREADS 256
 
+/* Depth of the scanner's work queues: the sequential scanner's pending-directory
+ * stack and the parallel scanner's result queue.  Bounds memory for a very wide
+ * or very deep tree while leaving ample headroom for normal scans. */
+#define SCANNER_RESULT_QUEUE_CAP 100
+
 typedef struct {
   bool use_metadata;
   /* Phase 4 metadata capture: -U/--atimes and -N/--crtimes tell the scanner to

src/client/scanner_filter.c

@@ -285,32 +285,34 @@ bool entry_passes_selection(const FileListSet* file_list, const FilterRuleList*
  * read xattrs is non-fatal: the file is transferred without them.  A symlink
  * entry reads the LINK's own xattrs (never the referent's) with the no-follow
  * variant; on Linux the VFS refuses xattrs on symlinks, so that yields NULL. */
-void scanner_capture_xattrs(const DirectoryScanner* scanner, File* file) {
+void scanner_capture_xattrs_opts(const ScannerOptions* options, File* file) {
-  if (!scanner || !file || !(scanner->options.preserve_xattrs || scanner->options.preserve_acls))
+  if (!options || !file || !(options->preserve_xattrs || options->preserve_acls))
     return;
-  file->xattrs = file->is_symlink
+  file->xattrs = file->is_symlink ? xattr_capture_path_nofollow(file->path, options->preserve_acls)
-                     ? xattr_capture_path_nofollow(file->path, scanner->options.preserve_acls)
+                                  : xattr_capture_path(file->path, options->preserve_acls);
-                     : xattr_capture_path(file->path, scanner->options.preserve_acls);
+}
+
+void scanner_capture_xattrs(const DirectoryScanner* scanner, File* file) {
+  if (!scanner)
+    return;
+  scanner_capture_xattrs_opts(&scanner->options, file);
 }
 
 /* Apply --hard-links (-H) detection to one regular File.  On a sibling (a
  * later member of an already-seen source inode) the File keeps the group id
  * and the first member's wire path but carries NO data payload (size 0); the
- * first member is left untouched (data present, link_first).  Allocation
+ * first member is left untouched (data present, link_first).  Returns false on
- * failure is fatal: the scanner is marked failed. */
+ * allocation failure (the caller marks the scan failed); the File stays usable
-void scanner_assign_hardlink(DirectoryScanner* scanner, HardLinkTable* table, File* file,
+ * either way. */
-                             const struct stat* stats) {
+bool scanner_assign_hardlink(HardLinkTable* table, File* file, const struct stat* stats) {
   if (!table || !file || !stats)
-    return;
+    return true;
   int gid;
   bool is_first;
   char* first_path = NULL;
   if (!hardlink_table_assign(table, file_wire_path(file), stats->st_dev, stats->st_ino, &gid,
-                             &is_first, &first_path)) {
+                             &is_first, &first_path))
-    if (scanner)
+    return false;
-      scanner->failed = true;
-    return;
-  }
   file->link_group = gid;
   file->link_first = is_first;
   if (!is_first) {
@@ -319,6 +321,7 @@ void scanner_assign_hardlink(DirectoryScanner* scanner, HardLinkTable* table, Fi
   } else {
     free(first_path);
   }
+  return true;
 }
 
 /* Phase 4 special/devices decision for one non-regular entry, matching rsync:
@@ -399,6 +402,76 @@ void scanner_note_nonreg(const ScannerOptions* options, const char* fs_path) {
   fflush(stdout);
 }
 
+/* Construct one non-directory File from an inspected entry.  Shared by the
+ * sequential and parallel scanners so entry construction has a single
+ * implementation: data size (or carried symlink), -R wire path, special/devices
+ * classification, hardlink group, metadata and xattr capture all happen here in
+ * the same order for both.  See the declaration for the ownership contract. */
+ScannerBuildStatus scanner_build_file_entry(const ScannerOptions* options, ScannerEntry* inspected,
+                                            const char* rel, File** out_file, bool* failed) {
+  *out_file = NULL;
+  if (failed)
+    *failed = false;
+  File* file = file_create(inspected->path);
+  if (!file) {
+    /* The File never existed, so drop the not-yet-transferred symlink target
+       here; the caller's entry teardown would otherwise double-free it. */
+    free(inspected->link_target);
+    inspected->link_target = NULL;
+    return SCANNER_BUILD_FAIL_CONTINUE;
+  }
+  if (inspected->is_symlink) {
+    file->is_symlink = true;
+    file->symlink_target = inspected->link_target;
+    inspected->link_target = NULL;
+  } else {
+    file->data->size = inspected->stats.st_size;
+  }
+  /* -R + --files-from uses the bare transfer-relative path; -R without
+     --files-from prefixes it.  Plain scans keep the source path. */
+  bool relative_mode = options->relative && options->file_list != NULL;
+  if (relative_mode) {
+    file->send_path = str_dup(rel);
+  } else if (options->relative_prefix) {
+    file->send_path = scanner_prefix_send_path(options->relative_prefix, rel);
+  }
+  if ((relative_mode || options->relative_prefix) && !file->send_path) {
+    file_destroy(file);
+    return SCANNER_BUILD_FAIL_BREAK;
+  }
+  /* --devices/--specials: a device/FIFO/socket entry marked for preservation
+     becomes a node to recreate (is_special, no data, rdev captured); an
+     unrequested non-regular entry is skipped (rsync default). */
+  ScannerSpecial special =
+      scanner_prepare_special(options->preserve_devices, options->preserve_specials,
+                              options->copy_devices, file, &inspected->stats);
+  if (special == SCANNER_SPECIAL_SKIP) {
+    scanner_note_nonreg(options, file->path);
+    file_destroy(file);
+    return SCANNER_BUILD_SKIP;
+  }
+  if (options->hardlinks && S_ISREG(inspected->stats.st_mode) &&
+      !scanner_assign_hardlink(options->hardlinks, file, &inspected->stats)) {
+    /* Allocation failure is non-fatal to this entry (it is still emitted) but
+       marks the scan failed, matching the historical inlined behaviour. */
+    if (failed)
+      *failed = true;
+  }
+  if (options->use_metadata) {
+    file->metadata = file_metadata_create(file->path, &inspected->stats, options->preserve_atimes,
+                                          options->preserve_crtimes);
+    if (!file->metadata) {
+      file_destroy(file);
+      return SCANNER_BUILD_FAIL_BREAK;
+    }
+  }
+  /* A hardlink sibling carries no data, so it carries no xattrs. */
+  if (!(file->link_group != 0 && !file->link_first))
+    scanner_capture_xattrs_opts(options, file);
+  *out_file = file;
+  return SCANNER_BUILD_OK;
+}
+
 /* rsync 3.4.1's `--info=mount` line, emitted when `-xx` drops a mount-point
  * directory: `[sender] skipping mount-point dir NAME` (the client is the
  * sender).  Plain `-x` keeps the empty directory and prints nothing, matching

src/client/scanner_internal.h

@@ -62,6 +62,17 @@ typedef enum {
   SCANNER_SPECIAL_SKIP,     /* non-regular entry not requested: skip */
 } ScannerSpecial;
 
+/* Result of scanner_build_file_entry().  The two failure variants preserve the
+ * sequential scanner's historical distinction between a failure before the
+ * File existed (which kept walking the directory) and one afterwards (which cut
+ * the chunk short); both mark the scan failed. */
+typedef enum {
+  SCANNER_BUILD_OK,            /* File built; caller owns it */
+  SCANNER_BUILD_SKIP,          /* non-regular entry not preserved; no File */
+  SCANNER_BUILD_FAIL_CONTINUE, /* failed before the File existed */
+  SCANNER_BUILD_FAIL_BREAK,    /* failed after the File existed */
+} ScannerBuildStatus;
+
 /* scanner_filter.c */
 void filter_node_destroy(void* item);
 FilterNode* filter_node_alloc(FilterNode* parent, FilterRuleList* own);
@@ -79,10 +90,21 @@ bool entry_passes_selection(const FileListSet* file_list, const FilterRuleList*
                             const FilterNode* node, const char* rel, const char* leaf, bool is_dir,
                             bool per_dir_filters, bool exclude_filter_files, bool* protect_out);
 void scanner_capture_xattrs(const DirectoryScanner* scanner, File* file);
-void scanner_assign_hardlink(DirectoryScanner* scanner, HardLinkTable* table, File* file,
+void scanner_capture_xattrs_opts(const ScannerOptions* options, File* file);
-                             const struct stat* stats);
+bool scanner_assign_hardlink(HardLinkTable* table, File* file, const struct stat* stats);
 ScannerSpecial scanner_prepare_special(bool preserve_devices, bool preserve_specials,
                                        bool copy_devices, File* file, const struct stat* stats);
+/* Build one non-directory transfer File from an inspected entry.  `rel` is the
+ * entry's transfer-root-relative path (used for the -R wire path); `inspected`
+ * supplies the on-disk path, stats and (for a carried symlink) the target whose
+ * ownership transfers to the File.  Populates data size, send_path, special-node
+ * state, hardlink group, metadata and xattrs.  On SCANNER_BUILD_OK the caller
+ * owns *out_file; on SCANNER_BUILD_SKIP it is NULL and the entry is dropped; on
+ * either failure it is NULL and the caller must mark the scan failed.  `*failed`
+ * additionally reports a non-fatal hardlink-table allocation failure, in which
+ * case a usable File is still returned. */
+ScannerBuildStatus scanner_build_file_entry(const ScannerOptions* options, ScannerEntry* inspected,
+                                            const char* rel, File** out_file, bool* failed);
 bool excluded_sink_append(ArrayList* list, mtx_t* mtx, const char* rel);
 void scanner_note_nonreg(const ScannerOptions* options, const char* fs_path);
 void scanner_note_mount(const ScannerOptions* options, const char* fs_path);

src/client/scanner_parallel.c

@@ -109,7 +109,7 @@ static void parallel_scanner_creation_failed(ParallelScanner* ps) {
 
 /* Initialize result queue and synchronization primitives. Returns true on success. */
 static bool parallel_scanner_init(ParallelScanner* ps) {
-  ps->result_queue = queue_create(100, chunk_destroy);
+  ps->result_queue = queue_create(SCANNER_RESULT_QUEUE_CAP, chunk_destroy);
   if (!ps->result_queue)
     return false;
   atomic_init(&ps->cancelled, false);
@@ -210,6 +210,157 @@ static Chunk* batch_files(ArrayList* files, unsigned long long chunk_size, Queue
   return first;
 }
 
+/* Record the delete-protection mirror of a root entry that
+ * scanner_inspect_entry() skipped (inspection == 0): a dereferenced symlink
+ * with no referent is a partial-transfer I/O error and a user-selection or size
+ * prune protects the entry's destination mirror. */
+static void scan_root_record_skipped(const ScannerOptions* options, const char* root_directory,
+                                     const char* name, const ScannerEntry* inspected,
+                                     ParallelScanner* ps) {
+  if (inspected->referent_error)
+    ps->io_error = true;
+  ArrayList* sink = NULL;
+  if (inspected->excluded)
+    sink = inspected->size_excluded ? options->size_skipped_paths : options->excluded_paths;
+  if (!sink)
+    return;
+  /* A root-level prune protects the destination mirror of the entry's wire
+     path: under -R + --files-from that is the bare relative name, otherwise it
+     is the full source path with a leading '/' removed (matching the
+     send_path/file_wire_path the scanner hands the sender). */
+  if (options->relative && options->file_list != NULL) {
+    if (!excluded_sink_append(sink, options->excluded_mutex, name))
+      ps->failed = true;
+  } else if (options->relative_prefix) {
+    char* wrel = scanner_prefix_send_path(options->relative_prefix, name);
+    if (!wrel) {
+      ps->failed = true;
+    } else {
+      if (!excluded_sink_append(sink, options->excluded_mutex, wrel))
+        ps->failed = true;
+      free(wrel);
+    }
+  } else {
+    char* abs_path = path_cat(root_directory, name);
+    if (!abs_path) {
+      ps->failed = true;
+    } else {
+      const char* rel = *abs_path == '/' ? abs_path + 1 : abs_path;
+      if (!excluded_sink_append(sink, options->excluded_mutex, rel))
+        ps->failed = true;
+      free(abs_path);
+    }
+  }
+}
+
+/* Record the delete-protection mirror of a root entry dropped by the
+ * --files-from allow-set or a filter rule.  Returns false only when the -R
+ * prefix could not be built (the caller must abandon the entry immediately);
+ * other allocation failures mark the scan failed but let the caller continue to
+ * the filter-notice step, matching the historical inlined flow. */
+static bool scan_root_record_protection(const ScannerOptions* options, const char* rel,
+                                        const char* name, const char* cur_path, bool protect,
+                                        bool passes, bool use_rel, ParallelScanner* ps) {
+  if (passes && !protect)
+    return true;
+  /* --files-from subset pruning is not a filter exclusion; -R bare-wire-path
+     exclusions are never recorded (see ScannerOptions.excluded_paths). */
+  bool files_from_prune = options->file_list && !file_list_affects(options->file_list, rel);
+  if ((!files_from_prune && !use_rel) || protect) {
+    const char* rel_path;
+    char* prefixed = NULL;
+    if (use_rel) {
+      /* -R + --files-from: the destination/wire path is the bare relative
+         name, not the source path. */
+      rel_path = rel;
+    } else if (options->relative_prefix) {
+      prefixed = scanner_prefix_send_path(options->relative_prefix, name);
+      if (!prefixed)
+        return false;
+      rel_path = prefixed;
+    } else {
+      rel_path = *cur_path == '/' ? cur_path + 1 : cur_path;
+    }
+    if (options->excluded_paths &&
+        !excluded_sink_append(options->excluded_paths, options->excluded_mutex, rel_path))
+      ps->failed = true;
+    free(prefixed);
+  }
+  return true;
+}
+
+/* Root-level directory node: apply -x/--one-file-system and either emit the
+ * mount-point directory (plain -x) or queue the directory for a worker. */
+static void scan_root_dir(const ScannerOptions* options, const char* cur_path, const char* rel,
+                          const struct stat* st, ArrayList* root_files, ArrayList* subdirs,
+                          dev_t root_dev, ParallelScanner* ps) {
+  if (!scanner_same_filesystem(options->one_file_system, root_dev, st->st_dev)) {
+    if (options->one_file_system > 1) {
+      /* -xx: drop the mount-point directory entirely (rsync) and print the
+         --info=mount line when enabled. */
+      scanner_note_mount(options, cur_path);
+      return;
+    }
+    /* -x/--one-file-system: emit the mount-point directory entry (empty) but do
+       not descend into it (see the sequential scanner for the same rule). */
+    File* mount = scanner_build_dir_file(cur_path, st, options);
+    if (!mount) {
+      ps->failed = true;
+      return;
+    }
+    if (options->relative_prefix) {
+      mount->send_path = scanner_prefix_send_path(options->relative_prefix, rel);
+      if (!mount->send_path) {
+        file_destroy(mount);
+        ps->failed = true;
+        return;
+      }
+    }
+    if (!array_list_add(root_files, mount)) {
+      file_destroy(mount);
+      ps->failed = true;
+    }
+    return;
+  }
+  char* dir = str_dup(cur_path);
+  if (!dir || !array_list_add(subdirs, dir)) {
+    free(dir);
+    ps->failed = true;
+  }
+}
+
+/* Build a non-directory root entry through the shared construction path and add
+ * it to `root_files`.  A non-regular entry the options do not preserve is
+ * dropped by the builder (which prints rsync's nonreg line); an allocation
+ * failure marks the scan failed. */
+static void scan_root_add_non_dir(const ScannerOptions* options, ScannerEntry* inspected,
+                                  const char* rel, ArrayList* root_files, ParallelScanner* ps) {
+  File* file = NULL;
+  bool failed = false;
+  ScannerBuildStatus status = scanner_build_file_entry(options, inspected, rel, &file, &failed);
+  if (failed || status == SCANNER_BUILD_FAIL_CONTINUE || status == SCANNER_BUILD_FAIL_BREAK)
+    ps->failed = true;
+  if (status != SCANNER_BUILD_OK)
+    return;
+  if (!array_list_add(root_files, file)) {
+    file_destroy(file);
+    ps->failed = true;
+  }
+}
+
+/* Regular file or carried symlink at the transfer root. */
+static void scan_root_file(const ScannerOptions* options, ScannerEntry* inspected, const char* rel,
+                           ArrayList* root_files, ParallelScanner* ps) {
+  scan_root_add_non_dir(options, inspected, rel, root_files, ps);
+}
+
+/* Device/FIFO/socket at the transfer root: recreated under --devices/--specials,
+ * otherwise dropped by the shared builder. */
+static void scan_root_special(const ScannerOptions* options, ScannerEntry* inspected,
+                              const char* rel, ArrayList* root_files, ParallelScanner* ps) {
+  scan_root_add_non_dir(options, inspected, rel, root_files, ps);
+}
+
 /* Scan one root-directory entry into either the subdirs or files list. */
 static void scan_root_entry(const ScannerOptions* options, const FilterNode* root_node,
                             const char* root_directory, const struct dirent* entry,
@@ -223,51 +374,16 @@ static void scan_root_entry(const ScannerOptions* options, const FilterNode* roo
     return;
   }
   if (inspection == 0) {
-    if (inspected.referent_error)
+    scan_root_record_skipped(options, root_directory, entry->d_name, &inspected, ps);
-      ps->io_error = true;
-    ArrayList* sink = NULL;
-    if (inspected.excluded)
-      sink = inspected.size_excluded ? options->size_skipped_paths : options->excluded_paths;
-    if (sink) {
-      /* A root-level prune protects the destination mirror of the entry's wire
-         path: under -R + --files-from that is the bare relative name, otherwise
-         it is the full source path with a leading '/' removed (matching the
-         send_path/file_wire_path the scanner hands the sender). */
-      if (options->relative && options->file_list != NULL) {
-        if (!excluded_sink_append(sink, options->excluded_mutex, entry->d_name))
-          ps->failed = true;
-      } else if (options->relative_prefix) {
-        char* wrel = scanner_prefix_send_path(options->relative_prefix, entry->d_name);
-        if (!wrel) {
-          ps->failed = true;
-        } else {
-          if (!excluded_sink_append(sink, options->excluded_mutex, wrel))
-            ps->failed = true;
-          free(wrel);
-        }
-      } else {
-        char* abs_path = path_cat(root_directory, entry->d_name);
-        if (!abs_path) {
-          ps->failed = true;
-        } else {
-          const char* rel = *abs_path == '/' ? abs_path + 1 : abs_path;
-          if (!excluded_sink_append(sink, options->excluded_mutex, rel))
-            ps->failed = true;
-          free(abs_path);
-        }
-      }
-    }
     return;
   }
   char* cur_path = inspected.path;
-  struct stat st = inspected.stats;
-  bool is_dir = inspected.is_directory;
   char* rel = str_dup(entry->d_name);
   if (!rel) {
-    free(cur_path);
     ps->failed = true;
-    return;
+    goto done;
   }
+  bool is_dir = inspected.is_directory;
   bool protect = false;
   bool passes = entry_passes_selection(options->file_list, options->base_filters, root_node, rel,
                                        entry->d_name, is_dir, options->per_dir_filters,
@@ -275,169 +391,28 @@ static void scan_root_entry(const ScannerOptions* options, const FilterNode* roo
   /* -R + --files-from: root-level files keep their bare relative send path. */
   bool use_rel = options->relative && options->file_list != NULL;
   if (!passes || protect) {
-    /* --files-from subset pruning is not a filter exclusion; -R bare-wire-path
+    if (!scan_root_record_protection(options, rel, entry->d_name, cur_path, protect, passes,
-       exclusions are never recorded (see ScannerOptions.excluded_paths). */
+                                     use_rel, ps)) {
-    bool files_from_prune = options->file_list && !file_list_affects(options->file_list, rel);
-    if ((!files_from_prune && !use_rel) || protect) {
-      const char* rel_path;
-      char* prefixed = NULL;
-      if (use_rel) {
-        /* -R + --files-from: the destination/wire path is the bare relative
-           name, not the source path. */
-        rel_path = rel;
-      } else if (options->relative_prefix) {
-        prefixed = scanner_prefix_send_path(options->relative_prefix, entry->d_name);
-        if (!prefixed) {
-          free(rel);
-          free(cur_path);
       ps->failed = true;
-          return;
+      goto done;
-        }
-        rel_path = prefixed;
-      } else {
-        rel_path = *cur_path == '/' ? cur_path + 1 : cur_path;
-      }
-      if (options->excluded_paths &&
-          !excluded_sink_append(options->excluded_paths, options->excluded_mutex, rel_path))
-        ps->failed = true;
-      free(prefixed);
     }
     if (!passes) {
       scanner_note_filter(options, entry->d_name);
-      free(rel);
+      goto done;
-      free(cur_path);
-      return;
     }
   }
   if (is_dir) {
-    if (!scanner_same_filesystem(options->one_file_system, root_dev, st.st_dev)) {
+    scan_root_dir(options, cur_path, rel, &inspected.stats, root_files, subdirs, root_dev, ps);
-      if (options->one_file_system > 1) {
+  } else if (S_ISCHR(inspected.stats.st_mode) || S_ISBLK(inspected.stats.st_mode) ||
-        /* -xx: drop the mount-point directory entirely (rsync) and print the
+             S_ISFIFO(inspected.stats.st_mode) || S_ISSOCK(inspected.stats.st_mode)) {
-           --info=mount line when enabled. */
+    scan_root_special(options, &inspected, rel, root_files, ps);
-        scanner_note_mount(options, cur_path);
+  } else {
+    scan_root_file(options, &inspected, rel, root_files, ps);
+  }
+done:
   free(rel);
   free(cur_path);
-        return;
-      }
-      /* -x/--one-file-system: emit the mount-point directory entry (empty) but
-         do not descend into it (see the sequential scanner for the same rule). */
-      File* mount = file_create(cur_path);
-      free(cur_path);
-      if (mount == NULL) {
-        free(rel);
-        ps->failed = true;
-        return;
-      }
-      mount->is_dir = true;
-      if (options->use_metadata) {
-        mount->metadata = file_metadata_create(mount->path, &st, options->preserve_atimes,
-                                               options->preserve_crtimes);
-        if (!mount->metadata) {
-          free(rel);
-          file_destroy(mount);
-          ps->failed = true;
-          return;
-        }
-      }
-      if (options->relative_prefix) {
-        mount->send_path = scanner_prefix_send_path(options->relative_prefix, rel);
-        if (!mount->send_path) {
-          free(rel);
-          file_destroy(mount);
-          ps->failed = true;
-          return;
-        }
-      }
-      free(rel);
-      if (!array_list_add(root_files, mount)) {
-        file_destroy(mount);
-        ps->failed = true;
-      }
-      return;
-    }
-    free(rel);
-    if (!array_list_add(subdirs, cur_path)) {
-      free(cur_path);
-      ps->failed = true;
-    }
-    return;
-  }
-  File* file = file_create(cur_path);
-  free(cur_path);
-  if (!file) {
-    free(rel);
   free(inspected.link_target);
-    inspected.link_target = NULL;
-    ps->failed = true;
-    return;
-  }
-  if (inspected.is_symlink) {
-    file->is_symlink = true;
-    file->symlink_target = inspected.link_target;
-    inspected.link_target = NULL;
-  } else {
-    file->data->size = st.st_size;
-  }
-  if (use_rel) {
-    file->send_path = rel;
-    rel = NULL;
-  } else if (options->relative_prefix) {
-    file->send_path = scanner_prefix_send_path(options->relative_prefix, rel);
-    free(rel);
-    rel = NULL;
-    if (!file->send_path) {
-      file_destroy(file);
-      ps->failed = true;
-      return;
-    }
-  }
-  ScannerSpecial special = scanner_prepare_special(
-      options->preserve_devices, options->preserve_specials, options->copy_devices, file, &st);
-  if (special == SCANNER_SPECIAL_SKIP) {
-    scanner_note_nonreg(ps->options, file->path);
-    free(rel);
-    file_destroy(file);
-    return;
-  }
-  if (options->hardlinks && S_ISREG(st.st_mode)) {
-    int gid;
-    bool is_first;
-    char* first_path = NULL;
-    if (!hardlink_table_assign((HardLinkTable*)options->hardlinks, file_wire_path(file), st.st_dev,
-                               st.st_ino, &gid, &is_first, &first_path)) {
-      ps->failed = true;
-    } else {
-      file->link_group = gid;
-      file->link_first = is_first;
-      if (!is_first) {
-        file->hardlink_target = first_path;
-        file->data->size = 0;
-      } else {
-        free(first_path);
-      }
-    }
-  }
-  if (options->use_metadata)
-    file->metadata =
-        file_metadata_create(file->path, &st, options->preserve_atimes, options->preserve_crtimes);
-  if (options->use_metadata && !file->metadata) {
-    free(rel);
-    file_destroy(file);
-    ps->failed = true;
-    return;
-  }
-  if ((options->preserve_xattrs || options->preserve_acls) &&
-      !(file->link_group != 0 && !file->link_first))
-    file->xattrs = file->is_symlink
-                       ? xattr_capture_path_nofollow(file->path, options->preserve_acls)
-                       : xattr_capture_path(file->path, options->preserve_acls);
-  if (!array_list_add(root_files, file)) {
-    free(rel);
-    file_destroy(file);
-    ps->failed = true;
-    return;
-  }
-  free(rel);
 }
 
 /* Scan the root directory itself, collecting root files and subdirectories.

src/server/receiver.c

@@ -797,15 +797,27 @@ int receiver_process_pending_ctx(Config* config, int file_descriptor, const Rece
     log_message(LOG_LEVEL_ERROR, "Did not receive FINISHED Status");
     goto receive_error;
   }
+  /* --delay-updates: publish every staged file BEFORE the deferred delete
+     commit, matching rsync's --delete-after ordering (all updates land first,
+     then extras are removed).  The single-threaded receiver stores files
+     synchronously, so every staged file is complete here.  The -m receiver
+     hands both publication and deletion to its caller via
+     pending_manifest/pending_plans; that caller publishes first, after its disk
+     writer has drained. */
+  bool handoff = state.pending_manifest != NULL || state.pending_plans != NULL;
+  if (!handoff && !config->dry_run && config->delay_updates && config->delay_context) {
+    if (!delay_updates_publish(config->delay_context, config)) {
+      send_status(file_descriptor, STATUS_ERROR);
+      goto fail;
+    }
+  }
   /* Commit-style (late) deletion: every data frame has been received and the
      sender proved the whole tree with STATUS_FINISHED.  The single-threaded
-     receiver stores files synchronously, so everything is on disk here and the
+     receiver stores files synchronously, so everything is on disk here (and a
-     deletion can be committed before the --delay-updates publication in
+     --delay-updates run has already published above).  The -m receiver passes
-     send_success (the walker skips the staging dir, so staged files are never
+     `pending_manifest` because its disk writer may still be draining; the
-     treated as extras).  The -m receiver passes `pending_manifest` because its
+     caller commits after the writer has joined so no extra file is removed
-     disk writer may still be draining; the caller commits after the writer has
+     unless the transfer is known to have succeeded. */
-     joined so no extra file is removed unless the transfer is known to have
-     succeeded. */
   if (state.deferred_manifest) {
     if (state.pending_manifest) {
       *state.pending_manifest = state.deferred_manifest;
@@ -989,21 +1001,10 @@ static bool receiver_send_success_frame(int fd, void* context_pointer) {
      nothing to publish and no directory times to stamp. */
   if (context->config->dry_run)
     return receiver_send_final_success(fd, context->config, &context->outcomes, final_status);
-  /* --delay-updates: the whole protocol stream (including manifest/delete
+  /* P7 Wave D: every child is now written and the --delay-updates publication
-     handling, which ran inside receiver_process) has succeeded and every
+     (done in receiver_process before the delete commit) plus the deferred
-     staged file was fully written.  Publish them atomically now, before the
+     deletion have both committed, so it is finally safe to stamp directory
-     success/outcome frame tells a --remove-source-files sender it may delete
+     times. */
-     its sources. */
-  if (context->config->delay_updates && context->config->delay_context) {
-    if (!delay_updates_publish(context->config->delay_context, context->config)) {
-      send_status(fd, STATUS_ERROR);
-      return false;
-    }
-  }
-  /* P7 Wave D: every child is now written and the delete / --delay-updates
-     phases have committed, so it is finally safe to stamp directory times.
-     This runs after the deferred deletion because receiver_process commits it
-     before calling this success frame. */
   dir_metadata_list_apply(&context->dir_times, context->config->receive_root_directory,
                           context->config);
   return receiver_send_final_success(fd, context->config, &context->outcomes, final_status);

src/server/server.c

@@ -981,12 +981,23 @@ static void server_run_mt_receiver(ServerSession* state) {
   thrd_join(writer, &writer_result);
   bool transfer_ok = receiver_result == thrd_success && writer_result == thrd_success;
   PipelineContextReceiver* context = state->context;
+  if (transfer_ok && !config->dry_run) {
+    /* --delay-updates: receive_thread has finished the whole protocol stream
+       and write_thread has drained its queue, so every staged file is complete.
+       Publish atomically BEFORE the deferred delete commit, matching rsync's
+       --delete-after ordering (all updates land first, then extras are
+       removed). */
+    if (config->delay_updates && config->delay_context &&
+        !delay_updates_publish(config->delay_context, config)) {
+      transfer_ok = false;
+    }
+  }
   if (transfer_ok && !config->dry_run) {
     /* Commit-style (late) deletion: receive_thread handed the keep-set
        manifest here instead of deleting while write_thread might still be
-       draining, so by now every file is on disk and the whole transfer is
+       draining, so by now every file is on disk (and a --delay-updates run has
-       known to have succeeded.  Remove the extras before publishing a
+       already published above) and the whole transfer is known to have
-       --delay-updates run; the walker skips the staging directory.  A
+       succeeded.  The walker skips the staging directory.  A
        server-contacting --dry-run deletes nothing (no manifest is sent). */
     if (context->deferred_manifest) {
       size_t deleted = 0;
@@ -1026,21 +1037,10 @@ static void server_run_mt_receiver(ServerSession* state) {
       delete_plan_session_destroy(context->deferred_plans);
       context->deferred_plans = NULL;
     }
-  }
+    /* P7 Wave D: all writers have joined and the --delay-updates publication
-  if (transfer_ok && !config->dry_run) {
+       plus the late deletion have committed above, so it is finally safe to
-    /* --delay-updates: receive_thread has finished the whole protocol stream
+       stamp directory times; a directory's mtime must not be clobbered by its
-       (including manifest/delete handling) and write_thread has drained its
+       children or by an extra removal. */
-       queue, so every staged file is complete.  Publish atomically before the
-       success/outcome frame so a --remove-source-files sender only learns of
-       files that were actually installed. */
-    if (config->delay_updates && config->delay_context &&
-        !delay_updates_publish(config->delay_context, config)) {
-      transfer_ok = false;
-    }
-    /* P7 Wave D: all writers have joined and the late deletion (and
-       --delay-updates publication) has committed above, so it is finally safe
-       to stamp directory times; a directory's mtime must not be clobbered by
-       its children or by an extra removal. */
     if (transfer_ok)
       dir_metadata_list_apply(&context->dir_times, config->receive_root_directory, config);
   }
@@ -1306,29 +1306,45 @@ static bool daemonize(void) {
   return true;
 }
 
-int main(int argc, char* argv[]) {
+/* Apply the process-wide policies shared by the stdio and listener
-  /* Capture the process umask now, while still single-threaded: the cached
+ * entrypoints: logging verbosity, signal handling, the parsed server
-   * value is what file_mode_base() uses, and reading it later would race with
+ * authorization policies, and the socket timeout floor.  Runs after CLI
-   * receiver threads creating files. */
+ * parsing and after the standalone --hash-credentials tool has been ruled
-  file_umask_capture();
+ * out. */
-  ServerCliOptions opts;
+static void configure_server_process(const ServerCliOptions* opts) {
-  char cli_err[512];
+  signal(SIGPIPE, SIG_IGN);
-  int parse_result = server_cli_parse(argc, argv, &opts, cli_err, sizeof(cli_err));
+  if (opts->verbose) {
-  if (parse_result == 1) {
+    set_log_level(LOG_LEVEL_DEBUG);
-    print_server_usage();
+    set_log_debug_flags(LOG_DEBUG_ALL);
-    return 0;
   }
-  if (parse_result < 0) {
+  if (opts->tls_ca && !opts->use_tls)
-    server_cli_options_free(&opts);
+    log_message(LOG_LEVEL_WARNING, "--ca has no effect without --tls");
-    fprintf(stderr, "Error: %s\n", cli_err);
+  /* Persist the parsed server policies into the process-global policy state
-    print_server_usage();
+   * BEFORE the stdio branch: an SSH-launched `--stdio` server (whose argv came
-    return 1;
+   * from the client via --remote-option and friends) must honor --allow-delete,
+   * --trust-sender and --client-cn exactly like the standalone listener. */
+  required_client_cn = opts->client_cn;
+  allow_delete = opts->allow_delete;
+  trust_sender = opts->trust_sender;
+  allow_unauthenticated = opts->allow_unauthenticated;
+  server_no_super = opts->no_super;
+  /* --stdio rejects --allow-super at parse time; force it off here as well so
+   * this process-global policy cannot be re-enabled by a future caller. */
+  server_allow_super = opts->allow_super && !opts->stdio_mode;
+  server_iconv_spec = opts->iconv_spec;
+  install_cleanup_handler(SIGINT);
+  install_cleanup_handler(SIGTERM);
+  /* Server-owned socket deadline floor: the client default --timeout=0 would
+   * otherwise leave accepted sockets without SO_RCVTIMEO/SO_SNDTIMEO and let a
+   * silent peer hold a connection (and its process slot) forever. */
+  tcp_set_timeouts(SERVER_IO_TIMEOUT_SEC, SERVER_IO_TIMEOUT_SEC);
 }
 
 /* --hash-credentials: standalone offline tool; read user:password lines and
-   * emit new-format credential-store lines, then exit. */
+ * emit new-format credential-store lines, then exit.  Consumes and frees
-  if (opts.hash_credentials_file) {
+ * opts. */
-    uint32_t iters = opts.hash_iterations_set ? opts.hash_iterations : CREDENTIAL_DEFAULT_ITERS;
+static int run_hash_credentials_tool(ServerCliOptions* opts) {
+  uint32_t iters = opts->hash_iterations_set ? opts->hash_iterations : CREDENTIAL_DEFAULT_ITERS;
   /* The output is secret material: if it is redirected to a regular file,
    * warn when that file is group/other-accessible (the store must be 0600). */
   struct stat out_st;
@@ -1338,53 +1354,29 @@ int main(int argc, char* argv[]) {
             "Warning: credential-store output is a group/other-accessible file; restrict it to "
             "mode 0600 (chmod 600)\n");
   char hash_err[512];
-    if (credentials_hash_file(opts.hash_credentials_file, iters, stdout, hash_err,
+  if (credentials_hash_file(opts->hash_credentials_file, iters, stdout, hash_err,
                             sizeof(hash_err)) != 0) {
     fprintf(stderr, "Error: %s\n", hash_err);
-      server_cli_options_free(&opts);
+    server_cli_options_free(opts);
     return 1;
   }
-    server_cli_options_free(&opts);
+  server_cli_options_free(opts);
   return 0;
 }
 
-  int exit_code = 0;
+/* SSH --stdio session: the transport is authenticated by sshd outside of
-  signal(SIGPIPE, SIG_IGN);
+ * FastSync, so the single connection is served over STDIN/STDOUT and the
-  if (opts.verbose) {
+ * process exits.  The destination root is authorized exactly like the listener
-    set_log_level(LOG_LEVEL_DEBUG);
+ * path.  Consumes and frees opts. */
-    set_log_debug_flags(LOG_DEBUG_ALL);
+static int run_stdio_server(ServerCliOptions* opts) {
-  }
-  if (opts.tls_ca && !opts.use_tls)
-    log_message(LOG_LEVEL_WARNING, "--ca has no effect without --tls");
-  /* Persist the parsed server policies into the process-global policy state
-   * BEFORE the stdio branch: an SSH-launched `--stdio` server (whose argv came
-   * from the client via --remote-option and friends) must honor --allow-delete,
-   * --trust-sender and --client-cn exactly like the standalone listener. */
-  required_client_cn = opts.client_cn;
-  allow_delete = opts.allow_delete;
-  trust_sender = opts.trust_sender;
-  allow_unauthenticated = opts.allow_unauthenticated;
-  server_no_super = opts.no_super;
-  /* --stdio rejects --allow-super at parse time; force it off here as well so
-   * this process-global policy cannot be re-enabled by a future caller. */
-  server_allow_super = opts.allow_super && !opts.stdio_mode;
-  server_iconv_spec = opts.iconv_spec;
-  install_cleanup_handler(SIGINT);
-  install_cleanup_handler(SIGTERM);
-  /* Server-owned socket deadline floor: the client default --timeout=0 would
-   * otherwise leave accepted sockets without SO_RCVTIMEO/SO_SNDTIMEO and let a
-   * silent peer hold a connection (and its process slot) forever. */
-  tcp_set_timeouts(SERVER_IO_TIMEOUT_SEC, SERVER_IO_TIMEOUT_SEC);
-
-  if (opts.stdio_mode) {
   /* SSH authenticates the stdio transport outside of FastSync. */
   allow_unauthenticated = true;
-    if (!configure_authorization(opts.destination_root)) {
+  if (!configure_authorization(opts->destination_root)) {
-      char* escaped = output_escape(opts.destination_root, false);
+    char* escaped = output_escape(opts->destination_root, false);
     fprintf(stderr, "Error: invalid destination root '%s'\n",
             escaped ? escaped : "<allocation failed>");
     free(escaped);
-      server_cli_options_free(&opts);
+    server_cli_options_free(opts);
     return 1;
   }
   io_set_fds(STDIN_FILENO, STDOUT_FILENO);
@@ -1393,34 +1385,32 @@ int main(int argc, char* argv[]) {
    * and are released by process exit. */
   handler(STDIN_FILENO);
   release_authorization();
-    server_cli_options_free(&opts);
+  server_cli_options_free(opts);
   return 0;
 }
 
-  int port = opts.port;
+/* Load the daemon config, apply --dparam overrides, resolve the effective
-  int bind_family = opts.bind_family;
+ * port/address, and surface the operator-facing module warnings.  On failure
-  const char* bind_address = opts.bind_address;
+ * the error is printed and false is returned. */
-
+static bool load_daemon_policy(ServerCliOptions* opts, int* port, const char** bind_address,
-  if (opts.daemon_mode) {
+                               char* err, size_t err_size) {
-    const char* config_path = opts.config_path ? opts.config_path : default_daemon_config_path();
+  const char* config_path = opts->config_path ? opts->config_path : default_daemon_config_path();
-    g_daemon_conf = daemon_conf_load(config_path, cli_err, sizeof(cli_err));
+  g_daemon_conf = daemon_conf_load(config_path, err, err_size);
   if (!g_daemon_conf) {
-      server_cli_options_free(&opts);
+    fprintf(stderr, "Error: %s\n", err);
-      fprintf(stderr, "Error: %s\n", cli_err);
+    return false;
-      return 1;
   }
-    for (int i = 0; i < opts.dparam_count; i++) {
+  for (int i = 0; i < opts->dparam_count; i++) {
-      if (daemon_conf_apply_dparam(g_daemon_conf, opts.dparams[i], cli_err, sizeof(cli_err)) != 0) {
+    if (daemon_conf_apply_dparam(g_daemon_conf, opts->dparams[i], err, err_size) != 0) {
-        fprintf(stderr, "Error: --dparam: %s\n", cli_err);
+      fprintf(stderr, "Error: --dparam: %s\n", err);
-        exit_code = 1;
+      return false;
-        goto out;
     }
   }
   /* Effective port: -p (highest) > --dparam port > config port (default 873). */
-    if (!opts.port_set)
+  if (!opts->port_set)
-      port = g_daemon_conf->global.port;
+    *port = g_daemon_conf->global.port;
-    if (!bind_address)
+  if (!*bind_address)
-      bind_address = g_daemon_conf->global.address;
+    *bind_address = g_daemon_conf->global.address;
   if (g_daemon_conf->module_count == 0)
     log_message(LOG_LEVEL_WARNING,
                 "daemon config has no modules; every connection will be refused");
@@ -1441,20 +1431,23 @@ int main(int argc, char* argv[]) {
                   "across all connection children)",
                   g_daemon_conf->modules[i].name, g_daemon_conf->modules[i].max_connections);
   }
-    /* Daemon credential store (Wave B).  --password-file and --early-input
+  return true;
-     * feed the same store, loaded BEFORE the listener forks so every
-     * connection child shares one read-only store.  Fail closed at startup: a
-     * module that declares `auth users` without a store (or with an empty
-     * store) refuses to start rather than serving a module whose credentials
-     * can never be verified. */
-    g_credentials =
-        credentials_load(opts.password_file, opts.early_input_file, cli_err, sizeof(cli_err));
-    if (!g_credentials) {
-      server_cli_options_free(&opts);
-      fprintf(stderr, "Error: %s\n", cli_err);
-      return 1;
 }
-    bool credential_source_given = opts.password_file != NULL || opts.early_input_file != NULL;
+
+/* Load the daemon credential store (Wave B) and enforce the fail-closed
+ * startup check: a module that declares `auth users` without a store (or with
+ * an empty store) refuses to start rather than serving a module whose
+ * credentials can never be verified.  On failure the error is printed and
+ * false is returned. */
+static bool validate_daemon_credentials(const ServerCliOptions* opts, char* err, size_t err_size) {
+  /* --password-file and --early-input feed the same store, loaded BEFORE the
+   * listener forks so every connection child shares one read-only store. */
+  g_credentials = credentials_load(opts->password_file, opts->early_input_file, err, err_size);
+  if (!g_credentials) {
+    fprintf(stderr, "Error: %s\n", err);
+    return false;
+  }
+  bool credential_source_given = opts->password_file != NULL || opts->early_input_file != NULL;
   for (int i = 0; i < g_daemon_conf->module_count; i++) {
     const DaemonModule* module = &g_daemon_conf->modules[i];
     if (module->auth_user_count == 0)
@@ -1464,16 +1457,14 @@ int main(int argc, char* argv[]) {
               "Error: module '%s' declares 'auth users' but no credential store was given "
               "(--password-file or --early-input); refusing to start (fail closed)\n",
               module->name);
-        server_cli_options_free(&opts);
+      return false;
-        return 1;
     }
     if (credentials_store_size(g_credentials) == 0) {
       fprintf(stderr,
               "Error: module '%s' declares 'auth users' but the credential store is empty; "
               "refusing to start (fail closed)\n",
               module->name);
-        server_cli_options_free(&opts);
+      return false;
-        return 1;
     }
     for (int j = 0; j < module->auth_user_count; j++) {
       if (!credentials_store_has(g_credentials, module->auth_users[j]))
@@ -1483,32 +1474,30 @@ int main(int argc, char* argv[]) {
                     module->name, module->auth_users[j]);
     }
   }
-    /* Shared cross-process registry for the per-module / per-source caps and
+  return true;
-     * the auth lockout.  Created HERE in the parent before any accept-loop
+}
-     * fork; every connection child inherits the mapping.  A failure degrades to
+
-     * "registry disabled" (the global cap and host ACLs still apply) rather
+/* Create the shared cross-process registry for the per-module / per-source
-     * than refusing to start. */
+ * caps and the auth lockout.  Called in the parent before any accept-loop fork;
-    g_daemon_limits = daemon_limits_create((int)g_daemon_conf->global.max_connections,
+ * every connection child inherits the mapping.  A failure degrades to
-                                           g_daemon_conf->module_count,
+ * "registry disabled" (the global cap and host ACLs still apply) rather than
-                                           g_daemon_conf->global.max_connections_per_host,
+ * refusing to start. */
-                                           g_daemon_conf->global.auth_lockout_threshold,
+static void create_daemon_limits(void) {
+  g_daemon_limits = daemon_limits_create(
+      (int)g_daemon_conf->global.max_connections, g_daemon_conf->module_count,
+      g_daemon_conf->global.max_connections_per_host, g_daemon_conf->global.auth_lockout_threshold,
       g_daemon_conf->global.auth_lockout_duration_sec);
   if (!g_daemon_limits)
     log_message(LOG_LEVEL_WARNING,
                 "daemon: could not allocate the shared connection registry; per-module / "
                 "per-host caps and the cross-process auth lockout are disabled (the global "
                 "'max connections' cap and host ACLs still apply)");
-  } else {
-    if (!configure_authorization(opts.destination_root)) {
-      char* escaped = output_escape(opts.destination_root, false);
-      fprintf(stderr, "Error: invalid destination root '%s'\n",
-              escaped ? escaped : "<allocation failed>");
-      free(escaped);
-      server_cli_options_free(&opts);
-      return 1;
-    }
 }
 
+/* Bind the listener, apply the daemon caps, set up TLS when requested, detach
+ * when daemonizing, and run the accept loop.  Returns the process exit code. */
+static int start_listener(ServerCliOptions* opts, int port, int bind_family,
+                          const char* bind_address) {
   ServerBindOptions bind_opts;
   bind_opts.bind_address = bind_address;
   bind_opts.family = bind_family;
@@ -1516,50 +1505,73 @@ int main(int argc, char* argv[]) {
   if (!g_server) {
     log_message(LOG_LEVEL_ERROR, "Failed to create server");
     release_authorization();
-    exit_code = 1;
+    return 1;
-    goto out;
   }
   if (g_daemon_conf)
     server_set_max_connections(g_server, (unsigned int)g_daemon_conf->global.max_connections);
   if (g_daemon_limits)
     server_set_limit_registry(g_server, g_daemon_limits);
-  if (opts.use_tls) {
+  if (opts->use_tls) {
-    if (!opts.tls_cert || !opts.tls_key || !opts.tls_ca || !opts.client_cn) {
+    if (!opts->tls_cert || !opts->tls_key || !opts->tls_ca || !opts->client_cn) {
       fprintf(stderr, "Error: --tls requires --cert, --key, --ca, and --client-cn\n");
       server_delete(&g_server);
       release_authorization();
-      exit_code = 1;
+      return 1;
-      goto out;
     }
     tls_global_init();
-    if (!server_create_tls(g_server, opts.tls_cert, opts.tls_key, opts.tls_ca)) {
+    if (!server_create_tls(g_server, opts->tls_cert, opts->tls_key, opts->tls_ca)) {
       log_message(LOG_LEVEL_ERROR, "Failed to set up TLS");
       server_delete(&g_server);
       release_authorization();
-      exit_code = 1;
+      return 1;
-      goto out;
     }
   }
-
   /* Detach after the listening socket (and TLS context) exist so the
    * background daemon inherits a fully-bound listener.  --no-detach runs in
    * the foreground, which is how tests drive the daemon. */
-  if (opts.daemon_mode && !opts.no_detach) {
+  if (opts->daemon_mode && !opts->no_detach) {
     if (!daemonize()) {
       log_message(LOG_LEVEL_ERROR, "Failed to daemonize");
       server_delete(&g_server);
       release_authorization();
-      exit_code = 1;
+      return 1;
-      goto out;
     }
   }
-
+  if (opts->use_tls)
-  if (opts.use_tls)
     server_listen_tls(g_server, handler);
   else
     server_listen(g_server, handler);
   server_delete(&g_server);
   release_authorization();
+  return 0;
+}
+
+/* Listener entrypoint: the daemon (config-driven, possibly detached) and the
+ * standalone TCP server share the same bind/TLS/listen path.  Consumes and
+ * frees opts. */
+static int run_daemon_server(ServerCliOptions* opts) {
+  int port = opts->port;
+  const char* bind_address = opts->bind_address;
+  char cli_err[512];
+  int exit_code = 0;
+
+  if (opts->daemon_mode) {
+    if (!load_daemon_policy(opts, &port, &bind_address, cli_err, sizeof(cli_err)) ||
+        !validate_daemon_credentials(opts, cli_err, sizeof(cli_err))) {
+      exit_code = 1;
+      goto out;
+    }
+    create_daemon_limits();
+  } else if (!configure_authorization(opts->destination_root)) {
+    char* escaped = output_escape(opts->destination_root, false);
+    fprintf(stderr, "Error: invalid destination root '%s'\n",
+            escaped ? escaped : "<allocation failed>");
+    free(escaped);
+    exit_code = 1;
+    goto out;
+  }
+
+  exit_code = start_listener(opts, port, opts->bind_family, bind_address);
 
 out:
   daemon_limits_destroy(g_daemon_limits);
@@ -1568,7 +1580,35 @@ out:
   g_daemon_conf = NULL;
   credentials_free(g_credentials);
   g_credentials = NULL;
-  server_cli_options_free(&opts);
+  server_cli_options_free(opts);
   return exit_code;
 }
+
+int main(int argc, char* argv[]) {
+  /* Capture the process umask now, while still single-threaded: the cached
+   * value is what file_mode_base() uses, and reading it later would race with
+   * receiver threads creating files. */
+  file_umask_capture();
+  ServerCliOptions opts;
+  char cli_err[512];
+  int parse_result = server_cli_parse(argc, argv, &opts, cli_err, sizeof(cli_err));
+  if (parse_result == 1) {
+    print_server_usage();
+    return 0;
+  }
+  if (parse_result < 0) {
+    server_cli_options_free(&opts);
+    fprintf(stderr, "Error: %s\n", cli_err);
+    print_server_usage();
+    return 1;
+  }
+
+  if (opts.hash_credentials_file)
+    return run_hash_credentials_tool(&opts);
+
+  configure_server_process(&opts);
+  if (opts.stdio_mode)
+    return run_stdio_server(&opts);
+  return run_daemon_server(&opts);
+}
 #endif

src/shared/compression.c

@@ -3,6 +3,7 @@
 #include "log.h"
 #include "protocol.h"
 #include "utils.h"
+#include <errno.h>
 #include <limits.h>
 #include <lz4.h>
 #include <stdatomic.h>
@@ -716,3 +717,379 @@ Data* data_decompress_limited(Data* compressed_data, size_t maximum_size) {
 Data* data_decompress(Data* compressed_data) {
   return data_decompress_limited(compressed_data, MAX_DECOMPRESSED_SIZE);
 }
+
+/* ---- streaming decompression ---- */
+
+#define STREAM_DECOMPRESS_OUT_CHUNK (256 * 1024)
+
+struct CompressionStreamDecompressor {
+  CompressionAlgo algo;
+  unsigned long long expected_out;
+  unsigned long long total;
+  int out_fd;
+  unsigned char* out_buf;
+  ZSTD_DCtx* dctx;
+  z_stream zs;
+  bool zs_initialized;
+  bool failed;
+};
+
+static bool stream_write_all(int fd, const void* data, size_t size) {
+  const unsigned char* p = data;
+  size_t done = 0;
+  while (done < size) {
+    ssize_t n = write(fd, p + done, size - done);
+    if (n < 0 && errno == EINTR)
+      continue;
+    if (n <= 0)
+      return false;
+    done += (size_t)n;
+  }
+  return true;
+}
+
+CompressionStreamDecompressor*
+compression_stream_decompressor_create(CompressionAlgo algo, unsigned long long expected_out) {
+  CompressionStreamDecompressor* d = calloc(1, sizeof(*d));
+  if (!d)
+    return NULL;
+  d->algo = algo;
+  d->expected_out = expected_out;
+  d->out_fd = -1;
+  d->out_buf = malloc(STREAM_DECOMPRESS_OUT_CHUNK);
+  if (!d->out_buf) {
+    free(d);
+    return NULL;
+  }
+  if (algo == COMPRESSION_ALGO_ZSTD) {
+    d->dctx = ZSTD_createDCtx();
+    if (!d->dctx) {
+      free(d->out_buf);
+      free(d);
+      return NULL;
+    }
+  } else if (algo == COMPRESSION_ALGO_ZLIB || algo == COMPRESSION_ALGO_ZLIBX) {
+    if (inflateInit(&d->zs) != Z_OK) {
+      free(d->out_buf);
+      free(d);
+      return NULL;
+    }
+    d->zs_initialized = true;
+  } else if (algo != COMPRESSION_ALGO_NONE) {
+    /* lz4's block format cannot be decompressed incrementally. */
+    free(d->out_buf);
+    free(d);
+    return NULL;
+  }
+  return d;
+}
+
+static bool stream_emit(CompressionStreamDecompressor* d, const void* buf, size_t len) {
+  if (len == 0)
+    return true;
+  if (d->expected_out != 0 && (d->total > d->expected_out || len > d->expected_out - d->total)) {
+    d->failed = true;
+    return false;
+  }
+  if (!stream_write_all(d->out_fd, buf, len)) {
+    d->failed = true;
+    return false;
+  }
+  d->total += len;
+  return true;
+}
+
+static bool stream_feed_none(CompressionStreamDecompressor* d, const void* in, size_t in_len,
+                             bool* done) {
+  if (!stream_emit(d, in, in_len))
+    return false;
+  /* NONE has no end marker; the caller knows the frame length. */
+  *done = true;
+  return true;
+}
+
+static bool stream_feed_zstd(CompressionStreamDecompressor* d, const void* in, size_t in_len,
+                             bool* done) {
+  ZSTD_inBuffer input = {in, in_len, 0};
+  while (input.pos < input.size) {
+    ZSTD_outBuffer output = {d->out_buf, STREAM_DECOMPRESS_OUT_CHUNK, 0};
+    size_t ret = ZSTD_decompressStream(d->dctx, &output, &input);
+    if (ZSTD_isError(ret)) {
+      d->failed = true;
+      return false;
+    }
+    if (!stream_emit(d, d->out_buf, output.pos))
+      return false;
+    if (ret == 0) {
+      *done = true;
+      /* Trailing bytes after a complete frame are malformed; stop consuming. */
+      if (input.pos < input.size) {
+        d->failed = true;
+        return false;
+      }
+      return true;
+    }
+  }
+  return true;
+}
+
+static bool stream_feed_zlib(CompressionStreamDecompressor* d, const void* in, size_t in_len,
+                             bool* done) {
+  d->zs.next_in = (Bytef*)in;
+  d->zs.avail_in = (uInt)in_len;
+  while (d->zs.avail_in > 0) {
+    d->zs.next_out = d->out_buf;
+    d->zs.avail_out = STREAM_DECOMPRESS_OUT_CHUNK;
+    int rc = inflate(&d->zs, Z_NO_FLUSH);
+    if (rc != Z_OK && rc != Z_STREAM_END && rc != Z_BUF_ERROR) {
+      d->failed = true;
+      return false;
+    }
+    size_t produced = STREAM_DECOMPRESS_OUT_CHUNK - d->zs.avail_out;
+    if (!stream_emit(d, d->out_buf, produced))
+      return false;
+    if (rc == Z_STREAM_END) {
+      *done = true;
+      return d->zs.avail_in == 0;
+    }
+    if (rc == Z_BUF_ERROR && produced == 0) {
+      /* Need more input. */
+      break;
+    }
+  }
+  return true;
+}
+
+bool compression_stream_decompressor_feed(CompressionStreamDecompressor* d, const void* in,
+                                          size_t in_len, int out_fd, bool* done) {
+  if (!d || d->failed)
+    return false;
+  d->out_fd = out_fd;
+  if (done)
+    *done = false;
+  switch (d->algo) {
+  case COMPRESSION_ALGO_NONE:
+    return stream_feed_none(d, in, in_len, done);
+  case COMPRESSION_ALGO_ZSTD:
+    return stream_feed_zstd(d, in, in_len, done);
+  case COMPRESSION_ALGO_ZLIB:
+  case COMPRESSION_ALGO_ZLIBX:
+    return stream_feed_zlib(d, in, in_len, done);
+  case COMPRESSION_ALGO_LZ4:
+    break;
+  }
+  d->failed = true;
+  return false;
+}
+
+unsigned long long compression_stream_decompressor_total(const CompressionStreamDecompressor* d) {
+  return d ? d->total : 0;
+}
+
+void compression_stream_decompressor_destroy(CompressionStreamDecompressor* d) {
+  if (!d)
+    return;
+  if (d->dctx)
+    ZSTD_freeDCtx(d->dctx);
+  if (d->zs_initialized)
+    inflateEnd(&d->zs);
+  free(d->out_buf);
+  free(d);
+}
+
+/* ---- streaming compression ---- */
+
+struct CompressionStreamCompressor {
+  CompressionAlgo algo;
+  int level;
+  ZSTD_CCtx* cctx;
+  z_stream zs;
+  bool zs_initialized;
+  unsigned char* out_buf;
+  bool failed;
+};
+
+bool compression_stream_compress_supported(CompressionAlgo algo) {
+  return algo == COMPRESSION_ALGO_ZSTD || algo == COMPRESSION_ALGO_ZLIB ||
+         algo == COMPRESSION_ALGO_ZLIBX;
+}
+
+CompressionStreamCompressor* compression_stream_compressor_create(CompressionAlgo algo, int level,
+                                                                  int threads) {
+  (void)threads;
+  if (!compression_algo_valid((int)algo) || algo == COMPRESSION_ALGO_LZ4)
+    return NULL;
+  CompressionStreamCompressor* c = calloc(1, sizeof(*c));
+  if (!c)
+    return NULL;
+  c->algo = algo;
+  c->level = level;
+  c->out_buf = malloc(STREAM_DECOMPRESS_OUT_CHUNK);
+  if (!c->out_buf) {
+    free(c);
+    return NULL;
+  }
+  if (algo == COMPRESSION_ALGO_ZSTD) {
+    c->cctx = ZSTD_createCCtx();
+    if (!c->cctx) {
+      free(c->out_buf);
+      free(c);
+      return NULL;
+    }
+  } else if (algo == COMPRESSION_ALGO_ZLIB || algo == COMPRESSION_ALGO_ZLIBX) {
+    if (deflateInit(&c->zs, level < 1 ? Z_DEFAULT_COMPRESSION : level) != Z_OK) {
+      free(c->out_buf);
+      free(c);
+      return NULL;
+    }
+    c->zs_initialized = true;
+  }
+  return c;
+}
+
+bool compression_stream_compressor_begin(CompressionStreamCompressor* c,
+                                         unsigned long long raw_size, int out_fd) {
+  if (!c || c->failed)
+    return false;
+  unsigned char hdr[1 + 4];
+  size_t hdr_len = 1;
+  hdr[0] = (unsigned char)c->algo;
+  if (c->algo == COMPRESSION_ALGO_ZLIB || c->algo == COMPRESSION_ALGO_ZLIBX) {
+    uint32_t size32 = raw_size > UINT32_MAX ? UINT32_MAX : (uint32_t)raw_size;
+    for (int i = 0; i < 4; i++)
+      hdr[1 + i] = (uint8_t)((size32 >> (8 * i)) & 0xff);
+    hdr_len = 5;
+  }
+  if (c->algo == COMPRESSION_ALGO_ZSTD) {
+    /* Pledge the source size and force the frame content-size field so the
+       receiver can decide whether to stream from the frame header alone. */
+    if (ZSTD_isError(ZSTD_CCtx_setPledgedSrcSize(c->cctx, raw_size)) ||
+        ZSTD_isError(ZSTD_CCtx_setParameter(c->cctx, ZSTD_c_compressionLevel, c->level)) ||
+        ZSTD_isError(ZSTD_CCtx_setParameter(c->cctx, ZSTD_c_contentSizeFlag, 1))) {
+      c->failed = true;
+      return false;
+    }
+    if (ZSTD_isError(ZSTD_CCtx_setParameter(c->cctx, ZSTD_c_checksumFlag, 0))) {
+      c->failed = true;
+      return false;
+    }
+  }
+  if (!stream_write_all(out_fd, hdr, hdr_len)) {
+    c->failed = true;
+    return false;
+  }
+  return true;
+}
+
+static bool stream_compress_zlib(CompressionStreamCompressor* c, const void* in, size_t in_len,
+                                 int out_fd, int flush) {
+  c->zs.next_in = (Bytef*)in;
+  c->zs.avail_in = (uInt)in_len;
+  do {
+    c->zs.next_out = c->out_buf;
+    c->zs.avail_out = STREAM_DECOMPRESS_OUT_CHUNK;
+    int rc = deflate(&c->zs, flush);
+    if (rc != Z_OK && rc != Z_STREAM_END && rc != Z_BUF_ERROR) {
+      c->failed = true;
+      return false;
+    }
+    size_t produced = STREAM_DECOMPRESS_OUT_CHUNK - c->zs.avail_out;
+    if (!stream_write_all(out_fd, c->out_buf, produced)) {
+      c->failed = true;
+      return false;
+    }
+    if (rc == Z_STREAM_END)
+      return true;
+    if (rc == Z_BUF_ERROR && produced == 0)
+      break;
+  } while (c->zs.avail_in > 0 || flush == Z_FINISH);
+  return true;
+}
+
+bool compression_stream_compressor_feed(CompressionStreamCompressor* c, const void* in,
+                                        size_t in_len, int out_fd) {
+  if (!c || c->failed)
+    return false;
+  if (c->algo == COMPRESSION_ALGO_NONE)
+    return stream_write_all(out_fd, in, in_len);
+  if (c->algo == COMPRESSION_ALGO_ZSTD) {
+    ZSTD_inBuffer input = {in, in_len, 0};
+    while (input.pos < input.size) {
+      ZSTD_outBuffer output = {c->out_buf, STREAM_DECOMPRESS_OUT_CHUNK, 0};
+      size_t ret = ZSTD_compressStream2(c->cctx, &output, &input, ZSTD_e_continue);
+      if (ZSTD_isError(ret)) {
+        c->failed = true;
+        return false;
+      }
+      if (!stream_write_all(out_fd, c->out_buf, output.pos)) {
+        c->failed = true;
+        return false;
+      }
+      if (output.pos == 0 && input.pos < input.size)
+        break; /* avoid spinning; zstd buffers the rest internally */
+    }
+    return true;
+  }
+  return stream_compress_zlib(c, in, in_len, out_fd, Z_NO_FLUSH);
+}
+
+bool compression_stream_compressor_finish(CompressionStreamCompressor* c, int out_fd) {
+  if (!c || c->failed)
+    return false;
+  if (c->algo == COMPRESSION_ALGO_NONE)
+    return true;
+  if (c->algo == COMPRESSION_ALGO_ZSTD) {
+    size_t ret;
+    do {
+      ZSTD_inBuffer input = {NULL, 0, 0};
+      ZSTD_outBuffer output = {c->out_buf, STREAM_DECOMPRESS_OUT_CHUNK, 0};
+      ret = ZSTD_compressStream2(c->cctx, &output, &input, ZSTD_e_end);
+      if (ZSTD_isError(ret)) {
+        c->failed = true;
+        return false;
+      }
+      if (!stream_write_all(out_fd, c->out_buf, output.pos)) {
+        c->failed = true;
+        return false;
+      }
+    } while (ret > 0);
+    return true;
+  }
+  return stream_compress_zlib(c, NULL, 0, out_fd, Z_FINISH);
+}
+
+void compression_stream_compressor_destroy(CompressionStreamCompressor* c) {
+  if (!c)
+    return;
+  if (c->cctx)
+    ZSTD_freeCCtx(c->cctx);
+  if (c->zs_initialized)
+    deflateEnd(&c->zs);
+  free(c->out_buf);
+  free(c);
+}
+
+unsigned long long compression_peek_frame_content_size(const void* buf, size_t len) {
+  if (!buf || len < 1)
+    return 0;
+  const uint8_t* p = buf;
+  uint8_t codec = p[0];
+  if (!compression_algo_valid(codec))
+    return 0;
+  if (codec == (uint8_t)COMPRESSION_ALGO_NONE)
+    return len - 1;
+  if (codec == (uint8_t)COMPRESSION_ALGO_ZSTD) {
+    if (len < 2)
+      return 0;
+    unsigned long long size = ZSTD_getFrameContentSize(p + 1, len - 1);
+    if (size == ZSTD_CONTENTSIZE_ERROR || size == ZSTD_CONTENTSIZE_UNKNOWN)
+      return 0;
+    return size;
+  }
+  if (len < 1 + LZ4_SIZE_PREFIX_LEN)
+    return 0;
+  uint32_t raw = 0;
+  for (int i = 0; i < LZ4_SIZE_PREFIX_LEN; i++)
+    raw |= (uint32_t)p[1 + i] << (8 * i);
+  return raw;
+}

src/shared/compression.h

@@ -81,6 +81,54 @@ Data* data_compress_with_threads(Data* data_to_compress, int compression_level,
 Data* data_decompress(Data* compressed_data);
 bool compression_should_skip_with_suffixes(const char* path, char* const* suffixes, int count);
 
+/* Streaming decompression for a payload too large to hold in memory.  The
+ * caller consumes the frame's leading codec byte (and, for lz4/zlib/zlibx, the
+ * 4-byte little-endian raw-size prefix) and then feeds the remaining frame
+ * bytes in bounded chunks; decompressed output is written straight to `out_fd`
+ * so neither the compressed nor the decompressed image is ever materialized.
+ * Only zstd (the default), zlib/zlibx and none support streaming; lz4's block
+ * format is one-shot, so its stream decompressor reports failure and the caller
+ * falls back (the whole-buffer path keeps its existing bound). */
+typedef struct CompressionStreamDecompressor CompressionStreamDecompressor;
+
+CompressionStreamDecompressor*
+compression_stream_decompressor_create(CompressionAlgo algo, unsigned long long expected_out);
+/* Feed one chunk.  Returns false on a malformed frame, an I/O error, or when the
+ * total output would exceed `expected_out` (when non-zero).  *done is set once
+ * the frame end has been reached. */
+bool compression_stream_decompressor_feed(CompressionStreamDecompressor* d, const void* in,
+                                          size_t in_len, int out_fd, bool* done);
+unsigned long long compression_stream_decompressor_total(const CompressionStreamDecompressor* d);
+void compression_stream_decompressor_destroy(CompressionStreamDecompressor* d);
+
+/* Peek the logical (decompressed) size from the leading bytes of a compressed
+ * frame (codec byte + header), returning 0 when it cannot be determined from
+ * the supplied prefix.  Used to decide whether a frame must take the streaming
+ * path before its body is read. */
+unsigned long long compression_peek_frame_content_size(const void* buf, size_t len);
+
+/* Streaming compression (sender side).  Compresses a source in bounded chunks
+ * into `out_fd` as one self-describing frame (codec byte, the lz4/zlib raw-size
+ * prefix, then the codec stream), so a whole file can be compressed without
+ * materializing it in memory.  zstd/zlib/zlibx/none are supported; lz4's block
+ * format is one-shot, so its create() returns NULL and the caller keeps the
+ * buffered path.  `raw_size` is the known source length (used for the zlib
+ * prefix and, for zstd, the frame content-size field). */
+typedef struct CompressionStreamCompressor CompressionStreamCompressor;
+
+/* True when `algo` can be stream-compressed (zstd/zlib/zlibx; lz4's block format
+ * is one-shot).  Used by the sender to decide whether an over-threshold source
+ * may stay unloaded. */
+bool compression_stream_compress_supported(CompressionAlgo algo);
+CompressionStreamCompressor* compression_stream_compressor_create(CompressionAlgo algo, int level,
+                                                                  int threads);
+bool compression_stream_compressor_begin(CompressionStreamCompressor* c,
+                                         unsigned long long raw_size, int out_fd);
+bool compression_stream_compressor_feed(CompressionStreamCompressor* c, const void* in,
+                                        size_t in_len, int out_fd);
+bool compression_stream_compressor_finish(CompressionStreamCompressor* c, int out_fd);
+void compression_stream_compressor_destroy(CompressionStreamCompressor* c);
+
 /* Release the calling thread's cached zstd contexts (compressor, decompressor
  * and scratch buffer).  The cache is thread-local and is also released
  * automatically when a worker thread exits (via a C11 tss destructor) and for

src/shared/delay_updates.c

@@ -8,13 +8,49 @@
 #include <errno.h>
 #include <fcntl.h>
 #include <libgen.h>
+#include <stdatomic.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 #include <sys/file.h>
 #include <sys/stat.h>
+#include <time.h>
 #include <unistd.h>
 
+/* Process-wide counter so two staging contexts created in the same process (or
+   within the same clock tick) can never pick the same name. */
+static unsigned long long delay_updates_next_sequence(void) {
+  static atomic_ullong sequence;
+  return atomic_fetch_add_explicit(&sequence, 1, memory_order_relaxed);
+}
+
+/* Build the per-run staging directory basename: the reserved prefix plus the
+   pid and an entropy token.  A fixed name could collide with a genuine
+   destination entry; the token makes such a collision vanishingly unlikely and,
+   if it ever happens, prepare() refuses to touch the existing directory. */
+static char* delay_updates_make_staging_name(void) {
+  unsigned long long entropy = 0;
+  int fd = open("/dev/urandom", O_RDONLY | O_CLOEXEC);
+  if (fd >= 0) {
+    ssize_t got = read(fd, &entropy, sizeof(entropy));
+    close(fd);
+    if (got != (ssize_t)sizeof(entropy))
+      entropy = 0;
+  }
+  if (entropy == 0)
+    entropy = ((unsigned long long)time(NULL) << 20) ^ ((unsigned long long)getpid() << 8) ^
+              delay_updates_next_sequence();
+  int length = snprintf(NULL, 0, DELAY_UPDATES_STAGING_DIR ".%ld.%llx", (long)getpid(), entropy);
+  if (length < 0)
+    return NULL;
+  char* name = malloc((size_t)length + 1);
+  if (!name)
+    return NULL;
+  snprintf(name, (size_t)length + 1, DELAY_UPDATES_STAGING_DIR ".%ld.%llx", (long)getpid(),
+           entropy);
+  return name;
+}
+
 DelayUpdatesContext* delay_updates_context_create(const char* root_directory) {
   if (!root_directory)
     return NULL;
@@ -26,8 +62,15 @@ DelayUpdatesContext* delay_updates_context_create(const char* root_directory) {
     free(context);
     return NULL;
   }
-  context->staging_root = path_cat(root_directory, DELAY_UPDATES_STAGING_DIR);
+  context->staging_name = delay_updates_make_staging_name();
+  if (!context->staging_name) {
+    free(context->root_directory);
+    free(context);
+    return NULL;
+  }
+  context->staging_root = path_cat(root_directory, context->staging_name);
   if (!context->staging_root) {
+    free(context->staging_name);
     free(context->root_directory);
     free(context);
     return NULL;
@@ -39,6 +82,7 @@ DelayUpdatesContext* delay_updates_context_create(const char* root_directory) {
   context->lock_fd = -1;
   if (mtx_init(&context->mutex, mtx_plain) != thrd_success) {
     free(context->staging_root);
+    free(context->staging_name);
     free(context->root_directory);
     free(context);
     return NULL;
@@ -54,6 +98,7 @@ void delay_updates_context_destroy(DelayUpdatesContext* context) {
     close(context->lock_fd);
   context->lock_fd = -1;
   free(context->staging_root);
+  free(context->staging_name);
   free(context->root_directory);
   for (size_t i = 0; i < context->count; i++) {
     free(context->entries[i].staged_path);
@@ -125,48 +170,81 @@ bool delay_updates_prepare(DelayUpdatesContext* context) {
     return false;
   if (context->prepared)
     return true;
-  int fd = file_open_private_dir(context->staging_root);
+  /* Create the per-run staging directory with O_EXCL semantics.  The name is
-  if (fd < 0) {
+     unique to this transfer, so if the path already exists it is NOT ours:
+     either a genuine destination entry that happens to share the name or a
+     leftover from another session.  Refuse rather than wipe it -- the old
+     fixed-name design could destroy a real destination entry.  A crash
+     leftover is never reused (the next run picks a fresh name). */
+  char* leaf = NULL;
+  int parent_fd = file_open_secure_parent(context->staging_root, &leaf, true);
+  if (parent_fd < 0) {
     int saved_errno = errno;
     char* escaped = output_escape(context->staging_root, false);
     log_message(LOG_LEVEL_ERROR, "could not create --delay-updates staging directory '%s': %s",
                 escaped ? escaped : "<allocation failed>", strerror(saved_errno));
     free(escaped);
+    free(leaf);
     return false;
   }
-  /* Hold an exclusive advisory lock on the staging directory for the whole
+  int fd = openat(parent_fd, leaf, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
-     transfer.  The staging directory name is fixed, so two simultaneous
+  if (fd >= 0) {
-     delayed transfers to the same destination root would otherwise share it
+    close(fd);
-     and destroy each other's staged files.  The lock makes the second session
+    close(parent_fd);
-     fail cleanly instead of corrupting the first.  The lock is released when
+    char* escaped = output_escape(context->staging_root, false);
-     the context (and its file descriptor) is destroyed. */
+    log_message(LOG_LEVEL_ERROR,
+                "--delay-updates staging directory '%s' already exists and is not owned by this "
+                "transfer; refusing to overwrite it",
+                escaped ? escaped : "<allocation failed>");
+    free(escaped);
+    free(leaf);
+    return false;
+  }
+  if (errno != ENOENT) {
+    int saved_errno = errno;
+    close(parent_fd);
+    char* escaped = output_escape(context->staging_root, false);
+    log_message(LOG_LEVEL_ERROR, "could not open --delay-updates staging directory '%s': %s",
+                escaped ? escaped : "<allocation failed>", strerror(saved_errno));
+    free(escaped);
+    free(leaf);
+    return false;
+  }
+  if (mkdirat(parent_fd, leaf, 0700) != 0) {
+    int saved_errno = errno;
+    close(parent_fd);
+    char* escaped = output_escape(context->staging_root, false);
+    log_message(LOG_LEVEL_ERROR, "could not create --delay-updates staging directory '%s': %s",
+                escaped ? escaped : "<allocation failed>", strerror(saved_errno));
+    free(escaped);
+    free(leaf);
+    return false;
+  }
+  fd = openat(parent_fd, leaf, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
+  close(parent_fd);
+  free(leaf);
+  if (fd < 0) {
+    int saved_errno = errno;
+    char* escaped = output_escape(context->staging_root, false);
+    log_message(LOG_LEVEL_ERROR, "could not open --delay-updates staging directory '%s': %s",
+                escaped ? escaped : "<allocation failed>", strerror(saved_errno));
+    free(escaped);
+    return false;
+  }
+  /* Keep the exclusive advisory lock as defense in depth: the unique name
+     already prevents two sessions from sharing a staging directory, but the
+     lock also catches an improbable same-name collision that raced between the
+     existence check above and the open. */
   if (flock(fd, LOCK_EX | LOCK_NB) != 0) {
     int saved_errno = errno;
     close(fd);
-    if (saved_errno == EWOULDBLOCK || saved_errno == EAGAIN) {
     char* escaped = output_escape(context->staging_root, false);
-      log_message(LOG_LEVEL_ERROR,
-                  "another --delay-updates transfer to '%s' is already in progress; refusing to "
-                  "share the staging directory",
-                  escaped ? escaped : "<allocation failed>");
-      free(escaped);
-    } else {
     log_message(LOG_LEVEL_ERROR, "could not lock --delay-updates staging directory '%s': %s",
-                  context->staging_root, strerror(saved_errno));
+                escaped ? escaped : "<allocation failed>", strerror(saved_errno));
-    }
+    free(escaped);
     return false;
   }
   context->lock_fd = fd;
-  /* Only now, with exclusive ownership, wipe leftovers from an interrupted
-     earlier transfer; this can never race with a live session. */
-  bool ok = delay_wipe_dir_fd(fd);
-  if (!ok) {
-    log_message(LOG_LEVEL_ERROR, "could not clear stale --delay-updates staging files under '%s'",
-                context->staging_root);
-    close(context->lock_fd);
-    context->lock_fd = -1;
-    return false;
-  }
   context->prepared = true;
   return true;
 }
@@ -264,12 +342,16 @@ static bool delay_publish_entry(DelayUpdatesContext* context, const Config* conf
                                 const StagedFileEntry* entry) {
   if (!delay_publish_backup(context, config, entry))
     return false;
-  /* --force: an incoming regular file/symlink may replace a destination
+  /* An incoming regular file/symlink may replace a destination DIRECTORY that
-     DIRECTORY (possibly non-empty).  The immediate-install path handles this in
+     blocks it.  rsync removes the blocker recursively when --delete or --force
-     file_receive; a --delay-updates run stages elsewhere and only discovers the
+     is active (its generator's "make way" deletion), and a --delay-updates run
-     blocking directory here, so clear it before the rename (rsync's
+     stages elsewhere so it only discovers the blocker here.  FastSync's
-     "could not make way for new regular file" without --force). */
+     immediate-install path clears it too; without --delete/--force a non-empty
-  if (config && config->force_delete && file_directory_exists_secure(entry->final_path)) {
+     blocker fails the run (rsync's "could not make way for new regular file").
+     use_delete is gated by the server --allow-delete policy, so a client can
+     never use this to bypass deletion authorization. */
+  if (config && (config->force_delete || config->use_delete) &&
+      file_directory_exists_secure(entry->final_path)) {
     if (!file_remove_tree_secure(entry->final_path)) {
       char* escaped = output_escape(entry->final_path, false);
       log_message(LOG_LEVEL_ERROR, "could not remove destination directory blocking '%s': %s",

src/shared/delay_updates.h

@@ -24,6 +24,7 @@ typedef struct {
    shared with the publish/cleanup phase that runs after the threads join. */
 typedef struct DelayUpdatesContext {
   char* root_directory; /* receive root the staging dir lives under */
+  char* staging_name;   /* per-run unique staging dir basename */
   char* staging_root;   /* root_directory/<staging dir name> */
   mtx_t mutex;
   StagedFileEntry* entries;
@@ -33,7 +34,11 @@ typedef struct DelayUpdatesContext {
   int lock_fd;   /* advisory exclusive flock held on the staging dir, or -1 */
 } DelayUpdatesContext;
 
-/* Name of the private staging subdirectory created under the receive root. */
+/* Reserved prefix for the private staging subdirectory created under the
+   receive root.  The actual directory name is per-run unique (the prefix plus a
+   pid/entropy token) so it can never clobber a genuine destination entry that
+   happens to share the name; the bare prefix is still what a --backup-dir must
+   not collide with. */
 #define DELAY_UPDATES_STAGING_DIR ".fastsync-stage"
 
 /* True when `dir` (ignoring a trailing "/") is the reserved staging directory

src/shared/delete.c

@@ -39,12 +39,31 @@ FilterAction delete_protect_verdict(const DeleteProtectRules* protect, const cha
                                     const char* leaf, bool is_dir) {
   if (!protect)
     return FILTER_ACTION_NONE;
+  /* rsync protects its own --backup files from the delete pass: a name ending
+     in the backup suffix is never an extra.  Checked before the filter rules so
+     an explicit exclude cannot be bypassed (the suffix is always a shield). */
+  if (protect->backup_suffix && protect->backup_suffix[0] != '\0') {
+    size_t name_len = strlen(leaf);
+    size_t suffix_len = strlen(protect->backup_suffix);
+    if (name_len > suffix_len &&
+        strcmp(leaf + (name_len - suffix_len), protect->backup_suffix) == 0)
+      return FILTER_ACTION_PROTECT;
+  }
   FilterAction action = filter_dir_rules_apply_side(protect->dir_rules, rel_path, leaf, is_dir);
   if (action != FILTER_ACTION_NONE)
     return action;
   return filter_rules_apply_side(protect->base_rules, rel_path, leaf, is_dir, FILTER_SIDE_RECEIVER);
 }
 
+const char* delete_backup_suffix(const Config* config) {
+  if (!config || !config->backup || config->ignore_existing)
+    return NULL;
+  const char* suffix = config->suffix ? config->suffix : "~";
+  if (!suffix[0] || strchr(suffix, '/'))
+    return NULL;
+  return suffix;
+}
+
 /* Classify a removed entry from its st_mode for the per-type delete counters. */
 DeleteEntryType delete_entry_type_of_mode(mode_t mode) {
   if (S_ISDIR(mode))
@@ -205,6 +224,235 @@ typedef struct {
   void* observer_context;      /* DELETE mode */
 } DeleteWalkState;
 
+/* The per-walk invariants threaded unchanged through every recursive descent:
+   the keep/synchronized-dir indexes, the destination mode and the protection
+   rules.  Bundling them keeps the recursive helpers below to a handful of
+   positional arguments. */
+typedef struct {
+  const PathIndex* keep;
+  const PathIndex* dirs;
+  DeleteWalkState* state;
+  const DeleteSkipEntry* skips;
+  int skip_count;
+  const DeleteProtectRules* protect;
+} DeleteWalkContext;
+
+/* Duplicate `path` with rsync's trailing-slash convention, used to report a
+   removed (or would-be-removed) directory.  Returns NULL on allocation
+   failure. */
+static char* with_trailing_slash(const char* path) {
+  size_t len = strlen(path);
+  char* copy = malloc(len + 2);
+  if (!copy)
+    return NULL;
+  memcpy(copy, path, len);
+  copy[len] = '/';
+  copy[len + 1] = '\0';
+  return copy;
+}
+
+/* Forward declaration: the ordered passes below recurse through the driver. */
+static bool delete_walk_fd(int dirfd, const char* rel_path, const DeleteWalkContext* ctx,
+                           bool parent_deletable, bool* all_removed);
+
+/* Descend into the child directory `name` of `dirfd`, walking it as part of the
+   current operation.  Returns false on a genuine open/walk failure; on success
+   *child_all_removed reports whether the child removed everything it held (so
+   the caller may rmdir it). */
+static bool delete_walk_child(int dirfd, const char* name, const char* child_rel,
+                              const DeleteWalkContext* ctx, bool deletable,
+                              bool* child_all_removed) {
+  *child_all_removed = false;
+  int childfd = openat(dirfd, name, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
+  if (childfd < 0)
+    return errno == ENOENT;
+  bool ok = delete_walk_fd(childfd, child_rel, ctx, deletable, child_all_removed);
+  close(childfd);
+  return ok;
+}
+
+/* Classify every entry up front (the verdict does not depend on processing
+   order) so the ordered passes below can act on it.  Sets shielded[]/is_extra[]
+   and reports through *local_survives whether anything in this directory stays
+   in place.  Returns false on a path-construction failure. */
+static bool delete_walk_classify(const char* rel_path, const DeleteDirEntry* entries, size_t count,
+                                 const DeleteWalkContext* ctx, bool deletable, bool at_root,
+                                 bool* shielded, bool* is_extra, bool* local_survives) {
+  bool ok = true;
+  for (size_t i = 0; i < count; i++) {
+    char* child_rel = path_cat((char*)rel_path, entries[i].name);
+    if (!child_rel) {
+      ok = false;
+      continue;
+    }
+    /* A --delay-updates run keeps its staging directory as a direct child of
+       the receive root, and basis-dir snapshots live below it too.  Their
+       contents are not manifest entries, so descending into them would delete
+       every staged / basis file as an "extra".  Only the staging name (a
+       top-level-only prefix) and the basis prefixes are protected: a nested
+       destination directory that happens to be called .fastsync-stage is
+       ordinary content. */
+    if (path_under_skip_prefix(child_rel, at_root, ctx->skips, ctx->skip_count)) {
+      shielded[i] = true;
+      *local_survives = true;
+    } else if (delete_protect_verdict(ctx->protect, child_rel, entries[i].name,
+                                      entries[i].is_dir) == FILTER_ACTION_PROTECT) {
+      /* A first-match protect rule shields the extra; for a directory the whole
+         subtree is shielded (rsync prunes an excluded directory), so do not
+         descend. */
+      shielded[i] = true;
+      *local_survives = true;
+    } else if (entries[i].is_dir) {
+      bool child_synced = ctx->dirs && path_index_contains(ctx->dirs, child_rel);
+      is_extra[i] = deletable && !child_synced && !keep_is_dir(ctx->keep, child_rel);
+      if (!is_extra[i])
+        *local_survives = true;
+    } else {
+      is_extra[i] = deletable && !keep_is_file(ctx->keep, child_rel);
+      if (!is_extra[i])
+        *local_survives = true;
+    }
+    free(child_rel);
+  }
+  return ok;
+}
+
+/* Pass 1: extraneous subdirectories, descending.  Recurses into each and, when
+   the child removed everything it held, records or removes it and charges the
+   budget. */
+static bool delete_walk_extra_dirs(int dirfd, const char* rel_path, const DeleteDirEntry* entries,
+                                   size_t dir_count, const DeleteWalkContext* ctx, bool deletable,
+                                   const bool* is_extra, bool* local_survives) {
+  bool ok = true;
+  for (size_t i = 0; i < dir_count; i++) {
+    if (!is_extra[i])
+      continue;
+    char* child_rel = path_cat((char*)rel_path, entries[i].name);
+    if (!child_rel) {
+      ok = false;
+      continue;
+    }
+    bool child_all_removed = false;
+    if (!delete_walk_child(dirfd, entries[i].name, child_rel, ctx, deletable, &child_all_removed))
+      ok = false;
+    if (child_all_removed && deletable) {
+      if (ctx->state->mode == DELETE_WALK_MODE_LIST) {
+        /* Record the directory with rsync's trailing slash. */
+        char* copy = with_trailing_slash(child_rel);
+        if (!copy) {
+          ok = false;
+        } else if (!array_list_add(ctx->state->out, copy)) {
+          free(copy);
+          ok = false;
+        } else {
+          (*ctx->state->recorded)++;
+        }
+      } else if (ctx->state->budget->deleted >= ctx->state->budget->max_delete) {
+        ctx->state->budget->limit_hit = true;
+        ctx->state->budget->skipped++;
+        *local_survives = true;
+      } else if (unlinkat(dirfd, entries[i].name, AT_REMOVEDIR) != 0) {
+        /* ENOENT: already gone (fine).  ENOTEMPTY/EEXIST: the directory still
+           holds entries the walker leaves in place (a protected excluded
+           prefix, a kept file the manifest protects, a symlink); rsync leaves
+           such a directory behind, so this is not an error.  Only genuine I/O
+           failures abort the deletion. */
+        if (errno != ENOENT && errno != ENOTEMPTY && errno != EEXIST)
+          ok = false;
+        *local_survives = true;
+      } else {
+        ctx->state->budget->deleted++;
+        /* rsync reports a removed directory with a trailing slash. */
+        if (ctx->state->observer) {
+          char* with_slash = with_trailing_slash(child_rel);
+          if (with_slash) {
+            ctx->state->observer(ctx->state->observer_context, with_slash, DELETE_ENTRY_DIR);
+            free(with_slash);
+          } else {
+            ctx->state->observer(ctx->state->observer_context, child_rel, DELETE_ENTRY_DIR);
+          }
+        }
+      }
+    } else {
+      *local_survives = true;
+    }
+    free(child_rel);
+  }
+  return ok;
+}
+
+/* Pass 2: extraneous files, descending. */
+static bool delete_walk_extra_files(int dirfd, const char* rel_path, const DeleteDirEntry* entries,
+                                    size_t dir_count, size_t count, const DeleteWalkContext* ctx,
+                                    const bool* is_extra, bool* local_survives) {
+  bool ok = true;
+  for (size_t i = dir_count; i < count; i++) {
+    if (!is_extra[i])
+      continue;
+    if (ctx->state->mode == DELETE_WALK_MODE_LIST) {
+      char* child_rel = path_cat((char*)rel_path, entries[i].name);
+      if (!child_rel) {
+        ok = false;
+        continue;
+      }
+      char* copy = str_dup(child_rel);
+      if (!copy || !array_list_add(ctx->state->out, copy)) {
+        free(copy);
+        ok = false;
+      } else {
+        (*ctx->state->recorded)++;
+      }
+      free(child_rel);
+    } else if (ctx->state->budget->deleted >= ctx->state->budget->max_delete) {
+      ctx->state->budget->limit_hit = true;
+      ctx->state->budget->skipped++;
+      *local_survives = true;
+    } else if (unlinkat(dirfd, entries[i].name, 0) != 0) {
+      if (errno != ENOENT)
+        ok = false;
+      *local_survives = true;
+    } else {
+      ctx->state->budget->deleted++;
+      char* child_rel = path_cat((char*)rel_path, entries[i].name);
+      if (child_rel) {
+        if (ctx->state->observer)
+          ctx->state->observer(ctx->state->observer_context, child_rel,
+                               delete_entry_type_of_mode(entries[i].mode));
+        char* escaped_path = output_escape(child_rel, log_get_8_bit_output());
+        fprintf(stderr, "  Deleted: %s\n", escaped_path ? escaped_path : "<allocation failed>");
+        free(escaped_path);
+      }
+      free(child_rel);
+    }
+  }
+  return ok;
+}
+
+/* Pass 3: kept subdirectories, ascending (rsync descends into these only after
+   the parent's own extras have been handled). */
+static bool delete_walk_kept_dirs(int dirfd, const char* rel_path, const DeleteDirEntry* entries,
+                                  size_t dir_count, const DeleteWalkContext* ctx, bool deletable,
+                                  const bool* is_extra, const bool* shielded,
+                                  bool* local_survives) {
+  bool ok = true;
+  for (size_t i = dir_count; i-- > 0;) {
+    if (is_extra[i] || shielded[i])
+      continue;
+    char* child_rel = path_cat((char*)rel_path, entries[i].name);
+    if (!child_rel) {
+      ok = false;
+      continue;
+    }
+    bool child_all_removed = false;
+    if (!delete_walk_child(dirfd, entries[i].name, child_rel, ctx, deletable, &child_all_removed))
+      ok = false;
+    /* A kept/synchronized directory is never removed. */
+    *local_survives = true;
+    free(child_rel);
+  }
+  return ok;
+}
+
 /* Remove the extras directly inside the directory open on `dirfd` (DELETE mode)
    or record the paths that WOULD be removed (LIST mode), recursing into every
    child directory so kept content below a synchronized prefix is reached.
@@ -219,11 +467,8 @@ typedef struct {
    descending name order, then extraneous files, then kept subdirectories in
    ascending order) rather than readdir() order, so `--max-delete` leaves the
    same survivors and the `--info=del`/dry-run line order matches rsync. */
-static bool delete_walk_fd(int dirfd, const char* rel_path, const PathIndex* keep,
+static bool delete_walk_fd(int dirfd, const char* rel_path, const DeleteWalkContext* ctx,
-                           const PathIndex* dirs, DeleteWalkState* state,
+                           bool parent_deletable, bool* all_removed) {
-                           const DeleteSkipEntry* skips, int skip_count,
-                           const DeleteProtectRules* protect, bool parent_deletable,
-                           bool* all_removed) {
   DeleteDirEntry* entries = NULL;
   size_t count = 0;
   bool collect_ok = true;
@@ -242,7 +487,7 @@ static bool delete_walk_fd(int dirfd, const char* rel_path, const PathIndex* kee
   /* A directory is deletable when it or ANY ancestor is synchronized; the
      `parent_deletable` flag carries that down the recursion so dest-only
      directories below a synchronized root are removed wholesale. */
-  bool deletable = parent_deletable || is_synced_dir(dirs, rel_path);
+  bool deletable = parent_deletable || is_synced_dir(ctx->dirs, rel_path);
   bool at_root = rel_path[0] == '\0';
 
   /* Reproduce rsync's traversal order: extraneous subdirectories in descending
@@ -255,182 +500,18 @@ static bool delete_walk_fd(int dirfd, const char* rel_path, const PathIndex* kee
   while (dir_count < count && entries[dir_count].is_dir)
     dir_count++;
 
-  /* Classify every entry up front (the verdict does not depend on processing
+  if (!delete_walk_classify(rel_path, entries, count, ctx, deletable, at_root, shielded, is_extra,
-     order) so the ordered passes below can act on it. */
+                            &local_survives))
-  for (size_t i = 0; i < count; i++) {
-    char* child_rel = path_cat((char*)rel_path, entries[i].name);
-    if (!child_rel) {
     operation_ok = false;
-      continue;
+  if (!delete_walk_extra_dirs(dirfd, rel_path, entries, dir_count, ctx, deletable, is_extra,
-    }
+                              &local_survives))
-    /* A --delay-updates run keeps its staging directory as a direct child of
-       the receive root, and basis-dir snapshots live below it too.  Their
-       contents are not manifest entries, so descending into them would delete
-       every staged / basis file as an "extra".  Only the staging name (a
-       top-level-only prefix) and the basis prefixes are protected: a nested
-       destination directory that happens to be called .fastsync-stage is
-       ordinary content. */
-    if (path_under_skip_prefix(child_rel, at_root, skips, skip_count)) {
-      shielded[i] = true;
-      local_survives = true;
-    } else if (delete_protect_verdict(protect, child_rel, entries[i].name, entries[i].is_dir) ==
-               FILTER_ACTION_PROTECT) {
-      /* A first-match protect rule shields the extra; for a directory the whole
-         subtree is shielded (rsync prunes an excluded directory), so do not
-         descend. */
-      shielded[i] = true;
-      local_survives = true;
-    } else if (entries[i].is_dir) {
-      bool child_synced = dirs && path_index_contains(dirs, child_rel);
-      is_extra[i] = deletable && !child_synced && !keep_is_dir(keep, child_rel);
-      if (!is_extra[i])
-        local_survives = true;
-    } else {
-      is_extra[i] = deletable && !keep_is_file(keep, child_rel);
-      if (!is_extra[i])
-        local_survives = true;
-    }
-    free(child_rel);
-  }
-
-  /* Pass 1: extraneous subdirectories, descending. */
-  for (size_t i = 0; i < dir_count; i++) {
-    if (!is_extra[i])
-      continue;
-    char* child_rel = path_cat((char*)rel_path, entries[i].name);
-    if (!child_rel) {
     operation_ok = false;
-      continue;
+  if (!delete_walk_extra_files(dirfd, rel_path, entries, dir_count, count, ctx, is_extra,
-    }
+                               &local_survives))
-    int childfd = openat(dirfd, entries[i].name, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
-    bool child_all_removed = false;
-    if (childfd >= 0) {
-      if (!delete_walk_fd(childfd, child_rel, keep, dirs, state, skips, skip_count, protect,
-                          deletable, &child_all_removed))
     operation_ok = false;
-      close(childfd);
+  if (!delete_walk_kept_dirs(dirfd, rel_path, entries, dir_count, ctx, deletable, is_extra,
-    } else if (errno != ENOENT) {
+                             shielded, &local_survives))
     operation_ok = false;
-    }
-    if (child_all_removed && deletable) {
-      if (state->mode == DELETE_WALK_MODE_LIST) {
-        /* Record the directory with rsync's trailing slash. */
-        size_t len = strlen(child_rel);
-        char* copy = malloc(len + 2);
-        if (!copy) {
-          operation_ok = false;
-        } else {
-          memcpy(copy, child_rel, len);
-          copy[len] = '/';
-          copy[len + 1] = '\0';
-          if (!array_list_add(state->out, copy)) {
-            free(copy);
-            operation_ok = false;
-          } else {
-            (*state->recorded)++;
-          }
-        }
-      } else if (state->budget->deleted >= state->budget->max_delete) {
-        state->budget->limit_hit = true;
-        state->budget->skipped++;
-        local_survives = true;
-      } else if (unlinkat(dirfd, entries[i].name, AT_REMOVEDIR) != 0) {
-        /* ENOENT: already gone (fine).  ENOTEMPTY/EEXIST: the directory still
-           holds entries the walker leaves in place (a protected excluded
-           prefix, a kept file the manifest protects, a symlink); rsync leaves
-           such a directory behind, so this is not an error.  Only genuine I/O
-           failures abort the deletion. */
-        if (errno != ENOENT && errno != ENOTEMPTY && errno != EEXIST)
-          operation_ok = false;
-        local_survives = true;
-      } else {
-        state->budget->deleted++;
-        /* rsync reports a removed directory with a trailing slash. */
-        if (state->observer) {
-          size_t len = strlen(child_rel);
-          char* with_slash = malloc(len + 2);
-          if (with_slash) {
-            memcpy(with_slash, child_rel, len);
-            with_slash[len] = '/';
-            with_slash[len + 1] = '\0';
-            state->observer(state->observer_context, with_slash, DELETE_ENTRY_DIR);
-            free(with_slash);
-          } else {
-            state->observer(state->observer_context, child_rel, DELETE_ENTRY_DIR);
-          }
-        }
-      }
-    } else {
-      local_survives = true;
-    }
-    free(child_rel);
-  }
-
-  /* Pass 2: extraneous files, descending. */
-  for (size_t i = dir_count; i < count; i++) {
-    if (!is_extra[i])
-      continue;
-    if (state->mode == DELETE_WALK_MODE_LIST) {
-      char* child_rel = path_cat((char*)rel_path, entries[i].name);
-      if (!child_rel) {
-        operation_ok = false;
-        continue;
-      }
-      char* copy = str_dup(child_rel);
-      if (!copy || !array_list_add(state->out, copy)) {
-        free(copy);
-        operation_ok = false;
-      } else {
-        (*state->recorded)++;
-      }
-      free(child_rel);
-    } else if (state->budget->deleted >= state->budget->max_delete) {
-      state->budget->limit_hit = true;
-      state->budget->skipped++;
-      local_survives = true;
-    } else if (unlinkat(dirfd, entries[i].name, 0) != 0) {
-      if (errno != ENOENT)
-        operation_ok = false;
-      local_survives = true;
-    } else {
-      state->budget->deleted++;
-      char* child_rel = path_cat((char*)rel_path, entries[i].name);
-      if (child_rel) {
-        if (state->observer)
-          state->observer(state->observer_context, child_rel,
-                          delete_entry_type_of_mode(entries[i].mode));
-        char* escaped_path = output_escape(child_rel, log_get_8_bit_output());
-        fprintf(stderr, "  Deleted: %s\n", escaped_path ? escaped_path : "<allocation failed>");
-        free(escaped_path);
-      }
-      free(child_rel);
-    }
-  }
-
-  /* Pass 3: kept subdirectories, ascending (rsync descends into these only
-     after the parent's own extras have been handled). */
-  for (size_t i = dir_count; i-- > 0;) {
-    if (is_extra[i] || shielded[i])
-      continue;
-    char* child_rel = path_cat((char*)rel_path, entries[i].name);
-    if (!child_rel) {
-      operation_ok = false;
-      continue;
-    }
-    int childfd = openat(dirfd, entries[i].name, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
-    bool child_all_removed = false;
-    if (childfd >= 0) {
-      if (!delete_walk_fd(childfd, child_rel, keep, dirs, state, skips, skip_count, protect,
-                          deletable, &child_all_removed))
-        operation_ok = false;
-      close(childfd);
-    } else if (errno != ENOENT) {
-      operation_ok = false;
-    }
-    /* A kept/synchronized directory is never removed. */
-    local_survives = true;
-    free(child_rel);
-  }
 
   free(shielded);
   free(is_extra);
@@ -485,8 +566,13 @@ bool delete_extras_list(const char* dest_root, const ArrayList* manifest,
                            .recorded = &recorded,
                            .observer = NULL,
                            .observer_context = NULL};
-  bool ok = delete_walk_fd(rootfd, "", &keep, have_dirs ? &dirs : NULL, &state, skips, skip_count,
+  DeleteWalkContext ctx = {.keep = &keep,
-                           protect, false, &all_removed);
+                           .dirs = have_dirs ? &dirs : NULL,
+                           .state = &state,
+                           .skips = skips,
+                           .skip_count = skip_count,
+                           .protect = protect};
+  bool ok = delete_walk_fd(rootfd, "", &ctx, false, &all_removed);
   if (close(rootfd) != 0)
     ok = false;
   path_index_free(&keep);
@@ -538,8 +624,13 @@ DeleteWalkResult delete_extras_limited_observed(const char* dest_root, const Arr
                            .recorded = NULL,
                            .observer = observer,
                            .observer_context = observer_context};
-  bool ok = delete_walk_fd(rootfd, "", &keep, have_dirs ? &dirs : NULL, &state, skips, skip_count,
+  DeleteWalkContext ctx = {.keep = &keep,
-                           protect, false, &all_removed);
+                           .dirs = have_dirs ? &dirs : NULL,
+                           .state = &state,
+                           .skips = skips,
+                           .skip_count = skip_count,
+                           .protect = protect};
+  bool ok = delete_walk_fd(rootfd, "", &ctx, false, &all_removed);
   if (close(rootfd) != 0)
     ok = false;
   path_index_free(&keep);
@@ -631,7 +722,13 @@ bool delete_skips_build(const Config* config, const ArrayList* protected_paths,
   }
   int idx = 0;
   if (config->delay_updates) {
-    out->entries[idx].prefix = DELAY_UPDATES_STAGING_DIR;
+    /* Protect this transfer's actual (per-run unique) staging directory.  The
+       runtime name is only known to the receiver-side context; fall back to the
+       reserved prefix for a context that was never created (e.g. a dry run). */
+    const char* staging_name = (config->delay_context && config->delay_context->staging_name)
+                                   ? config->delay_context->staging_name
+                                   : DELAY_UPDATES_STAGING_DIR;
+    out->entries[idx].prefix = staging_name;
     out->entries[idx].top_level_only = true;
     idx++;
   }

src/shared/delete.h

@@ -37,6 +37,11 @@ typedef enum {
 typedef struct {
   const FilterRuleList* base_rules;
   const FilterRuleList* dir_rules;
+  /* When non-NULL and non-empty, a destination entry whose name ends with this
+     suffix is protected from deletion.  rsync never treats a --backup file as
+     an extra, so a backup created at --delay-updates publication (or a
+     pre-existing one) survives the delete-after pass. */
+  const char* backup_suffix;
 } DeleteProtectRules;
 
 /* rsync's first-match-wins receiver verdict for one candidate extra: the
@@ -48,6 +53,11 @@ typedef struct {
 FilterAction delete_protect_verdict(const DeleteProtectRules* protect, const char* rel_path,
                                     const char* leaf, bool is_dir);
 
+/* The backup suffix the delete walker must shield from deletion, or NULL when
+   --backup is inactive or the configured suffix is unusable (empty, or holding
+   a path separator).  Matches the suffix file_save uses for backups. */
+const char* delete_backup_suffix(const Config* config);
+
 /* One protected entry for the delete walker.  When top_level_only is true the
    prefix is skipped only as a DIRECT child of dest_root (the --delay-updates
    staging directory, which must not hide genuine extras inside a nested

src/shared/delete_commit.c

@@ -170,7 +170,8 @@ static bool delete_extras_budgeted_observed(const Config* config, const DeleteMa
   size_t deleted = 0;
   size_t skipped = 0;
   DeleteProtectRules protect = {.base_rules = config->protect_rules,
-                                .dir_rules = manifest->per_dir_rules};
+                                .dir_rules = manifest->per_dir_rules,
+                                .backup_suffix = delete_backup_suffix(config)};
   DeleteWalkResult result = delete_extras_limited_observed(
       config->receive_root_directory, manifest->keeps, manifest->dirs, remaining, skips.entries,
       skips.count, &protect, &deleted, &skipped, observer, observer_context);
@@ -226,6 +227,166 @@ static void prefixed_delete_observer(void* context, const char* rel, DeleteEntry
    --max-delete budget: once it is exhausted the remaining requests are skipped
    and counted.  Returns false only on a genuine error (a confinement failure on
    a validated path or an I/O error), which fails the run. */
+
+/* How one missing-args request leaves the driver loop.  The original walker
+   `continue`s past an invalid/protected/absent/budget-skipped request (without
+   breaking) but stops after a request that ran to completion while an error is
+   pending; NEXT/STOP preserve that control flow exactly. */
+typedef enum { MISSING_ARG_NEXT, MISSING_ARG_STOP } MissingArgStep;
+
+/* Remove a NON-empty missing-args directory recursively (--delete/--force in
+   effect): walk its contents through the budgeted extras walker so every removed
+   file/dir counts toward --max-delete (rsync parity), then remove the now-empty
+   directory itself, which costs one more budget unit.  A run that hits the cap
+   leaves the remaining entries in place.  The observer is wrapped so the nested
+   walk reports receive-root-relative paths.  Sets the *removed and *ok outputs. */
+static void delete_nonempty_missing_dir(const char* full, const char* rel,
+                                        DeleteBudgetState* budget, DeletePathObserver observer,
+                                        void* observer_context, bool* removed, bool* ok) {
+  ArrayList* no_keeps = array_list_create(free);
+  /* Never let an accounting slip (deleted > max_delete) underflow the remaining
+     budget into SIZE_MAX, which would grant unlimited deletions. */
+  size_t remaining =
+      budget->deleted >= budget->max_delete ? 0 : budget->max_delete - budget->deleted;
+  size_t contents_deleted = 0;
+  size_t contents_skipped = 0;
+  PrefixedDeleteObserver nested = {observer, observer_context, rel};
+  DeleteWalkResult walk =
+      no_keeps ? delete_extras_limited_observed(full, no_keeps, NULL, remaining, NULL, 0, NULL,
+                                                &contents_deleted, &contents_skipped,
+                                                observer ? prefixed_delete_observer : NULL,
+                                                observer ? &nested : NULL)
+               : DELETE_WALK_ERROR;
+  if (no_keeps)
+    array_list_delete(no_keeps);
+  budget->deleted += contents_deleted;
+  budget->skipped += contents_skipped;
+  if (walk == DELETE_WALK_LIMIT_REACHED) {
+    budget->limit_hit = true;
+  } else if (walk != DELETE_WALK_OK) {
+    *ok = false;
+  } else if (budget->deleted >= budget->max_delete) {
+    budget->limit_hit = true;
+    budget->skipped++;
+  } else if (file_remove_tree_secure(full)) {
+    /* The shared `if (removed)` tail charges this directory exactly once;
+       counting it here too would consume two budget units. */
+    *removed = true;
+  } else {
+    *ok = false;
+  }
+}
+
+/* Remove one missing-args destination mirror.  `skips` holds the receiver
+   artifacts (staging directory, basis snapshots) that stay protected.  Returns
+   MISSING_ARG_STOP when the driver loop must stop (a completed removal left a
+   genuine error pending) and MISSING_ARG_NEXT otherwise; *ok accumulates the
+   overall success across the whole run. */
+static MissingArgStep delete_one_missing_arg(const Config* config, const char* rel,
+                                             const DeleteSkipSet* skips, DeleteBudgetState* budget,
+                                             DeletePathObserver observer, void* observer_context,
+                                             bool* ok) {
+  if (!rel || *rel == '\0' || *rel == '/' || has_path_traversal(rel)) {
+    /* Defensive only: receive_manifest_entries already validated every
+       section identically, so a controlled peer never reaches this branch. */
+    log_message(LOG_LEVEL_ERROR, "invalid missing-args delete path");
+    *ok = false;
+    return MISSING_ARG_NEXT;
+  }
+  bool at_root = strchr(rel, '/') == NULL;
+  if (path_under_skip_prefix(rel, at_root, skips->entries, skips->count)) {
+    char* escaped = output_escape(rel, log_get_8_bit_output());
+    log_message(LOG_LEVEL_WARNING,
+                "missing-args path '%s' is protected (staging directory or basis snapshot); "
+                "not deleting",
+                escaped ? escaped : "<allocation failed>");
+    free(escaped);
+    return MISSING_ARG_NEXT;
+  }
+  char* full = path_cat(config->receive_root_directory, rel);
+  if (!full) {
+    *ok = false;
+    return MISSING_ARG_NEXT;
+  }
+  char* leaf = NULL;
+  int parent_fd = file_open_secure_parent(full, &leaf, false);
+  if (parent_fd < 0) {
+    /* The mirror's parent directory may itself not exist on the destination
+       (a deeper missing entry whose leading directories were never created).
+       That is a no-op -- there is nothing to delete -- matching
+       file_remove_tree_secure's absent-path handling; only a genuine I/O
+       error (EACCES, a symlink loop, ...) fails the run. */
+    bool absent = errno == ENOENT || errno == ENOTDIR;
+    free(full);
+    free(leaf);
+    if (!absent)
+      *ok = false;
+    return MISSING_ARG_NEXT;
+  }
+  struct stat st;
+  if (fstatat(parent_fd, leaf, &st, AT_SYMLINK_NOFOLLOW) != 0) {
+    /* Already absent: nothing to delete (a no-op, not a deletion). */
+    if (errno != ENOENT)
+      *ok = false;
+    close(parent_fd);
+    free(leaf);
+    free(full);
+    return MISSING_ARG_NEXT;
+  }
+  /* An entry that exists is one deletion: skip it (and count it) when the
+     shared --max-delete budget is already exhausted. */
+  if (budget->deleted >= budget->max_delete) {
+    budget->limit_hit = true;
+    budget->skipped++;
+    close(parent_fd);
+    free(leaf);
+    free(full);
+    return MISSING_ARG_NEXT;
+  }
+  bool removed = false;
+  if (S_ISDIR(st.st_mode)) {
+    if (unlinkat(parent_fd, leaf, AT_REMOVEDIR) == 0) {
+      removed = true;
+    } else if (errno == ENOTEMPTY || errno == EEXIST) {
+      close(parent_fd);
+      parent_fd = -1;
+      free(leaf);
+      leaf = NULL;
+      if (config->use_delete || config->force_delete) {
+        delete_nonempty_missing_dir(full, rel, budget, observer, observer_context, &removed, ok);
+      } else {
+        char* escaped = output_escape(rel, log_get_8_bit_output());
+        log_message(LOG_LEVEL_WARNING,
+                    "missing-args destination '%s' is a non-empty directory; use --force or "
+                    "--delete to remove it",
+                    escaped ? escaped : "<allocation failed>");
+        free(escaped);
+      }
+    } else if (errno != ENOENT) {
+      *ok = false;
+    }
+  } else {
+    if (unlinkat(parent_fd, leaf, 0) == 0) {
+      removed = true;
+    } else if (errno != ENOENT) {
+      *ok = false;
+    }
+  }
+  if (removed) {
+    budget->deleted++;
+    if (observer)
+      observer(observer_context, rel, delete_entry_type_of_mode(st.st_mode));
+    char* escaped = output_escape(rel, log_get_8_bit_output());
+    fprintf(stderr, "  Deleted: %s\n", escaped ? escaped : "<allocation failed>");
+    free(escaped);
+  }
+  if (parent_fd >= 0)
+    close(parent_fd);
+  free(leaf);
+  free(full);
+  return *ok ? MISSING_ARG_NEXT : MISSING_ARG_STOP;
+}
+
 static bool delete_missing_args_budgeted_observed(const Config* config,
                                                   const DeleteManifest* manifest,
                                                   DeleteBudgetState* budget,
@@ -245,141 +406,8 @@ static bool delete_missing_args_budgeted_observed(const Config* config,
   bool ok = true;
   for (int i = 0; i < manifest->missing->size; i++) {
     const char* rel = (const char*)manifest->missing->items[i];
-    if (!rel || *rel == '\0' || *rel == '/' || has_path_traversal(rel)) {
+    if (delete_one_missing_arg(config, rel, &skips, budget, observer, observer_context, &ok) ==
-      /* Defensive only: receive_manifest_entries already validated every
+        MISSING_ARG_STOP)
-         section identically, so a controlled peer never reaches this branch. */
-      log_message(LOG_LEVEL_ERROR, "invalid missing-args delete path");
-      ok = false;
-      continue;
-    }
-    bool at_root = strchr(rel, '/') == NULL;
-    if (path_under_skip_prefix(rel, at_root, skips.entries, skips.count)) {
-      char* escaped = output_escape(rel, log_get_8_bit_output());
-      log_message(LOG_LEVEL_WARNING,
-                  "missing-args path '%s' is protected (staging directory or basis snapshot); "
-                  "not deleting",
-                  escaped ? escaped : "<allocation failed>");
-      free(escaped);
-      continue;
-    }
-    char* full = path_cat(config->receive_root_directory, rel);
-    if (!full) {
-      ok = false;
-      continue;
-    }
-    char* leaf = NULL;
-    int parent_fd = file_open_secure_parent(full, &leaf, false);
-    if (parent_fd < 0) {
-      /* The mirror's parent directory may itself not exist on the destination
-         (a deeper missing entry whose leading directories were never created).
-         That is a no-op -- there is nothing to delete -- matching
-         file_remove_tree_secure's absent-path handling; only a genuine I/O
-         error (EACCES, a symlink loop, ...) fails the run. */
-      bool absent = errno == ENOENT || errno == ENOTDIR;
-      free(full);
-      free(leaf);
-      if (!absent)
-        ok = false;
-      continue;
-    }
-    struct stat st;
-    if (fstatat(parent_fd, leaf, &st, AT_SYMLINK_NOFOLLOW) != 0) {
-      /* Already absent: nothing to delete (a no-op, not a deletion). */
-      if (errno != ENOENT)
-        ok = false;
-      close(parent_fd);
-      free(leaf);
-      free(full);
-      continue;
-    }
-    /* An entry that exists is one deletion: skip it (and count it) when the
-       shared --max-delete budget is already exhausted. */
-    if (budget->deleted >= budget->max_delete) {
-      budget->limit_hit = true;
-      budget->skipped++;
-      close(parent_fd);
-      free(leaf);
-      free(full);
-      continue;
-    }
-    bool removed = false;
-    if (S_ISDIR(st.st_mode)) {
-      if (unlinkat(parent_fd, leaf, AT_REMOVEDIR) == 0) {
-        removed = true;
-      } else if (errno == ENOTEMPTY || errno == EEXIST) {
-        close(parent_fd);
-        parent_fd = -1;
-        free(leaf);
-        leaf = NULL;
-        if (config->use_delete || config->force_delete) {
-          /* Remove the contents entry-by-entry through the budgeted extras
-             walker so every deleted file/dir counts toward --max-delete (rsync
-             parity); the now-empty directory itself costs one more.  A run that
-             hits the cap leaves the remaining entries in place. */
-          ArrayList* no_keeps = array_list_create(free);
-          /* Never let an accounting slip (deleted > max_delete) underflow the
-             remaining budget into SIZE_MAX, which would grant unlimited
-             deletions. */
-          size_t remaining =
-              budget->deleted >= budget->max_delete ? 0 : budget->max_delete - budget->deleted;
-          size_t contents_deleted = 0;
-          size_t contents_skipped = 0;
-          PrefixedDeleteObserver nested = {observer, observer_context, rel};
-          DeleteWalkResult walk =
-              no_keeps ? delete_extras_limited_observed(full, no_keeps, NULL, remaining, NULL, 0,
-                                                        NULL, &contents_deleted, &contents_skipped,
-                                                        observer ? prefixed_delete_observer : NULL,
-                                                        observer ? &nested : NULL)
-                       : DELETE_WALK_ERROR;
-          if (no_keeps)
-            array_list_delete(no_keeps);
-          budget->deleted += contents_deleted;
-          budget->skipped += contents_skipped;
-          if (walk == DELETE_WALK_LIMIT_REACHED) {
-            budget->limit_hit = true;
-          } else if (walk != DELETE_WALK_OK) {
-            ok = false;
-          } else if (budget->deleted >= budget->max_delete) {
-            budget->limit_hit = true;
-            budget->skipped++;
-          } else if (file_remove_tree_secure(full)) {
-            /* The shared `if (removed)` tail charges this directory exactly
-               once; counting it here too would consume two budget units. */
-            removed = true;
-          } else {
-            ok = false;
-          }
-        } else {
-          char* escaped = output_escape(rel, log_get_8_bit_output());
-          log_message(LOG_LEVEL_WARNING,
-                      "missing-args destination '%s' is a non-empty directory; use --force or "
-                      "--delete to remove it",
-                      escaped ? escaped : "<allocation failed>");
-          free(escaped);
-        }
-      } else if (errno != ENOENT) {
-        ok = false;
-      }
-    } else {
-      if (unlinkat(parent_fd, leaf, 0) == 0) {
-        removed = true;
-      } else if (errno != ENOENT) {
-        ok = false;
-      }
-    }
-    if (removed) {
-      budget->deleted++;
-      if (observer)
-        observer(observer_context, rel, delete_entry_type_of_mode(st.st_mode));
-      char* escaped = output_escape(rel, log_get_8_bit_output());
-      fprintf(stderr, "  Deleted: %s\n", escaped ? escaped : "<allocation failed>");
-      free(escaped);
-    }
-    if (parent_fd >= 0)
-      close(parent_fd);
-    free(leaf);
-    free(full);
-    if (!ok)
       break;
   }
   delete_skips_free(&skips);
@@ -398,7 +426,8 @@ bool manifest_would_delete_list(const Config* config, const DeleteManifest* mani
   if (!delete_skips_build(config, manifest->protected, NULL, true, &skips))
     return false;
   DeleteProtectRules protect = {.base_rules = config->protect_rules,
-                                .dir_rules = manifest->per_dir_rules};
+                                .dir_rules = manifest->per_dir_rules,
+                                .backup_suffix = delete_backup_suffix(config)};
   bool ok = delete_extras_list(config->receive_root_directory, manifest->keeps, manifest->dirs,
                                skips.entries, skips.count, &protect, out, count_out);
   delete_skips_free(&skips);

src/shared/delete_plan.c

@@ -801,6 +801,7 @@ static bool build_plan_skips(const Config* config, const DeletePlanSession* sess
                              PlanSkips* out) {
   out->protect.base_rules = config->protect_rules;
   out->protect.dir_rules = session->per_dir_rules;
+  out->protect.backup_suffix = delete_backup_suffix(config);
   /* The per-directory plan walk keeps each basis path verbatim (it does not
      convert an absolute under-root path to its root-relative form, unlike the
      whole-tree commit walk). */

src/shared/delta.c

@@ -1,10 +1,12 @@
 #include "delta.h"
 #include "log.h"
 #include "protocol.h"
+#include <errno.h>
 #include <stdint.h>
 #include <limits.h>
 #include <stdlib.h>
 #include <string.h>
+#include <unistd.h>
 
 #define XXH_STATIC_LINKING_ONLY
 #define XXH_IMPLEMENTATION
@@ -82,6 +84,64 @@ DeltaSignature* delta_signature_create_seeded(const void* old_file_data, uint64_
   return sig;
 }
 
+/* Bounded read of exactly `len` bytes at `off`; retries on EINTR. */
+static bool pread_all(int fd, void* buf, size_t len, uint64_t off) {
+  uint8_t* p = buf;
+  size_t done = 0;
+  while (done < len) {
+    ssize_t n = pread(fd, p + done, len - done, (off_t)(off + done));
+    if (n < 0 && errno == EINTR)
+      continue;
+    if (n <= 0)
+      return false;
+    done += (size_t)n;
+  }
+  return true;
+}
+
+DeltaSignature* delta_signature_create_fd_seeded(int fd, uint64_t old_file_size,
+                                                 uint32_t block_size, uint32_t seed) {
+  if (fd < 0 || old_file_size == 0 || block_size == 0 || block_size > DELTA_BLOCK_SIZE_MAX ||
+      old_file_size > UINT32_MAX * (uint64_t)block_size)
+    return NULL;
+  uint32_t block_count = (uint32_t)((old_file_size + block_size - 1) / block_size);
+  /* Bound the signature's own memory (block_count * sizeof(DeltaBlockSig)). */
+  if (block_count == 0 || block_count > MAX_DELTA_BLOCKS)
+    return NULL;
+  DeltaSignature* sig = protocol_alloc(sizeof(DeltaSignature));
+  if (!sig)
+    return NULL;
+  sig->file_size = old_file_size;
+  sig->block_size = block_size;
+  sig->block_count = block_count;
+  sig->blocks = protocol_alloc((size_t)block_count * sizeof(DeltaBlockSig));
+  if (!sig->blocks) {
+    free(sig);
+    return NULL;
+  }
+  uint8_t* block = malloc(block_size);
+  if (!block) {
+    free(sig->blocks);
+    free(sig);
+    return NULL;
+  }
+  for (uint32_t i = 0; i < block_count; i++) {
+    uint64_t offset = (uint64_t)i * block_size;
+    uint32_t len =
+        (uint32_t)((old_file_size - offset < block_size) ? (old_file_size - offset) : block_size);
+    if (!pread_all(fd, block, len, offset)) {
+      free(block);
+      free(sig->blocks);
+      free(sig);
+      return NULL;
+    }
+    sig->blocks[i].adler32 = delta_adler32(block, len);
+    sig->blocks[i].xxhash = delta_xxhash32_seeded(block, len, seed);
+  }
+  free(block);
+  return sig;
+}
+
 Data* delta_signature_serialize(const DeltaSignature* sig) {
   if (!sig)
     return NULL;
@@ -687,6 +747,130 @@ void* delta_apply(const void* old_data, uint64_t old_size, const Delta* delta,
   return output;
 }
 
+void* delta_apply_fd(int src_fd, uint64_t old_size, const Delta* delta, uint32_t block_size) {
+  if (!delta || block_size == 0 || block_size > DELTA_BLOCK_SIZE_MAX ||
+      (delta->instruction_count > 0 && !delta->instructions) || delta->new_file_size == 0 ||
+      delta->new_file_size > SIZE_MAX)
+    return NULL;
+
+  void* output = protocol_alloc((size_t)delta->new_file_size);
+  if (!output)
+    return NULL;
+
+  uint8_t* out = (uint8_t*)output;
+  uint64_t out_pos = 0;
+
+  for (uint32_t i = 0; i < delta->instruction_count; i++) {
+    if (delta->instructions[i].type == DELTA_INSTR_BLOCK_MATCH) {
+      uint64_t src_offset = (uint64_t)delta->instructions[i].match.block_index * block_size;
+      if (src_offset > UINT64_MAX - delta->instructions[i].match.block_offset) {
+        free(output);
+        return NULL;
+      }
+      src_offset += delta->instructions[i].match.block_offset;
+      uint32_t len = delta->instructions[i].match.length;
+
+      if (src_offset > old_size || (uint64_t)len > old_size - src_offset ||
+          out_pos > delta->new_file_size || (uint64_t)len > delta->new_file_size - out_pos) {
+        free(output);
+        return NULL;
+      }
+      if (!pread_all(src_fd, out + out_pos, len, src_offset)) {
+        free(output);
+        return NULL;
+      }
+      out_pos += len;
+    } else if (delta->instructions[i].type == DELTA_INSTR_LITERAL) {
+      uint32_t len = delta->instructions[i].literal.length;
+      if (out_pos > delta->new_file_size || (uint64_t)len > delta->new_file_size - out_pos) {
+        free(output);
+        return NULL;
+      }
+      memcpy(out + out_pos, delta->instructions[i].literal.data, len);
+      out_pos += len;
+    } else {
+      free(output);
+      return NULL;
+    }
+  }
+
+  if (out_pos != delta->new_file_size) {
+    free(output);
+    return NULL;
+  }
+  return output;
+}
+
+bool delta_apply_to_fd(const void* old_data, int src_fd, uint64_t old_size, const Delta* delta,
+                       uint32_t block_size, int dst_fd) {
+  if (!delta || (old_data == NULL && src_fd < 0) || block_size == 0 ||
+      block_size > DELTA_BLOCK_SIZE_MAX || (delta->instruction_count > 0 && !delta->instructions))
+    return false;
+  const int chunk = 1 << 20;
+  uint8_t* buf = malloc((size_t)chunk);
+  if (!buf)
+    return false;
+  uint64_t out_pos = 0;
+  bool ok = true;
+  for (uint32_t i = 0; i < delta->instruction_count && ok; i++) {
+    uint64_t src_offset = 0;
+    uint64_t len = 0;
+    const uint8_t* lit = NULL;
+    if (delta->instructions[i].type == DELTA_INSTR_BLOCK_MATCH) {
+      src_offset = (uint64_t)delta->instructions[i].match.block_index * block_size;
+      if (src_offset > UINT64_MAX - delta->instructions[i].match.block_offset ||
+          src_offset + delta->instructions[i].match.block_offset > old_size) {
+        ok = false;
+        break;
+      }
+      src_offset += delta->instructions[i].match.block_offset;
+      len = delta->instructions[i].match.length;
+      if (len > old_size - src_offset) {
+        ok = false;
+        break;
+      }
+    } else if (delta->instructions[i].type == DELTA_INSTR_LITERAL) {
+      lit = delta->instructions[i].literal.data;
+      len = delta->instructions[i].literal.length;
+    } else {
+      ok = false;
+      break;
+    }
+    if (out_pos > delta->new_file_size || len > delta->new_file_size - out_pos) {
+      ok = false;
+      break;
+    }
+    uint64_t done = 0;
+    while (ok && done < len) {
+      size_t want = (len - done) < (uint64_t)chunk ? (size_t)(len - done) : (size_t)chunk;
+      if (lit) {
+        memcpy(buf, lit + done, want);
+      } else if (old_data) {
+        memcpy(buf, (const uint8_t*)old_data + src_offset + done, want);
+      } else if (!pread_all(src_fd, buf, want, src_offset + done)) {
+        ok = false;
+        break;
+      }
+      const uint8_t* p = buf;
+      size_t written = 0;
+      while (written < want) {
+        ssize_t n = write(dst_fd, p + written, want - written);
+        if (n < 0 && errno == EINTR)
+          continue;
+        if (n <= 0) {
+          ok = false;
+          break;
+        }
+        written += (size_t)n;
+      }
+      done += want;
+    }
+    out_pos += len;
+  }
+  free(buf);
+  return ok && out_pos == delta->new_file_size;
+}
+
 void delta_destroy(Delta* delta) {
   if (!delta)
     return;

src/shared/delta.h

@@ -61,6 +61,13 @@ DeltaSignature* delta_signature_create(const void* old_file_data, uint64_t old_f
  * identical to the unseeded function. */
 DeltaSignature* delta_signature_create_seeded(const void* old_file_data, uint64_t old_file_size,
                                               uint32_t block_size, uint32_t seed);
+/* Streaming equivalent of delta_signature_create_seeded: reads the basis blocks
+ * from `fd` in bounded chunks, so an arbitrarily large basis can be signed
+ * without materializing it.  The signature itself is bounded (MAX_DELTA_BLOCKS
+ * entries); an over-large basis returns NULL and the caller falls back to a
+ * whole-file transfer. */
+DeltaSignature* delta_signature_create_fd_seeded(int fd, uint64_t old_file_size,
+                                                 uint32_t block_size, uint32_t seed);
 Data* delta_signature_serialize(const DeltaSignature* sig);
 DeltaSignature* delta_signature_deserialize(const Data* data);
 void delta_signature_destroy(DeltaSignature* sig);
@@ -75,6 +82,15 @@ Delta* delta_compute_seeded(const void* new_file_data, uint64_t new_file_size,
 Data* delta_serialize(const Delta* delta);
 Delta* delta_deserialize(const Data* data);
 void* delta_apply(const void* old_data, uint64_t old_size, const Delta* delta, uint32_t block_size);
+/* Streaming equivalent of delta_apply: matched blocks are read from `src_fd` as
+ * they are emitted, so the basis never has to be resident. */
+void* delta_apply_fd(int src_fd, uint64_t old_size, const Delta* delta, uint32_t block_size);
+/* Fully streamed reconstruction: matched blocks come from `old_data` (when
+ * non-NULL) or are read from `src_fd`, and the reconstructed bytes are written
+ * straight to `dst_fd` in bounded chunks, so a reconstructed file larger than
+ * memory is never materialized. */
+bool delta_apply_to_fd(const void* old_data, int src_fd, uint64_t old_size, const Delta* delta,
+                       uint32_t block_size, int dst_fd);
 void delta_destroy(Delta* delta);
 
 bool delta_should_attempt(uint64_t old_size, uint64_t new_size, uint64_t max_file_size);

src/shared/file.c

@@ -16,6 +16,8 @@
 
 #include "data.h"
 #include "checksum.h"
+#include "chunk.h"
+#include "compression.h"
 #include "delta.h"
 #include "file.h"
 #include "file_store.h"
@@ -209,6 +211,7 @@ File* file_create(const char* path) {
   file->dir_time_only = false;
   file->basis_link = NULL;
   file->basis_copy = NULL;
+  file->data_spool = false;
   file->link_group = 0;
   file->link_first = false;
   file->hardlink_target = NULL;
@@ -238,8 +241,11 @@ void file_destroy(void* item) {
   file->send_path = NULL;
   free(file->basis_link);
   file->basis_link = NULL;
+  if (file->data_spool && file->basis_copy)
+    unlink(file->basis_copy);
   free(file->basis_copy);
   file->basis_copy = NULL;
+  file->data_spool = false;
   free(file->hardlink_target);
   file->hardlink_target = NULL;
   free(file->symlink_target);
@@ -381,6 +387,261 @@ size_t file_content_to_buffer(File* file) {
   return bytes_read;
 }
 
+/* ---- Streamed whole-file payload receive ----
+ *
+ * A whole-file data frame is a uint64 length followed by that many bytes.  When
+ * the logical payload is at or below the receiver's streaming bound the
+ * historical charged whole-buffer path is kept (decompressing in one shot for
+ * `-z`).  Above the bound the frame is read in bounded chunks and written into
+ * a spool temp file next to the destination, decompressing incrementally for
+ * zstd/zlib (lz4's block format cannot be streamed and keeps the buffered path).
+ * The spool is then installed by the existing bounded-buffer basis-copy path
+ * (file_copy_basis_stream_attrs), so the atomic temp+rename store, --partial/
+ * --partial-dir, --delay-updates, --preallocate and metadata/xattr application
+ * are all reused unchanged.  Only bounded buffers (64 KiB read chunk + the
+ * decompressor's 256 KiB output window) are ever live. */
+
+#define FILE_PAYLOAD_READ_CHUNK (64 * 1024)
+#define FILE_PAYLOAD_PEEK_MAX 32
+
+/* Create a confined spool temp file in the destination's directory.  Returns an
+ * open write fd and an owned absolute path, or -1 (errno set).  The parent walk
+ * creates missing directories exactly as a normal store would. */
+static int file_spool_create(const char* dest_path, char** out_spool_path) {
+  *out_spool_path = NULL;
+  char* leaf = NULL;
+  int dirfd = file_open_secure_parent(dest_path, &leaf, true);
+  free(leaf);
+  if (dirfd < 0)
+    return -1;
+  char name[64];
+  for (unsigned int i = 0; i < 100; i++) {
+    snprintf(name, sizeof(name), ".fastsync-spool.%ld.%llu", (long)getpid(), next_temp_sequence());
+    int fd = openat(dirfd, name, O_WRONLY | O_CREAT | O_EXCL | O_CLOEXEC | O_NOFOLLOW, 0600);
+    if (fd < 0) {
+      if (errno != EEXIST)
+        break;
+      continue;
+    }
+    char* copy = str_dup(dest_path);
+    const char* dir = copy ? dirname(copy) : NULL;
+    size_t need = dir ? strlen(dir) + 1 + strlen(name) + 1 : 0;
+    char* full = need ? malloc(need) : NULL;
+    if (!full) {
+      free(copy);
+      close(fd);
+      unlinkat(dirfd, name, 0);
+      close(dirfd);
+      return -1;
+    }
+    snprintf(full, need, "%s/%s", dir, name);
+    free(copy);
+    close(dirfd);
+    *out_spool_path = full;
+    return fd;
+  }
+  close(dirfd);
+  return -1;
+}
+
+int file_spool_for_payload(const char* dest_path, char** out_spool_path) {
+  return file_spool_create(dest_path, out_spool_path);
+}
+
+bool file_receive_payload(int fd, bool compress, unsigned long long expected_size,
+                          const char* dest_path, unsigned long long stream_limit, Data** out_buffer,
+                          char** out_spool, unsigned long long* out_size) {
+  if (out_buffer)
+    *out_buffer = NULL;
+  if (out_spool)
+    *out_spool = NULL;
+  if (out_size)
+    *out_size = 0;
+  if (!out_buffer || !out_spool || !out_size || !dest_path)
+    return false;
+
+  unsigned long long frame_size = 0;
+  if (!receive_n_data(fd, &frame_size, sizeof(frame_size)))
+    return false;
+  /* A frame larger than MAX_DATA_PAYLOAD_SIZE is allowed only on the streamed
+     path, which never materializes it; the buffered path's
+     receive_data_alloc/body enforces the historical bound itself.  Only a size
+     unrepresentable on this platform is rejected here. */
+  if (frame_size > SIZE_MAX) {
+    log_message(LOG_LEVEL_ERROR, "Data size %llu is not representable", frame_size);
+    return false;
+  }
+
+  unsigned char prefix[FILE_PAYLOAD_PEEK_MAX];
+  size_t prefix_len = 0;
+  bool stream;
+  if (expected_size != 0) {
+    /* The check frame already told us the logical size: no need to peek. */
+    stream = expected_size > stream_limit;
+  } else if (!compress) {
+    stream = frame_size > stream_limit;
+  } else {
+    /* Non-incremental compressed frame with unknown logical size: peek the
+       header to decide, so a small compressed frame that expands past the bound
+       still streams instead of allocating the whole logical image. */
+    size_t want = frame_size < sizeof(prefix) ? (size_t)frame_size : sizeof(prefix);
+    if (want > 0 && !receive_n_data(fd, prefix, want))
+      return false;
+    prefix_len = want;
+    unsigned long long logical = compression_peek_frame_content_size(prefix, prefix_len);
+    stream = logical != 0 ? logical > stream_limit : frame_size > stream_limit;
+  }
+
+  if (!stream) {
+    Data* frame;
+    if (prefix_len > 0) {
+      frame = receive_data_alloc(fd, frame_size);
+      if (!frame)
+        return false;
+      memcpy(frame->data, prefix, prefix_len);
+      if (frame_size > prefix_len &&
+          !receive_n_data(fd, (char*)frame->data + prefix_len, (size_t)(frame_size - prefix_len))) {
+        data_destroy(frame);
+        return false;
+      }
+    } else {
+      frame = receive_data_body(fd, frame_size);
+      if (!frame)
+        return false;
+    }
+    if (compress) {
+      unsigned long long bound =
+          expected_size != 0 ? expected_size : (unsigned long long)MAX_RECEIVE_WHOLE_FILE_SIZE;
+      Data* uncompressed = data_decompress_limited(frame, (size_t)bound);
+      ProtocolSession* owner = frame->owner;
+      data_destroy(frame);
+      if (!uncompressed)
+        return false;
+      if (expected_size != 0 && uncompressed->size != expected_size) {
+        data_destroy(uncompressed);
+        return false;
+      }
+      if (!data_charge_session(uncompressed, owner, uncompressed->size)) {
+        data_destroy(uncompressed);
+        return false;
+      }
+      *out_buffer = uncompressed;
+      *out_size = uncompressed->size;
+      return true;
+    }
+    *out_buffer = frame;
+    *out_size = frame->size;
+    return true;
+  }
+
+  /* Streamed path. */
+  int spool_fd = file_spool_create(dest_path, out_spool);
+  if (spool_fd < 0)
+    return false;
+
+  CompressionStreamDecompressor* dec = NULL;
+  size_t header_len = 0;
+  unsigned long long learned_size = expected_size;
+  if (compress) {
+    unsigned char hdr[1 + 4];
+    size_t hdr_have = 0;
+    if (prefix_len >= 1) {
+      hdr[0] = prefix[0];
+      hdr_have = 1;
+    } else {
+      if (!receive_n_data(fd, hdr, 1))
+        goto stream_fail;
+      hdr_have = 1;
+    }
+    CompressionAlgo algo = (CompressionAlgo)hdr[0];
+    if (!compression_algo_valid((int)algo) || algo == COMPRESSION_ALGO_LZ4)
+      goto stream_fail;
+    header_len = (algo == COMPRESSION_ALGO_ZLIB || algo == COMPRESSION_ALGO_ZLIBX) ? 5 : 1;
+    while (hdr_have < header_len) {
+      size_t need = header_len - hdr_have;
+      if (prefix_len > hdr_have) {
+        size_t avail = prefix_len - hdr_have;
+        size_t take = avail < need ? avail : need;
+        memcpy(hdr + hdr_have, prefix + hdr_have, take);
+        hdr_have += take;
+      } else {
+        if (!receive_n_data(fd, hdr + hdr_have, need))
+          goto stream_fail;
+        hdr_have = header_len;
+      }
+    }
+    if (header_len == 5) {
+      uint32_t raw = 0;
+      for (int i = 0; i < 4; i++)
+        raw |= (uint32_t)hdr[1 + i] << (8 * i);
+      if (expected_size != 0 && raw != expected_size)
+        goto stream_fail;
+      if (learned_size == 0)
+        learned_size = raw;
+    }
+    dec = compression_stream_decompressor_create(algo, learned_size);
+    if (!dec)
+      goto stream_fail;
+  }
+
+  if (frame_size < header_len)
+    goto stream_fail;
+  {
+    unsigned long long body_remaining = frame_size - header_len;
+    if (prefix_len > header_len) {
+      size_t avail = prefix_len - header_len;
+      if (compress) {
+        bool done = false;
+        if (!compression_stream_decompressor_feed(dec, prefix + header_len, avail, spool_fd, &done))
+          goto stream_fail;
+      } else if (!write_all(spool_fd, prefix + header_len, avail)) {
+        goto stream_fail;
+      }
+      body_remaining -= avail;
+    }
+    unsigned char buf[FILE_PAYLOAD_READ_CHUNK];
+    while (body_remaining > 0) {
+      size_t want = body_remaining < sizeof(buf) ? (size_t)body_remaining : (size_t)sizeof(buf);
+      if (!receive_n_data(fd, buf, want))
+        goto stream_fail;
+      if (compress) {
+        bool done = false;
+        if (!compression_stream_decompressor_feed(dec, buf, want, spool_fd, &done))
+          goto stream_fail;
+      } else if (!write_all(spool_fd, buf, want)) {
+        goto stream_fail;
+      }
+      body_remaining -= want;
+    }
+  }
+
+  {
+    unsigned long long total = compress ? compression_stream_decompressor_total(dec) : frame_size;
+    if (learned_size != 0 && total != learned_size)
+      goto stream_fail;
+    *out_size = total;
+  }
+  if (dec)
+    compression_stream_decompressor_destroy(dec);
+  if (close(spool_fd) != 0) {
+    spool_fd = -1;
+    goto stream_fail;
+  }
+  return true;
+
+stream_fail:
+  if (dec)
+    compression_stream_decompressor_destroy(dec);
+  if (spool_fd >= 0)
+    close(spool_fd);
+  if (*out_spool) {
+    unlink(*out_spool);
+    free(*out_spool);
+    *out_spool = NULL;
+  }
+  return false;
+}
+
 /* ---- Secure filesystem primitives ---- */
 
 bool file_path_exists_secure(const char* path) {

src/shared/file.h

@@ -177,6 +177,24 @@ bool file_copy_basis_stream_attrs(const char* path, const char* basis_path,
                                   const FileMetadata* metadata, FileAttrPolicy policy, bool update,
                                   bool use_fsync, const FileXattrList* xattrs, bool fake_super,
                                   const char* temp_dir);
+/* Receive one length-prefixed whole-file data frame, streaming the payload
+ * through a bounded buffer when it (or its known logical size) exceeds
+ * `stream_limit`.  On success exactly one of *out_buffer / *out_spool is set:
+ *   - *out_buffer: the historical charged whole-buffer Data (caller destroys);
+ *   - *out_spool: an owned temp path holding the payload, installed through the
+ *     File's basis_copy field with File.data_spool set so file_destroy unlinks
+ *     it.  The destination policy/metadata/atomic-store handling is then the
+ *     existing bounded-buffer basis install (file_copy_basis_stream_attrs).
+ * `expected_size` (0 = unknown) is the logical size from the check frame;
+ * `dest_path` locates the spool next to the destination; `compress` selects
+ * incremental decompression.  Returns false on any framing/I/O/size error. */
+bool file_receive_payload(int fd, bool compress, unsigned long long expected_size,
+                          const char* dest_path, unsigned long long stream_limit, Data** out_buffer,
+                          char** out_spool, unsigned long long* out_size);
+/* Create a confined spool temp file next to `dest_path`; returns an open write
+ * fd and an owned absolute path (to be installed via File.basis_copy with
+ * File.data_spool set, and unlinked by file_destroy). */
+int file_spool_for_payload(const char* dest_path, char** out_spool_path);
 /* Protocol 2.28.0 receiver-stat variants: like the two above but additionally
  * report through `dirs_created` (when non-NULL) how many parent directories the
  * confined secure walk had to create that lie strictly below `count_floor` (a

src/shared/file_receive.c

@@ -58,36 +58,41 @@ File* file_receive(const Config* config, int file_descriptor) {
     file_destroy(file);
     return NULL;
   }
-  Data* file_data = receive_data_limited(file_descriptor, MAX_RECEIVE_WHOLE_FILE_SIZE);
+  bool compress =
-  if (file_data == NULL) {
+      config->use_compression && !compression_should_skip_with_suffixes(
+                                     file->path, config->skip_compress_suffixes,
+                                     config->skip_compress_set ? config->skip_compress_count : -1);
+  char* dest_path = path_cat(config->receive_root_directory, file->path);
+  if (!dest_path) {
     file_destroy(file);
     return NULL;
   }
-  if (config->use_compression &&
+  Data* buffer = NULL;
-      !compression_should_skip_with_suffixes(file->path, config->skip_compress_suffixes,
+  char* spool = NULL;
-                                             config->skip_compress_set ? config->skip_compress_count
+  unsigned long long size = 0;
-                                                                       : -1)) {
+  bool ok = file_receive_payload(file_descriptor, compress, 0, dest_path,
-    Data* file_data_uncompressed = data_decompress_limited(file_data, MAX_RECEIVE_WHOLE_FILE_SIZE);
+                                 protocol_whole_file_receive_limit(), &buffer, &spool, &size);
-    ProtocolSession* owner = file_data->owner;
+  free(dest_path);
-    data_destroy(file_data);
+  if (!ok) {
-    if (file_data_uncompressed == NULL) {
     file_destroy(file);
     return NULL;
   }
-    if (!data_charge_session(file_data_uncompressed, owner, file_data_uncompressed->size)) {
+  if (spool) {
-      data_destroy(file_data_uncompressed);
+    Data* reserved = data_create_reserve((size_t)size);
+    if (!reserved) {
+      unlink(spool);
+      free(spool);
       file_destroy(file);
       return NULL;
     }
-    if (file_data_uncompressed->size > MAX_FILE_DATA_SIZE) {
-      data_destroy(file_data_uncompressed);
-      file_destroy(file);
-      return NULL;
-    }
-    file_data = file_data_uncompressed;
-  }
     data_destroy(file->data);
-  file->data = file_data;
+    file->data = reserved;
+    file->basis_copy = spool;
+    file->data_spool = true;
+  } else {
+    data_destroy(file->data);
+    file->data = buffer;
+  }
   return file;
 }
 
@@ -97,12 +102,14 @@ bool dir_metadata_should_capture(const Config* config) {
   /* Directory metadata is captured when a directory attribute is actually
    * requested: -p/--perms (directory modes), -t/--times (directory mtimes,
    * unless -O/--omit-dir-times suppresses them), -o/-g (directory ownership),
-   * or -X/-A (directory xattrs/ACLs).  --atimes/-U alone does not pull
+   * -X/-A (directory xattrs/ACLs), or --fake-super (whose reserved %stat record
-   * directory metadata (matching the original dir-time bundle). */
+   * is written on the directory itself, so its metadata must travel).
+   * --atimes/-U alone does not pull directory metadata (matching the original
+   * dir-time bundle). */
   return config && config->use_metadata &&
          (config->preserve_perms || (config->preserve_times && !config->omit_dir_times) ||
           config->preserve_owner || config->preserve_group || config->preserve_xattrs ||
-          config->preserve_acls);
+          config->preserve_acls || config->fake_super);
 }
 
 void dir_time_list_init(DirTimeList* list) {
@@ -200,12 +207,19 @@ void dir_metadata_list_apply(const DirTimeList* list, const char* root_directory
   bool apply_times = config->preserve_times && !config->omit_dir_times;
   bool apply_mode = config->preserve_perms;
   bool apply_xattrs = config->use_xattrs;
+  bool apply_fake_super = config->fake_super;
   /* Ownership is applied through the active identity snapshot (which no-ops
-   * unless an ownership request is active), and xattrs only when -X/-A was
+   * unless an ownership request is active), xattrs only when -X/-A was
-   * negotiated.  Times/mode keep their own per-attribute gates. */
+   * negotiated, and the --fake-super record whenever the flag is active.
-  bool have_any = apply_times || apply_mode || apply_xattrs || identity_active_enabled();
+   * Times/mode keep their own per-attribute gates. */
+  bool have_any =
+      apply_times || apply_mode || apply_xattrs || apply_fake_super || identity_active_enabled();
   if (!have_any)
     return;
+  /* Built once: the --fake-super replay uses it to apply only the recorded
+   * permission bits (the special bits stay in the record, exactly like the
+   * regular-file fake-super receiver). */
+  FileAttrPolicy policy = file_attr_policy_from_config(config);
   for (size_t i = 0; i < list->count; i++) {
     char* dir_path = path_cat(root_directory, list->paths[i]);
     if (!dir_path)
@@ -255,10 +269,12 @@ void dir_metadata_list_apply(const DirTimeList* list, const char* root_directory
         free(escaped_path);
       }
     }
-    if (apply_mode) {
+    /* The final directory mode (after any --chmod) is computed once so the
+       --fake-super record can carry it even when the on-disk replay is
+       restricted to the permission bits below. */
     mode_t dir_mode = list->entries[i].mode;
     bool mode_ready = true;
-      if (config->chmod_spec && *config->chmod_spec &&
+    if (apply_mode && config->chmod_spec && *config->chmod_spec &&
         !chmod_apply(dir_mode, config->chmod_spec, &dir_mode)) {
       char* escaped_path = output_escape(dir_path, log_get_8_bit_output());
       log_message(LOG_LEVEL_WARNING, "Failed to apply --chmod to directory %s",
@@ -266,7 +282,13 @@ void dir_metadata_list_apply(const DirTimeList* list, const char* root_directory
       free(escaped_path);
       mode_ready = false;
     }
-      if (mode_ready) {
+    /* Under --fake-super the normal fchmod below still applies the mode, but
+       the fake-super replay that follows narrows the on-disk result to the
+       recorded permission bits (the full mode, including setuid/setgid/sticky,
+       lives only in the record).  Keeping the normal fchmod first means a
+       filesystem without xattr support still gets the directory mode rather than
+       silently losing it. */
+    if (apply_mode && mode_ready) {
       /* rsync -p copies the source directory mode exactly, including
        * group/other write and the setgid/sticky bits.  Setuid/setgid/sticky
        * are super-user activities: when the connection forbade them
@@ -286,6 +308,19 @@ void dir_metadata_list_apply(const DirTimeList* list, const char* root_directory
         free(escaped_path);
       }
     }
+    /* --fake-super: park the directory's full stat (rsync 3.4.1's exact
+       grammar) on the directory ITSELF, then replay only the recorded
+       permission bits fd-relative.  The special bits live only in the record
+       and the recorded ownership is never real-chowned: the resolved ids are
+       stored for a later privileged restore, exactly like the file path.  Runs
+       before the xattr apply so a mode change cannot clobber the ACL mask. */
+    if (apply_fake_super && dir_fd >= 0) {
+      uint32_t store_uid = 0;
+      uint32_t store_gid = 0;
+      identity_resolve_storage_ids((int32_t)list->entries[i].uid, (int32_t)list->entries[i].gid,
+                                   &store_uid, &store_gid);
+      fake_super_store_fd(dir_fd, store_uid, store_gid, (uint32_t)dir_mode, 0, 0);
+      fake_super_restore_fd(dir_fd, policy);
     }
     /* xattrs/ACLs last: a mode change can rewrite the ACL mask, so the ACL
        xattrs must be (re)applied after fchmod. */

src/shared/file_save.c

@@ -817,6 +817,21 @@ static FileSaveResult file_save_directory_to_disk(const FileSavePlan* plan, bool
   } else if (ok && identity_copy_as_active()) {
     ok = false;
   }
+  /* --fake-super: park the directory's full stat in rsync's reserved
+     user.rsync.%stat xattr as soon as the directory exists.  This makes even a
+     direct file_save_to_disk_full() caller -- which never runs the deferred
+     DirTimeList pass -- produce an rsync-readable fake-super record.  The record
+     carries the full mode/uid/gid; the permission bits are replayed by the
+     deferred pass (never inline, so a restrictive mode cannot block child
+     creation) and the recorded ownership is never real-chowned.  Best-effort:
+     fake_super_store_fd() logs and skips a failure, never failing the entry. */
+  if (ok && plan->config && plan->config->fake_super && file->metadata && dir_fd >= 0) {
+    uint32_t store_uid = 0;
+    uint32_t store_gid = 0;
+    identity_resolve_storage_ids((int32_t)file->metadata->uid, (int32_t)file->metadata->gid,
+                                 &store_uid, &store_gid);
+    fake_super_store_fd(dir_fd, store_uid, store_gid, (uint32_t)file->metadata->mode, 0, 0);
+  }
   /* The final source MODE is deliberately NOT applied inline.  A restrictive
      source mode (for example 0555) would make the directory unwritable before
      its children are created, so a non-root receiver fails each child with

src/shared/file_send.c

@@ -44,12 +44,138 @@ bool file_send_single_calls(File* file, int file_descriptor, bool use_metadata,
                                           send_path, NULL, -1, 0, false);
 }
 
+/* Stream-compress a whole source file into a temp file, then transmit it as the
+ * normal length-prefixed data frame.  Used when the source was too large to
+ * load (File.data.data == NULL): the source is read in bounded chunks through a
+ * streaming codec, so neither the raw nor the compressed image is held in
+ * memory.  The compressed length must be known before the frame is sent (the
+ * wire is length-prefixed), so the stream lands in a private temp file first. */
+bool file_send_compressed_stream_with_skip(File* file, int file_descriptor, bool use_metadata,
+                                           int compression_level, bool send_path,
+                                           char* const* skip_suffixes, int skip_count,
+                                           int compression_threads, bool send_xattrs) {
+  /* The caller has already decided this file compresses; the skip list is part
+     of the public signature for symmetry with the buffered path. */
+  (void)skip_suffixes;
+  (void)skip_count;
+  if (!file || !file->path || !file->data || file->data->size == 0 || file->data->data != NULL)
+    return false;
+  CompressionAlgo algo = compression_get_algo();
+  CompressionStreamCompressor* compressor =
+      compression_stream_compressor_create(algo, compression_level, compression_threads);
+  if (!compressor) {
+    log_message(LOG_LEVEL_ERROR, "streaming compression is not available for this codec; "
+                                 "the source was not loaded for the buffered path");
+    return false;
+  }
+
+  int src = file_open_for_read(file->path);
+  if (src < 0) {
+    compression_stream_compressor_destroy(compressor);
+    return false;
+  }
+  struct stat src_st;
+  if (fstat(src, &src_st) != 0 || !S_ISREG(src_st.st_mode) ||
+      (unsigned long long)src_st.st_size < file->data->size) {
+    close(src);
+    compression_stream_compressor_destroy(compressor);
+    return false;
+  }
+
+  const char* tmpdir = getenv("TMPDIR");
+  if (!tmpdir || tmpdir[0] == '\0')
+    tmpdir = "/tmp";
+  size_t tmplen = strlen(tmpdir) + strlen("/fastsync-z-XXXXXX") + 1;
+  char* tmpl = malloc(tmplen);
+  if (!tmpl) {
+    close(src);
+    compression_stream_compressor_destroy(compressor);
+    return false;
+  }
+  snprintf(tmpl, tmplen, "%s/fastsync-z-XXXXXX", tmpdir);
+  int tmp_fd = mkstemp(tmpl);
+  if (tmp_fd < 0) {
+    log_perror("Could not create compression temp file");
+    free(tmpl);
+    close(src);
+    compression_stream_compressor_destroy(compressor);
+    return false;
+  }
+  unlink(tmpl);
+  free(tmpl);
+
+  bool ok = compression_stream_compressor_begin(compressor, file->data->size, tmp_fd);
+  unsigned char buf[64 * 1024];
+  unsigned long long remaining = file->data->size;
+  while (ok && remaining > 0) {
+    size_t want = remaining < sizeof(buf) ? (size_t)remaining : sizeof(buf);
+    ssize_t got = read(src, buf, want);
+    if (got <= 0) {
+      ok = false;
+      break;
+    }
+    if (!compression_stream_compressor_feed(compressor, buf, (size_t)got, tmp_fd))
+      ok = false;
+    remaining -= (unsigned long long)got;
+  }
+  if (ok)
+    ok = compression_stream_compressor_finish(compressor, tmp_fd);
+  close(src);
+  compression_stream_compressor_destroy(compressor);
+  if (!ok) {
+    close(tmp_fd);
+    return false;
+  }
+  struct stat tmp_st;
+  if (fstat(tmp_fd, &tmp_st) != 0 || tmp_st.st_size < 0) {
+    close(tmp_fd);
+    return false;
+  }
+  unsigned long long compressed_size = (unsigned long long)tmp_st.st_size;
+  if (lseek(tmp_fd, 0, SEEK_SET) == (off_t)-1) {
+    close(tmp_fd);
+    return false;
+  }
+
+  ok = true;
+  if (send_path && !send_wire_str(file_descriptor, file_wire_path(file)))
+    ok = false;
+  if (ok && use_metadata && !metadata_send(file_descriptor, file->metadata))
+    ok = false;
+  if (ok && send_xattrs && !xattr_send(file_descriptor, file ? file->xattrs : NULL))
+    ok = false;
+  if (ok && !send_n_data(file_descriptor, &compressed_size, sizeof(compressed_size)))
+    ok = false;
+  unsigned long long left = compressed_size;
+  while (ok && left > 0) {
+    size_t want = left < sizeof(buf) ? (size_t)left : sizeof(buf);
+    ssize_t got = read(tmp_fd, buf, want);
+    if (got <= 0 || !send_n_data(file_descriptor, buf, (size_t)got)) {
+      ok = false;
+      break;
+    }
+    left -= (unsigned long long)got;
+  }
+  close(tmp_fd);
+  return ok;
+}
+
 bool file_send_single_calls_with_skip(File* file, int file_descriptor, bool use_metadata,
                                       int compression_level, bool send_path,
                                       char* const* skip_suffixes, int skip_count,
                                       int compression_threads, bool send_xattrs) {
-  if (!file || !file->path || !file->data || (file->data->size != 0 && !file->data->data))
+  if (!file || !file->path || !file->data)
     return false;
+  /* A source too large to load is streamed: compression streams through a
+   * temp file, no compression must have taken the sendfile path instead. */
+  if (file->data->size != 0 && file->data->data == NULL) {
+    if (compression_level <= 0 ||
+        compression_should_skip_with_suffixes(file->path, skip_suffixes, skip_count))
+      return false;
+    return file_send_compressed_stream_with_skip(file, file_descriptor, use_metadata,
+                                                 compression_level, send_path, skip_suffixes,
+                                                 skip_count, compression_threads, send_xattrs);
+  }
   const Data* data_to_send = file->data;
   Data* compressed_data = NULL;
   if (compression_level > 0 &&

src/shared/file_send.h

@@ -13,6 +13,12 @@ bool file_send_single_calls_with_skip(File* file, int file_descriptor, bool use_
                                       int compression_level, bool send_path,
                                       char* const* skip_suffixes, int skip_count,
                                       int compression_threads, bool send_xattrs);
+/* Stream-compress an unloaded whole source file into a temp file and send it as
+ * the usual data frame (see file_send.c). */
+bool file_send_compressed_stream_with_skip(File* file, int file_descriptor, bool use_metadata,
+                                           int compression_level, bool send_path,
+                                           char* const* skip_suffixes, int skip_count,
+                                           int compression_threads, bool send_xattrs);
 bool file_send_sendfile(File* file, int file_descriptor, bool use_metadata, int compression_level,
                         bool send_path);
 bool file_send_sendfile_with_skip(File* file, int file_descriptor, bool use_metadata,

src/shared/file_types.h

@@ -62,6 +62,11 @@ typedef struct {
    * This lets a basis larger than any whole-file bound materialize without
    * buffering it; the source metadata on `metadata` is applied afterwards. */
   char* basis_copy;
+  /* Receiver-only.  When set, `basis_copy` points at a receiver-created spool
+   * temp file holding a STREAMED whole-file payload (rather than a --copy-dest
+   * basis).  file_destroy unlinks it after the install consumes it, so an
+   * over-limit file leaves no scratch behind. */
+  bool data_spool;
   /* --hard-links (-H), sender + receiver wire state.  link_group is a run-local
    * id shared by every member of one source inode (0 = not part of a group).
    * The FIRST member (link_first == true) carries its data on the wire and is

src/shared/identity.c

@@ -276,7 +276,7 @@ static bool identity_wire_map_valid(const IdentityMap* map) {
   }
   if (map->to < IDENTITY_CURRENT)
     return false;
-  if (map->to_name && strlen(map->to_name) > 255)
+  if (map->to_name && strlen(map->to_name) > IDENTITY_MAX_NAME_LEN)
     return false;
   return true;
 }

src/shared/identity.h

@@ -6,6 +6,11 @@
 #include <stdint.h>
 #include <sys/types.h>
 
+/* Maximum length of a receiver-resolved identity name in a FROM:TO map's TO
+ * field.  Bounded so a malicious/huge name can never cross the wire (see
+ * identity_wire_map_valid). */
+#define IDENTITY_MAX_NAME_LEN 255
+
 /*
  * Identity mapping: --numeric-ids / --usermap / --groupmap / --chown / --copy-as.
  *

src/shared/incremental_check.c

@@ -44,99 +44,53 @@ bool receive_file_xattrs(File* file, int fd, const Config* config) {
   return true;
 }
 
-static File* receive_delta_file(int fd, const Config* config, const char* check_path,
+static bool receive_file_payload_into(File* file, int fd, const Config* config,
-                                void* old_data, unsigned long long old_size, bool* failed) {
+                                      const char* dest_path, unsigned long long expected_size);
-  if (!old_data) {
-    free(old_data); /* defensive: old_data is always non-NULL today */
-    *failed = true;
-    return NULL;
-  }
 
-  DeltaSignature* sig = delta_signature_create_seeded(old_data, old_size, config->delta_block_size,
+/* Receive a STATUS_DELTA_DATA response: the sender's delta against the basis we
-                                                      (uint32_t)config->checksum_seed);
+   signed.  Deserializes, decompresses and applies the delta (in memory or
-  if (!sig) {
+   through a spool temp file), then receives the metadata/xattr block and
-    free(old_data);
+   installs the reconstructed payload.  Takes ownership of `old_data` and `sig`,
-    *failed = true;
+   releasing both on every path. */
-    return NULL;
+static File* receive_delta_data_branch(int fd, const Config* config, const char* check_path,
-  }
+                                       void* old_data, unsigned long long old_size, int basis_fd,
+                                       DeltaSignature* sig, bool* failed) {
+  Data* raw_delta = NULL;
+  Delta* delta = NULL;
+  void* new_data = NULL;
+  char* spool = NULL;
+  File* file = NULL;
 
-  Data* sig_data = delta_signature_serialize(sig);
+  raw_delta = receive_data_limited(fd, MAX_RECEIVE_WHOLE_FILE_SIZE);
-  if (!sig_data) {
+  if (!raw_delta)
-    delta_signature_destroy(sig);
+    goto fail;
-    free(old_data);
-    *failed = true;
-    return NULL;
-  }
 
-  bool sig_sent = send_status(fd, STATUS_DELTA_SIGNATURE) && send_data(fd, sig_data);
-  data_destroy(sig_data);
-
-  if (!sig_sent) {
-    delta_signature_destroy(sig);
-    free(old_data);
-    *failed = true;
-    return NULL;
-  }
-
-  Status resp;
-  if (!receive_status(fd, &resp)) {
-    delta_signature_destroy(sig);
-    free(old_data);
-    *failed = true;
-    return NULL;
-  }
-
-  if (resp == STATUS_DELTA_DATA) {
-    Data* delta_data = receive_data_limited(fd, MAX_RECEIVE_WHOLE_FILE_SIZE);
-    if (!delta_data) {
-      delta_signature_destroy(sig);
-      free(old_data);
-      *failed = true;
-      return NULL;
-    }
-
-    Data* raw_delta = delta_data;
   if (config->use_compression &&
-        !compression_should_skip_with_suffixes(
+      !compression_should_skip_with_suffixes(check_path, config->skip_compress_suffixes,
-            check_path, config->skip_compress_suffixes,
+                                             config->skip_compress_set ? config->skip_compress_count
-            config->skip_compress_set ? config->skip_compress_count : -1)) {
+                                                                       : -1)) {
-      ProtocolSession* owner = delta_data->owner;
+    ProtocolSession* owner = raw_delta->owner;
-      raw_delta = data_decompress_limited(delta_data, MAX_RECEIVE_WHOLE_FILE_SIZE);
+    Data* decompressed = data_decompress_limited(raw_delta, MAX_RECEIVE_WHOLE_FILE_SIZE);
-      data_destroy(delta_data);
+    data_destroy(raw_delta);
-      if (!raw_delta) {
+    raw_delta = decompressed;
-        free(old_data);
+    if (!raw_delta)
-        delta_signature_destroy(sig);
+      goto fail;
-        *failed = true;
-        return NULL;
-      }
     /* Charge the decompressed delta to the connection budget (the paired
        wire buffer's charge was just released). */
-      if (!data_charge_session(raw_delta, owner, raw_delta->size)) {
+    if (!data_charge_session(raw_delta, owner, raw_delta->size))
-        data_destroy(raw_delta);
+      goto fail;
-        free(old_data);
-        delta_signature_destroy(sig);
-        *failed = true;
-        return NULL;
-      }
   }
 
-    Delta* delta = delta_deserialize(raw_delta);
+  delta = delta_deserialize(raw_delta);
   data_destroy(raw_delta);
-    if (!delta) {
+  raw_delta = NULL;
-      free(old_data);
+  if (!delta)
-      delta_signature_destroy(sig);
+    goto fail;
-      *failed = true;
-      return NULL;
-    }
 
   uint64_t new_size = delta->new_file_size;
-    if (new_size > MAX_RECEIVE_WHOLE_FILE_SIZE || new_size > SIZE_MAX) {
+  if (new_size > SIZE_MAX) {
-      delta_destroy(delta);
-      free(old_data);
-      delta_signature_destroy(sig);
     send_status(fd, STATUS_ERROR);
-      *failed = true;
+    goto fail;
-      return NULL;
   }
   /* Wire-stats tally: bytes taken straight from the basis file (matched
      delta blocks) and bytes shipped literally (protocol 2.28.0).  Computed
@@ -149,134 +103,180 @@ static File* receive_delta_file(int fd, const Config* config, const char* check_
     else if (delta->instructions[k].type == DELTA_INSTR_LITERAL)
       literal += delta->instructions[k].literal.length;
   }
-    void* new_data = delta_apply(old_data, old_size, delta, config->delta_block_size);
+
+  /* A reconstructed file above the streaming bound is written into a spool
+     temp file through delta_apply_to_fd; a smaller one keeps the historical
+     in-memory reconstruction. */
+  if (new_size > protocol_whole_file_receive_limit()) {
+    char* dest_path = path_cat(config->receive_root_directory, check_path);
+    int spool_fd = dest_path ? file_spool_for_payload(dest_path, &spool) : -1;
+    free(dest_path);
+    if (spool_fd < 0) {
+      send_status(fd, STATUS_ERROR);
+      goto fail;
+    }
+    bool applied =
+        delta_apply_to_fd(old_data, basis_fd, old_size, delta, config->delta_block_size, spool_fd);
+    if (close(spool_fd) != 0)
+      applied = false;
     delta_destroy(delta);
-
+    delta = NULL;
-    if (!new_data) {
+    if (!applied) {
-      free(old_data);
+      send_status(fd, STATUS_ERROR);
-      delta_signature_destroy(sig);
+      goto fail;
-      *failed = true;
+    }
-      return NULL;
+  } else {
+    new_data = old_data ? delta_apply(old_data, old_size, delta, config->delta_block_size)
+                        : delta_apply_fd(basis_fd, old_size, delta, config->delta_block_size);
+    delta_destroy(delta);
+    delta = NULL;
+    if (!new_data)
+      goto fail;
   }
 
-    File* file = file_create(check_path);
+  file = file_create(check_path);
-    if (!file) {
+  if (!file)
-      free(new_data);
+    goto fail;
-      free(old_data);
-      delta_signature_destroy(sig);
-      *failed = true;
-      return NULL;
-    }
   file->matched_bytes = matched;
   file->literal_bytes = literal;
 
   if (config->use_metadata) {
     int meta_ok = 1;
     file->metadata = metadata_receive(fd, &meta_ok);
-      if (!meta_ok) {
+    if (!meta_ok)
-        file_destroy(file);
+      goto fail;
-        free(new_data);
-        free(old_data);
-        delta_signature_destroy(sig);
-        *failed = true;
-        return NULL;
-      }
-    }
-    if (!receive_file_xattrs(file, fd, config)) {
-      file_destroy(file);
-      free(new_data);
-      free(old_data);
-      delta_signature_destroy(sig);
-      *failed = true;
-      return NULL;
   }
+  if (!receive_file_xattrs(file, fd, config))
+    goto fail;
 
-    Data* replacement = data_create(new_data, (size_t)new_size);
+  if (spool) {
-    if (replacement == NULL) {
+    Data* reserved = data_create_reserve((size_t)new_size);
-      file_destroy(file);
+    if (reserved == NULL) {
-      free(old_data);
-      delta_signature_destroy(sig);
       send_status(fd, STATUS_ERROR);
-      *failed = true;
+      goto fail;
-      return NULL;
+    }
+    data_destroy(file->data);
+    file->data = reserved;
+    file->basis_copy = spool;
+    spool = NULL; /* ownership moved into file->basis_copy */
+    file->data_spool = true;
+  } else {
+    Data* replacement = data_create(new_data, (size_t)new_size);
+    new_data = NULL; /* data_create owns, and frees, the buffer on failure */
+    if (replacement == NULL) {
+      send_status(fd, STATUS_ERROR);
+      goto fail;
     }
     data_destroy(file->data);
     file->data = replacement;
+  }
 
   free(old_data);
   delta_signature_destroy(sig);
   return file;
+
+fail:
+  free(new_data);
+  if (spool) {
+    unlink(spool);
+    free(spool);
+  }
+  file_destroy(file);
+  delta_destroy(delta);
+  data_destroy(raw_delta);
+  free(old_data);
+  delta_signature_destroy(sig);
+  *failed = true;
+  return NULL;
 }
 
-  if (resp == STATUS_NEXT) {
+/* Receive a STATUS_NEXT response: the sender declined the delta and will send
+   the whole file.  Releases the basis signature and snapshot, then receives the
+   metadata/xattr block and the full payload.  Takes ownership of `old_data` and
+   `sig`, releasing both immediately. */
+static File* receive_next_branch(int fd, const Config* config, const char* check_path,
+                                 unsigned long long expected_size, void* old_data,
+                                 DeltaSignature* sig, bool* failed) {
   delta_signature_destroy(sig);
   free(old_data);
 
   File* file = file_create(check_path);
-    if (!file) {
+  if (!file)
-      *failed = true;
+    goto fail;
-      return NULL;
-    }
 
   if (config->use_metadata) {
     int meta_ok = 1;
     file->metadata = metadata_receive(fd, &meta_ok);
-      if (!meta_ok) {
+    if (!meta_ok)
-        file_destroy(file);
+      goto fail;
-        *failed = true;
-        return NULL;
-      }
-    }
-    if (!receive_file_xattrs(file, fd, config)) {
-      file_destroy(file);
-      *failed = true;
-      return NULL;
   }
+  if (!receive_file_xattrs(file, fd, config))
+    goto fail;
 
-    Data* file_data = receive_data_limited(fd, MAX_RECEIVE_WHOLE_FILE_SIZE);
+  char* dest_path = path_cat(config->receive_root_directory, check_path);
-    if (file_data == NULL) {
+  if (!dest_path)
-      file_destroy(file);
+    goto fail;
-      *failed = true;
+  bool payload_ok = receive_file_payload_into(file, fd, config, dest_path, expected_size);
-      return NULL;
+  free(dest_path);
-    }
+  if (!payload_ok)
-
+    goto fail;
-    if (config->use_compression &&
-        !compression_should_skip_with_suffixes(
-            file->path, config->skip_compress_suffixes,
-            config->skip_compress_set ? config->skip_compress_count : -1)) {
-      Data* uncompressed = data_decompress_limited(file_data, MAX_RECEIVE_WHOLE_FILE_SIZE);
-      ProtocolSession* owner = file_data->owner;
-      data_destroy(file_data);
-      if (uncompressed == NULL) {
-        file_destroy(file);
-        *failed = true;
-        return NULL;
-      }
-      if (!data_charge_session(uncompressed, owner, uncompressed->size)) {
-        data_destroy(uncompressed);
-        file_destroy(file);
-        send_status(fd, STATUS_ERROR);
-        *failed = true;
-        return NULL;
-      }
-      if (uncompressed->size > MAX_FILE_DATA_SIZE) {
-        data_destroy(uncompressed);
-        file_destroy(file);
-        send_status(fd, STATUS_ERROR);
-        *failed = true;
-        return NULL;
-      }
-      file_data = uncompressed;
-    }
-
-    data_destroy(file->data);
-    file->data = file_data;
   return file;
+
+fail:
+  file_destroy(file);
+  *failed = true;
+  return NULL;
 }
 
+/* Delta handshake dispatcher: sign the basis, ship the signature, then hand the
+   response off to the matching branch helper.  Takes ownership of `old_data`
+   (and, once created, `sig`); sets `*failed` on every error path. */
+static File* receive_delta_file(int fd, const Config* config, const char* check_path,
+                                void* old_data, unsigned long long old_size,
+                                unsigned long long expected_size, int basis_fd, bool* failed) {
+  /* The basis is either an in-memory snapshot (the destination file, bounded) or
+   * a confined descriptor (a --fuzzy sibling, possibly larger than memory) that
+   * is signed/applied in bounded chunks. */
+  Data* sig_data = NULL;
+  Status resp = STATUS_ERROR;
+  bool sig_sent = false;
+  /* A delta check needs at least one basis source: the in-memory destination
+   * snapshot or a confined basis descriptor. */
+  if (!old_data && basis_fd < 0) {
+    *failed = true;
+    return NULL;
+  }
+  DeltaSignature* sig =
+      old_data ? delta_signature_create_seeded(old_data, old_size, config->delta_block_size,
+                                               (uint32_t)config->checksum_seed)
+               : delta_signature_create_fd_seeded(basis_fd, old_size, config->delta_block_size,
+                                                  (uint32_t)config->checksum_seed);
+  if (!sig)
+    goto fail;
+
+  sig_data = delta_signature_serialize(sig);
+  if (!sig_data)
+    goto fail;
+
+  sig_sent = send_status(fd, STATUS_DELTA_SIGNATURE) && send_data(fd, sig_data);
+  data_destroy(sig_data);
+  sig_data = NULL;
+
+  if (!sig_sent || !receive_status(fd, &resp))
+    goto fail;
+
+  if (resp == STATUS_DELTA_DATA)
+    return receive_delta_data_branch(fd, config, check_path, old_data, old_size, basis_fd, sig,
+                                     failed);
+
+  if (resp == STATUS_NEXT)
+    return receive_next_branch(fd, config, check_path, expected_size, old_data, sig, failed);
+
+  send_status(fd, STATUS_ERROR);
+fail:
+  data_destroy(sig_data);
   delta_signature_destroy(sig);
   free(old_data);
-  send_status(fd, STATUS_ERROR);
   *failed = true;
   return NULL;
 }
@@ -640,28 +640,29 @@ static bool fuzzy_candidate_better(const FuzzyCandidate* cand, const FuzzyCandid
   return strcmp(cand->name, best->name) < 0;
 }
 
-/* Search the destination directory that will contain `check_path` for a
+/* Search the destination directory that will contain `check_path` for a similar
- * similar regular file usable as a --fuzzy delta basis and return its full
+ * regular file usable as a --fuzzy delta basis and return an open, confined
- * content in a malloc'd (protocol_alloc) buffer.  Returns NULL (with *out_size
+ * read descriptor to it (with *out_size set).  Returns -1 (with *out_size 0)
- * = 0) when no candidate qualifies, which means the caller performs the normal
+ * when no candidate qualifies, which means the caller performs the normal
- * whole-file transfer. */
+ * whole-file transfer.  The basis is signed/applied by streaming its descriptor,
-static void* fuzzy_basis_find_and_load(const Config* config, const char* check_path,
+ * so no whole-basis buffer is ever needed and its size is not capped. */
+static int fuzzy_basis_find_and_open(const Config* config, const char* check_path,
                                      unsigned long long check_size, time_t check_mtime,
                                      long check_mtime_nsec, unsigned long long* out_size) {
   *out_size = 0;
   if (!config || !config->receive_root_directory || !config->fuzzy || !config->use_delta ||
-      !check_path || check_size > MAX_RECEIVE_WHOLE_FILE_SIZE)
+      !check_path)
-    return NULL;
+    return -1;
 
   char* full_path = path_cat(config->receive_root_directory, check_path);
   if (!full_path)
-    return NULL;
+    return -1;
   char* leaf = NULL;
   int dir_fd = file_open_secure_parent(full_path, &leaf, false);
   if (dir_fd < 0 || !leaf) {
     free(leaf);
     free(full_path);
-    return NULL;
+    return -1;
   }
   size_t target_len = strlen(leaf);
   /* A target basename longer than FUZZY_NAME_LIMIT can never pass the name gate
@@ -670,7 +671,7 @@ static void* fuzzy_basis_find_and_load(const Config* config, const char* check_p
     close(dir_fd);
     free(leaf);
     free(full_path);
-    return NULL;
+    return -1;
   }
 
   int scanfd = dup(dir_fd);
@@ -678,7 +679,7 @@ static void* fuzzy_basis_find_and_load(const Config* config, const char* check_p
     close(dir_fd);
     free(leaf);
     free(full_path);
-    return NULL;
+    return -1;
   }
   DIR* dir = fdopendir(scanfd);
   if (!dir) {
@@ -686,7 +687,7 @@ static void* fuzzy_basis_find_and_load(const Config* config, const char* check_p
     close(dir_fd);
     free(leaf);
     free(full_path);
-    return NULL;
+    return -1;
   }
 
   /* The weighted-distance scratch row is allocated once per scan (not once per
@@ -697,7 +698,7 @@ static void* fuzzy_basis_find_and_load(const Config* config, const char* check_p
     close(dir_fd);
     free(leaf);
     free(full_path);
-    return NULL;
+    return -1;
   }
   int fname_suf_len = 0;
   const char* fname_suf = fuzzy_find_suffix(leaf, (int)target_len, &fname_suf_len);
@@ -727,7 +728,7 @@ static void* fuzzy_basis_find_and_load(const Config* config, const char* check_p
     if (fstatat(dir_fd, name, &st, AT_SYMLINK_NOFOLLOW) != 0 || !S_ISREG(st.st_mode))
       continue;
     unsigned long long cand_size = (unsigned long long)st.st_size;
-    if (cand_size == 0 || cand_size > MAX_RECEIVE_WHOLE_FILE_SIZE)
+    if (cand_size == 0)
       continue;
     long cand_nsec = 0;
 #ifdef __linux__
@@ -771,7 +772,7 @@ static void* fuzzy_basis_find_and_load(const Config* config, const char* check_p
   if (exact.name[0])
     best = exact;
 
-  void* basis = NULL;
+  int basis_fd = -1;
   if (best.name[0]) {
     /* O_NONBLOCK: a name raced to a FIFO between the fstatat gate and this open
        would otherwise block the receive thread forever on open(2); with it the
@@ -781,29 +782,55 @@ static void* fuzzy_basis_find_and_load(const Config* config, const char* check_p
     if (fd >= 0) {
       struct stat st;
       if (fstat(fd, &st) == 0 && S_ISREG(st.st_mode) &&
-          (unsigned long long)st.st_size == best.size && best.size <= SIZE_MAX) {
+          (unsigned long long)st.st_size == best.size) {
-        basis = protocol_alloc((size_t)best.size);
+        basis_fd = fd;
-        if (basis) {
+      } else {
-          size_t got = 0;
-          while (got < (size_t)best.size) {
-            ssize_t n = read(fd, (char*)basis + got, (size_t)best.size - got);
-            if (n <= 0) {
-              free(basis);
-              basis = NULL;
-              break;
-            }
-            got += (size_t)n;
-          }
-        }
-      }
         close(fd);
       }
     }
+  }
   close(dir_fd);
   free(full_path);
-  if (basis)
+  if (basis_fd >= 0)
     *out_size = best.size;
-  return basis;
+  return basis_fd;
+}
+
+/* Receive one whole-file data frame into `file`.  A payload at or below the
+ * receiver's streaming bound keeps the historical charged whole-buffer path; a
+ * larger one is streamed into a spool temp file (decompressing incrementally)
+ * and installed through the File's basis_copy field.  `expected_size` is the
+ * logical size from the check frame (0 when unknown, e.g. the non-incremental
+ * path). */
+static bool receive_file_payload_into(File* file, int fd, const Config* config,
+                                      const char* dest_path, unsigned long long expected_size) {
+  bool compress =
+      config->use_compression && !compression_should_skip_with_suffixes(
+                                     file->path, config->skip_compress_suffixes,
+                                     config->skip_compress_set ? config->skip_compress_count : -1);
+  Data* buffer = NULL;
+  char* spool = NULL;
+  unsigned long long size = 0;
+  if (!file_receive_payload(fd, compress, expected_size, dest_path,
+                            protocol_whole_file_receive_limit(), &buffer, &spool, &size)) {
+    return false;
+  }
+  if (spool) {
+    Data* reserved = data_create_reserve((size_t)size);
+    if (!reserved) {
+      unlink(spool);
+      free(spool);
+      return false;
+    }
+    data_destroy(file->data);
+    file->data = reserved;
+    file->basis_copy = spool;
+    file->data_spool = true;
+  } else {
+    data_destroy(file->data);
+    file->data = buffer;
+  }
+  return true;
 }
 
 /* Read the remainder of a full-file transfer after the receiver has already
@@ -811,7 +838,8 @@ static void* fuzzy_basis_find_and_load(const Config* config, const char* check_p
  * data frame, and return an owned File.  Shared by the plain full-transfer path
  * and the --append-verify prefix-mismatch fallback (a clean full transfer
  * instead of a corrupt prefix+tail blend). */
-static File* receive_full_file(int fd, const Config* config, const char* path) {
+static File* receive_full_file(int fd, const Config* config, const char* path,
+                               unsigned long long expected_size) {
   File* file = file_create(path);
   if (!file)
     return NULL;
@@ -827,36 +855,17 @@ static File* receive_full_file(int fd, const Config* config, const char* path) {
     file_destroy(file);
     return NULL;
   }
-  Data* file_data = receive_data_limited(fd, MAX_RECEIVE_WHOLE_FILE_SIZE);
+  char* dest_path = path_cat(config->receive_root_directory, path);
-  if (file_data == NULL) {
+  if (!dest_path) {
     file_destroy(file);
     return NULL;
   }
-  if (config->use_compression &&
+  bool ok = receive_file_payload_into(file, fd, config, dest_path, expected_size);
-      !compression_should_skip_with_suffixes(file->path, config->skip_compress_suffixes,
+  free(dest_path);
-                                             config->skip_compress_set ? config->skip_compress_count
+  if (!ok) {
-                                                                       : -1)) {
-    Data* uncompressed = data_decompress_limited(file_data, MAX_RECEIVE_WHOLE_FILE_SIZE);
-    ProtocolSession* owner = file_data->owner;
-    data_destroy(file_data);
-    if (uncompressed == NULL) {
     file_destroy(file);
     return NULL;
   }
-    if (!data_charge_session(uncompressed, owner, uncompressed->size)) {
-      data_destroy(uncompressed);
-      file_destroy(file);
-      return NULL;
-    }
-    if (uncompressed->size > MAX_FILE_DATA_SIZE) {
-      data_destroy(uncompressed);
-      file_destroy(file);
-      return NULL;
-    }
-    file_data = uncompressed;
-  }
-  data_destroy(file->data);
-  file->data = file_data;
   return file;
 }
 
@@ -974,13 +983,12 @@ static IncrementalCheckOutcome incremental_check_receive_request(IncrementalChec
       return INCREMENTAL_ERROR;
   }
 
-  /* A basis-configured run may materialize a file larger than the whole-file
+  /* No file-size refusal: a whole-file payload larger than the historical
-     payload bound: a basis hit is streamed from the basis path (bounded
+     whole-file bound is streamed through a bounded buffer (see
-     buffers), so the check size is not itself an allocation.  Every other
+     file_receive_payload).  Only a size that cannot be represented on this
-     path (delta/append/full) still applies MAX_RECEIVE_WHOLE_FILE_SIZE, and a
+     platform is rejected. */
-     miss simply falls through to the normal transfer with its own bound. */
+  if (state->check_size > SIZE_MAX) {
-  if (!config_has_basis(config) && state->check_size > MAX_RECEIVE_WHOLE_FILE_SIZE) {
+    send_error_detail(fd, "check size is not representable");
-    send_error_detail(fd, "check size exceeds receiver limit");
     return INCREMENTAL_ERROR;
   }
 
@@ -1411,7 +1419,7 @@ static IncrementalCheckOutcome incremental_check_try_append_resume(IncrementalCh
       close(state->old_fd);
       state->old_fd = -1;
     }
-    *out_file = receive_full_file(fd, config, check_path);
+    *out_file = receive_full_file(fd, config, check_path, check_size);
     return INCREMENTAL_FILE;
   }
 
@@ -1428,20 +1436,24 @@ static IncrementalCheckOutcome incremental_check_try_append_resume(IncrementalCh
   if (config->use_metadata) {
     int meta_ok = 1;
     meta = metadata_receive(fd, &meta_ok);
-    if (!meta_ok)
+    if (!meta_ok) {
+      file_metadata_destroy(meta);
       return INCREMENTAL_ERROR;
     }
+  }
   if (config->use_xattrs) {
     int xok = 0;
     append_xattrs = xattr_receive(fd, &xok, config->preserve_acls);
     if (!xok) {
       xattr_list_free(append_xattrs);
+      file_metadata_destroy(meta);
       return INCREMENTAL_ERROR;
     }
   }
   Data* tail = receive_data_limited(fd, MAX_RECEIVE_WHOLE_FILE_SIZE);
   if (tail == NULL) {
     xattr_list_free(append_xattrs);
+    file_metadata_destroy(meta);
     return INCREMENTAL_ERROR;
   }
   if (config->use_compression &&
@@ -1453,16 +1465,19 @@ static IncrementalCheckOutcome incremental_check_try_append_resume(IncrementalCh
     data_destroy(tail);
     if (uncompressed == NULL) {
       xattr_list_free(append_xattrs);
+      file_metadata_destroy(meta);
       return INCREMENTAL_ERROR;
     }
     if (!data_charge_session(uncompressed, owner, uncompressed->size)) {
       data_destroy(uncompressed);
       xattr_list_free(append_xattrs);
+      file_metadata_destroy(meta);
       return INCREMENTAL_ERROR;
     }
     if (uncompressed->size > MAX_FILE_DATA_SIZE) {
       data_destroy(uncompressed);
       xattr_list_free(append_xattrs);
+      file_metadata_destroy(meta);
       return INCREMENTAL_ERROR;
     }
     tail = uncompressed;
@@ -1475,6 +1490,7 @@ static IncrementalCheckOutcome incremental_check_try_append_resume(IncrementalCh
     send_status(fd, STATUS_ERROR);
     data_destroy(tail);
     xattr_list_free(append_xattrs);
+    file_metadata_destroy(meta);
     return INCREMENTAL_ERROR;
   }
   size_t full_size = (size_t)check_size;
@@ -1482,6 +1498,7 @@ static IncrementalCheckOutcome incremental_check_try_append_resume(IncrementalCh
   if (!full) {
     data_destroy(tail);
     xattr_list_free(append_xattrs);
+    file_metadata_destroy(meta);
     return INCREMENTAL_ERROR;
   }
   if (old_size > 0 && state->old_data)
@@ -1496,6 +1513,7 @@ static IncrementalCheckOutcome incremental_check_try_append_resume(IncrementalCh
   if (!file) {
     free(full);
     xattr_list_free(append_xattrs);
+    file_metadata_destroy(meta);
     return INCREMENTAL_ERROR;
   }
   file->metadata = meta;
@@ -1516,8 +1534,9 @@ static IncrementalCheckOutcome incremental_check_try_delta(IncrementalCheckState
                                                            bool try_delta, File** out_file) {
   if (try_delta && state->old_data != NULL) {
     bool delta_failed = false;
-    File* delta_file = receive_delta_file(state->fd, state->config, state->check_path,
+    File* delta_file =
-                                          state->old_data, state->old_size, &delta_failed);
+        receive_delta_file(state->fd, state->config, state->check_path, state->old_data,
+                           state->old_size, state->check_size, -1, &delta_failed);
     state->old_data = NULL; /* receive_delta_file consumes the snapshot on every path */
     if (delta_file) {
       *out_file = delta_file;
@@ -1540,14 +1559,14 @@ static IncrementalCheckOutcome incremental_check_try_fuzzy(IncrementalCheckState
   if (!config->fuzzy || !config->use_delta)
     return INCREMENTAL_CONTINUE;
   unsigned long long fuzzy_size = 0;
-  void* fuzzy_basis = fuzzy_basis_find_and_load(config, state->check_path, state->check_size,
+  int fuzzy_fd = fuzzy_basis_find_and_open(config, state->check_path, state->check_size,
                                            (time_t)state->check_mtime,
                                            (long)state->check_mtime_nsec, &fuzzy_size);
-  if (fuzzy_basis != NULL) {
+  if (fuzzy_fd >= 0) {
     bool fuzzy_failed = false;
-    File* fuzzy_file = receive_delta_file(state->fd, config, state->check_path, fuzzy_basis,
+    File* fuzzy_file = receive_delta_file(state->fd, config, state->check_path, NULL, fuzzy_size,
-                                          fuzzy_size, &fuzzy_failed);
+                                          state->check_size, fuzzy_fd, &fuzzy_failed);
-    fuzzy_basis = NULL; /* receive_delta_file consumes the buffer on every path */
+    close(fuzzy_fd);
     if (fuzzy_file) {
       *out_file = fuzzy_file;
       return INCREMENTAL_FILE;
@@ -1555,7 +1574,6 @@ static IncrementalCheckOutcome incremental_check_try_fuzzy(IncrementalCheckState
     if (fuzzy_failed)
       return INCREMENTAL_ERROR;
   }
-  free(fuzzy_basis);
   return INCREMENTAL_CONTINUE;
 }
 
@@ -1567,7 +1585,7 @@ static File* incremental_check_receive_full(IncrementalCheckState* state) {
     close(state->old_fd);
     state->old_fd = -1;
   }
-  return receive_full_file(state->fd, state->config, state->check_path);
+  return receive_full_file(state->fd, state->config, state->check_path, state->check_size);
 }
 
 /* Core implementation.  `would_transfer` (may be NULL) is set true only on the

src/shared/protocol.c

@@ -38,6 +38,27 @@ static atomic_ullong io_bytes_read = 0;
 
 static unsigned long long global_bwlimit(void);
 
+/* Runtime whole-file receive bound (see protocol.h).  Resolved once; an
+ * override can only LOWER the ceiling, never raise it above the protocol
+ * constant, so the wire/security bound is unchanged.  A parse failure or a
+ * non-positive value leaves the default in place. */
+unsigned long long protocol_whole_file_receive_limit(void) {
+  static atomic_ullong cached = 0;
+  unsigned long long value = atomic_load_explicit(&cached, memory_order_relaxed);
+  if (value != 0)
+    return value;
+  value = MAX_RECEIVE_WHOLE_FILE_SIZE;
+  const char* env = getenv("FASTSYNC_MAX_WHOLE_FILE_SIZE");
+  if (env && env[0] != '\0') {
+    char* end = NULL;
+    unsigned long long parsed = strtoull(env, &end, 10);
+    if (end && *end == '\0' && parsed > 0 && parsed < value)
+      value = parsed;
+  }
+  atomic_store_explicit(&cached, value, memory_order_relaxed);
+  return value;
+}
+
 /* ------------------------------------------------------------------------- *
  * Transport vtable implementations.
  *
@@ -769,17 +790,11 @@ bool protocol_send_data(ProtocolSession* session, const Data* data) {
   return true;
 }
 
-Data* protocol_receive_data_limited(ProtocolSession* session, unsigned long long maximum_size) {
+Data* protocol_receive_data_alloc(ProtocolSession* session, unsigned long long size) {
   if (!session)
     return NULL;
-  unsigned long long size = 0;
+  if (size > MAX_DATA_PAYLOAD_SIZE)
-  if (!protocol_receive_n_data(session, &size, sizeof(unsigned long long)))
     return NULL;
-  if (size > MAX_DATA_PAYLOAD_SIZE || size > maximum_size) {
-    log_message(LOG_LEVEL_ERROR, "Data size %llu exceeds maximum %llu", size,
-                (unsigned long long)MAX_DATA_PAYLOAD_SIZE);
-    return NULL;
-  }
   if (size > SIZE_MAX)
     return NULL;
   size_t allocation_size = size == 0 ? 1 : (size_t)size;
@@ -794,12 +809,6 @@ Data* protocol_receive_data_limited(ProtocolSession* session, unsigned long long
     protocol_release_memory_for_session(session, allocation_size);
     return NULL;
   }
-  if (!protocol_receive_n_data(session, data, (size_t)size)) {
-    free(data);
-    protocol_release_memory_for_session(session, allocation_size);
-    return NULL;
-  }
-  log_debug_message(LOG_DEBUG_PROTO, "Received %llu data", size);
   Data* result = data_create(data, (size_t)size);
   if (!result) {
     protocol_release_memory_for_session(session, allocation_size);
@@ -810,6 +819,32 @@ Data* protocol_receive_data_limited(ProtocolSession* session, unsigned long long
   return result;
 }
 
+Data* protocol_receive_data_body(ProtocolSession* session, unsigned long long size) {
+  Data* result = protocol_receive_data_alloc(session, size);
+  if (!result)
+    return NULL;
+  if (!protocol_receive_n_data(session, result->data, (size_t)size)) {
+    data_destroy(result);
+    return NULL;
+  }
+  log_debug_message(LOG_DEBUG_PROTO, "Received %llu data", size);
+  return result;
+}
+
+Data* protocol_receive_data_limited(ProtocolSession* session, unsigned long long maximum_size) {
+  if (!session)
+    return NULL;
+  unsigned long long size = 0;
+  if (!protocol_receive_n_data(session, &size, sizeof(unsigned long long)))
+    return NULL;
+  if (size > MAX_DATA_PAYLOAD_SIZE || size > maximum_size) {
+    log_message(LOG_LEVEL_ERROR, "Data size %llu exceeds maximum %llu", size,
+                (unsigned long long)MAX_DATA_PAYLOAD_SIZE);
+    return NULL;
+  }
+  return protocol_receive_data_body(session, size);
+}
+
 bool protocol_send_int(ProtocolSession* session, int data) {
   if (!protocol_send_n_data(session, &data, sizeof(int)))
     return false;
@@ -1114,6 +1149,12 @@ Data* receive_data(int fd) {
 Data* receive_data_limited(int fd, unsigned long long maximum_size) {
   return protocol_receive_data_limited(legacy_session(fd, -1), maximum_size);
 }
+Data* receive_data_body(int fd, unsigned long long size) {
+  return protocol_receive_data_body(legacy_session(fd, -1), size);
+}
+Data* receive_data_alloc(int fd, unsigned long long size) {
+  return protocol_receive_data_alloc(legacy_session(fd, -1), size);
+}
 bool send_int(int fd, int data) {
   return protocol_send_int(legacy_session(-1, fd), data);
 }

src/shared/protocol.h

@@ -32,6 +32,16 @@
 /* Maximum allowed data payload size for receive_data (whole-file bound) */
 #define MAX_DATA_PAYLOAD_SIZE MAX_RECEIVE_WHOLE_FILE_SIZE
 
+/* Runtime whole-file receive bound.  It defaults to MAX_RECEIVE_WHOLE_FILE_SIZE
+ * and exists so the test suite can lower the ceiling (via the
+ * FASTSYNC_MAX_WHOLE_FILE_SIZE environment variable, a byte count) and exercise
+ * the streaming path with a small, fast transfer.  A payload at or below the
+ * bound keeps the historical whole-buffer path; a larger one is streamed
+ * through a bounded buffer.  The value is resolved once per process and never
+ * exceeds the compile-time ceiling, so a malicious environment cannot raise it
+ * beyond the protocol limit. */
+unsigned long long protocol_whole_file_receive_limit(void);
+
 /* Maximum chunk size (64 MB) — prevents unbounded allocation from the wire */
 #define MAX_CHUNK_SIZE (64ULL * 1024 * 1024)
 /* Files larger than this are not kept fully in memory while loading: the
@@ -370,6 +380,15 @@ char* receive_str_redacted(int file_descriptor);
 bool send_data(int file_descriptor, const Data* data);
 Data* receive_data(int file_descriptor);
 Data* receive_data_limited(int file_descriptor, unsigned long long maximum_size);
+/* Read exactly `size` bytes as a charged Data body.  The length-prefixed
+ * receive_data_limited() reads the header itself; this variant is for callers
+ * that must inspect the declared size (and possibly stream the body instead)
+ * before allocating.  `size` must already be within MAX_DATA_PAYLOAD_SIZE. */
+Data* receive_data_body(int file_descriptor, unsigned long long size);
+/* Allocate (and charge) a `size`-byte Data body without reading it; the caller
+ * fills `result->data` itself.  Used when a frame's leading bytes must be
+ * inspected before the rest of the body is read. */
+Data* receive_data_alloc(int file_descriptor, unsigned long long size);
 bool send_int(int file_descriptor, int data);
 bool receive_int(int file_descriptor, int* data);
 bool send_status(int file_descriptor, Status status);

src/shared/transport_tcp.c

@@ -134,7 +134,7 @@ Server* server_create_ex(int port, const ServerBindOptions* bind_opts) {
 
   server->file_descriptor = file_descriptor;
   server->ssl_ctx = NULL;
-  server->max_connections = 100;
+  server->max_connections = SERVER_DEFAULT_MAX_CONNECTIONS;
   server->active_connections = 0;
   server->limit_registry = NULL;
 

src/shared/transport_tcp.h

@@ -11,6 +11,10 @@
  * stored here so the transport layer does not depend on daemon config. */
 struct DaemonLimitRegistry;
 
+/* Connection cap applied by server_create_ex() until the daemon's configured
+ * `max connections` overrides it via server_set_max_connections(). */
+#define SERVER_DEFAULT_MAX_CONNECTIONS 100
+
 typedef struct Server {
   struct sockaddr_storage address;
   unsigned int address_length;

src/shared/xattr.c

@@ -450,7 +450,7 @@ void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, uint
   if (len <= 0 || (size_t)len >= sizeof(record))
     return;
   if (fsetxattr(fd, FAKESUPER_XATTR, record, (size_t)len, 0) != 0) {
-    log_message(LOG_LEVEL_WARNING, "--fake-super: could not store %s on destination file: %s",
+    log_message(LOG_LEVEL_WARNING, "--fake-super: could not store %s on destination entry: %s",
                 FAKESUPER_XATTR, strerror(errno));
   }
 }
@@ -550,7 +550,7 @@ bool fake_super_restore_fd(int fd, FileAttrPolicy policy) {
     } else if (metadata_mode_for_policy((mode_t)(ul_mode & 0777U), cur.st_mode, policy, &want)) {
       if (fchmod(fd, want) != 0)
         log_message(LOG_LEVEL_WARNING,
-                    "--fake-super: could not restore mode on destination file: %s",
+                    "--fake-super: could not restore mode on destination entry: %s",
                     strerror(errno));
     }
   }

src/shared/xattr.h

@@ -140,21 +140,25 @@ bool xattr_apply_path_nofollow(int parent_fd, const char* leaf, const FileXattrL
 /* --fake-super: write the source uid/gid/mode/rdev record into the reserved
  * FAKESUPER_XATTR on `fd`, using rsync 3.4.1's exact grammar (see the key
  * comment above).  `mode` is the full st_mode including its S_IFMT bits.
- * Best-effort (logged, never fatal).  Only meaningful when metadata was
+ * `fd` may be a regular file, a faked char/block device (written as a regular
- * transmitted so the values exist. */
+ * file), or a DIRECTORY: rsync stores a directory's faked mode/uid/gid in the
+ * reserved xattr on the directory itself.  Best-effort (logged, never fatal).
+ * Only meaningful when metadata was transmitted so the values exist. */
 void fake_super_store_fd(int fd, uint32_t uid, uint32_t gid, uint32_t mode, uint32_t rdev_major,
                          uint32_t rdev_minor);
 
 /* --fake-super replay: parse the FAKESUPER_XATTR record previously written on
  * `fd` by fake_super_store_fd and re-apply the recorded permission bits
- * fd-relative.  The recorded uid/gid are deliberately NOT chowned for real:
+ * fd-relative.  `fd` may be a regular file, a faked device, or a DIRECTORY;
- * --fake-super only RECORDS ownership (the caller stores the resolved mapping
+ * fgetxattr/fchmod work identically on a directory descriptor.  The recorded
- * via identity_resolve_storage_ids), it never performs a real chown.  The
+ * uid/gid are deliberately NOT chowned for real: --fake-super only RECORDS
+ * ownership (the caller stores the resolved mapping via
+ * identity_resolve_storage_ids), it never performs a real chown.  The
  * recorded rdev is retained for a later privileged restore but is not acted on
  * here.  Best-effort: absence of the xattr or a malformed record is a silent
  * no-op that never fails the transfer.  The MODE leg is applied only when
  * policy.perms||policy.executability, and the recorded special bits
- * (setuid/setgid/sticky) are NOT applied to the real file -- exactly like
+ * (setuid/setgid/sticky) are NOT applied to the real entry -- exactly like
  * rsync's fake-super receiver, which stores the full mode in the xattr but
  * strips the special bits on disk.  mtime is not part of the record; the normal
  * metadata path carries it (policy.times) exactly as rsync sets the file's own

test_probe/dst/workspace/test_probe/src/a.txt

@@ -0,0 +1 @@
+hello

test_probe/dst/workspace/test_probe/src/extra.txt

@@ -0,0 +1 @@
+x

test_probe/dst/workspace/test_probe/src/extradir/in.txt

@@ -0,0 +1 @@
+y

test_probe/dst/workspace/test_probe/src/extralink

@@ -0,0 +1 @@
+a.txt

test_probe/dst/workspace/test_probe/src/link

@@ -0,0 +1 @@
+b.txt

test_probe/dst/workspace/test_probe/src/sub/b.txt

@@ -0,0 +1 @@
+world

test_probe/dst/workspace/test_probe/src/sub/deep/d.txt

@@ -0,0 +1 @@
+deep

test_probe/dst/workspace/test_probe/src/sub/rel

@@ -0,0 +1 @@
+../a.txt

test_probe/src/a.txt

@@ -0,0 +1 @@
+hello

test_probe/src/link

@@ -0,0 +1 @@
+b.txt

test_probe/src/sub/b.txt

@@ -0,0 +1 @@
+world

test_probe/src/sub/deep/d.txt

@@ -0,0 +1 @@
+deep

test_probe/src/sub/rel

@@ -0,0 +1 @@
+../a.txt

test_probe2/dst/workspace/test_probe2/src/a.txt

@@ -0,0 +1 @@
+hello

test_probe2/dst/workspace/test_probe2/src/sub/b.txt

@@ -0,0 +1 @@
+world

test_probe2/src/a.txt

@@ -0,0 +1 @@
+hello

test_probe2/src/sub/b.txt

@@ -0,0 +1 @@
+world

test_probe3/dst/workspace/test_probe3/src/a.txt

@@ -0,0 +1 @@
+hello

test_probe3/dst/workspace/test_probe3/src/sub/b.txt

@@ -0,0 +1 @@
+world

test_probe3/src/a.txt

@@ -0,0 +1 @@
+hello

test_probe3/src/sub/b.txt

@@ -0,0 +1 @@
+world

tests/integration/common.py

@@ -34,7 +34,7 @@ class ServerManager:
         self._proc = None
         self._port = None
 
-    def start(self, extra_args=None):
+    def start(self, extra_args=None, env=None):
         self.stop()
         self._port = _find_free_port()
         # Plain TCP is intentionally explicit in the server; integration tests
@@ -42,7 +42,11 @@ class ServerManager:
         cmd = SERVER_CMD + ["-p", str(self._port), "--allow-unauthenticated"]
         if extra_args:
             cmd += extra_args
-        self._proc = subprocess.Popen(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
+        proc_env = dict(os.environ)
+        if env:
+            proc_env.update(env)
+        self._proc = subprocess.Popen(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
+                                      env=proc_env)
         _wait_for_port(self._port, timeout=5)
 
     def stop(self):

tests/integration/test_differential_parity.py

@@ -106,6 +106,15 @@ def seed_backup(_src, rroot, froot):
         _mk(os.path.join(root, "a.txt"), b"OLD-CONTENT\n", _OLD_MTIME)
 
 
+def seed_delay_updates(_src, rroot, froot):
+    """A changed file plus an extra, so --delay-updates (and its implied
+    --delete-after) has both a publication and a deletion to order."""
+    for root in (rroot, froot):
+        _mk(os.path.join(root, "a.txt"), b"OLD-CONTENT\n", _OLD_MTIME)
+        _mk(os.path.join(root, "extra.txt"), b"extra\n", _OLD_MTIME)
+        _mk(os.path.join(root, "extradir", "z.txt"), b"z\n", _OLD_MTIME)
+
+
 def seed_size_only(_src, rroot, froot):
     for root in (rroot, froot):
         _mk(os.path.join(root, "a.txt"), b"XXXXXXXXXXX\n", _OLD_MTIME)
@@ -305,6 +314,14 @@ _CASES = [
     H.Case("delete_commit", "basic", ["-a", "--delete-after"], seed=seed_extras,
            fastsync_flags=["-a", "--delete-commit"], server_args=DELETE,
            ref="FastSync-only --delete-commit == rsync --delete-after"),
+    # #317: --delay-updates stages under a per-run unique name and publishes
+    # every update before the implied --delete-after removes extras.
+    H.Case("delay_updates", "basic", ["-a", "--delay-updates"],
+           seed=seed_delay_updates, ref="--delay-updates stages then publishes"),
+    H.Case("delay_updates_delete", "basic",
+           ["-a", "--delay-updates", "--delete"], seed=seed_delay_updates,
+           server_args=DELETE, ci=True,
+           ref="--delay-updates implies --delete-after (publish before delete)"),
     H.Case("delete_excluded", "filters",
            ["-a", "--delete", "--delete-excluded", "--exclude=*.log"],
            seed=seed_delete_excluded, server_args=DELETE, ref="--delete-excluded"),

tests/integration/test_features.py

@@ -3089,12 +3089,12 @@ class TestDelayUpdates:
             "staging directory left behind after a successful delayed transfer"
 
     @pytest.mark.skipif(shutil.which("rsync") is None, reason="rsync not installed")
-    def test_delay_updates_staging_name_collision_residual(self):
+    def test_delay_updates_staging_name_collision_preserved(self):
-        """Documented residual (RSYNC_COMPAT.md `--delay-updates` row): FastSync
+        """rsync parity (RSYNC_COMPAT.md `--delay-updates` row): the receiver
-        uses a fixed `.fastsync-stage` staging name and wipes a pre-existing tree
+        stages under a per-run unique name, so a genuine pre-existing
-        of that name at the start of a delayed run (crash-leftover cleanup),
+        destination entry named like the reserved staging prefix (`.fastsync-
-        even without `--delete`; rsync leaves a genuine destination entry of that
+        stage`) is never wiped -- even without `--delete`.  rsync likewise
-        name untouched.  Pins the divergence that keeps the row Divergent."""
+        leaves a real destination entry of its own temp name untouched."""
         source = self._make_source("delay_collide_src")
         rdst = os.path.join(TEST_DATA_DIR, "delay_collide_rdst")
         fdst = os.path.join(TEST_DATA_DIR, "delay_collide_fdst")
@@ -3120,8 +3120,11 @@ class TestDelayUpdates:
             result, _ = run_client(source, fdst, flags=["--delay-updates"],
                                    port=server.port)
         assert result.returncode == 0, (result.stderr or result.stdout)[:300]
-        assert not os.path.exists(os.path.join(fdst, self.STAGING)), \
+        assert _read_file(os.path.join(fdst, self.STAGING, "keepme.txt")) == b"genuine user data\n", \
-            "FastSync did not wipe the reserved staging name (residual changed)"
+            "FastSync destroyed a genuine destination entry named like the staging prefix"
+        # The per-run staging directory itself is removed after a clean run.
+        leftovers = [n for n in os.listdir(fdst) if n.startswith(self.STAGING + ".")]
+        assert leftovers == [], f"per-run staging directories left behind: {leftovers}"
 
     @pytest.mark.parametrize("mt", [False, True])
     def test_delay_updates_incremental_rerun_no_leftovers(self, shared_server, mt):
@@ -3161,10 +3164,11 @@ class TestDelayUpdates:
 
     @pytest.mark.parametrize("mt", [False, True])
     def test_delete_with_delay_updates(self, mt):
-        """--delete runs before publication, so the delete walker must not treat
+        """rsync parity: --delay-updates implies --delete-after, so every staged
-        the staging directory as a set of extras: a changed file must still be
+        update is published first and the genuine extras are removed only after
-        published after genuine extras are removed.  Uses its own server started
+        that (the delete walker must never treat the staging directory as a set
-        with --allow-delete (the shared session server refuses deletion)."""
+        of extras).  Uses its own server started with --allow-delete (the shared
+        session server refuses deletion)."""
         source = os.path.join(TEST_DATA_DIR, "delay_delete_src")
         dest = os.path.join(TEST_DATA_DIR, "delay_delete_dst")
         clean_dir(source)
@@ -3194,6 +3198,65 @@ class TestDelayUpdates:
             assert not os.path.exists(os.path.join(received, "extra.txt")), \
                 "genuine extra file was not deleted"
             assert not os.path.isdir(os.path.join(dest, self.STAGING))
+            assert [n for n in os.listdir(dest) if n.startswith(self.STAGING + ".")] == []
+
+    @pytest.mark.parametrize("mt", [False, True])
+    def test_delay_updates_delete_keeps_backup(self, mt):
+        """The --backup/--delay-updates interplay: the old destination file is
+        moved aside at publication, and that backup survives the implied
+        --delete-after pass (rsync never treats a backup file as an extra)."""
+        source = os.path.join(TEST_DATA_DIR, "delay_bak_del_src")
+        dest = os.path.join(TEST_DATA_DIR, "delay_bak_del_dst")
+        clean_dir(source)
+        clean_dir(dest)
+        with open(os.path.join(source, "f.txt"), "wb") as fh:
+            fh.write(b"NEW")
+        received = get_dest_received_dir(dest, source)
+        os.makedirs(received, exist_ok=True)
+        with open(os.path.join(received, "f.txt"), "wb") as fh:
+            fh.write(b"OLD")
+        os.utime(os.path.join(received, "f.txt"), (1_500_000_000, 1_500_000_000))
+        # A pre-existing backup-looking extra must also be shielded.
+        with open(os.path.join(received, "stale.txt~"), "wb") as fh:
+            fh.write(b"stale backup")
+        with ServerManager() as server:
+            server.start(extra_args=["--allow-delete"])
+            flags = ["--delete", "--backup", "--delay-updates"] + (["--threads"] if mt else [])
+            result, _ = run_client(source, dest, flags=flags, port=server.port)
+        assert result.returncode == 0, (result.stderr or result.stdout)[:300]
+        assert _read_file(os.path.join(received, "f.txt")) == b"NEW"
+        assert _read_file(os.path.join(received, "f.txt~")) == b"OLD", \
+            "the publication backup was removed by the delete-after pass"
+        assert os.path.exists(os.path.join(received, "stale.txt~")), \
+            "a pre-existing backup-suffixed entry was deleted"
+
+    @pytest.mark.parametrize("mt", [False, True])
+    def test_delay_updates_failed_run_leaves_no_staged_files(self, shared_server, mt):
+        """A run that fails before publication installs nothing and removes the
+        per-run staging directory (no staged leftovers)."""
+        source = os.path.join(TEST_DATA_DIR, "delay_fail_src")
+        dest = os.path.join(TEST_DATA_DIR, "delay_fail_dst")
+        clean_dir(source)
+        clean_dir(dest)
+        with open(os.path.join(source, "top.txt"), "wb") as fh:
+            fh.write(b"top\n")
+        os.makedirs(os.path.join(source, "sub"))
+        with open(os.path.join(source, "sub", "deep.txt"), "wb") as fh:
+            fh.write(b"deep\n")
+        # Plant a regular file where the "sub" directory must be created so the
+        # nested publish fails (the top-level file still publishes first).
+        received = get_dest_received_dir(dest, source)
+        os.makedirs(received)
+        with open(os.path.join(received, "sub"), "wb") as fh:
+            fh.write(b"blocker")
+
+        flags = ["--delay-updates"] + (["--threads"] if mt else [])
+        result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
+        assert result.returncode != 0, "a blocked nested publish must fail the run"
+        assert not os.path.lexists(os.path.join(received, "sub", "deep.txt")), \
+            "a staged file appeared despite the failed run"
+        assert [n for n in os.listdir(dest) if n.startswith(self.STAGING + ".")] == [], \
+            "the per-run staging directory survived a failed run"
 
     def test_delay_updates_rejects_reserved_backup_dir(self):
         """--backup-dir equal to the internal staging name must be rejected so
@@ -6979,6 +7042,72 @@ class TestExtendedAttributes:
             f"is not interoperable: ours={rec!r} rsync={out_rec!r}"
         )
 
+    @pytest.mark.ci
+    def test_fake_super_directory_rsync_interop(self, shared_server):
+        """#319: --fake-super fakes DIRECTORIES too.  A recursive -a
+        --fake-super run must write rsync 3.4.1's `user.rsync.%stat` record on
+        the directory itself (full mode with S_IFDIR + special bits, rdev 0,0,
+        uid:gid), replay only the permission bits on disk, and real rsync must
+        read the tree and re-emit the identical record."""
+        rsync = shutil.which("rsync")
+        if rsync is None:
+            pytest.skip("rsync not installed")
+        source, dest = self._source_and_dest("fakesuper_dir_interop")
+        sub = os.path.join(source, "subdir")
+        os.makedirs(sub)
+        with open(os.path.join(sub, "f.txt"), "wb") as fh:
+            fh.write(b"dir interop\n")
+        if not _xattr_supported(sub):
+            pytest.skip("filesystem does not support user xattrs")
+        # A special bit (setgid) is exactly what a fake-super record exists to
+        # carry: rsync only re-emits a directory record when there is something
+        # it cannot represent on disk (a special bit, or a mode it would widen
+        # to keep the owner's rwx).  Skip cleanly when the filesystem drops it.
+        os.chmod(sub, 0o2751)
+        if stat.S_IMODE(os.stat(sub).st_mode) & 0o7000 == 0:
+            pytest.skip("filesystem drops directory special bits")
+        uid = os.stat(sub).st_uid
+
+        result, _ = run_client(source, dest, flags=["-a", "--fake-super"],
+                               port=shared_server.port)
+        assert result.returncode == 0, \
+            f"-a --fake-super dir sync failed: {(result.stderr or result.stdout)[:300]}"
+        received = get_dest_received_dir(dest, source)
+        dst_sub = os.path.join(received, "subdir")
+        assert os.path.isdir(dst_sub), "the directory entry was not transferred"
+
+        rec = os.getxattr(dst_sub, "user.rsync.%stat").decode()
+        fields = rec.split()
+        assert len(fields) == 3, f"unexpected rsync fake-super record {rec!r}"
+        mode_field, rdev_field, owner_field = fields
+        assert rdev_field == "0,0", f"directory rdev must be 0,0, got {rdev_field!r}"
+        assert int(mode_field, 8) & 0o170000 == stat.S_IFDIR, (
+            f"recorded mode {mode_field!r} must carry S_IFDIR"
+        )
+        assert int(mode_field, 8) & 0o7777 == 0o2751, (
+            f"recorded mode {mode_field!r} must carry the full source mode 02751"
+        )
+        assert owner_field.split(":")[0] == str(uid), \
+            f"recorded uid {owner_field!r} != source uid {uid}"
+        # Permission bits only on disk: the setgid bit stays in the record.
+        assert stat.S_IMODE(os.stat(dst_sub).st_mode) == 0o751, (
+            "the directory's special bits must not be installed on disk"
+        )
+
+        # Real rsync reads FastSync's directory record and re-emits it verbatim.
+        out = os.path.join(TEST_DATA_DIR, "fakesuper_dir_interop_rsync")
+        clean_dir(out)
+        rs = subprocess.run([rsync, "-aX", "--fake-super", received + "/", out + "/"],
+                            capture_output=True, text=True, timeout=120)
+        assert rs.returncode == 0, (
+            f"rsync could not read FastSync's fake-super directory tree: {rs.stderr[:300]}"
+        )
+        out_rec = os.getxattr(os.path.join(out, "subdir"), "user.rsync.%stat").decode()
+        assert out_rec == rec, (
+            "rsync re-emitted a different directory fake-super record; FastSync's "
+            f"grammar is not interoperable: ours={rec!r} rsync={out_rec!r}"
+        )
+
     @pytest.mark.ci
     def test_directory_xattrs_preserved(self, shared_server):
         """#286.3: -aX must preserve user.* xattrs on DIRECTORIES, not just files."""

tests/integration/test_large_stream.py

@@ -0,0 +1,222 @@
+"""#318: whole-file streaming above the receiver's 256 MiB ceiling.
+
+The receiver's historical whole-file bound (``MAX_RECEIVE_WHOLE_FILE_SIZE``,
+256 MiB) refused any single-file payload above it.  The transfer engine now
+streams such a payload (and the basis read/verify/hash) through a bounded buffer
+and spools it to a temp file, so arbitrarily large single files transfer without
+being materialized in memory.
+
+To exercise the streaming path deterministically and quickly, these tests lower
+the receiver bound with the test-only ``FASTSYNC_MAX_WHOLE_FILE_SIZE`` hook (it
+can only lower, never raise, the protocol ceiling) and transfer a file a few
+times larger than the lowered bound.  A real >256 MiB transfer is covered once,
+unmarked, so it runs in the full suite but not the fast PR gate.
+"""
+import hashlib
+import os
+import random
+import shutil
+import sys
+
+import pytest
+
+sys.path.insert(0, os.path.dirname(__file__))
+from common import (  # noqa: E402
+    ServerManager,
+    TEST_DATA_DIR,
+    clean_dir,
+    get_dest_received_dir,
+    run_client,
+)
+
+LOW_BOUND = 1024 * 1024
+FILE_SIZE = 3 * 1024 * 1024
+OLD_MTIME = 1_500_000_000
+
+
+@pytest.fixture(scope="module")
+def small_bound_server():
+    """A server whose whole-file streaming bound is 1 MiB."""
+    server = ServerManager()
+    server.start(extra_args=["--allow-super"],
+                 env={"FASTSYNC_MAX_WHOLE_FILE_SIZE": str(LOW_BOUND)})
+    yield server
+    server.stop()
+
+
+def _payload(n):
+    rng = random.Random(0xC0FFEE)
+    return rng.randbytes(n)
+
+
+def _write(path, data, mtime=None):
+    os.makedirs(os.path.dirname(path), exist_ok=True)
+    with open(path, "wb") as fh:
+        fh.write(data)
+    if mtime is not None:
+        os.utime(path, (mtime, mtime))
+
+
+def _resolved(dest, source, rel):
+    return os.path.join(get_dest_received_dir(dest, source), rel)
+
+
+def _no_spool_leftovers(dest):
+    leftovers = []
+    for root, _dirs, files in os.walk(dest):
+        leftovers += [os.path.join(root, f) for f in files if ".fastsync-spool." in f]
+    return leftovers
+
+
+class TestStreamedWholeFile:
+    """A file above the (lowered) bound transfers correctly in every mode."""
+
+    @pytest.mark.parametrize(
+        "flags",
+        [
+            ["-a"],
+            ["-a", "--incremental"],
+            ["-a", "-z"],
+            ["-a", "--incremental", "-z"],
+            ["-a", "--threads", "--incremental"],
+            ["-a", "--inplace"],
+            ["-a", "--partial"],
+        ],
+    )
+    def test_above_bound_transfers(self, small_bound_server, flags):
+        tag = "_".join(f.strip("-") for f in flags) or "default"
+        source = os.path.join(TEST_DATA_DIR, f"stream_src_{tag}")
+        dest = os.path.join(TEST_DATA_DIR, f"stream_dst_{tag}")
+        clean_dir(source)
+        clean_dir(dest)
+        data = _payload(FILE_SIZE)
+        _write(os.path.join(source, "big.bin"), data, OLD_MTIME)
+        if "--inplace" in flags:
+            # --inplace only matters when the destination already exists.
+            _write(_resolved(dest, source, "big.bin"), b"stale", OLD_MTIME)
+
+        result, _ = run_client(source, dest, flags=flags, port=small_bound_server.port)
+        assert result.returncode == 0, (result.stderr or result.stdout)[:400]
+
+        got = os.path.join(get_dest_received_dir(dest, source), "big.bin")
+        assert os.path.exists(got), "streamed file was not written"
+        with open(got, "rb") as fh:
+            assert fh.read() == data, "streamed file content mismatch"
+        assert _no_spool_leftovers(dest) == [], "a spool temp file leaked"
+
+
+class TestStreamedBasis:
+    """A basis above the bound is streamed, not refused (compare/copy/link)."""
+
+    def _seed(self, dest, source, data):
+        clean_dir(source)
+        clean_dir(dest)
+        _write(os.path.join(source, "big.bin"), data, OLD_MTIME)
+        # FastSync resolves a relative basis DIR against the destination and
+        # appends the transfer-relative name.
+        _write(os.path.join(dest, "basis", "big.bin"), data, OLD_MTIME)
+
+    def test_compare_dest_above_bound(self, small_bound_server):
+        source = os.path.join(TEST_DATA_DIR, "sbasis_cmp_src")
+        dest = os.path.join(TEST_DATA_DIR, "sbasis_cmp_dst")
+        data = _payload(FILE_SIZE)
+        self._seed(dest, source, data)
+        result, _ = run_client(source, dest,
+                               flags=["-a", "--compare-dest=basis", "--incremental"],
+                               port=small_bound_server.port)
+        assert result.returncode == 0, (result.stderr or result.stdout)[:400]
+        # compare-dest never copies: an already-present destination stays sparse.
+        assert not os.path.exists(_resolved(dest, source, "big.bin"))
+
+    def test_copy_dest_above_bound(self, small_bound_server):
+        source = os.path.join(TEST_DATA_DIR, "sbasis_cpy_src")
+        dest = os.path.join(TEST_DATA_DIR, "sbasis_cpy_dst")
+        data = _payload(FILE_SIZE)
+        self._seed(dest, source, data)
+        result, _ = run_client(source, dest,
+                               flags=["-a", "--copy-dest=basis", "--incremental"],
+                               port=small_bound_server.port)
+        assert result.returncode == 0, (result.stderr or result.stdout)[:400]
+        got = _resolved(dest, source, "big.bin")
+        assert os.path.exists(got)
+        with open(got, "rb") as fh:
+            assert fh.read() == data
+        assert os.stat(got).st_ino != os.stat(os.path.join(dest, "basis", "big.bin")).st_ino
+        assert _no_spool_leftovers(dest) == []
+
+    def test_link_dest_above_bound(self, small_bound_server):
+        source = os.path.join(TEST_DATA_DIR, "sbasis_lnk_src")
+        dest = os.path.join(TEST_DATA_DIR, "sbasis_lnk_dst")
+        data = _payload(FILE_SIZE)
+        self._seed(dest, source, data)
+        result, _ = run_client(source, dest,
+                               flags=["-a", "--link-dest=basis", "--incremental"],
+                               port=small_bound_server.port)
+        assert result.returncode == 0, (result.stderr or result.stdout)[:400]
+        got = _resolved(dest, source, "big.bin")
+        assert os.path.exists(got)
+        with open(got, "rb") as fh:
+            assert fh.read() == data
+        assert os.stat(got).st_ino == os.stat(os.path.join(dest, "basis", "big.bin")).st_ino
+
+
+class TestFuzzyAboveBound:
+    """-y/--fuzzy reuses a basis above the bound by streaming its signature."""
+
+    def test_fuzzy_oversized_sibling(self, small_bound_server):
+        source = os.path.join(TEST_DATA_DIR, "sfuzzy_src")
+        dest = os.path.join(TEST_DATA_DIR, "sfuzzy_dst")
+        clean_dir(source)
+        clean_dir(dest)
+        base = _payload(FILE_SIZE)
+        sibling = bytearray(base)
+        sibling[FILE_SIZE // 2:FILE_SIZE // 2 + 4096] = bytes(
+            (b + 1) % 256 for b in sibling[FILE_SIZE // 2:FILE_SIZE // 2 + 4096])
+        _write(os.path.join(source, "report_v2.txt"), base)
+        _write(os.path.join(get_dest_received_dir(dest, source), "report_v1.txt"), bytes(sibling))
+        result, _ = run_client(
+            source, dest,
+            flags=["-a", "--incremental", "--delta", "--fuzzy", "--stats"],
+            port=small_bound_server.port)
+        assert result.returncode == 0, (result.stderr or result.stdout)[:400]
+        got = _resolved(dest, source, "report_v2.txt")
+        with open(got, "rb") as fh:
+            assert fh.read() == base, "fuzzy reconstruction mismatch"
+        assert _no_spool_leftovers(dest) == []
+
+
+class TestRealLargeFile:
+    """A real >256 MiB transfer, run only in the full (non-PR-gate) suite."""
+
+    def test_real_300mib_transfer(self, shared_server):
+        source = os.path.join(TEST_DATA_DIR, "real_large_src")
+        dest = os.path.join(TEST_DATA_DIR, "real_large_dst")
+        clean_dir(source)
+        clean_dir(dest)
+        n = 300 * 1024 * 1024
+        # Deterministic, compressible pattern written in bounded chunks.
+        chunk = bytes(range(256)) * 4096
+        digest = hashlib.sha256()
+        with open(os.path.join(source, "big.bin"), "wb") as fh:
+            written = 0
+            while written < n:
+                piece = chunk[: min(len(chunk), n - written)]
+                fh.write(piece)
+                digest.update(piece)
+                written += len(piece)
+
+        result, _ = run_client(source, dest, flags=["-a", "--incremental"],
+                               port=shared_server.port)
+        assert result.returncode == 0, (result.stderr or result.stdout)[:400]
+        got = os.path.join(get_dest_received_dir(dest, source), "big.bin")
+        assert os.path.getsize(got) == n
+        got_digest = hashlib.sha256()
+        with open(got, "rb") as fh:
+            while True:
+                block = fh.read(1 << 20)
+                if not block:
+                    break
+                got_digest.update(block)
+        assert got_digest.hexdigest() == digest.hexdigest()
+        shutil.rmtree(source, ignore_errors=True)
+        shutil.rmtree(dest, ignore_errors=True)

tests/test_client_cli.c

@@ -2977,6 +2977,48 @@ static void test_parse_args_delay_updates() {
   config_delete(cfg);
 }
 
+/* rsync parity: --delay-updates implies --delete-after when --delete is
+   active (all updates publish first, then extras are removed).  An explicit
+   other timing is overridden; without --delete no timing is set. */
+static void test_parse_args_delay_updates_implies_delete_after() {
+  Config* cfg = config_create();
+  char* argv[] = {"fastsync", "--delay-updates", "--delete", "/src", "/dst"};
+  int positional_args[2];
+  int positional_count = 0;
+  EXPECT_EQ_INT(parse_args(cfg, 5, argv, positional_args, &positional_count), 0);
+  EXPECT_TRUE(cfg->use_delete);
+  EXPECT_TRUE(cfg->delete_after);
+  EXPECT_FALSE(cfg->delete_before);
+  EXPECT_FALSE(cfg->delete_during);
+  EXPECT_FALSE(cfg->delete_delay);
+  cfg->send_directory = str_dup("/src");
+  cfg->receive_root_directory = str_dup("/dst");
+  EXPECT_TRUE(validate_config(cfg));
+  config_delete(cfg);
+
+  /* An explicit conflicting timing is normalized to delete-after. */
+  cfg = config_create();
+  char* argv_before[] = {"fastsync", "--delay-updates", "--delete-before", "/src", "/dst"};
+  positional_count = 0;
+  EXPECT_EQ_INT(parse_args(cfg, 5, argv_before, positional_args, &positional_count), 0);
+  EXPECT_TRUE(cfg->use_delete);
+  EXPECT_TRUE(cfg->delete_after);
+  EXPECT_FALSE(cfg->delete_before);
+  config_delete(cfg);
+
+  /* Without --delete there is no deletion, so no timing is selected. */
+  cfg = config_create();
+  char* argv_alone[] = {"fastsync", "--delay-updates", "/src", "/dst"};
+  positional_count = 0;
+  EXPECT_EQ_INT(parse_args(cfg, 4, argv_alone, positional_args, &positional_count), 0);
+  EXPECT_FALSE(cfg->use_delete);
+  EXPECT_FALSE(cfg->delete_after);
+  EXPECT_FALSE(cfg->delete_before);
+  EXPECT_FALSE(cfg->delete_during);
+  EXPECT_FALSE(cfg->delete_delay);
+  config_delete(cfg);
+}
+
 /* rsync rejects --delay-updates with --inplace; FastSync must too. */
 static void test_validate_config_delay_updates_rejects_inplace() {
   Config* cfg = valid_client_config();
@@ -5312,6 +5354,7 @@ void test_client_cli() {
   test_parse_args_checksum_seed();
   test_parse_args_temp_dir();
   test_parse_args_delay_updates();
+  test_parse_args_delay_updates_implies_delete_after();
   test_validate_config_delay_updates_rejects_inplace();
   test_validate_config_delay_updates_rejects_reserved_backup_dir();
   test_parse_args_files_from();

tests/test_delay_updates.c

@@ -87,8 +87,10 @@ static void test_delay_updates_no_final_before_publish() {
   const char* final_path = "test_delay_tmp/sub/file.txt";
   /* Before publication the final destination must not contain the file. */
   EXPECT_FALSE(file_path_exists_secure(final_path));
-  /* The complete staged copy must live inside the staging tree. */
+  /* The complete staged copy must live inside the per-run staging tree. */
-  char* staged = path_cat("test_delay_tmp/.fastsync-stage", "/sub/file.txt");
+  EXPECT_NOT_NULL(cfg->delay_context->staging_name);
+  EXPECT_EQ_INT(strncmp(cfg->delay_context->staging_name, ".fastsync-stage.", 16), 0);
+  char* staged = path_cat(cfg->delay_context->staging_root, "/sub/file.txt");
   EXPECT_NOT_NULL(staged);
   // cppcheck-suppress knownConditionTrueFalse
   if (staged) {
@@ -125,6 +127,8 @@ static void test_delay_updates_publish_installs_files() {
   const char* final_path = "test_delay_pub_tmp/sub/file.txt";
   EXPECT_FALSE(file_path_exists_secure(final_path));
 
+  char* staging_root = str_dup(cfg->delay_context->staging_root);
+  EXPECT_NOT_NULL(staging_root);
   EXPECT_TRUE(delay_updates_publish(cfg->delay_context, cfg));
   /* After a successful publish the file is installed and staging is gone. */
   char* content = read_all(final_path);
@@ -134,7 +138,8 @@ static void test_delay_updates_publish_installs_files() {
     EXPECT_EQ_STR(content, "published payload");
     free(content);
   }
-  EXPECT_FALSE(file_path_exists_secure("test_delay_pub_tmp/.fastsync-stage"));
+  EXPECT_FALSE(file_path_exists_secure(staging_root));
+  free(staging_root);
 
 out:
   file_destroy(f);
@@ -158,11 +163,17 @@ static void test_delay_updates_cleanup_removes_staged() {
     goto out;
 
   EXPECT_EQ_INT(file_save_to_disk_full(root, f, cfg), FILE_SAVE_WRITTEN);
-  EXPECT_TRUE(file_path_exists_secure("test_delay_clean_tmp/.fastsync-stage/sub/file.txt"));
+  char* staged_file = path_cat(cfg->delay_context->staging_root, "/sub/file.txt");
+  char* staging_root = str_dup(cfg->delay_context->staging_root);
+  EXPECT_NOT_NULL(staged_file);
+  EXPECT_NOT_NULL(staging_root);
+  EXPECT_TRUE(file_path_exists_secure(staged_file));
 
   delay_updates_cleanup(cfg->delay_context);
-  EXPECT_FALSE(file_path_exists_secure("test_delay_clean_tmp/.fastsync-stage"));
+  EXPECT_FALSE(file_path_exists_secure(staging_root));
   EXPECT_FALSE(file_path_exists_secure("test_delay_clean_tmp/sub/file.txt"));
+  free(staged_file);
+  free(staging_root);
 
 out:
   file_destroy(f);
@@ -274,6 +285,55 @@ out:
   remove_tree(root);
 }
 
+/* Every context picks its own staging directory name, so two delayed
+   transfers to the same root can never share (and corrupt) a staging tree. */
+static void test_delay_updates_unique_staging_name() {
+  DelayUpdatesContext* first = delay_updates_context_create("test_delay_uniq_tmp");
+  DelayUpdatesContext* second = delay_updates_context_create("test_delay_uniq_tmp");
+  EXPECT_NOT_NULL(first);
+  EXPECT_NOT_NULL(second);
+  /* cppcheck-suppress knownConditionTrueFalse -- the EXPECT_NOT_NULL checks above return on NULL */
+  if (first && second) {
+    EXPECT_EQ_INT(strncmp(first->staging_name, ".fastsync-stage.", 16), 0);
+    EXPECT_EQ_INT(strncmp(second->staging_name, ".fastsync-stage.", 16), 0);
+    EXPECT_TRUE(strcmp(first->staging_name, second->staging_name) != 0);
+    EXPECT_TRUE(strcmp(first->staging_root, second->staging_root) != 0);
+  }
+  delay_updates_context_destroy(first);
+  delay_updates_context_destroy(second);
+}
+
+/* A pre-existing destination entry at the exact (random) staging path is not
+   ours: prepare() must refuse rather than wipe it. */
+static void test_delay_updates_prepare_refuses_non_owned_collision() {
+  const char* root = "test_delay_collide_tmp";
+  remove_tree(root);
+  DelayUpdatesContext* context = delay_updates_context_create(root);
+  EXPECT_NOT_NULL(context);
+  // cppcheck-suppress knownConditionTrueFalse
+  if (!context)
+    return;
+  /* Plant a genuine directory with user data at the exact staging path. */
+  EXPECT_TRUE(file_ensure_directory_secure(context->staging_root));
+  char* inner = path_cat(context->staging_root, "keepme.txt");
+  EXPECT_NOT_NULL(inner);
+  // cppcheck-suppress knownConditionTrueFalse
+  if (inner) {
+    EXPECT_TRUE(file_write_to_disk(inner, "genuine", 7, false, false));
+    EXPECT_FALSE(delay_updates_prepare(context));
+    char* content = read_all(inner);
+    EXPECT_NOT_NULL(content);
+    // cppcheck-suppress knownConditionTrueFalse
+    if (content) {
+      EXPECT_EQ_STR(content, "genuine");
+      free(content);
+    }
+    free(inner);
+  }
+  delay_updates_context_destroy(context);
+  remove_tree(root);
+}
+
 /* The reserved staging name must be recognizable for validation, including
    with a trailing slash. */
 static void test_delay_updates_reserved_name_helper() {
@@ -288,6 +348,8 @@ static void test_delay_updates_reserved_name_helper() {
 
 void test_delay_updates() {
   test_delay_updates_reserved_name_helper();
+  test_delay_updates_unique_staging_name();
+  test_delay_updates_prepare_refuses_non_owned_collision();
   test_delay_updates_no_final_before_publish();
   test_delay_updates_publish_installs_files();
   test_delay_updates_cleanup_removes_staged();

tests/test_file.c

@@ -5,6 +5,8 @@
 #include "file.h"
 #include "file_receive.h"
 #include "data.h"
+#include "compression.h"
+#include "delta.h"
 #include "config.h"
 #include "charset.h"
 #include "utils.h"
@@ -2465,7 +2467,172 @@ static void test_manifest_would_delete_protects_absolute_basis() {
   free(extra);
 }
 
+/* #318: a whole-file payload above the streaming bound must be written to a
+ * spool temp file in bounded chunks, not materialized in memory.  Exercises the
+ * raw and zstd-compressed paths and asserts the exact bytes land in the spool. */
+static void test_file_receive_payload_streams(void) {
+  const char* dir = "test_file_stream_tmp";
+  char dest_path[512];
+  snprintf(dest_path, sizeof(dest_path), "%s/out.bin", dir);
+  mkdir(dir, 0777);
+
+  const unsigned long long stream_limit = 4096;
+  size_t size = 20000;
+  unsigned char* payload = malloc(size);
+  EXPECT_NOT_NULL(payload);
+  for (size_t i = 0; i < size; i++)
+    payload[i] = (unsigned char)((i * 7 + 3) & 0xff);
+
+  /* Raw (uncompressed) streamed payload. */
+  {
+    int p[2];
+    EXPECT_EQ_INT(pipe(p), 0);
+    unsigned long long hdr = size;
+    EXPECT_TRUE(send_n_data(p[1], &hdr, sizeof(hdr)));
+    EXPECT_TRUE(send_n_data(p[1], payload, size));
+    Data* buffer = NULL;
+    char* spool = NULL;
+    unsigned long long out_size = 0;
+    EXPECT_TRUE(file_receive_payload(p[0], false, size, dest_path, stream_limit, &buffer, &spool,
+                                     &out_size));
+    EXPECT_NULL(buffer);
+    EXPECT_NOT_NULL(spool);
+    EXPECT_TRUE(out_size == size);
+    /* cppcheck-suppress knownConditionTrueFalse -- EXPECT_NOT_NULL/EXPECT_TRUE above returns on
+     * failure */
+    if (spool) {
+      FILE* fh = fopen(spool, "rb");
+      EXPECT_NOT_NULL(fh);
+      /* cppcheck-suppress knownConditionTrueFalse -- EXPECT_NOT_NULL/EXPECT_TRUE above returns on
+       * failure */
+      if (fh) {
+        unsigned char* got = malloc(size);
+        EXPECT_TRUE(fread(got, 1, size, fh) == size);
+        EXPECT_EQ_INT(memcmp(got, payload, size), 0);
+        free(got);
+        fclose(fh);
+      }
+      unlink(spool);
+      free(spool);
+    }
+    close(p[0]);
+    close(p[1]);
+  }
+
+  /* zstd-compressed payload whose logical size exceeds the bound: the frame is
+   * decompressed incrementally straight into the spool. */
+  {
+    unsigned char* copy = malloc(size);
+    EXPECT_NOT_NULL(copy);
+    memcpy(copy, payload, size);
+    Data* raw = data_create(copy, size); /* data_create takes ownership of copy */
+    Data* compressed = data_compress_codec(raw, COMPRESSION_ALGO_ZSTD, 3, 0);
+    data_destroy(raw);
+    EXPECT_NOT_NULL(compressed);
+    /* cppcheck-suppress knownConditionTrueFalse -- EXPECT_NOT_NULL/EXPECT_TRUE above returns on
+     * failure */
+    if (compressed) {
+      int p[2];
+      EXPECT_EQ_INT(pipe(p), 0);
+      EXPECT_TRUE(send_data(p[1], compressed));
+      Data* buffer = NULL;
+      char* spool = NULL;
+      unsigned long long out_size = 0;
+      EXPECT_TRUE(file_receive_payload(p[0], true, size, dest_path, stream_limit, &buffer, &spool,
+                                       &out_size));
+      EXPECT_NULL(buffer);
+      EXPECT_NOT_NULL(spool);
+      EXPECT_TRUE(out_size == size);
+      /* cppcheck-suppress knownConditionTrueFalse -- EXPECT_NOT_NULL/EXPECT_TRUE above returns on
+       * failure */
+      if (spool) {
+        FILE* fh = fopen(spool, "rb");
+        EXPECT_NOT_NULL(fh);
+        /* cppcheck-suppress knownConditionTrueFalse -- EXPECT_NOT_NULL/EXPECT_TRUE above returns on
+         * failure */
+        if (fh) {
+          unsigned char* got = malloc(size);
+          EXPECT_TRUE(fread(got, 1, size, fh) == size);
+          EXPECT_EQ_INT(memcmp(got, payload, size), 0);
+          free(got);
+          fclose(fh);
+        }
+        unlink(spool);
+        free(spool);
+      }
+      close(p[0]);
+      close(p[1]);
+      data_destroy(compressed);
+    }
+  }
+
+  free(payload);
+  unlink(dest_path);
+  rmdir(dir);
+}
+
+/* #318: the fd-based delta helpers must match the in-memory ones and stream the
+ * reconstruction to a descriptor without allocating the whole output. */
+static void test_delta_stream_helpers(void) {
+  const unsigned char basis[] = {0x11, 0x22, 0x33, 0x44};
+  const char* basis_path = "test_file_delta_basis.bin";
+  int bfd = open(basis_path, O_RDWR | O_CREAT | O_TRUNC, 0600);
+  EXPECT_TRUE(bfd >= 0);
+  EXPECT_TRUE(write(bfd, basis, sizeof(basis)) == (ssize_t)sizeof(basis));
+  EXPECT_TRUE(lseek(bfd, 0, SEEK_SET) == 0);
+
+  DeltaSignature* fd_sig = delta_signature_create_fd_seeded(bfd, sizeof(basis), 4, 0);
+  DeltaSignature* mem_sig = delta_signature_create_seeded(basis, sizeof(basis), 4, 0);
+  EXPECT_NOT_NULL(fd_sig);
+  EXPECT_NOT_NULL(mem_sig);
+  /* cppcheck-suppress knownConditionTrueFalse -- EXPECT_NOT_NULL/EXPECT_TRUE above returns on
+   * failure */
+  if (fd_sig && mem_sig) {
+    EXPECT_TRUE(fd_sig->block_count == mem_sig->block_count);
+    for (uint32_t i = 0; i < fd_sig->block_count; i++) {
+      EXPECT_TRUE(fd_sig->blocks[i].adler32 == mem_sig->blocks[i].adler32);
+      EXPECT_TRUE(fd_sig->blocks[i].xxhash == mem_sig->blocks[i].xxhash);
+    }
+  }
+  delta_signature_destroy(fd_sig);
+  delta_signature_destroy(mem_sig);
+
+  DeltaInstruction instrs[2];
+  instrs[0].type = DELTA_INSTR_BLOCK_MATCH;
+  instrs[0].match.block_index = 0;
+  instrs[0].match.block_offset = 0;
+  instrs[0].match.length = 4;
+  instrs[1].type = DELTA_INSTR_LITERAL;
+  instrs[1].literal.data = (uint8_t*)"XY";
+  instrs[1].literal.length = 2;
+  Delta delta;
+  delta.new_file_size = 6;
+  delta.instruction_count = 2;
+  delta.instructions = instrs;
+  delta.delta_size = 0;
+
+  int out[2];
+  EXPECT_EQ_INT(pipe(out), 0);
+  EXPECT_TRUE(delta_apply_to_fd(NULL, bfd, sizeof(basis), &delta, 4, out[1]));
+  close(out[1]);
+  unsigned char got[6] = {0};
+  size_t total = 0;
+  while (total < sizeof(got)) {
+    ssize_t n = read(out[0], got + total, sizeof(got) - total);
+    if (n <= 0)
+      break;
+    total += (size_t)n;
+  }
+  EXPECT_TRUE(total == sizeof(got));
+  EXPECT_TRUE(memcmp(got, "\x11\x22\x33\x44XY", 6) == 0);
+  close(out[0]);
+  close(bfd);
+  unlink(basis_path);
+}
+
 void test_file() {
+  test_file_receive_payload_streams();
+  test_delta_stream_helpers();
   test_file_create();
   test_file_special_rdev_valid();
   test_file_destroy_null();

tests/test_xattr.c

@@ -892,6 +892,114 @@ static void test_symlink_frame_carries_xattrs() {
   EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
 }
 
+/* --fake-super for DIRECTORIES: rsync stores a directory's faked mode/uid/gid
+ * in `user.rsync.%stat` on the directory itself.  fake_super_store_fd() and
+ * fake_super_restore_fd() operate on a directory descriptor exactly like a
+ * file: the full mode (with S_IFDIR + special bits) is recorded, only the
+ * permission bits are replayed on disk, and the owner is never real-chowned.
+ * Guarded on filesystem xattr support. */
+static void test_fake_super_directory_fd_roundtrip() {
+  const char* root = "test_fake_super_dirfd_tmp";
+  const char* path = "test_fake_super_dirfd_tmp/subdir";
+  rmdir(path);
+  rmdir(root);
+  EXPECT_EQ_INT(mkdir(root, 0700), 0);
+  if (setxattr(root, "user.fastsync-dirprobe", "p", 1, 0) != 0) {
+    rmdir(root);
+    return; /* skip silently when the filesystem has no xattr support */
+  }
+  removexattr(root, "user.fastsync-dirprobe");
+  EXPECT_EQ_INT(mkdir(path, 0755), 0);
+
+  int fd = open(path, O_RDONLY | O_DIRECTORY | O_CLOEXEC);
+  EXPECT_TRUE(fd >= 0);
+  FileAttrPolicy policy = {true, true, false, false, true};
+
+  /* No record yet: restore is a silent no-op on a directory too. */
+  EXPECT_FALSE(fake_super_restore_fd(fd, policy));
+
+  struct stat before;
+  EXPECT_EQ_INT(fstat(fd, &before), 0);
+  fake_super_store_fd(fd, 2222, 3333, S_IFDIR | 01777, 0, 0);
+  char value[64];
+  ssize_t got = fgetxattr(fd, FAKESUPER_XATTR, value, sizeof(value));
+  /* S_IFDIR | 01777 == 0041777 -> "41777 0,0 2222:3333" */
+  EXPECT_EQ_INT((int)got, 19);
+  EXPECT_TRUE(got == 19 && memcmp(value, "41777 0,0 2222:3333", 19) == 0);
+
+  EXPECT_TRUE(fake_super_restore_fd(fd, policy));
+  struct stat after;
+  EXPECT_EQ_INT(fstat(fd, &after), 0);
+  /* The sticky bit is stored in the record but never installed on disk. */
+  EXPECT_EQ_INT((int)(after.st_mode & 07777), 0777);
+  EXPECT_EQ_INT((int)(after.st_mode & (S_ISUID | S_ISGID | S_ISVTX)), 0);
+  EXPECT_EQ_INT((int)after.st_uid, (int)before.st_uid);
+  EXPECT_EQ_INT((int)after.st_gid, (int)before.st_gid);
+
+  close(fd);
+  removexattr(path, FAKESUPER_XATTR);
+  rmdir(path);
+  rmdir(root);
+}
+
+/* The deferred directory-metadata pass is where a recursive -a --fake-super
+ * transfer stamps each directory: dir_metadata_list_apply() must park the
+ * directory's full stat in the reserved xattr and replay only its permission
+ * bits on disk.  This is the recursive-path counterpart of the explicit
+ * --dirs store in file_save_directory_to_disk().  Guarded on xattr support. */
+static void test_fake_super_directory_deferred_apply() {
+  const char* root = "test_fake_super_dirdir_tmp";
+  const char* leaf = "subdir";
+  const char* path = "test_fake_super_dirdir_tmp/subdir";
+  rmdir(path);
+  rmdir(root);
+  EXPECT_EQ_INT(mkdir(root, 0700), 0);
+  if (setxattr(root, "user.fastsync-dirprobe", "p", 1, 0) != 0) {
+    rmdir(root);
+    return; /* skip silently when the filesystem has no xattr support */
+  }
+  removexattr(root, "user.fastsync-dirprobe");
+  EXPECT_EQ_INT(mkdir(path, 0755), 0);
+
+  FileMetadata m;
+  memset(&m, 0, sizeof(m));
+  m.mode = S_IFDIR | 02751;
+  m.uid = 1001;
+  m.gid = 1002;
+  m.mtime_sec = 1234567890;
+
+  Config* config = config_create();
+  EXPECT_NOT_NULL(config);
+  config->use_metadata = true;
+  config->preserve_perms = true;
+  config->preserve_times = true;
+  config->fake_super = true;
+
+  identity_clear_active();
+  DirTimeList list;
+  dir_time_list_init(&list);
+  EXPECT_TRUE(dir_time_list_add(&list, leaf, &m, NULL));
+  dir_metadata_list_apply(&list, root, config);
+  dir_time_list_free(&list);
+
+  char value[64];
+  ssize_t got = getxattr(path, FAKESUPER_XATTR, value, sizeof(value));
+  /* S_IFDIR | 02751 -> "42751 0,0 1001:1002" (resolved ids == source ids). */
+  EXPECT_EQ_INT((int)got, 19);
+  EXPECT_TRUE(got == 19 && memcmp(value, "42751 0,0 1001:1002", 19) == 0);
+
+  struct stat st;
+  EXPECT_EQ_INT(stat(path, &st), 0);
+  /* Only the permission bits land on disk; setgid stays in the record. */
+  EXPECT_EQ_INT((int)(st.st_mode & 07777), 0751);
+  EXPECT_EQ_INT((int)(st.st_mode & (S_ISUID | S_ISGID | S_ISVTX)), 0);
+
+  config_delete(config);
+  removexattr(path, FAKESUPER_XATTR);
+  rmdir(path);
+  rmdir(root);
+}
+
 void test_xattr() {
   test_xattr_list_clone();
   test_xattr_capture_symlink_nofollow();
@@ -910,5 +1018,7 @@ void test_xattr() {
   test_fake_super_rsync_format();
   test_fake_super_no_real_chown();
   test_fake_super_storage_resolution();
+  test_fake_super_directory_fd_roundtrip();
+  test_fake_super_directory_deferred_apply();
   test_file_save_directory_applies_xattrs();
 }