HANDOFF.md

@@ -94,6 +94,21 @@
     now **111 ✅ / 14 ⚠️ / 32 ❌ = 157**; differential + unit tests added in
     `test_features.py`, `test_option_parity.py`, `test_delete_plan.c`,
     `test_delete_delay_budget_parity.py`, `test_delete_timing_parity.py`.
+12. **No-wire parity track 2b** on `feat/parity-2.28` (no protocol change):
+    `--progress`/`-P`/`--info=progress` (when not `--quiet`) now run an opt-in
+    paths-only metadata pre-count (no file reads/hashing) that supplies rsync's
+    full file-list total for the `to-chk` denominator and the directory names,
+    and emits per-directory/symlink/special name lines, in both the sequential
+    and `--threads` paths. `--delete-during`/`--delete-delay` reuse their
+    keep-set pre-scan instead of a second walk; non-progress runs are
+    unaffected. Differential tests (`progress`/`progress_threads` over a new
+    `multidir` corpus) match rsync's name set and `to-chk` denominator on a
+    fresh transfer, and the single-file byte-identical test still passes;
+    emission order (rsync's sorted depth-first vs FastSync's readdir/BFS stream)
+    plus re-run over-naming (unconditional `./`, ancestor dirs named with a
+    transferred child, and no quick-check for symlinks/empty dirs) remain the
+    caveats, so the row stays ⚠️ and the matrix is unchanged at
+    **111 ✅ / 14 ⚠️ / 32 ❌ = 157**.
 
 ## Next steps
 1. **Merge PR #284** (`dev` -> `main`) once reviewed (protected branch).

RSYNC_COMPAT.md

@@ -39,6 +39,8 @@ matrix is **111 ✅ / 13 ⚠️ / 33 ❌ = 157**.
 
 **Parity track 1 (no-wire, on `feat/parity-2.28`).** Three residual burn-downs that required no protocol change: (1) `-n --delete` now propagates the filter-excluded and size-pruned protected prefixes (and the synchronized-directory scope) into the dry-run keep-set manifest, so its would-delete report matches rsync for source-derived protections and no longer lists the file merely being updated (the destination-only exclude/filter residual and readdir ordering remain, leaving the row ⚠️); (2) `--delete-delay` now charges `--max-delete` on actual removals and re-scans a queued directory at commit to remove content created after the plan, with an independent cap bounding the deferred list and the row's prior caveat closed (ordering of partial removals remains, leaving the row ⚠️); and (3) `--info=name2` now emits `NAME is uptodate` and `--info=name` emits the leading `./` root name line (only the root-line trigger condition and the receiver-side `skip` wording remain, leaving the row ⚠️). The matrix is now **111 ✅ / 14 ⚠️ / 32 ❌ = 157**.
 
+**Parity track 2b (no-wire, on `feat/parity-2.28`).** An opt-in, paths-only metadata pre-count for `--progress`/`-P`/`--info=progress` (only when not `--quiet`) now gives the `to-chk` denominator rsync's full file-list total (every regular file, directory, symlink and special plus the transfer root) and emits the per-directory/symlink/special name lines, in both the sequential and `--threads` paths; `--delete-during`/`--delete-delay` reuse their keep-set pre-scan so no second walk happens, and non-progress runs are unaffected. A fresh multi-directory differential against rsync 3.4.1 matches the name set and the `to-chk` denominator, while a single-file transfer stays byte-identical; the remaining caveats are emission order (rsync's sorted depth-first vs FastSync's readdir/BFS stream, so the `to-chk` numerator and interleaving differ) and re-run/receiver-state-driven over-naming (unconditional `./`, ancestor dirs emitted with a transferred child, and no quick-check for symlinks/empty dirs), keeping the row ⚠️. The matrix is unchanged at **111 ✅ / 14 ⚠️ / 32 ❌ = 157**.
+
 **Parity completion wave (protocol 2.23.0 → 2.26.0).** This wave closed the
 remaining gaps the rsync-parity wave left open (delete timing, wire counters and
 output, codec breadth, general `-R`/`-d`, the full filter grammar, receiver-side
@@ -85,7 +87,7 @@ Every one of those has an entry below with its remaining caveats.
 | `--stats` | Give transfer stats | ⚠️ Caveat | Prints transfer statistics. Protocol 2.25.0 populates the receiver-only counters the sender cannot observe (`Matched data`, `Number of deleted files`) from the receiver's `STATUS_STATS` report; the sender tracks the scanned file list per type so `Number of files` carries rsync's `(reg: X, dir: Y, link: Z, special: W)` breakdown (directories come from the scanner's captured directory list, present for `-a`/`-t`/`-p`), `Number of regular files transferred` excludes symlinks/specials and up-to-date files, `Total file size` includes symlink target lengths, and `Total transferred file size` counts only transferred files. **Protocol 2.28.0 extends `STATUS_STATS`** with receiver-observed `literal_bytes` and the four `created_*` counters: `Number of created files` now carries rsync's `(reg/dir/link/special)` breakdown (the receiver reports which destination entries it newly created, including implicitly-created parent directories below the transfer root) and `Literal data` is exact for a delta transfer (the receiver counts the literal fragments it stored, not the whole source size) — all differential-tested in the sequential and `--threads` paths against rsync 3.4.1 for fresh-create, update and delta shapes. **Remaining divergences:** a recursive scan that preserves no directory attribute (`-r` without `-t`/`-p`) captures no directory entries, so the `dir:` category is then omitted from `Number of files`; rsync's per-type breakdown on `Number of deleted files` is not reproduced; and `Total bytes sent`/`received` are FastSync wire bytes framed differently from rsync's, so they are not numerically comparable |
 | `-h`, `--human-readable` | Human-readable numbers | ✅ Parity | Formats transfer byte and rate counts using rsync's **decimal** (base-1000) units, matching rsync `-h` (e.g. `1.23M`), not binary units. **A lone `-h` with no transfer arguments prints help instead** (protocol 2.26.0), matching the rsync idiom; `-h` alongside a transfer remains human-readable |
 | `-i`, `--itemize-changes` | Per-file change summary | ✅ Parity | Prints rsync-style `>f+++++++++` lines to stdout only for files actually sent (also under `-j`/`--threads`); unchanged files print nothing, matching single-`-i` behavior |
-| `--progress` | Show progress | ⚠️ Caveat | Protocol 2.25.0 prints rsync-style per-file progress blocks (percent, transferred/total bytes, rate, elapsed, `(xfr#N, to-chk=M/T)`) fed by the receiver's `STATUS_STATS`, in both the sequential and `--threads` send paths. FastSync now also prints rsync's leading `./` transfer-root line and counts that root entry in the `to-chk` denominator, so a **single-file transfer's name lines and deterministic frames are byte-identical to rsync** (differential test). **Remaining divergences:** for a multi-directory tree rsync prints a per-directory name line and its `to-chk` denominator includes every directory/symlink/special entry; FastSync's streaming scan emits only file-name lines and counts just the root plus transferred files (a full flist pre-count would be needed), and the rate/ETA are wall-clock dependent |
+| `--progress` | Show progress | ⚠️ Caveat | Protocol 2.25.0 prints rsync-style per-file progress blocks (percent, transferred/total bytes, rate, elapsed, `(xfr#N, to-chk=M/T)`) fed by the receiver's `STATUS_STATS`, in both the sequential and `--threads` send paths. FastSync also prints rsync's leading `./` transfer-root line and, when progress is requested (`--progress`/`-P`/`--info=progress`) and not `--quiet`, runs a **paths-only metadata pre-scan** (no file reads, no hashing) that supplies rsync's file-list total `T` for the `to-chk` denominator and the directory names; `--delete-during`/`--delete-delay` reuse their existing keep-set pre-scan instead of walking twice, and non-progress runs are untouched. Per-directory name lines are emitted (trailing `/`), and symlink (` -> target`) and special entries are named too, so a **fresh multi-directory tree's name set and `to-chk` denominator match rsync 3.4.1** (differential test, sequential and `--threads`) and a **single-file transfer's name lines and deterministic frames remain byte-identical** to rsync. **Remaining divergences:** rsync emits entries in sorted depth-first order while FastSync streams them in the scanner's readdir/BFS order, so the interleaving and the `to-chk` numerator differ (the denominator matches); the leading `./` root line is emitted unconditionally rather than keyed off rsync's root-attribute-change decision, and an ancestor directory line is emitted whenever a child transfers (rsync suppresses it when the directory itself is unchanged); on a re-run, entries without a quick-check (symlinks, empty directories) are still named where rsync stays silent; and the rate/ETA are wall-clock dependent |
 | `-P` | Same as --partial --progress | ✅ Parity | Parses to `--partial` + `--progress`. The independent `--partial` retention semantics are rsync parity: an interrupted write retains the already-written temp at the destination (best-effort) so a later `--append`/`--append-verify` can resume. Progress presentation is owned by the `--progress` row; there is no separate `-P` divergence |
 | `--out-format=FORMAT` | Custom output format | ❌ Divergent | Per-transfer template on stdout; tokens `%f` `%n` `%l` `%b` `%c` `%C` `%i` `%M` `%o` `%U` `%G` `%t` `%%`. `%C` now uses the negotiated transfer algorithm (`--checksum-choice`, default `xxh128`, seed 0) and renders every algorithm exactly like rsync — xxh128 high-then-low, xxh64/xxh3 big-endian, md5/md4/sha1 standard hex, `none` a blank 2-char column — differential-tested across all algorithms. `%f`/`%n`/`%l`/`%i`/`%M`/`%U`/`%G`/`%B` also match. **Reclassified because `%b`/`%c` are protocol-specific and cannot match:** a differential against rsync 3.4.1 shows whole-file `%c = 16` for both, but rsync whole-file `%b = filesize + 27 + transfer-digest-bytes` (39 for a 0-byte file; 43/35/47 for xxh128/xxh64/sha1 on a 12-byte file) while FastSync `%b` counts its own framing; in delta mode rsync `%c = 16 + 6·ceil(filesize/block_size)` (verified at block sizes 512/700/1024/2048) while FastSync counts its own signature handshake, and rsync `%b` is its token stream. FastSync's wire bytes are a different quantity, so exact `%b`/delta-`%c` equality is impossible |
 | `--log-file=FILE` | Log to file | ✅ Parity | `log_file` config field |

src/client/client_send.c

@@ -275,10 +275,61 @@ static void transfer_stats_note_transferred(TransferStats* stats, const File* fi
 
 static const char* delete_display_path(const Config* config, const char* path);
 
+/* Paths-only pre-count of the source file list, built once at transfer start
+ * when progress output is requested.  rsync's `to-chk` denominator is the whole
+ * file list -- every regular file, directory, symlink and special plus the
+ * transfer root -- while the streaming scan never emits directories.  A
+ * metadata-only walk (no file reads, no hashing) supplies that total and the
+ * directory names, so the opt-in pass leaves non-progress runs untouched. */
+typedef struct {
+  unsigned long long total;
+  ArrayList* dir_paths; /* owned char* in transfer-relative display form */
+} ProgressPrecount;
+
 static bool g_progress_active;
 static unsigned long long g_progress_xferred;
-static unsigned long long g_progress_seen;
+static unsigned long long g_progress_index;
+static unsigned long long g_progress_total;
 static struct timespec g_progress_file_start;
+static ProgressPrecount g_progress_precount;
+static PathIndex g_progress_dir_index;
+static bool g_progress_dir_index_valid;
+static StrHashSet g_progress_emitted;
+static bool g_progress_emitted_valid;
+static ArrayList* g_progress_emitted_keys;
+
+static bool progress_requested(const Config* config) {
+  return config != NULL && !config->quiet &&
+         (config->show_progress || (config->info_level & LOG_INFO_PROGRESS) != 0);
+}
+
+static void progress_precount_dispose(ProgressPrecount* p) {
+  if (p->dir_paths != NULL) {
+    array_list_delete(p->dir_paths);
+    p->dir_paths = NULL;
+  }
+  p->total = 0;
+}
+
+static void client_progress_cleanup(void) {
+  if (g_progress_dir_index_valid) {
+    path_index_free(&g_progress_dir_index);
+    g_progress_dir_index_valid = false;
+  }
+  if (g_progress_emitted_valid) {
+    str_hash_set_free(&g_progress_emitted);
+    g_progress_emitted_valid = false;
+  }
+  if (g_progress_emitted_keys != NULL) {
+    array_list_delete(g_progress_emitted_keys);
+    g_progress_emitted_keys = NULL;
+  }
+  progress_precount_dispose(&g_progress_precount);
+  g_progress_active = false;
+  g_progress_total = 0;
+  g_progress_index = 0;
+  g_progress_xferred = 0;
+}
 
 static void progress_first_frame(unsigned long long size, char* out, size_t out_size) {
   char ofs_buf[32];
@@ -315,12 +366,12 @@ static void progress_final_frame(unsigned long long size, char* out, size_t out_
   unsigned long long remain = (unsigned long long)(diff_ms / 1000);
   snprintf(rembuf, sizeof(rembuf), "%4u:%02u:%02u", (unsigned)(remain / 3600),
            (unsigned)((remain / 60) % 60), (unsigned)(remain % 60));
-  /* rsync's `to-chk` denominator is the whole file list, which includes the
+  /* rsync's `to-chk` denominator is the whole file list (the pre-count); the
-     transfer-root directory FastSync never emits as a transfer entry.  Count
+     numerator falls as each entry is processed, root first.  Without a
-     that root entry so a single-file transfer matches rsync exactly. */
+     pre-count (the paths-only walk failed) fall back to the transferred-file
-  unsigned long long total = g_progress_seen + 1;
+     count so the single-file layout stays intact. */
-  unsigned long long to_chk =
+  unsigned long long total = g_progress_total > 0 ? g_progress_total : g_progress_xferred + 1;
-      g_progress_seen > g_progress_xferred ? g_progress_seen - g_progress_xferred : 0;
+  unsigned long long to_chk = total > g_progress_index ? total - g_progress_index - 1 : 0;
   snprintf(out, out_size, "\r%15s %3d%% %7.2f%s %s (xfr#%llu, to-chk=%llu/%llu)\n", ofs_buf, 100,
            rate, units, rembuf, g_progress_xferred, to_chk, total);
 }
@@ -368,12 +419,73 @@ static void print_delete_reports(const Config* config, const ArrayList* paths) {
   fflush(stdout);
 }
 
+static void client_progress_emit_ancestors(const Config* config, const char* rel) {
+  if (!g_progress_dir_index_valid || !g_progress_emitted_valid || g_progress_emitted_keys == NULL ||
+      rel == NULL)
+    return;
+  size_t rel_len = strlen(rel);
+  for (size_t i = 0; i < rel_len; i++) {
+    if (rel[i] != '/')
+      continue;
+    char* prefix = malloc(i + 1);
+    if (prefix == NULL)
+      return;
+    memcpy(prefix, rel, i);
+    prefix[i] = '\0';
+    if (path_index_contains(&g_progress_dir_index, prefix) &&
+        !str_hash_set_lookup(&g_progress_emitted, prefix)) {
+      char* key = str_dup(prefix);
+      if (key != NULL && array_list_add(g_progress_emitted_keys, key)) {
+        str_hash_set_insert_ref(&g_progress_emitted, key);
+        char* escaped = output_escape(prefix, config->eight_bit_output);
+        printf("%s/\n", escaped ? escaped : prefix);
+        free(escaped);
+        g_progress_index++;
+      } else {
+        free(key);
+      }
+    }
+    free(prefix);
+  }
+}
+
+/* rsync's --info=name/progress line for one entry: transfer-relative name (a
+ * trailing slash for directories) plus the ` -> target` symlink suffix. */
+static char* progress_entry_line(const File* file, const char* rel) {
+  const char* arrow = NULL;
+  const char* target = NULL;
+  if (file->is_symlink && file->symlink_target != NULL) {
+    arrow = " -> ";
+    target = file->symlink_target;
+  } else if (file->link_group != 0 && !file->link_first && file->hardlink_target != NULL) {
+    arrow = " => ";
+    target = file->hardlink_target;
+  }
+  size_t rel_len = strlen(rel);
+  bool dir_slash = file->is_dir && (rel_len == 0 || rel[rel_len - 1] != '/');
+  size_t extra = (dir_slash ? 1u : 0u) + (target != NULL ? 4u + strlen(target) : 0u);
+  char* line = malloc(rel_len + extra + 1);
+  if (line == NULL)
+    return NULL;
+  memcpy(line, rel, rel_len);
+  size_t off = rel_len;
+  if (dir_slash)
+    line[off++] = '/';
+  if (target != NULL) {
+    memcpy(line + off, arrow, 4);
+    off += 4;
+    memcpy(line + off, target, strlen(target));
+    off += strlen(target);
+  }
+  line[off] = '\0';
+  return line;
+}
+
 static void client_progress_begin(const Config* config) {
   change_reset_name_root();
-  g_progress_active =
+  g_progress_active = progress_requested(config);
-      (config->show_progress || info_flag_enabled(config, LOG_INFO_PROGRESS)) && !config->quiet;
   g_progress_xferred = 0;
-  g_progress_seen = 0;
+  g_progress_index = 1; /* the transfer root is file-list entry #0 */
   if (!g_progress_active) {
     /* `--info=flist` prints rsync's file-list header even without progress. */
     if (!config->quiet && info_flag_enabled(config, LOG_INFO_FLIST)) {
@@ -395,12 +507,14 @@ static void client_progress_begin(const Config* config) {
 static void client_progress_file(const Config* config, const File* file) {
   if (!g_progress_active || file == NULL || !file->data)
     return;
-  g_progress_seen++;
   g_progress_xferred++;
   unsigned long long size = file->data->size;
   if (!config->itemize_changes && config->out_format == NULL) {
-    const char* name = delete_display_path(config, file_wire_path(file));
+    const char* rel = delete_display_path(config, file_wire_path(file));
-    printf("%s\n", name ? name : "");
+    client_progress_emit_ancestors(config, rel);
+    char* escaped = output_escape(rel, config->eight_bit_output);
+    printf("%s\n", escaped ? escaped : (rel ? rel : ""));
+    free(escaped);
   }
   clock_gettime(CLOCK_MONOTONIC, &g_progress_file_start);
   char frame[160];
@@ -408,9 +522,41 @@ static void client_progress_file(const Config* config, const File* file) {
   fputs(frame, stdout);
   progress_final_frame(size, frame, sizeof(frame));
   fputs(frame, stdout);
+  g_progress_index++;
   fflush(stdout);
 }
 
+/* Emit the name line for a transferred non-regular entry (directory, symlink,
+ * special or hard-link sibling): rsync prints these in the file list but has no
+ * progress frame for them. */
+static void client_progress_name(const Config* config, const File* file) {
+  if (!g_progress_active || file == NULL)
+    return;
+  const char* rel = delete_display_path(config, file_wire_path(file));
+  if (!config->itemize_changes && config->out_format == NULL) {
+    client_progress_emit_ancestors(config, rel);
+    char* line = progress_entry_line(file, rel ? rel : "");
+    if (line != NULL) {
+      char* escaped = output_escape(line, config->eight_bit_output);
+      printf("%s\n", escaped ? escaped : line);
+      free(escaped);
+      free(line);
+      fflush(stdout);
+    }
+  }
+  g_progress_index++;
+}
+
+/* An entry the receiver already had prints no name under --progress but still
+ * occupies a file-list slot in the `to-chk` numerator. */
+static void client_progress_uptodate(const Config* config, const File* file) {
+  (void)config;
+  (void)file;
+  if (!g_progress_active)
+    return;
+  g_progress_index++;
+}
+
 /* Compiled scanner inputs that are shared read-only across scanner instances
  * and, in -m mode, across worker threads. `base_filters` owns the compiled
  * command-line + -C rules; the FileListSet allow-set lives in the Config.
@@ -555,6 +701,126 @@ static void prepared_scanner_destroy(PreparedScanner* prepared) {
   prepared->relative_prefix = NULL;
 }
 
+static bool progress_precount_add_dir(ProgressPrecount* p, const char* path) {
+  if (path == NULL || path[0] == '\0')
+    return true;
+  char* dup = str_dup(path);
+  if (dup == NULL)
+    return false;
+  if (array_list_add(p->dir_paths, dup))
+    return true;
+  free(dup);
+  return false;
+}
+
+/* Metadata-only walk collecting the full file-list total and every directory
+ * name.  It uses its own scanner (fresh filter compilation and hard-link table)
+ * so the data pass's link-group state is never perturbed. */
+static bool progress_precount_scan(const Config* config, ProgressPrecount* out) {
+  out->dir_paths = array_list_create(free);
+  if (out->dir_paths == NULL)
+    return false;
+  out->total = 0;
+  PreparedScanner prepared;
+  memset(&prepared, 0, sizeof(prepared));
+  if (!prepare_scanner(config, 0, &prepared)) {
+    progress_precount_dispose(out);
+    return false;
+  }
+  ScannerOptions local = prepared.options;
+  local.list_dirs = true;
+  local.note_nonreg = false;
+  local.use_metadata = false;
+  local.preserve_xattrs = false;
+  local.preserve_acls = false;
+  local.checksum = false;
+  local.capture_dir_times = false;
+  local.excluded_paths = NULL;
+  local.size_skipped_paths = NULL;
+  local.synced_dirs = NULL;
+  local.plan_dirs = NULL;
+  local.dir_entries = NULL;
+  local.dir_entries_mutex = NULL;
+  local.hardlinks = NULL;
+  DirectoryScanner* scanner = directory_scanner_create_with_options(config->send_directory, &local);
+  bool ok = scanner != NULL;
+  if (scanner != NULL) {
+    Chunk* chunk;
+    while (ok && (chunk = directory_scanner_next(scanner)) != NULL) {
+      out->total += (unsigned long long)chunk->element_count;
+      for (int i = 0; i < chunk->element_count && ok; i++) {
+        const File* f = chunk->items[i];
+        if (f != NULL && f->is_dir)
+          ok = progress_precount_add_dir(out, delete_display_path(config, file_wire_path(f)));
+      }
+      chunk_destroy(chunk);
+    }
+    if (ok && directory_scanner_failed(scanner))
+      ok = false;
+    directory_scanner_destroy(scanner);
+  }
+  prepared_scanner_destroy(&prepared);
+  if (!ok) {
+    progress_precount_dispose(out);
+    return false;
+  }
+  out->total += 1; /* the transfer root "." */
+  return true;
+}
+
+/* Reuse the --delete-during/--delete-delay keep-set pre-scan: its traversed
+ * directory list already holds every directory and `non_dir_count` the entries
+ * counted during that same pass, so progress costs no second walk. */
+static bool progress_precount_from_plan_dirs(const Config* config, const ArrayList* plan_dirs,
+                                             unsigned long long non_dir_count,
+                                             ProgressPrecount* out) {
+  out->dir_paths = array_list_create(free);
+  if (out->dir_paths == NULL)
+    return false;
+  out->total = non_dir_count + 1;
+  for (int i = 0; i < plan_dirs->size; i++) {
+    const char* path = (const char*)plan_dirs->items[i];
+    const char* rel = config->send_directory != NULL
+                          ? utils_strip_transfer_root(path, config->send_directory)
+                          : path;
+    if (!progress_precount_add_dir(out, rel)) {
+      progress_precount_dispose(out);
+      return false;
+    }
+  }
+  out->total += (unsigned long long)out->dir_paths->size;
+  return true;
+}
+
+/* Build the optional progress pre-count.  A failed pre-count is non-fatal: the
+ * transfer proceeds and the progress denominator falls back to the transferred
+ * file count. */
+static void client_progress_prepare(const Config* config, const ArrayList* plan_dirs,
+                                    unsigned long long plan_non_dir_count) {
+  client_progress_cleanup();
+  g_progress_active = progress_requested(config);
+  if (!g_progress_active)
+    return;
+  bool ok = plan_dirs != NULL ? progress_precount_from_plan_dirs(
+                                    config, plan_dirs, plan_non_dir_count, &g_progress_precount)
+                              : progress_precount_scan(config, &g_progress_precount);
+  if (!ok) {
+    g_progress_total = 0;
+    return;
+  }
+  g_progress_total = g_progress_precount.total;
+  if (g_progress_precount.dir_paths != NULL && g_progress_precount.dir_paths->size > 0 &&
+      path_index_build(&g_progress_dir_index,
+                       (const char* const*)g_progress_precount.dir_paths->items,
+                       (size_t)g_progress_precount.dir_paths->size))
+    g_progress_dir_index_valid = true;
+  if (str_hash_set_init(&g_progress_emitted, (size_t)(g_progress_precount.dir_paths != NULL
+                                                          ? g_progress_precount.dir_paths->size + 1
+                                                          : 1)))
+    g_progress_emitted_valid = true;
+  g_progress_emitted_keys = array_list_create(free);
+}
+
 /* -R/--relative implied directories: rsync transmits the metadata of the
  * parent directories implied by the source path (every prefix component above
  * the source root) so the receiver applies their attributes to the created
@@ -1569,9 +1835,12 @@ static bool send_delete_manifest_early(Client* client, ArrayList* manifest,
    directory and *io_error_out reports it (the caller still performs the
    deletion but reports the run as errored). */
 static bool scan_paths_only(const Config* config, const ScannerOptions* options,
-                            ArrayList* manifest, DeletePlanSender* plans, bool* io_error_out) {
+                            ArrayList* manifest, DeletePlanSender* plans, bool* io_error_out,
+                            unsigned long long* non_dir_count_out) {
   if (io_error_out)
     *io_error_out = false;
+  if (non_dir_count_out)
+    *non_dir_count_out = 0;
   ScannerOptions local = *options;
   /* The pre-scan is a paths-only pass with no client output; it must not emit
      --info=nonreg lines (the data pass does that once). */
@@ -1582,6 +1851,13 @@ static bool scan_paths_only(const Config* config, const ScannerOptions* options,
   bool ok = true;
   Chunk* chunk;
   while ((chunk = directory_scanner_next(scanner)) != NULL) {
+    if (non_dir_count_out) {
+      for (int i = 0; i < chunk->element_count; i++) {
+        const File* f = chunk->items[i];
+        if (f && !f->is_dir)
+          (*non_dir_count_out)++;
+      }
+    }
     if (manifest && !add_chunk_to_manifest(manifest, chunk)) {
       ok = false;
       chunk_destroy(chunk);
@@ -2417,6 +2693,7 @@ static int send_chunk_with_removal(Client* client, Chunk* chunk, Config* config,
       if (!send_directory_entry(client, f, config))
         return -1;
       change_emit_dir_sent(config, f);
+      client_progress_name(config, f);
       continue;
     }
     /* --hard-links/-H sibling: a later member of a hard-link group that has no
@@ -2430,6 +2707,7 @@ static int send_chunk_with_removal(Client* client, Chunk* chunk, Config* config,
           !send_wire_str(client->file_descriptor, f->hardlink_target))
         return -1;
       change_emit_file_sent(config, f);
+      client_progress_name(config, f);
       continue;
     }
     /* Symlink entry (-l / -k keep-as-symlink): only the target rides the wire. */
@@ -2437,6 +2715,7 @@ static int send_chunk_with_removal(Client* client, Chunk* chunk, Config* config,
       if (!send_symlink_entry(client, f, config))
         return -1;
       change_emit_file_sent(config, f);
+      client_progress_name(config, f);
       continue;
     }
     /* --devices/--specials: a device/special node is recreated on the receiver,
@@ -2445,6 +2724,7 @@ static int send_chunk_with_removal(Client* client, Chunk* chunk, Config* config,
       if (!file_send_special(f, client->file_descriptor, config->use_metadata))
         return -1;
       change_emit_file_sent(config, f);
+      client_progress_name(config, f);
       continue;
     }
     bool stream = f->data->data == NULL && f->data->size > 0;
@@ -2458,6 +2738,7 @@ static int send_chunk_with_removal(Client* client, Chunk* chunk, Config* config,
     if (rc == 1) {
       source_file_destroy(source);
       change_emit_file_uptodate(config, f);
+      client_progress_uptodate(config, f);
       continue;
     }
     if (rc < 0) {
@@ -3004,6 +3285,7 @@ int send_files(Config* config) {
   bool delete_per_dir = config->use_delete && config_delete_timing_per_dir(config) && !config->dirs;
   bool send_failed = false;
   bool had_scan_io = false;
+  unsigned long long per_dir_non_dir_count = 0;
   PreparedScanner prepared;
   memset(&prepared, 0, sizeof(prepared));
   if (!config_send(client->file_descriptor, config))
@@ -3065,7 +3347,7 @@ int send_files(Config* config) {
     if (!early_manifest)
       goto send_fail;
     bool prescan_ok =
-        scan_paths_only(config, &prepared.options, early_manifest, NULL, &had_scan_io);
+        scan_paths_only(config, &prepared.options, early_manifest, NULL, &had_scan_io, NULL);
     bool early_ok = false;
     bool skip_delete = false;
     if (prescan_ok) {
@@ -3107,7 +3389,8 @@ int send_files(Config* config) {
     if (!plan_sender || !plan_dirs)
       goto send_fail;
     prepared.options.plan_dirs = plan_dirs;
-    bool prescan_ok = scan_paths_only(config, &prepared.options, NULL, plan_sender, &had_scan_io);
+    bool prescan_ok = scan_paths_only(config, &prepared.options, NULL, plan_sender, &had_scan_io,
+                                      &per_dir_non_dir_count);
     bool plans_ok = false;
     bool skip_delete = false;
     if (prescan_ok) {
@@ -3145,6 +3428,11 @@ int send_files(Config* config) {
     if (!manifest)
       goto send_fail;
   }
+  /* --progress/--info=progress: pre-count the file list for rsync's to-chk
+     denominator.  When a --delete-during/--delete-delay pre-scan already ran,
+     reuse its traversed directory list instead of walking the tree again. */
+  if (progress_requested(config))
+    client_progress_prepare(config, plan_dirs, per_dir_non_dir_count);
   /* Phase 6: compute the client-only stop deadline once at transfer start.  The
      early-delete pre-scan above deliberately ignores it so the keep-set (and
      its committed deletion) is always complete and correct. */
@@ -3366,6 +3654,7 @@ send_fail:
   if (scanner)
     directory_scanner_destroy(scanner);
   prepared_scanner_destroy(&prepared);
+  client_progress_cleanup();
   disconnect_transfer_client(client);
   protocol_session_unbind();
   client_set_abort_armed(false);
@@ -3449,6 +3738,7 @@ int send_files_multithreaded(Config** config_ptr) {
       stop_condition_make(config->stop_after_mins > 0, config->stop_after_mins, config->stop_at_set,
                           config->stop_at, now_mono);
   bool collect_excluded = config->use_delete && !config->delete_excluded;
+  unsigned long long pre_scan_non_dir = 0;
   if (config->use_delete) {
     if (collect_excluded) {
       context->excluded_paths = array_list_create(free);
@@ -3507,8 +3797,9 @@ int send_files_multithreaded(Config** config_ptr) {
         prepared_ok = prepared_ok && context->manifest != NULL;
       }
       bool prebuilt =
-          prepared_ok && scan_paths_only(config, &prepared.options, context->manifest,
+          prepared_ok &&
-                                         context->delete_plans, &context->scan_had_io_error);
+          scan_paths_only(config, &prepared.options, context->manifest, context->delete_plans,
+                          &context->scan_had_io_error, &pre_scan_non_dir);
       prepared_scanner_destroy(&prepared);
       if (per_dir && prebuilt) {
         const char* walk_root = delete_plan_walk_root(config, context->synced_dirs);
@@ -3571,6 +3862,11 @@ int send_files_multithreaded(Config** config_ptr) {
     pipeline_context_sender_destroy(context);
     return 1;
   }
+  /* --progress/--info=progress: pre-count the file list for rsync's to-chk
+     denominator, reusing a --delete-during/--delete-delay pre-scan when one
+     already ran. */
+  if (progress_requested(config))
+    client_progress_prepare(config, context->plan_dirs, pre_scan_non_dir);
 
   thrd_t scanner, loader, sender;
   bool scanner_created = false;
@@ -3596,6 +3892,7 @@ int send_files_multithreaded(Config** config_ptr) {
     if (scanner_created)
       thrd_join(scanner, NULL);
     pipeline_context_sender_destroy(context);
+    client_progress_cleanup();
     return 1;
   }
 
@@ -3615,6 +3912,7 @@ int send_files_multithreaded(Config** config_ptr) {
      transfer (exit 23).  A --max-delete-capped commit is a successful transfer
      that rsync reports with exit code 25. */
   pipeline_context_sender_destroy(context);
+  client_progress_cleanup();
   client_set_abort_armed(false);
   if (!sender_ok)
     return 1;

tests/integration/parity_harness.py

@@ -54,6 +54,7 @@ STDOUT_NONE = None
 STDOUT_ITEMIZE = "itemize"
 STDOUT_OUTFMT = "outfmt"
 STDOUT_STATS = "stats"
+STDOUT_PROGRESS = "progress"
 
 # rsync --stats lines that are protocol-independent and must match exactly.
 # `Number of files` and `Number of created files` carry rsync's per-type
@@ -73,6 +74,8 @@ STATS_KEYS = (
 )
 
 _ITEMIZE_RE = re.compile(r"^(<|>|c|h|\.|\*)[fdLDS][.+\-][.+\-][.+\-][.+\-]")
+_PROGRESS_TOTAL_RE = re.compile(r"to-chk=\d+/(\d+)")
+_PROGRESS_XFR_RE = re.compile(r"xfr#(\d+)")
 
 
 @dataclass
@@ -184,6 +187,24 @@ def corpus_empty_dir(root: str) -> None:
     _write(os.path.join(root, "nonempty", "f.txt"), b"f\n")
 
 
+def corpus_multidir(root: str) -> None:
+    """Multi-directory tree for the --progress file-list naming/denominator.
+
+    Nested files, a directory-only branch, an empty directory and a symlink
+    exercise every file-list entry type rsync counts in `to-chk` but FastSync's
+    streaming scanner never emits as a transfer entry.
+    """
+    clean_dir(root)
+    _write(os.path.join(root, "a.txt"), b"alpha\n")
+    _write(os.path.join(root, "b.txt"), b"bravo\n")
+    _write(os.path.join(root, "sub1", "c.txt"), b"charlie\n")
+    _write(os.path.join(root, "sub1", "deep", "d.txt"), b"delta\n")
+    _write(os.path.join(root, "sub2", "e.txt"), b"echo\n")
+    os.symlink("a.txt", os.path.join(root, "link1"))
+    os.makedirs(os.path.join(root, "emptydir"), exist_ok=True)
+    os.utime(os.path.join(root, "emptydir"), (_SRC_MTIME, _SRC_MTIME))
+
+
 def corpus_relative(root: str) -> None:
     """Tree for the -R/--files-from cases."""
     clean_dir(root)
@@ -216,6 +237,7 @@ CORPORA: Dict[str, Callable[[str], None]] = {
     "sparse": corpus_sparse,
     "filters": corpus_filters,
     "empty_dir": corpus_empty_dir,
+    "multidir": corpus_multidir,
     "relative": corpus_relative,
     "iconv": corpus_iconv,
 }
@@ -382,6 +404,37 @@ def normalize_stdout(text: str, mode: Optional[str]) -> object:
                 if line.startswith(key + ":"):
                     found[key] = _parse_bytes(line.split(":", 1)[1])
         return found
+    if mode == STDOUT_PROGRESS:
+        # rsync prints the file-list entries in sorted depth-first order while
+        # FastSync's streaming scan emits them in readdir/BFS order; only the
+        # entry set and deterministic fields are compared.  The transfer-root
+        # `./` line's trigger condition is a separate documented residual, and
+        # the per-frame rate/elapsed/xfr#/to-chk numerator are wall-clock- or
+        # order-dependent, so only the `to-chk` denominator and the name set are
+        # asserted.
+        names = []
+        totals = set()
+        max_xfr = 0
+        for line in (text or "").splitlines():
+            line = line.rstrip()
+            if not line:
+                continue
+            if "%" in line:
+                m = _PROGRESS_TOTAL_RE.search(line)
+                if m:
+                    totals.add(int(m.group(1)))
+                mx = _PROGRESS_XFR_RE.search(line)
+                if mx:
+                    max_xfr = max(max_xfr, int(mx.group(1)))
+                continue
+            if line == "sending incremental file list":
+                continue
+            if line.startswith("created directory "):
+                continue
+            if line == "./":
+                continue
+            names.append(line)
+        return {"names": sorted(names), "total": sorted(totals), "xfr": max_xfr}
     # raw
     return sorted(l.rstrip() for l in (text or "").splitlines() if l.strip())
 
@@ -393,6 +446,8 @@ def stdout_diff(rsync_out: str, fs_out: str, mode: Optional[str]) -> List[str]:
         return []
     if mode == STDOUT_STATS:
         return [f"stats rsync={r}", f"stats fastsync={f}"]
+    if mode == STDOUT_PROGRESS:
+        return [f"progress rsync={r}", f"progress fastsync={f}"]
     return list(difflib.unified_diff(
         [str(x) for x in r], [str(x) for x in f],
         fromfile="rsync", tofile="fastsync", lineterm=""))

tests/integration/test_differential_parity.py

@@ -173,6 +173,12 @@ _CASES = [
            stdout=H.STDOUT_OUTFMT, ref="--out-format %n %l"),
     H.Case("out_format_i_n", "basic", ["-a", "--out-format=%i %n"],
            stdout=H.STDOUT_OUTFMT, ref="--out-format %i %n"),
+    H.Case("progress", "multidir", ["-a", "--progress"], stdout=H.STDOUT_PROGRESS,
+           ci=True, ref="--progress multi-directory file list"),
+    H.Case("progress_threads", "multidir", ["-a", "--progress"],
+           fastsync_flags=["-a", "--progress", "--threads"],
+           stdout=H.STDOUT_PROGRESS, ci=True,
+           ref="--progress multi-directory file list (--threads)"),
 
     # --- transfer modifications -------------------------------------------
     H.Case("update", "basic", ["-a", "--update"], seed=seed_update,

tests/integration/test_output_parity.py

@@ -289,6 +289,40 @@ def _make_one_file(root, name="f.bin", size=100):
         fh.write(bytes((i * 7 + 3) & 0xFF for i in range(size)))
 
 
+def _make_multidir_tree(root):
+    """Multi-directory corpus for the --progress file-list tests: nested files,
+    a directory-only branch, an empty directory and a symlink."""
+    clean_dir(root)
+    for rel, data in (("a.txt", b"alpha\n"), ("b.txt", b"bravo\n"),
+                      ("sub1/c.txt", b"charlie\n"), ("sub1/deep/d.txt", b"delta\n"),
+                      ("sub2/e.txt", b"echo\n")):
+        path = os.path.join(root, rel)
+        os.makedirs(os.path.dirname(path), exist_ok=True)
+        with open(path, "wb") as fh:
+            fh.write(data)
+    os.symlink("a.txt", os.path.join(root, "link1"))
+    os.makedirs(os.path.join(root, "emptydir"), exist_ok=True)
+
+
+def _parse_progress(text):
+    """Name lines and the `to-chk` denominators from a --progress run."""
+    names = []
+    totals = set()
+    for line in text.splitlines():
+        line = line.rstrip()
+        if not line or line == "sending incremental file list":
+            continue
+        if "%" in line:
+            match = re.search(r"to-chk=\d+/(\d+)", line)
+            if match:
+                totals.add(int(match.group(1)))
+            continue
+        if line == "./":  # root-line trigger is a separate documented residual
+            continue
+        names.append(line)
+    return sorted(names), totals
+
+
 def _pick_stats(text, keys):
     out = {}
     for line in text.splitlines():
@@ -493,6 +527,62 @@ class TestWireStatsParity:
         assert "to-chk=0/2" in fast_lines[-1], fast_lines[-1]
         assert fast_lines[-1] == rsync_lines[-1], (rsync_lines[-1], fast_lines[-1])
 
+    @requires_rsync
+    @pytest.mark.ci
+    @pytest.mark.parametrize("mt", [False, True])
+    def test_progress_multidir_file_list_matches_rsync(self, shared_server, mt):
+        """A multi-directory tree: the paths-only pre-count must reproduce
+        rsync's file-list set and `to-chk` denominator.  Per-directory name
+        lines are emitted for directories, symlinks and the empty directory; the
+        name set and the denominator (every entry plus the transfer root) match
+        rsync, while the emitted *order* remains a documented residual (rsync
+        sorts depth-first, FastSync streams in readdir/BFS order)."""
+        source = os.path.join(TEST_DATA_DIR, "wire_pgmd_src")
+        dest = os.path.join(TEST_DATA_DIR, "wire_pgmd_dst")
+        rdst = os.path.join(TEST_DATA_DIR, "wire_pgmd_rdst")
+        _make_multidir_tree(source)
+        clean_dir(dest)
+        clean_dir(rdst)
+
+        rsync_result = _rsync(["-a", "--progress", source + "/", rdst + "/"])
+        assert rsync_result.returncode == 0, rsync_result.stderr
+        flags = ["-a", "--progress"] + (["--threads"] if mt else [])
+        result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
+        assert result.returncode == 0, result.stderr[:300]
+
+        rsync_names, rsync_totals = _parse_progress(rsync_result.stdout)
+        fast_names, fast_totals = _parse_progress(result.stdout)
+        assert sorted(rsync_names) == [
+            "a.txt", "b.txt", "emptydir/", "link1 -> a.txt", "sub1/",
+            "sub1/c.txt", "sub1/deep/", "sub1/deep/d.txt", "sub2/", "sub2/e.txt",
+        ], rsync_names
+        assert fast_names == rsync_names, (rsync_names, fast_names)
+        # 10 entries + the transfer-root "." counted by rsync's file list.
+        assert rsync_totals == {11}, rsync_totals
+        assert fast_totals == rsync_totals, (rsync_totals, fast_totals)
+
+    @pytest.mark.ci
+    def test_progress_delete_during_reuses_pre_scan(self):
+        """--delete-during + --progress reuses the keep-set pre-scan instead of
+        walking the tree a second time: the file-list total and directory name
+        lines are identical to a plain --progress run."""
+        source = os.path.join(TEST_DATA_DIR, "wire_pgdel_src")
+        dest = os.path.join(TEST_DATA_DIR, "wire_pgdel_dst")
+        _make_multidir_tree(source)
+        clean_dir(dest)
+        server = ServerManager()
+        server.start(extra_args=["--allow-super", "--allow-delete"])
+        try:
+            result, _ = run_client(source, dest, flags=["-a", "--progress", "--delete-during"],
+                                   port=server.port)
+        finally:
+            server.stop()
+        assert result.returncode == 0, result.stderr[:300]
+        names, totals = _parse_progress(result.stdout)
+        assert totals == {11}, totals
+        assert "sub1/" in names and "sub1/deep/" in names and "emptydir/" in names, names
+        assert "link1 -> a.txt" in names, names
+
     @requires_rsync
     @pytest.mark.ci
     @pytest.mark.parametrize("mt", [False, True])

tests/test_scanner.c

@@ -1240,6 +1240,55 @@ static int collect_scan_info_parallel(ParallelScanner* scanner, const char* root
   return failed ? -1 : count;
 }
 
+/* The --progress paths-only pre-count relies on `list_dirs` emitting every
+ * directory (including empty ones) exactly once, alongside the files and
+ * symlinks the streaming scanner already emits. */
+static void test_scanner_list_dirs_counts_every_entry() {
+  const char* root = "test_scan_listdirs";
+  EXPECT_EQ_INT(mkdir(root, 0755), 0);
+  EXPECT_EQ_INT(mkdir("test_scan_listdirs/sub1", 0755), 0);
+  EXPECT_EQ_INT(mkdir("test_scan_listdirs/sub1/deep", 0755), 0);
+  EXPECT_EQ_INT(mkdir("test_scan_listdirs/sub2", 0755), 0);
+  EXPECT_EQ_INT(mkdir("test_scan_listdirs/emptydir", 0755), 0);
+  create_test_file("test_scan_listdirs/a.txt", "a");
+  create_test_file("test_scan_listdirs/sub1/c.txt", "c");
+  create_test_file("test_scan_listdirs/sub1/deep/d.txt", "d");
+  create_test_file("test_scan_listdirs/sub2/e.txt", "e");
+  EXPECT_EQ_INT(symlink("a.txt", "test_scan_listdirs/link1"), 0);
+
+  ScannerOptions options = {0};
+  options.list_dirs = true;
+  options.emit_empty_dirs = true;
+  options.follow_symlinks = true; /* -l/--links: carry symlinks, don't skip */
+  ScanInfo infos[16];
+  int count = collect_scan_info(root, &options, infos, 16);
+  EXPECT_EQ_INT(count, 9);
+  int dirs = 0;
+  for (int i = 0; i < count; i++) {
+    if (infos[i].is_dir)
+      dirs++;
+  }
+  EXPECT_EQ_INT(dirs, 4);
+  EXPECT_TRUE(scan_info_present(infos, count, "sub1", true, NULL));
+  EXPECT_TRUE(scan_info_present(infos, count, "sub1/deep", true, NULL));
+  EXPECT_TRUE(scan_info_present(infos, count, "sub2", true, NULL));
+  EXPECT_TRUE(scan_info_present(infos, count, "emptydir", true, NULL));
+  EXPECT_TRUE(scan_info_present(infos, count, "a.txt", false, ""));
+  EXPECT_TRUE(scan_info_present(infos, count, "sub1/c.txt", false, ""));
+  EXPECT_TRUE(scan_info_present(infos, count, "link1", false, ""));
+
+  unlink("test_scan_listdirs/a.txt");
+  unlink("test_scan_listdirs/sub1/c.txt");
+  unlink("test_scan_listdirs/sub1/deep/d.txt");
+  unlink("test_scan_listdirs/sub2/e.txt");
+  unlink("test_scan_listdirs/link1");
+  rmdir("test_scan_listdirs/sub1/deep");
+  rmdir("test_scan_listdirs/sub1");
+  rmdir("test_scan_listdirs/sub2");
+  rmdir("test_scan_listdirs/emptydir");
+  rmdir(root);
+}
+
 /* -d without --files-from emits exactly the source-root directory (empty) and
  * never descends. */
 static void test_dirs_no_descent() {
@@ -1667,6 +1716,7 @@ void test_scanner() {
   test_per_dir_filter_override(true);
   test_dirs_no_descent();
   test_dirs_files_from();
+  test_scanner_list_dirs_counts_every_entry();
   test_files_from_relative_send_path();
   test_scanner_captures_directory_times();
   test_scanner_chunk_ownership();