fix(fs): recreate empty dirs, replace blocking file; -R --no-implied-dirs implied parents
- Recursive scans emit a directory entry for every traversed directory that produced no transferred child, so empty source dirs (and dirs emptied by filtering) are recreated like rsync; -m prunes them, --files-from/--list-only never emit implicit dirs. - A directory entry now replaces a destination regular file (rsync removes the non-directory) instead of aborting; confined to the secure parent fd. - -R --no-implied-dirs --files-from: stop refusing a listed file whose parent is not listed; create the implied parent with default attributes (no source metadata is captured for it), matching rsync 3.4.1. - Differential gate: drop min_size/empty_dirs_recursive/dirs_plain allowlist entries (dirs_plain now hands FastSync the same trailing-slash source as rsync); add differential test for the files-from implied parent. - Docs: -d row -> Parity, tally 108/24/24. No wire change (PROTOCOL_VERSION stays 2.26.0).
This commit is contained in:
+9
-9
@@ -6,8 +6,8 @@ This document maps rsync's full feature set to FastSync's current implementation
|
||||
|
||||
| Status | Count | Description |
|
||||
|--------|-------|-------------|
|
||||
| ✅ Parity | 107 | Reproduces rsync's semantics for this option's scope |
|
||||
| ⚠️ Caveat | 25 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
|
||||
| ✅ Parity | 108 | Reproduces rsync's semantics for this option's scope |
|
||||
| ⚠️ Caveat | 24 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
|
||||
| ❌ Divergent | 24 | Rejected, an accepted no-op, deliberately non-rsync (native config/auth/batch, privileged namespaces, safe-subset privilege), or impossible on any portable filesystem call |
|
||||
| **Total** | **156** | One row per rsync option/feature group; a row may name several spellings |
|
||||
|
||||
@@ -101,8 +101,8 @@ Every one of those has an entry below with its remaining caveats.
|
||||
|------|-------------------|-----------------|-------|
|
||||
| `-r`, `--recursive` | Recurse into directories | ✅ Parity | Default behavior |
|
||||
| `-R`, `--relative` | Use relative path names | ✅ Parity | Protocol 2.26.0 implements rsync's general `-R` path semantics: without a cut the source argument is mirrored in full below the destination root; a `/./` cut in the source argument (`src/./foo`) makes everything after the cut the destination prefix, so the layout matches rsync's relative reconstruction; and `--files-from` entries land under their bare relative path. The delete manifest derives from the sent (relative) paths and is scoped to the transferred prefix subtree, so `--delete` cannot remove destination content outside that prefix (a blocker fix). Works single-threaded and under `-j`/`--threads` |
|
||||
| `--no-implied-dirs` | Don't send implied dirs with -R | ✅ Parity | With `-R`, rsync creates the ancestor directories implied by a listed path and, with `--no-implied-dirs`, omits them from the transfer so the destination directories keep the destination's own mode/mtime. Protocol 2.26.0 matches this: the implied-dir walk applies the transfer's per-attribute metadata only to explicitly transferred directories, and a differential test verifies the modes and mtimes of the implied parents against rsync with and without the flag. Works single-threaded and under `-j`/`--threads` |
|
||||
| `-d`, `--dirs`, `--old-dirs`, `--old-d` | Transfer dirs without recursing | ⚠️ Caveat | Protocol 2.26.0 implements rsync's one-level `-d` listing for `dir`, `dir/` and `.`: the source's immediate contents are transferred (files with content, directories as explicit entries), matching rsync's destination tree in a differential test. `--dirs --files-from` transfers exactly the listed items — a listed directory is created empty and a listed file with content — under the same `-R` layout rules. Directory entries cross as `STATUS_MKDIR` and appear in the delete manifest, so `--delete` prunes correctly and an empty listed directory survives. Directory times are applied at the end of the transfer; modes/ownership follow the per-attribute policy. **Remaining divergence:** a plain recursive `-a` scan still does not create empty source directories (directory entries are record-only unless `-d`/`--files-from` explicitly lists a directory), and under `--delay-updates` directories are created immediately while only regular files are staged (see the recursive-empty-directory residual in the completion-wave section) |
|
||||
| `--no-implied-dirs` | Don't send implied dirs with -R | ✅ Parity | With `-R`, rsync creates the ancestor directories implied by a listed path and, with `--no-implied-dirs`, omits their attributes from the transfer so they keep the destination's own state (or are created with default attributes when absent). Protocol 2.26.0 matches this: without `--files-from` the implied-dir walk applies per-attribute metadata only to explicitly transferred directories, and with `-R --files-from` a listed file whose parent is not itself listed is placed normally — the missing implied parent is created with default attributes (not the source's) and the file transfers with `rc 0`, exactly like rsync 3.4.1 (a differential test verifies the modes and mtimes with and without the flag). Works single-threaded and under `-j`/`--threads` |
|
||||
| `-d`, `--dirs`, `--old-dirs`, `--old-d` | Transfer dirs without recursing | ✅ Parity | Protocol 2.26.0 implements rsync's one-level `-d` listing for `dir`, `dir/` and `.`: the source's immediate contents are transferred (files with content, directories as explicit entries), matching rsync's destination tree in a differential test. `--dirs --files-from` transfers exactly the listed items — a listed directory is created empty and a listed file with content — under the same `-R` layout rules. A plain recursive scan also recreates empty source directories now: the scanner emits a payload-less directory entry (with metadata) for every traversed directory that produced no transferred or descended child, unless `-m/--prune-empty-dirs` suppresses it or the run is `--files-from`/`--list-only` (a directory emptied by filtering is recreated too, matching rsync). Directory entries cross as `STATUS_MKDIR` and appear in the delete manifest, so `--delete` prunes correctly and an empty listed directory survives; an incoming directory replaces a destination regular file (rsync removes the non-directory and creates the directory), verified differentially. Directory times are applied at the end of the transfer; modes/ownership follow the per-attribute policy. Under `--delay-updates` directories are created immediately while only regular files are staged, exactly as rsync does |
|
||||
| `--mkpath` | Create missing path components | ✅ Parity | Wire option (client → server). At connection start the server creates the client's destination root directory (and any missing leading components below its own authorized root) when `--mkpath` is set, failing the connection cleanly if it cannot. Without `--mkpath` a destination root that does not exist yet is rejected up front (rsync semantics), so the flag is the only way to transfer into a not-yet-created destination directory. Creation is confined by the same secure mkdir walk as file writes (`O_NOFOLLOW`, no `..`) |
|
||||
| `--inc-recursive`, `--no-inc-recursive` | Incremental recursion mode | ❌ Divergent | rsync's man-page-only scanning-mode switch (and its short aliases). FastSync always performs a single full recursive scan, so both spellings are rejected as unknown options rather than accepted as a no-op; there is no incremental-recursion engine to toggle. A genuine implementation would be a scan-architecture change with no benefit for FastSync's push model |
|
||||
|
||||
@@ -144,7 +144,7 @@ Every one of those has an entry below with its remaining caveats.
|
||||
| `--max-delete=NUM` | Max files to delete | ✅ Parity | `max_delete` config field (default -1 = no client limit; 0 = delete nothing). **Protocol 2.23.0 matches rsync's partial semantics:** the receiver deletes up to NUM entries (regular files, symlinks and empty directories; each directory removal counts as one) and then **stops deleting, skips the rest, and reports the run as partial**. The client prints a "deletions stopped due to `--max-delete` limit" message and exits **25** (rsync's `RERR_PARTIAL`), not a hard failure — the transfer itself succeeded. NUM only applies together with `--delete` (it is inert otherwise, matching rsync). A client NUM below the server hard bound `MAX_SERVER_DELETE_COUNT` (100000) replaces it; a NUM above it never raises that cap. Deleting an entire destination with no limit is still bounded by the server's 100000-entry ceiling. `--delete-missing-args` exact-path deletions and the ordinary extras walk draw from the same budget, matching rsync |
|
||||
| `--ignore-errors` | Delete even with I/O errors | ⚠️ Caveat | Sender-side, client-only config field. rsync suppresses `--delete` when the transfer had I/O errors; FastSync's equivalent is a source-scan I/O error (an unreadable directory, e.g. EACCES). By default the scan aborts the run so no deletion happens. With `--ignore-errors` the scan continues past the unreadable directory, the readable tree is transferred, deletion still runs (the unreadable directory's mirror is treated as an extra), and the run exits 23 (`RERR_PARTIAL`), matching rsync. **Remaining divergence / uncertainty:** the EACCES differential is not exercised in CI because the runner is root (mode 000 is still readable), so this rests on source inspection plus the setpriv integration test |
|
||||
| `--force` | Force deletion of non-empty dirs | ✅ Parity | `force_delete` receiver config field (crosses the wire). rsync's `--force` lets an incoming non-directory replace a destination directory; FastSync implements exactly that: when a regular file (or symlink) is written to a path that is currently a (possibly non-empty) destination directory, `--force` removes that directory tree first — confined to the receive root and symlink-safe (O_NOFOLLOW fd walk, symlinks removed by name, never followed) — so the install can place the file. **Protocol 2.23.0 honors `--force` on the `--delay-updates` publication path too**, not only the immediate-install path. Without `--force` such a write fails and the run aborts. Gated by the server `--allow-delete` policy (a client cannot use `--force` to remove a destination tree on a server that forbids deletion) |
|
||||
| `-m`, `--prune-empty-dirs` | Prune empty dir chains | ✅ Parity | `-m`/`--prune-empty-dirs` (Phase 7 Wave A freed the rsync short `-m`; FastSync multithreading is now `-j`/`--threads`). FastSync's recursive transfer records directory times but never CREATES an empty directory (a `STATUS_DIR_TIMES` entry is record-only, and `--dirs` empty entries are pruned by this flag), so empty directories are inherently never transferred (which is rsync's `-m` behavior) and truly-empty destination directory chains are removed by `--delete` regardless of this flag. The flag's additional real effect is on the `--dirs` explicit directory-entry generator: a plain `-d <empty-dir>` run omits the empty source directory's entry, so nothing is created at the destination (no `STATUS_MKDIR`, no `-i`/`--out-format` change line, and an existing empty mirror becomes an extra that `--delete` prunes). Explicitly `--files-from`-listed directories always pass through (documented `--files-from` behavior). A directory that still holds an excluded-but-protected file survives, matching the `--delete-excluded` default |
|
||||
| `-m`, `--prune-empty-dirs` | Prune empty dir chains | ✅ Parity | `-m`/`--prune-empty-dirs` (Phase 7 Wave A freed the rsync short `-m`; FastSync multithreading is now `-j`/`--threads`). A recursive transfer now recreates empty source directories by default (rsync parity); this flag suppresses that emission, so an empty directory (physically empty, or emptied by filtering) is not created and a true empty-directory chain is removed by `--delete`, matching rsync's `-m`. It also affects the `--dirs` explicit directory-entry generator: a plain `-d <empty-dir>` run omits the empty source directory's entry, so nothing is created at the destination (no `STATUS_MKDIR`, no `-i`/`--out-format` change line, and an existing empty mirror becomes an extra that `--delete` prunes). Explicitly `--files-from`-listed directories always pass through (documented `--files-from` behavior; `--files-from` runs never emit implicit empty directories). A directory that still holds an excluded-but-protected file survives, matching the `--delete-excluded` default |
|
||||
|
||||
**Deletion-timing implementation notes (Phase 3):** the delete flags above are
|
||||
real. Two new config booleans (`delete_during`, `delete_delay`) join the already
|
||||
@@ -879,7 +879,7 @@ These are the last compatibility items and the closing phase toward rsync flag p
|
||||
|
||||
**Wave D — Times superstructure & arg-protection no-ops (✅ implemented, `--secluded-args` ❌).** `-O`/`--omit-dir-times` and `-J`/`--omit-link-times` are now **real modifiers** (both `🔄 → ✅ Implemented`), reversing the old "never preserves directory/symlink times" divergence:
|
||||
|
||||
- **Directory times.** The recursive scanner captures every traversed source directory's metadata (mtime, plus atime under `-U`) into a per-transfer list — two paths are covered: the sequential `DirectoryScanner` captures each opened directory (including the transfer root), and the parallel scanner captures both the root in `parallel_scanner_create_with_options` and each worker's subdirectories in `open_next_directory` (appends are guarded by a mutex shared with the sender's pipeline context). The sender transmits them in trailing `STATUS_DIR_TIMES` frames (each: int count + count × (wire path, metadata) pairs) sent **after all file data and after the optional delete manifest**, just before `STATUS_FINISHED`. A tree larger than `MAX_MANIFEST_ENTRIES` (1 048 576) directories is chunked into repeated frames, each within the receiver's per-frame bound. A dir-time entry is RECORD-ONLY (`file->dir_time_only`): `file_save_to_disk_full` returns `FILE_SAVE_SKIPPED` without creating anything, so a source directory that was empty (or pruned by `-m/--prune-empty-dirs`) is never resurrected. The receiver accumulates received directory metadata in a `DirTimeList` and applies it only at the very end — after the entire stream, after the commit-style `--delete` deletion, and after `--delay-updates` publication — because creating or removing a child bumps the parent's mtime. Application is fd-relative/walk-confined (`file_open_secure_parent` + `utimensat(..., AT_SYMLINK_NOFOLLOW)`) and best-effort per entry: an absent path (an intentionally uncreated empty dir) is skipped QUIETLY and only a real existing directory is stamped. `-O` (config boolean, already on the wire) makes the receiver skip the whole set. The single-threaded sink applies in `receiver_send_success_frame`; the `-j`/`--threads` sink accumulates in `write_thread` and server.c applies after both threads join and the deletion commits.
|
||||
- **Directory times.** The recursive scanner captures every traversed source directory's metadata (mtime, plus atime under `-U`) into a per-transfer list — two paths are covered: the sequential `DirectoryScanner` captures each opened directory (including the transfer root), and the parallel scanner captures both the root in `parallel_scanner_create_with_options` and each worker's subdirectories in `open_next_directory` (appends are guarded by a mutex shared with the sender's pipeline context). The sender transmits them in trailing `STATUS_DIR_TIMES` frames (each: int count + count × (wire path, metadata) pairs) sent **after all file data and after the optional delete manifest**, just before `STATUS_FINISHED`. A tree larger than `MAX_MANIFEST_ENTRIES` (1 048 576) directories is chunked into repeated frames, each within the receiver's per-frame bound. A dir-time entry is RECORD-ONLY (`file->dir_time_only`): `file_save_to_disk_full` returns `FILE_SAVE_SKIPPED` without creating anything (the directory's creation, when it is empty, is now carried by a separate `STATUS_MKDIR` entry the scanner emits for every directory that produced no transferred child, and `-m/--prune-empty-dirs` suppresses that). The receiver accumulates received directory metadata in a `DirTimeList` and applies it only at the very end — after the entire stream, after the commit-style `--delete` deletion, and after `--delay-updates` publication — because creating or removing a child bumps the parent's mtime. Application is fd-relative/walk-confined (`file_open_secure_parent` + `utimensat(..., AT_SYMLINK_NOFOLLOW)`) and best-effort per entry: an absent path (an intentionally uncreated empty dir) is skipped QUIETLY and only a real existing directory is stamped. `-O` (config boolean, already on the wire) makes the receiver skip the whole set. The single-threaded sink applies in `receiver_send_success_frame`; the `-j`/`--threads` sink accumulates in `write_thread` and server.c applies after both threads join and the deletion commits.
|
||||
- **Symlink times/owner/mode.** `STATUS_SYMLINK` already carried metadata; the receiver now applies it with no-follow primitives only: `utimensat(..., AT_SYMLINK_NOFOLLOW)`, best-effort `fchmodat(..., AT_SYMLINK_NOFOLLOW)` (honest no-op where unsupported, e.g. Linux), and policy-gated `fchownat(..., AT_SYMLINK_NOFOLLOW)` via a new `identity_apply_ownership_link` that shares the identity resolver with the fd path. `-J` suppresses only the timestamps; ownership stays governed by the identity opt-in (`--numeric-ids`/`--usermap`/`--groupmap`/`--chown`) exactly like regular files. A symlink has no children, so this is applied immediately at creation.
|
||||
- **Wire:** the shared `STATUS_DIR_TIMES` frame (and metadata on `STATUS_MKDIR` for `--dirs` entries) is a frame-sequence change, so `PROTOCOL_VERSION` was bumped **2.16.0 → 2.17.0**; every version-sensitive test (`--protocol` accepted/rejected values) was updated. The config-frame layout itself is unchanged (the omit booleans already crossed). Non-metadata and `--no-preserve` transfers send no `STATUS_DIR_TIMES` frame and no directory metadata, keeping them byte-identical.
|
||||
|
||||
@@ -893,7 +893,7 @@ These are the last compatibility items and the closing phase toward rsync flag p
|
||||
|
||||
**Wire:** two trailing config-frame blocks after the `--iconv` spec, in fixed order — `send_privilege_options`/`receive_privilege_options` (one `super_mode` int, validated `0..2`), then `send_copy_as_options`/`receive_copy_as_options` (presence int + two int32 ids, validated `>= 0`, with `copy_as_set ⇒ use_metadata`). `PROTOCOL_VERSION` bumped **2.17.0 → 2.18.0**. **Divergences from rsync:** rsync's `--super` elevates the receiver and `--copy-as` actually switches its credentials; FastSync never elevates and only permits/forwards confined attempts, and `--copy-as` forces ownership rather than switching identity.
|
||||
|
||||
**Honest status after the parity-completion wave (protocol 2.26.0), updated by the rsync-parity-stats pass.** ✅ Parity 107 / ⚠️ Caveat 25 / ❌ Divergent 24 = 156 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. This reclassification makes every difference explicit. The completion wave closed 23 previously-caveated rows (9 that triage showed were already parity, plus 14 genuine fixes) and turned the 17 inherently non-rsync rows — native daemon config/auth, the FastSync batch container, the safe-subset device/privilege flags, `-X`'s privileged namespaces, `--fake-super`'s native xattr format, and the `--old-args` no-op — into explicit ❌ divergences. The stats pass flipped `--delete-delay` to ✅ (actual-removal accounting) and reclassified `--out-format` to ❌ (protocol-specific `%b`/delta-`%c`), and sharpened the `--stats`/`--progress`/`--checksum-choice` residuals. The remaining ⚠️ rows are the ones with a documented residual (see the row notes and the **Parity Completion Wave (protocol 2.26.0)** section below).
|
||||
**Honest status after the parity-completion wave (protocol 2.26.0), updated by the rsync-parity-stats and rsync-parity-fs passes.** ✅ Parity 108 / ⚠️ Caveat 24 / ❌ Divergent 24 = 156 rows. Earlier revisions of this document reported "143 ✅ / 0 divergence / 0 partial"; that conflated "parsed and tested" with "rsync parity", because many rows carried documented behavioral differences and some short options were not parsed at all. This reclassification makes every difference explicit. The completion wave closed 23 previously-caveated rows (9 that triage showed were already parity, plus 14 genuine fixes) and turned the 17 inherently non-rsync rows — native daemon config/auth, the FastSync batch container, the safe-subset device/privilege flags, `-X`'s privileged namespaces, `--fake-super`'s native xattr format, and the `--old-args` no-op — into explicit ❌ divergences. The stats pass flipped `--delete-delay` to ✅ (actual-removal accounting) and reclassified `--out-format` to ❌ (protocol-specific `%b`/delta-`%c`), and sharpened the `--stats`/`--progress`/`--checksum-choice` residuals. The fs pass flipped `-d/--dirs` to ✅: recursive transfers now recreate empty source directories (and replace a blocking destination non-directory with an incoming directory), and `-R --no-implied-dirs --files-from` places a listed file under a missing implied parent with default attributes instead of refusing. The remaining ⚠️ rows are the ones with a documented residual (see the row notes and the **Parity Completion Wave (protocol 2.26.0)** section below).
|
||||
|
||||
**Preserve-attribute split (protocol 2.21.0 → 2.22.0) — ✅ implemented.** FastSync splits the former single metadata bundle into four independent, rsync-compatible per-attribute flags — `-p/--perms`, `-t/--times`, `-o/--owner`, `-g/--group` — each with a negation (`--no-perms`/`--no-times`/`--no-owner`/`--no-group`, short `--no-p`/`--no-t`/`--no-o`/`--no-g`), plus `--no-preserve` clearing all four. `-a/--archive` is now full rsync `-rlptgoD` (owner and group included, though their application stays privilege-gated), `-A/--acls` implies `-p`, `-X/--xattrs` does not, `-E/--executability` sets only executability, and `-U`/`-N` do not imply `-t`. `--incremental`/`--delta` still auto-preserve perms+times unless the user explicitly negated them. Wire: the binary config frame gains four appended booleans (`preserve_perms`/`preserve_times`/`preserve_owner`/`preserve_group`) after `omit_link_times`, so `PROTOCOL_VERSION` is bumped **2.21.0 → 2.22.0**; the fixed-width `FileMetadata` layout is unchanged and the receiver gates the metadata frame on a derived `use_metadata`. Receiver behavior: each attribute is applied independently, directory modes are applied under `-p` (at the end of the transfer, alongside dir times), symlink mode under `-p`, and `-O/--omit-dir-times` suppresses directory times only. Documented divergences as of 2.22.0, **all but (d)/(e) removed by the rsync-parity wave (protocol 2.23.0)**: (a) the mode-masking divergence is **gone** — under `-p` the source mode is now copied exactly, including `S_IWGRP`/`S_IWOTH` and setuid/setgid/sticky; (b) a brand-new file without `-p` still gets `source_mode & ~umask` when metadata is present (else the historical fixed `0644`), and a new *directory* without `-p` still uses FastSync's `0755` default; (c) the `--chmod`-implies-`-p` divergence is **gone** — `--chmod` no longer implies `-p` (rsync parity); (d) `-o`/`-g` map by name on the receiver with a raw-numeric fallback (only numeric ids cross the wire); (e) a daemon module without `client owner = yes` does not refuse a plain `-a`/`-o`/`-g` — it forces super off, applies no ownership, and logs a warning, while explicit `--chown`/`--usermap`/`--groupmap`/`--numeric-ids`/`--copy-as`/`--super` are still refused.
|
||||
|
||||
@@ -1145,8 +1145,8 @@ These remain after the wave; the individual rows carry the precise wording.
|
||||
half-swap/charset-declaration caveat.
|
||||
- **Basis dirs** do not re-apply attributes on a match, keep the
|
||||
`--size-only` mtime caveat, and share the 256 MiB whole-file cap; **`--fuzzy`**
|
||||
has a different tie-break order; recursive transfers still do not create empty
|
||||
directories; and **`--bwlimit`** rejects rsync's `0`/decimal/suffixed rates.
|
||||
has a different tie-break order; and **`--bwlimit`** rejects rsync's
|
||||
`0`/decimal/suffixed rates.
|
||||
- **`--inc-recursive`/`--no-inc-recursive`** are not implemented (rejected).
|
||||
|
||||
### Intentional divergences (explicit ❌ rows)
|
||||
|
||||
@@ -412,6 +412,10 @@ static bool prepare_scanner(const Config* config, int num_threads, PreparedScann
|
||||
options->exclude_per_dir_filter_files = config->per_dir_filter_count >= 2;
|
||||
options->dirs = config->dirs;
|
||||
options->relative = config->relative;
|
||||
/* --no-implied-dirs only has meaning with -R (rsync): without it the option
|
||||
is a documented no-op, so the scanner must not suppress directory
|
||||
metadata. */
|
||||
options->no_implied_dirs = config->no_implied_dirs && config->relative;
|
||||
/* -R/--relative outside --files-from reconstructs every destination path from
|
||||
* the source spec (rsync's '/./' cut point). With --files-from the listed
|
||||
* entry already supplies the bare relative path, so no prefix is built. */
|
||||
@@ -572,72 +576,6 @@ static const char* delete_plan_walk_root(const Config* config, const ArrayList*
|
||||
return marker;
|
||||
}
|
||||
|
||||
/* True when some --files-from entry is an ancestor-or-equal directory of
|
||||
* `rel` (an empty entry -- the whole tree "." -- counts as the root). */
|
||||
static bool file_list_ancestor_listed(const FileListSet* set, const char* rel) {
|
||||
if (!set)
|
||||
return true;
|
||||
for (int i = 0; i < set->count; i++) {
|
||||
const char* listed = set->entries[i];
|
||||
if (listed[0] == '\0')
|
||||
return true;
|
||||
size_t n = strlen(listed);
|
||||
if (strncmp(rel, listed, n) == 0 && (rel[n] == '/' || rel[n] == '\0'))
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/* --no-implied-dirs (meaningful only with -R + --files-from): a listed file
|
||||
* may only be placed when its parent directory (or one of its ancestors) is
|
||||
* itself an explicitly listed entry. rsync omits a file whose implied parent
|
||||
* directory is suppressed, and an explicitly listed file that cannot be placed
|
||||
* fails the transfer; FastSync fails the whole run up front with a clear error
|
||||
* (it has no per-entry skip channel). Without -R or --files-from the option
|
||||
* has no effect. */
|
||||
static bool no_implied_dirs_files_from_valid(const Config* config) {
|
||||
if (!config->no_implied_dirs || !config->relative)
|
||||
return true;
|
||||
const FileListSet* set = (const FileListSet*)config->files_from_set;
|
||||
if (!set)
|
||||
return true;
|
||||
for (int i = 0; i < set->count; i++) {
|
||||
const char* entry = set->entries[i];
|
||||
if (entry[0] == '\0')
|
||||
continue;
|
||||
char* full = path_cat(config->send_directory, entry);
|
||||
if (!full)
|
||||
return false;
|
||||
struct stat st;
|
||||
bool is_file = lstat(full, &st) == 0 && S_ISREG(st.st_mode);
|
||||
free(full);
|
||||
if (!is_file)
|
||||
continue;
|
||||
const char* slash = strrchr(entry, '/');
|
||||
if (!slash)
|
||||
continue; /* top-level file: its parent is the receive root */
|
||||
size_t parent_len = (size_t)(slash - entry);
|
||||
if (parent_len == 0)
|
||||
continue;
|
||||
char* parent = malloc(parent_len + 1);
|
||||
if (!parent)
|
||||
return false;
|
||||
memcpy(parent, entry, parent_len);
|
||||
parent[parent_len] = '\0';
|
||||
bool listed = file_list_ancestor_listed(set, parent);
|
||||
if (!listed) {
|
||||
log_message(LOG_LEVEL_ERROR,
|
||||
"--no-implied-dirs: cannot place file '%s': parent directory '%s' is not "
|
||||
"explicitly listed (list the directory or drop --no-implied-dirs)",
|
||||
entry, parent);
|
||||
}
|
||||
free(parent);
|
||||
if (!listed)
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/* The destination-relative mirror path for a missing --files-from entry: where
|
||||
a PRESENT entry with the same name would have been written. With -R that is
|
||||
the entry's bare relative path (the bare wire path the receiver uses);
|
||||
@@ -751,7 +689,7 @@ static bool files_from_list_check(const Config* config, ArrayList* missing_dest,
|
||||
"--ignore-missing-args: ignored %d missing --files-from entr%s", *skipped_out,
|
||||
*skipped_out == 1 ? "y" : "ies");
|
||||
}
|
||||
return no_implied_dirs_files_from_valid(config);
|
||||
return true;
|
||||
}
|
||||
|
||||
/* Basis directories are honored by the receiver's per-file incremental check,
|
||||
|
||||
+65
-3
@@ -830,7 +830,8 @@ static bool scanner_capture_dir_time(ArrayList* dir_entries, mtx_t* mutex, const
|
||||
const char* fs_path, bool relative_mode,
|
||||
const char* relative_prefix, bool preserve_atimes,
|
||||
bool preserve_crtimes, bool preserve_xattrs,
|
||||
bool preserve_acls) {
|
||||
bool preserve_acls, bool no_implied_dirs,
|
||||
const FileListSet* file_list) {
|
||||
if (!dir_entries || !root_path || !fs_path)
|
||||
return true;
|
||||
struct stat st;
|
||||
@@ -839,6 +840,13 @@ static bool scanner_capture_dir_time(ArrayList* dir_entries, mtx_t* mutex, const
|
||||
char* rel = scanner_path_relative(root_path, fs_path);
|
||||
if (!rel)
|
||||
return true;
|
||||
/* --no-implied-dirs: an implied parent directory (not listed, and not under
|
||||
a listed directory) keeps the destination's own/default attributes, so its
|
||||
source metadata is not transmitted. */
|
||||
if (no_implied_dirs && file_list && !file_list_dir_in_scope(file_list, rel)) {
|
||||
free(rel);
|
||||
return true;
|
||||
}
|
||||
if (relative_mode && rel[0] == '\0') {
|
||||
/* -R + --files-from: the transfer root itself has no bare relative wire
|
||||
path (matches the -R scan, which never emits the root). */
|
||||
@@ -902,6 +910,43 @@ static bool scanner_capture_dir_time(ArrayList* dir_entries, mtx_t* mutex, const
|
||||
return true;
|
||||
}
|
||||
|
||||
/* Recursive scan: emit a payload-less directory entry for the directory that
|
||||
* just finished scanning. rsync creates every source directory at the
|
||||
* destination; FastSync otherwise creates one only implicitly through a
|
||||
* transferred child, so a directory emptied on the transfer side (physically
|
||||
* empty, or all of its entries filtered out) would never appear. The transfer
|
||||
* root is skipped (it maps to the receive root, which already exists), as are
|
||||
* --files-from (only listed items and their implied parents transfer),
|
||||
* --list-only (directory lines are emitted by the caller) and
|
||||
* -m/--prune-empty-dirs. Returns false on allocation failure. */
|
||||
static bool scanner_emit_empty_dir(DirectoryScanner* scanner, ArrayList* chunk_data) {
|
||||
if (!scanner->current_path || !scanner->current_rel || scanner->current_rel[0] == '\0')
|
||||
return true;
|
||||
struct stat st;
|
||||
if (stat(scanner->current_path, &st) != 0 || !S_ISDIR(st.st_mode))
|
||||
return true;
|
||||
File* dir = scanner_build_dir_file(scanner->current_path, &st, &scanner->options);
|
||||
if (!dir)
|
||||
return false;
|
||||
if (scanner->relative_mode) {
|
||||
dir->send_path = str_dup(scanner->current_rel);
|
||||
} else if (scanner->options.relative_prefix) {
|
||||
dir->send_path =
|
||||
scanner_prefix_send_path(scanner->options.relative_prefix, scanner->current_rel);
|
||||
}
|
||||
if ((scanner->relative_mode || scanner->options.relative_prefix) && !dir->send_path) {
|
||||
file_destroy(dir);
|
||||
return false;
|
||||
}
|
||||
if (scanner->options.preserve_xattrs || scanner->options.preserve_acls)
|
||||
dir->xattrs = xattr_capture_path(scanner->current_path, scanner->options.preserve_acls);
|
||||
if (!array_list_add(chunk_data, dir)) {
|
||||
file_destroy(dir);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/* Open the next queued directory and set up its filter context. Returns 1 when
|
||||
a directory is open, 0 when the queue is exhausted, and -1 on a fatal error.
|
||||
A directory that cannot be opened is an I/O error: it is recorded on the
|
||||
@@ -925,6 +970,7 @@ static int open_next_directory(DirectoryScanner* scanner) {
|
||||
* the directory that enqueued them. */
|
||||
const FilterNode* inherited = scanner->at_seed_dir ? scanner->seed_node : de->context;
|
||||
scanner->at_seed_dir = false;
|
||||
scanner->current_dir_produced = false;
|
||||
free(de);
|
||||
|
||||
free(scanner->current_rel);
|
||||
@@ -985,7 +1031,8 @@ static int open_next_directory(DirectoryScanner* scanner) {
|
||||
scanner->options.dir_entries, scanner->options.dir_entries_mutex, scanner->root_path,
|
||||
scanner->current_path, scanner->relative_mode, scanner->options.relative_prefix,
|
||||
scanner->options.preserve_atimes, scanner->options.preserve_crtimes,
|
||||
scanner->options.preserve_xattrs, scanner->options.preserve_acls)) {
|
||||
scanner->options.preserve_xattrs, scanner->options.preserve_acls,
|
||||
scanner->options.no_implied_dirs, scanner->options.file_list)) {
|
||||
closedir(scanner->current_dir);
|
||||
scanner->current_dir = NULL;
|
||||
free(scanner->current_path);
|
||||
@@ -1350,10 +1397,21 @@ Chunk* directory_scanner_next(DirectoryScanner* scanner) {
|
||||
|
||||
const struct dirent* entry = readdir(scanner->current_dir);
|
||||
if (entry == NULL) {
|
||||
/* The directory is exhausted: if nothing was transferred or descended
|
||||
from it, recreate it at the destination as an explicit entry. */
|
||||
if (!scanner->current_dir_produced && !scanner->options.prune_empty_dirs &&
|
||||
!scanner->options.list_dirs && scanner->options.file_list == NULL) {
|
||||
if (!scanner_emit_empty_dir(scanner, chunk_data))
|
||||
scanner->failed = true;
|
||||
}
|
||||
closedir(scanner->current_dir);
|
||||
scanner->current_dir = NULL;
|
||||
free(scanner->current_path);
|
||||
scanner->current_path = NULL;
|
||||
if (scanner->failed) {
|
||||
array_list_delete(chunk_data);
|
||||
return NULL;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -1488,6 +1546,7 @@ Chunk* directory_scanner_next(DirectoryScanner* scanner) {
|
||||
scanner->failed = true;
|
||||
break;
|
||||
}
|
||||
scanner->current_dir_produced = true;
|
||||
free(cur_path);
|
||||
continue;
|
||||
}
|
||||
@@ -1502,6 +1561,7 @@ Chunk* directory_scanner_next(DirectoryScanner* scanner) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
scanner->current_dir_produced = true;
|
||||
int next_depth = scanner->current_depth + 1;
|
||||
if (scanner->options.max_depth <= 0 || next_depth < scanner->options.max_depth) {
|
||||
DirEntry* de = dir_entry_create(cur_path, next_depth, scanner->current_node);
|
||||
@@ -1577,6 +1637,7 @@ Chunk* directory_scanner_next(DirectoryScanner* scanner) {
|
||||
scanner->failed = true;
|
||||
break;
|
||||
}
|
||||
scanner->current_dir_produced = true;
|
||||
chunk_data_size += file->data->size;
|
||||
if (chunk_data_size > scanner->options.chunk_size) {
|
||||
free(rel_copy);
|
||||
@@ -2206,7 +2267,8 @@ ParallelScanner* parallel_scanner_create_with_options(const char* root_directory
|
||||
root_directory, options->relative && options->file_list != NULL,
|
||||
options->relative_prefix, options->preserve_atimes,
|
||||
options->preserve_crtimes, options->preserve_xattrs,
|
||||
options->preserve_acls)) {
|
||||
options->preserve_acls, options->no_implied_dirs,
|
||||
options->file_list)) {
|
||||
array_list_delete(root_files);
|
||||
array_list_delete(subdirs);
|
||||
parallel_scanner_destroy(ps);
|
||||
|
||||
@@ -151,6 +151,11 @@ typedef struct {
|
||||
bool capture_dir_times;
|
||||
ArrayList* dir_entries;
|
||||
mtx_t* dir_entries_mutex;
|
||||
/* --no-implied-dirs with -R + --files-from: a directory that is only an
|
||||
* implied parent of a listed entry (not itself listed, nor below a listed
|
||||
* directory) must not carry source metadata; it is created with default
|
||||
* attributes at the destination, matching rsync. */
|
||||
bool no_implied_dirs;
|
||||
} ScannerOptions;
|
||||
|
||||
/* Internal per-scanner filter state. FilterNode chains represent the ordered
|
||||
@@ -168,6 +173,11 @@ typedef struct {
|
||||
int current_depth;
|
||||
dev_t root_dev;
|
||||
bool failed;
|
||||
/* Recursive scan: whether the open directory yielded any transferred or
|
||||
descended entry. When it did not, closing it emits a directory entry so
|
||||
the empty source directory is recreated at the destination (rsync
|
||||
parity). */
|
||||
bool current_dir_produced;
|
||||
/* Phase 2 (files-from / filter layer). */
|
||||
char* root_path; /* transfer root (fs path) for rel computation */
|
||||
char* current_rel; /* rel path of the open directory ("" == root) */
|
||||
|
||||
+3
-2
@@ -103,8 +103,9 @@ void print_usage(void) {
|
||||
printf(" -R, --relative With --files-from, preserve each listed entry's relative path\n");
|
||||
printf(" below the destination root instead of mirroring the full\n");
|
||||
printf(" source path (no effect without --files-from)\n");
|
||||
printf(" --no-implied-dirs With -R --files-from, refuse to place a listed file whose\n");
|
||||
printf(" parent directory is not itself listed\n");
|
||||
printf(" --no-implied-dirs With -R, do not apply the source metadata of a listed file's\n");
|
||||
printf(" implied parent directories (they are still created with\n");
|
||||
printf(" default attributes)\n");
|
||||
printf(" --mkpath Create the destination root directory on the server when it\n");
|
||||
printf(" does not exist yet\n");
|
||||
printf(" --exclude <pattern>, --exclude=<pattern> Exclude files matching pattern\n");
|
||||
|
||||
+4
-2
@@ -424,8 +424,10 @@ typedef struct Config {
|
||||
* repeated -F adds --filter='- .rsync-filter' so they are excluded too. */
|
||||
int per_dir_filter_count;
|
||||
bool one_file_system; /* -x/--one-file-system: do not cross filesystem boundaries */
|
||||
/* --no-implied-dirs: client-only. With -R + --files-from, refuse to place a
|
||||
* listed file whose ancestor directory is not itself explicitly listed. */
|
||||
/* --no-implied-dirs: client-only. With -R, do not transfer the source
|
||||
* metadata of the parent directories implied by a listed path; an unlisted
|
||||
* implied parent is still created (with default attributes) so the listed
|
||||
* file can be placed, matching rsync. */
|
||||
bool no_implied_dirs;
|
||||
/* -d/--dirs: client-only. Transfer the directory entries named by the
|
||||
* source argument / --files-from list without recursing into contents. */
|
||||
|
||||
@@ -807,6 +807,26 @@ bool file_ensure_directory_secure(const char* path) {
|
||||
} else if (errno == EEXIST) {
|
||||
dir_fd = openat(parent_fd, leaf, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
|
||||
}
|
||||
} else if (dir_fd < 0 && (errno == ENOTDIR || errno == ELOOP)) {
|
||||
/* rsync replaces a destination non-directory (regular file or symlink)
|
||||
with an incoming directory. Confined to the already-opened secure
|
||||
parent fd: the leaf is unlinked by name (never followed) and only a
|
||||
non-directory is ever removed, so this cannot escape the authorized
|
||||
root or remove a pre-existing directory tree. A symlink is left alone:
|
||||
replacing it is not required for FastSync's transferred directories and
|
||||
keeps --keep-dirlinks semantics untouched. */
|
||||
struct stat leaf_st;
|
||||
if (fstatat(parent_fd, leaf, &leaf_st, AT_SYMLINK_NOFOLLOW) == 0 &&
|
||||
!S_ISDIR(leaf_st.st_mode) && !S_ISLNK(leaf_st.st_mode)) {
|
||||
if (unlinkat(parent_fd, leaf, 0) == 0) {
|
||||
if (mkdirat(parent_fd, leaf, (mode_t)(0777 & ~(mode_t)file_process_umask())) == 0) {
|
||||
created = true;
|
||||
} else if (errno != EEXIST) {
|
||||
/* leave dir_fd < 0 so the caller sees the failure */
|
||||
}
|
||||
dir_fd = openat(parent_fd, leaf, O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
|
||||
}
|
||||
}
|
||||
}
|
||||
bool ok = dir_fd >= 0;
|
||||
/* --copy-as owns a directory this call just created (the final component;
|
||||
|
||||
@@ -25,40 +25,6 @@ re-triaged when the row moves.
|
||||
|
||||
# case id -> {aspect: "reason (ref: RSYNC_COMPAT.md ...)"}
|
||||
CAVEATS = {
|
||||
# A source subtree whose only files are all filtered out (here, by
|
||||
# --min-size) is left behind as an empty directory by rsync but not by
|
||||
# FastSync: the recursive scanner keeps directory entries record-only, so
|
||||
# `sub/deep` is only implicit through the (skipped) file. This is the
|
||||
# documented recursive-empty-directory residual, not a payload/selection
|
||||
# bug.
|
||||
"min_size": {
|
||||
"tree": "recursive transfer does not create a source directory that "
|
||||
"becomes empty after --min-size filtering (FastSync directory "
|
||||
"entries are record-only). ref: RSYNC_COMPAT.md `-d/--dirs` "
|
||||
"row and completion-wave residual ('recursive transfers still "
|
||||
"do not create empty directories').",
|
||||
},
|
||||
# FastSync's recursive scanner keeps directory entries record-only, so a
|
||||
# plain `-d`/recursive source whose only role for a directory is that entry
|
||||
# (empty dir, or a dir emptied by filtering) is not created on the
|
||||
# destination. rsync creates it. `--dirs`/`--files-from`-listed
|
||||
# directories DO cross as explicit entries (covered by the passing
|
||||
# `empty_dirs_files_from` / `files_from` cases).
|
||||
"empty_dirs_recursive": {
|
||||
"tree": "recursive transfer does not create empty source directories. "
|
||||
"ref: RSYNC_COMPAT.md `-d/--dirs` row and completion-wave "
|
||||
"residual ('recursive transfers still do not create empty "
|
||||
"directories').",
|
||||
},
|
||||
# A plain `-d` invocation: FastSync's `--source-dir` treats the argument as
|
||||
# the directory entry itself (creates the empty source-root mirror), while
|
||||
# rsync's `src/` trailing-slash form lists the immediate contents.
|
||||
"dirs_plain": {
|
||||
"tree": "plain -d semantics: FastSync creates the source-root directory "
|
||||
"entry (its documented --dirs files-from behavior) instead of "
|
||||
"rsync's one-level contents listing for a `src/` argument. "
|
||||
"ref: RSYNC_COMPAT.md `-d/--dirs` row (⚠️).",
|
||||
},
|
||||
# --max-delete stops the extras walk part-way and exits 25 in both
|
||||
# implementations; which of the remaining extras survives depends on
|
||||
# deletion order, which neither tool specifies. The exit code and the
|
||||
|
||||
@@ -89,6 +89,9 @@ class Case:
|
||||
ignore_paths: Tuple[str, ...] = ()
|
||||
extra_check: Optional[Callable] = None
|
||||
files_from: Optional[Tuple[str, ...]] = None
|
||||
# rsync receives ``src + "/"``; FastSync mirrors the path it is given, so a
|
||||
# trailing-slash-sensitive case must hand FastSync the same form.
|
||||
fs_src_suffix: str = ""
|
||||
ci: bool = False
|
||||
ref: str = ""
|
||||
|
||||
@@ -418,6 +421,7 @@ def run_differential( # noqa: PLR0913 (explicit scenario parameters)
|
||||
ignore_paths: Tuple[str, ...] = (),
|
||||
extra_check: Optional[Callable] = None,
|
||||
files_from: Optional[Tuple[str, ...]] = None,
|
||||
fs_src_suffix: str = "",
|
||||
) -> Dict[str, object]:
|
||||
"""Run one rsync/FastSync pair and return the diff aspects.
|
||||
|
||||
@@ -447,7 +451,7 @@ def run_differential( # noqa: PLR0913 (explicit scenario parameters)
|
||||
fs_flags.append(f"--files-from={list_path}")
|
||||
|
||||
rs = run_rsync(src, rdst, rs_flags)
|
||||
fs_result, _ = run_fastsync(src, fdst, fs_flags, server.port)
|
||||
fs_result, _ = run_fastsync(src + fs_src_suffix, fdst, fs_flags, server.port)
|
||||
|
||||
class _View:
|
||||
"""Adapter so tree_diff/extra_check keep the Case-shaped interface."""
|
||||
@@ -486,7 +490,7 @@ def execute_case(case: Case, server) -> Dict[str, object]:
|
||||
layout=case.layout, seed=case.seed, stdout=case.stdout,
|
||||
compare_modes=case.compare_modes, compare_hardlinks=case.compare_hardlinks,
|
||||
ignore_paths=case.ignore_paths, extra_check=case.extra_check,
|
||||
files_from=case.files_from,
|
||||
files_from=case.files_from, fs_src_suffix=case.fs_src_suffix,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -218,7 +218,8 @@ _CASES = [
|
||||
H.Case("files_from", "relative", ["--dirs", "-R"],
|
||||
files_from=("dir1", "sub/x.txt"), layout=H.RELATIVE, ci=True,
|
||||
ref="-d/--dirs + --files-from"),
|
||||
H.Case("dirs_plain", "basic", ["-d"], ref="-d/--dirs (plain)"),
|
||||
H.Case("dirs_plain", "basic", ["-d"], fs_src_suffix="/",
|
||||
ref="-d/--dirs (plain)"),
|
||||
H.Case("empty_dirs_recursive", "empty_dir", ["-a"],
|
||||
ref="recursive empty-directory residual"),
|
||||
H.Case("empty_dirs_files_from", "empty_dir", ["--dirs", "-R"],
|
||||
|
||||
@@ -3523,23 +3523,25 @@ class TestMissingArgs:
|
||||
|
||||
|
||||
class TestNoImpliedDirs:
|
||||
"""--no-implied-dirs (only meaningful with -R + --files-from) refuses to
|
||||
place a listed file whose parent directory is not itself listed."""
|
||||
"""--no-implied-dirs (meaningful with -R) omits the source metadata of a
|
||||
listed path's implied parent directories but still creates those parents
|
||||
with default attributes, matching rsync 3.4.1."""
|
||||
|
||||
def _make(self):
|
||||
return _make_relative_source("noimplied_src")
|
||||
|
||||
@pytest.mark.parametrize("mt", [False, True])
|
||||
def test_implied_dir_only_fails_entry(self, shared_server, mt):
|
||||
def test_implied_dir_created_with_default_attrs(self, shared_server, mt):
|
||||
source = self._make()
|
||||
dest = os.path.join(TEST_DATA_DIR, "noimplied_dst")
|
||||
clean_dir(dest)
|
||||
lst = _write_rel_list(b"a/b.txt\n") # "a" itself is not listed
|
||||
flags = ["--files-from", lst, "-R", "--no-implied-dirs"] + (["--threads"] if mt else [])
|
||||
result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
|
||||
assert result.returncode != 0, "implied parent directory was not rejected"
|
||||
assert "--no-implied-dirs" in (result.stderr or result.stdout)
|
||||
assert not os.path.exists(os.path.join(dest, "a", "b.txt"))
|
||||
assert result.returncode == 0, \
|
||||
f"implied parent directory was not created: {result.stderr[:200]}"
|
||||
assert os.path.isdir(os.path.join(dest, "a")), "implied parent 'a' was not created"
|
||||
assert _read_file(os.path.join(dest, "a", "b.txt")) == b"nested\n"
|
||||
|
||||
@pytest.mark.parametrize("mt", [False, True])
|
||||
def test_listed_dir_allows_file(self, shared_server, mt):
|
||||
@@ -3666,9 +3668,10 @@ class TestDirs:
|
||||
files.extend(os.path.relpath(os.path.join(root, n), mirror) for n in names)
|
||||
assert files == [], f"--dirs descended into contents: {files}"
|
||||
|
||||
def test_dirs_listed_dir_colliding_with_file_fails(self, shared_server):
|
||||
"""A listed directory that already exists as a regular file at the
|
||||
destination fails the transfer cleanly instead of clobbering the file."""
|
||||
def test_dirs_listed_dir_replaces_blocking_file(self, shared_server):
|
||||
"""rsync parity: a listed directory replaces a regular file already at
|
||||
its destination path (rsync removes the non-directory and creates the
|
||||
directory)."""
|
||||
source = self._make()
|
||||
dest = os.path.join(TEST_DATA_DIR, "dirs_coll_dst")
|
||||
clean_dir(dest)
|
||||
@@ -3678,8 +3681,10 @@ class TestDirs:
|
||||
lst = _write_rel_list(b"dir1\n")
|
||||
result, _ = run_client(source, dest, flags=["--files-from", lst, "--dirs", "-R"],
|
||||
port=shared_server.port)
|
||||
assert result.returncode != 0, "dir entry over an existing file did not fail"
|
||||
assert os.path.isfile(blocker), "blocking regular file was clobbered"
|
||||
assert result.returncode == 0, \
|
||||
f"dir entry over an existing file failed: {(result.stderr or result.stdout)[:300]}"
|
||||
assert os.path.isdir(blocker) and not os.path.islink(blocker), \
|
||||
"blocking regular file was not replaced by the incoming directory"
|
||||
|
||||
|
||||
class TestMkpath:
|
||||
@@ -6690,11 +6695,10 @@ class TestDirectoryAndSymlinkTimes:
|
||||
|
||||
@pytest.mark.ci
|
||||
@pytest.mark.parametrize("mt", [False, True])
|
||||
def test_preserve_does_not_create_empty_source_dir(self, shared_server, mt):
|
||||
"""P7 Wave D #1: a captured-but-EMPTY source directory is never created
|
||||
at the destination. The scanner records its time (it is transmitted via
|
||||
STATUS_DIR_TIMES), but the receiver treats that entry as record-only, so
|
||||
`-a` keeps the documented "empty dirs are never transferred" behavior."""
|
||||
def test_preserve_creates_empty_source_dir(self, shared_server, mt):
|
||||
"""rsync parity: a recursive `-a` transfer recreates an empty source
|
||||
directory at the destination (the scanner emits it as an explicit
|
||||
directory entry)."""
|
||||
source = os.path.join(TEST_DATA_DIR, f"empty_dir_{'m' if mt else 's'}_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, f"empty_dir_{'m' if mt else 's'}_dst")
|
||||
clean_dir(source)
|
||||
@@ -6705,8 +6709,8 @@ class TestDirectoryAndSymlinkTimes:
|
||||
flags = ["-a"] + (["--threads"] if mt else [])
|
||||
received = self._run(source, dest, flags, shared_server)
|
||||
assert os.path.isfile(os.path.join(received, "keep.txt")), "regular file missing"
|
||||
assert not os.path.lexists(os.path.join(received, "empty_sub")), \
|
||||
f"-a created an empty source directory at {received}/empty_sub"
|
||||
assert os.path.isdir(os.path.join(received, "empty_sub")), \
|
||||
f"-a did not recreate the empty source directory at {received}/empty_sub"
|
||||
|
||||
@pytest.mark.ci
|
||||
@pytest.mark.parametrize("mt", [False, True])
|
||||
@@ -6729,10 +6733,10 @@ class TestDirectoryAndSymlinkTimes:
|
||||
|
||||
@pytest.mark.ci
|
||||
@pytest.mark.parametrize("mt", [False, True])
|
||||
def test_collision_at_dir_time_path_does_not_abort(self, shared_server, mt):
|
||||
"""P7 Wave D #1: a pre-existing regular file at a source-empty-dir's
|
||||
mirror path must not abort the transfer (the old mkdir failed and failed
|
||||
the run) and must not be clobbered."""
|
||||
def test_collision_at_empty_dir_path_replaces_blocker(self, shared_server, mt):
|
||||
"""rsync parity: a pre-existing regular file at a source empty-dir's
|
||||
mirror path is replaced by the incoming directory (rsync removes the
|
||||
non-directory and creates the directory); the run succeeds."""
|
||||
source = os.path.join(TEST_DATA_DIR, f"dirtime_collide_{'m' if mt else 's'}_src")
|
||||
dest = os.path.join(TEST_DATA_DIR, f"dirtime_collide_{'m' if mt else 's'}_dst")
|
||||
clean_dir(source)
|
||||
@@ -6751,10 +6755,8 @@ class TestDirectoryAndSymlinkTimes:
|
||||
assert result.returncode == 0, \
|
||||
f"-a aborted on a pre-existing file at an empty-dir path: " \
|
||||
f"{(result.stderr or result.stdout)[:400]}"
|
||||
assert os.path.isfile(blocker) and not os.path.islink(blocker), \
|
||||
"the pre-existing blocker was replaced by a directory"
|
||||
with open(blocker, "rb") as fh:
|
||||
assert fh.read() == b"pre-existing blocker\n", "the blocker file was clobbered"
|
||||
assert os.path.isdir(blocker) and not os.path.islink(blocker), \
|
||||
"the pre-existing blocker was not replaced by the incoming directory"
|
||||
assert os.path.isfile(os.path.join(received, "keep.txt")), "regular file missing"
|
||||
|
||||
|
||||
|
||||
@@ -111,6 +111,45 @@ class TestRelativeGeneral:
|
||||
assert int(rs.st_mtime) == int(fs.st_mtime), \
|
||||
f"mtime mismatch for {rel} with {extra}"
|
||||
|
||||
@requires_rsync
|
||||
@pytest.mark.ci
|
||||
def test_no_implied_dirs_files_from_matches_rsync(self, shared_server):
|
||||
"""-R --no-implied-dirs --files-from: a listed file whose parent is not
|
||||
itself listed still transfers; the implied parent is created with
|
||||
default attributes (rsync 3.4.1 parity)."""
|
||||
source = _make_tree(os.path.join(TEST_DATA_DIR, "sel_nidff_src"))
|
||||
# Make the implied parent unmistakably non-default on the source so a
|
||||
# wrongly-applied attribute would be observable.
|
||||
os.chmod(os.path.join(source, "foo"), 0o700)
|
||||
os.chmod(os.path.join(source, "foo", "bar"), 0o711)
|
||||
os.utime(os.path.join(source, "foo"), (978307200, 978307200))
|
||||
os.utime(os.path.join(source, "foo", "bar"), (978307200, 978307200))
|
||||
lst = os.path.join(TEST_DATA_DIR, "sel_nidff_list")
|
||||
with open(lst, "w") as fh:
|
||||
fh.write("foo/bar/baz/f.txt\n")
|
||||
dest = os.path.join(TEST_DATA_DIR, "sel_nidff_dst")
|
||||
rdst = os.path.join(TEST_DATA_DIR, "sel_nidff_rdst")
|
||||
clean_dir(dest)
|
||||
clean_dir(rdst)
|
||||
r = _rsync(["-rlpt", "-R", "--no-implied-dirs", "--files-from=" + lst,
|
||||
source + "/", rdst + "/"])
|
||||
assert r.returncode == 0, r.stderr
|
||||
result, _ = run_client(source, dest, flags=[
|
||||
"-rlpt", "-R", "--no-implied-dirs", "--files-from", lst],
|
||||
port=shared_server.port)
|
||||
assert result.returncode == 0, result.stderr[:300]
|
||||
assert _tree(rdst) == _tree(dest), "implied-parent layout mismatch"
|
||||
# The implied parents exist on both sides and carry the run-time default
|
||||
# attributes, not the source's (non-default) ones.
|
||||
for rel in ("foo", "foo/bar", "foo/bar/baz"):
|
||||
rs = os.stat(os.path.join(rdst, rel))
|
||||
fs = os.stat(os.path.join(dest, rel))
|
||||
assert (rs.st_mode & 0o7777) == (fs.st_mode & 0o7777), \
|
||||
f"mode mismatch for implied {rel}"
|
||||
# The listed file is transferred with its content.
|
||||
with open(os.path.join(dest, "foo", "bar", "baz", "f.txt"), "rb") as fh:
|
||||
assert fh.read() == b"deep\n"
|
||||
|
||||
|
||||
class TestDirsOneLevel:
|
||||
"""#13: -d with a trailing slash (or '.') lists the source's immediate
|
||||
|
||||
Reference in New Issue
Block a user