test(parity): option-wave rsync differentials + docs (109/21/26)
- tests/integration/test_option_parity.py: bwlimit parse matrix + throttle rate, --info flist/name/nonreg/del(dry+real+itemize)/remove, real-setpriv --ignore-errors, rsync-daemon -M forwarding evidence, filter protect/risk destination-only divergence pin. - RSYNC_COMPAT.md: tally 109/21/26; --bwlimit and --ignore-errors -> Parity, --filter and -M -> Divergent with differential rationale, --info residual narrowed to the categories with no client-observable event. - HANDOFF/README: protocol 2.27.0, option-wave summary.
This commit is contained in:
+18
-2
@@ -8,8 +8,8 @@
|
|||||||
- **Release PR #284 (`dev` -> `main`)** open, CI green (run 553).
|
- **Release PR #284 (`dev` -> `main`)** open, CI green (run 553).
|
||||||
`main` is protected: it needs review/approval to merge.
|
`main` is protected: it needs review/approval to merge.
|
||||||
https://gitea.tap-tap.win/TapTap/FastSync/pulls/284
|
https://gitea.tap-tap.win/TapTap/FastSync/pulls/284
|
||||||
- **`PROTOCOL_VERSION` = `"2.26.0"`** (`src/shared/config.h`); CMake
|
- **`PROTOCOL_VERSION` = `"2.27.0"`** (`src/shared/config.h`); CMake
|
||||||
`project(FastFileTransfer VERSION 2.26.0)`.
|
`project(FastFileTransfer VERSION 2.27.0)`.
|
||||||
- Working tree clean; no wave worktrees remain.
|
- Working tree clean; no wave worktrees remain.
|
||||||
|
|
||||||
## What landed this session
|
## What landed this session
|
||||||
@@ -41,6 +41,22 @@
|
|||||||
6. **Rsync-parity wave (protocol 2.23.0)** on `feat/rsync-parity`: rsync short options/clustering/attached values (`-r`/`-b`/`-L`/`-B`, `-av`, `-aAX`, `-B1000`, `-essh`, `-MOPT`), `-c` checksum quick-check, `--checksum-choice`/`--compress-choice` validation and seed randomization, rsync timeout/max-alloc defaults, temp-dir confinement + `EXDEV` fallback, ownership/mapping parity (numeric-ids modifier, map ranges/`*`/empty-FROM, `--chown`+map conflicts, fake-super resolved-owner record), verbatim symlink storage with rsync `--safe-links`/`--munge-links`, socket recreation under `--specials`, `--chmod` 3.4.1 semantics, and delete scoping + `--max-delete` partial/exit-25. Wire: appended delete-manifest synchronized-directory section and `STATUS_DELETE_LIMIT`.
|
6. **Rsync-parity wave (protocol 2.23.0)** on `feat/rsync-parity`: rsync short options/clustering/attached values (`-r`/`-b`/`-L`/`-B`, `-av`, `-aAX`, `-B1000`, `-essh`, `-MOPT`), `-c` checksum quick-check, `--checksum-choice`/`--compress-choice` validation and seed randomization, rsync timeout/max-alloc defaults, temp-dir confinement + `EXDEV` fallback, ownership/mapping parity (numeric-ids modifier, map ranges/`*`/empty-FROM, `--chown`+map conflicts, fake-super resolved-owner record), verbatim symlink storage with rsync `--safe-links`/`--munge-links`, socket recreation under `--specials`, `--chmod` 3.4.1 semantics, and delete scoping + `--max-delete` partial/exit-25. Wire: appended delete-manifest synchronized-directory section and `STATUS_DELETE_LIMIT`.
|
||||||
7. **Parity-completion wave (protocol 2.24.0 → 2.26.0)** on `feat/parity-completion`: per-directory delete plans (`STATUS_DELETE_PLAN`) for `--delete-during`/`--delete-delay`; receiver `STATUS_STATS` counters feeding `--stats`/`--progress` and `--out-format %b/%c/%C`, plus `-n --delete` lines; `lz4`/`zlib`/`zlibx` compression and `md4`/`sha1`/`none` checksums with `auto` negotiation (default `xxh128`/`zstd`); general `-R`/`--no-implied-dirs`/`-d`; the full filter grammar (`merge`/`dir-merge`/`hide`/`show`/`protect`/`risk`/`clear` + modifiers) and corrected `-F`/`-FF`; receiver-side `--chown`/map TO-name resolution; absolute basis dirs + `--link-dest` relink; receiver-side `--ignore-existing` short-circuit; `--preallocate` over `--sparse` via `fallocate(2)`; `--iconv=.`/`-`/`--no-iconv`; lone `-h` help; aliases `--ignore-non-existing`/`--protect-args`/`--msgs2stderr`; and the full `--info`/`--debug` vocabulary. `RSYNC_COMPAT.md` reclassifies the matrix to 106 ✅ / 27 ⚠️ / 23 ❌; the later rsync-parity-stats pass (`fix/parity-stats`) moves it to 107 ✅ / 25 ⚠️ / 24 ❌ (see item 8).
|
7. **Parity-completion wave (protocol 2.24.0 → 2.26.0)** on `feat/parity-completion`: per-directory delete plans (`STATUS_DELETE_PLAN`) for `--delete-during`/`--delete-delay`; receiver `STATUS_STATS` counters feeding `--stats`/`--progress` and `--out-format %b/%c/%C`, plus `-n --delete` lines; `lz4`/`zlib`/`zlibx` compression and `md4`/`sha1`/`none` checksums with `auto` negotiation (default `xxh128`/`zstd`); general `-R`/`--no-implied-dirs`/`-d`; the full filter grammar (`merge`/`dir-merge`/`hide`/`show`/`protect`/`risk`/`clear` + modifiers) and corrected `-F`/`-FF`; receiver-side `--chown`/map TO-name resolution; absolute basis dirs + `--link-dest` relink; receiver-side `--ignore-existing` short-circuit; `--preallocate` over `--sparse` via `fallocate(2)`; `--iconv=.`/`-`/`--no-iconv`; lone `-h` help; aliases `--ignore-non-existing`/`--protect-args`/`--msgs2stderr`; and the full `--info`/`--debug` vocabulary. `RSYNC_COMPAT.md` reclassifies the matrix to 106 ✅ / 27 ⚠️ / 23 ❌; the later rsync-parity-stats pass (`fix/parity-stats`) moves it to 107 ✅ / 25 ⚠️ / 24 ❌ (see item 8).
|
||||||
8. **rsync-parity-stats pass** on `fix/parity-stats` (no wire change, `PROTOCOL_VERSION` stays `2.26.0`): `--delete-delay` now counts/budgets only entries actually removed (a refilled deferred directory that survives `ENOTEMPTY` is not counted; `--max-delete` partial-delete count matches rsync); `--stats` gained the `(reg/dir/link/special)` `Number of files` breakdown and now counts only regular files actually stored for `Number of regular files transferred`/transferred size/literal data (up-to-date re-runs report 0); `Total file size` includes symlink target lengths; `--progress` prints the leading `./` root line and counts it in `to-chk` so a single-file transfer matches rsync; and `%C` uses the selected transfer checksum with `checksum_digest_file` supporting md4/sha1/none, byte-identical to rsync for every algorithm. `--out-format` reclassified ❌ (`%b`/delta-`%c` are protocol-specific). Differential + regression tests added; full suite + ASan + clang-format + cppcheck clean.
|
8. **rsync-parity-stats pass** on `fix/parity-stats` (no wire change, `PROTOCOL_VERSION` stays `2.26.0`): `--delete-delay` now counts/budgets only entries actually removed (a refilled deferred directory that survives `ENOTEMPTY` is not counted; `--max-delete` partial-delete count matches rsync); `--stats` gained the `(reg/dir/link/special)` `Number of files` breakdown and now counts only regular files actually stored for `Number of regular files transferred`/transferred size/literal data (up-to-date re-runs report 0); `Total file size` includes symlink target lengths; `--progress` prints the leading `./` root line and counts it in `to-chk` so a single-file transfer matches rsync; and `%C` uses the selected transfer checksum with `checksum_digest_file` supporting md4/sha1/none, byte-identical to rsync for every algorithm. `--out-format` reclassified ❌ (`%b`/delta-`%c` are protocol-specific). Differential + regression tests added; full suite + ASan + clang-format + cppcheck clean.
|
||||||
|
9. **Option-parity wave (protocol 2.26.0 → 2.27.0, on `fix/parity-options`):**
|
||||||
|
`--bwlimit` now ports rsync 3.4.1's units/quantization and paces like its
|
||||||
|
leaky bucket; `--ignore-errors` reproduces rsync's default (an I/O error
|
||||||
|
skips deletion unless the flag is set; the readable tree still transfers and
|
||||||
|
the run exits 23) across every delete timing; the `--info` categories with a
|
||||||
|
FastSync event (`name`/`flist`/`del`/`remove`/`nonreg`/`progress`) emit
|
||||||
|
rsync's line format, with real-run `deleting`/`*deleting` lines carried over
|
||||||
|
the new trailing config bool `report_deletes` (golden wire updated by
|
||||||
|
`tests/test_config.c`). Two residuals were reclassified **divergent**: `-M`
|
||||||
|
over daemon/TCP (no argv channel in FastSync's binary config handshake;
|
||||||
|
rsync-daemon differential pins the rsync behavior) and receiver-side
|
||||||
|
`protect`/`risk` re-derivation for destination-only entries (would need a
|
||||||
|
receiver filter engine; differential pins the divergence). Matrix now
|
||||||
|
**109 ✅ / 21 ⚠️ / 26 ❌**. New `tests/integration/test_option_parity.py`
|
||||||
|
holds the rsync differentials (bwlimit parse+rate, info lines, real-setpriv
|
||||||
|
`--ignore-errors`, rsync-daemon `-M`, filter-protect pin).
|
||||||
|
|
||||||
## Next steps
|
## Next steps
|
||||||
1. **Merge PR #284** (`dev` -> `main`) once reviewed (protected branch).
|
1. **Merge PR #284** (`dev` -> `main`) once reviewed (protected branch).
|
||||||
|
|||||||
@@ -789,7 +789,7 @@ before the module list, before authentication, and the connecting peer address
|
|||||||
|
|
||||||
## Protocol and Security
|
## Protocol and Security
|
||||||
|
|
||||||
FastSync protocol version `2.26.0` is shared by the client and server. The
|
FastSync protocol version `2.27.0` is shared by the client and server. The
|
||||||
current protocol is sender-driven and includes configuration negotiation,
|
current protocol is sender-driven and includes configuration negotiation,
|
||||||
including the maximum allocation limit, incremental checks, checksums,
|
including the maximum allocation limit, incremental checks, checksums,
|
||||||
manifests, keep-alives, abort handling, per-file remove-source results, and
|
manifests, keep-alives, abort handling, per-file remove-source results, and
|
||||||
|
|||||||
+23
-9
@@ -6,9 +6,9 @@ This document maps rsync's full feature set to FastSync's current implementation
|
|||||||
|
|
||||||
| Status | Count | Description |
|
| Status | Count | Description |
|
||||||
|--------|-------|-------------|
|
|--------|-------|-------------|
|
||||||
| ✅ Parity | 107 | Reproduces rsync's semantics for this option's scope |
|
| ✅ Parity | 109 | Reproduces rsync's semantics for this option's scope |
|
||||||
| ⚠️ Caveat | 25 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
|
| ⚠️ Caveat | 21 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
|
||||||
| ❌ Divergent | 24 | Rejected, an accepted no-op, deliberately non-rsync (native config/auth/batch, privileged namespaces, safe-subset privilege), or impossible on any portable filesystem call |
|
| ❌ Divergent | 26 | Rejected, an accepted no-op, deliberately non-rsync (native config/auth/batch, privileged namespaces, safe-subset privilege), or impossible on any portable filesystem call |
|
||||||
| **Total** | **156** | One row per rsync option/feature group; a row may name several spellings |
|
| **Total** | **156** | One row per rsync option/feature group; a row may name several spellings |
|
||||||
|
|
||||||
This matrix reports honest rsync parity, not "implemented" as a synonym for
|
This matrix reports honest rsync parity, not "implemented" as a synonym for
|
||||||
@@ -21,6 +21,20 @@ the batch container, `--fake-super`'s xattr format, `--copy-as` credential
|
|||||||
switching), or impossible (`-N`/`--crtimes`). The counts are derived from the
|
switching), or impossible (`-N`/`--crtimes`). The counts are derived from the
|
||||||
rows below; update them together with the table.
|
rows below; update them together with the table.
|
||||||
|
|
||||||
|
**Option wave (protocol 2.26.0 → 2.27.0).** A differential pass against rsync
|
||||||
|
3.4.1 over the remaining option caveats. `--bwlimit` now parses rsync's units
|
||||||
|
exactly and paces like rsync's leaky bucket; `--ignore-errors` reproduces
|
||||||
|
rsync's default (an I/O error skips deletion unless the flag is set, while the
|
||||||
|
readable tree still transfers and the run exits 23); the `--info` categories
|
||||||
|
that map to a FastSync event (`name`, `flist`, `del`, `remove`, `nonreg`,
|
||||||
|
`progress`) now emit rsync's line format, including real-run `deleting PATH` /
|
||||||
|
`*deleting` lines carried over a new `report_deletes` wire bool; and two
|
||||||
|
genuinely non-interoperable residuals are reclassified divergent (`-M` over a
|
||||||
|
daemon/TCP connection, which FastSync's binary config handshake has no argv
|
||||||
|
channel for, and receiver-side `protect`/`risk` re-derivation for
|
||||||
|
destination-only entries, which would need a receiver filter engine). That moves
|
||||||
|
the matrix to **109 ✅ / 21 ⚠️ / 26 ❌ = 156**.
|
||||||
|
|
||||||
**Parity completion wave (protocol 2.23.0 → 2.26.0).** This wave closed the
|
**Parity completion wave (protocol 2.23.0 → 2.26.0).** This wave closed the
|
||||||
remaining gaps the rsync-parity wave left open (delete timing, wire counters and
|
remaining gaps the rsync-parity wave left open (delete timing, wire counters and
|
||||||
output, codec breadth, general `-R`/`-d`, the full filter grammar, receiver-side
|
output, codec breadth, general `-R`/`-d`, the full filter grammar, receiver-side
|
||||||
@@ -51,7 +65,7 @@ Every one of those has an entry below with its remaining caveats.
|
|||||||
| `-q`, `--quiet` | Suppress non-error messages | ✅ Parity | Suppresses client output while preserving errors |
|
| `-q`, `--quiet` | Suppress non-error messages | ✅ Parity | Suppresses client output while preserving errors |
|
||||||
| `--help` | Show help | ✅ Parity | Prints usage and exits. A lone `-h` with no other transfer arguments also prints help (protocol 2.26.0), matching the rsync idiom; `-h` alongside a transfer keeps its rsync meaning of `--human-readable` (see that row) |
|
| `--help` | Show help | ✅ Parity | Prints usage and exits. A lone `-h` with no other transfer arguments also prints help (protocol 2.26.0), matching the rsync idiom; `-h` alongside a transfer keeps its rsync meaning of `--human-readable` (see that row) |
|
||||||
| `-V`, `--version` | Print version | ✅ Parity | |
|
| `-V`, `--version` | Print version | ✅ Parity | |
|
||||||
| `--info=FLAGS` | Fine-grained info verbosity | ⚠️ Caveat | Protocol 2.26.0 accepts rsync 3.4.1's full `--info` vocabulary — `backup`, `copy`, `del`, `flist`, `misc`, `mount`, `name`, `nonreg`, `progress`, `remove`, `skip`, `stats`, `symsafe`, `all`, `none` — with optional level suffixes (`--info=stats2`), so a valid rsync invocation is never rejected up front. The categories that map to a FastSync channel emit (`copy`, `name`, `misc`, `skip`, `stats`); the remaining rsync categories are accepted silently, with no output. `none` suppresses info output, explicit flags override `--verbose`, and a genuinely unknown name is still rejected by name (matching rsync). **Caveat:** many accepted rsync categories produce no output (e.g. `del`, `flist`, `remove`, `progress`, `symsafe`, `mount`, `nonreg`, `backup`), so e.g. `--info=progress` is accepted for CLI compatibility only; `name` maps to the `copy` channel rather than rsync's per-file name output |
|
| `--info=FLAGS` | Fine-grained info verbosity | ⚠️ Caveat | Accepts rsync 3.4.1's full `--info` vocabulary — `backup`, `copy`, `del`, `flist`, `misc`, `mount`, `name`, `nonreg`, `progress`, `remove`, `skip`, `stats`, `symsafe`, `all`, `none` — with optional level suffixes (`--info=stats2`), so a valid rsync invocation is never rejected up front. Protocol 2.27.0 wires the categories that map to a real FastSync event, matching rsync's line format: `name` prints the updated entry names (with the ` -> target` link suffix), `flist` prints `sending incremental file list`, `del` prints `deleting PATH` (or `*deleting PATH` under `-i`/`--out-format`) for both dry-run would-delete and real deletions (real runs carry the removed paths over the new `report_deletes` wire bool), `remove` prints `sender removed PATH`, `nonreg` prints `skipping non-regular file "NAME"`, `progress` drives the per-file progress output, and `copy`/`misc`/`skip`/`stats` keep their existing channels. `none` suppresses info output, explicit flags override `--verbose`, and a genuinely unknown name is still rejected by name (matching rsync). **Caveat:** the categories with no client-observable event stay accepted-but-silent — `symsafe`, `mount`, and `backup` (the backup happens on the receiver, which FastSync's protocol does not echo back); `name` level 2 (`is uptodate` lines) and the leading `./` root name line are not emitted; and `skip` maps to FastSync's sender-side skip logging rather than rsync's receiver-side "not creating new file" lines |
|
||||||
| `--debug=FLAGS` | Fine-grained debug verbosity | ⚠️ Caveat | Protocol 2.26.0 accepts rsync 3.4.1's full `--debug` vocabulary with optional level suffixes. FastSync emits for its own channels (`io`, `proto`, `pack`, `util`, plus the aliases `hl`/`owner`); the rsync-only categories (`acl`, `filter`, `send`, ...) are accepted silently. `--debug=help` lists the flags; a genuinely unknown name is rejected by name. **Caveat:** most accepted rsync categories produce no output (e.g. `acl`, `filter`, `send`, `flist`, `del`, `deltasum`, `hash`, `recv`, `time`), so they are accepted for CLI compatibility only |
|
| `--debug=FLAGS` | Fine-grained debug verbosity | ⚠️ Caveat | Protocol 2.26.0 accepts rsync 3.4.1's full `--debug` vocabulary with optional level suffixes. FastSync emits for its own channels (`io`, `proto`, `pack`, `util`, plus the aliases `hl`/`owner`); the rsync-only categories (`acl`, `filter`, `send`, ...) are accepted silently. `--debug=help` lists the flags; a genuinely unknown name is rejected by name. **Caveat:** most accepted rsync categories produce no output (e.g. `acl`, `filter`, `send`, `flist`, `del`, `deltasum`, `hash`, `recv`, `time`), so they are accepted for CLI compatibility only |
|
||||||
| `--stderr=MODE` | Change stderr output mode | ❌ Divergent | `errors` (default) and `all` are supported; `client` is rejected with a clear error (`--stderr=client is not supported`) because FastSync has no rsync client-message channel — the rejection itself is the documented behavior (Phase 7 Wave B decision). The modes that exist work; the missing rsync channel cannot be emulated without a wire change |
|
| `--stderr=MODE` | Change stderr output mode | ❌ Divergent | `errors` (default) and `all` are supported; `client` is rejected with a clear error (`--stderr=client is not supported`) because FastSync has no rsync client-message channel — the rejection itself is the documented behavior (Phase 7 Wave B decision). The modes that exist work; the missing rsync channel cannot be emulated without a wire change |
|
||||||
| `--msgs2stderr`, `--no-msgs2stderr` | Deprecated `--stderr` aliases | ⚠️ Caveat | `--msgs2stderr` maps to `--stderr=all` (supported, matching rsync). `--no-msgs2stderr` is rsync's spelling of `--stderr=client`, which FastSync has no client-message channel for, so it maps to the errors-only default instead of reproducing rsync's client mode. See `--stderr=MODE` |
|
| `--msgs2stderr`, `--no-msgs2stderr` | Deprecated `--stderr` aliases | ⚠️ Caveat | `--msgs2stderr` maps to `--stderr=all` (supported, matching rsync). `--no-msgs2stderr` is rsync's spelling of `--stderr=client`, which FastSync has no client-message channel for, so it maps to the errors-only default instead of reproducing rsync's client mode. See `--stderr=MODE` |
|
||||||
@@ -81,7 +95,7 @@ Every one of those has an entry below with its remaining caveats.
|
|||||||
|------|-------------------|-----------------|-------|
|
|------|-------------------|-----------------|-------|
|
||||||
| `--exclude-from=FILE` | Read exclude patterns from file | ✅ Parity | Reads patterns from file |
|
| `--exclude-from=FILE` | Read exclude patterns from file | ✅ Parity | Reads patterns from file |
|
||||||
| `--include-from=FILE` | Read include patterns from file | ✅ Parity | Reads patterns from file |
|
| `--include-from=FILE` | Read include patterns from file | ✅ Parity | Reads patterns from file |
|
||||||
| `--filter=RULE` | Add file-filtering rule | ⚠️ Caveat | The short `-f` **is** bound to `--filter` (the old FastSync sendfile conflict is gone; sendfile is long-only `--sendfile`), and `-f RULE`, `-f=RULE`, `--filter=RULE` and the two-argument form all parse. Protocol 2.26.0 implements rsync's filter grammar: `+`/`-`, `include`/`exclude`, a leading `/` anchor (to the transfer root or a `.rsync-filter` file's directory), a trailing `/` dir-only rule, and the `merge`/`.`, `dir-merge`/`:`, `hide`/`H`, `show`/`S`, `protect`/`P`, `risk`/`R` and `clear`/`!` words, including the `:`/`.` modifiers. First match wins; the filter layer is independent of `--exclude`/`--include`. **Remaining divergence:** the receiver-mirror protection a `protect`/`risk` rule produces is derived from the sender's source traversal, so a rule that would match only a destination-only entry is not re-derived on the receiver; destination-only deletion protection continues to come from the ordinary sender-derived protected-prefix mechanism |
|
| `--filter=RULE` | Add file-filtering rule | ❌ Divergent | The short `-f` **is** bound to `--filter` (the old FastSync sendfile conflict is gone; sendfile is long-only `--sendfile`), and `-f RULE`, `-f=RULE`, `--filter=RULE` and the two-argument form all parse. Protocol 2.26.0 implements rsync's filter grammar: `+`/`-`, `include`/`exclude`, a leading `/` anchor (to the transfer root or a `.rsync-filter` file's directory), a trailing `/` dir-only rule, and the `merge`/`.`, `dir-merge`/`:`, `hide`/`H`, `show`/`S`, `protect`/`P`, `risk`/`R` and `clear`/`!` words, including the `:`/`.` modifiers. First match wins; the filter layer is independent of `--exclude`/`--include`. **Reclassified because the receiver-side `protect`/`risk` semantics cannot be reproduced:** rsync maintains an independent filter engine on the receiver and re-applies every rule to the destination during deletion, so a `P *.log` rule protects a destination-only `extra.log` (differential test vs rsync 3.4.1 pins this). FastSync is sender-derived: its delete protection is the set of source paths the scan actually pruned, so a rule that matches only a destination-only entry is never re-derived and the extra is deleted. Closing this would require shipping the whole (including per-directory merge) filter grammar to, and re-implementing rsync's dual-sided engine on, the receiver — a protocol/architecture change out of proportion to the residual |
|
||||||
| `--files-from=FILE` | Read source file list from file | ✅ Parity | Entries are paths relative to the source root (leading `./` stripped, `..`/absolute rejected at parse time, blank lines ignored; NUL-delimited with `-0`). A listed regular file is transferred; a listed directory transfers its whole subtree (FastSync recursion is always on). Non-listed paths are pruned by the scanner; the delete manifest is scoped to the listed directory subtrees. A listed entry that does not exist is a hard error unless `--ignore-missing-args`/`--delete-missing-args` is given. **An empty list is a zero-transfer success (exit 0), matching rsync 3.4.1** — the earlier claim that rsync reports "no source files specified" was wrong. Scalability note: `file_list_affects` is O(list size) per scanned entry, so a very large list against a huge tree is quadratic (the documented bound) |
|
| `--files-from=FILE` | Read source file list from file | ✅ Parity | Entries are paths relative to the source root (leading `./` stripped, `..`/absolute rejected at parse time, blank lines ignored; NUL-delimited with `-0`). A listed regular file is transferred; a listed directory transfers its whole subtree (FastSync recursion is always on). Non-listed paths are pruned by the scanner; the delete manifest is scoped to the listed directory subtrees. A listed entry that does not exist is a hard error unless `--ignore-missing-args`/`--delete-missing-args` is given. **An empty list is a zero-transfer success (exit 0), matching rsync 3.4.1** — the earlier claim that rsync reports "no source files specified" was wrong. Scalability note: `file_list_affects` is O(list size) per scanned entry, so a very large list against a huge tree is quadratic (the documented bound) |
|
||||||
| `-0`, `--from0` | Delimit *-from files with NULs | ✅ Parity | `--files-from` entries become NUL-delimited; the flag may appear before or after `--files-from` on the command line. NUL mode preserves entry bytes exactly (trailing CR/LF are part of the name; only newline mode trims them) |
|
| `-0`, `--from0` | Delimit *-from files with NULs | ✅ Parity | `--files-from` entries become NUL-delimited; the flag may appear before or after `--files-from` on the command line. NUL mode preserves entry bytes exactly (trailing CR/LF are part of the name; only newline mode trims them) |
|
||||||
| `--max-size=SIZE` | Skip files larger than SIZE | ✅ Parity | `max_size` in scanner |
|
| `--max-size=SIZE` | Skip files larger than SIZE | ✅ Parity | `max_size` in scanner |
|
||||||
@@ -142,7 +156,7 @@ Every one of those has an entry below with its remaining caveats.
|
|||||||
| `--delete-after` | Delete after transfer | ✅ Parity | Implies `--delete`. The delete-after timing is also what plain `--delete` does: the keep-set manifest closes the data stream and the receiver commits the bounded deletion only after the terminal `STATUS_FINISHED` proves the whole transfer (every data frame received and stored) succeeded. A failed or aborted transfer removes nothing |
|
| `--delete-after` | Delete after transfer | ✅ Parity | Implies `--delete`. The delete-after timing is also what plain `--delete` does: the keep-set manifest closes the data stream and the receiver commits the bounded deletion only after the terminal `STATUS_FINISHED` proves the whole transfer (every data frame received and stored) succeeded. A failed or aborted transfer removes nothing |
|
||||||
| `--delete-excluded` | Also delete excluded files | ⚠️ Caveat | `delete_excluded` config field. Under `--delete` FastSync protects (rsync's default) the destination mirror of paths the sender's source scan pruned by the user-selection rules — the `--filter`/`-F`/`-C` layer and the legacy `--exclude`/`--include` layer. The sender transmits those concrete pruned paths as **protected prefixes** in the delete-manifest frame (see the Phase-3 notes below); the walker never descends into or removes them. `--delete-excluded` opts back in: the sender sends an empty protected list, so the excluded destination mirrors become ordinary extras and are removed. **`--max-size`/`--min-size` pruned mirrors are a separate, always-on protection** (protocol 2.23.0, rsync parity): size-pruned source mirrors survive `--delete` even with `--delete-excluded`. Divergences (documented): protection is derived only from what the source scan actually pruned — a stray destination-only file that happens to match an exclude rule is not protected (FastSync never re-applies rules to the destination, keeping deletion sender-derived) |
|
| `--delete-excluded` | Also delete excluded files | ⚠️ Caveat | `delete_excluded` config field. Under `--delete` FastSync protects (rsync's default) the destination mirror of paths the sender's source scan pruned by the user-selection rules — the `--filter`/`-F`/`-C` layer and the legacy `--exclude`/`--include` layer. The sender transmits those concrete pruned paths as **protected prefixes** in the delete-manifest frame (see the Phase-3 notes below); the walker never descends into or removes them. `--delete-excluded` opts back in: the sender sends an empty protected list, so the excluded destination mirrors become ordinary extras and are removed. **`--max-size`/`--min-size` pruned mirrors are a separate, always-on protection** (protocol 2.23.0, rsync parity): size-pruned source mirrors survive `--delete` even with `--delete-excluded`. Divergences (documented): protection is derived only from what the source scan actually pruned — a stray destination-only file that happens to match an exclude rule is not protected (FastSync never re-applies rules to the destination, keeping deletion sender-derived) |
|
||||||
| `--max-delete=NUM` | Max files to delete | ✅ Parity | `max_delete` config field (default -1 = no client limit; 0 = delete nothing). **Protocol 2.23.0 matches rsync's partial semantics:** the receiver deletes up to NUM entries (regular files, symlinks and empty directories; each directory removal counts as one) and then **stops deleting, skips the rest, and reports the run as partial**. The client prints a "deletions stopped due to `--max-delete` limit" message and exits **25** (rsync's `RERR_PARTIAL`), not a hard failure — the transfer itself succeeded. NUM only applies together with `--delete` (it is inert otherwise, matching rsync). A client NUM below the server hard bound `MAX_SERVER_DELETE_COUNT` (100000) replaces it; a NUM above it never raises that cap. Deleting an entire destination with no limit is still bounded by the server's 100000-entry ceiling. `--delete-missing-args` exact-path deletions and the ordinary extras walk draw from the same budget, matching rsync |
|
| `--max-delete=NUM` | Max files to delete | ✅ Parity | `max_delete` config field (default -1 = no client limit; 0 = delete nothing). **Protocol 2.23.0 matches rsync's partial semantics:** the receiver deletes up to NUM entries (regular files, symlinks and empty directories; each directory removal counts as one) and then **stops deleting, skips the rest, and reports the run as partial**. The client prints a "deletions stopped due to `--max-delete` limit" message and exits **25** (rsync's `RERR_PARTIAL`), not a hard failure — the transfer itself succeeded. NUM only applies together with `--delete` (it is inert otherwise, matching rsync). A client NUM below the server hard bound `MAX_SERVER_DELETE_COUNT` (100000) replaces it; a NUM above it never raises that cap. Deleting an entire destination with no limit is still bounded by the server's 100000-entry ceiling. `--delete-missing-args` exact-path deletions and the ordinary extras walk draw from the same budget, matching rsync |
|
||||||
| `--ignore-errors` | Delete even with I/O errors | ⚠️ Caveat | Sender-side, client-only config field. rsync suppresses `--delete` when the transfer had I/O errors; FastSync's equivalent is a source-scan I/O error (an unreadable directory, e.g. EACCES). By default the scan aborts the run so no deletion happens. With `--ignore-errors` the scan continues past the unreadable directory, the readable tree is transferred, deletion still runs (the unreadable directory's mirror is treated as an extra), and the run exits 23 (`RERR_PARTIAL`), matching rsync. **Remaining divergence / uncertainty:** the EACCES differential is not exercised in CI because the runner is root (mode 000 is still readable), so this rests on source inspection plus the setpriv integration test |
|
| `--ignore-errors` | Delete even with I/O errors | ✅ Parity | Sender-side, client-only config field. Matches rsync's semantics exactly: an unreadable source subdirectory is always skipped so the readable tree transfers (the transfer root itself stays fatal), and the run reports rsync's partial-transfer exit **23**. Deletion policy follows rsync: by default an I/O error suppresses deletion (`IO error encountered -- skipping file deletion`), while `--ignore-errors` lets the deletion commit. The decision applies to every timing (`--delete`, `--delete-before`, `--delete-during`, `--delete-delay`, `--delete-after`) in both the sequential and `--threads` send paths. Differential-tested against rsync 3.4.1 with both tools run as an unprivileged user (mode-000 source directory); the reference build's root-only gate still excludes the EACCES differential, but the setpriv differential test exercises it. The piece that stays FastSync-specific is documented under the recursive-empty-directory residual: FastSync never emits an unreadable (or empty) directory entry, so that mirror is an extra that a run with `--ignore-errors` removes, where rsync emits the directory and keeps its mirror |
|
||||||
| `--force` | Force deletion of non-empty dirs | ✅ Parity | `force_delete` receiver config field (crosses the wire). rsync's `--force` lets an incoming non-directory replace a destination directory; FastSync implements exactly that: when a regular file (or symlink) is written to a path that is currently a (possibly non-empty) destination directory, `--force` removes that directory tree first — confined to the receive root and symlink-safe (O_NOFOLLOW fd walk, symlinks removed by name, never followed) — so the install can place the file. **Protocol 2.23.0 honors `--force` on the `--delay-updates` publication path too**, not only the immediate-install path. Without `--force` such a write fails and the run aborts. Gated by the server `--allow-delete` policy (a client cannot use `--force` to remove a destination tree on a server that forbids deletion) |
|
| `--force` | Force deletion of non-empty dirs | ✅ Parity | `force_delete` receiver config field (crosses the wire). rsync's `--force` lets an incoming non-directory replace a destination directory; FastSync implements exactly that: when a regular file (or symlink) is written to a path that is currently a (possibly non-empty) destination directory, `--force` removes that directory tree first — confined to the receive root and symlink-safe (O_NOFOLLOW fd walk, symlinks removed by name, never followed) — so the install can place the file. **Protocol 2.23.0 honors `--force` on the `--delay-updates` publication path too**, not only the immediate-install path. Without `--force` such a write fails and the run aborts. Gated by the server `--allow-delete` policy (a client cannot use `--force` to remove a destination tree on a server that forbids deletion) |
|
||||||
| `-m`, `--prune-empty-dirs` | Prune empty dir chains | ✅ Parity | `-m`/`--prune-empty-dirs` (Phase 7 Wave A freed the rsync short `-m`; FastSync multithreading is now `-j`/`--threads`). FastSync's recursive transfer records directory times but never CREATES an empty directory (a `STATUS_DIR_TIMES` entry is record-only, and `--dirs` empty entries are pruned by this flag), so empty directories are inherently never transferred (which is rsync's `-m` behavior) and truly-empty destination directory chains are removed by `--delete` regardless of this flag. The flag's additional real effect is on the `--dirs` explicit directory-entry generator: a plain `-d <empty-dir>` run omits the empty source directory's entry, so nothing is created at the destination (no `STATUS_MKDIR`, no `-i`/`--out-format` change line, and an existing empty mirror becomes an extra that `--delete` prunes). Explicitly `--files-from`-listed directories always pass through (documented `--files-from` behavior). A directory that still holds an excluded-but-protected file survives, matching the `--delete-excluded` default |
|
| `-m`, `--prune-empty-dirs` | Prune empty dir chains | ✅ Parity | `-m`/`--prune-empty-dirs` (Phase 7 Wave A freed the rsync short `-m`; FastSync multithreading is now `-j`/`--threads`). FastSync's recursive transfer records directory times but never CREATES an empty directory (a `STATUS_DIR_TIMES` entry is record-only, and `--dirs` empty entries are pruned by this flag), so empty directories are inherently never transferred (which is rsync's `-m` behavior) and truly-empty destination directory chains are removed by `--delete` regardless of this flag. The flag's additional real effect is on the `--dirs` explicit directory-entry generator: a plain `-d <empty-dir>` run omits the empty source directory's entry, so nothing is created at the destination (no `STATUS_MKDIR`, no `-i`/`--out-format` change line, and an existing empty mirror becomes an extra that `--delete` prunes). Explicitly `--files-from`-listed directories always pass through (documented `--files-from` behavior). A directory that still holds an excluded-but-protected file survives, matching the `--delete-excluded` default |
|
||||||
|
|
||||||
@@ -659,8 +673,8 @@ targets verbatim, matching rsync.
|
|||||||
| `--address=ADDRESS` | Bind address for outgoing socket | ✅ Parity | Binds the outgoing client socket to a local source address before `connect()` (resolved with the same `-4`/`-6` family hints as the destination). Local socket concern: never crosses the wire |
|
| `--address=ADDRESS` | Bind address for outgoing socket | ✅ Parity | Binds the outgoing client socket to a local source address before `connect()` (resolved with the same `-4`/`-6` family hints as the destination). Local socket concern: never crosses the wire |
|
||||||
| `-4`, `--ipv4` | Prefer IPv4 | ✅ Parity | Forces `AF_INET` in the `getaddrinfo` hints for client destination/source resolution and the server bind (see the Phase 5, Wave B note). Mutually exclusive with `-6` |
|
| `-4`, `--ipv4` | Prefer IPv4 | ✅ Parity | Forces `AF_INET` in the `getaddrinfo` hints for client destination/source resolution and the server bind (see the Phase 5, Wave B note). Mutually exclusive with `-6` |
|
||||||
| `-6`, `--ipv6` | Prefer IPv6 | ✅ Parity | Forces `AF_INET6` in the `getaddrinfo` hints for client destination/source resolution and the server bind. Mutually exclusive with `-4` |
|
| `-6`, `--ipv6` | Prefer IPv6 | ✅ Parity | Forces `AF_INET6` in the `getaddrinfo` hints for client destination/source resolution and the server bind. Mutually exclusive with `-4` |
|
||||||
| `--remote-option=OPT`, `-M` | Send an option only to the remote side | ⚠️ Caveat | Each value is appended to the remote server invocation over SSH as an individually single-quote-escaped shell word in `ssh_build_remote_command()`. Values are validated (non-empty, no control characters) and shell metacharacters cannot break out of the quoting (`;`, `&`, `\|`, <code>`</code>, `$`, `(`, `)`, quotes are neutralized), so a value cannot inject an arbitrary remote command and a subsequent `--` on the client line cannot be turned into one. The short `-M` form (`-M OPT`, `-M=OPT`, and rsync-style attached `-MOPT`) is available, matching rsync; metadata mode moved to long-only `--preserve`. **Divergence:** `-M` is only meaningful for the SSH transport (`user@host:path`); a daemon (`host::module/path`) or local TCP destination **rejects** it (there is no remote command line to append to), whereas rsync applies it to its own remote process on every transport. The options never cross the binary config frame |
|
| `--remote-option=OPT`, `-M` | Send an option only to the remote side | ❌ Divergent | Each value is appended to the remote server invocation over SSH as an individually single-quote-escaped shell word in `ssh_build_remote_command()`. Values are validated (non-empty, no control characters) and shell metacharacters cannot break out of the quoting (`;`, `&`, `\|`, <code>`</code>, `$`, `(`, `)`, quotes are neutralized), so a value cannot inject an arbitrary remote command and a subsequent `--` on the client line cannot be turned into one. The short `-M` form (`-M OPT`, `-M=OPT`, and rsync-style attached `-MOPT`) is available, matching rsync; metadata mode moved to long-only `--preserve`. **Reclassified because the daemon/TCP case cannot be reproduced:** `-M` is only meaningful for the SSH transport (`user@host:path`); a daemon (`host::module/path`) or local TCP destination **rejects** it, whereas rsync forwards it to its own remote process on every transport. A differential test starts a real rsync daemon and shows `-M--totally-bogus` reaching the remote parser (`unknown option`) while a valid `-M--safe-links` is accepted. FastSync's daemon handshake is a fixed binary config frame with no per-connection argv channel; adding one would let a client set arbitrary server-side options (the same class of divergence as the native daemon config/auth), so the safe subset stays SSH-only |
|
||||||
| `--bwlimit=RATE` | Limit I/O bandwidth | ⚠️ Caveat | Token-bucket throttling of the transfer I/O (Kibibytes/second). **Divergence:** FastSync accepts only a positive integer; rsync additionally accepts `0` (no limit) and decimal/suffixed rates (`1.5`, `1.5m`, `100K`), so those rsync spellings are rejected. The limit is a local I/O concern and is not negotiated on the wire |
|
| `--bwlimit=RATE` | Limit I/O bandwidth | ✅ Parity | A faithful port of rsync 3.4.1's `parse_size_arg(bwlimit_arg, 'K', "bwlimit", 512, -1, True)`: a bare value is KiB/s, `K`/`M`/`G`/`T`/`P` are binary suffixes, `KB`/`MB` are decimal, `KiB`/`MiB` are binary, decimals are accepted and quantized to whole KiB exactly like rsync's `(size + 512) / 1024`, `0` (or an empty value) means "no limit", and any other value below the 512-byte floor is rejected. The token bucket's burst capacity is ~100 ms of bandwidth, matching the point at which rsync's leaky bucket starts sleeping, so a throttled transfer paces like rsync (4 MiB at `--bwlimit=1024`/`2048` matches rsync within ~4%). Differential-tested: the accept/reject matrix and the wall-clock rate both match rsync 3.4.1. The limit is a local I/O concern and is not negotiated on the wire |
|
||||||
|
|
||||||
## 14. Daemon Mode
|
## 14. Daemon Mode
|
||||||
|
|
||||||
@@ -733,7 +747,7 @@ modes or links.
|
|||||||
| `--stop-after=MINS` | Stop after N minutes | ✅ Parity | Client-only sender stop deadline (Phase 6): computing `--stop-after=MINS` (a positive minute count; 0/negative/garbage rejected) and `--stop-at=TIME` (`HH:MM`, `HH:MM:SS`, or `now+N[smhd]`; a past time stops immediately). The transfer stops ELEGANTLY at the next chunk boundary: everything already fully sent is kept and applied, the run returns 0, and --delete (late/delete-after timing) does NOT wipe the destination — when the scan is cut short the partial keep-set manifest is suppressed with a warning (the delete walk is skipped rather than acting on an incomplete keep-set, so unscanned source mirrors survive). `--delete-before`/`--delete-during` still run their complete pre-scan (which ignores the deadline). Local client-only fields: never serialized into the wire config frame, so no PROTOCOL_VERSION bump. `--stop-after` uses CLOCK_MONOTONIC; `--stop-at` uses the wall clock. Works single-threaded and under `-j`/`--threads` (multithreaded). Divergence: rsync computes `--stop-after` from the run start; FastSync likewise. When both are given, the earlier of the two deadlines wins (checked per iteration). See the Phase-6 stop notes below |
|
| `--stop-after=MINS` | Stop after N minutes | ✅ Parity | Client-only sender stop deadline (Phase 6): computing `--stop-after=MINS` (a positive minute count; 0/negative/garbage rejected) and `--stop-at=TIME` (`HH:MM`, `HH:MM:SS`, or `now+N[smhd]`; a past time stops immediately). The transfer stops ELEGANTLY at the next chunk boundary: everything already fully sent is kept and applied, the run returns 0, and --delete (late/delete-after timing) does NOT wipe the destination — when the scan is cut short the partial keep-set manifest is suppressed with a warning (the delete walk is skipped rather than acting on an incomplete keep-set, so unscanned source mirrors survive). `--delete-before`/`--delete-during` still run their complete pre-scan (which ignores the deadline). Local client-only fields: never serialized into the wire config frame, so no PROTOCOL_VERSION bump. `--stop-after` uses CLOCK_MONOTONIC; `--stop-at` uses the wall clock. Works single-threaded and under `-j`/`--threads` (multithreaded). Divergence: rsync computes `--stop-after` from the run start; FastSync likewise. When both are given, the earlier of the two deadlines wins (checked per iteration). See the Phase-6 stop notes below |
|
||||||
| `--stop-at=TIME` | Stop at specified time | ✅ Parity | Deadline transfer stop (client-only, never serialized). Protocol 2.26.0 accepts rsync's full date/time grammar (`2030-12-31T23:59`, `2030/12/31T23:59`, `2030-12-31`, `12-31`, `14:00`, `:59`, `1`) in addition to FastSync's `HH:MM[:SS]` and `now+N[smhd]`; a past time stops immediately. Everything already transferred is kept and an early stop suppresses the late `--delete` keep-set so unscanned source mirrors survive. Works single-threaded and under `-j`/`--threads` |
|
| `--stop-at=TIME` | Stop at specified time | ✅ Parity | Deadline transfer stop (client-only, never serialized). Protocol 2.26.0 accepts rsync's full date/time grammar (`2030-12-31T23:59`, `2030/12/31T23:59`, `2030-12-31`, `12-31`, `14:00`, `:59`, `1`) in addition to FastSync's `HH:MM[:SS]` and `now+N[smhd]`; a past time stops immediately. Everything already transferred is kept and an early stop suppresses the late `--delete` keep-set so unscanned source mirrors survive. Works single-threaded and under `-j`/`--threads` |
|
||||||
| `--fsync` | Fsync every written file before publication | ✅ Parity | |
|
| `--fsync` | Fsync every written file before publication | ✅ Parity | |
|
||||||
| `--protocol=NUM` | Force older protocol version | ❌ Divergent | Forces the wire protocol version for this transfer. FastSync has exactly ONE wire format (`PROTOCOL_VERSION`, currently 2.26.0) with no downgrade/backward-compat code paths, so `--protocol=2.26.0` is accepted (it sets the version claim the client sends, which the server already requires to match exactly) and **every other value is rejected up front** with a clear error before any connection — it does not and cannot speak an older or virtual wire format. Divergence from rsync (which negotiates a range and downgrades to an integer 0..31): FastSync's honest contract is force-to-the-one-supported-value; a genuine downgrade would require a per-version compatibility layer that does not exist. Client-only; the server-side exact-match check is unchanged. `--protocol=2.25.0`/`2.24.0`/`2.23.0`/`2.22.0`/`2.21.0`/`2.20.0`/`2.19.0`/`2.18.0`/`2.17.0`/`2.16.0`/`2.15.0`/`216`/`31`/garbage are all rejected. See the Phase-6 protocol note below |
|
| `--protocol=NUM` | Force older protocol version | ❌ Divergent | Forces the wire protocol version for this transfer. FastSync has exactly ONE wire format (`PROTOCOL_VERSION`, currently 2.27.0) with no downgrade/backward-compat code paths, so `--protocol=2.27.0` is accepted (it sets the version claim the client sends, which the server already requires to match exactly) and **every other value is rejected up front** with a clear error before any connection — it does not and cannot speak an older or virtual wire format. Divergence from rsync (which negotiates a range and downgrades to an integer 0..31): FastSync's honest contract is force-to-the-one-supported-value; a genuine downgrade would require a per-version compatibility layer that does not exist. Client-only; the server-side exact-match check is unchanged. `--protocol=2.26.0`/`2.25.0`/`2.24.0`/`2.23.0`/`2.22.0`/`2.21.0`/`2.20.0`/`2.19.0`/`2.18.0`/`2.17.0`/`2.16.0`/`2.15.0`/`216`/`31`/garbage are all rejected. See the Phase-6 protocol note below |
|
||||||
| `--iconv=CONVERT_SPEC` | Charset conversion | ⚠️ Caveat | Charset conversion of FILE NAMES (not content) at the protocol boundary via iconv(3): `--iconv=LOCAL[,REMOTE]` — the sender converts each local filename LOCAL→REMOTE before transmitting, and the receiver converts each wire filename REMOTE→LOCAL before creating/writing. The full CONVERT_SPEC is serialized into the config frame as a new trailing string field so the peer knows the wire charset; **PROTOCOL_VERSION bumped 2.15.0 → 2.16.0**. `LOCAL[,REMOTE]` parse: single charset ⇒ LOCAL==REMOTE (identity both ways); garbage rejected up front; protocol 2.26.0 additionally accepts `--iconv=.` (the locale's default charset for both directions), `--iconv=-` and `--no-iconv` (disable conversion). Validation probes BOTH directions (a spec that only opens one way is refused, as is a NUL-emitting target charset like utf-16/utf-32/ucs-2, since filenames cannot contain NUL). An unrepresentable name (EILSEQ/EINVAL) fails that path cleanly with a logged `--iconv: cannot convert file name ...` and is never written mangled/truncated. Conversion is applied at EVERY wire-path site (regular/MKDIR/hardlink path+target/symlink path+target/SPECIAL, the delete manifest, the incremental-check path, and the `-s`/`chunk_serialize` embedded blob path), on both client and server (`--iconv` is also a server/daemon option). Zero overhead when unset. See the Phase-6 iconv notes below |
|
| `--iconv=CONVERT_SPEC` | Charset conversion | ⚠️ Caveat | Charset conversion of FILE NAMES (not content) at the protocol boundary via iconv(3): `--iconv=LOCAL[,REMOTE]` — the sender converts each local filename LOCAL→REMOTE before transmitting, and the receiver converts each wire filename REMOTE→LOCAL before creating/writing. The full CONVERT_SPEC is serialized into the config frame as a new trailing string field so the peer knows the wire charset; **PROTOCOL_VERSION bumped 2.15.0 → 2.16.0**. `LOCAL[,REMOTE]` parse: single charset ⇒ LOCAL==REMOTE (identity both ways); garbage rejected up front; protocol 2.26.0 additionally accepts `--iconv=.` (the locale's default charset for both directions), `--iconv=-` and `--no-iconv` (disable conversion). Validation probes BOTH directions (a spec that only opens one way is refused, as is a NUL-emitting target charset like utf-16/utf-32/ucs-2, since filenames cannot contain NUL). An unrepresentable name (EILSEQ/EINVAL) fails that path cleanly with a logged `--iconv: cannot convert file name ...` and is never written mangled/truncated. Conversion is applied at EVERY wire-path site (regular/MKDIR/hardlink path+target/symlink path+target/SPECIAL, the delete manifest, the incremental-check path, and the `-s`/`chunk_serialize` embedded blob path), on both client and server (`--iconv` is also a server/daemon option). Zero overhead when unset. See the Phase-6 iconv notes below |
|
||||||
| `--checksum-seed=NUM` | Set checksum seed | ✅ Parity | Sets the seed for FastSync's whole-file xxHash digest (full 64-bit seed) and for the delta path's per-block xxHash32 strong checksum (low 32 bits of the seed). **As of protocol 2.23.0 a seed of `0` — the default when the flag is unset — is randomized per transfer and the chosen seed is sent to the receiver**, exactly like rsync, so two runs against different content do not share a predictable seed; an explicit non-zero seed is used verbatim, so an explicit seed deterministically reproduces every computed digest on BOTH endpoints (the seed crosses in the config frame). `--checksum-choice=md5` has no seed and ignores it (documented). The value is a strict decimal 0..2⁶⁴-1 (blank, signed, or non-numeric values are rejected). Like rsync, a seed only matters where a digest is actually computed (`--checksum` or a basis-dir run, or a delta transfer); it does not by itself enable `--checksum`/`--delta` |
|
| `--checksum-seed=NUM` | Set checksum seed | ✅ Parity | Sets the seed for FastSync's whole-file xxHash digest (full 64-bit seed) and for the delta path's per-block xxHash32 strong checksum (low 32 bits of the seed). **As of protocol 2.23.0 a seed of `0` — the default when the flag is unset — is randomized per transfer and the chosen seed is sent to the receiver**, exactly like rsync, so two runs against different content do not share a predictable seed; an explicit non-zero seed is used verbatim, so an explicit seed deterministically reproduces every computed digest on BOTH endpoints (the seed crosses in the config frame). `--checksum-choice=md5` has no seed and ignores it (documented). The value is a strict decimal 0..2⁶⁴-1 (blank, signed, or non-numeric values are rejected). Like rsync, a seed only matters where a digest is actually computed (`--checksum` or a basis-dir run, or a delta transfer); it does not by itself enable `--checksum`/`--delta` |
|
||||||
| `--secluded-args`, `-s` | Use protocol to send args | ❌ Divergent | Accepted for CLI compatibility (including the rsync short `-s`, Phase 7 Wave A) but a documented **no-op / divergence**. rsync's `-s` protects arguments from shell expansion by shipping them over the protocol; FastSync never passes remote arguments through a shell expansion boundary in the first place — its SSH transport builds the remote argv as **single-quote-escaped shell words** (`ssh_build_remote_command`), so the injection/leak that `-s` guards against does not exist and there is nothing to "seclude". Implementing a true arg-send protocol would mean replacing the argv-based SSH launch with an in-band argument channel, a large redesign of the transport that buys no security here. Chunk serialization remains the long-only `--chunk-serialization`. |
|
| `--secluded-args`, `-s` | Use protocol to send args | ❌ Divergent | Accepted for CLI compatibility (including the rsync short `-s`, Phase 7 Wave A) but a documented **no-op / divergence**. rsync's `-s` protects arguments from shell expansion by shipping them over the protocol; FastSync never passes remote arguments through a shell expansion boundary in the first place — its SSH transport builds the remote argv as **single-quote-escaped shell words** (`ssh_build_remote_command`), so the injection/leak that `-s` guards against does not exist and there is nothing to "seclude". Implementing a true arg-send protocol would mean replacing the argv-based SSH launch with an in-band argument channel, a large redesign of the transport that buys no security here. Chunk serialization remains the long-only `--chunk-serialization`. |
|
||||||
|
|||||||
@@ -70,10 +70,9 @@ static bool strbuf_append(StrBuf* buf, const char* text) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
bool change_list_enabled(const Config* config) {
|
bool change_list_enabled(const Config* config) {
|
||||||
return config != NULL &&
|
return config != NULL && (config->itemize_changes || config->out_format != NULL ||
|
||||||
(config->itemize_changes || config->out_format != NULL ||
|
(config->log_file != NULL && config->log_file_format != NULL) ||
|
||||||
(config->log_file != NULL && config->log_file_format != NULL) ||
|
(config->info_level & LOG_INFO_NAME) != 0);
|
||||||
(config->info_level & LOG_INFO_NAME) != 0);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/* ---- Itemize code ---- */
|
/* ---- Itemize code ---- */
|
||||||
|
|||||||
+10
-10
@@ -501,7 +501,9 @@ static bool is_accepted_debug_category(const char* name) {
|
|||||||
|
|
||||||
static bool is_accepted_info_category(const char* name) {
|
static bool is_accepted_info_category(const char* name) {
|
||||||
static const char* const categories[] = {
|
static const char* const categories[] = {
|
||||||
"mount", "syms", "symsafe",
|
"mount",
|
||||||
|
"syms",
|
||||||
|
"symsafe",
|
||||||
};
|
};
|
||||||
for (size_t i = 0; i < sizeof(categories) / sizeof(categories[0]); i++) {
|
for (size_t i = 0; i < sizeof(categories) / sizeof(categories[0]); i++) {
|
||||||
if (strcmp(name, categories[i]) == 0)
|
if (strcmp(name, categories[i]) == 0)
|
||||||
@@ -1846,9 +1848,9 @@ static int parse_bwlimit_value(const char* value, unsigned long long* bytes_per_
|
|||||||
long long mult;
|
long long mult;
|
||||||
while (*arg >= '0' && *arg <= '9')
|
while (*arg >= '0' && *arg <= '9')
|
||||||
arg++;
|
arg++;
|
||||||
if (*arg != '\0' &&
|
if (*arg != '\0' && (*arg == '.' || *arg == localeconv()->decimal_point[0]))
|
||||||
(*arg == '.' || *arg == localeconv()->decimal_point[0]))
|
for (arg++; *arg >= '0' && *arg <= '9'; arg++) {
|
||||||
for (arg++; *arg >= '0' && *arg <= '9'; arg++) {}
|
}
|
||||||
|
|
||||||
char suffix = *arg && *arg != '+' && *arg != '-' ? *arg++ : 'K';
|
char suffix = *arg && *arg != '+' && *arg != '-' ? *arg++ : 'K';
|
||||||
switch (suffix) {
|
switch (suffix) {
|
||||||
@@ -1932,8 +1934,7 @@ static int set_bwlimit_option(const char* value) {
|
|||||||
if (parse_bwlimit_value(value, &bytes_per_sec) != 0)
|
if (parse_bwlimit_value(value, &bytes_per_sec) != 0)
|
||||||
return -1;
|
return -1;
|
||||||
io_set_bwlimit(bytes_per_sec);
|
io_set_bwlimit(bytes_per_sec);
|
||||||
log_info_message(LOG_INFO_MISC, "Set bandwidth limit to %llu KB/s",
|
log_info_message(LOG_INFO_MISC, "Set bandwidth limit to %llu KB/s", bytes_per_sec / 1024);
|
||||||
bytes_per_sec / 1024);
|
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -2651,10 +2652,9 @@ static int cli_finalize_config(Config* config, bool verbose, bool no_delta, bool
|
|||||||
/* --info=del on a real --delete run asks the receiver to report the paths it
|
/* --info=del on a real --delete run asks the receiver to report the paths it
|
||||||
actually removed; the report rides the STATUS_STATS path list, so the wire
|
actually removed; the report rides the STATUS_STATS path list, so the wire
|
||||||
stats frame must be negotiated too. */
|
stats frame must be negotiated too. */
|
||||||
config->report_deletes =
|
config->report_deletes = config->use_delete && !config->dry_run &&
|
||||||
config->use_delete && !config->dry_run &&
|
((config->info_level & LOG_INFO_DEL) != 0 || config->itemize_changes ||
|
||||||
((config->info_level & LOG_INFO_DEL) != 0 || config->itemize_changes ||
|
config->out_format != NULL);
|
||||||
config->out_format != NULL);
|
|
||||||
config->report_stats = config->stats || config->show_progress ||
|
config->report_stats = config->stats || config->show_progress ||
|
||||||
(config->info_level & LOG_INFO_PROGRESS) || format_needs_wire ||
|
(config->info_level & LOG_INFO_PROGRESS) || format_needs_wire ||
|
||||||
config->report_deletes || (config->dry_run && config->use_delete);
|
config->report_deletes || (config->dry_run && config->use_delete);
|
||||||
|
|||||||
@@ -347,8 +347,8 @@ static void print_delete_reports(const Config* config, const ArrayList* paths) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
static void client_progress_begin(const Config* config) {
|
static void client_progress_begin(const Config* config) {
|
||||||
g_progress_active = (config->show_progress || info_flag_enabled(config, LOG_INFO_PROGRESS)) &&
|
g_progress_active =
|
||||||
!config->quiet;
|
(config->show_progress || info_flag_enabled(config, LOG_INFO_PROGRESS)) && !config->quiet;
|
||||||
g_progress_xferred = 0;
|
g_progress_xferred = 0;
|
||||||
g_progress_seen = 0;
|
g_progress_seen = 0;
|
||||||
if (!g_progress_active) {
|
if (!g_progress_active) {
|
||||||
@@ -3307,7 +3307,8 @@ int send_files(Config* config) {
|
|||||||
/* rsync default: a scan I/O error suppresses deletion unless
|
/* rsync default: a scan I/O error suppresses deletion unless
|
||||||
--ignore-errors, even in the late (commit) modes. Drop the keep-set so
|
--ignore-errors, even in the late (commit) modes. Drop the keep-set so
|
||||||
the receiver removes nothing; the readable tree still transferred. */
|
the receiver removes nothing; the readable tree still transferred. */
|
||||||
bool late_delete = (manifest || config->delete_missing_args) && !delete_early && !delete_per_dir;
|
bool late_delete =
|
||||||
|
(manifest || config->delete_missing_args) && !delete_early && !delete_per_dir;
|
||||||
if (late_delete && !ignore_errors_allows_delete(config, had_scan_io)) {
|
if (late_delete && !ignore_errors_allows_delete(config, had_scan_io)) {
|
||||||
log_message(LOG_LEVEL_WARNING, "IO error encountered -- skipping file deletion");
|
log_message(LOG_LEVEL_WARNING, "IO error encountered -- skipping file deletion");
|
||||||
if (manifest) {
|
if (manifest) {
|
||||||
|
|||||||
@@ -218,7 +218,7 @@ typedef struct {
|
|||||||
int completed;
|
int completed;
|
||||||
Chunk* initial_chunk;
|
Chunk* initial_chunk;
|
||||||
ProtocolSession* allocation_session;
|
ProtocolSession* allocation_session;
|
||||||
FilterNode* root_filter_node; /* root .rsync-filter context (owned by ps) */
|
FilterNode* root_filter_node; /* root .rsync-filter context (owned by ps) */
|
||||||
const ScannerOptions* options; /* borrowed scan options (--info=nonreg output) */
|
const ScannerOptions* options; /* borrowed scan options (--info=nonreg output) */
|
||||||
} ParallelScanner;
|
} ParallelScanner;
|
||||||
|
|
||||||
|
|||||||
@@ -417,8 +417,7 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver
|
|||||||
--max-delete-capped commit still succeeds and the transfer proceeds;
|
--max-delete-capped commit still succeeds and the transfer proceeds;
|
||||||
the terminal success frame reports the cap. */
|
the terminal success frame reports the cap. */
|
||||||
size_t deleted = 0;
|
size_t deleted = 0;
|
||||||
DeletePathObserver observer =
|
DeletePathObserver observer = sink->deleted_paths ? receiver_record_deleted_path : NULL;
|
||||||
sink->deleted_paths ? receiver_record_deleted_path : NULL;
|
|
||||||
DeleteCommitResult deletion =
|
DeleteCommitResult deletion =
|
||||||
(config->use_delete || config->delete_missing_args)
|
(config->use_delete || config->delete_missing_args)
|
||||||
? manifest_delete_all_observed(config, manifest, &deleted, observer,
|
? manifest_delete_all_observed(config, manifest, &deleted, observer,
|
||||||
@@ -506,8 +505,7 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver
|
|||||||
deferred_manifest = NULL;
|
deferred_manifest = NULL;
|
||||||
} else {
|
} else {
|
||||||
size_t deleted = 0;
|
size_t deleted = 0;
|
||||||
DeletePathObserver observer =
|
DeletePathObserver observer = sink->deleted_paths ? receiver_record_deleted_path : NULL;
|
||||||
sink->deleted_paths ? receiver_record_deleted_path : NULL;
|
|
||||||
DeleteCommitResult deletion = manifest_delete_all_observed(
|
DeleteCommitResult deletion = manifest_delete_all_observed(
|
||||||
config, deferred_manifest, &deleted, observer, (void*)sink->deleted_paths);
|
config, deferred_manifest, &deleted, observer, (void*)sink->deleted_paths);
|
||||||
receiver_tally_deleted(sink, deleted);
|
receiver_tally_deleted(sink, deleted);
|
||||||
|
|||||||
+5
-8
@@ -956,11 +956,9 @@ void handler(int file_descriptor) {
|
|||||||
server-contacting --dry-run deletes nothing (no manifest is sent). */
|
server-contacting --dry-run deletes nothing (no manifest is sent). */
|
||||||
if (context->deferred_manifest) {
|
if (context->deferred_manifest) {
|
||||||
size_t deleted = 0;
|
size_t deleted = 0;
|
||||||
DeletePathObserver observer =
|
DeletePathObserver observer = config->report_deletes ? receiver_record_deleted_path : NULL;
|
||||||
config->report_deletes ? receiver_record_deleted_path : NULL;
|
DeleteCommitResult deletion = manifest_delete_all_observed(
|
||||||
DeleteCommitResult deletion =
|
config, context->deferred_manifest, &deleted, observer, (void*)context->deleted_paths);
|
||||||
manifest_delete_all_observed(config, context->deferred_manifest, &deleted, observer,
|
|
||||||
(void*)context->deleted_paths);
|
|
||||||
context->stats.deleted_files += deleted;
|
context->stats.deleted_files += deleted;
|
||||||
if (deletion == DELETE_COMMIT_ERROR) {
|
if (deletion == DELETE_COMMIT_ERROR) {
|
||||||
transfer_ok = false;
|
transfer_ok = false;
|
||||||
@@ -979,9 +977,8 @@ void handler(int file_descriptor) {
|
|||||||
/* Defence in depth (the enclosing block already excludes dry-run): a
|
/* Defence in depth (the enclosing block already excludes dry-run): a
|
||||||
-n run never commits a deletion. */
|
-n run never commits a deletion. */
|
||||||
if (config->report_deletes)
|
if (config->report_deletes)
|
||||||
delete_plan_session_set_delete_observer(context->deferred_plans,
|
delete_plan_session_set_delete_observer(
|
||||||
receiver_record_deleted_path,
|
context->deferred_plans, receiver_record_deleted_path, (void*)context->deleted_paths);
|
||||||
(void*)context->deleted_paths);
|
|
||||||
DeleteCommitResult deletion =
|
DeleteCommitResult deletion =
|
||||||
config->dry_run ? DELETE_COMMIT_OK
|
config->dry_run ? DELETE_COMMIT_OK
|
||||||
: delete_plan_session_commit(context->deferred_plans, config);
|
: delete_plan_session_commit(context->deferred_plans, config);
|
||||||
|
|||||||
+2
-2
@@ -268,8 +268,8 @@ typedef enum SuperMode { SUPER_MODE_AUTO = 0, SUPER_MODE_ON = 1, SUPER_MODE_OFF
|
|||||||
* real (non-dry-run) deletion. It is set only when --info=del is requested with
|
* real (non-dry-run) deletion. It is set only when --info=del is requested with
|
||||||
* --delete; the transfer decision itself is unchanged. */
|
* --delete; the transfer decision itself is unchanged. */
|
||||||
#define CONFIG_WIRE_OUTPUT_FIELDS(X) \
|
#define CONFIG_WIRE_OUTPUT_FIELDS(X) \
|
||||||
X(report_dest_info, bool, false, BOOL) X(report_stats, bool, false, BOOL) \
|
X(report_dest_info, bool, false, BOOL) \
|
||||||
X(report_deletes, bool, false, BOOL)
|
X(report_stats, bool, false, BOOL) X(report_deletes, bool, false, BOOL)
|
||||||
|
|
||||||
/* Codec-negotiation wave (protocol 2.26.0). compression_algo is the concrete
|
/* Codec-negotiation wave (protocol 2.26.0). compression_algo is the concrete
|
||||||
* codec the client selected for this transfer (a CompressionAlgo id) and is the
|
* codec the client selected for this transfer (a CompressionAlgo id) and is the
|
||||||
|
|||||||
@@ -791,9 +791,9 @@ static bool apply_missing(DeletePlanSession* session, const Config* config) {
|
|||||||
size_t deleted = 0;
|
size_t deleted = 0;
|
||||||
size_t skipped = 0;
|
size_t skipped = 0;
|
||||||
bool limit = false;
|
bool limit = false;
|
||||||
bool ok = manifest_delete_missing_args_limited_observed(
|
bool ok = manifest_delete_missing_args_limited_observed(config, &manifest, remaining, &deleted,
|
||||||
config, &manifest, remaining, &deleted, &skipped, &limit, session->observer,
|
&skipped, &limit, session->observer,
|
||||||
session->observer_context);
|
session->observer_context);
|
||||||
session->deleted += deleted;
|
session->deleted += deleted;
|
||||||
session->planned += deleted;
|
session->planned += deleted;
|
||||||
session->skipped += skipped;
|
session->skipped += skipped;
|
||||||
@@ -893,8 +893,8 @@ static bool apply_deferred_path(DeletePlanSession* session, const Config* config
|
|||||||
return ok;
|
return ok;
|
||||||
}
|
}
|
||||||
|
|
||||||
void delete_plan_session_set_delete_observer(DeletePlanSession* session, DeletePathObserver observer,
|
void delete_plan_session_set_delete_observer(DeletePlanSession* session,
|
||||||
void* context) {
|
DeletePathObserver observer, void* context) {
|
||||||
if (!session)
|
if (!session)
|
||||||
return;
|
return;
|
||||||
session->observer = observer;
|
session->observer = observer;
|
||||||
|
|||||||
@@ -84,7 +84,7 @@ size_t delete_plan_session_deleted(const DeletePlanSession* session);
|
|||||||
truly removes (including the deferred --delete-delay commit), so the receiver
|
truly removes (including the deferred --delete-delay commit), so the receiver
|
||||||
can report rsync's `deleting PATH` lines through the terminal STATUS_STATS
|
can report rsync's `deleting PATH` lines through the terminal STATUS_STATS
|
||||||
record. Pass NULL/0 to clear. */
|
record. Pass NULL/0 to clear. */
|
||||||
void delete_plan_session_set_delete_observer(DeletePlanSession* session, DeletePathObserver observer,
|
void delete_plan_session_set_delete_observer(DeletePlanSession* session,
|
||||||
void* context);
|
DeletePathObserver observer, void* context);
|
||||||
|
|
||||||
#endif
|
#endif
|
||||||
|
|||||||
@@ -3261,10 +3261,9 @@ static bool delete_extras_budgeted_observed(const Config* config, DeleteManifest
|
|||||||
remaining = budget->max_delete - budget->deleted;
|
remaining = budget->max_delete - budget->deleted;
|
||||||
size_t deleted = 0;
|
size_t deleted = 0;
|
||||||
size_t skipped = 0;
|
size_t skipped = 0;
|
||||||
DeleteWalkResult result =
|
DeleteWalkResult result = delete_extras_limited_observed(
|
||||||
delete_extras_limited_observed(config->receive_root_directory, manifest->keeps, manifest->dirs,
|
config->receive_root_directory, manifest->keeps, manifest->dirs, remaining, skips, used,
|
||||||
remaining, skips, used, &deleted, &skipped, observer,
|
&deleted, &skipped, observer, observer_context);
|
||||||
observer_context);
|
|
||||||
if (owned_prefixes) {
|
if (owned_prefixes) {
|
||||||
for (int i = 0; i < config->basis_count; i++)
|
for (int i = 0; i < config->basis_count; i++)
|
||||||
free(owned_prefixes[i]);
|
free(owned_prefixes[i]);
|
||||||
@@ -3592,8 +3591,8 @@ bool manifest_delete_missing_args_limited_observed(const Config* config, DeleteM
|
|||||||
void* observer_context) {
|
void* observer_context) {
|
||||||
DeleteBudgetState budget = {
|
DeleteBudgetState budget = {
|
||||||
.max_delete = max_delete, .deleted = 0, .skipped = 0, .limit_hit = false};
|
.max_delete = max_delete, .deleted = 0, .skipped = 0, .limit_hit = false};
|
||||||
bool ok = delete_missing_args_budgeted_observed(config, manifest, &budget, observer,
|
bool ok =
|
||||||
observer_context);
|
delete_missing_args_budgeted_observed(config, manifest, &budget, observer, observer_context);
|
||||||
if (deleted)
|
if (deleted)
|
||||||
*deleted = budget.deleted;
|
*deleted = budget.deleted;
|
||||||
if (skipped)
|
if (skipped)
|
||||||
|
|||||||
@@ -132,7 +132,8 @@ bool manifest_delete_missing_args_limited(const Config* config, DeleteManifest*
|
|||||||
bool manifest_delete_missing_args_limited_observed(const Config* config, DeleteManifest* manifest,
|
bool manifest_delete_missing_args_limited_observed(const Config* config, DeleteManifest* manifest,
|
||||||
size_t max_delete, size_t* deleted,
|
size_t max_delete, size_t* deleted,
|
||||||
size_t* skipped, bool* limit_hit,
|
size_t* skipped, bool* limit_hit,
|
||||||
DeletePathObserver observer, void* observer_context);
|
DeletePathObserver observer,
|
||||||
|
void* observer_context);
|
||||||
/* Outcome of committing a delete manifest. LIMIT_REACHED reports rsync's
|
/* Outcome of committing a delete manifest. LIMIT_REACHED reports rsync's
|
||||||
partial --max-delete result: the budget allowed some deletions and the rest
|
partial --max-delete result: the budget allowed some deletions and the rest
|
||||||
were skipped (the run still stores all file data but the client exits 25). */
|
were skipped (the run still stores all file data but the client exits 25). */
|
||||||
|
|||||||
+2
-1
@@ -876,7 +876,8 @@ DeleteWalkResult delete_extras_limited_observed(const char* dest_root, const Arr
|
|||||||
const ArrayList* synced_dirs, size_t max_delete,
|
const ArrayList* synced_dirs, size_t max_delete,
|
||||||
const DeleteSkipEntry* skips, int skip_count,
|
const DeleteSkipEntry* skips, int skip_count,
|
||||||
size_t* deleted_out, size_t* skipped_out,
|
size_t* deleted_out, size_t* skipped_out,
|
||||||
DeletePathObserver observer, void* observer_context) {
|
DeletePathObserver observer,
|
||||||
|
void* observer_context) {
|
||||||
if (deleted_out)
|
if (deleted_out)
|
||||||
*deleted_out = 0;
|
*deleted_out = 0;
|
||||||
if (skipped_out)
|
if (skipped_out)
|
||||||
|
|||||||
+2
-1
@@ -145,7 +145,8 @@ DeleteWalkResult delete_extras_limited_observed(const char* dest_root, const Arr
|
|||||||
const ArrayList* synced_dirs, size_t max_delete,
|
const ArrayList* synced_dirs, size_t max_delete,
|
||||||
const DeleteSkipEntry* skips, int skip_count,
|
const DeleteSkipEntry* skips, int skip_count,
|
||||||
size_t* deleted_out, size_t* skipped_out,
|
size_t* deleted_out, size_t* skipped_out,
|
||||||
DeletePathObserver observer, void* observer_context);
|
DeletePathObserver observer,
|
||||||
|
void* observer_context);
|
||||||
|
|
||||||
DeleteWalkResult delete_extras_limited(const char* dest_root, const ArrayList* manifest,
|
DeleteWalkResult delete_extras_limited(const char* dest_root, const ArrayList* manifest,
|
||||||
const ArrayList* synced_dirs, size_t max_delete,
|
const ArrayList* synced_dirs, size_t max_delete,
|
||||||
|
|||||||
@@ -0,0 +1,455 @@
|
|||||||
|
"""Differential parity tests for the option wave (bwlimit, --info=*, -M,
|
||||||
|
--ignore-errors, --filter protect).
|
||||||
|
|
||||||
|
Every differential here runs the SAME scenario with real ``rsync 3.4.1`` and
|
||||||
|
with fastsync and compares the observable result, so the modules are skipped
|
||||||
|
when rsync is unavailable. The privilege-dependent --ignore-errors differential
|
||||||
|
drops the client to an unprivileged uid so a mode-000 source directory is
|
||||||
|
genuinely unreadable; it is marked ``setpriv`` (run as root locally, excluded
|
||||||
|
from the root PR gate exactly like the other privilege tests).
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
sys.path.insert(0, os.path.dirname(__file__))
|
||||||
|
from common import ( # noqa: E402
|
||||||
|
CLIENT_CMD,
|
||||||
|
TEST_DATA_DIR,
|
||||||
|
ServerManager,
|
||||||
|
clean_dir,
|
||||||
|
get_dest_received_dir,
|
||||||
|
run_client,
|
||||||
|
)
|
||||||
|
|
||||||
|
RSYNC = shutil.which("rsync")
|
||||||
|
requires_rsync = pytest.mark.skipif(RSYNC is None, reason="rsync 3.4.1 not installed")
|
||||||
|
|
||||||
|
|
||||||
|
def _rsync(args, timeout=120, as_nobody=False):
|
||||||
|
env = dict(os.environ, LC_ALL="C")
|
||||||
|
cmd = [RSYNC] + args
|
||||||
|
if as_nobody:
|
||||||
|
cmd = ["setpriv", "--reuid=65534", "--regid=65534", "--clear-groups"] + cmd
|
||||||
|
return subprocess.run(cmd, capture_output=True, text=True, env=env, timeout=timeout)
|
||||||
|
|
||||||
|
|
||||||
|
def _write(path, content):
|
||||||
|
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||||
|
with open(path, "wb") as fh:
|
||||||
|
fh.write(content)
|
||||||
|
|
||||||
|
|
||||||
|
class TestBwlimitParity:
|
||||||
|
"""--bwlimit must accept rsync 3.4.1's spellings and pace like it."""
|
||||||
|
|
||||||
|
ACCEPTED = ["100", "0", "1.5", "100K", "100KB", "100KiB", "1M", "1MB", "1m", "1G", "512"]
|
||||||
|
REJECTED = ["-1", "abc", "1x", "1 000"]
|
||||||
|
|
||||||
|
@requires_rsync
|
||||||
|
@pytest.mark.ci
|
||||||
|
def test_parse_acceptance_matches_rsync(self, shared_server):
|
||||||
|
source = os.path.join(TEST_DATA_DIR, "bwp_src")
|
||||||
|
clean_dir(source)
|
||||||
|
_write(os.path.join(source, "f.txt"), b"payload\n")
|
||||||
|
|
||||||
|
for value in self.ACCEPTED + self.REJECTED:
|
||||||
|
rdst = os.path.join(TEST_DATA_DIR, "bwp_rdst")
|
||||||
|
clean_dir(rdst)
|
||||||
|
rsync_result = _rsync(["-a", "--bwlimit=" + value, source + "/", rdst + "/"])
|
||||||
|
dest = os.path.join(TEST_DATA_DIR, "bwp_dst")
|
||||||
|
clean_dir(dest)
|
||||||
|
result, _ = run_client(source, dest, flags=["-a", "--bwlimit=" + value],
|
||||||
|
port=shared_server.port)
|
||||||
|
assert (result.returncode == 0) == (rsync_result.returncode == 0), (
|
||||||
|
f"--bwlimit={value}: fastsync rc={result.returncode} "
|
||||||
|
f"({(result.stderr or result.stdout)[:120]!r}) "
|
||||||
|
f"rsync rc={rsync_result.returncode} ({rsync_result.stderr[:120]!r})"
|
||||||
|
)
|
||||||
|
|
||||||
|
@requires_rsync
|
||||||
|
@pytest.mark.ci
|
||||||
|
def test_throttle_rate_matches_rsync(self, shared_server):
|
||||||
|
"""A 4 MiB transfer at --bwlimit=2048 (2 MiB/s) must take about the same
|
||||||
|
wall-clock time for both tools (~2 s with rsync's leaky bucket)."""
|
||||||
|
source = os.path.join(TEST_DATA_DIR, "bwt_src")
|
||||||
|
clean_dir(source)
|
||||||
|
_write(os.path.join(source, "big.bin"), os.urandom(4 * 1024 * 1024))
|
||||||
|
dest = os.path.join(TEST_DATA_DIR, "bwt_dst")
|
||||||
|
rdst = os.path.join(TEST_DATA_DIR, "bwt_rdst")
|
||||||
|
|
||||||
|
clean_dir(rdst)
|
||||||
|
start = time.monotonic()
|
||||||
|
rsync_result = _rsync(["-a", "--bwlimit=2048", source + "/", rdst + "/"])
|
||||||
|
rsync_secs = time.monotonic() - start
|
||||||
|
assert rsync_result.returncode == 0, rsync_result.stderr
|
||||||
|
|
||||||
|
clean_dir(dest)
|
||||||
|
result, fast_secs = run_client(source, dest, flags=["-a", "--bwlimit=2048"],
|
||||||
|
port=shared_server.port)
|
||||||
|
assert result.returncode == 0, (result.stderr or result.stdout)[:200]
|
||||||
|
assert fast_secs > 1.0, f"fastsync throttled too little: {fast_secs:.2f}s"
|
||||||
|
# Both rendezvous near 2 s; allow a generous band for CI scheduling.
|
||||||
|
assert abs(fast_secs - rsync_secs) < 1.0, (
|
||||||
|
f"fastsync {fast_secs:.2f}s vs rsync {rsync_secs:.2f}s"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _output_tree(root):
|
||||||
|
clean_dir(root)
|
||||||
|
os.makedirs(os.path.join(root, "sub"))
|
||||||
|
_write(os.path.join(root, "a.txt"), b"top\n")
|
||||||
|
_write(os.path.join(root, "sub", "b.txt"), b"nested\n")
|
||||||
|
os.symlink("a.txt", os.path.join(root, "link"))
|
||||||
|
|
||||||
|
|
||||||
|
class TestInfoParity:
|
||||||
|
"""The --info categories that map to a FastSync event must print rsync's
|
||||||
|
line format."""
|
||||||
|
|
||||||
|
@requires_rsync
|
||||||
|
@pytest.mark.ci
|
||||||
|
def test_info_flist_matches_rsync(self, shared_server):
|
||||||
|
source = os.path.join(TEST_DATA_DIR, "inf_fl_src")
|
||||||
|
dest = os.path.join(TEST_DATA_DIR, "inf_fl_dst")
|
||||||
|
rdst = os.path.join(TEST_DATA_DIR, "inf_fl_rdst")
|
||||||
|
_output_tree(source)
|
||||||
|
clean_dir(dest)
|
||||||
|
clean_dir(rdst)
|
||||||
|
rsync_result = _rsync(["-a", "--info=flist", source + "/", rdst + "/"])
|
||||||
|
assert rsync_result.returncode == 0, rsync_result.stderr
|
||||||
|
result, _ = run_client(source, dest, flags=["-a", "--info=flist"],
|
||||||
|
port=shared_server.port)
|
||||||
|
assert result.returncode == 0, (result.stderr or result.stdout)[:200]
|
||||||
|
assert "sending incremental file list" in result.stdout
|
||||||
|
assert "sending incremental file list" in rsync_result.stdout
|
||||||
|
|
||||||
|
@requires_rsync
|
||||||
|
@pytest.mark.ci
|
||||||
|
def test_info_name_matches_rsync(self, shared_server):
|
||||||
|
source = os.path.join(TEST_DATA_DIR, "inf_nm_src")
|
||||||
|
dest = os.path.join(TEST_DATA_DIR, "inf_nm_dst")
|
||||||
|
rdst = os.path.join(TEST_DATA_DIR, "inf_nm_rdst")
|
||||||
|
_output_tree(source)
|
||||||
|
clean_dir(dest)
|
||||||
|
clean_dir(rdst)
|
||||||
|
rsync_result = _rsync(["-a", "--info=name", source + "/", rdst + "/"])
|
||||||
|
assert rsync_result.returncode == 0, rsync_result.stderr
|
||||||
|
result, _ = run_client(source, dest, flags=["-a", "--info=name"],
|
||||||
|
port=shared_server.port)
|
||||||
|
assert result.returncode == 0, (result.stderr or result.stdout)[:200]
|
||||||
|
|
||||||
|
def entries(text):
|
||||||
|
# Compare the transferred entries only: rsync also prints the
|
||||||
|
# transfer-root `./` and every directory (FastSync records dirs),
|
||||||
|
# which are a separate documented divergence.
|
||||||
|
out = []
|
||||||
|
for line in text.splitlines():
|
||||||
|
if not line or line.startswith("sending ") or line.startswith("created "):
|
||||||
|
continue
|
||||||
|
if line == "./" or line.endswith("/"):
|
||||||
|
continue
|
||||||
|
out.append(line)
|
||||||
|
return sorted(out)
|
||||||
|
|
||||||
|
assert entries(result.stdout) == entries(rsync_result.stdout), (
|
||||||
|
f"rsync={entries(rsync_result.stdout)} fastsync={entries(result.stdout)}"
|
||||||
|
)
|
||||||
|
|
||||||
|
@requires_rsync
|
||||||
|
@pytest.mark.ci
|
||||||
|
def test_info_nonreg_matches_rsync(self, shared_server):
|
||||||
|
source = os.path.join(TEST_DATA_DIR, "inf_nr_src")
|
||||||
|
dest = os.path.join(TEST_DATA_DIR, "inf_nr_dst")
|
||||||
|
rdst = os.path.join(TEST_DATA_DIR, "inf_nr_rdst")
|
||||||
|
clean_dir(source)
|
||||||
|
os.mkfifo(os.path.join(source, "fifo"))
|
||||||
|
_write(os.path.join(source, "a.txt"), b"a\n")
|
||||||
|
clean_dir(dest)
|
||||||
|
clean_dir(rdst)
|
||||||
|
rsync_result = _rsync(["-rlt", "--info=nonreg", source + "/", rdst + "/"])
|
||||||
|
assert rsync_result.returncode == 0, rsync_result.stderr
|
||||||
|
result, _ = run_client(source, dest, flags=["-rlt", "--info=nonreg"],
|
||||||
|
port=shared_server.port)
|
||||||
|
assert result.returncode == 0, (result.stderr or result.stdout)[:200]
|
||||||
|
rsync_lines = sorted(l for l in rsync_result.stdout.splitlines()
|
||||||
|
if l.startswith("skipping non-regular"))
|
||||||
|
fast_lines = sorted(l for l in result.stdout.splitlines()
|
||||||
|
if l.startswith("skipping non-regular"))
|
||||||
|
assert fast_lines == rsync_lines, (rsync_lines, fast_lines)
|
||||||
|
assert fast_lines, "no non-regular skip line emitted"
|
||||||
|
|
||||||
|
@requires_rsync
|
||||||
|
@pytest.mark.ci
|
||||||
|
def test_info_del_real_matches_rsync(self, shared_server):
|
||||||
|
source = os.path.join(TEST_DATA_DIR, "inf_dl_src")
|
||||||
|
dest = os.path.join(TEST_DATA_DIR, "inf_dl_dst")
|
||||||
|
rdst = os.path.join(TEST_DATA_DIR, "inf_dl_rdst")
|
||||||
|
clean_dir(source)
|
||||||
|
_write(os.path.join(source, "keep.txt"), b"keep\n")
|
||||||
|
clean_dir(rdst)
|
||||||
|
_write(os.path.join(rdst, "extra.txt"), b"x\n")
|
||||||
|
_write(os.path.join(rdst, "extra2.txt"), b"y\n")
|
||||||
|
rsync_result = _rsync(["-a", "--delete", "--info=del", source + "/", rdst + "/"])
|
||||||
|
assert rsync_result.returncode == 0, rsync_result.stderr
|
||||||
|
rsync_lines = sorted(l for l in rsync_result.stdout.splitlines()
|
||||||
|
if l.startswith("deleting "))
|
||||||
|
|
||||||
|
clean_dir(dest)
|
||||||
|
received = get_dest_received_dir(dest, source)
|
||||||
|
_write(os.path.join(received, "extra.txt"), b"x\n")
|
||||||
|
_write(os.path.join(received, "extra2.txt"), b"y\n")
|
||||||
|
with ServerManager() as server:
|
||||||
|
server.start(extra_args=["--allow-delete"])
|
||||||
|
result, _ = run_client(source, dest, flags=["-a", "--delete", "--info=del"],
|
||||||
|
port=server.port)
|
||||||
|
assert result.returncode == 0, (result.stderr or result.stdout)[:200]
|
||||||
|
fast_lines = sorted(l for l in result.stdout.splitlines()
|
||||||
|
if l.startswith("deleting "))
|
||||||
|
assert fast_lines == rsync_lines, (rsync_lines, fast_lines)
|
||||||
|
assert fast_lines, "no deletion lines emitted"
|
||||||
|
|
||||||
|
@requires_rsync
|
||||||
|
@pytest.mark.ci
|
||||||
|
def test_info_del_itemize_real_matches_rsync(self, shared_server):
|
||||||
|
source = os.path.join(TEST_DATA_DIR, "inf_di_src")
|
||||||
|
dest = os.path.join(TEST_DATA_DIR, "inf_di_dst")
|
||||||
|
rdst = os.path.join(TEST_DATA_DIR, "inf_di_rdst")
|
||||||
|
clean_dir(source)
|
||||||
|
_write(os.path.join(source, "keep.txt"), b"keep\n")
|
||||||
|
clean_dir(rdst)
|
||||||
|
_write(os.path.join(rdst, "extra.txt"), b"x\n")
|
||||||
|
rsync_result = _rsync(["-a", "-i", "--delete", source + "/", rdst + "/"])
|
||||||
|
assert rsync_result.returncode == 0, rsync_result.stderr
|
||||||
|
rsync_lines = sorted(l for l in rsync_result.stdout.splitlines()
|
||||||
|
if l.startswith("*deleting"))
|
||||||
|
|
||||||
|
clean_dir(dest)
|
||||||
|
received = get_dest_received_dir(dest, source)
|
||||||
|
_write(os.path.join(received, "extra.txt"), b"x\n")
|
||||||
|
with ServerManager() as server:
|
||||||
|
server.start(extra_args=["--allow-delete"])
|
||||||
|
result, _ = run_client(source, dest, flags=["-a", "-i", "--delete"],
|
||||||
|
port=server.port)
|
||||||
|
assert result.returncode == 0, (result.stderr or result.stdout)[:200]
|
||||||
|
fast_lines = sorted(l for l in result.stdout.splitlines()
|
||||||
|
if l.startswith("*deleting"))
|
||||||
|
assert fast_lines == rsync_lines, (rsync_lines, fast_lines)
|
||||||
|
|
||||||
|
@requires_rsync
|
||||||
|
@pytest.mark.ci
|
||||||
|
def test_info_del_dry_run_matches_rsync(self, shared_server):
|
||||||
|
source = os.path.join(TEST_DATA_DIR, "inf_dd_src")
|
||||||
|
dest = os.path.join(TEST_DATA_DIR, "inf_dd_dst")
|
||||||
|
rdst = os.path.join(TEST_DATA_DIR, "inf_dd_rdst")
|
||||||
|
clean_dir(source)
|
||||||
|
_write(os.path.join(source, "keep.txt"), b"keep\n")
|
||||||
|
clean_dir(rdst)
|
||||||
|
_write(os.path.join(rdst, "extra.txt"), b"x\n")
|
||||||
|
rsync_result = _rsync(["-a", "-n", "--delete", "--info=del", source + "/", rdst + "/"])
|
||||||
|
assert rsync_result.returncode == 0, rsync_result.stderr
|
||||||
|
rsync_lines = sorted(l for l in rsync_result.stdout.splitlines()
|
||||||
|
if l.startswith("deleting "))
|
||||||
|
|
||||||
|
clean_dir(dest)
|
||||||
|
received = get_dest_received_dir(dest, source)
|
||||||
|
_write(os.path.join(received, "extra.txt"), b"x\n")
|
||||||
|
with ServerManager() as server:
|
||||||
|
server.start(extra_args=["--allow-delete"])
|
||||||
|
result, _ = run_client(source, dest, flags=["-a", "-n", "--delete", "--info=del"],
|
||||||
|
port=server.port)
|
||||||
|
assert result.returncode == 0, (result.stderr or result.stdout)[:200]
|
||||||
|
fast_lines = sorted(l for l in result.stdout.splitlines()
|
||||||
|
if l.startswith("deleting "))
|
||||||
|
assert fast_lines == rsync_lines, (rsync_lines, fast_lines)
|
||||||
|
|
||||||
|
@requires_rsync
|
||||||
|
@pytest.mark.ci
|
||||||
|
def test_info_remove_matches_rsync(self, shared_server):
|
||||||
|
tag = "inf_rm"
|
||||||
|
rsync_src = os.path.join(TEST_DATA_DIR, f"{tag}_rsrc")
|
||||||
|
rsync_dst = os.path.join(TEST_DATA_DIR, f"{tag}_rdst")
|
||||||
|
fast_src = os.path.join(TEST_DATA_DIR, f"{tag}_fsrc")
|
||||||
|
fast_dst = os.path.join(TEST_DATA_DIR, f"{tag}_fdst")
|
||||||
|
for root in (rsync_src, rsync_dst, fast_src, fast_dst):
|
||||||
|
clean_dir(root)
|
||||||
|
_write(os.path.join(rsync_src, "a.txt"), b"a\n")
|
||||||
|
_write(os.path.join(rsync_src, "sub", "b.txt"), b"b\n")
|
||||||
|
_write(os.path.join(fast_src, "a.txt"), b"a\n")
|
||||||
|
_write(os.path.join(fast_src, "sub", "b.txt"), b"b\n")
|
||||||
|
|
||||||
|
rsync_result = _rsync(["-a", "--remove-source-files", "--info=remove",
|
||||||
|
rsync_src + "/", rsync_dst + "/"])
|
||||||
|
assert rsync_result.returncode == 0, rsync_result.stderr
|
||||||
|
rsync_lines = sorted(l for l in rsync_result.stdout.splitlines()
|
||||||
|
if l.startswith("sender removed "))
|
||||||
|
|
||||||
|
result, _ = run_client(fast_src, fast_dst,
|
||||||
|
flags=["-a", "--remove-source-files", "--info=remove"],
|
||||||
|
port=shared_server.port)
|
||||||
|
assert result.returncode == 0, (result.stderr or result.stdout)[:200]
|
||||||
|
fast_lines = sorted(l for l in result.stdout.splitlines()
|
||||||
|
if l.startswith("sender removed "))
|
||||||
|
assert fast_lines == rsync_lines, (rsync_lines, fast_lines)
|
||||||
|
assert fast_lines, "no source-removal lines emitted"
|
||||||
|
|
||||||
|
|
||||||
|
class TestIgnoreErrorsParity:
|
||||||
|
"""--ignore-errors: a source I/O error skips deletion by default; the flag
|
||||||
|
lets deletion proceed. Both exit 23. Run the client as an unprivileged user
|
||||||
|
so the mode-000 directory is genuinely unreadable."""
|
||||||
|
|
||||||
|
@pytest.mark.setpriv
|
||||||
|
def test_delete_after_io_error_matches_rsync(self):
|
||||||
|
if os.geteuid() != 0 or shutil.which("setpriv") is None:
|
||||||
|
pytest.skip("requires root + setpriv to drop privileges for the client")
|
||||||
|
tag = f"ie_{os.getpid()}"
|
||||||
|
source = os.path.join(TEST_DATA_DIR, f"{tag}_src")
|
||||||
|
rsync_dst = os.path.join(TEST_DATA_DIR, f"{tag}_rdst")
|
||||||
|
dest = os.path.join(TEST_DATA_DIR, f"{tag}_dst")
|
||||||
|
clean_dir(source)
|
||||||
|
clean_dir(rsync_dst)
|
||||||
|
clean_dir(dest)
|
||||||
|
_write(os.path.join(source, "top.txt"), b"top\n")
|
||||||
|
_write(os.path.join(source, "locked", "blocked.txt"), b"blocked\n")
|
||||||
|
os.chmod(os.path.join(source, "locked"), 0)
|
||||||
|
os.chmod(TEST_DATA_DIR, 0o777)
|
||||||
|
os.chmod(source, 0o755)
|
||||||
|
os.chmod(rsync_dst, 0o777)
|
||||||
|
os.chmod(dest, 0o777)
|
||||||
|
try:
|
||||||
|
for ignore in (False, True):
|
||||||
|
flags = ["-a", "--delete-after"] + (["--ignore-errors"] if ignore else [])
|
||||||
|
# rsync side
|
||||||
|
_write(os.path.join(rsync_dst, "extra.txt"), b"x\n")
|
||||||
|
os.chmod(os.path.join(rsync_dst, "extra.txt"), 0o666)
|
||||||
|
rres = _rsync(flags + [source + "/", rsync_dst + "/"], as_nobody=True)
|
||||||
|
rsync_extra = os.path.exists(os.path.join(rsync_dst, "extra.txt"))
|
||||||
|
|
||||||
|
# fastsync side
|
||||||
|
received = get_dest_received_dir(dest, source)
|
||||||
|
_write(os.path.join(received, "extra.txt"), b"x\n")
|
||||||
|
os.chmod(os.path.join(received, "extra.txt"), 0o666)
|
||||||
|
with ServerManager() as server:
|
||||||
|
server.start(extra_args=["--allow-delete"])
|
||||||
|
fflags = (["--delete", "--ignore-errors"] if ignore else ["--delete"])
|
||||||
|
cmd = CLIENT_CMD + ["--source-dir", source, "--dest-dir", dest,
|
||||||
|
"--save-to-disk", "--server-port", str(server.port)] + fflags
|
||||||
|
fres = subprocess.run(
|
||||||
|
["setpriv", "--reuid=65534", "--regid=65534", "--clear-groups"] + cmd,
|
||||||
|
text=True, capture_output=True)
|
||||||
|
fast_extra = os.path.exists(os.path.join(received, "extra.txt"))
|
||||||
|
|
||||||
|
assert rres.returncode == 23, (ignore, rres.returncode, rres.stderr[:200])
|
||||||
|
assert fres.returncode == 23, (ignore, fres.returncode, fres.stderr[:200])
|
||||||
|
assert rsync_extra == fast_extra, (
|
||||||
|
f"ignore_errors={ignore}: rsync extra={rsync_extra} fastsync extra={fast_extra}"
|
||||||
|
)
|
||||||
|
assert fast_extra is (not ignore), (ignore, fast_extra)
|
||||||
|
finally:
|
||||||
|
os.chmod(os.path.join(source, "locked"), 0o755)
|
||||||
|
|
||||||
|
|
||||||
|
class TestRemoteOptionDaemon:
|
||||||
|
"""rsync forwards -M/--remote-option to its remote process over a daemon
|
||||||
|
connection; FastSync's daemon has no per-connection argv channel and rejects
|
||||||
|
it. This pins the documented divergence with evidence."""
|
||||||
|
|
||||||
|
@requires_rsync
|
||||||
|
def test_rsync_forwards_M_over_daemon_and_fastsync_rejects(self, tmp_path):
|
||||||
|
import socket
|
||||||
|
|
||||||
|
with socket.socket() as probe:
|
||||||
|
probe.bind(("127.0.0.1", 0))
|
||||||
|
port = probe.getsockname()[1]
|
||||||
|
|
||||||
|
module_root = tmp_path / "mod"
|
||||||
|
module_root.mkdir()
|
||||||
|
os.chmod(module_root, 0o777)
|
||||||
|
source = tmp_path / "src"
|
||||||
|
source.mkdir()
|
||||||
|
(source / "a.txt").write_bytes(b"hello\n")
|
||||||
|
conf = tmp_path / "rsyncd.conf"
|
||||||
|
conf.write_text(
|
||||||
|
f"port = {port}\nuse chroot = no\n[m]\npath = {module_root}\nread only = no\n"
|
||||||
|
)
|
||||||
|
daemon = subprocess.Popen(
|
||||||
|
[RSYNC, "--daemon", "--no-detach", "--port", str(port), "--config", str(conf)],
|
||||||
|
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||||
|
try:
|
||||||
|
deadline = time.monotonic() + 5
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
try:
|
||||||
|
with socket.create_connection(("127.0.0.1", port), timeout=0.3):
|
||||||
|
break
|
||||||
|
except OSError:
|
||||||
|
time.sleep(0.05)
|
||||||
|
else:
|
||||||
|
pytest.skip("rsync daemon did not start")
|
||||||
|
|
||||||
|
# A well-formed -M option is forwarded and accepted by the daemon...
|
||||||
|
ok = _rsync(["-a", "-M--safe-links", source.as_posix() + "/",
|
||||||
|
f"rsync://127.0.0.1:{port}/m/"])
|
||||||
|
# ...and a bogus one is rejected ON THE REMOTE with "unknown option",
|
||||||
|
# which proves the option reached the daemon's parser.
|
||||||
|
bogus = _rsync(["-a", "-M--totally-bogus", source.as_posix() + "/",
|
||||||
|
f"rsync://127.0.0.1:{port}/m/"])
|
||||||
|
assert bogus.returncode != 0
|
||||||
|
assert "unknown option" in (bogus.stderr + bogus.stdout), bogus.stderr
|
||||||
|
del ok
|
||||||
|
finally:
|
||||||
|
daemon.terminate()
|
||||||
|
try:
|
||||||
|
daemon.wait(timeout=5)
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
daemon.kill()
|
||||||
|
|
||||||
|
# FastSync rejects -M for a non-SSH transport up front.
|
||||||
|
dest = os.path.join(TEST_DATA_DIR, "ro_dst")
|
||||||
|
clean_dir(dest)
|
||||||
|
result, _ = run_client(source.as_posix(), dest, flags=["-a", "-M--safe-links"])
|
||||||
|
assert result.returncode != 0
|
||||||
|
assert "remote-option" in (result.stderr + result.stdout)
|
||||||
|
|
||||||
|
|
||||||
|
class TestFilterProtectDivergence:
|
||||||
|
"""Documented residual: a protect rule that matches only a destination-only
|
||||||
|
entry is not re-derived on the receiver (FastSync derives delete protection
|
||||||
|
from the source scan), so rsync protects the extra but FastSync removes it."""
|
||||||
|
|
||||||
|
@requires_rsync
|
||||||
|
@pytest.mark.ci
|
||||||
|
def test_protect_dest_only_divergence(self, shared_server):
|
||||||
|
source = os.path.join(TEST_DATA_DIR, "fpd_src")
|
||||||
|
dest = os.path.join(TEST_DATA_DIR, "fpd_dst")
|
||||||
|
rdst = os.path.join(TEST_DATA_DIR, "fpd_rdst")
|
||||||
|
clean_dir(source)
|
||||||
|
_write(os.path.join(source, "keep.txt"), b"keep\n")
|
||||||
|
clean_dir(rdst)
|
||||||
|
_write(os.path.join(rdst, "extra.log"), b"extra\n")
|
||||||
|
_write(os.path.join(rdst, "other.txt"), b"other\n")
|
||||||
|
|
||||||
|
rsync_result = _rsync(["-a", "--delete", "--filter=P *.log", source + "/", rdst + "/"])
|
||||||
|
assert rsync_result.returncode == 0, rsync_result.stderr
|
||||||
|
assert os.path.exists(os.path.join(rdst, "extra.log")), "rsync did not protect extra.log"
|
||||||
|
|
||||||
|
clean_dir(dest)
|
||||||
|
received = get_dest_received_dir(dest, source)
|
||||||
|
_write(os.path.join(received, "extra.log"), b"extra\n")
|
||||||
|
_write(os.path.join(received, "other.txt"), b"other\n")
|
||||||
|
with ServerManager() as server:
|
||||||
|
server.start(extra_args=["--allow-delete"])
|
||||||
|
result, _ = run_client(source, dest,
|
||||||
|
flags=["-a", "--delete", "--filter=P *.log"],
|
||||||
|
port=server.port)
|
||||||
|
assert result.returncode == 0, (result.stderr or result.stdout)[:200]
|
||||||
|
# Pin the known divergence: FastSync deletes the destination-only file.
|
||||||
|
assert not os.path.exists(os.path.join(received, "extra.log")), (
|
||||||
|
"FastSync now protects destination-only P matches; the --filter row may be "
|
||||||
|
"upgradable to full parity"
|
||||||
|
)
|
||||||
|
assert not os.path.exists(os.path.join(received, "other.txt"))
|
||||||
@@ -1361,8 +1361,8 @@ static void test_parse_args_rsync_flag_vocabulary_accepted() {
|
|||||||
int positional_count = 0;
|
int positional_count = 0;
|
||||||
|
|
||||||
EXPECT_EQ_INT(parse_args(cfg, 4, argv, positional_args, &positional_count), 0);
|
EXPECT_EQ_INT(parse_args(cfg, 4, argv, positional_args, &positional_count), 0);
|
||||||
EXPECT_EQ_INT(cfg->info_level, LOG_INFO_BACKUP | LOG_INFO_DEL | LOG_INFO_FLIST |
|
EXPECT_EQ_INT(cfg->info_level, LOG_INFO_BACKUP | LOG_INFO_DEL | LOG_INFO_FLIST | LOG_INFO_NONREG |
|
||||||
LOG_INFO_NONREG | LOG_INFO_PROGRESS | LOG_INFO_REMOVE);
|
LOG_INFO_PROGRESS | LOG_INFO_REMOVE);
|
||||||
EXPECT_EQ_INT(cfg->debug_level, 0);
|
EXPECT_EQ_INT(cfg->debug_level, 0);
|
||||||
config_delete(cfg);
|
config_delete(cfg);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -244,6 +244,7 @@ static void test_write_thread_done() {
|
|||||||
* and queue_destroy which would double-free since we created them
|
* and queue_destroy which would double-free since we created them
|
||||||
* in this test. Let me just free the context directly. */
|
* in this test. Let me just free the context directly. */
|
||||||
array_list_delete(ctx->would_delete);
|
array_list_delete(ctx->would_delete);
|
||||||
|
array_list_delete(ctx->deleted_paths);
|
||||||
mtx_destroy(&ctx->mutex);
|
mtx_destroy(&ctx->mutex);
|
||||||
cnd_destroy(&ctx->condition_not_full);
|
cnd_destroy(&ctx->condition_not_full);
|
||||||
cnd_destroy(&ctx->condition_not_empty);
|
cnd_destroy(&ctx->condition_not_empty);
|
||||||
@@ -329,6 +330,7 @@ static void test_receiver_enqueue_byte_budget() {
|
|||||||
|
|
||||||
/* Tear down: the second file is still queued and is freed by queue_destroy. */
|
/* Tear down: the second file is still queued and is freed by queue_destroy. */
|
||||||
array_list_delete(ctx->would_delete);
|
array_list_delete(ctx->would_delete);
|
||||||
|
array_list_delete(ctx->deleted_paths);
|
||||||
mtx_destroy(&ctx->mutex);
|
mtx_destroy(&ctx->mutex);
|
||||||
cnd_destroy(&ctx->condition_not_full);
|
cnd_destroy(&ctx->condition_not_full);
|
||||||
cnd_destroy(&ctx->condition_not_empty);
|
cnd_destroy(&ctx->condition_not_empty);
|
||||||
|
|||||||
Reference in New Issue
Block a user