TapTap/FastSync
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

feat(p7-cli-namespace): rename colliding short flags to rsync parity (Wave A)

Browse Source 
-c -> --checksum, -m -> --prune-empty-dirs, -M -> --remote-option,
-f -> --filter, -s -> --secluded-args, -p -> --perms, -T -> --temp-dir;
-a/--archive is now real rsync -rlptgoD (links+metadata+devices+specials).

FastSync's own flags moved to long-form-only or new shorts:
-j/--threads (multithreading), --preserve (metadata), --sendfile,
--chunk-serialization, --timeout, --ssh-port. Client-side only; the
wire config fields are unchanged (no PROTOCOL_VERSION bump). The server
keeps -p as its port. Docs (README, RSYNC_COMPAT summary 129->132) and
unit/integration tests updated. 37/37 unit, 400-pass integration.
This commit is contained in:
TapTap
2026-09-11 14:20:15 +02:00
parent 15363c0018
commit 66e82f9384
12 changed files with 329 additions and 280 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+31 -26
View File
@@ -88,16 +88,18 @@ partial, alternate, and planned behavior.
| Argument | Description | | Argument | Description |
|----------|-------------| |----------|-------------|
| Positional | `<source> <dest>` — automatic SSH detection if dest contains `:` | | Positional | `<source> <dest>` — automatic SSH detection if dest contains `:` |
| `-c [level]` | Compression with optional level (1–22, default 5) | | `-c, --checksum` | Verify content by checksum instead of size+mtime |
| `-z [level]` | Alias for `-c` | | `-z, --compress [level]` | Enable streaming zstd compression (level 1–22, default 5) |
| `-a, --archive` | Archive mode: enables `-c -m -M` (no `-s`) | | `-a, --archive` | rsync archive mode (`-rlptgoD`): links, metadata, devices and specials (not compression/multithreading) |
| `-m` | Multithreading mode | | `-j, --threads` | Multithreading mode |
| `-s` | Chunk serialization (batch all files per chunk) | | `-m` | rsync `--prune-empty-dirs` (short form now rsync-parity) |
| `--secluded-args` | Accepted as an rsync compatibility option with no effect; `-s` remains chunk serialization. | | `--chunk-serialization` | Chunk serialization (batch all files per chunk; long form only) |
| `-f, --sendfile` | Sendfile zero-copy. Incompatible with `-c` / `-s`. TCP only. | | `-s` | rsync `--secluded-args` compatibility no-op (remote SSH argv is already injection-safe) |
| `-M, --preserve` | Preserve supported file metadata (mode and mtime; ownership and atime are unsupported) | | `--sendfile` | Sendfile zero-copy. Incompatible with compression / chunk serialization. TCP only. Long form only. |
| `--preserve` | Preserve supported file metadata (mode and mtime; ownership and atime are unsupported) |
| `-n, --dry-run` | Scan and print what would be transferred | | `-n, --dry-run` | Scan and print what would be transferred |
| `-p <port>` | SSH port (default: 22) | | `-p, --perms` | Preserve permission bits (part of the metadata bundle) |
| `--ssh-port <port>` | SSH port (default: 22) |
| `-v, --verbose` | Enable debug logging | | `-v, --verbose` | Enable debug logging |
| `-q, --quiet` | Suppress non-error output | | `-q, --quiet` | Suppress non-error output |
| `--progress` | Show real-time transfer speed | | `--progress` | Show real-time transfer speed |
@@ -113,7 +115,7 @@ partial, alternate, and planned behavior.
| `--max-size <n>` | Skip files larger than n bytes | | `--max-size <n>` | Skip files larger than n bytes |
| `--min-size <n>` | Skip files smaller than n bytes | | `--min-size <n>` | Skip files smaller than n bytes |
| `--max-alloc <SIZE>` | Maximum single allocation (binary units: B, K, M, G, T, P, E; default 1G) | | `--max-alloc <SIZE>` | Maximum single allocation (binary units: B, K, M, G, T, P, E; default 1G) |
| `--incremental` | Skip files unchanged since last transfer (size + mtime). Auto-enables `--preserve`. Incompatible with `-s`. | | `--incremental` | Skip files unchanged since last transfer (size + mtime). Auto-enables `--preserve`. Incompatible with `--chunk-serialization`. |
| `--existing` | Skip files not already present at the destination; update existing files normally. | | `--existing` | Skip files not already present at the destination; update existing files normally. |
| `--bwlimit <KB/s>` | Bandwidth limit in kilobytes per second | | `--bwlimit <KB/s>` | Bandwidth limit in kilobytes per second |
| `--chunk-size <n>` | Chunk size in bytes (default: 10485760) | | `--chunk-size <n>` | Chunk size in bytes (default: 10485760) |
@@ -350,16 +352,16 @@ features without changing the meaning of ordinary compatibility options.
| Option | Purpose | | Option | Purpose |
|---|---| |---|---|
| `-m` | Enable the multithreaded scanner/loader/sender pipeline. | | `-j`, `--threads` | Enable the multithreaded scanner/loader/sender pipeline. |
| `-c [level]`, `-z [level]` | Enable streaming zstd compression, levels 1-22. | | `-z [level]`, `--compress [level]` | Enable streaming zstd compression, levels 1-22. |
| `--compress-level <n>` | Set the zstd compression level. | | `--compress-level <n>` | Set the zstd compression level. |
| `--zc <alg>` | Alias for `--compress-choice`. FastSync supports `zstd` and `none`. | | `--zc <alg>` | Alias for `--compress-choice`. FastSync supports `zstd` and `none`. |
| `--zl <n>` | Alias for `--compress-level`. | | `--zl <n>` | Alias for `--compress-level`. |
| `--skip-compress <list>` | Skip compression for comma-separated suffixes; incompatible with `-s`. | | `--skip-compress <list>` | Skip compression for comma-separated suffixes; incompatible with `--chunk-serialization`. |
| `--compress-threads <n>` | Use `n` zstd compression workers. Requires compression and a zstd build with threaded support; the setting affects sender CPU work only. | | `--compress-threads <n>` | Use `n` zstd compression workers. Requires compression and a zstd build with threaded support; the setting affects sender CPU work only. |
| `--chunk-size <bytes>` | Set the transfer chunk size. | | `--chunk-size <bytes>` | Set the transfer chunk size. |
| `-s` | Enable FastSync chunk serialization. | | `--chunk-serialization` | Enable FastSync chunk serialization (long form only; `-s` is rsync's `--secluded-args`). |
| `-f`, `--sendfile` | Use TCP `sendfile()` zero-copy transfer. Incompatible with compression and chunk serialization. | | `--sendfile` | Use TCP `sendfile()` zero-copy transfer. Incompatible with compression and chunk serialization. Long form only. |
| `--delta` | Use FastSync-native block delta transfer. Requires `--incremental`. | | `--delta` | Use FastSync-native block delta transfer. Requires `--incremental`. |
| `--delta-block <bytes>` | Set the FastSync delta block size. | | `--delta-block <bytes>` | Set the FastSync delta block size. |
| `--delta-max <bytes>` | Limit files eligible for FastSync delta transfer. | | `--delta-max <bytes>` | Limit files eligible for FastSync delta transfer. |
@@ -372,15 +374,18 @@ features without changing the meaning of ordinary compatibility options.
| `--timeout <seconds>` | Set I/O timeout. | | `--timeout <seconds>` | Set I/O timeout. |
| `--contimeout <seconds>` | Set connection timeout. | | `--contimeout <seconds>` | Set connection timeout. |
Current short-option conflicts are tracked as compatibility work. In Short-option conflicts with rsync have been resolved for the CLI namespace
particular, FastSync currently uses `-p` for SSH port, `-s` for chunk (Phase 7): `-c` is now rsync's `--checksum`, `-m` is `--prune-empty-dirs`, `-M`
serialization, and `-S` for sparse handling. These meanings must be reconciled is `--remote-option`, `-f` is `--filter`, `-s` is `--secluded-args`, `-p` is
before FastSync can claim full rsync CLI compatibility. `--perms`, and `-T` is `--temp-dir`. FastSync's own flags were renamed to
long-form-only or new shorts: multithreading is `-j`/`--threads`, metadata
is `--preserve`, sendfile is `--sendfile`, chunk serialization is
`--chunk-serialization`, timeout is `--timeout`, and SSH port is `--ssh-port`.
`-a`/`--archive` is now real rsync archive (`-rlptgoD`).
`--secluded-args` is accepted as a long-form compatibility no-op. It does not `--secluded-args` (and its short form `-s`) is accepted as a compatibility
change FastSync's transport or protocol behavior. The rsync short form `-s` is no-op. It does not change FastSync's transport or protocol behavior, because
intentionally not aliased because it remains FastSync's chunk-serialization remote SSH argv is already built injection-safe.
option.
## Client Options ## Client Options
@@ -388,7 +393,7 @@ option.
| Option | Description | | Option | Description |
|---|---| |---|---|
| `-a`, `--archive` | Enable current archive preset. Full rsync archive semantics are planned. | | `-a`, `--archive` | rsync archive mode (`-rlptgoD`): links, metadata, devices and specials. |
| `-n`, `--dry-run` | Scan and report without writing files. | | `-n`, `--dry-run` | Scan and report without writing files. |
| `--delete` | Request removal of destination entries absent from the source. The server must allow deletion. Default timing is delete-after: extras are removed only after the whole transfer succeeded. | | `--delete` | Request removal of destination entries absent from the source. The server must allow deletion. Default timing is delete-after: extras are removed only after the whole transfer succeeded. |
| `--delete-before` | Delete extras before the transfer starts (implies `--delete`). | | `--delete-before` | Delete extras before the transfer starts (implies `--delete`). |
@@ -417,7 +422,7 @@ use with `--partial`. |
| Option | Description | | Option | Description |
|---|---| |---|---|
| `-M`, `--preserve` | Preserve supported file metadata, currently mode and modification time. | | `--preserve` | Preserve supported file metadata, currently mode and modification time (long form only). |
| `-l`, `--links` | Request symlink preservation; | `-l`, `--links` | Request symlink preservation;
link-target transfer remains incomplete. | link-target transfer remains incomplete. |
| `--copy-links` | Copy symlink referents. | | `--copy-links` | Copy symlink referents. |
@@ -440,7 +445,7 @@ link-target transfer remains incomplete. |
| Option | Description | | Option | Description |
|---|---| |---|---|
| `-p <port>` | SSH port in the current CLI. This conflicts with rsync's `-p` permissions option and is planned for correction. | | `--ssh-port <port>` | SSH port for the SSH transport (default: 22). Note the short `-p` is now rsync's `--perms`. |
| `--fastsync-server-path <path>` | Remote FastSync server path for SSH mode. | | `--fastsync-server-path <path>` | Remote FastSync server path for SSH mode. |
| `--source-dir <path>` | Set the source directory explicitly. | | `--source-dir <path>` | Set the source directory explicitly. |
| `--dest-dir <path>` | Set the destination directory explicitly. | | `--dest-dir <path>` | Set the destination directory explicitly. |
RSYNC_COMPAT.md
+15 -15
View File
@@ -6,8 +6,8 @@ This document maps rsync's full feature set to FastSync's current implementation
| Status | Count | Description | | Status | Count | Description |
|--------|-------|-------------| |--------|-------|-------------|
| ✅ Implemented | 129 | Feature works end-to-end | | ✅ Implemented | 132 | Feature works end-to-end |
| 🔀 Alt Arg | 3 | Functionality exists but under different flag/semantics | | 🔀 Alt Arg | 0 | Functionality exists but under different flag/semantics |
| ⚠️ Partial | 10 | Flag parsed/stored but behavior incomplete | | ⚠️ Partial | 10 | Flag parsed/stored but behavior incomplete |
| 🔄 Compatibility No-op | 3 | Flag is accepted for CLI compatibility but has no effect | | 🔄 Compatibility No-op | 3 | Flag is accepted for CLI compatibility but has no effect |
| ❌ Not Implemented | 2 | Flag not recognized or no behavior | | ❌ Not Implemented | 2 | Flag not recognized or no behavior |
@@ -19,7 +19,7 @@ This document maps rsync's full feature set to FastSync's current implementation
| Flag | Rsync Description | FastSync Status | Notes | | Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------| |------|-------------------|-----------------|-------|
| `-a`, `--archive` | Archive mode is -rlptgoD | 🔀 Alt Arg | Maps to -c -m -M (compression + multithread + metadata) | | `-a`, `--archive` | Archive mode is -rlptgoD | ✅ Implemented | Phase 7 Wave A: real rsync archive. `-a`/`--archive` now implies `--links` + metadata (perms/times/group/owner as FastSync's broad bundle) + `--devices` + `--specials`. FastSync is always recursive, so no `-r` is needed. It no longer implies compression or multithreading (those moved to `-z`/`-j`). The short-option namespace is now rsync-parity (see the Phase 7 note) |
| `-v`, `--verbose` | Increase verbosity | ✅ Implemented | Sets `log_level=DEBUG` | | `-v`, `--verbose` | Increase verbosity | ✅ Implemented | Sets `log_level=DEBUG` |
| `-q`, `--quiet` | Suppress non-error messages | ✅ Implemented | Suppresses client output while preserving errors | | `-q`, `--quiet` | Suppress non-error messages | ✅ Implemented | Suppresses client output while preserving errors |
| `--help` | Show help | ✅ Implemented | Prints usage and exits; `-h` is not accepted | | `--help` | Show help | ✅ Implemented | Prints usage and exits; `-h` is not accepted |
@@ -97,7 +97,7 @@ This document maps rsync's full feature set to FastSync's current implementation
| `--backup-dir=DIR` | Backup directory hierarchy | ✅ Implemented | `backup_dir` config field | | `--backup-dir=DIR` | Backup directory hierarchy | ✅ Implemented | `backup_dir` config field |
| `--suffix=SUFFIX` | Backup suffix (default ~) | ✅ Implemented | `suffix` config field | | `--suffix=SUFFIX` | Backup suffix (default ~) | ✅ Implemented | `suffix` config field |
| `--delay-updates` | Put updated files in place at end | ✅ Implemented | Successfully received files are staged under a private 0700 `.fastsync-stage` dir inside the receive root and atomically renamed into their final destinations only after the whole transfer (manifest/delete handling included) succeeds, just before the success/outcome frame is sent. The delete walker deliberately skips the staging dir at the receive root, so `--delete` removes genuine extras but never the staged files (deletion runs before publication; rsync's delete-after ordering is not implemented). `--existing`/`--ignore-existing`/`--update` decide against the final destination path at stage time; `--backup` moves the old file aside at publication. Incompatible with `--inplace` and with `--backup-dir=.fastsync-stage` (the internal staging name is reserved; both are rejected). The staging dir name is fixed, so two simultaneous delayed transfers to the same destination root are serialized with an exclusive advisory lock held for the whole transfer: the second session fails cleanly instead of corrupting the first. Aborting or failing before publication installs nothing and removes the staging tree; a crash between stage and publish leaves staged leftovers that the next delayed run wipes at start (process death releases the lock). A stage→publish failure aborts the transfer (best-effort cleanup of the not-yet-published staged files; already-published files are not rolled back). Works in single-threaded and `-m` modes | | `--delay-updates` | Put updated files in place at end | ✅ Implemented | Successfully received files are staged under a private 0700 `.fastsync-stage` dir inside the receive root and atomically renamed into their final destinations only after the whole transfer (manifest/delete handling included) succeeds, just before the success/outcome frame is sent. The delete walker deliberately skips the staging dir at the receive root, so `--delete` removes genuine extras but never the staged files (deletion runs before publication; rsync's delete-after ordering is not implemented). `--existing`/`--ignore-existing`/`--update` decide against the final destination path at stage time; `--backup` moves the old file aside at publication. Incompatible with `--inplace` and with `--backup-dir=.fastsync-stage` (the internal staging name is reserved; both are rejected). The staging dir name is fixed, so two simultaneous delayed transfers to the same destination root are serialized with an exclusive advisory lock held for the whole transfer: the second session fails cleanly instead of corrupting the first. Aborting or failing before publication installs nothing and removes the staging tree; a crash between stage and publish leaves staged leftovers that the next delayed run wipes at start (process death releases the lock). A stage→publish failure aborts the transfer (best-effort cleanup of the not-yet-published staged files; already-published files are not rolled back). Works in single-threaded and `-m` modes |
| `-T`, `--temp-dir=DIR` | Create temporary files in DIR | ✅ Implemented | `--temp-dir` only; `-T` stays FastSync's `--timeout` alias. Scratch dir is resolved under the receive root; temp copies use a unique name there and are atomically renamed into place. If the scratch dir and destination are on different filesystems the atomic rename fails with EXDEV and the file save fails, which aborts the whole transfer (FastSync has no per-file skip/resume on a save error; rsync's non-atomic copy fallback is deliberately not used). `--inplace` and `--partial-dir` writes bypass the scratch dir | | `-T`, `--temp-dir=DIR` | Create temporary files in DIR | ✅ Implemented | `--temp-dir` with the rsync short `-T` (Phase 7 Wave A; the timeout alias moved to long-only `--timeout`). Scratch dir is resolved under the receive root; temp copies use a unique name there and are atomically renamed into place. If the scratch dir and destination are on different filesystems the atomic rename fails with EXDEV and the file save fails, which aborts the whole transfer (FastSync has no per-file skip/resume on a save error; rsync's non-atomic copy fallback is deliberately not used). `--inplace` and `--partial-dir` writes bypass the scratch dir |
## 7. Deletion ## 7. Deletion
@@ -112,7 +112,7 @@ This document maps rsync's full feature set to FastSync's current implementation
| `--max-delete=NUM` | Max files to delete | ✅ Implemented | `max_delete` config field (default -1 = no client limit; 0 = delete nothing). NUM bounds a `--delete` run with rsync's all-or-nothing semantics: the receiver rehearses the deletion first and, if the destination holds more than NUM extras, deletes NOTHING and fails the transfer with a distinct `--max-delete` error. A run at or below NUM deletes exactly the extras. NUM only applies together with `--delete` (it is inert otherwise, matching rsync). The hard server bound `MAX_SERVER_DELETE_COUNT` (100000) still caps the walk; a NUM above it never raises that cap, and exceeding the server bound is its own all-or-nothing error. Directories count toward the limit (each removed empty directory is one deletion), like rsync | | `--max-delete=NUM` | Max files to delete | ✅ Implemented | `max_delete` config field (default -1 = no client limit; 0 = delete nothing). NUM bounds a `--delete` run with rsync's all-or-nothing semantics: the receiver rehearses the deletion first and, if the destination holds more than NUM extras, deletes NOTHING and fails the transfer with a distinct `--max-delete` error. A run at or below NUM deletes exactly the extras. NUM only applies together with `--delete` (it is inert otherwise, matching rsync). The hard server bound `MAX_SERVER_DELETE_COUNT` (100000) still caps the walk; a NUM above it never raises that cap, and exceeding the server bound is its own all-or-nothing error. Directories count toward the limit (each removed empty directory is one deletion), like rsync |
| `--ignore-errors` | Delete even with I/O errors | ✅ Implemented | Sender-side, client-only config field. rsync suppresses `--delete` when the transfer had I/O errors; FastSync's equivalent is a source-scan I/O error (an unreadable directory, e.g. EACCES): by default the scan aborts the run so no deletion happens. With `--ignore-errors` the scan continues past the unreadable directory, the readable tree is transferred and the deletion still runs (the mirror of the unreadable directory is treated as an extra). The run still exits non-zero (the error is reported, matching rsync's error status). Divergence: without the flag FastSync aborts the whole run on the scan error, whereas rsync transfers the rest of the tree and merely skips the deletion; both leave the deletion undone | | `--ignore-errors` | Delete even with I/O errors | ✅ Implemented | Sender-side, client-only config field. rsync suppresses `--delete` when the transfer had I/O errors; FastSync's equivalent is a source-scan I/O error (an unreadable directory, e.g. EACCES): by default the scan aborts the run so no deletion happens. With `--ignore-errors` the scan continues past the unreadable directory, the readable tree is transferred and the deletion still runs (the mirror of the unreadable directory is treated as an extra). The run still exits non-zero (the error is reported, matching rsync's error status). Divergence: without the flag FastSync aborts the whole run on the scan error, whereas rsync transfers the rest of the tree and merely skips the deletion; both leave the deletion undone |
| `--force` | Force deletion of non-empty dirs | ✅ Implemented | `force_delete` receiver config field (crosses the wire). rsync's `--force` lets an incoming non-directory replace a destination directory; FastSync implements exactly that: when a regular file is written to a path that is currently a (possibly non-empty) destination directory, `--force` removes that directory tree first — confined to the receive root and symlink-safe (O_NOFOLLOW fd walk, symlinks removed by name, never followed) — so the atomic install can place the file. Without `--force` such a write fails and the run aborts. Divergence: `--force` acts on the immediate-install path only; under `--delay-updates` a blocking directory is not cleared (publication renames over regular files) | | `--force` | Force deletion of non-empty dirs | ✅ Implemented | `force_delete` receiver config field (crosses the wire). rsync's `--force` lets an incoming non-directory replace a destination directory; FastSync implements exactly that: when a regular file is written to a path that is currently a (possibly non-empty) destination directory, `--force` removes that directory tree first — confined to the receive root and symlink-safe (O_NOFOLLOW fd walk, symlinks removed by name, never followed) — so the atomic install can place the file. Without `--force` such a write fails and the run aborts. Divergence: `--force` acts on the immediate-install path only; under `--delay-updates` a blocking directory is not cleared (publication renames over regular files) |
| `--prune-empty-dirs` | Prune empty dir chains | ✅ Implemented | Long-only: FastSync's `-m` is already multithreading (recorded divergence — rsync's `-m` short form is not reassigned). FastSync's recursive transfer never emits directory entries, so empty directories are inherently never transferred (which is rsync's `-m` behavior) and truly-empty destination directory chains are removed by `--delete` regardless of this flag. The flag's additional real effect is on the `--dirs` explicit directory-entry generator: a plain `-d <empty-dir>` run omits the empty source directory's entry, so nothing is created at the destination (no `STATUS_MKDIR`, no `-i`/`--out-format` change line, and an existing empty mirror becomes an extra that `--delete` prunes). Explicitly `--files-from`-listed directories always pass through (documented `--files-from` behavior). A directory that still holds an excluded-but-protected file survives, matching the `--delete-excluded` default | | `-m`, `--prune-empty-dirs` | Prune empty dir chains | ✅ Implemented | `-m`/`--prune-empty-dirs` (Phase 7 Wave A freed the rsync short `-m`; FastSync multithreading is now `-j`/`--threads`). FastSync's recursive transfer never emits directory entries, so empty directories are inherently never transferred (which is rsync's `-m` behavior) and truly-empty destination directory chains are removed by `--delete` regardless of this flag. The flag's additional real effect is on the `--dirs` explicit directory-entry generator: a plain `-d <empty-dir>` run omits the empty source directory's entry, so nothing is created at the destination (no `STATUS_MKDIR`, no `-i`/`--out-format` change line, and an existing empty mirror becomes an extra that `--delete` prunes). Explicitly `--files-from`-listed directories always pass through (documented `--files-from` behavior). A directory that still holds an excluded-but-protected file survives, matching the `--delete-excluded` default |
**Deletion-timing implementation notes (Phase 3):** the delete flags above are **Deletion-timing implementation notes (Phase 3):** the delete flags above are
real. Two new config booleans (`delete_during`, `delete_delay`) join the already real. Two new config booleans (`delete_during`, `delete_delay`) join the already
@@ -238,7 +238,7 @@ why plain `--append` works on the normal atomic path, not only with `--inplace`.
| Flag | Rsync Description | FastSync Status | Notes | | Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------| |------|-------------------|-----------------|-------|
| `-M`, `--preserve` | Preserve file metadata | ✅ Implemented | Mode, uid, gid, mtime | | `-M`, `--preserve` | Preserve file metadata | ✅ Implemented | Mode, uid, gid, mtime |
| `-p`, `--perms` | Preserve permissions | 🔀 Alt Arg | `-p` means SSH port; permissions preserved via `-M`/`--preserve` | | `-p`, `--perms` | Preserve permissions | ✅ Implemented | Phase 7 Wave A: `-p`/`--perms` now preserve permission bits, folded into FastSync's broad metadata bundle (`--preserve`); the SSH port moved to `--ssh-port`. rsync-parity short form |
| `-o`, `--owner` | Preserve owner | ✅ Implemented | Part of -M | | `-o`, `--owner` | Preserve owner | ✅ Implemented | Part of -M |
| `-g`, `--group` | Preserve group | ✅ Implemented | Part of -M | | `-g`, `--group` | Preserve group | ✅ Implemented | Part of -M |
| `-t`, `--times` | Preserve modification times | ✅ Implemented | Part of -M | | `-t`, `--times` | Preserve modification times | ✅ Implemented | Part of -M |
@@ -597,7 +597,7 @@ now transmits targets (the prior behavior was broken/partial); its status moved
| Flag | Rsync Description | FastSync Status | Notes | | Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------| |------|-------------------|-----------------|-------|
| `-z`, `--compress` | Compress file data | 🔀 Alt Arg | Always uses zstd (rsync supports multiple algorithms) | | `-z`, `--compress` | Compress file data | ✅ Implemented | Always uses zstd (rsync supports multiple algorithms — a documented divergence, selectable via `--compress-choice`). Phase 7 Wave A: `-z` is now the compression short form; `-c` is rsync's `--checksum` |
| `--compress-choice=STR`, `--zc=STR` | Choose compression algorithm | ✅ Implemented | FastSync supports `zstd` and `none` | | `--compress-choice=STR`, `--zc=STR` | Choose compression algorithm | ✅ Implemented | FastSync supports `zstd` and `none` |
| `--compress-level=NUM`, `--zl=NUM` | Set compression level | ✅ Implemented | 1-22, default 5 | | `--compress-level=NUM`, `--zl=NUM` | Set compression level | ✅ Implemented | 1-22, default 5 |
| `--compress-threads=NUM` | Set compression threads | ✅ Implemented | `compression_threads` config field (client-only; does not cross the wire). Sets the number of worker threads used by the zstd compression pool to NUM (1..64; 0/garbage/oversized rejected up front). Accepted in both `--compress-threads=NUM` and two-argument `--compress-threads NUM` forms. Composes with `-c`/compression; under the `-m` multithreaded pipeline it parallelizes compressed chunk encoding. See test_tcp.py `-c --compress-threads=2` and test_client_cli.c | | `--compress-threads=NUM` | Set compression threads | ✅ Implemented | `compression_threads` config field (client-only; does not cross the wire). Sets the number of worker threads used by the zstd compression pool to NUM (1..64; 0/garbage/oversized rejected up front). Accepted in both `--compress-threads=NUM` and two-argument `--compress-threads NUM` forms. Composes with `-c`/compression; under the `-m` multithreaded pipeline it parallelizes compressed chunk encoding. See test_tcp.py `-c --compress-threads=2` and test_client_cli.c |
@@ -616,7 +616,7 @@ now transmits targets (the prior behavior was broken/partial); its status moved
| `--address=ADDRESS` | Bind address for outgoing socket | ✅ Implemented | Binds the outgoing client socket to a local source address before `connect()` (resolved with the same `-4`/`-6` family hints as the destination). Local socket concern: never crosses the wire | | `--address=ADDRESS` | Bind address for outgoing socket | ✅ Implemented | Binds the outgoing client socket to a local source address before `connect()` (resolved with the same `-4`/`-6` family hints as the destination). Local socket concern: never crosses the wire |
| `-4`, `--ipv4` | Prefer IPv4 | ✅ Implemented | Forces `AF_INET` in the `getaddrinfo` hints for client destination/source resolution and the server bind (see the Phase 5, Wave B note). Mutually exclusive with `-6` | | `-4`, `--ipv4` | Prefer IPv4 | ✅ Implemented | Forces `AF_INET` in the `getaddrinfo` hints for client destination/source resolution and the server bind (see the Phase 5, Wave B note). Mutually exclusive with `-6` |
| `-6`, `--ipv6` | Prefer IPv6 | ✅ Implemented | Forces `AF_INET6` in the `getaddrinfo` hints for client destination/source resolution and the server bind. Mutually exclusive with `-4` | | `-6`, `--ipv6` | Prefer IPv6 | ✅ Implemented | Forces `AF_INET6` in the `getaddrinfo` hints for client destination/source resolution and the server bind. Mutually exclusive with `-4` |
| `--remote-option=OPT`, `-M` | Send an option only to the remote side | ✅ Implemented | Long form only; each value is appended to the remote server invocation over SSH as an individually single-quote-escaped shell word in `ssh_build_remote_command()`. Values are validated (non-empty, no control characters) and shell metacharacters cannot break out of the quoting (`;`, `&`, `|`, <code>`</code>, `$`, `(`, `)`, quotes are neutralized), so a value cannot inject an arbitrary remote command and a subsequent `--` on the client line cannot be turned into one. The options never cross the binary config frame. Divergence: the short `-M` form is intentionally unavailable because `-M` is already FastSync's metadata-preservation flag/multiplier (see Phase 5 notes below) | | `--remote-option=OPT`, `-M` | Send an option only to the remote side | ✅ Implemented | Each value is appended to the remote server invocation over SSH as an individually single-quote-escaped shell word in `ssh_build_remote_command()`. Values are validated (non-empty, no control characters) and shell metacharacters cannot break out of the quoting (`;`, `&`, `|`, <code>`</code>, `$`, `(`, `)`, quotes are neutralized), so a value cannot inject an arbitrary remote command and a subsequent `--` on the client line cannot be turned into one. The options never cross the binary config frame. Phase 7 Wave A: the short `-M` form is now available (as `-M OPT` and `-M=OPT`), matching rsync; metadata mode moved to long-only `--preserve` |
## 14. Daemon Mode ## 14. Daemon Mode
@@ -675,7 +675,7 @@ now transmits targets (the prior behavior was broken/partial); its status moved
| `--protocol=NUM` | Force older protocol version | ✅ Implemented | Forces the wire protocol version for this transfer. FastSync has exactly ONE wire format (`PROTOCOL_VERSION`, currently 2.16.0) with no downgrade/backward-compat code paths, so `--protocol=2.16.0` is accepted (it sets the version claim the client sends, which the server already requires to match exactly) and **every other value is rejected up front** with a clear error before any connection — it does not and cannot speak an older or virtual wire format. Divergence from rsync (which negotiates a range and downgrades to an integer 0..31): FastSync's honest contract is force-to-the-one-supported-value; a genuine downgrade would require a per-version compatibility layer that does not exist. Client-only; the server-side exact-match check is unchanged. `--protocol=2.16`/`2.15.0`/`2.17.0`/`216`/`31`/garbage are all rejected. See the Phase-6 protocol note below | | `--protocol=NUM` | Force older protocol version | ✅ Implemented | Forces the wire protocol version for this transfer. FastSync has exactly ONE wire format (`PROTOCOL_VERSION`, currently 2.16.0) with no downgrade/backward-compat code paths, so `--protocol=2.16.0` is accepted (it sets the version claim the client sends, which the server already requires to match exactly) and **every other value is rejected up front** with a clear error before any connection — it does not and cannot speak an older or virtual wire format. Divergence from rsync (which negotiates a range and downgrades to an integer 0..31): FastSync's honest contract is force-to-the-one-supported-value; a genuine downgrade would require a per-version compatibility layer that does not exist. Client-only; the server-side exact-match check is unchanged. `--protocol=2.16`/`2.15.0`/`2.17.0`/`216`/`31`/garbage are all rejected. See the Phase-6 protocol note below |
| `--iconv=CONVERT_SPEC` | Charset conversion | ✅ Implemented | Charset conversion of FILE NAMES (not content) at the protocol boundary via iconv(3): `--iconv=LOCAL[,REMOTE]` — the sender converts each local filename LOCAL→REMOTE before transmitting, and the receiver converts each wire filename REMOTE→LOCAL before creating/writing. The full CONVERT_SPEC is serialized into the config frame as a new trailing string field so the peer knows the wire charset; **PROTOCOL_VERSION bumped 2.15.0 → 2.16.0**. `LOCAL[,REMOTE]` parse: single charset ⇒ LOCAL==REMOTE (identity both ways); garbage rejected up front. Validation probes BOTH directions (a spec that only opens one way is refused, as is a NUL-emitting target charset like utf-16/utf-32/ucs-2, since filenames cannot contain NUL). An unrepresentable name (EILSEQ/EINVAL) fails that path cleanly with a logged `--iconv: cannot convert file name ...` and is never written mangled/truncated. Conversion is applied at EVERY wire-path site (regular/MKDIR/hardlink path+target/symlink path+target/SPECIAL, the delete manifest, the incremental-check path, and the `-s`/`chunk_serialize` embedded blob path), on both client and server (`--iconv` is also a server/daemon option). Zero overhead when unset. See the Phase-6 iconv notes below | | `--iconv=CONVERT_SPEC` | Charset conversion | ✅ Implemented | Charset conversion of FILE NAMES (not content) at the protocol boundary via iconv(3): `--iconv=LOCAL[,REMOTE]` — the sender converts each local filename LOCAL→REMOTE before transmitting, and the receiver converts each wire filename REMOTE→LOCAL before creating/writing. The full CONVERT_SPEC is serialized into the config frame as a new trailing string field so the peer knows the wire charset; **PROTOCOL_VERSION bumped 2.15.0 → 2.16.0**. `LOCAL[,REMOTE]` parse: single charset ⇒ LOCAL==REMOTE (identity both ways); garbage rejected up front. Validation probes BOTH directions (a spec that only opens one way is refused, as is a NUL-emitting target charset like utf-16/utf-32/ucs-2, since filenames cannot contain NUL). An unrepresentable name (EILSEQ/EINVAL) fails that path cleanly with a logged `--iconv: cannot convert file name ...` and is never written mangled/truncated. Conversion is applied at EVERY wire-path site (regular/MKDIR/hardlink path+target/symlink path+target/SPECIAL, the delete manifest, the incremental-check path, and the `-s`/`chunk_serialize` embedded blob path), on both client and server (`--iconv` is also a server/daemon option). Zero overhead when unset. See the Phase-6 iconv notes below |
| `--checksum-seed=NUM` | Set checksum seed | ✅ Implemented | Sets the seed for FastSync's whole-file xxHash64 digest (full 64-bit seed) and for the delta path's per-block xxHash32 strong checksum (low 32 bits of the seed). An explicit seed deterministically changes every computed digest on BOTH endpoints (sender and receiver share the seed via the config frame, protocol 2.10.0), so identical runs with the same seed skip the same files and a changed seed changes the digests — the explicit-seed path that makes xxHash comparisons deterministic. `--checksum-choice=md5` has no seed and ignores it (documented). The value is a strict decimal 0..2⁶⁴-1 (blank, signed, or non-numeric values are rejected). Like rsync, a seed only matters where a digest is actually computed (`--checksum` or a basis-dir run, or a delta transfer); it does not by itself enable `--checksum`/`--delta`. Divergence from rsync: the default is seed 0, and FastSync never randomizes the seed (rsync uses a random per-transfer seed when `--checksum-seed` is unset); FastSync's unset default therefore reproduces its historical byte-for-byte behavior | | `--checksum-seed=NUM` | Set checksum seed | ✅ Implemented | Sets the seed for FastSync's whole-file xxHash64 digest (full 64-bit seed) and for the delta path's per-block xxHash32 strong checksum (low 32 bits of the seed). An explicit seed deterministically changes every computed digest on BOTH endpoints (sender and receiver share the seed via the config frame, protocol 2.10.0), so identical runs with the same seed skip the same files and a changed seed changes the digests — the explicit-seed path that makes xxHash comparisons deterministic. `--checksum-choice=md5` has no seed and ignores it (documented). The value is a strict decimal 0..2⁶⁴-1 (blank, signed, or non-numeric values are rejected). Like rsync, a seed only matters where a digest is actually computed (`--checksum` or a basis-dir run, or a delta transfer); it does not by itself enable `--checksum`/`--delta`. Divergence from rsync: the default is seed 0, and FastSync never randomizes the seed (rsync uses a random per-transfer seed when `--checksum-seed` is unset); FastSync's unset default therefore reproduces its historical byte-for-byte behavior |
| `--secluded-args` | Use protocol to send args | 🔄 Compatibility No-op | Accepted for CLI compatibility; it does not change FastSync transport or protocol behavior. `-s` remains chunk serialization. | | `--secluded-args`, `-s` | Use protocol to send args | 🔄 Compatibility No-op | Accepted for CLI compatibility, including the rsync short `-s` (Phase 7 Wave A); it does not change FastSync transport or protocol behavior, because remote SSH argv is already built injection-safe (single-quote-escaped). Chunk serialization is the long-only `--chunk-serialization`. |
| `--no-OPTION` | Turn off implied option | ✅ Supported | Supported boolean FastSync options and archive-implied options; unsafe or value-taking options are rejected. | | `--no-OPTION` | Turn off implied option | ✅ Supported | Supported boolean FastSync options and archive-implied options; unsafe or value-taking options are rejected. |
--- ---
@@ -683,7 +683,7 @@ now transmits targets (the prior behavior was broken/partial); its status moved
## Implementation Difficulty Plan ## Implementation Difficulty Plan
**Phase 5 notes (remote-option wave):** `--remote-option=OPT` (long form only) and `--trust-sender` landed here. **Phase 5 notes (remote-option wave):** `--remote-option=OPT` (long form only) and `--trust-sender` landed here.
- `--remote-option` is CLIENT-only and never serialized into the binary config frame. On the SSH transport the client forwards each value to the remote server by appending it to the remote command line in `ssh_build_remote_command()`, after ` --stdio`, as an individually single-quoted shell word (`'...'` with `'\''` for embedded quotes). Values are validated at CLI parse time (non-empty; no ASCII control characters) and rejected otherwise, and a non-conforming value is refused again in the command builder, so shell metacharacters (`;`, `&`, `|`, backticks, `$()`, quotes) can never break out of the quoting to inject an unrelated remote command — including after a client-side `--` separator, whose arguments are never forwarded anyway. Because the remote options affect the *remote server invocation*, not the transmitted config, the wire frame layout is unchanged, but `PROTOCOL_VERSION` was bumped **2.13.0 → 2.14.0** as the Phase-5 lockstep release marker (a 2.14 client against a 2.13 server fails the version check cleanly rather than the old server rejecting an unfamiliar forwarded argv later). Divergence: rsync's short `-M` form of `--remote-option` is intentionally NOT implemented, because `-M` is already FastSync's metadata-preservation mode/multiplier. - `--remote-option` is CLIENT-only and never serialized into the binary config frame. On the SSH transport the client forwards each value to the remote server by appending it to the remote command line in `ssh_build_remote_command()`, after ` --stdio`, as an individually single-quoted shell word (`'...'` with `'\''` for embedded quotes). Values are validated at CLI parse time (non-empty; no ASCII control characters) and rejected otherwise, and a non-conforming value is refused again in the command builder, so shell metacharacters (`;`, `&`, `|`, backticks, `$()`, quotes) can never break out of the quoting to inject an unrelated remote command — including after a client-side `--` separator, whose arguments are never forwarded anyway. Because the remote options affect the *remote server invocation*, not the transmitted config, the wire frame layout is unchanged, but `PROTOCOL_VERSION` was bumped **2.13.0 → 2.14.0** as the Phase-5 lockstep release marker (a 2.14 client against a 2.13 server fails the version check cleanly rather than the old server rejecting an unfamiliar forwarded argv later). Divergence: rsync's short `-M` form of `--remote-option` was intentionally NOT implemented at that time because `-M` was FastSync metadata mode; **Phase 7 Wave A later freed `-M` for `--remote-option` and moved metadata to long-only `--preserve`** (see the Sending Options table).
- `--trust-sender` is a receiver-local policy: it never crosses the wire (the sender's value is never serialized, so a wire peer can never enable it). On the receiving process it skips the up-front re-validation of the incoming file list (empty/`..` path rejection and the escaping-symlink-target containment), trusting the sender's list instead of double-checking — fewer checks, faster, and potentially unsafe, matching rsync. It is OFF by default (`config.trust_sender`). As a deliberate safety floor, the low-level fd-relative confinement primitives are NOT disabled: `file_open_secure_parent()` (O_NOFOLLOW walk, `..` rejection, root containment) and leaf/destination confinement still hold, so even under `--trust-sender` a hostile sender cannot write or create a symlink outside the authorized root — the relaxation only removes the redundant list-layer double-checks, never the root-confinement guarantees. - `--trust-sender` is a receiver-local policy: it never crosses the wire (the sender's value is never serialized, so a wire peer can never enable it). On the receiving process it skips the up-front re-validation of the incoming file list (empty/`..` path rejection and the escaping-symlink-target containment), trusting the sender's list instead of double-checking — fewer checks, faster, and potentially unsafe, matching rsync. It is OFF by default (`config.trust_sender`). As a deliberate safety floor, the low-level fd-relative confinement primitives are NOT disabled: `file_open_secure_parent()` (O_NOFOLLOW walk, `..` rejection, root containment) and leaf/destination confinement still hold, so even under `--trust-sender` a hostile sender cannot write or create a symlink outside the authorized root — the relaxation only removes the redundant list-layer double-checks, never the root-confinement guarantees.
The estimates below cover the currently unimplemented features in this document. They assume one engineer familiar with the codebase, include implementation and focused tests, and exclude production rollout time. A feature should not be marked implemented until its behavior is tested in both local and SSH/TCP paths where applicable. The estimates below cover the currently unimplemented features in this document. They assume one engineer familiar with the codebase, include implementation and focused tests, and exclude production rollout time. A feature should not be marked implemented until its behavior is tested in both local and SSH/TCP paths where applicable.
@@ -798,7 +798,7 @@ These are the hardest compatibility items because they require durable formats o
These are the last compatibility items and the closing phase toward rsync flag parity. Per the project decision: every rsync flag (short **and** long) that is *possible* gets real rsync-parity behavior; anything physically impossible becomes an explicit **Impossible/Divergence** status (accepted for CLI compatibility, safely inert, with coverage tests proving that); and the two privilege flags are deferred to the final wave pending an explicit privilege-model decision. The remaining `⚠️ Partial`, `🔄 Compatibility No-op`, `🔀 Alt Arg`, and `❌ Not Implemented` rows in the Summary are this phase's scope. All Wave A renames are **client-side only** (the wire config fields `use_compression`/`use_metadata`/`use_sendfile`/`use_chunk_serialization` are unchanged), so they require **no `PROTOCOL_VERSION` bump**. These are the last compatibility items and the closing phase toward rsync flag parity. Per the project decision: every rsync flag (short **and** long) that is *possible* gets real rsync-parity behavior; anything physically impossible becomes an explicit **Impossible/Divergence** status (accepted for CLI compatibility, safely inert, with coverage tests proving that); and the two privilege flags are deferred to the final wave pending an explicit privilege-model decision. The remaining `⚠️ Partial`, `🔄 Compatibility No-op`, `🔀 Alt Arg`, and `❌ Not Implemented` rows in the Summary are this phase's scope. All Wave A renames are **client-side only** (the wire config fields `use_compression`/`use_metadata`/`use_sendfile`/`use_chunk_serialization` are unchanged), so they require **no `PROTOCOL_VERSION` bump**.
**Wave A — CLI namespace parity (rename colliding FastSync short flags).** This is the prerequisite for all short-flag parity: it frees the short letters rsync needs and makes the three `🔀 Alt Arg` rows real. Renames touch `src/client/client_cli.c` `OPTION_TABLE` + the legacy bool table, `src/client/usage.c`, the README, and integration fixtures that reference the old short flags; the server's independent little CLI stays as-is (`-p` there remains the server port). Includes an opportunistic mechanical CLI refactor (align the client tables; dedupe where the pattern is identical). A fresh rsync-collision audit fixes the final short letters for the freed FastSync flags during this wave. Flags already matching rsync (`-e`/`--rsh`, `-S`/`--sparse`, `-H`, `-K`, `-k`, `-A`, `-X`, `-x`, `-C`, `-R`, `-d`, `-u`, `-W`, `-I`, `-l`, `-o`) are untouched. **Wave A — CLI namespace parity (rename colliding FastSync short flags) — ✅ implemented.** This freed the short letters rsync needs and made the three `🔀 Alt Arg` rows real. `-c`→`--checksum`, `-m`→`--prune-empty-dirs`, `-M`→`--remote-option`, `-f`→`--filter`, `-s`→`--secluded-args`, `-p`→`--perms`, `-T`→`--temp-dir`, `-a`/`--archive`→real `-rlptgoD`. FastSync's own flags moved to long-form-only or new shorts: `-j`/`--threads` (multithreading), `--preserve` (metadata), `--sendfile`, `--chunk-serialization`, `--timeout`, `--ssh-port`. The server's independent little CLI keeps `-p` as its port. All client-side, no wire change, no `PROTOCOL_VERSION` bump. Unit tests 37/37, full integration 400 passed, cppcheck and clang-format clean.
| FastSync flag today | rsync wants that name | Proposed rename | | FastSync flag today | rsync wants that name | Proposed rename |
|---------------------|----------------------|-----------------| |---------------------|----------------------|-----------------|
@@ -857,10 +857,10 @@ Ranked by user demand, implementation complexity, and interoperability impact (_
| Feature | Description | | Feature | Description |
|---------|-------------| |---------|-------------|
| `-m` | Multithreaded pipeline (scanner/loader/sender) | | `-j` / `--threads` | Multithreaded pipeline (scanner/loader/sender) (renamed from `-m` in Phase 7 Wave A; `-m` is now rsync `--prune-empty-dirs`) |
| `-s` | Chunk serialization mode | | `--chunk-serialization` | Chunk serialization mode (long form only; `-s` is now rsync `--secluded-args`) |
| `-f` / `--sendfile` | Zero-copy sendfile() syscall (TCP only) | | `--sendfile` | Zero-copy sendfile() syscall (TCP only) (long form only; `-f` is now rsync `--filter`) |
| `-c [level]` | zstd compression level (1-22) | | `-z [level]` / `--compress` | zstd compression level (1-22) (`-c` is now rsync `--checksum`) |
| `--chunk-size` | Configurable chunk size | | `--chunk-size` | Configurable chunk size |
| `--tls` | TLS encryption (mutual auth) | | `--tls` | TLS encryption (mutual auth) |
| `--fastsync-server-path` | Path to fastsync-server binary | | `--fastsync-server-path` | Path to fastsync-server binary |
src/client/client_cli.c
+51 -34
View File
@@ -554,7 +554,7 @@ static const OptionEntry OPTION_TABLE[] = {
{"--stats", NULL, OPT_FLAG, offsetof(Config, stats)}, {"--stats", NULL, OPT_FLAG, offsetof(Config, stats)},
{"--human-readable", "-h", OPT_FLAG, offsetof(Config, human_readable)}, {"--human-readable", "-h", OPT_FLAG, offsetof(Config, human_readable)},
{"--partial", NULL, OPT_FLAG, offsetof(Config, partial)}, {"--partial", NULL, OPT_FLAG, offsetof(Config, partial)},
{"--secluded-args", NULL, OPT_NOOP, 0}, {"--secluded-args", "-s", OPT_NOOP, 0},
{"--update", "-u", OPT_FLAG, offsetof(Config, update)}, {"--update", "-u", OPT_FLAG, offsetof(Config, update)},
{"--old-args", NULL, OPT_FLAG, offsetof(Config, old_args)}, {"--old-args", NULL, OPT_FLAG, offsetof(Config, old_args)},
{"--rsh", "-e", OPT_STRING, offsetof(Config, rsh_command)}, {"--rsh", "-e", OPT_STRING, offsetof(Config, rsh_command)},
@@ -573,7 +573,7 @@ static const OptionEntry OPTION_TABLE[] = {
{"--append", NULL, OPT_FLAG, offsetof(Config, append)}, {"--append", NULL, OPT_FLAG, offsetof(Config, append)},
{"--append-verify", NULL, OPT_FLAG, offsetof(Config, append_verify)}, {"--append-verify", NULL, OPT_FLAG, offsetof(Config, append_verify)},
{"--fsync", NULL, OPT_FLAG, offsetof(Config, use_fsync)}, {"--fsync", NULL, OPT_FLAG, offsetof(Config, use_fsync)},
{"--checksum", NULL, OPT_FLAG, offsetof(Config, checksum)}, {"--checksum", "-c", OPT_FLAG, offsetof(Config, checksum)},
{"--8-bit-output", "-8", OPT_FLAG, offsetof(Config, eight_bit_output)}, {"--8-bit-output", "-8", OPT_FLAG, offsetof(Config, eight_bit_output)},
{"--itemize-changes", "-i", OPT_FLAG, offsetof(Config, itemize_changes)}, {"--itemize-changes", "-i", OPT_FLAG, offsetof(Config, itemize_changes)},
{"--list-only", NULL, OPT_FLAG, offsetof(Config, list_only)}, {"--list-only", NULL, OPT_FLAG, offsetof(Config, list_only)},
@@ -620,7 +620,7 @@ static const OptionEntry OPTION_TABLE[] = {
{"--max-delete", NULL, OPT_NONNEG_INT, offsetof(Config, max_delete)}, {"--max-delete", NULL, OPT_NONNEG_INT, offsetof(Config, max_delete)},
{"--ignore-errors", NULL, OPT_FLAG, offsetof(Config, ignore_errors)}, {"--ignore-errors", NULL, OPT_FLAG, offsetof(Config, ignore_errors)},
{"--force", NULL, OPT_FLAG, offsetof(Config, force_delete)}, {"--force", NULL, OPT_FLAG, offsetof(Config, force_delete)},
{"--prune-empty-dirs", NULL, OPT_FLAG, offsetof(Config, prune_empty_dirs)}, {"--prune-empty-dirs", "-m", OPT_FLAG, offsetof(Config, prune_empty_dirs)},
{"--ignore-missing-args", NULL, OPT_FLAG, offsetof(Config, ignore_missing_args)}, {"--ignore-missing-args", NULL, OPT_FLAG, offsetof(Config, ignore_missing_args)},
{"--delete-missing-args", NULL, OPT_FLAG, offsetof(Config, delete_missing_args)}, {"--delete-missing-args", NULL, OPT_FLAG, offsetof(Config, delete_missing_args)},
@@ -635,7 +635,7 @@ static const OptionEntry OPTION_TABLE[] = {
/* --rsync-path is rsync's spelling for the same "server program path"; it /* --rsync-path is rsync's spelling for the same "server program path"; it
* is a pure alias for fastsync_server_path (never a distinct field). */ * is a pure alias for fastsync_server_path (never a distinct field). */
{"--rsync-path", NULL, OPT_STRING, offsetof(Config, fastsync_server_path)}, {"--rsync-path", NULL, OPT_STRING, offsetof(Config, fastsync_server_path)},
{"--temp-dir", NULL, OPT_STRING, offsetof(Config, temp_dir)}, {"--temp-dir", "-T", OPT_STRING, offsetof(Config, temp_dir)},
{"--partial-dir", NULL, OPT_STRING, offsetof(Config, partial_dir)}, {"--partial-dir", NULL, OPT_STRING, offsetof(Config, partial_dir)},
{"--suffix", NULL, OPT_STRING, offsetof(Config, suffix)}, {"--suffix", NULL, OPT_STRING, offsetof(Config, suffix)},
{"--compress-choice", "--zc", OPT_STRING, offsetof(Config, compress_choice)}, {"--compress-choice", "--zc", OPT_STRING, offsetof(Config, compress_choice)},
@@ -668,10 +668,10 @@ static const OptionEntry OPTION_TABLE[] = {
{"--xattrs", "-X", OPT_FLAG, offsetof(Config, preserve_xattrs)}, {"--xattrs", "-X", OPT_FLAG, offsetof(Config, preserve_xattrs)},
{"--acls", "-A", OPT_FLAG, offsetof(Config, preserve_acls)}, {"--acls", "-A", OPT_FLAG, offsetof(Config, preserve_acls)},
{"--fake-super", NULL, OPT_FLAG, offsetof(Config, fake_super)}, {"--fake-super", NULL, OPT_FLAG, offsetof(Config, fake_super)},
/* Long-form-only: rsync's -M short form of --remote-option is INTENTIONALLY /* rsync's -M/--remote-option: -M is now the short alias for --remote-option
* unavailable because -M already means metadata mode in FastSync (a * (metadata mode is long-only --preserve), handled in the parse loop where
* documented divergence; see RSYNC_COMPAT.md). --trust-sender is a local * --remote-option is parsed. --trust-sender is a local receiver policy and
* receiver policy and never travels to the remote peer. */ * never travels to the remote peer. */
{"--trust-sender", NULL, OPT_FLAG, offsetof(Config, trust_sender)}, {"--trust-sender", NULL, OPT_FLAG, offsetof(Config, trust_sender)},
}; };
@@ -696,17 +696,17 @@ static const NegatableOption NEGATABLE_OPTIONS[] = {
{"sparse", "S", offsetof(Config, preserve_sparse)}, {"sparse", "S", offsetof(Config, preserve_sparse)},
{"inplace", NULL, offsetof(Config, inplace)}, {"inplace", NULL, offsetof(Config, inplace)},
{"preallocate", NULL, offsetof(Config, preallocate)}, {"preallocate", NULL, offsetof(Config, preallocate)},
{"checksum", NULL, offsetof(Config, checksum)}, {"checksum", "c", offsetof(Config, checksum)},
{"from0", NULL, offsetof(Config, from0)}, {"from0", NULL, offsetof(Config, from0)},
{"cvs-exclude", NULL, offsetof(Config, cvs_exclude)}, {"cvs-exclude", NULL, offsetof(Config, cvs_exclude)},
/* These options are also implied by --archive or handled outside the table. */ /* These options are also implied by --archive or handled outside the table. */
{"compress", "c", offsetof(Config, use_compression)}, {"compress", NULL, offsetof(Config, use_compression)},
{"compress", "z", offsetof(Config, use_compression)}, {"compress", "z", offsetof(Config, use_compression)},
{"multithreading", "m", offsetof(Config, use_multithreading)}, {"multithreading", "j", offsetof(Config, use_multithreading)},
{"preserve", "M", offsetof(Config, use_metadata)}, {"preserve", NULL, offsetof(Config, use_metadata)},
{"sendfile", "f", offsetof(Config, use_sendfile)}, {"sendfile", NULL, offsetof(Config, use_sendfile)},
{"chunk-serialization", "s", offsetof(Config, use_chunk_serialization)}, {"chunk-serialization", NULL, offsetof(Config, use_chunk_serialization)},
{"xattrs", "X", offsetof(Config, preserve_xattrs)}, {"xattrs", "X", offsetof(Config, preserve_xattrs)},
{"acls", "A", offsetof(Config, preserve_acls)}, {"acls", "A", offsetof(Config, preserve_acls)},
{"fake-super", NULL, offsetof(Config, fake_super)}, {"fake-super", NULL, offsetof(Config, fake_super)},
@@ -1029,17 +1029,35 @@ int parse_args(Config* config, int argc, char* argv[], int* positional_args,
config->preserve_specials = true; config->preserve_specials = true;
log_info_message(LOG_INFO_MISC, "Enabled preservation of device and special files (-D)"); log_info_message(LOG_INFO_MISC, "Enabled preservation of device and special files (-D)");
} else if (opt_is(argv[i], "-a", "--archive")) { } else if (opt_is(argv[i], "-a", "--archive")) {
config->use_compression = /* Real rsync archive (-rlptgoD). FastSync is always recursive and always
!config->compress_choice || strcmp(config->compress_choice, "zstd") == 0; * preserves hard-link/other transfer semantics per its own flags, so -a
config->use_multithreading = true; * implies links, full metadata (perms/times/group/owner as FastSync's
* broad bundle), devices and specials. Compression and multithreading
* are NOT implied (they are no longer part of archive mode). */
config->follow_symlinks = true;
config->use_metadata = true; config->use_metadata = true;
log_info_message(LOG_INFO_MISC, "Enabled archive mode (-c -m -M)"); config->preserve_devices = true;
} else if (opt_is(argv[i], "-p", NULL)) { config->preserve_specials = true;
log_info_message(LOG_INFO_MISC,
"Enabled archive mode (-rlptgoD: links, metadata, devices, specials)");
} else if (opt_is(argv[i], "-p", "--perms")) {
/* rsync -p/--perms: preserve permission bits. Folded into FastSync's
* broad metadata bundle (mode/mtime travel together). */
config->use_metadata = true;
log_info_message(LOG_INFO_MISC, "Enabled permission preservation");
} else if (opt_is(argv[i], "--ssh-port", NULL)) {
if (i + 1 >= argc) { if (i + 1 >= argc) {
log_message(LOG_LEVEL_ERROR, "missing argument for %s", argv[i]); log_message(LOG_LEVEL_ERROR, "missing argument for %s", argv[i]);
return -1; return -1;
} }
if (set_positive_int_option(&config->ssh_port, argv[++i], "-p") != 0) if (set_positive_int_option(&config->ssh_port, argv[++i], "--ssh-port") != 0)
return -1;
if (config->ssh_port > 65535) {
log_message(LOG_LEVEL_ERROR, "SSH port must be 1-65535");
return -1;
}
} else if (strncmp(argv[i], "--ssh-port=", 11) == 0) {
if (set_positive_int_option(&config->ssh_port, argv[i] + 11, "--ssh-port") != 0)
return -1; return -1;
if (config->ssh_port > 65535) { if (config->ssh_port > 65535) {
log_message(LOG_LEVEL_ERROR, "SSH port must be 1-65535"); log_message(LOG_LEVEL_ERROR, "SSH port must be 1-65535");
@@ -1085,7 +1103,7 @@ int parse_args(Config* config, int argc, char* argv[], int* positional_args,
config->delta_max_file_size = val; config->delta_max_file_size = val;
else else
log_message(LOG_LEVEL_WARNING, "--delta-max value %llu too small, using default", val); log_message(LOG_LEVEL_WARNING, "--delta-max value %llu too small, using default", val);
} else if (opt_is(argv[i], "-c", "-z")) { } else if (opt_is(argv[i], "-z", "--compress")) {
config->use_compression = config->use_compression =
!config->compress_choice || strcmp(config->compress_choice, "zstd") == 0; !config->compress_choice || strcmp(config->compress_choice, "zstd") == 0;
log_info_message(LOG_INFO_MISC, "Enabled Compression"); log_info_message(LOG_INFO_MISC, "Enabled Compression");
@@ -1102,20 +1120,20 @@ int parse_args(Config* config, int argc, char* argv[], int* positional_args,
i++; i++;
} }
} }
} else if (opt_is(argv[i], "-M", "--preserve")) { } else if (opt_is(argv[i], "--preserve", NULL)) {
config->use_metadata = true; config->use_metadata = true;
log_info_message(LOG_INFO_MISC, "Enabled metadata preservation"); log_info_message(LOG_INFO_MISC, "Enabled metadata preservation");
} else if (opt_is(argv[i], "-E", "--executability")) { } else if (opt_is(argv[i], "-E", "--executability")) {
config->use_metadata = true; config->use_metadata = true;
config->use_executability = true; config->use_executability = true;
log_info_message(LOG_INFO_MISC, "Enabled executable permission preservation"); log_info_message(LOG_INFO_MISC, "Enabled executable permission preservation");
} else if (opt_is(argv[i], "-f", "--sendfile")) { } else if (opt_is(argv[i], "--sendfile", NULL)) {
config->use_sendfile = true; config->use_sendfile = true;
log_info_message(LOG_INFO_MISC, "Enabled sendfile"); log_info_message(LOG_INFO_MISC, "Enabled sendfile");
} else if (opt_is(argv[i], "-m", NULL)) { } else if (opt_is(argv[i], "-j", "--threads")) {
config->use_multithreading = true; config->use_multithreading = true;
log_info_message(LOG_INFO_MISC, "Enabled Multithreading"); log_info_message(LOG_INFO_MISC, "Enabled Multithreading");
} else if (opt_is(argv[i], "-s", NULL)) { } else if (opt_is(argv[i], "--chunk-serialization", NULL)) {
config->use_chunk_serialization = true; config->use_chunk_serialization = true;
log_info_message(LOG_INFO_MISC, "Enabled Chunk Serialization"); log_info_message(LOG_INFO_MISC, "Enabled Chunk Serialization");
} else if (opt_is(argv[i], "--server-port", NULL)) { } else if (opt_is(argv[i], "--server-port", NULL)) {
@@ -1210,7 +1228,10 @@ int parse_args(Config* config, int argc, char* argv[], int* positional_args,
} else if (strncmp(argv[i], "--filter=", 9) == 0) { } else if (strncmp(argv[i], "--filter=", 9) == 0) {
if (config_add_filter(config, argv[i] + 9) != 0) if (config_add_filter(config, argv[i] + 9) != 0)
return -1; return -1;
} else if (opt_is(argv[i], "--filter", NULL)) { } else if (strncmp(argv[i], "-f=", 3) == 0) {
if (config_add_filter(config, argv[i] + 3) != 0)
return -1;
} else if (opt_is(argv[i], "--filter", "-f")) {
if (i + 1 >= argc) { if (i + 1 >= argc) {
log_message(LOG_LEVEL_ERROR, "missing argument for %s", argv[i]); log_message(LOG_LEVEL_ERROR, "missing argument for %s", argv[i]);
return -1; return -1;
@@ -1248,13 +1269,6 @@ int parse_args(Config* config, int argc, char* argv[], int* positional_args,
} else if (opt_is(argv[i], "--info", NULL)) { } else if (opt_is(argv[i], "--info", NULL)) {
if (i + 1 >= argc || parse_info_flags(argv[++i], config) != 0) if (i + 1 >= argc || parse_info_flags(argv[++i], config) != 0)
return -1; return -1;
} else if (opt_is(argv[i], "-T", NULL)) {
if (i + 1 >= argc) {
log_message(LOG_LEVEL_ERROR, "missing argument for %s", argv[i]);
return -1;
}
if (set_positive_int_option(&config->timeout, argv[++i], "-T") != 0)
return -1;
} else if (strncmp(argv[i], "--skip-compress=", 16) == 0) { } else if (strncmp(argv[i], "--skip-compress=", 16) == 0) {
if (parse_skip_compress(config, argv[i] + 16) != 0) if (parse_skip_compress(config, argv[i] + 16) != 0)
return -1; return -1;
@@ -1308,7 +1322,10 @@ int parse_args(Config* config, int argc, char* argv[], int* positional_args,
} else if (strncmp(argv[i], "--remote-option=", 16) == 0) { } else if (strncmp(argv[i], "--remote-option=", 16) == 0) {
if (config_add_remote_option(config, argv[i] + 16, "--remote-option") != 0) if (config_add_remote_option(config, argv[i] + 16, "--remote-option") != 0)
return -1; return -1;
} else if (opt_is(argv[i], "--remote-option", NULL)) { } else if (strncmp(argv[i], "-M=", 3) == 0) {
if (config_add_remote_option(config, argv[i] + 3, "-M") != 0)
return -1;
} else if (opt_is(argv[i], "--remote-option", "-M")) {
if (i + 1 >= argc) { if (i + 1 >= argc) {
log_message(LOG_LEVEL_ERROR, "missing argument for --remote-option"); log_message(LOG_LEVEL_ERROR, "missing argument for --remote-option");
return -1; return -1;
src/client/usage.c
+22 -23
View File
@@ -17,12 +17,14 @@ void print_usage(void) {
printf(" /local/path TCP transport (requires server on localhost:8080)\n"); printf(" /local/path TCP transport (requires server on localhost:8080)\n");
printf("\n"); printf("\n");
printf("Options:\n"); printf("Options:\n");
printf(" -c [level] Enable compression (level 1-22, default 5)\n"); printf(" -c, --checksum Verify content by checksum instead of size+mtime\n");
printf(" -z [level] Alias for -c\n"); printf(" -z, --compress [level] Enable compression (level 1-22, default 5)\n");
printf(" -a, --archive Archive mode (-c -m -M)\n"); printf(" -a, --archive rsync archive mode (-rlptgoD): links, metadata,\n");
printf(" devices and specials (not compression/multithreading)\n");
printf(" -n, --dry-run Show what would be transferred\n"); printf(" -n, --dry-run Show what would be transferred\n");
printf(" --remove-source-files Remove regular source files after successful transfer\n"); printf(" --remove-source-files Remove regular source files after successful transfer\n");
printf(" -p <port> SSH port (default: 22)\n"); printf(" -p, --perms Preserve permission bits (part of the metadata bundle)\n");
printf(" --ssh-port <port> SSH port (default: 22)\n");
printf(" -e, --rsh <command> Remote shell to launch on the client for the SSH\n"); printf(" -e, --rsh <command> Remote shell to launch on the client for the SSH\n");
printf(" transport (default: ssh). The command may include\n"); printf(" transport (default: ssh). The command may include\n");
printf(" arguments, e.g. -e \"ssh -p 2222\"\n"); printf(" arguments, e.g. -e \"ssh -p 2222\"\n");
@@ -80,9 +82,8 @@ void print_usage(void) {
printf(" entry's destination mirror receiver-side. Independent of\n"); printf(" entry's destination mirror receiver-side. Independent of\n");
printf(" --delete (it does not imply --delete; a non-empty directory\n"); printf(" --delete (it does not imply --delete; a non-empty directory\n");
printf(" mirror is removed only with --force or --delete)\n"); printf(" mirror is removed only with --force or --delete)\n");
printf(" --prune-empty-dirs Do not transfer empty directory entries (--dirs mode);\n"); printf(" -m, --prune-empty-dirs Do not transfer empty directory entries (--dirs mode);\n");
printf(" recursive transfers never send empty dirs. rsync's -m\n"); printf(" recursive transfers never send empty dirs\n");
printf(" short form stays FastSync multithreading\n");
printf(" Note: each timing flag implies --delete. Combining a timing flag with\n"); printf(" Note: each timing flag implies --delete. Combining a timing flag with\n");
printf(" --no-delete (in either order) is rejected as a config error.\n"); printf(" --no-delete (in either order) is rejected as a config error.\n");
printf(" --ignore-existing Skip files that already exist on receiver\n"); printf(" --ignore-existing Skip files that already exist on receiver\n");
@@ -105,8 +106,8 @@ void print_usage(void) {
printf(" --files-from <file> Read the source file list from FILE (paths relative to the " printf(" --files-from <file> Read the source file list from FILE (paths relative to the "
"source root)\n"); "source root)\n");
printf(" -0, --from0 Entries in --files-from are NUL-delimited\n"); printf(" -0, --from0 Entries in --files-from are NUL-delimited\n");
printf(" --filter=RULE rsync-style filter rule (+/- include/exclude; repeatable; the\n"); printf(" -f, --filter=RULE rsync-style filter rule (+/- include/exclude; repeatable;\n");
printf(" rsync -f short form conflicts with FastSync sendfile -f)\n"); printf(" both --filter=RULE and the -f RULE / -f=RULE short forms work)\n");
printf(" -C, --cvs-exclude Auto-ignore common CVS/SCM files (.git/, .svn/, *.o, *~, ...)\n"); printf(" -C, --cvs-exclude Auto-ignore common CVS/SCM files (.git/, .svn/, *.o, *~, ...)\n");
printf(" -F Apply per-directory .rsync-filter files during the scan\n"); printf(" -F Apply per-directory .rsync-filter files during the scan\n");
printf(" --max-size <n> Skip files larger than n bytes\n"); printf(" --max-size <n> Skip files larger than n bytes\n");
@@ -143,10 +144,11 @@ void print_usage(void) {
DELTA_BLOCK_SIZE_DEFAULT); DELTA_BLOCK_SIZE_DEFAULT);
printf(" --delta-max <n> Max file size for delta transfer (default: %llu)\n", printf(" --delta-max <n> Max file size for delta transfer (default: %llu)\n",
DELTA_MAX_FILE_SIZE); DELTA_MAX_FILE_SIZE);
printf(" -m Enable multithreading\n"); printf(" -j, --threads Enable multithreading\n");
printf(" -s Enable chunk serialization\n"); printf(" --chunk-serialization Enable chunk serialization (long form only)\n");
printf(" --secluded-args Accept rsync compatibility option (no effect)\n"); printf(" -s, --secluded-args Protect-args compatibility option (no effect; remote\n");
printf(" -f Enable sendfile (TCP only, not with -c or -s)\n"); printf(" SSH argv is already built injection-safe)\n");
printf(" --sendfile Enable sendfile zero-copy (TCP only; long form only)\n");
printf(" --compress-choice <alg> Compression algorithm (default: zstd)\n"); printf(" --compress-choice <alg> Compression algorithm (default: zstd)\n");
printf(" --zc <alg> Alias for --compress-choice\n"); printf(" --zc <alg> Alias for --compress-choice\n");
printf(" -v, --verbose Enable debug logging\n"); printf(" -v, --verbose Enable debug logging\n");
@@ -154,7 +156,7 @@ void print_usage(void) {
printf(" --debug=FLAGS Fine-grained debug logging (use --debug=help for flags)\n"); printf(" --debug=FLAGS Fine-grained debug logging (use --debug=help for flags)\n");
printf(" --info=FLAGS Fine-grained info: copy,misc,skip,stats,all,none\n"); printf(" --info=FLAGS Fine-grained info: copy,misc,skip,stats,all,none\n");
printf(" none suppresses info even with --verbose\n"); printf(" none suppresses info even with --verbose\n");
printf(" -M, --preserve Preserve file metadata\n"); printf(" --preserve Preserve file metadata (long form only)\n");
printf(" -E, --executability Preserve executable permission bits\n"); printf(" -E, --executability Preserve executable permission bits\n");
printf(" -X, --xattrs Preserve user extended attributes (user.* only;\n"); printf(" -X, --xattrs Preserve user extended attributes (user.* only;\n");
printf(" privileged security.*/trusted.* namespaces are\n"); printf(" privileged security.*/trusted.* namespaces are\n");
@@ -178,8 +180,8 @@ void print_usage(void) {
printf(" USER:GROUP, USER (owner only), :GROUP (group only); a\n"); printf(" USER:GROUP, USER (owner only), :GROUP (group only); a\n");
printf(" value of * means the current/root user as appropriate.\n"); printf(" value of * means the current/root user as appropriate.\n");
printf(" Names resolve on the source machine; @N for numerics.\n"); printf(" Names resolve on the source machine; @N for numerics.\n");
printf(" Note: -M is already FastSync's preserve flag; these use\n"); printf(" (Metadata is enabled with --preserve; -M now means\n");
printf(" long forms only.\n"); printf(" rsync's --remote-option.)\n");
printf(" --chunk-size <n> Chunk size in bytes (default: %d)\n", DEFAULT_CHUNK_SIZE); printf(" --chunk-size <n> Chunk size in bytes (default: %d)\n", DEFAULT_CHUNK_SIZE);
printf(" --source-dir <path> Source directory\n"); printf(" --source-dir <path> Source directory\n");
printf(" --dest-dir <path> Destination directory\n"); printf(" --dest-dir <path> Destination directory\n");
@@ -197,8 +199,7 @@ void print_usage(void) {
printf(" --cert <path> TLS certificate file (PEM)\n"); printf(" --cert <path> TLS certificate file (PEM)\n");
printf(" --key <path> TLS private key file (PEM)\n"); printf(" --key <path> TLS private key file (PEM)\n");
printf(" --ca <path> TLS CA certificate file (PEM)\n"); printf(" --ca <path> TLS CA certificate file (PEM)\n");
printf(" --timeout <sec> I/O timeout in seconds (default: 30)\n"); printf(" --timeout <sec> I/O timeout in seconds (default: 30; long form only)\n");
printf(" -T <sec> Alias for --timeout\n");
printf(" --contimeout <sec> Connection timeout in seconds (default: 10)\n"); printf(" --contimeout <sec> Connection timeout in seconds (default: 10)\n");
printf(" --stop-after=MINS Stop the transfer after MINS minutes (a positive\n"); printf(" --stop-after=MINS Stop the transfer after MINS minutes (a positive\n");
printf(" integer); whatever was already transferred is kept\n"); printf(" integer); whatever was already transferred is kept\n");
@@ -227,17 +228,15 @@ void print_usage(void) {
printf(" --stderr=MODE Route logging to stderr: errors or all\n"); printf(" --stderr=MODE Route logging to stderr: errors or all\n");
printf(" --partial Keep partial files on interrupted transfer\n"); printf(" --partial Keep partial files on interrupted transfer\n");
printf(" --partial-dir <dir> Directory for partial files\n"); printf(" --partial-dir <dir> Directory for partial files\n");
printf(" --temp-dir <dir> Scratch dir for temp files before atomic install\n"); printf(" -T, --temp-dir <dir> Scratch dir for temp files before atomic install\n");
printf(" --fastsync-server-path <path>\n"); printf(" --fastsync-server-path <path>\n");
printf(" Path to fastsync-server on remote (default: fastsync-server)\n"); printf(" Path to fastsync-server on remote (default: fastsync-server)\n");
printf( printf(
" --old-args Disable safe SSH command argument quoting (legacy compatibility)\n"); " --old-args Disable safe SSH command argument quoting (legacy compatibility)\n");
printf(" --remote-option=OPT Append OPT to the REMOTE server invocation over SSH\n"); printf(" -M, --remote-option=OPT Append OPT to the REMOTE server invocation over SSH\n");
printf(" (repeatable; each value is single-quote-escaped on the remote\n"); printf(" (repeatable; each value is single-quote-escaped on the remote\n");
printf(" command line; empty values and values with control characters\n"); printf(" command line; empty values and values with control characters\n");
printf(" are rejected). Long form only: rsync's -M short form is NOT\n"); printf(" are rejected; -M OPT, -M=OPT and --remote-option=OPT work)\n");
printf(" available because -M already means metadata preservation in\n");
printf(" FastSync (documented divergence)\n");
printf(" --trust-sender Trust the remote sender's file list: the receiver skips its\n"); printf(" --trust-sender Trust the remote sender's file list: the receiver skips its\n");
printf(" own up-front path-traversal/containment re-validation of the\n"); printf(" own up-front path-traversal/containment re-validation of the\n");
printf(" incoming file list (fewer checks, faster, potentially unsafe).\n"); printf(" incoming file list (fewer checks, faster, potentially unsafe).\n");
tests/integration/test_append.py
+1 -1
View File
@@ -138,7 +138,7 @@ class TestAppend:
self._place(source, REL, prefix + added) self._place(source, REL, prefix + added)
self._place(self._dest_file(source, dest, ""), REL, prefix) self._place(self._dest_file(source, dest, ""), REL, prefix)
result, _ = run_client(source, dest, flags=["--append", "-m"], port=shared_server.port) result, _ = run_client(source, dest, flags=["--append", "--threads"], port=shared_server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"--append -m failed: {(result.stderr or result.stdout)[:400]}" f"--append -m failed: {(result.stderr or result.stdout)[:400]}"
assert self._read(self._dest_file(source, dest, ""), REL) == prefix + added assert self._read(self._dest_file(source, dest, ""), REL) == prefix + added
tests/integration/test_features.py
+117 -117
View File
@@ -108,7 +108,7 @@ class TestDeviceSpecial:
self._setup() self._setup()
os.mkfifo(os.path.join(DEVICE_SOURCE, "pipe.fifo")) os.mkfifo(os.path.join(DEVICE_SOURCE, "pipe.fifo"))
result, _ = run_client(DEVICE_SOURCE, DEVICE_DEST, result, _ = run_client(DEVICE_SOURCE, DEVICE_DEST,
flags=["-m", "--remove-source-files", "--specials"], flags=["--threads", "--remove-source-files", "--specials"],
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, ( assert result.returncode == 0, (
f"Exit {result.returncode}: {result.stderr[:300]}" f"Exit {result.returncode}: {result.stderr[:300]}"
@@ -129,7 +129,7 @@ class TestDeviceSpecial:
src_dev = os.path.join(DEVICE_SOURCE, "realdev") src_dev = os.path.join(DEVICE_SOURCE, "realdev")
os.mknod(src_dev, stat.S_IFCHR | 0o666, os.makedev(1, 3)) os.mknod(src_dev, stat.S_IFCHR | 0o666, os.makedev(1, 3))
result, _ = run_client(DEVICE_SOURCE, DEVICE_DEST, result, _ = run_client(DEVICE_SOURCE, DEVICE_DEST,
flags=["-m", "--remove-source-files", "--devices"], flags=["--threads", "--remove-source-files", "--devices"],
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, ( assert result.returncode == 0, (
f"Exit {result.returncode}: {result.stderr[:300]}" f"Exit {result.returncode}: {result.stderr[:300]}"
@@ -263,7 +263,7 @@ class TestRemoveSourceFiles:
os.makedirs(os.path.join(source, "directory")) os.makedirs(os.path.join(source, "directory"))
os.symlink("one.txt", os.path.join(source, "link.txt")) os.symlink("one.txt", os.path.join(source, "link.txt"))
result, _ = run_client(source, dest, flags=["--remove-source-files", "-m"], result, _ = run_client(source, dest, flags=["--remove-source-files", "--threads"],
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, f"Remove-source sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"Remove-source sync failed: {result.stderr[:200]}"
assert not os.path.exists(os.path.join(source, "one.txt")) assert not os.path.exists(os.path.join(source, "one.txt"))
@@ -323,7 +323,7 @@ class TestRemoveSourceFiles:
# The seed run preserves timestamps (-M) so the destination copy has the # The seed run preserves timestamps (-M) so the destination copy has the
# source's exact mtime; otherwise the incremental skip would depend on # source's exact mtime; otherwise the incremental skip would depend on
# both writes landing in the same whole second (a race). # both writes landing in the same whole second (a race).
result, _ = run_client(source, dest, flags=["-M"], port=shared_server.port) result, _ = run_client(source, dest, flags=["--preserve"], port=shared_server.port)
assert result.returncode == 0 assert result.returncode == 0
result, _ = run_client(source, dest, result, _ = run_client(source, dest,
flags=["--remove-source-files", "--incremental"], flags=["--remove-source-files", "--incremental"],
@@ -352,7 +352,7 @@ class TestArchiveMode:
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, dur = run_client( result, dur = run_client(
SOURCE_DIR, DEST_DIR, SOURCE_DIR, DEST_DIR,
flags=["--archive", "--no-compress", "--no-m", "--no-preserve"], flags=["--archive", "--no-links", "--no-preserve"],
port=shared_server.port, port=shared_server.port,
) )
if result.returncode != 0: if result.returncode != 0:
@@ -477,7 +477,7 @@ class TestSkipCompress:
f.write((b"skip compression case test\n" * 100)) f.write((b"skip compression case test\n" * 100))
result, _ = run_client( result, _ = run_client(
SOURCE_DIR, DEST_DIR, SOURCE_DIR, DEST_DIR,
flags=["-c", "--skip-compress=.txt"], flags=["-z", "--skip-compress=.txt"],
port=shared_server.port, port=shared_server.port,
) )
assert result.returncode == 0, f"Skip-compress sync failed: {(result.stderr or result.stdout)[:200]}" assert result.returncode == 0, f"Skip-compress sync failed: {(result.stderr or result.stdout)[:200]}"
@@ -489,7 +489,7 @@ class TestSkipCompress:
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client( result, _ = run_client(
SOURCE_DIR, DEST_DIR, SOURCE_DIR, DEST_DIR,
flags=["-c", "--skip-compress="], flags=["-z", "--skip-compress="],
port=shared_server.port, port=shared_server.port,
) )
assert result.returncode == 0, f"Empty skip-compress sync failed: {(result.stderr or result.stdout)[:200]}" assert result.returncode == 0, f"Empty skip-compress sync failed: {(result.stderr or result.stdout)[:200]}"
@@ -503,7 +503,7 @@ class TestSkipCompress:
path = os.path.join(SOURCE_DIR, "incremental-skip.TXT") path = os.path.join(SOURCE_DIR, "incremental-skip.TXT")
with open(path, "wb") as f: with open(path, "wb") as f:
f.write(b"original skipped content\n") f.write(b"original skipped content\n")
flags = ["-c", "-M", "--skip-compress=.txt"] flags = ["-z", "--preserve", "--skip-compress=.txt"]
result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=flags, port=shared_server.port) result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"Initial sync failed: {(result.stderr or result.stdout)[:200]}" assert result.returncode == 0, f"Initial sync failed: {(result.stderr or result.stdout)[:200]}"
with open(path, "wb") as f: with open(path, "wb") as f:
@@ -521,7 +521,7 @@ class TestSkipCompress:
def test_skip_compress_rejects_chunk_serialization(self, shared_server): def test_skip_compress_rejects_chunk_serialization(self, shared_server):
result, _ = run_client( result, _ = run_client(
SOURCE_DIR, DEST_DIR, SOURCE_DIR, DEST_DIR,
flags=["-c", "-s", "--skip-compress=.txt"], flags=["-z", "--chunk-serialization", "--skip-compress=.txt"],
port=shared_server.port, port=shared_server.port,
) )
assert result.returncode != 0 assert result.returncode != 0
@@ -620,7 +620,7 @@ class TestIncremental:
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, dur = run_client( result, dur = run_client(
SOURCE_DIR, DEST_DIR, SOURCE_DIR, DEST_DIR,
flags=["-M"], flags=["--preserve"],
port=shared_server.port, port=shared_server.port,
) )
assert result.returncode == 0, f"First sync failed: {result.stderr[:100]}" assert result.returncode == 0, f"First sync failed: {result.stderr[:100]}"
@@ -628,7 +628,7 @@ class TestIncremental:
start = time.monotonic() start = time.monotonic()
result, dur = run_client( result, dur = run_client(
SOURCE_DIR, DEST_DIR, SOURCE_DIR, DEST_DIR,
flags=["-M", "--incremental"], flags=["--preserve", "--incremental"],
port=shared_server.port, port=shared_server.port,
) )
incremental_time = time.monotonic() - start incremental_time = time.monotonic() - start
@@ -645,7 +645,7 @@ class TestIncremental:
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client( result, _ = run_client(
SOURCE_DIR, DEST_DIR, SOURCE_DIR, DEST_DIR,
flags=["-M"], flags=["--preserve"],
port=shared_server.port, port=shared_server.port,
) )
assert result.returncode == 0 assert result.returncode == 0
@@ -656,7 +656,7 @@ class TestIncremental:
result, dur = run_client( result, dur = run_client(
SOURCE_DIR, DEST_DIR, SOURCE_DIR, DEST_DIR,
flags=["-M", "--incremental"], flags=["--preserve", "--incremental"],
port=shared_server.port, port=shared_server.port,
) )
assert result.returncode == 0 assert result.returncode == 0
@@ -673,7 +673,7 @@ class TestIncremental:
def test_checksum_detects_same_size_and_mtime_change(self, shared_server): def test_checksum_detects_same_size_and_mtime_change(self, shared_server):
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["-M"], port=shared_server.port) result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["--preserve"], port=shared_server.port)
assert result.returncode == 0 assert result.returncode == 0
received = get_dest_received_dir(DEST_DIR, SOURCE_DIR) received = get_dest_received_dir(DEST_DIR, SOURCE_DIR)
@@ -685,7 +685,7 @@ class TestIncremental:
os.utime(received_file, (source_stat.st_atime, source_stat.st_mtime)) os.utime(received_file, (source_stat.st_atime, source_stat.st_mtime))
result, _ = run_client(SOURCE_DIR, DEST_DIR, result, _ = run_client(SOURCE_DIR, DEST_DIR,
flags=["-M", "--incremental", "--checksum"], flags=["--preserve", "--incremental", "--checksum"],
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, f"Checksum sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"Checksum sync failed: {result.stderr[:200]}"
with open(received_file, "rb") as f: with open(received_file, "rb") as f:
@@ -693,7 +693,7 @@ class TestIncremental:
def test_size_only_skips_same_size_with_different_mtime(self, shared_server): def test_size_only_skips_same_size_with_different_mtime(self, shared_server):
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["-M"], port=shared_server.port) result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["--preserve"], port=shared_server.port)
assert result.returncode == 0 assert result.returncode == 0
received = get_dest_received_dir(DEST_DIR, SOURCE_DIR) received = get_dest_received_dir(DEST_DIR, SOURCE_DIR)
@@ -704,7 +704,7 @@ class TestIncremental:
result, _ = run_client( result, _ = run_client(
SOURCE_DIR, DEST_DIR, SOURCE_DIR, DEST_DIR,
flags=["-M", "--incremental", "--size-only"], flags=["--preserve", "--incremental", "--size-only"],
port=shared_server.port, port=shared_server.port,
) )
assert result.returncode == 0, f"Size-only sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"Size-only sync failed: {result.stderr[:200]}"
@@ -713,7 +713,7 @@ class TestIncremental:
def test_ignore_times_transfers_same_size_and_mtime(self, shared_server): def test_ignore_times_transfers_same_size_and_mtime(self, shared_server):
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["-M"], port=shared_server.port) result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["--preserve"], port=shared_server.port)
assert result.returncode == 0 assert result.returncode == 0
received = get_dest_received_dir(DEST_DIR, SOURCE_DIR) received = get_dest_received_dir(DEST_DIR, SOURCE_DIR)
@@ -725,7 +725,7 @@ class TestIncremental:
os.utime(received_file, (source_stat.st_atime, source_stat.st_mtime)) os.utime(received_file, (source_stat.st_atime, source_stat.st_mtime))
result, _ = run_client(SOURCE_DIR, DEST_DIR, result, _ = run_client(SOURCE_DIR, DEST_DIR,
flags=["-M", "--incremental", "--ignore-times"], flags=["--preserve", "--incremental", "--ignore-times"],
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, f"Ignore-times sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"Ignore-times sync failed: {result.stderr[:200]}"
with open(received_file, "rb") as f: with open(received_file, "rb") as f:
@@ -733,7 +733,7 @@ class TestIncremental:
def test_modify_window_allows_subsecond_mtime_difference(self, shared_server): def test_modify_window_allows_subsecond_mtime_difference(self, shared_server):
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["-M"], port=shared_server.port) result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["--preserve"], port=shared_server.port)
assert result.returncode == 0 assert result.returncode == 0
received = get_dest_received_dir(DEST_DIR, SOURCE_DIR) received = get_dest_received_dir(DEST_DIR, SOURCE_DIR)
@@ -746,7 +746,7 @@ class TestIncremental:
source_stat.st_mtime_ns - 1500000000)) source_stat.st_mtime_ns - 1500000000))
result, _ = run_client(SOURCE_DIR, DEST_DIR, result, _ = run_client(SOURCE_DIR, DEST_DIR,
flags=["-M", "--incremental", "--modify-window=2"], flags=["--preserve", "--incremental", "--modify-window=2"],
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, f"Modify-window sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"Modify-window sync failed: {result.stderr[:200]}"
with open(received_file, "rb") as f: with open(received_file, "rb") as f:
@@ -754,7 +754,7 @@ class TestIncremental:
def test_whole_file_disables_delta_and_keeps_compression(self, shared_server): def test_whole_file_disables_delta_and_keeps_compression(self, shared_server):
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["-M"], port=shared_server.port) result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["--preserve"], port=shared_server.port)
assert result.returncode == 0 assert result.returncode == 0
source_file = os.path.join(SOURCE_DIR, "medium.txt") source_file = os.path.join(SOURCE_DIR, "medium.txt")
@@ -764,7 +764,7 @@ class TestIncremental:
result, _ = run_client( result, _ = run_client(
SOURCE_DIR, SOURCE_DIR,
DEST_DIR, DEST_DIR,
flags=["-M", "--incremental", "--delta", "-W", "-c"], flags=["--preserve", "--incremental", "--delta", "-W", "-z"],
port=shared_server.port, port=shared_server.port,
) )
assert result.returncode == 0, f"Whole-file sync failed: {(result.stderr or result.stdout)[:200]}" assert result.returncode == 0, f"Whole-file sync failed: {(result.stderr or result.stdout)[:200]}"
@@ -795,11 +795,11 @@ class TestChecksumChoice:
@pytest.mark.parametrize("mt", [False, True]) @pytest.mark.parametrize("mt", [False, True])
def test_unchanged_skipped_and_bytes_preserved(self, shared_server, algo, mt): def test_unchanged_skipped_and_bytes_preserved(self, shared_server, algo, mt):
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["-M"], port=shared_server.port) result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["--preserve"], port=shared_server.port)
assert result.returncode == 0, f"seed sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"seed sync failed: {result.stderr[:200]}"
flags = (["-M", "--incremental", "--checksum", f"--checksum-choice={algo}"] + flags = (["--preserve", "--incremental", "--checksum", f"--checksum-choice={algo}"] +
(["-m"] if mt else [])) (["--threads"] if mt else []))
result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=flags, port=shared_server.port) result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"checksum {algo} run failed: {result.stderr[:200]}" assert result.returncode == 0, f"checksum {algo} run failed: {result.stderr[:200]}"
@@ -816,7 +816,7 @@ class TestChecksumChoice:
@pytest.mark.parametrize("mt", [False, True]) @pytest.mark.parametrize("mt", [False, True])
def test_changed_same_size_mtime_redetected(self, shared_server, algo, mt): def test_changed_same_size_mtime_redetected(self, shared_server, algo, mt):
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["-M"], port=shared_server.port) result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["--preserve"], port=shared_server.port)
assert result.returncode == 0 assert result.returncode == 0
received = get_dest_received_dir(DEST_DIR, SOURCE_DIR) received = get_dest_received_dir(DEST_DIR, SOURCE_DIR)
@@ -827,8 +827,8 @@ class TestChecksumChoice:
f.write(b"DDDDDDDDDDDD") # same size, different content f.write(b"DDDDDDDDDDDD") # same size, different content
os.utime(received_file, (source_stat.st_atime, source_stat.st_mtime)) os.utime(received_file, (source_stat.st_atime, source_stat.st_mtime))
flags = (["-M", "--incremental", "--checksum", f"--checksum-choice={algo}"] + flags = (["--preserve", "--incremental", "--checksum", f"--checksum-choice={algo}"] +
(["-m"] if mt else [])) (["--threads"] if mt else []))
result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=flags, port=shared_server.port) result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"checksum {algo} redetect failed: {result.stderr[:200]}" assert result.returncode == 0, f"checksum {algo} redetect failed: {result.stderr[:200]}"
with open(received_file, "rb") as f: with open(received_file, "rb") as f:
@@ -840,10 +840,10 @@ class TestChecksumChoice:
# small handshake travels, not the payloads. Proxy byte counts are not # small handshake travels, not the payloads. Proxy byte counts are not
# available for -m (multithreaded connections), so single-thread only. # available for -m (multithreaded connections), so single-thread only.
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["-M"], port=shared_server.port) result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["--preserve"], port=shared_server.port)
assert result.returncode == 0 assert result.returncode == 0
flags = ["-M", "--incremental", "--checksum", f"--checksum-choice={algo}"] flags = ["--preserve", "--incremental", "--checksum", f"--checksum-choice={algo}"]
proxy = CountingProxy(shared_server.port) proxy = CountingProxy(shared_server.port)
cmd = (CLIENT_CMD + ["--source-dir", SOURCE_DIR, "--dest-dir", DEST_DIR, cmd = (CLIENT_CMD + ["--source-dir", SOURCE_DIR, "--dest-dir", DEST_DIR,
"--save-to-disk", "--server-port", str(proxy.port)] + flags) "--save-to-disk", "--server-port", str(proxy.port)] + flags)
@@ -855,8 +855,8 @@ class TestChecksumChoice:
@pytest.mark.parametrize("mt", [False, True]) @pytest.mark.parametrize("mt", [False, True])
def test_seed_is_deterministic_and_preserves_content(self, shared_server, mt): def test_seed_is_deterministic_and_preserves_content(self, shared_server, mt):
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
flags = ["-M", "--incremental", "--checksum", flags = ["--preserve", "--incremental", "--checksum",
"--checksum-choice=xxh64", "--checksum-seed=987654"] + (["-m"] if mt else []) "--checksum-choice=xxh64", "--checksum-seed=987654"] + (["--threads"] if mt else [])
first, _ = run_client(SOURCE_DIR, DEST_DIR, flags=flags, port=shared_server.port) first, _ = run_client(SOURCE_DIR, DEST_DIR, flags=flags, port=shared_server.port)
assert first.returncode == 0, f"seeded run failed: {first.stderr[:200]}" assert first.returncode == 0, f"seeded run failed: {first.stderr[:200]}"
@@ -894,7 +894,7 @@ class TestChecksumChoice:
big = os.path.join(source, "big.bin") big = os.path.join(source, "big.bin")
with open(big, "wb") as f: with open(big, "wb") as f:
f.write(bytes(range(256)) * 200) # 51200 bytes > delta 16K floor f.write(bytes(range(256)) * 200) # 51200 bytes > delta 16K floor
result, _ = run_client(source, dest, flags=["-M"], port=shared_server.port) result, _ = run_client(source, dest, flags=["--preserve"], port=shared_server.port)
assert result.returncode == 0, f"seed delta seed failed: {result.stderr[:200]}" assert result.returncode == 0, f"seed delta seed failed: {result.stderr[:200]}"
# Edit a region so the receiver must match a changed block with the seed. # Edit a region so the receiver must match a changed block with the seed.
@@ -906,8 +906,8 @@ class TestChecksumChoice:
# edit and the prior sync share a second. Setting an old dest mtime # edit and the prior sync share a second. Setting an old dest mtime
# guarantees the delta path is exercised deterministically. # guarantees the delta path is exercised deterministically.
os.utime(os.path.join(get_dest_received_dir(dest, source), "big.bin"), (0, 0)) os.utime(os.path.join(get_dest_received_dir(dest, source), "big.bin"), (0, 0))
flags = (["-M", "--incremental", "--delta", "--checksum-seed=314159"] + flags = (["--preserve", "--incremental", "--delta", "--checksum-seed=314159"] +
(["-m"] if mt else [])) (["--threads"] if mt else []))
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"seed delta run failed: {result.stderr[:200]}" assert result.returncode == 0, f"seed delta run failed: {result.stderr[:200]}"
received = get_dest_received_dir(dest, source) received = get_dest_received_dir(dest, source)
@@ -975,7 +975,7 @@ class TestExisting:
@pytest.mark.ci @pytest.mark.ci
def test_existing_updates_existing_and_skips_new(self, shared_server): def test_existing_updates_existing_and_skips_new(self, shared_server):
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["-M"], port=shared_server.port) result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["--preserve"], port=shared_server.port)
assert result.returncode == 0, f"Initial sync failed: {(result.stderr or result.stdout)[:200]}" assert result.returncode == 0, f"Initial sync failed: {(result.stderr or result.stdout)[:200]}"
received = get_dest_received_dir(DEST_DIR, SOURCE_DIR) received = get_dest_received_dir(DEST_DIR, SOURCE_DIR)
@@ -988,7 +988,7 @@ class TestExisting:
try: try:
result, _ = run_client(SOURCE_DIR, DEST_DIR, result, _ = run_client(SOURCE_DIR, DEST_DIR,
flags=["-M", "--existing"], port=shared_server.port) flags=["--preserve", "--existing"], port=shared_server.port)
assert result.returncode == 0, f"--existing sync failed: {(result.stderr or result.stdout)[:200]}" assert result.returncode == 0, f"--existing sync failed: {(result.stderr or result.stdout)[:200]}"
with open(os.path.join(received, "small.txt"), "rb") as f: with open(os.path.join(received, "small.txt"), "rb") as f:
@@ -1060,7 +1060,7 @@ class TestDelete:
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client( result, _ = run_client(
SOURCE_DIR, DEST_DIR, SOURCE_DIR, DEST_DIR,
flags=["-M"], flags=["--preserve"],
port=shared_server.port, port=shared_server.port,
) )
assert result.returncode == 0 assert result.returncode == 0
@@ -1076,7 +1076,7 @@ class TestDelete:
result, dur = run_client( result, dur = run_client(
SOURCE_DIR, DEST_DIR, SOURCE_DIR, DEST_DIR,
flags=["-M", "--delete"], flags=["--preserve", "--delete"],
port=shared_server.port, port=shared_server.port,
) )
@@ -1119,7 +1119,7 @@ class TestProgress:
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, dur = run_client( result, dur = run_client(
SOURCE_DIR, DEST_DIR, SOURCE_DIR, DEST_DIR,
flags=["-m", "-h", "--progress"], flags=["--threads", "-h", "--progress"],
port=shared_server.port, port=shared_server.port,
) )
assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr[:100]}" assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr[:100]}"
@@ -1145,7 +1145,7 @@ class TestInfo:
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client( result, _ = run_client(
SOURCE_DIR, DEST_DIR, SOURCE_DIR, DEST_DIR,
flags=["-m", "--info=stats"], flags=["--threads", "--info=stats"],
port=shared_server.port, port=shared_server.port,
) )
assert result.returncode == 0, f"Info stats sync failed: {(result.stderr or result.stdout)[:200]}" assert result.returncode == 0, f"Info stats sync failed: {(result.stderr or result.stdout)[:200]}"
@@ -1277,7 +1277,7 @@ class TestRemoveSourceFilesSkips:
assert result.returncode == 0 assert result.returncode == 0
result, _ = run_client(source, dest, result, _ = run_client(source, dest,
flags=["--remove-source-files", "--ignore-existing", "-m"], flags=["--remove-source-files", "--ignore-existing", "--threads"],
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, f"Sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"Sync failed: {result.stderr[:200]}"
# Destination already has the file, so the receiver (writer thread) # Destination already has the file, so the receiver (writer thread)
@@ -1425,7 +1425,7 @@ class TestOneFileSystem:
source = os.path.join(TEST_DATA_DIR, "ofs_m_src") source = os.path.join(TEST_DATA_DIR, "ofs_m_src")
self._make_tree(source) self._make_tree(source)
self._assert_full_tree_transferred(source, os.path.join(TEST_DATA_DIR, "ofs_m_dst"), self._assert_full_tree_transferred(source, os.path.join(TEST_DATA_DIR, "ofs_m_dst"),
shared_server.port, ["-m", "--one-file-system"]) shared_server.port, ["--threads", "--one-file-system"])
def test_x_skips_other_device_mountpoint(self, shared_server): def test_x_skips_other_device_mountpoint(self, shared_server):
if os.geteuid() != 0 or shutil.which("mount") is None or shutil.which("umount") is None: if os.geteuid() != 0 or shutil.which("mount") is None or shutil.which("umount") is None:
@@ -1518,7 +1518,7 @@ class TestTempDir:
source = self._make_source("tempdir_src") source = self._make_source("tempdir_src")
dest = os.path.join(TEST_DATA_DIR, "tempdir_dst") dest = os.path.join(TEST_DATA_DIR, "tempdir_dst")
clean_dir(dest) clean_dir(dest)
flags = ["--temp-dir=scratch"] + (["-m"] if mt else []) flags = ["--temp-dir=scratch"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"temp-dir sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"temp-dir sync failed: {result.stderr[:200]}"
received = get_dest_received_dir(dest, source) received = get_dest_received_dir(dest, source)
@@ -1629,7 +1629,7 @@ class TestListOnly:
def test_list_only_multithreaded(self): def test_list_only_multithreaded(self):
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["--list-only", "-m"]) result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["--list-only", "--threads"])
assert result.returncode == 0, f"list-only -m failed: {result.stderr[:200]}" assert result.returncode == 0, f"list-only -m failed: {result.stderr[:200]}"
for full_path in _source_files(): for full_path in _source_files():
assert full_path in result.stdout, f"list-only -m omitted {full_path}" assert full_path in result.stdout, f"list-only -m omitted {full_path}"
@@ -1643,7 +1643,7 @@ class TestItemizeChanges:
def test_first_run_prints_sent_lines(self, shared_server): def test_first_run_prints_sent_lines(self, shared_server):
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client(SOURCE_DIR, DEST_DIR, result, _ = run_client(SOURCE_DIR, DEST_DIR,
flags=["-M", "-i"], port=shared_server.port) flags=["--preserve", "-i"], port=shared_server.port)
assert result.returncode == 0, f"itemize sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"itemize sync failed: {result.stderr[:200]}"
sent_lines = {">f+++++++++ " + p for p in _source_files()} sent_lines = {">f+++++++++ " + p for p in _source_files()}
assert sent_lines <= set(result.stdout.splitlines()), ( assert sent_lines <= set(result.stdout.splitlines()), (
@@ -1652,10 +1652,10 @@ class TestItemizeChanges:
def test_incremental_second_run_prints_no_line_for_unchanged(self, shared_server): def test_incremental_second_run_prints_no_line_for_unchanged(self, shared_server):
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["-M"], port=shared_server.port) result, _ = run_client(SOURCE_DIR, DEST_DIR, flags=["--preserve"], port=shared_server.port)
assert result.returncode == 0, f"seed sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"seed sync failed: {result.stderr[:200]}"
result, _ = run_client(SOURCE_DIR, DEST_DIR, result, _ = run_client(SOURCE_DIR, DEST_DIR,
flags=["-M", "-i", "--incremental"], flags=["--preserve", "-i", "--incremental"],
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, f"incremental itemize failed: {result.stderr[:200]}" assert result.returncode == 0, f"incremental itemize failed: {result.stderr[:200]}"
itemized = [line for line in result.stdout.splitlines() if line and line[0] in ">.<c"] itemized = [line for line in result.stdout.splitlines() if line and line[0] in ">.<c"]
@@ -1664,7 +1664,7 @@ class TestItemizeChanges:
def test_multithreaded_emits_same_itemize_lines(self, shared_server): def test_multithreaded_emits_same_itemize_lines(self, shared_server):
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client(SOURCE_DIR, DEST_DIR, result, _ = run_client(SOURCE_DIR, DEST_DIR,
flags=["-M", "-i", "-m"], port=shared_server.port) flags=["--preserve", "-i", "--threads"], port=shared_server.port)
assert result.returncode == 0, f"itemize -m sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"itemize -m sync failed: {result.stderr[:200]}"
sent_lines = {">f+++++++++ " + p for p in _source_files()} sent_lines = {">f+++++++++ " + p for p in _source_files()}
assert sent_lines <= set(result.stdout.splitlines()), ( assert sent_lines <= set(result.stdout.splitlines()), (
@@ -1690,14 +1690,14 @@ class TestItemizeChanges:
with open(untouched, "wb") as fh: with open(untouched, "wb") as fh:
fh.write(b"stable\n") fh.write(b"stable\n")
result, _ = run_client(source, dest, flags=["-M"], port=shared_server.port) result, _ = run_client(source, dest, flags=["--preserve"], port=shared_server.port)
assert result.returncode == 0, f"seed sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"seed sync failed: {result.stderr[:200]}"
with open(changed, "wb") as fh: with open(changed, "wb") as fh:
fh.write(b"edited payload\n") fh.write(b"edited payload\n")
result, _ = run_client(source, dest, result, _ = run_client(source, dest,
flags=["-M", "-i", "--incremental"], flags=["--preserve", "-i", "--incremental"],
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, f"incremental itemize failed: {result.stderr[:200]}" assert result.returncode == 0, f"incremental itemize failed: {result.stderr[:200]}"
itemized = [line for line in result.stdout.splitlines() if line.startswith(">f")] itemized = [line for line in result.stdout.splitlines() if line.startswith(">f")]
@@ -1724,7 +1724,7 @@ class TestOutFormat:
def test_out_format_multithreaded_matches_single(self, shared_server): def test_out_format_multithreaded_matches_single(self, shared_server):
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
result, _ = run_client(SOURCE_DIR, DEST_DIR, result, _ = run_client(SOURCE_DIR, DEST_DIR,
flags=["--out-format=%f %l", "-m"], port=shared_server.port) flags=["--out-format=%f %l", "--threads"], port=shared_server.port)
assert result.returncode == 0, f"out-format -m sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"out-format -m sync failed: {result.stderr[:200]}"
expected = {f"{p} {os.path.getsize(p)}" for p in _source_files()} expected = {f"{p} {os.path.getsize(p)}" for p in _source_files()}
got = set(result.stdout.splitlines()) got = set(result.stdout.splitlines())
@@ -1767,7 +1767,7 @@ class TestLogFileFormat:
result, _ = run_client( result, _ = run_client(
source, source,
dest, dest,
flags=["--log-file", log_path, "--log-file-format=%f %l", "-m"], flags=["--log-file", log_path, "--log-file-format=%f %l", "--threads"],
port=shared_server.port, port=shared_server.port,
) )
assert result.returncode == 0, f"log-file -m sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"log-file -m sync failed: {result.stderr[:200]}"
@@ -1812,7 +1812,7 @@ class TestDelayUpdates:
result, _ = run_client(source, plain_dest, port=shared_server.port) result, _ = run_client(source, plain_dest, port=shared_server.port)
assert result.returncode == 0, f"plain sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"plain sync failed: {result.stderr[:200]}"
flags = ["--delay-updates"] + (["-m"] if mt else []) flags = ["--delay-updates"] + (["--threads"] if mt else [])
result, _ = run_client(source, delay_dest, flags=flags, port=shared_server.port) result, _ = run_client(source, delay_dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"delay-updates sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"delay-updates sync failed: {result.stderr[:200]}"
@@ -1834,7 +1834,7 @@ class TestDelayUpdates:
source = self._make_source("delay_rerun_src") source = self._make_source("delay_rerun_src")
dest = os.path.join(TEST_DATA_DIR, "delay_rerun_dst") dest = os.path.join(TEST_DATA_DIR, "delay_rerun_dst")
clean_dir(dest) clean_dir(dest)
flags = ["--delay-updates", "-M", "--incremental"] + (["-m"] if mt else []) flags = ["--delay-updates", "--preserve", "--incremental"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"first delayed sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"first delayed sync failed: {result.stderr[:200]}"
@@ -1853,7 +1853,7 @@ class TestDelayUpdates:
source = self._make_source("delay_rsf_src") source = self._make_source("delay_rsf_src")
dest = os.path.join(TEST_DATA_DIR, "delay_rsf_dst") dest = os.path.join(TEST_DATA_DIR, "delay_rsf_dst")
clean_dir(dest) clean_dir(dest)
flags = ["--remove-source-files", "--delay-updates"] + (["-m"] if mt else []) flags = ["--remove-source-files", "--delay-updates"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"delayed remove-source sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"delayed remove-source sync failed: {result.stderr[:200]}"
@@ -1891,7 +1891,7 @@ class TestDelayUpdates:
fh.write(b"BBBB") fh.write(b"BBBB")
os.remove(os.path.join(source, "extra.txt")) os.remove(os.path.join(source, "extra.txt"))
flags = ["--delete", "--delay-updates"] + (["-m"] if mt else []) flags = ["--delete", "--delay-updates"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"delete+delay-updates sync failed: {result.stderr[:200]}" f"delete+delay-updates sync failed: {result.stderr[:200]}"
@@ -1945,7 +1945,7 @@ class TestDelayUpdates:
with open(os.path.join(received, "sub"), "wb") as fh: with open(os.path.join(received, "sub"), "wb") as fh:
fh.write(b"blocks the nested destination directory") fh.write(b"blocks the nested destination directory")
flags = ["--delay-updates"] + (["-m"] if mt else []) flags = ["--delay-updates"] + (["--threads"] if mt else [])
if remove_source_files: if remove_source_files:
flags += ["--remove-source-files"] flags += ["--remove-source-files"]
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
@@ -1982,7 +1982,7 @@ class TestDelayUpdates:
fh.write(b"changed on source") fh.write(b"changed on source")
with open(os.path.join(source, "deliver.txt"), "wb") as fh: with open(os.path.join(source, "deliver.txt"), "wb") as fh:
fh.write(b"new file") fh.write(b"new file")
flags = ["--remove-source-files", "--ignore-existing", "--delay-updates"] + (["-m"] if mt else []) flags = ["--remove-source-files", "--ignore-existing", "--delay-updates"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"delayed skip sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"delayed skip sync failed: {result.stderr[:200]}"
# keep.txt already existed at the destination: receiver skip -> source stays. # keep.txt already existed at the destination: receiver skip -> source stays.
@@ -2045,7 +2045,7 @@ class TestRelativeFilesFrom:
dest = os.path.join(TEST_DATA_DIR, "rel_dst") dest = os.path.join(TEST_DATA_DIR, "rel_dst")
clean_dir(dest) clean_dir(dest)
lst = _write_rel_list(b"top.txt\nsub/x.txt\n") lst = _write_rel_list(b"top.txt\nsub/x.txt\n")
flags = ["--files-from", lst, "-R"] + (["-m"] if mt else []) flags = ["--files-from", lst, "-R"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"-R files-from sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"-R files-from sync failed: {result.stderr[:200]}"
assert _read_file(os.path.join(dest, "sub", "x.txt")) == b"x\n", \ assert _read_file(os.path.join(dest, "sub", "x.txt")) == b"x\n", \
@@ -2064,7 +2064,7 @@ class TestRelativeFilesFrom:
source = _make_relative_source("rel_only_src") source = _make_relative_source("rel_only_src")
dest = os.path.join(TEST_DATA_DIR, "rel_only_dst") dest = os.path.join(TEST_DATA_DIR, "rel_only_dst")
clean_dir(dest) clean_dir(dest)
flags = ["-R"] + (["-m"] if mt else []) flags = ["-R"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"-R alone sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"-R alone sync failed: {result.stderr[:200]}"
received = get_dest_received_dir(dest, source) received = get_dest_received_dir(dest, source)
@@ -2077,7 +2077,7 @@ class TestRelativeFilesFrom:
dest = os.path.join(TEST_DATA_DIR, "rel_noR_dst") dest = os.path.join(TEST_DATA_DIR, "rel_noR_dst")
clean_dir(dest) clean_dir(dest)
lst = _write_rel_list(b"sub/x.txt\n") lst = _write_rel_list(b"sub/x.txt\n")
flags = ["--files-from", lst] + (["-m"] if mt else []) flags = ["--files-from", lst] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"files-from sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"files-from sync failed: {result.stderr[:200]}"
received = get_dest_received_dir(dest, source) received = get_dest_received_dir(dest, source)
@@ -2140,7 +2140,7 @@ class TestMissingArgs:
dest = os.path.join(TEST_DATA_DIR, "mg_default_dst") dest = os.path.join(TEST_DATA_DIR, "mg_default_dst")
clean_dir(dest) clean_dir(dest)
lst = _write_rel_list(b"a.txt\ngone.txt\nsub/b.txt\n") lst = _write_rel_list(b"a.txt\ngone.txt\nsub/b.txt\n")
flags = ["--files-from", lst] + (["-m"] if mt else []) flags = ["--files-from", lst] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode != 0, "a listed-but-missing entry did not fail the run" assert result.returncode != 0, "a listed-but-missing entry did not fail the run"
assert "gone.txt" in (result.stderr or result.stdout) assert "gone.txt" in (result.stderr or result.stdout)
@@ -2154,7 +2154,7 @@ class TestMissingArgs:
dest = os.path.join(TEST_DATA_DIR, "mg_ignore_dst") dest = os.path.join(TEST_DATA_DIR, "mg_ignore_dst")
clean_dir(dest) clean_dir(dest)
lst = _write_rel_list(b"a.txt\ngone.txt\nsub/b.txt\n") lst = _write_rel_list(b"a.txt\ngone.txt\nsub/b.txt\n")
flags = ["--files-from", lst, "--ignore-missing-args"] + (["-m"] if mt else []) flags = ["--files-from", lst, "--ignore-missing-args"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"ignore-missing-args sync failed: {result.stderr[:300]}" assert result.returncode == 0, f"ignore-missing-args sync failed: {result.stderr[:300]}"
received = get_dest_received_dir(dest, source) received = get_dest_received_dir(dest, source)
@@ -2171,7 +2171,7 @@ class TestMissingArgs:
dest = os.path.join(TEST_DATA_DIR, "mg_all_missing_dst") dest = os.path.join(TEST_DATA_DIR, "mg_all_missing_dst")
clean_dir(dest) clean_dir(dest)
lst = _write_rel_list(b"gone1.txt\ngone2.txt\n") lst = _write_rel_list(b"gone1.txt\ngone2.txt\n")
flags = ["--files-from", lst, "--ignore-missing-args"] + (["-m"] if mt else []) flags = ["--files-from", lst, "--ignore-missing-args"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"all-missing run should succeed (rsync parity): {result.stderr[:300]}" f"all-missing run should succeed (rsync parity): {result.stderr[:300]}"
@@ -2184,7 +2184,7 @@ class TestMissingArgs:
dest = os.path.join(TEST_DATA_DIR, "mg_empty_dst") dest = os.path.join(TEST_DATA_DIR, "mg_empty_dst")
clean_dir(dest) clean_dir(dest)
lst = _write_rel_list(b"") lst = _write_rel_list(b"")
flags = ["--files-from", lst, "--ignore-missing-args"] + (["-m"] if mt else []) flags = ["--files-from", lst, "--ignore-missing-args"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode != 0, "an empty --files-from list must stay a hard error" assert result.returncode != 0, "an empty --files-from list must stay a hard error"
assert "contains no entries" in (result.stderr or result.stdout) assert "contains no entries" in (result.stderr or result.stdout)
@@ -2201,7 +2201,7 @@ class TestMissingArgs:
server.start(extra_args=["--allow-delete"]) server.start(extra_args=["--allow-delete"])
seed = _write_rel_list(b"a.txt\nsub/b.txt\n") seed = _write_rel_list(b"a.txt\nsub/b.txt\n")
result, _ = run_client(source, dest, result, _ = run_client(source, dest,
flags=["--files-from", seed, "-R"] + (["-m"] if mt else []), flags=["--files-from", seed, "-R"] + (["--threads"] if mt else []),
port=server.port) port=server.port)
assert result.returncode == 0, f"seed -R sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"seed -R sync failed: {result.stderr[:200]}"
assert os.path.isfile(os.path.join(dest, "a.txt")) assert os.path.isfile(os.path.join(dest, "a.txt"))
@@ -2214,7 +2214,7 @@ class TestMissingArgs:
fh.write("unrelated") fh.write("unrelated")
lst = _write_rel_list(b"a.txt\ngone.txt\nsub/b.txt\n") lst = _write_rel_list(b"a.txt\ngone.txt\nsub/b.txt\n")
flags = ["--files-from", lst, "-R", "--delete-missing-args"] + (["-m"] if mt else []) flags = ["--files-from", lst, "-R", "--delete-missing-args"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode == 0, f"delete-missing sync failed: {result.stderr[:300]}" assert result.returncode == 0, f"delete-missing sync failed: {result.stderr[:300]}"
assert not os.path.exists(os.path.join(dest, "gone.txt")), \ assert not os.path.exists(os.path.join(dest, "gone.txt")), \
@@ -2227,7 +2227,7 @@ class TestMissingArgs:
# Now with --delete the unrelated extra is an ordinary extra and must go. # Now with --delete the unrelated extra is an ordinary extra and must go.
lst2 = _write_rel_list(b"a.txt\ngone.txt\nsub/b.txt\n") lst2 = _write_rel_list(b"a.txt\ngone.txt\nsub/b.txt\n")
flags2 = ["--files-from", lst2, "-R", "--delete-missing-args", "--delete"] + \ flags2 = ["--files-from", lst2, "-R", "--delete-missing-args", "--delete"] + \
(["-m"] if mt else []) (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags2, port=server.port) result, _ = run_client(source, dest, flags=flags2, port=server.port)
assert result.returncode == 0, f"delete-missing + delete sync failed: {result.stderr[:300]}" assert result.returncode == 0, f"delete-missing + delete sync failed: {result.stderr[:300]}"
assert not os.path.exists(os.path.join(dest, "unrelated.txt")), \ assert not os.path.exists(os.path.join(dest, "unrelated.txt")), \
@@ -2287,7 +2287,7 @@ class TestMissingArgs:
lst = _write_rel_list(b"a.txt\nprot/gone.txt\n") lst = _write_rel_list(b"a.txt\nprot/gone.txt\n")
flags = ["--files-from", lst, "--filter=- prot/", "--delete-missing-args", flags = ["--files-from", lst, "--filter=- prot/", "--delete-missing-args",
"--delete"] + (["-m"] if mt else []) "--delete"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode == 0, f"delete-missing exclude sync failed: {result.stderr[:300]}" assert result.returncode == 0, f"delete-missing exclude sync failed: {result.stderr[:300]}"
assert not os.path.exists(os.path.join(received, "prot", "gone.txt")), \ assert not os.path.exists(os.path.join(received, "prot", "gone.txt")), \
@@ -2314,7 +2314,7 @@ class TestMissingArgs:
server.start(extra_args=["--allow-delete"]) server.start(extra_args=["--allow-delete"])
lst = _write_rel_list(b"a.txt\ngone.txt\ngone2.txt\n") lst = _write_rel_list(b"a.txt\ngone.txt\ngone2.txt\n")
flags = ["--files-from", lst, "-R", "--delete-missing-args", "--delete-before"] + \ flags = ["--files-from", lst, "-R", "--delete-missing-args", "--delete-before"] + \
(["-m"] if mt else []) (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode == 0, f"early delete-missing sync failed: {result.stderr[:300]}" assert result.returncode == 0, f"early delete-missing sync failed: {result.stderr[:300]}"
assert not os.path.exists(os.path.join(dest, "gone.txt")), \ assert not os.path.exists(os.path.join(dest, "gone.txt")), \
@@ -2353,7 +2353,7 @@ class TestMissingArgs:
lst = _write_rel_list(b"a.txt\nsub/gone.txt\n") lst = _write_rel_list(b"a.txt\nsub/gone.txt\n")
flags = ["--files-from", lst, "--delete-missing-args", "--delete"] + rel_flags + \ flags = ["--files-from", lst, "--delete-missing-args", "--delete"] + rel_flags + \
(["-m"] if mt else []) (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"deep missing-entry sync failed: {result.stderr[:300]}" f"deep missing-entry sync failed: {result.stderr[:300]}"
@@ -2373,7 +2373,7 @@ class TestMissingArgs:
clean_dir(dest) clean_dir(dest)
lst = _write_rel_list(b"a.txt\ngone.txt\n") lst = _write_rel_list(b"a.txt\ngone.txt\n")
flags = ["--files-from", lst, "--dirs", "-R", "--ignore-missing-args"] + \ flags = ["--files-from", lst, "--dirs", "-R", "--ignore-missing-args"] + \
(["-m"] if mt else []) (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"--dirs ignore-missing sync failed: {result.stderr[:300]}" assert result.returncode == 0, f"--dirs ignore-missing sync failed: {result.stderr[:300]}"
assert _read_file(os.path.join(dest, "a.txt")) == b"a\n", \ assert _read_file(os.path.join(dest, "a.txt")) == b"a\n", \
@@ -2395,7 +2395,7 @@ class TestNoImpliedDirs:
dest = os.path.join(TEST_DATA_DIR, "noimplied_dst") dest = os.path.join(TEST_DATA_DIR, "noimplied_dst")
clean_dir(dest) clean_dir(dest)
lst = _write_rel_list(b"a/b.txt\n") # "a" itself is not listed lst = _write_rel_list(b"a/b.txt\n") # "a" itself is not listed
flags = ["--files-from", lst, "-R", "--no-implied-dirs"] + (["-m"] if mt else []) flags = ["--files-from", lst, "-R", "--no-implied-dirs"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode != 0, "implied parent directory was not rejected" assert result.returncode != 0, "implied parent directory was not rejected"
assert "--no-implied-dirs" in (result.stderr or result.stdout) assert "--no-implied-dirs" in (result.stderr or result.stdout)
@@ -2407,7 +2407,7 @@ class TestNoImpliedDirs:
dest = os.path.join(TEST_DATA_DIR, "noimplied_ok_dst") dest = os.path.join(TEST_DATA_DIR, "noimplied_ok_dst")
clean_dir(dest) clean_dir(dest)
lst = _write_rel_list(b"a\na/b.txt\n") lst = _write_rel_list(b"a\na/b.txt\n")
flags = ["--files-from", lst, "-R", "--no-implied-dirs"] + (["-m"] if mt else []) flags = ["--files-from", lst, "-R", "--no-implied-dirs"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"listed dir + file sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"listed dir + file sync failed: {result.stderr[:200]}"
assert _read_file(os.path.join(dest, "a", "b.txt")) == b"nested\n" assert _read_file(os.path.join(dest, "a", "b.txt")) == b"nested\n"
@@ -2418,7 +2418,7 @@ class TestNoImpliedDirs:
dest = os.path.join(TEST_DATA_DIR, "noimplied_noR_dst") dest = os.path.join(TEST_DATA_DIR, "noimplied_noR_dst")
clean_dir(dest) clean_dir(dest)
lst = _write_rel_list(b"a/b.txt\n") lst = _write_rel_list(b"a/b.txt\n")
flags = ["--files-from", lst, "--no-implied-dirs"] + (["-m"] if mt else []) flags = ["--files-from", lst, "--no-implied-dirs"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, "--no-implied-dirs without -R changed behavior" assert result.returncode == 0, "--no-implied-dirs without -R changed behavior"
received = get_dest_received_dir(dest, source) received = get_dest_received_dir(dest, source)
@@ -2446,7 +2446,7 @@ class TestDirs:
source = self._make() source = self._make()
dest = os.path.join(TEST_DATA_DIR, "dirs_dst") dest = os.path.join(TEST_DATA_DIR, "dirs_dst")
clean_dir(dest) clean_dir(dest)
flags = [flag] + (["-m"] if mt else []) flags = [flag] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"{flag} sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"{flag} sync failed: {result.stderr[:200]}"
self._assert_only_empty_mirror(dest, source) self._assert_only_empty_mirror(dest, source)
@@ -2458,7 +2458,7 @@ class TestDirs:
clean_dir(dest) clean_dir(dest)
# A listed directory is created empty; a listed file is transferred. # A listed directory is created empty; a listed file is transferred.
lst = _write_rel_list(b"dir1\nsub/x.txt\n") lst = _write_rel_list(b"dir1\nsub/x.txt\n")
flags = ["--files-from", lst, "--dirs", "-R"] + (["-m"] if mt else []) flags = ["--files-from", lst, "--dirs", "-R"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"dirs files-from sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"dirs files-from sync failed: {result.stderr[:200]}"
assert os.path.isdir(os.path.join(dest, "dir1")), "listed dir was not created" assert os.path.isdir(os.path.join(dest, "dir1")), "listed dir was not created"
@@ -2476,7 +2476,7 @@ class TestDirs:
dest = os.path.join(TEST_DATA_DIR, "dirs_ff_noR_dst") dest = os.path.join(TEST_DATA_DIR, "dirs_ff_noR_dst")
clean_dir(dest) clean_dir(dest)
lst = _write_rel_list(b"dir1\n") lst = _write_rel_list(b"dir1\n")
flags = ["--files-from", lst, "--dirs"] + (["-m"] if mt else []) flags = ["--files-from", lst, "--dirs"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"dirs files-from no-R sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"dirs files-from no-R sync failed: {result.stderr[:200]}"
received = get_dest_received_dir(dest, source) received = get_dest_received_dir(dest, source)
@@ -2495,7 +2495,7 @@ class TestDirs:
dest = os.path.join(TEST_DATA_DIR, "dirs_s_dst") dest = os.path.join(TEST_DATA_DIR, "dirs_s_dst")
clean_dir(dest) clean_dir(dest)
lst = _write_rel_list(b"dir1\nsub/x.txt\n") lst = _write_rel_list(b"dir1\nsub/x.txt\n")
flags = ["--files-from", lst, "--dirs", "-R", "-s"] + (["-m"] if mt else []) flags = ["--files-from", lst, "--dirs", "-R", "--chunk-serialization"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"dirs -s sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"dirs -s sync failed: {result.stderr[:200]}"
assert os.path.isdir(os.path.join(dest, "dir1")), "listed dir was not created" assert os.path.isdir(os.path.join(dest, "dir1")), "listed dir was not created"
@@ -2554,7 +2554,7 @@ class TestMkpath:
shutil.rmtree(dest, ignore_errors=True) shutil.rmtree(dest, ignore_errors=True)
with ServerManager() as server: with ServerManager() as server:
server.start() server.start()
flags = ["-m"] if mt else [] flags = ["--threads"] if mt else []
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode != 0, "missing destination root did not fail without --mkpath" assert result.returncode != 0, "missing destination root did not fail without --mkpath"
assert not os.path.exists(dest), "missing root was created without --mkpath" assert not os.path.exists(dest), "missing root was created without --mkpath"
@@ -2566,7 +2566,7 @@ class TestMkpath:
shutil.rmtree(os.path.join(TEST_DATA_DIR, "deep"), ignore_errors=True) shutil.rmtree(os.path.join(TEST_DATA_DIR, "deep"), ignore_errors=True)
with ServerManager() as server: with ServerManager() as server:
server.start() server.start()
flags = ["--mkpath"] + (["-m"] if mt else []) flags = ["--mkpath"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode == 0, f"--mkpath sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"--mkpath sync failed: {result.stderr[:200]}"
received = get_dest_received_dir(dest, source) received = get_dest_received_dir(dest, source)
@@ -2656,7 +2656,7 @@ class TestDeleteTiming:
with open(extra, "wb") as fh: with open(extra, "wb") as fh:
fh.write(b"should be deleted") fh.write(b"should be deleted")
flags = [flag] + (["-m"] if mt else []) flags = [flag] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"{flag} sync failed: {(result.stderr or result.stdout)[:300]}" f"{flag} sync failed: {(result.stderr or result.stdout)[:300]}"
@@ -2687,7 +2687,7 @@ class TestDeleteTiming:
with open(blocker, "wb") as fh: with open(blocker, "wb") as fh:
fh.write(b"blocks the nested destination directory") fh.write(b"blocks the nested destination directory")
flags = [flag] + (["-m"] if mt else []) flags = [flag] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"{flag} (early delete) did not remove the blocker in time: " \ f"{flag} (early delete) did not remove the blocker in time: " \
@@ -2721,7 +2721,7 @@ class TestDeleteTiming:
with open(blocker, "wb") as fh: with open(blocker, "wb") as fh:
fh.write(b"blocks the nested destination directory") fh.write(b"blocks the nested destination directory")
flags = [flag] + (["-m"] if mt else []) flags = [flag] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode != 0, \ assert result.returncode != 0, \
f"{flag} (mt={mt}) unexpectedly succeeded (deletion must be deferred)" f"{flag} (mt={mt}) unexpectedly succeeded (deletion must be deferred)"
@@ -2804,7 +2804,7 @@ class TestDeletePolicy:
self._write(os.path.join(received, "extra.txt"), b"extra\n") self._write(os.path.join(received, "extra.txt"), b"extra\n")
# Default: the excluded mirrors survive --delete, genuine extras die. # Default: the excluded mirrors survive --delete, genuine extras die.
flags = ["--exclude", "*.log", timing] + (["-m"] if mt else []) flags = ["--exclude", "*.log", timing] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"default delete sync failed: {(result.stderr or result.stdout)[:300]}" f"default delete sync failed: {(result.stderr or result.stdout)[:300]}"
@@ -2817,7 +2817,7 @@ class TestDeletePolicy:
# --delete-excluded: excluded mirrors are extras again and die. # --delete-excluded: excluded mirrors are extras again and die.
self._write(os.path.join(received, "extra.txt"), b"extra\n") self._write(os.path.join(received, "extra.txt"), b"extra\n")
flags = ["--exclude", "*.log", timing, "--delete-excluded"] + (["-m"] if mt else []) flags = ["--exclude", "*.log", timing, "--delete-excluded"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"--delete-excluded sync failed: {(result.stderr or result.stdout)[:300]}" f"--delete-excluded sync failed: {(result.stderr or result.stdout)[:300]}"
@@ -2846,7 +2846,7 @@ class TestDeletePolicy:
assert result.returncode == 0, f"seed sync failed: {result.stderr[:200]}" assert result.returncode == 0, f"seed sync failed: {result.stderr[:200]}"
received = get_dest_received_dir(dest, source) received = get_dest_received_dir(dest, source)
flags = ["--filter=- skipdir/", "--delete"] + (["-m"] if mt else []) flags = ["--filter=- skipdir/", "--delete"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"default delete sync failed: {(result.stderr or result.stdout)[:300]}" f"default delete sync failed: {(result.stderr or result.stdout)[:300]}"
@@ -2855,7 +2855,7 @@ class TestDeletePolicy:
assert os.path.exists(os.path.join(received, "skipdir", "deep", "b.log")), \ assert os.path.exists(os.path.join(received, "skipdir", "deep", "b.log")), \
"nested excluded dir content was deleted under plain --delete" "nested excluded dir content was deleted under plain --delete"
flags = ["--filter=- skipdir/", "--delete", "--delete-excluded"] + (["-m"] if mt else []) flags = ["--filter=- skipdir/", "--delete", "--delete-excluded"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"--delete-excluded sync failed: {(result.stderr or result.stdout)[:300]}" f"--delete-excluded sync failed: {(result.stderr or result.stdout)[:300]}"
@@ -2882,7 +2882,7 @@ class TestDeletePolicy:
self._write(os.path.join(received, name), b"extra\n") self._write(os.path.join(received, name), b"extra\n")
extras.append(os.path.join(received, name)) extras.append(os.path.join(received, name))
flags = ["--max-delete=2", timing] + (["-m"] if mt else []) flags = ["--max-delete=2", timing] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode != 0, \ assert result.returncode != 0, \
f"--max-delete=2 with 4 extras unexpectedly succeeded: {result.stderr[:300]}" f"--max-delete=2 with 4 extras unexpectedly succeeded: {result.stderr[:300]}"
@@ -2906,7 +2906,7 @@ class TestDeletePolicy:
received = get_dest_received_dir(dest, source) received = get_dest_received_dir(dest, source)
for i in range(3): for i in range(3):
self._write(os.path.join(received, f"e{i}.txt"), b"extra\n") self._write(os.path.join(received, f"e{i}.txt"), b"extra\n")
flags = ["--max-delete=3", "--delete"] + (["-m"] if mt else []) flags = ["--max-delete=3", "--delete"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"--max-delete=3 with 3 extras failed: {(result.stderr or result.stdout)[:300]}" f"--max-delete=3 with 3 extras failed: {(result.stderr or result.stdout)[:300]}"
@@ -2943,7 +2943,7 @@ class TestDeletePolicy:
assert os.path.exists(os.path.join(received, "sub", "old.txt")), \ assert os.path.exists(os.path.join(received, "sub", "old.txt")), \
"non-empty dir content was lost although the run failed without --force" "non-empty dir content was lost although the run failed without --force"
flags = ["--force"] + (["-m"] if mt else []) flags = ["--force"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"--force run failed: {(result.stderr or result.stdout)[:300]}" f"--force run failed: {(result.stderr or result.stdout)[:300]}"
@@ -3000,7 +3000,7 @@ class TestDeletePolicy:
assert os.listdir(received) == [] assert os.listdir(received) == []
# prune-empty-dirs: the empty mirror is pruned by --delete. # prune-empty-dirs: the empty mirror is pruned by --delete.
flags = ["--dirs", "--prune-empty-dirs", "--delete"] + (["-m"] if mt else []) flags = ["--dirs", "--prune-empty-dirs", "--delete"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"--dirs --prune-empty-dirs --delete failed: {(result.stderr or result.stdout)[:300]}" f"--dirs --prune-empty-dirs --delete failed: {(result.stderr or result.stdout)[:300]}"
@@ -3012,7 +3012,7 @@ class TestDeletePolicy:
clean_dir(dest2) clean_dir(dest2)
with ServerManager() as server: with ServerManager() as server:
server.start(extra_args=["--allow-delete"]) server.start(extra_args=["--allow-delete"])
flags = ["--dirs", "--prune-empty-dirs", "-i"] + (["-m"] if mt else []) flags = ["--dirs", "--prune-empty-dirs", "-i"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest2, flags=flags, port=server.port) result, _ = run_client(source, dest2, flags=flags, port=server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"--dirs --prune-empty-dirs failed: {(result.stderr or result.stdout)[:300]}" f"--dirs --prune-empty-dirs failed: {(result.stderr or result.stdout)[:300]}"
@@ -3047,7 +3047,7 @@ class TestDeletePolicy:
os.makedirs(os.path.join(received, "empty", "chain")) os.makedirs(os.path.join(received, "empty", "chain"))
for prune in ([], ["--prune-empty-dirs"]): for prune in ([], ["--prune-empty-dirs"]):
flags = prune + ["--delete"] + (["-m"] if mt else []) flags = prune + ["--delete"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"prune recursive sync failed: {(result.stderr or result.stdout)[:300]}" f"prune recursive sync failed: {(result.stderr or result.stdout)[:300]}"
@@ -3058,7 +3058,7 @@ class TestDeletePolicy:
assert _read_file(os.path.join(received, "keep.txt")) == b"kept\n" assert _read_file(os.path.join(received, "keep.txt")) == b"kept\n"
# An excluded file's mirror is protected: the dir that holds it stays. # An excluded file's mirror is protected: the dir that holds it stays.
flags = ["--exclude", "*.log", "--delete", "--prune-empty-dirs"] + (["-m"] if mt else []) flags = ["--exclude", "*.log", "--delete", "--prune-empty-dirs"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=server.port) result, _ = run_client(source, dest, flags=flags, port=server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"prune recursive sync failed: {(result.stderr or result.stdout)[:300]}" f"prune recursive sync failed: {(result.stderr or result.stdout)[:300]}"
@@ -3100,7 +3100,7 @@ class TestDeletePolicy:
# Default: scan error aborts the run; nothing is deleted. # Default: scan error aborts the run; nothing is deleted.
self._write(os.path.join(received, "extra.txt"), b"extra\n") self._write(os.path.join(received, "extra.txt"), b"extra\n")
flags = ["--delete"] + (["-m"] if mt else []) flags = ["--delete"] + (["--threads"] if mt else [])
result = self._run_client_as_nobody(source, dest, server.port, flags) result = self._run_client_as_nobody(source, dest, server.port, flags)
assert result.returncode != 0, "unreadable source dir did not fail the run" assert result.returncode != 0, "unreadable source dir did not fail the run"
assert os.path.exists(os.path.join(received, "extra.txt")), \ assert os.path.exists(os.path.join(received, "extra.txt")), \
@@ -3108,7 +3108,7 @@ class TestDeletePolicy:
# --ignore-errors: the readable tree transfers, deletion still runs. # --ignore-errors: the readable tree transfers, deletion still runs.
self._write(os.path.join(received, "extra.txt"), b"extra\n") self._write(os.path.join(received, "extra.txt"), b"extra\n")
flags = ["--delete", "--ignore-errors"] + (["-m"] if mt else []) flags = ["--delete", "--ignore-errors"] + (["--threads"] if mt else [])
result = self._run_client_as_nobody(source, dest, server.port, flags) result = self._run_client_as_nobody(source, dest, server.port, flags)
assert not os.path.exists(os.path.join(received, "extra.txt")), \ assert not os.path.exists(os.path.join(received, "extra.txt")), \
f"--ignore-errors did not keep deletion active: {result.stderr[:300]}" f"--ignore-errors did not keep deletion active: {result.stderr[:300]}"
@@ -3143,7 +3143,7 @@ class TestDeletePolicy:
try: try:
os.chmod(source, 0) os.chmod(source, 0)
self._write(os.path.join(received, "extra.txt"), b"extra\n") self._write(os.path.join(received, "extra.txt"), b"extra\n")
flags = [timing, "--ignore-errors"] + (["-m"] if mt else []) flags = [timing, "--ignore-errors"] + (["--threads"] if mt else [])
result = self._run_client_as_nobody(source, dest, server.port, flags) result = self._run_client_as_nobody(source, dest, server.port, flags)
assert result.returncode != 0, \ assert result.returncode != 0, \
f"unreadable source root with {timing} (mt={mt}) unexpectedly succeeded" f"unreadable source root with {timing} (mt={mt}) unexpectedly succeeded"
@@ -3371,7 +3371,7 @@ class TestBasisDestDirs:
dest = os.path.join(TEST_DATA_DIR, "basis_link_mt_dst") dest = os.path.join(TEST_DATA_DIR, "basis_link_mt_dst")
clean_dir(dest) clean_dir(dest)
basis = self._seed_basis(dest, source, "mtbasis", self._basis_tree("mt")) basis = self._seed_basis(dest, source, "mtbasis", self._basis_tree("mt"))
result, _ = run_client(source, dest, flags=["--link-dest=mtbasis", "-m"], result, _ = run_client(source, dest, flags=["--link-dest=mtbasis", "--threads"],
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, f"-m link-dest failed: {result.stderr[:300]}" assert result.returncode == 0, f"-m link-dest failed: {result.stderr[:300]}"
received = get_dest_received_dir(dest, source) received = get_dest_received_dir(dest, source)
@@ -3665,7 +3665,7 @@ class TestFuzzy:
with open(os.path.join(source, self.NEW_NAME), "wb") as fh: with open(os.path.join(source, self.NEW_NAME), "wb") as fh:
fh.write(new_bytes) fh.write(new_bytes)
flags = ["--fuzzy"] + (["-m"] if mt else []) flags = ["--fuzzy"] + (["--threads"] if mt else [])
result, proxy = self._run_measured(source, dest, flags, shared_server.port) result, proxy = self._run_measured(source, dest, flags, shared_server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"--fuzzy {'-m ' if mt else ''}rename failed: {(result.stderr or result.stdout)[:300]}" f"--fuzzy {'-m ' if mt else ''}rename failed: {(result.stderr or result.stdout)[:300]}"
@@ -3926,7 +3926,7 @@ class TestIdentityMapping:
with open(os.path.join(source, "f.txt"), "wb") as f: with open(os.path.join(source, "f.txt"), "wb") as f:
f.write(b"hello identity") f.write(b"hello identity")
result, _ = run_client(source, dest, result, _ = run_client(source, dest,
flags=["-M", "--numeric-ids"], flags=["--preserve", "--numeric-ids"],
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"exit {result.returncode}: {(result.stderr or '')[:200]}" f"exit {result.returncode}: {(result.stderr or '')[:200]}"
@@ -3943,7 +3943,7 @@ class TestIdentityMapping:
f.write(b"mapped") f.write(b"mapped")
result, _ = run_client( result, _ = run_client(
source, dest, source, dest,
flags=["-M", "--usermap=@1000:@1001", "--groupmap=@100:@101", "--chown=@2000:@2001"], flags=["--preserve", "--usermap=@1000:@1001", "--groupmap=@100:@101", "--chown=@2000:@2001"],
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"exit {result.returncode}: {(result.stderr or '')[:200]}" f"exit {result.returncode}: {(result.stderr or '')[:200]}"
@@ -3962,7 +3962,7 @@ class TestIdentityMapping:
f.write(b"owner") f.write(b"owner")
os.chown(src_file, 12345, 12346) os.chown(src_file, 12345, 12346)
result, _ = run_client(source, dest, result, _ = run_client(source, dest,
flags=["-M", "--numeric-ids"], flags=["--preserve", "--numeric-ids"],
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"exit {result.returncode}: {(result.stderr or '')[:200]}" f"exit {result.returncode}: {(result.stderr or '')[:200]}"
@@ -3984,7 +3984,7 @@ class TestIdentityMapping:
f.write(b"root chown") f.write(b"root chown")
os.chown(src_file, 1, 1) os.chown(src_file, 1, 1)
result, _ = run_client(source, dest, result, _ = run_client(source, dest,
flags=["-M", "--chown=@12345:@54321"], flags=["--preserve", "--chown=@12345:@54321"],
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"exit {result.returncode}: {(result.stderr or '')[:200]}" f"exit {result.returncode}: {(result.stderr or '')[:200]}"
@@ -4013,7 +4013,7 @@ class TestHardLinks:
fh.write(b"independent content\n" * 2000) fh.write(b"independent content\n" * 2000)
return src return src
@pytest.mark.parametrize("flags", [[], ["-m"], ["--delay-updates"]]) @pytest.mark.parametrize("flags", [[], ["--threads"], ["--delay-updates"]])
def test_hard_links_preserved(self, shared_server, flags): def test_hard_links_preserved(self, shared_server, flags):
src = self._make_source("hl_src") src = self._make_source("hl_src")
dest = os.path.join(TEST_DATA_DIR, "hl_dst") dest = os.path.join(TEST_DATA_DIR, "hl_dst")
@@ -4041,7 +4041,7 @@ class TestHardLinks:
src = self._make_source("hl_reject_src") src = self._make_source("hl_reject_src")
dest = os.path.join(TEST_DATA_DIR, "hl_reject_dst") dest = os.path.join(TEST_DATA_DIR, "hl_reject_dst")
clean_dir(dest) clean_dir(dest)
result, _ = run_client(src, dest, flags=["-H", "-s"], port=shared_server.port) result, _ = run_client(src, dest, flags=["-H", "--chunk-serialization"], port=shared_server.port)
assert result.returncode != 0, "-H with -s was accepted" assert result.returncode != 0, "-H with -s was accepted"
def test_hard_links_rejects_append(self, shared_server): def test_hard_links_rejects_append(self, shared_server):
@@ -4136,7 +4136,7 @@ class TestAtimes:
source = os.path.join(TEST_DATA_DIR, f"atime_{'m' if mt else 's'}_src") source = os.path.join(TEST_DATA_DIR, f"atime_{'m' if mt else 's'}_src")
dest = os.path.join(TEST_DATA_DIR, f"atime_{'m' if mt else 's'}_dst") dest = os.path.join(TEST_DATA_DIR, f"atime_{'m' if mt else 's'}_dst")
src_file, atime = self._make_source(source, dest) src_file, atime = self._make_source(source, dest)
flags = ["-U"] + (["-m"] if mt else []) flags = ["-U"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"-U failed: {(result.stderr or result.stdout)[:300]}" f"-U failed: {(result.stderr or result.stdout)[:300]}"
@@ -4158,7 +4158,7 @@ class TestAtimes:
dest = os.path.join(TEST_DATA_DIR, f"atime_ctrl_{'m' if mt else 's'}_dst") dest = os.path.join(TEST_DATA_DIR, f"atime_ctrl_{'m' if mt else 's'}_dst")
src_file, atime = self._make_source(source, dest) src_file, atime = self._make_source(source, dest)
now = time.time() now = time.time()
flags = (["-m"] if mt else []) flags = (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"control run failed: {(result.stderr or '')[:200]}" assert result.returncode == 0, f"control run failed: {(result.stderr or '')[:200]}"
received = get_dest_received_dir(dest, source) received = get_dest_received_dir(dest, source)
@@ -4191,7 +4191,7 @@ class TestOpenNoatime:
atime = 730486800 # 1993-02-11, distinct and far from now atime = 730486800 # 1993-02-11, distinct and far from now
os.utime(path, ns=(atime * 10**9, atime * 10**9)) os.utime(path, ns=(atime * 10**9, atime * 10**9))
flags = ["--open-noatime"] + (["-m"] if mt else []) flags = ["--open-noatime"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"--open-noatime failed: {(result.stderr or result.stdout)[:300]}" f"--open-noatime failed: {(result.stderr or result.stdout)[:300]}"
@@ -4220,7 +4220,7 @@ class TestCrtimes:
with open(path, "wb") as f: with open(path, "wb") as f:
f.write(payload) f.write(payload)
flags = ["-N"] + (["-m"] if mt else []) flags = ["-N"] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"-N failed: {(result.stderr or result.stdout)[:300]}" f"-N failed: {(result.stderr or result.stdout)[:300]}"
@@ -4267,7 +4267,7 @@ class TestOmitTimes:
clean_dir(dest) clean_dir(dest)
with open(os.path.join(source, "a.txt"), "wb") as f: with open(os.path.join(source, "a.txt"), "wb") as f:
f.write(b"omit times content\n") f.write(b"omit times content\n")
flags = [flag] + (["-m"] if mt else []) flags = [flag] + (["--threads"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port) result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"{flag} failed: {(result.stderr or result.stdout)[:300]}" f"{flag} failed: {(result.stderr or result.stdout)[:300]}"
@@ -4289,7 +4289,7 @@ class TestOmitTimes:
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"-d -O failed: {(result.stderr or result.stdout)[:300]}" f"-d -O failed: {(result.stderr or result.stdout)[:300]}"
result, _ = run_client(source, dest, flags=["-M", "-O", "-J"], result, _ = run_client(source, dest, flags=["--preserve", "-O", "-J"],
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"-M -O -J failed: {(result.stderr or result.stdout)[:300]}" f"-M -O -J failed: {(result.stderr or result.stdout)[:300]}"
@@ -4536,7 +4536,7 @@ class TestExtendedAttributes:
if not _xattr_supported(f): if not _xattr_supported(f):
pytest.skip("filesystem does not support user xattrs") pytest.skip("filesystem does not support user xattrs")
os.setxattr(f, "user.k", b"v") os.setxattr(f, "user.k", b"v")
result, _ = run_client(source, dest, flags=["-X", "-m"], port=shared_server.port) result, _ = run_client(source, dest, flags=["-X", "--threads"], port=shared_server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"-X -m sync failed: {(result.stderr or result.stdout)[:300]}" f"-X -m sync failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source) received = get_dest_received_dir(dest, source)
@@ -4552,7 +4552,7 @@ class TestExtendedAttributes:
pytest.skip("filesystem does not support xattrs") pytest.skip("filesystem does not support xattrs")
acl_blob = None acl_blob = None
if shutil.which("setfacl") is not None: if shutil.which("setfacl") is not None:
acl = subprocess.run(["setfacl", "-m", "o::r", f], capture_output=True, text=True) acl = subprocess.run(["setfacl", "--threads", "o::r", f], capture_output=True, text=True)
if acl.returncode == 0: if acl.returncode == 0:
try: try:
acl_blob = os.getxattr(f, "system.posix_acl_access") acl_blob = os.getxattr(f, "system.posix_acl_access")
@@ -4662,7 +4662,7 @@ class TestConnectivityClientOptions:
source, dest = self._source_and_dest("connopt_zlib") source, dest = self._source_and_dest("connopt_zlib")
with open(os.path.join(source, "text.txt"), "wb") as f: with open(os.path.join(source, "text.txt"), "wb") as f:
f.write(b"compress me\n" * 4096) f.write(b"compress me\n" * 4096)
result, _ = run_client(source, dest, flags=["--blocking-io", "-c"], result, _ = run_client(source, dest, flags=["--blocking-io", "-z"],
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"--blocking-io -c failed: {(result.stderr or result.stdout)[:300]}" f"--blocking-io -c failed: {(result.stderr or result.stdout)[:300]}"
tests/integration/test_iconv.py
+1 -1
View File
@@ -243,7 +243,7 @@ def test_iconv_chunk_serialization_blob(shared_server):
_place_bytes(source, name, data=b"blob\n") _place_bytes(source, name, data=b"blob\n")
result, _ = run_client( result, _ = run_client(
source, dest, flags=["--iconv=iso-8859-1,utf-8", "-s"], port=shared_server.port source, dest, flags=["--iconv=iso-8859-1,utf-8", "--chunk-serialization"], port=shared_server.port
) )
assert result.returncode == 0, (result.stderr or result.stdout)[:400] assert result.returncode == 0, (result.stderr or result.stdout)[:400]
tests/integration/test_ssh.py
+8 -8
View File
@@ -97,31 +97,31 @@ class TestSSHStandard:
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
def test_multithreading(self): def test_multithreading(self):
r = _run_ssh_test("SSH Multithreading (-m)", ["-m"]) r = _run_ssh_test("SSH Multithreading (-m)", ["--threads"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
def test_compression(self): def test_compression(self):
r = _run_ssh_test("SSH Compression (-c)", ["-c"]) r = _run_ssh_test("SSH Compression (-c)", ["-z"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
def test_chunk_serialization(self): def test_chunk_serialization(self):
r = _run_ssh_test("SSH Chunk Serialization (-s)", ["-s"]) r = _run_ssh_test("SSH Chunk Serialization (-s)", ["--chunk-serialization"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
def test_compression_chunk(self): def test_compression_chunk(self):
r = _run_ssh_test("SSH Compression + Chunk (-c -s)", ["-c", "-s"]) r = _run_ssh_test("SSH Compression + Chunk (-c -s)", ["-z", "--chunk-serialization"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
def test_multithread_compression(self): def test_multithread_compression(self):
r = _run_ssh_test("SSH Multithread + Compression (-m -c)", ["-m", "-c"]) r = _run_ssh_test("SSH Multithread + Compression (-m -c)", ["--threads", "-z"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
def test_multithread_chunk(self): def test_multithread_chunk(self):
r = _run_ssh_test("SSH Multithread + Chunk (-m -s)", ["-m", "-s"]) r = _run_ssh_test("SSH Multithread + Chunk (-m -s)", ["--threads", "--chunk-serialization"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
def test_all_flags(self): def test_all_flags(self):
r = _run_ssh_test("SSH All Flags (-m -c -s)", ["-m", "-c", "-s"]) r = _run_ssh_test("SSH All Flags (-m -c -s)", ["--threads", "-z", "--chunk-serialization"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
@@ -178,7 +178,7 @@ class TestSSHConnectivity:
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
def test_blocking_io_with_compression(self): def test_blocking_io_with_compression(self):
r = _run_ssh_test("SSH --blocking-io -c", ["--blocking-io", "-c"]) r = _run_ssh_test("SSH --blocking-io -c", ["--blocking-io", "-z"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
def test_trust_sender(self): def test_trust_sender(self):
tests/integration/test_stop.py
+1 -1
View File
@@ -231,7 +231,7 @@ class TestStopDelete:
"""-m immediate stop + --delete: the completion tail must not read the """-m immediate stop + --delete: the completion tail must not read the
still-appendable manifest (no race) and must not delete the mirrors.""" still-appendable manifest (no race) and must not delete the mirrors."""
source, dest = self._seed("del_mt", shared_server.port, many=True) source, dest = self._seed("del_mt", shared_server.port, many=True)
result, _ = run_client(source, dest, flags=["-m", "--delete", "--stop-at=now+0s"], result, _ = run_client(source, dest, flags=["--threads", "--delete", "--stop-at=now+0s"],
port=shared_server.port) port=shared_server.port)
assert result.returncode == 0, \ assert result.returncode == 0, \
f"-m --delete immediate stop failed: {(result.stderr or result.stdout)[:400]}" f"-m --delete immediate stop failed: {(result.stderr or result.stdout)[:400]}"
tests/integration/test_tcp.py
+12 -12
View File
@@ -30,11 +30,11 @@ def _run_tcp_test(name, port, flags, use_metadata=True, posix=False):
clean_dir(DEST_DIR) clean_dir(DEST_DIR)
if posix: if posix:
result, dur = run_client_posix(SOURCE_DIR, DEST_DIR, result, dur = run_client_posix(SOURCE_DIR, DEST_DIR,
flags=(["-M"] if use_metadata else []) + flags, flags=(["--preserve"] if use_metadata else []) + flags,
port=port) port=port)
else: else:
result, dur = run_client(SOURCE_DIR, DEST_DIR, result, dur = run_client(SOURCE_DIR, DEST_DIR,
flags=(["-M"] if use_metadata else []) + flags, flags=(["--preserve"] if use_metadata else []) + flags,
port=port) port=port)
if result.returncode != 0: if result.returncode != 0:
@@ -68,45 +68,45 @@ class TestTCPStandard:
class TestTCPFlags: class TestTCPFlags:
@pytest.mark.ci @pytest.mark.ci
def test_multithreading(self, shared_server): def test_multithreading(self, shared_server):
r = _run_tcp_test("Multithreading (-m)", shared_server.port, ["-m"]) r = _run_tcp_test("Multithreading (-m)", shared_server.port, ["--threads"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
@pytest.mark.ci @pytest.mark.ci
def test_compression(self, shared_server): def test_compression(self, shared_server):
r = _run_tcp_test("Compression (-c)", shared_server.port, ["-c"]) r = _run_tcp_test("Compression (-c)", shared_server.port, ["-z"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
def test_compression_threads(self, shared_server): def test_compression_threads(self, shared_server):
r = _run_tcp_test("Compression threads (-c --compress-threads=2)", shared_server.port, r = _run_tcp_test("Compression threads (-c --compress-threads=2)", shared_server.port,
["-c", "--compress-threads=2"]) ["-z", "--compress-threads=2"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
def test_chunk_serialization(self, shared_server): def test_chunk_serialization(self, shared_server):
r = _run_tcp_test("Chunk Serialization (-s)", shared_server.port, ["-s"]) r = _run_tcp_test("Chunk Serialization (-s)", shared_server.port, ["--chunk-serialization"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
def test_compression_chunk(self, shared_server): def test_compression_chunk(self, shared_server):
r = _run_tcp_test("Compression + Chunk (-c -s)", shared_server.port, ["-c", "-s"]) r = _run_tcp_test("Compression + Chunk (-c -s)", shared_server.port, ["-z", "--chunk-serialization"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
def test_multithread_compression(self, shared_server): def test_multithread_compression(self, shared_server):
r = _run_tcp_test("Multithreading + Compression (-m -c)", shared_server.port, ["-m", "-c"]) r = _run_tcp_test("Multithreading + Compression (-m -c)", shared_server.port, ["--threads", "-z"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
def test_multithread_chunk(self, shared_server): def test_multithread_chunk(self, shared_server):
r = _run_tcp_test("Multithreading + Chunk (-m -s)", shared_server.port, ["-m", "-s"]) r = _run_tcp_test("Multithreading + Chunk (-m -s)", shared_server.port, ["--threads", "--chunk-serialization"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
def test_all_flags(self, shared_server): def test_all_flags(self, shared_server):
r = _run_tcp_test("Multithread + Compression + Chunk (-m -c -s)", shared_server.port, ["-m", "-c", "-s"]) r = _run_tcp_test("Multithread + Compression + Chunk (-m -c -s)", shared_server.port, ["--threads", "-z", "--chunk-serialization"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
def test_sendfile(self, shared_server): def test_sendfile(self, shared_server):
r = _run_tcp_test("Sendfile (-f)", shared_server.port, ["-f"]) r = _run_tcp_test("Sendfile (-f)", shared_server.port, ["--sendfile"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
def test_sendfile_multithread(self, shared_server): def test_sendfile_multithread(self, shared_server):
r = _run_tcp_test("Sendfile + Multithreading (-f -m)", shared_server.port, ["-f", "-m"]) r = _run_tcp_test("Sendfile + Multithreading (-f -m)", shared_server.port, ["--sendfile", "--threads"])
assert r["status"] == "Success", r["error"] assert r["status"] == "Success", r["error"]
tests/integration/test_tls.py
+2 -2
View File
@@ -132,7 +132,7 @@ class TestTLSBasic:
]) ])
result, dur = run_client( result, dur = run_client(
SOURCE_DIR, DEST_DIR, SOURCE_DIR, DEST_DIR,
flags=["-c", "--tls", flags=["-z", "--tls",
"--cert", certs["client_cert"], "--key", certs["client_key"], "--cert", certs["client_cert"], "--key", certs["client_key"],
"--ca", certs["ca"]], "--ca", certs["ca"]],
port=server.port, port=server.port,
@@ -157,7 +157,7 @@ class TestTLSBasic:
]) ])
result, dur = run_client( result, dur = run_client(
SOURCE_DIR, DEST_DIR, SOURCE_DIR, DEST_DIR,
flags=["-m", "--tls", flags=["--threads", "--tls",
"--cert", certs["client_cert"], "--key", certs["client_key"], "--cert", certs["client_cert"], "--key", certs["client_key"],
"--ca", certs["ca"]], "--ca", certs["ca"]],
port=server.port, port=server.port,
tests/test_client_cli.c
+68 -40
View File
@@ -103,19 +103,24 @@ static void test_cli_help() {
config_delete(cfg); config_delete(cfg);
} }
/* Test that --archive sets compression, multithreading, and metadata */ /* Test that --archive's config bundle matches rsync -rlptgoD semantics:
* links + metadata + devices + specials, and NOT compression/multithreading. */
static void test_cli_archive_flags() { static void test_cli_archive_flags() {
Config* cfg = config_create(); Config* cfg = config_create();
EXPECT_NOT_NULL(cfg); EXPECT_NOT_NULL(cfg);
/* Simulate --archive flag */ /* Simulate the --archive flag's implied bundle. */
cfg->use_compression = true; cfg->follow_symlinks = true;
cfg->use_multithreading = true;
cfg->use_metadata = true; cfg->use_metadata = true;
cfg->preserve_devices = true;
cfg->preserve_specials = true;
EXPECT_TRUE(cfg->use_compression); EXPECT_TRUE(cfg->follow_symlinks);
EXPECT_TRUE(cfg->use_multithreading);
EXPECT_TRUE(cfg->use_metadata); EXPECT_TRUE(cfg->use_metadata);
EXPECT_TRUE(cfg->preserve_devices);
EXPECT_TRUE(cfg->preserve_specials);
EXPECT_FALSE(cfg->use_compression);
EXPECT_FALSE(cfg->use_multithreading);
config_delete(cfg); config_delete(cfg);
} }
@@ -325,10 +330,10 @@ static void test_parse_args_fake_super() {
config_delete(cfg); config_delete(cfg);
} }
/* Test parse_args with valid port */ /* Test parse_args with valid SSH port (long form; -p is now rsync --perms) */
static void test_parse_args_valid_port() { static void test_parse_args_valid_port() {
Config* cfg = config_create(); Config* cfg = config_create();
char* argv[] = {"fastsync", "-p", "2222", "/src", "/dst"}; char* argv[] = {"fastsync", "--ssh-port", "2222", "/src", "/dst"};
int positional_args[2]; int positional_args[2];
int positional_count = 0; int positional_count = 0;
@@ -417,7 +422,7 @@ static void test_parse_args_rejects_invalid_chmod() {
/* Test parse_args rejects port > 65535 */ /* Test parse_args rejects port > 65535 */
static void test_parse_args_invalid_port() { static void test_parse_args_invalid_port() {
Config* cfg = config_create(); Config* cfg = config_create();
char* argv[] = {"fastsync", "-p", "99999", "/src", "/dst"}; char* argv[] = {"fastsync", "--ssh-port", "99999", "/src", "/dst"};
int positional_args[2]; int positional_args[2];
int positional_count = 0; int positional_count = 0;
@@ -430,7 +435,7 @@ static void test_parse_args_invalid_port() {
/* Test parse_args rejects non-numeric port */ /* Test parse_args rejects non-numeric port */
static void test_parse_args_non_numeric_port() { static void test_parse_args_non_numeric_port() {
Config* cfg = config_create(); Config* cfg = config_create();
char* argv[] = {"fastsync", "-p", "abc", "/src", "/dst"}; char* argv[] = {"fastsync", "--ssh-port", "abc", "/src", "/dst"};
int positional_args[2]; int positional_args[2];
int positional_count = 0; int positional_count = 0;
@@ -453,10 +458,10 @@ static void test_parse_args_invalid_server_port() {
config_delete(cfg); config_delete(cfg);
} }
/* Test parse_args rejects invalid compression level */ /* Test parse_args rejects invalid compression level (-z/--compress) */
static void test_parse_args_invalid_compression_level() { static void test_parse_args_invalid_compression_level() {
Config* cfg = config_create(); Config* cfg = config_create();
char* argv[] = {"fastsync", "-c", "25", "/src", "/dst"}; char* argv[] = {"fastsync", "-z", "25", "/src", "/dst"};
int positional_args[2]; int positional_args[2];
int positional_count = 0; int positional_count = 0;
@@ -466,10 +471,10 @@ static void test_parse_args_invalid_compression_level() {
config_delete(cfg); config_delete(cfg);
} }
/* Test parse_args accepts valid compression level */ /* Test parse_args accepts valid compression level (-z/--compress) */
static void test_parse_args_valid_compression_level() { static void test_parse_args_valid_compression_level() {
Config* cfg = config_create(); Config* cfg = config_create();
char* argv[] = {"fastsync", "-c", "10", "/src", "/dst"}; char* argv[] = {"fastsync", "-z", "10", "/src", "/dst"};
int positional_args[2]; int positional_args[2];
int positional_count = 0; int positional_count = 0;
@@ -977,11 +982,12 @@ static void test_parse_args_hard_links() {
config_delete(cfg); config_delete(cfg);
} }
/* -H/--hard-links violates the per-file streaming requirement of -s and the /* -H/--hard-links violates the per-file streaming requirement of
* payload-bearing tail-resume of --append: both combos are rejected up front. */ * --chunk-serialization and the payload-bearing tail-resume of --append: both
* combos are rejected up front. */
static void test_validate_config_hard_links_incompatible_modes() { static void test_validate_config_hard_links_incompatible_modes() {
Config* cfg = config_create(); Config* cfg = config_create();
char* argv_s[] = {"fastsync", "-H", "-s", "/src", "/dst"}; char* argv_s[] = {"fastsync", "-H", "--chunk-serialization", "/src", "/dst"};
int positional_args[2]; int positional_args[2];
int positional_count = 0; int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, argv_s, positional_args, &positional_count), 0); EXPECT_EQ_INT(parse_args(cfg, 5, argv_s, positional_args, &positional_count), 0);
@@ -1049,33 +1055,52 @@ static void test_parse_args_archive() {
int ret = parse_args(cfg, 4, argv, positional_args, &positional_count); int ret = parse_args(cfg, 4, argv, positional_args, &positional_count);
EXPECT_EQ_INT(ret, 0); EXPECT_EQ_INT(ret, 0);
EXPECT_TRUE(cfg->use_compression); EXPECT_TRUE(cfg->follow_symlinks);
EXPECT_TRUE(cfg->use_multithreading);
EXPECT_TRUE(cfg->use_metadata); EXPECT_TRUE(cfg->use_metadata);
EXPECT_TRUE(cfg->preserve_devices);
EXPECT_TRUE(cfg->preserve_specials);
EXPECT_FALSE(cfg->use_compression);
EXPECT_FALSE(cfg->use_multithreading);
config_delete(cfg); config_delete(cfg);
} }
/* Negations must override archive's implied options in argument order. */ /* Negations must override archive's implied options in argument order. Note:
* archive implies devices+specials, and device/special preservation itself
* forces metadata transmission (re-creating a node needs the metadata mode), so
* --no-preserve cannot turn metadata back off while archive keeps devices/specials
* on -- that is the correct interaction, not a bug. A link negation does work. */
static void test_parse_args_negations() { static void test_parse_args_negations() {
Config* cfg = config_create(); Config* cfg = config_create();
char* argv[] = {"fastsync", "--archive", "--no-compress", "--no-m", char* argv[] = {"fastsync", "--archive", "--no-links", "--no-preserve",
"--no-preserve", "--no-dry-run", "/src", "/dst"}; "--no-dry-run", "/src", "/dst"};
int positional_args[2]; int positional_args[2];
int positional_count = 0; int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 8, argv, positional_args, &positional_count), 0); EXPECT_EQ_INT(parse_args(cfg, 7, argv, positional_args, &positional_count), 0);
EXPECT_FALSE(cfg->use_compression); EXPECT_FALSE(cfg->follow_symlinks);
EXPECT_FALSE(cfg->use_multithreading); EXPECT_TRUE(cfg->use_metadata);
EXPECT_FALSE(cfg->use_metadata);
EXPECT_FALSE(cfg->dry_run); EXPECT_FALSE(cfg->dry_run);
EXPECT_EQ_INT(positional_count, 2); EXPECT_EQ_INT(positional_count, 2);
config_delete(cfg); config_delete(cfg);
} }
/* --no-preserve negates an explicit --preserve when nothing forces metadata back
* on (no devices/specials). */
static void test_parse_args_negate_preserve_without_devices() {
Config* cfg = config_create();
char* argv[] = {"fastsync", "--preserve", "--no-preserve", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, argv, positional_args, &positional_count), 0);
EXPECT_FALSE(cfg->use_metadata);
config_delete(cfg);
}
static void test_parse_args_negation_order() { static void test_parse_args_negation_order() {
Config* cfg = config_create(); Config* cfg = config_create();
char* argv[] = {"fastsync", "--no-z", "-c", "/src", "/dst"}; char* argv[] = {"fastsync", "--no-z", "-z", "/src", "/dst"};
int positional_args[2]; int positional_args[2];
int positional_count = 0; int positional_count = 0;
@@ -1428,9 +1453,11 @@ static void test_parse_args_secluded_args() {
config_delete(cfg); config_delete(cfg);
} }
static void test_parse_args_short_s_remains_chunk_serialization() { static void test_parse_args_chunk_serialization_long_form() {
Config* cfg = config_create(); Config* cfg = config_create();
char* argv[] = {"fastsync", "-s", "/src", "/dst"}; /* Chunk serialization is now long-form-only (the short -s is rsync's
* --secluded-args no-op). */
char* argv[] = {"fastsync", "--chunk-serialization", "/src", "/dst"};
int positional_args[2]; int positional_args[2];
int positional_count = 0; int positional_count = 0;
@@ -1651,7 +1678,7 @@ static void test_parse_args_fuzzy_respects_no_incremental() {
* sendfile (-f) modes reject -- mirroring the --delta constraint checks. */ * sendfile (-f) modes reject -- mirroring the --delta constraint checks. */
static void test_validate_config_fuzzy_incompatible_modes() { static void test_validate_config_fuzzy_incompatible_modes() {
Config* cfg = config_create(); Config* cfg = config_create();
char* argv[] = {"fastsync", "--fuzzy", "-s", "/src", "/dst"}; char* argv[] = {"fastsync", "--fuzzy", "--chunk-serialization", "/src", "/dst"};
int positional_args[2]; int positional_args[2];
int positional_count = 0; int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, argv, positional_args, &positional_count), 0); EXPECT_EQ_INT(parse_args(cfg, 5, argv, positional_args, &positional_count), 0);
@@ -1662,7 +1689,7 @@ static void test_validate_config_fuzzy_incompatible_modes() {
config_delete(cfg); config_delete(cfg);
cfg = config_create(); cfg = config_create();
char* sendfile_argv[] = {"fastsync", "--fuzzy", "-f", "/src", "/dst"}; char* sendfile_argv[] = {"fastsync", "--fuzzy", "--sendfile", "/src", "/dst"};
positional_count = 0; positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, sendfile_argv, positional_args, &positional_count), 0); EXPECT_EQ_INT(parse_args(cfg, 5, sendfile_argv, positional_args, &positional_count), 0);
cfg->send_directory = str_dup("/src"); cfg->send_directory = str_dup("/src");
@@ -2819,18 +2846,18 @@ static void test_parse_args_remote_option_rejects_bad_values() {
config_delete(cfg); config_delete(cfg);
} }
/* A short -M form must NOT be accepted as --remote-option: -M stays FastSync /* Since the Phase-7 CLI-namespace pass, -M is rsync's --remote-option short
* metadata mode (documented divergence). */ * form (FastSync metadata mode is long-only --preserve): it consumes the next
static void test_parse_args_remote_option_no_short_M() { * argv as a remote-option value and must NOT set FastSync metadata mode. */
static void test_parse_args_remote_option_short_M() {
Config* cfg = valid_client_config(); Config* cfg = valid_client_config();
EXPECT_NOT_NULL(cfg); EXPECT_NOT_NULL(cfg);
/* -M followed by a remote-option-looking word still means metadata mode. */ char* argv[] = {"fastsync", "-M", "--trust-sender", "--source-dir", "/src", "--dest-dir", "/dst"};
char* argv[] = {"fastsync", "-M", "-v", "--source-dir", "/src", "--dest-dir", "/dst"};
int positional_args[2]; int positional_args[2];
int positional_count = 0; int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 7, argv, positional_args, &positional_count), 0); EXPECT_EQ_INT(parse_args(cfg, 7, argv, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->use_metadata); EXPECT_FALSE(cfg->use_metadata);
EXPECT_EQ_INT(cfg->remote_option_count, 0); EXPECT_EQ_INT(cfg->remote_option_count, 1);
config_delete(cfg); config_delete(cfg);
} }
@@ -2955,6 +2982,7 @@ void test_client_cli() {
test_parse_args_rejects_invalid_info_flag(); test_parse_args_rejects_invalid_info_flag();
test_parse_args_archive(); test_parse_args_archive();
test_parse_args_negations(); test_parse_args_negations();
test_parse_args_negate_preserve_without_devices();
test_parse_args_negation_order(); test_parse_args_negation_order();
test_parse_args_no_preserve_blocks_implicit_metadata(); test_parse_args_no_preserve_blocks_implicit_metadata();
test_parse_args_rejects_unsafe_negation(); test_parse_args_rejects_unsafe_negation();
@@ -2970,7 +2998,7 @@ void test_client_cli() {
test_parse_args_stderr_modes(); test_parse_args_stderr_modes();
test_parse_args_rejects_unsupported_stderr_modes(); test_parse_args_rejects_unsupported_stderr_modes();
test_parse_args_secluded_args(); test_parse_args_secluded_args();
test_parse_args_short_s_remains_chunk_serialization(); test_parse_args_chunk_serialization_long_form();
test_parse_args_symlink_trust(); test_parse_args_symlink_trust();
test_parse_args_whole_file(); test_parse_args_whole_file();
test_parse_args_fuzzy_implies_delta(); test_parse_args_fuzzy_implies_delta();
@@ -3021,7 +3049,7 @@ void test_client_cli() {
test_parse_args_remote_option_space_form(); test_parse_args_remote_option_space_form();
test_parse_args_remote_option_missing_value(); test_parse_args_remote_option_missing_value();
test_parse_args_remote_option_rejects_bad_values(); test_parse_args_remote_option_rejects_bad_values();
test_parse_args_remote_option_no_short_M(); test_parse_args_remote_option_short_M();
test_parse_args_no_motd(); test_parse_args_no_motd();
test_parse_args_password_file(); test_parse_args_password_file();
} }