Release v2.28.0 #304

Merged
TapTap merged 39 commits from dev into main 2026-09-20 01:17:26 +02:00
17 changed files with 686 additions and 25 deletions
Showing only changes of commit eff9852038 - Show all commits
+8 -5
View File
@@ -117,9 +117,12 @@ matrix is classified as parity, caveat, or divergent in
per-type `Number of files` breakdown is not reproduced. `--progress` prints
rsync-style per-file blocks (without rsync's leading `./` line).
- Codecs match rsync 3.4.1: `zstd`/`lz4`/`zlib`/`zlibx` compression and
`xxh128`/`xxh3`/`xxh64`/`md5`/`md4`/`sha1`/`none` checksums, negotiated with
`auto`; `zlibx` behaves as `zlib`, and the transfer checksum is not separately
selectable.
`xxh128`/`xxh3`/`xxh64`/`md5`/`md4`/`sha1`/`none` checksums. `auto` honors
`RSYNC_COMPRESS_LIST`/`RSYNC_CHECKSUM_LIST` and otherwise follows rsync's
compiled-in order. An omitted `--compress-level` uses the codec's rsync
default (zstd 3, zlib/zlibx 6, lz4 ignored); `zlib`/`zlibx` share the
literal-only zlib path (rsync's zlibx semantics), and the transfer checksum is
not separately selectable.
The detailed flag matrix is maintained in
[`RSYNC_COMPAT.md`](RSYNC_COMPAT.md). It reports each row as **parity**,
@@ -505,8 +508,8 @@ features without changing the meaning of ordinary compatibility options.
|---|---|
| `-j`, `--threads[=N]` | Enable the multithreaded scanner/loader/sender pipeline. `N` (1–256) sets the parallel scanner worker count; bare `-j`/`--threads` uses the default. |
| `-z [level]`, `--compress [level]` | Enable streaming compression (default `zstd`), levels 1-22. |
| `--compress-level <n>` | Set the compression level. |
| `--zc <alg>` | Alias for `--compress-choice`. FastSync supports `zstd` (default), `lz4`, `zlib`, `zlibx`, `none`, and `auto`; `zlibx` behaves as `zlib`. |
| `--compress-level <n>` | Set the compression level (1-22). Omitted, each codec uses its rsync default: zstd 3, zlib/zlibx 6, lz4 ignored. |
| `--zc <alg>` | Alias for `--compress-choice`. FastSync supports `zstd` (default), `lz4`, `zlib`, `zlibx`, `none`, and `auto`; `zlib`/`zlibx` share the same literal-only zlib path. |
| `--zl <n>` | Alias for `--compress-level`. |
| `--skip-compress <list>` | Skip compression for `/`- or `,`-separated suffixes; defaults to rsync 3.4.1's built-in list. Incompatible with `--chunk-serialization`. |
| `--compress-threads <n>` | Use `n` zstd compression workers. Requires compression and a zstd build with threaded support; the setting affects sender CPU work only. |
+13 -10
View File
@@ -6,8 +6,8 @@ This document maps rsync's full feature set to FastSync's current implementation
| Status | Count | Description |
|--------|-------|-------------|
| ✅ Parity | 111 | Reproduces rsync's semantics for this option's scope |
| ⚠️ Caveat | 14 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
| ✅ Parity | 113 | Reproduces rsync's semantics for this option's scope |
| ⚠️ Caveat | 12 | Wired and tested, but carries a documented behavioral difference from rsync (named in the row and/or the wave notes) |
| ❌ Divergent | 32 | Rejected, an accepted no-op, deliberately non-rsync (native config/auth/batch, privileged namespaces, safe-subset privilege), or impossible on any portable filesystem call |
| **Total** | **157** | One row per rsync option/feature group; a row may name several spellings |
@@ -41,6 +41,8 @@ matrix is **111 ✅ / 13 ⚠️ / 33 ❌ = 157**.
**Parity track 2b (no-wire, on `feat/parity-2.28`).** An opt-in, paths-only metadata pre-count for `--progress`/`-P`/`--info=progress` (only when not `--quiet`) now gives the `to-chk` denominator rsync's full file-list total (every regular file, directory, symlink and special plus the transfer root) and emits the per-directory/symlink/special name lines, in both the sequential and `--threads` paths; `--delete-during`/`--delete-delay` reuse their keep-set pre-scan so no second walk happens, and non-progress runs are unaffected. A fresh multi-directory differential against rsync 3.4.1 matches the name set and the `to-chk` denominator, while a single-file transfer stays byte-identical; the remaining caveats are emission order (rsync's sorted depth-first vs FastSync's readdir/BFS stream, so the `to-chk` numerator and interleaving differ) and re-run/receiver-state-driven over-naming (unconditional `./`, ancestor dirs emitted with a transferred child, and no quick-check for symlinks/empty dirs), keeping the row ⚠️. The matrix is unchanged at **111 ✅ / 14 ⚠️ / 32 ❌ = 157**.
**Parity track 3a (no-wire, on `feat/parity-2.28`).** Three codec refinements, none of which change the wire layout (`PROTOCOL_VERSION` stays 2.28.0): (1) an omitted `--compress-level` now resolves to rsync 3.4.1's per-codec default (zstd 3, zlib/zlibx 6, lz4 ignored) and an explicit level is clamped per codec (zstd 1-22, zlib/zlibx 1-9), verified against `rsync --debug=NSTR1`; (2) `auto` now honors `RSYNC_COMPRESS_LIST`/`RSYNC_CHECKSUM_LIST` (rsync's whitespace-separated preference syntax, unknown names skipped, first supported wins, all-unknown is exit 4) before the compiled-in order, with an explicit `--zc`/`--cc` still winning; and (3) `--compress-choice=zlibx` is reclassified out of the caveat list — FastSync's zlib stream already carries only the literal/delta bytes (rsync's zlibx semantics) and is observably identical to `--zc=zlib`, so the zlib/zlibx aliasing is only an implementation detail. The matrix is now **113 ✅ / 12 ⚠️ / 32 ❌ = 157**.
**Parity completion wave (protocol 2.23.0 → 2.26.0).** This wave closed the
remaining gaps the rsync-parity wave left open (delete timing, wire counters and
output, codec breadth, general `-R`/`-d`, the full filter grammar, receiver-side
@@ -649,7 +651,7 @@ targets verbatim, matching rsync.
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `--checksum` | Skip based on checksum | ✅ Parity | `-c`/`--checksum` compares per-file whole-file content digests to skip unchanged files. **As of protocol 2.23.0 the short `-c` implies the checksum quick-check**, so a plain `-c` run verifies content rather than only affecting the `--incremental` handshake. The digest algorithm is `xxh128` by default (protocol 2.26.0's negotiated default) and is selectable via `--checksum-choice`/`--cc` (`xxh128`/`xxh3`/`xxh64`/`xxhash`/`md5`/`md4`/`sha1`/`none`/`auto`, plus rsync's two-name form) and `--checksum-seed=NUM` (see those rows) |
| `--checksum-choice=STR`, `--cc=STR` | Choose checksum algorithm | ⚠️ Caveat | Real algorithm selection for the per-file whole-file digest used by the `--incremental`/`--checksum` handshake and basis-dir verification. **Protocol 2.26.0 accepts rsync 3.4.1's full set** — `xxh128` (the negotiated default), `xxh3`, `xxh64`, `xxhash`, `md5`, `md4`, `sha1`, `none`, `auto`, and the two-name `transfer,pre-transfer` form — with rsync's exit-4 rejection of an unknown name and of `none` on the transfer side when `--checksum` is on. `--cc=ALG` and space forms both parse. The algorithm id and seed cross the wire; the receiver hashes its old file with the same algorithm+seed and the per-file `STATUS_CHECK` handshake carries a bounded digest pinned to the negotiated length. `checksum_digest_file` now streams **every** supported algorithm (md4 via the self-contained RFC 1320 code, sha1/md5 via EVP, none as an empty digest), so the streaming path matches its contract, and `--out-format %C` uses the selected **transfer** half of a two-name choice and renders each algorithm byte-for-byte like rsync (xxh128 high-then-low, xxh64/xxh3 big-endian, md5/md4/sha1 standard hex, none a blank 2-char column) — differential-tested across all algorithms. **Remaining divergences:** rsync uses this choice for the block checksum on the wire too, while FastSync selects only the whole-file comparison digest and keeps the delta BLOCK strong checksum at xxHash32; the `RSYNC_CHECKSUM_LIST` environment variable is not consulted; and `auto` always resolves deterministically to the first supported entry in rsync's preference order rather than probing the peer |
| `--checksum-choice=STR`, `--cc=STR` | Choose checksum algorithm | ⚠️ Caveat | Real algorithm selection for the per-file whole-file digest used by the `--incremental`/`--checksum` handshake and basis-dir verification. **Protocol 2.26.0 accepts rsync 3.4.1's full set** — `xxh128` (the negotiated default), `xxh3`, `xxh64`, `xxhash`, `md5`, `md4`, `sha1`, `none`, `auto`, and the two-name `transfer,pre-transfer` form — with rsync's exit-4 rejection of an unknown name and of `none` on the transfer side when `--checksum` is on. `--cc=ALG` and space forms both parse. The algorithm id and seed cross the wire; the receiver hashes its old file with the same algorithm+seed and the per-file `STATUS_CHECK` handshake carries a bounded digest pinned to the negotiated length. `checksum_digest_file` now streams **every** supported algorithm (md4 via the self-contained RFC 1320 code, sha1/md5 via EVP, none as an empty digest), so the streaming path matches its contract, and `--out-format %C` uses the selected **transfer** half of a two-name choice and renders each algorithm byte-for-byte like rsync (xxh128 high-then-low, xxh64/xxh3 big-endian, md5/md4/sha1 standard hex, none a blank 2-char column) — differential-tested across all algorithms. **Remaining divergence:** rsync uses this choice for the block checksum on the wire too, while FastSync selects only the whole-file comparison digest and keeps the delta BLOCK strong checksum at xxHash32. `auto` now consults `RSYNC_CHECKSUM_LIST` (rsync's whitespace-separated preference list; unknown names skipped, first supported wins, all-unknown is exit 4) before the compiled-in order; because both peers run the identical build this deterministic resolution needs no rsync peer probe, and an explicit `--cc` still wins. The list is differential-tested through `--out-format %C` (byte-identical digests to rsync for md5/sha1/xxh3) |
| `--compare-dest=DIR` | Compare dest files relative to DIR | ❌ Divergent | DIR is a receiver-side basis; protocol 2.26.0 uses an absolute path verbatim (rsync semantics) and resolves a relative path below the destination root (`..` components are rejected, `//` collapsed and trailing `/` dropped). On the receiver's per-file check (implies `--incremental`) an exact match = same size + mtime (unless `--size-only`; `-I` disables matching) **and** equal xxHash64 of the sender's file; a match suppresses the data transfer. compare-dest never copies: it only skips a file the destination does **not** already hold (sparse destination, rsync parity), and is consulted before the normal delta/full paths. Repeatable; searched in command-line order, first match wins. **Reclassified Divergent (differential evidence):** FastSync verifies a basis hit's content with xxHash64 while rsync's `--size-only` quick check trusts size (and mtime) alone, so with a same-size/different-content basis rsync skips/links the *wrong* basis content while FastSync transfers the source — a deliberate safety-stricter behavior that cannot match rsync (see `test_basis_dir_size_only_content_residual` in `tests/integration/test_parity_quickwins.py`). Attribute-only differences on a match are also not re-applied (data is skipped so the sender never sends metadata). Sizing: FastSync's whole-file payload limit is 256 MiB on **every** transfer path (not basis-specific); rsync applies basis dirs to arbitrary sizes, so FastSync refuses a basis run whose source contains a larger file up front with a clear error before any transfer. Wire: a basis-count field is always present on the config frame (protocol 2.9.0, so clients and servers must both be 2.9.0) |
| `--copy-dest=DIR` | Include copies of unchanged files | ❌ Divergent | Same basis rules as `--compare-dest`, but an exact match materializes a **local copy** of the DIR file into the destination (via the normal atomic temp+rename store path, so `--existing`/`--ignore-existing`/`--update`/`--backup`/`--delay-updates` all still apply) instead of transferring data. Repeatable; command-line order = priority. Content is xxHash64-verified before the copy. **Reclassified Divergent** for the same basis-hit verification divergence as `--compare-dest`: rsync's `--size-only` size/quick-check match rings a same-size/different-content file as unchanged and copies the wrong basis bytes, while FastSync's xxHash verification transfers the source (differential test `test_basis_dir_size_only_content_residual`); a basis-hit also keeps whatever metadata the copy derived from the basis rather than rsync's "copy + fix attributes" in attribute-only cases. Requires `--incremental` (implied); incompatible with `-s`. Wire: protocol 2.9.0 |
| `--link-dest=DIR` | Hardlink to files when unchanged | ❌ Divergent | Same basis rules as `--copy-dest`, but an exact match installs an atomic **hard link** to the DIR file (temp hard link + rename) so no data or disk space is used; where the link is impossible (basis on another filesystem, filesystem refuses links) it falls back cleanly to a byte-identical local copy, never a corrupt/partial file. `--delay-updates` stages the link and publishes by rename, so the final entry stays a real hard link. Repeatable (searched in command-line order, first match wins). Content is xxHash64-verified before linking. **Reclassified Divergent** for the shared basis-hit divergence: with `--size-only` a same-size/different-content basis is linked by rsync (installing wrong content) but FastSync detects the xxHash mismatch and transfers the source (differential test `test_basis_dir_size_only_content_residual`). Other inherent caveats: protocol 2.26.0 re-links an already up-to-date destination file to the basis; a link keeps the basis inode's own mode/uid/gid and mtime — metadata is never written through the shared inode (that would mutate the basis file), so a later `--inplace` run that rewrites such a destination path **will mutate the basis snapshot** through the shared inode (use `--copy-dest` when the destination must stay independently writable); a `--remove-source-files` source satisfied by a basis dir is treated as skipped and therefore **retained** (never removed); basis dirs are excluded from `--delete`. Requires `--incremental` (implied); incompatible with `-s`. Wire: protocol 2.9.0 |
@@ -659,9 +661,9 @@ targets verbatim, matching rsync.
| Flag | Rsync Description | FastSync Status | Notes |
|------|-------------------|-----------------|-------|
| `-z`, `--compress` | Compress file data | ⚠️ Caveat | Streaming compression. **Protocol 2.26.0 implements rsync 3.4.1's codec set** (`zstd` default, `lz4`, `zlib`, `zlibx`, `none`), selectable via `--compress-choice`/`--zc` and negotiated with `auto`. `-z` is the compression short form; `-c` is rsync's `--checksum`. `--skip-compress` applies rsync 3.4.1's default suffix list when no list is given. **Remaining codec divergence:** `zlibx` is treated as `zlib`, per-codec level defaults are not mirrored, and `auto` does not probe the peer |
| `--compress-choice=STR`, `--zc=STR` | Choose compression algorithm | ⚠️ Caveat | Protocol 2.26.0 accepts rsync 3.4.1's compiled-in choices — `zstd` (default), `lz4`, `zlib`, `zlibx`, `none`, `auto` — and rejects an unknown name with exit 4 like rsync. The negotiated codec id crosses the wire (`compression_algo`), so the receiver decodes with the sender's codec. `--zc` is the alias. **Remaining divergences:** `zlibx` behaves as `zlib` (there is no separate zlibx path), the per-codec compression-level defaults differ from rsync's, and `auto` always resolves to the first supported entry in rsync's preference order rather than probing the peer (no `RSYNC_COMPRESS_LIST` handling) |
| `--compress-level=NUM`, `--zl=NUM` | Set compression level | ✅ Parity | 1-22, default 5 |
| `-z`, `--compress` | Compress file data | ✅ Parity | Streaming compression. **Protocol 2.26.0 implements rsync 3.4.1's codec set** (`zstd` default, `lz4`, `zlib`, `zlibx`, `none`), selectable via `--compress-choice`/`--zc` and negotiated with `auto`. `-z` is the compression short form; `-c` is rsync's `--checksum`. `--skip-compress` applies rsync 3.4.1's default suffix list when no list is given. **Track 3a closes the codec caveats:** `zlibx` is no longer a divergence — FastSync's zlib stream already carries only the delta/token (literal) bytes, which is exactly rsync's zlibx semantics, so `--zc=zlib` and `--zc=zlibx` land the same tree/stdout/exit (differential `test_compress_codec_matches_rsync_bytes`) and the zlib/zlibx aliasing is only an implementation detail. Each codec now uses rsync's own default `--compress-level` (zstd 3, zlib/zlibx 6, lz4 ignored) and `auto` consults `RSYNC_COMPRESS_LIST` before the compiled-in order; the deterministic same-build resolution needs no peer probe |
| `--compress-choice=STR`, `--zc=STR` | Choose compression algorithm | ✅ Parity | Protocol 2.26.0 accepts rsync 3.4.1's compiled-in choices — `zstd` (default), `lz4`, `zlib`, `zlibx`, `none`, `auto` — and rejects an unknown name with exit 4 like rsync. The negotiated codec id crosses the wire (`compression_algo`), so the receiver decodes with the sender's codec. `--zc` is the alias. `auto` now resolves through `RSYNC_COMPRESS_LIST` (whitespace-separated; unknown names skipped, first supported wins, all-unknown is exit 4) and then the compiled-in order, and an explicit `--zc` wins; the deterministic same-build resolution needs no peer probe. `zlib`/`zlibx` share FastSync's literal-only zlib path, which is rsync's zlibx behavior and is observably identical for both, so `zlibx` is not a divergence (the aliasing is an implementation detail) |
| `--compress-level=NUM`, `--zl=NUM` | Set compression level | ✅ Parity | Accepted range 1-22. When omitted, rsync 3.4.1's **per-codec default** applies: zstd 3 (`ZSTD_CLEVEL_DEFAULT`), zlib/zlibx 6 (`Z_DEFAULT_COMPRESSION` resolved), lz4 ignored (no tunable level; FastSync keeps a positive gate value and `lz4_compress` ignores it, so the bytes match rsync). An explicit level is clamped per codec like rsync's `init_compression_level()`: zstd 1-22, zlib/zlibx 1-9, lz4 ignored. Verified against `rsync --debug=NSTR1`, which reports the same effective level per codec |
| `--compress-threads=NUM` | Set compression threads | ✅ Parity | `compression_threads` config field (client-only; does not cross the wire). Sets the number of worker threads used by the zstd compression pool to NUM (1..64; 0/garbage/oversized rejected up front). Accepted in both `--compress-threads=NUM` and two-argument `--compress-threads NUM` forms. Composes with `-z`/compression; under the `-j`/`--threads` multithreaded pipeline it parallelizes compressed chunk encoding. See test_tcp.py `-z --compress-threads=2` and test_client_cli.c |
| `--skip-compress=LIST` | Skip compress for suffixes | ✅ Parity | Comma-separated (or `/`-separated, as in rsync) case-insensitive suffix list; a leading dot is optional; an empty list skips none. **When the option is omitted, rsync 3.4.1's built-in default suffix list applies** (`3g2 3gp 7z aac … zip zst`); an explicit list replaces that default entirely, matching rsync. A user-supplied list is a client-side compression choice; incompatible with FastSync chunk serialization (`-s`) |
@@ -1111,10 +1113,11 @@ wire protocol three times (full rationale in `src/shared/config.h`):
`sha1`, `none`, `auto`, plus the two-name `transfer,pre-transfer` form.
Unknown names and `none` on the transfer side under `--checksum` exit 4 like
rsync.
- **Remaining codec residuals:** `zlibx` behaves as `zlib`; the transfer
checksum is not independently selectable (only the whole-file comparison
digest is); per-codec level defaults differ; and `RSYNC_CHECKSUM_LIST`/
`RSYNC_COMPRESS_LIST` are not consulted.
- **Codec residuals:** the transfer checksum is not independently selectable
(only the whole-file comparison digest is); `zlib`/`zlibx` share FastSync's
literal-only zlib path (rsync's zlibx behavior, observably identical for both,
so `zlibx` is not a divergence). Per-codec compression-level defaults and
`RSYNC_COMPRESS_LIST`/`RSYNC_CHECKSUM_LIST` are implemented (track 3a).
### Selection, paths, and filters
+61 -8
View File
@@ -161,10 +161,16 @@ static int set_compression_choice(Config* config, const char* value) {
return -1;
}
int algo;
if (strcasecmp(value, "auto") == 0)
algo = (int)compression_negotiate_default();
else
if (strcasecmp(value, "auto") == 0) {
algo = compression_choice_resolve();
if (algo < 0) {
log_message(LOG_LEVEL_ERROR, "RSYNC_COMPRESS_LIST names no supported compression algorithm");
config->cli_exit_code = 4;
return -1;
}
} else {
algo = compression_algo_from_name(value);
}
if (algo < 0) {
log_message(LOG_LEVEL_ERROR,
"--compress-choice '%s' is not a supported algorithm; FastSync supports zstd, "
@@ -228,16 +234,25 @@ static int set_checksum_choice(Config* config, const char* value) {
config->cli_exit_code = 4;
return -1;
}
ChecksumAlgo negotiated = checksum_negotiate_default();
int negotiated = -1;
if (rc1 == 1 || rc2 == 1) {
negotiated = checksum_choice_resolve();
if (negotiated < 0) {
log_message(LOG_LEVEL_ERROR, "RSYNC_CHECKSUM_LIST names no supported checksum algorithm");
config->cli_exit_code = 4;
return -1;
}
}
if (rc1 == 1)
transfer = (int)negotiated;
transfer = negotiated;
if (!name2)
pre = transfer;
else if (rc2 == 1)
pre = (int)negotiated;
pre = negotiated;
config->checksum_algo = pre;
config->checksum_transfer_algo = transfer;
config->checksum_choice_set = true;
/* rsync: "none" for the transfer checksum forces --whole-file. */
if (transfer == (int)CHECKSUM_ALGO_NONE)
config->whole_file = true;
@@ -1464,6 +1479,8 @@ static bool cli_handle_table_option(CliParseCtx* ctx) {
ctx->exit_code = -1;
return true;
}
if (entry->offset == offsetof(Config, compression_level))
config->compression_level_set = true;
if (entry->offset == offsetof(Config, chmod_spec)) {
mode_t ignored;
if (!chmod_apply(0, config->chmod_spec, &ignored)) {
@@ -1748,6 +1765,7 @@ static bool cli_handle_transfer_flags(CliParseCtx* ctx) {
return true;
}
config->compression_level = (int)level;
config->compression_level_set = true;
log_info_message(LOG_INFO_MISC, "Set Compression level to %ld", level);
ctx->i++;
}
@@ -2550,8 +2568,43 @@ static int cli_finalize_config(Config* config, bool verbose, bool no_delta, bool
config->use_compression = (algo != (int)COMPRESSION_ALGO_NONE);
}
}
if (config->use_compression && config->compression_algo == (int)COMPRESSION_ALGO_NONE)
config->compression_algo = (int)compression_negotiate_default();
/* A bare -z (no --compress-choice) resolves like rsync's "auto": the
* RSYNC_COMPRESS_LIST preference list first, then the compiled-in order. A
* list that names no supported codec is rsync's failed negotiation (exit 4). */
if (config->use_compression && !config->compress_choice) {
int resolved = compression_choice_resolve();
if (resolved < 0) {
log_message(LOG_LEVEL_ERROR, "RSYNC_COMPRESS_LIST names no supported compression algorithm");
config->cli_exit_code = 4;
return -1;
}
config->compression_algo = resolved;
if (resolved == (int)COMPRESSION_ALGO_NONE)
config->use_compression = false;
}
/* Apply rsync's per-codec compression level: an explicit --compress-level is
* clamped to the codec's range, otherwise the codec's own default is used. */
if (config->use_compression) {
CompressionAlgo algo = (CompressionAlgo)config->compression_algo;
config->compression_level = config->compression_level_set
? compression_clamp_level(algo, config->compression_level)
: compression_default_level(algo);
log_debug_message(LOG_DEBUG_UTIL, "Client compression: %s (level %d)",
compression_algo_name(algo), config->compression_level);
}
/* The negotiated checksum is always resolved (rsync negotiates one for the
* delta strong sum even without --checksum): RSYNC_CHECKSUM_LIST first, then
* the compiled-in order. An explicit --checksum-choice already set it. */
if (!config->checksum_choice_set) {
int resolved = checksum_choice_resolve();
if (resolved < 0) {
log_message(LOG_LEVEL_ERROR, "RSYNC_CHECKSUM_LIST names no supported checksum algorithm");
config->cli_exit_code = 4;
return -1;
}
config->checksum_algo = resolved;
config->checksum_transfer_algo = resolved;
}
/* rsync parity: "none" as the pre-transfer checksum cannot be combined with
* --checksum (exit 4). The check runs here because --checksum may appear on
* either side of --checksum-choice. */
+15 -1
View File
@@ -1,4 +1,5 @@
#include "checksum.h"
#include "utils.h"
#include <fcntl.h>
#include <openssl/evp.h>
#include <string.h>
@@ -396,7 +397,7 @@ uint8_t checksum_digest_len(ChecksumAlgo algo) {
return 0;
}
ChecksumAlgo checksum_negotiate_default(void) {
static ChecksumAlgo compiled_checksum_preference_first(void) {
/* rsync 3.4.1 default preference order; every entry is compiled in, so this
* resolves to xxh128. */
static const ChecksumAlgo preference[] = {
@@ -409,3 +410,16 @@ ChecksumAlgo checksum_negotiate_default(void) {
}
return CHECKSUM_ALGO_XXH64;
}
int checksum_choice_resolve(void) {
bool specified = false;
int env = env_choice_first("RSYNC_CHECKSUM_LIST", checksum_algo_from_name, &specified);
if (specified)
return env; /* -1 = the list named no supported checksum */
return (int)compiled_checksum_preference_first();
}
ChecksumAlgo checksum_negotiate_default(void) {
int resolved = checksum_choice_resolve();
return resolved >= 0 ? (ChecksumAlgo)resolved : compiled_checksum_preference_first();
}
+7
View File
@@ -72,4 +72,11 @@ uint8_t checksum_digest_len(ChecksumAlgo algo);
* xxh128 xxh3 xxh64 md5 md4 sha1 none). Used to resolve "auto". */
ChecksumAlgo checksum_negotiate_default(void);
/* Resolve "auto" the way rsync does: the first supported name in
* RSYNC_CHECKSUM_LIST (whitespace-separated, client half ends at '&'), then the
* compiled-in preference order when the variable is unset/blank. Returns -1
* when the variable is set but names no supported checksum (rsync's failed
* negotiation), otherwise a valid ChecksumAlgo id. */
int checksum_choice_resolve(void);
#endif /* CHECKSUM_H */
+53 -1
View File
@@ -2,6 +2,7 @@
#include "data.h"
#include "log.h"
#include "protocol.h"
#include "utils.h"
#include <limits.h>
#include <lz4.h>
#include <stdatomic.h>
@@ -119,7 +120,7 @@ bool compression_algo_enabled(CompressionAlgo algo) {
return algo != COMPRESSION_ALGO_NONE;
}
CompressionAlgo compression_negotiate_default(void) {
static CompressionAlgo compiled_preference_first(void) {
/* rsync 3.4.1 default preference order; every entry is compiled in, so this
* resolves to zstd. */
static const CompressionAlgo preference[] = {
@@ -133,6 +134,57 @@ CompressionAlgo compression_negotiate_default(void) {
return COMPRESSION_ALGO_ZSTD;
}
int compression_choice_resolve(void) {
bool specified = false;
int env = env_choice_first("RSYNC_COMPRESS_LIST", compression_algo_from_name, &specified);
if (specified)
return env; /* -1 = the list named no supported codec */
return (int)compiled_preference_first();
}
CompressionAlgo compression_negotiate_default(void) {
int resolved = compression_choice_resolve();
return resolved >= 0 ? (CompressionAlgo)resolved : compiled_preference_first();
}
int compression_default_level(CompressionAlgo algo) {
switch (algo) {
case COMPRESSION_ALGO_ZSTD:
return ZSTD_CLEVEL_DEFAULT;
case COMPRESSION_ALGO_ZLIB:
case COMPRESSION_ALGO_ZLIBX:
return 6; /* rsync resolves zlib's Z_DEFAULT_COMPRESSION (-1) to 6 */
case COMPRESSION_ALGO_LZ4:
return 1; /* rsync lz4 level is 0/ignored; positive keeps the gate on */
case COMPRESSION_ALGO_NONE:
return 0;
}
return 0;
}
int compression_clamp_level(CompressionAlgo algo, int level) {
switch (algo) {
case COMPRESSION_ALGO_ZSTD:
if (level < 1)
return 1;
if (level > 22)
return 22;
return level;
case COMPRESSION_ALGO_ZLIB:
case COMPRESSION_ALGO_ZLIBX:
if (level < 1)
return 1;
if (level > 9)
return 9;
return level;
case COMPRESSION_ALGO_LZ4:
return 1; /* ignored by lz4_compress; keeps the "compress" gate on */
case COMPRESSION_ALGO_NONE:
return 0;
}
return level;
}
void compression_set_algo(CompressionAlgo algo) {
if (compression_algo_valid((int)algo))
atomic_store(&g_compression_algo, (int)algo);
+20
View File
@@ -35,6 +35,26 @@ bool compression_algo_valid(int algo);
* "auto". */
CompressionAlgo compression_negotiate_default(void);
/* Resolve "auto" the way rsync does: the first supported name in
* RSYNC_COMPRESS_LIST (whitespace-separated, client half ends at '&'), then the
* compiled-in preference order when the variable is unset/blank. Returns -1
* when the variable is set but names no supported codec (rsync's failed
* negotiation), otherwise a valid CompressionAlgo id. */
int compression_choice_resolve(void);
/* rsync 3.4.1's per-codec default level, applied when the user did not pass
* --compress-level/--zl. zstd uses ZSTD_CLEVEL_DEFAULT (3) and zlib/zlibx the
* resolved Z_DEFAULT_COMPRESSION (6). lz4 has no tunable level in rsync
* (always the default acceleration); FastSync returns a positive placeholder so
* its "level > 0" compression gate stays engaged, and lz4_compress ignores the
* value, so the output is identical to rsync's. none is 0. */
int compression_default_level(CompressionAlgo algo);
/* Clamp an explicit --compress-level to the codec's accepted range the way
* rsync's init_compression_level() does: zstd 1..22, zlib/zlibx 1..9, lz4
* ignored (fixed positive placeholder), none 0. */
int compression_clamp_level(CompressionAlgo algo, int level);
/* True when the algorithm actually compresses (i.e. is not NONE). */
bool compression_algo_enabled(CompressionAlgo algo);
+2
View File
@@ -70,6 +70,8 @@ static void config_set_defaults(Config* config) {
config->ignore_missing_args = false;
config->checksum_transfer_algo = CHECKSUM_ALGO_DEFAULT;
config->cli_exit_code = 0;
config->compression_level_set = false;
config->checksum_choice_set = false;
config->filters = NULL;
config->files_from = NULL;
config->files_from_set = NULL;
+6
View File
@@ -419,6 +419,12 @@ typedef struct Config {
* for an unsupported checksum/compress algorithm) so main() can mirror it. */
int checksum_transfer_algo;
int cli_exit_code;
/* Client-only "the user explicitly chose" bits. They let the per-codec
* default level / checksum list be applied only when the corresponding
* rsync option was omitted (an explicit --compress-level / --checksum-choice
* always wins). Never serialized. */
bool compression_level_set;
bool checksum_choice_set;
// Issue #129: Advanced file selection. These fields are CLIENT-ONLY: they are
// never serialized to the wire (the receiver must not learn them).
+42
View File
@@ -2,6 +2,7 @@
#include "array_list.h"
#include "log.h"
#include <arpa/inet.h>
#include <ctype.h>
#include <dirent.h>
#include <errno.h>
#include <fcntl.h>
@@ -142,6 +143,47 @@ char* str_dup(const char* string) {
return new_string;
}
int env_choice_first(const char* env_name, int (*resolve)(const char*), bool* specified) {
if (specified)
*specified = false;
if (!env_name || !resolve)
return -1;
const char* env = getenv(env_name);
if (!env)
return -1;
bool saw_nonblank = false;
const char* p = env;
while (*p) {
if (*p == '&')
break;
if (isspace((unsigned char)*p)) {
p++;
continue;
}
saw_nonblank = true;
char token[64];
size_t len = 0;
while (*p && *p != '&' && !isspace((unsigned char)*p)) {
if (len < sizeof(token) - 1)
token[len++] = *p;
p++;
}
token[len] = '\0';
if (len > 0) {
int id = resolve(token);
if (id >= 0) {
if (specified)
*specified = true;
return id;
}
}
}
if (specified)
*specified = saw_nonblank;
return -1;
}
#define STR_HASH_SET_MIN_CAPACITY 16
static size_t str_hash_set_hash(const char* key, size_t len) {
+11
View File
@@ -81,6 +81,17 @@ bool path_index_has_descendant(const PathIndex* index, const char* path);
char* str_dup(const char* string);
char* output_escape(const char* string, bool eight_bit_output);
/* Resolve the first supported name from a rsync algorithm-preference
* environment variable (RSYNC_COMPRESS_LIST / RSYNC_CHECKSUM_LIST). `resolve`
* maps a case-insensitive name to an algorithm id (>= 0) or -1 for an unknown
* name. rsync's syntax is a whitespace-separated list (comma/colon are NOT
* separators); the client-side half ends at '&'. Unknown entries are skipped
* and the first resolvable one wins. *specified is set true when the variable
* holds at least one non-blank character. Returns the first resolvable id, or
* -1 when the variable is unset/blank or names no supported algorithm. */
int env_choice_first(const char* env_name, int (*resolve)(const char*), bool* specified);
/* Upper bound on one line/token read from a local list file (--files-from,
* --exclude-from/--include-from, .rsync-filter). Mirrors MAX_STRING_SIZE and
* stops a hostile multi-gigabyte line from forcing unbounded allocation. */
+213
View File
@@ -0,0 +1,213 @@
"""Differential tests for the client CLI's codec defaults and env lists.
Track 3a of the rsync-parity plan pins two rsync 3.4.1 behaviors that are
resolved entirely on the client:
* the per-codec default ``--compress-level`` (zstd 3, zlib/zlibx 6, lz4
ignored) applied when the user omits ``--compress-level``/``--zl``, with an
explicit level clamped to the codec's range; and
* the ``RSYNC_COMPRESS_LIST`` / ``RSYNC_CHECKSUM_LIST`` preference lists that
rsync's ``auto`` consults before its compiled-in order (whitespace-separated,
unknown names skipped, first supported wins, all-unknown is exit 4).
The rsync side is observed through ``--debug=NSTR1``; FastSync publishes its
resolved codec/level through ``--debug=util``. The checksum side is confirmed
byte-for-byte through ``--out-format %C``. The rsync-based tests skip cleanly
when rsync is not installed.
"""
import os
import re
import shutil
import subprocess
import sys
import pytest
sys.path.insert(0, os.path.dirname(__file__))
from common import (
TEST_DATA_DIR,
run_client,
clean_dir,
get_dest_received_dir,
)
RSYNC = shutil.which("rsync")
requires_rsync = pytest.mark.skipif(RSYNC is None, reason="rsync 3.4.1 not installed")
CODEC_ROOT = os.path.join(TEST_DATA_DIR, "cli_differential")
_COMPRESS_RE = re.compile(r"compress(?:ion)?: (\w+) \(level (-?\d+)\)")
def _rsync(args):
env = dict(os.environ, LC_ALL="C")
return subprocess.run([RSYNC] + args, capture_output=True, text=True, env=env, timeout=120)
def _scratch(tag):
path = os.path.join(CODEC_ROOT, tag)
clean_dir(path)
os.makedirs(path, exist_ok=True)
return path
def _make_corpus(root):
clean_dir(root)
os.makedirs(root, exist_ok=True)
with open(os.path.join(root, "big.bin"), "wb") as fh:
fh.write(b"FastSync codec payload " * 4096)
with open(os.path.join(root, "small.txt"), "wb") as fh:
fh.write(b"hello codec world\n" * 32)
return root
def _rsync_compress_level(choice, level):
src = _make_corpus(_scratch(f"lvl_src_{choice}_{level}"))
dst = _scratch(f"lvl_rsync_{choice}_{level}")
args = ["-a", "-z", f"--zc={choice}"]
if level is not None:
args.append(f"--zl={level}")
args += ["--debug=NSTR1", src + "/", dst + "/"]
result = _rsync(args)
assert result.returncode == 0, result.stderr
match = _COMPRESS_RE.search(result.stdout + result.stderr)
assert match, (result.stdout, result.stderr)
return match.group(1), int(match.group(2))
def _fastsync_compress_level(choice, level, shared_server):
src = _make_corpus(_scratch(f"lvl_src_fs_{choice}_{level}"))
dst = _scratch(f"lvl_fs_{choice}_{level}")
args = ["-a", "-z", f"--zc={choice}"]
if level is not None:
args.append(f"--zl={level}")
args += ["-v", "--debug=util"]
result, _ = run_client(src, dst, flags=args, port=shared_server.port)
assert result.returncode == 0, (result.stderr or result.stdout)[:300]
match = _COMPRESS_RE.search(result.stdout)
assert match, result.stdout[:500]
return match.group(1), int(match.group(2))
class TestPerCodecCompressionLevelDefaults:
"""``--compress-level`` defaults and clamping match rsync per codec."""
# FastSync uses a positive lz4 placeholder because its "level > 0" gate
# enables compression; lz4_compress ignores the value, so rsync's level 0
# and FastSync's level 1 produce the same bytes.
CASES = [
("zstd", None, 3),
("zlib", None, 6),
("zlibx", None, 6),
("lz4", None, 1),
("zstd", 10, 10),
("zlib", 15, 9),
("zlib", 3, 3),
("lz4", 15, 1),
]
@requires_rsync
@pytest.mark.ci
@pytest.mark.parametrize("choice,level,fs_level", CASES)
def test_level_matches_rsync(self, choice, level, fs_level, shared_server):
rsync_algo, rsync_level = _rsync_compress_level(choice, level)
fs_algo, fs_level_actual = _fastsync_compress_level(choice, level, shared_server)
assert rsync_algo == choice
assert fs_algo == choice
if choice == "lz4":
assert rsync_level == 0 and fs_level_actual > 0
else:
assert rsync_level == fs_level
assert fs_level_actual == fs_level
class TestEnvPreferenceLists:
"""``RSYNC_COMPRESS_LIST`` / ``RSYNC_CHECKSUM_LIST`` drive auto like rsync."""
# (env value, expected codec, rsync level, FastSync level)
COMPRESS_CASES = [
("zlib lz4", "zlib", 6, 6),
("lz4 zstd", "lz4", 0, 1),
("bogus zstd zlib", "zstd", 3, 3),
(" ", "zstd", 3, 3),
]
@requires_rsync
@pytest.mark.ci
@pytest.mark.parametrize("env,algo,rsync_level,fs_level", COMPRESS_CASES)
def test_compress_list_matches_rsync(self, env, algo, rsync_level, fs_level, shared_server,
monkeypatch):
monkeypatch.setenv("RSYNC_COMPRESS_LIST", env)
src = _make_corpus(_scratch(f"envc_src_{algo}"))
rdst = _scratch(f"envc_rsync_{algo}")
rs = _rsync(["-a", "-z", "--debug=NSTR1", src + "/", rdst + "/"])
assert rs.returncode == 0, rs.stderr
rm = _COMPRESS_RE.search(rs.stdout + rs.stderr)
assert rm, (rs.stdout, rs.stderr)
assert rm.group(1) == algo
assert int(rm.group(2)) == rsync_level
fdst = _scratch(f"envc_fs_{algo}")
result, _ = run_client(src, fdst, flags=["-a", "-z", "-v", "--debug=util"],
port=shared_server.port)
assert result.returncode == 0, (result.stderr or result.stdout)[:300]
fm = _COMPRESS_RE.search(result.stdout)
assert fm, result.stdout[:500]
assert fm.group(1) == algo
assert int(fm.group(2)) == fs_level
received = get_dest_received_dir(fdst, src)
assert _tree_bytes(received) == _tree_bytes(src)
@requires_rsync
@pytest.mark.ci
@pytest.mark.parametrize("env,algo", [("md5", "md5"), ("sha1", "sha1"), ("xxh3 md5", "xxh3")])
def test_checksum_list_matches_rsync(self, env, algo, shared_server, monkeypatch):
monkeypatch.setenv("RSYNC_CHECKSUM_LIST", env)
src = _make_corpus(_scratch(f"envcc_src_{algo}"))
rdst = _scratch(f"envcc_rsync_{algo}")
rs = _rsync(["-a", "--checksum", "--out-format=%C %n", src + "/", rdst + "/"])
assert rs.returncode == 0, rs.stderr
rs_digests = _digests(rs.stdout)
fdst = _scratch(f"envcc_fs_{algo}")
result, _ = run_client(src, fdst, flags=["-a", "--checksum", "--out-format=%C %n"],
port=shared_server.port)
assert result.returncode == 0, (result.stderr or result.stdout)[:300]
assert _digests(result.stdout) == rs_digests
@requires_rsync
@pytest.mark.ci
def test_all_unknown_lists_fail_like_rsync(self, shared_server, monkeypatch):
src = _make_corpus(_scratch("envbad_src"))
monkeypatch.setenv("RSYNC_COMPRESS_LIST", "bogus")
rs = _rsync(["-a", "-z", src + "/", _scratch("envbad_rsync_c") + "/"])
assert rs.returncode == 4, rs.stderr
result, _ = run_client(src, _scratch("envbad_fs_c"), flags=["-a", "-z"],
port=shared_server.port)
assert result.returncode == 4, (result.stderr or result.stdout)[:200]
monkeypatch.setenv("RSYNC_CHECKSUM_LIST", "bogus")
rs = _rsync(["-a", src + "/", _scratch("envbad_rsync_s") + "/"])
assert rs.returncode == 4, rs.stderr
result, _ = run_client(src, _scratch("envbad_fs_s"), flags=["-a"],
port=shared_server.port)
assert result.returncode == 4, (result.stderr or result.stdout)[:200]
def _tree_bytes(root):
out = {}
for dirpath, _dirs, files in os.walk(root):
for name in files:
path = os.path.join(dirpath, name)
with open(path, "rb") as fh:
out[os.path.relpath(path, root)] = fh.read()
return out
def _digests(output):
out = {}
for line in output.splitlines():
parts = line.split()
if len(parts) == 2 and parts[0]:
out[parts[1]] = parts[0]
return out
+6
View File
@@ -43,6 +43,7 @@
#include "test_utils.h"
#include "test_xattr.h"
#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
// Define global test state variables
@@ -52,6 +53,11 @@ bool current_test_failed = false;
int main() {
signal(SIGPIPE, SIG_IGN);
/* The codec/checksum resolvers consult rsync's preference-list environment
* variables; clear them so a developer's shell cannot change test outcomes.
* The env-specific tests set and restore their own values. */
unsetenv("RSYNC_COMPRESS_LIST");
unsetenv("RSYNC_CHECKSUM_LIST");
printf("\033[1;36m=== RUNNING UNIT TESTS ===\033[0m\n\n");
RUN_TEST(test_queue);
+23
View File
@@ -265,6 +265,28 @@ static void test_checksum_digest_file_matches_oneshot(void) {
free(data);
}
/* RSYNC_CHECKSUM_LIST precedence, syntax and fallback. */
static void test_checksum_choice_env_list() {
unsetenv("RSYNC_CHECKSUM_LIST");
EXPECT_EQ_INT(checksum_choice_resolve(), (int)CHECKSUM_ALGO_XXH128);
EXPECT_EQ_INT((int)checksum_negotiate_default(), (int)CHECKSUM_ALGO_XXH128);
setenv("RSYNC_CHECKSUM_LIST", "bogus md5 xxh3", 1);
EXPECT_EQ_INT(checksum_choice_resolve(), (int)CHECKSUM_ALGO_MD5);
setenv("RSYNC_CHECKSUM_LIST", "SHA1", 1);
EXPECT_EQ_INT(checksum_choice_resolve(), (int)CHECKSUM_ALGO_SHA1);
/* Whitespace-separated only: comma is not a separator in rsync's syntax. */
setenv("RSYNC_CHECKSUM_LIST", "md5,xxh3", 1);
EXPECT_EQ_INT(checksum_choice_resolve(), -1);
setenv("RSYNC_CHECKSUM_LIST", " ", 1);
EXPECT_EQ_INT(checksum_choice_resolve(), (int)CHECKSUM_ALGO_XXH128);
unsetenv("RSYNC_CHECKSUM_LIST");
}
void test_checksum(void) {
test_checksum_xxh64_seed0();
test_checksum_xxh64_empty();
@@ -281,4 +303,5 @@ void test_checksum(void) {
test_checksum_truncated_buffer_rejected();
test_checksum_null_empty_digest();
test_checksum_digest_file_matches_oneshot();
test_checksum_choice_env_list();
}
+116
View File
@@ -2431,6 +2431,119 @@ static void test_parse_args_rejects_invalid_compression_choice() {
config_delete(cfg);
}
/* rsync gives each codec its own default --compress-level; an omitted level
* resolves to that default and an explicit one is clamped to the codec range. */
static void test_parse_args_per_codec_compression_level_defaults() {
unsetenv("RSYNC_COMPRESS_LIST");
struct {
const char* choice;
int level;
} cases[] = {
{"zstd", 3},
{"zlib", 6},
{"zlibx", 6},
{"lz4", 1},
};
for (size_t i = 0; i < sizeof(cases) / sizeof(cases[0]); i++) {
Config* cfg = config_create();
char* argv[] = {"fastsync", "-z", "--compress-choice", (char*)cases[i].choice, "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 6, argv, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->use_compression);
EXPECT_EQ_INT(cfg->compression_level, cases[i].level);
config_delete(cfg);
}
/* Bare -z resolves to the zstd default. */
Config* cfg = config_create();
char* bare[] = {"fastsync", "-z", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, bare, positional_args, &positional_count), 0);
EXPECT_EQ_INT(cfg->compression_level, 3);
config_delete(cfg);
/* An explicit level wins unchanged for zstd... */
cfg = config_create();
char* zv[] = {"fastsync", "-z", "--compress-level", "10", "/src", "/dst"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, zv, positional_args, &positional_count), 0);
EXPECT_EQ_INT(cfg->compression_level, 10);
config_delete(cfg);
/* ...but zlib clamps an over-range level to 9 like rsync. */
cfg = config_create();
char* zc[] = {"fastsync", "-z", "--compress-choice", "zlib", "--compress-level", "15",
"/src", "/dst"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 8, zc, positional_args, &positional_count), 0);
EXPECT_EQ_INT(cfg->compression_level, 9);
config_delete(cfg);
}
/* RSYNC_COMPRESS_LIST drives the bare -z ("auto") resolution. */
static void test_parse_args_compression_env_list() {
Config* cfg = config_create();
char* argv[] = {"fastsync", "-z", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
setenv("RSYNC_COMPRESS_LIST", "zlib lz4", 1);
EXPECT_EQ_INT(parse_args(cfg, 4, argv, positional_args, &positional_count), 0);
EXPECT_EQ_INT(cfg->compression_algo, (int)COMPRESSION_ALGO_ZLIB);
EXPECT_EQ_INT(cfg->compression_level, 6);
config_delete(cfg);
/* An explicit --compress-choice beats the env list. */
cfg = config_create();
char* explicit_argv[] = {"fastsync", "-z", "--compress-choice", "zstd", "/src", "/dst"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 6, explicit_argv, positional_args, &positional_count), 0);
EXPECT_EQ_INT(cfg->compression_algo, (int)COMPRESSION_ALGO_ZSTD);
config_delete(cfg);
/* A list with no supported name is rsync's failed negotiation (exit 4). */
setenv("RSYNC_COMPRESS_LIST", "bogus", 1);
cfg = config_create();
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv, positional_args, &positional_count), -1);
EXPECT_EQ_INT(cfg->cli_exit_code, 4);
config_delete(cfg);
unsetenv("RSYNC_COMPRESS_LIST");
}
/* RSYNC_CHECKSUM_LIST drives the default checksum choice. */
static void test_parse_args_checksum_env_list() {
Config* cfg = config_create();
char* argv[] = {"fastsync", "--checksum", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
setenv("RSYNC_CHECKSUM_LIST", "md5", 1);
EXPECT_EQ_INT(parse_args(cfg, 4, argv, positional_args, &positional_count), 0);
EXPECT_EQ_INT(cfg->checksum_algo, (int)CHECKSUM_ALGO_MD5);
EXPECT_EQ_INT(cfg->checksum_transfer_algo, (int)CHECKSUM_ALGO_MD5);
config_delete(cfg);
/* An explicit --cc wins. */
cfg = config_create();
char* cc_argv[] = {"fastsync", "--checksum", "--cc=sha1", "/src", "/dst"};
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, cc_argv, positional_args, &positional_count), 0);
EXPECT_EQ_INT(cfg->checksum_algo, (int)CHECKSUM_ALGO_SHA1);
config_delete(cfg);
/* A list with no supported name is rsync's failed negotiation (exit 4). */
setenv("RSYNC_CHECKSUM_LIST", "bogus", 1);
cfg = config_create();
positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv, positional_args, &positional_count), -1);
EXPECT_EQ_INT(cfg->cli_exit_code, 4);
config_delete(cfg);
unsetenv("RSYNC_CHECKSUM_LIST");
}
/* Every value-taking table option accepts an inline "--opt=value" form. */
static void test_parse_args_table_equals_size_options() {
Config* cfg = config_create();
@@ -4682,6 +4795,9 @@ void test_client_cli() {
test_parse_args_compression_alias_equals();
test_parse_args_rejects_invalid_compression_level_equals();
test_parse_args_rejects_invalid_compression_choice();
test_parse_args_per_codec_compression_level_defaults();
test_parse_args_compression_env_list();
test_parse_args_checksum_env_list();
test_parse_args_table_equals_size_options();
test_parse_args_table_equals_string_and_int_options();
test_parse_args_missing_argument_diagnostic();
+53
View File
@@ -4,6 +4,7 @@
#include "data.h"
#include "file.h"
#include "utils.h"
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/wait.h>
@@ -390,6 +391,56 @@ static void test_codec_name_mapping() {
EXPECT_TRUE(compression_algo_enabled(COMPRESSION_ALGO_ZSTD));
}
/* rsync 3.4.1's per-codec default levels and its clamping ranges. */
static void test_codec_level_defaults_and_clamp() {
EXPECT_EQ_INT(compression_default_level(COMPRESSION_ALGO_ZSTD), ZSTD_CLEVEL_DEFAULT);
EXPECT_EQ_INT(compression_default_level(COMPRESSION_ALGO_ZSTD), 3);
EXPECT_EQ_INT(compression_default_level(COMPRESSION_ALGO_ZLIB), 6);
EXPECT_EQ_INT(compression_default_level(COMPRESSION_ALGO_ZLIBX), 6);
/* lz4 has no tunable level; a positive placeholder keeps the codec engaged. */
EXPECT_TRUE(compression_default_level(COMPRESSION_ALGO_LZ4) > 0);
EXPECT_EQ_INT(compression_default_level(COMPRESSION_ALGO_NONE), 0);
EXPECT_EQ_INT(compression_clamp_level(COMPRESSION_ALGO_ZSTD, 1), 1);
EXPECT_EQ_INT(compression_clamp_level(COMPRESSION_ALGO_ZSTD, 22), 22);
EXPECT_EQ_INT(compression_clamp_level(COMPRESSION_ALGO_ZSTD, 23), 22);
EXPECT_EQ_INT(compression_clamp_level(COMPRESSION_ALGO_ZSTD, 0), 1);
EXPECT_EQ_INT(compression_clamp_level(COMPRESSION_ALGO_ZLIB, 15), 9);
EXPECT_EQ_INT(compression_clamp_level(COMPRESSION_ALGO_ZLIBX, 15), 9);
EXPECT_EQ_INT(compression_clamp_level(COMPRESSION_ALGO_ZLIB, 1), 1);
EXPECT_TRUE(compression_clamp_level(COMPRESSION_ALGO_LZ4, 20) > 0);
EXPECT_EQ_INT(compression_clamp_level(COMPRESSION_ALGO_NONE, 20), 0);
}
/* RSYNC_COMPRESS_LIST precedence, syntax and fallback. */
static void test_codec_choice_env_list() {
unsetenv("RSYNC_COMPRESS_LIST");
EXPECT_EQ_INT(compression_choice_resolve(), (int)COMPRESSION_ALGO_ZSTD);
EXPECT_EQ_INT((int)compression_negotiate_default(), (int)COMPRESSION_ALGO_ZSTD);
/* Unknown entries are skipped; the first supported wins. */
setenv("RSYNC_COMPRESS_LIST", "bogus zlib lz4", 1);
EXPECT_EQ_INT(compression_choice_resolve(), (int)COMPRESSION_ALGO_ZLIB);
/* Case-insensitive. */
setenv("RSYNC_COMPRESS_LIST", "ZSTD", 1);
EXPECT_EQ_INT(compression_choice_resolve(), (int)COMPRESSION_ALGO_ZSTD);
/* Whitespace-separated; the client half ends at '&'. */
setenv("RSYNC_COMPRESS_LIST", "lz4 zlib & zstd", 1);
EXPECT_EQ_INT(compression_choice_resolve(), (int)COMPRESSION_ALGO_LZ4);
/* Blank falls back to the compiled-in order. */
setenv("RSYNC_COMPRESS_LIST", " ", 1);
EXPECT_EQ_INT(compression_choice_resolve(), (int)COMPRESSION_ALGO_ZSTD);
/* rsync's syntax has no comma/colon separator: this is one unknown name. */
setenv("RSYNC_COMPRESS_LIST", "bogus,lz4", 1);
EXPECT_EQ_INT(compression_choice_resolve(), -1);
unsetenv("RSYNC_COMPRESS_LIST");
}
/* The process-global codec selects what the legacy wrappers produce. */
static void test_codec_global_selection() {
Data* original = data_create_empty(64);
@@ -423,5 +474,7 @@ void test_compression() {
test_chunk_compress_decompress_roundtrip();
test_codec_roundtrips();
test_codec_name_mapping();
test_codec_level_defaults_and_clamp();
test_codec_choice_env_list();
test_codec_global_selection();
}
+37
View File
@@ -10,6 +10,7 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <strings.h>
#include <sys/socket.h>
#include <sys/stat.h>
#include <threads.h>
@@ -578,7 +579,43 @@ static void test_getdelim_bounded() {
fclose(fp);
}
static int test_env_resolver(const char* name) {
if (strcasecmp(name, "alpha") == 0)
return 10;
if (strcasecmp(name, "beta") == 0)
return 20;
return -1;
}
static void test_env_choice_first_parsing() {
const char* var = "FASTSYNC_TEST_CHOICE_LIST";
bool specified = true;
unsetenv(var);
EXPECT_EQ_INT(env_choice_first(var, test_env_resolver, &specified), -1);
EXPECT_FALSE(specified);
/* Unknown entries are skipped, case-insensitive, first supported wins. */
setenv(var, "bogus BETA alpha", 1);
EXPECT_EQ_INT(env_choice_first(var, test_env_resolver, &specified), 20);
EXPECT_TRUE(specified);
/* The client half ends at '&'. */
setenv(var, "alpha & beta", 1);
EXPECT_EQ_INT(env_choice_first(var, test_env_resolver, &specified), 10);
/* Blank means "unspecified"; all-unknown means "specified but no match". */
setenv(var, " ", 1);
EXPECT_EQ_INT(env_choice_first(var, test_env_resolver, &specified), -1);
EXPECT_FALSE(specified);
setenv(var, "nope,alpha", 1);
EXPECT_EQ_INT(env_choice_first(var, test_env_resolver, &specified), -1);
EXPECT_TRUE(specified);
unsetenv(var);
}
void test_shared_utils() {
test_env_choice_first_parsing();
test_path_index_bounded();
test_path_index_semantics();
test_getdelim_bounded();