From 405e5b3b0050a5490f98b624d7e04bce78ccb5cc Mon Sep 17 00:00:00 2001 From: TapTap Date: Tue, 1 Sep 2026 21:15:11 +0200 Subject: [PATCH 1/3] docs: document missing rsync flags --- RSYNC_COMPAT.md | 37 +++++++++++++++++++++++++------------ 1 file changed, 25 insertions(+), 12 deletions(-) diff --git a/RSYNC_COMPAT.md b/RSYNC_COMPAT.md index 4b236ec..72d1c44 100644 --- a/RSYNC_COMPAT.md +++ b/RSYNC_COMPAT.md @@ -8,9 +8,9 @@ This document maps rsync's full feature set to FastSync's current implementation |--------|-------|-------------| | ✅ Implemented | 34 | Feature works end-to-end | | 🔀 Alt Arg | 3 | Functionality exists but under different flag/semantics | -| ⚠️ Partial | 1 | Flag parsed/stored but behavior incomplete | -| ❌ Not Implemented | 98 | Flag not recognized or no behavior | -| **Total** | **136** | | +| ⚠️ Partial | 2 | Flag parsed/stored but behavior incomplete | +| ❌ Not Implemented | 111 | Flag not recognized or no behavior | +| **Total** | **150** | | --- @@ -30,6 +30,7 @@ This document maps rsync's full feature set to FastSync's current implementation | `--exclude=PATTERN` | Exclude files matching pattern | ✅ Implemented | Glob matching in scanner | | `--include=PATTERN` | Include files matching pattern | ✅ Implemented | Glob matching in scanner | | `-C`, `--cvs-exclude` | Auto-ignore CVS files | ❌ Not Implemented | Removed because it had no effect | +| `--no-OPTION` | Turn off an implied option | ❌ Not Implemented | Rsync supports this for options such as `--no-D`; FastSync does not | ## 2. Modifying Output @@ -43,7 +44,7 @@ This document maps rsync's full feature set to FastSync's current implementation | `--out-format=FORMAT` | Custom output format | ❌ Not Implemented | Removed because it had no effect | | `--log-file=FILE` | Log to file | ✅ Implemented | `log_file` config field | | `--log-file-format=FMT` | Log format | ❌ Not Implemented | | -| `--8-bit-output` | Leave high-bit chars unescaped | ❌ Not Implemented | | +| `-8`, `--8-bit-output` | Leave high-bit chars unescaped | ❌ Not Implemented | | | `--list-only` | List files instead of copying | ❌ Not Implemented | Removed because it had no effect | ## 3. File Selection @@ -52,7 +53,8 @@ This document maps rsync's full feature set to FastSync's current implementation |------|-------------------|-----------------|-------| | `--exclude-from=FILE` | Read exclude patterns from file | ✅ Implemented | Reads patterns from file | | `--include-from=FILE` | Read include patterns from file | ✅ Implemented | Reads patterns from file | -| `--filter=RULE` | Add file-filtering rule | ❌ Not Implemented | Removed because it had no effect | +| `-f`, `--filter=RULE` | Add a file-filtering rule | ❌ Not Implemented | `-f` is FastSync's sendfile flag | +| `-F` | Add the default `.rsync-filter` rules | ❌ Not Implemented | | | `--files-from=FILE` | Read source file list from file | ❌ Not Implemented | Removed because it had no effect | | `-0`, `--from0` | Delimit *-from files with NULs | ❌ Not Implemented | | | `--max-size=SIZE` | Skip files larger than SIZE | ✅ Implemented | `max_size` in scanner | @@ -63,6 +65,7 @@ This document maps rsync's full feature set to FastSync's current implementation | `--existing` | Skip creating new files on receiver | ❌ Not Implemented | | | `--ignore-existing` | Skip updating existing files | ❌ Not Implemented | | | `--remove-source-files` | Sender removes synced files | ❌ Not Implemented | | +| `-x`, `--one-file-system` | Do not cross filesystem boundaries | ❌ Not Implemented | | ## 4. Directory Options @@ -72,6 +75,7 @@ This document maps rsync's full feature set to FastSync's current implementation | `-R`, `--relative` | Use relative path names | ❌ Not Implemented | Removed because it had no effect | | `--no-implied-dirs` | Don't send implied dirs with -R | ❌ Not Implemented | | | `-d`, `--dirs` | Transfer dirs without recursing | ❌ Not Implemented | | +| `--old-dirs`, `--old-d` | Transfer directories without recursing | ❌ Not Implemented | Compatibility aliases for `--dirs` | | `--mkpath` | Create missing path components | ❌ Not Implemented | | ## 5. Transfer Modifications @@ -83,7 +87,7 @@ This document maps rsync's full feature set to FastSync's current implementation | `--append` | Append data to shorter files | ❌ Not Implemented | Removed because it had no effect | | `--append-verify` | Append with old-data checksum | ❌ Not Implemented | Removed because it had no effect | | `-W`, `--whole-file` | Copy whole file (no delta) | ❌ Not Implemented | | -| `--block-size=SIZE` | Force checksum block-size | ⚠️ Partial | Parsed as `--delta-block`; controls delta transfer block size | +| `-B`, `--block-size=SIZE` | Force checksum block-size | ⚠️ Partial | Parsed as `--delta-block`; controls delta transfer block size | ## 6. Destination Handling @@ -94,12 +98,14 @@ This document maps rsync's full feature set to FastSync's current implementation | `--backup-dir=DIR` | Backup directory hierarchy | ✅ Implemented | `backup_dir` config field | | `--suffix=SUFFIX` | Backup suffix (default ~) | ✅ Implemented | `suffix` config field | | `--delay-updates` | Put updated files in place at end | ❌ Not Implemented | | +| `-T`, `--temp-dir=DIR` | Create temporary files in DIR | ❌ Not Implemented | `-T` is FastSync's timeout alias | ## 7. Deletion | Flag | Rsync Description | FastSync Status | Notes | |------|-------------------|-----------------|-------| | `--delete` | Delete extraneous files from dest | ✅ Implemented | `use_delete` config field | +| `--del` | Delete during transfer | ❌ Not Implemented | Alias for `--delete-during` | | `--delete-before` | Delete before transfer | ❌ Not Implemented | Removed because it had no effect | | `--delete-during` | Delete during transfer | ❌ Not Implemented | | | `--delete-delay` | Find deletions during, delete after | ❌ Not Implemented | | @@ -108,7 +114,7 @@ This document maps rsync's full feature set to FastSync's current implementation | `--max-delete=NUM` | Max files to delete | ❌ Not Implemented | Removed because it had no effect | | `--ignore-errors` | Delete even with I/O errors | ❌ Not Implemented | | | `--force` | Force deletion of non-empty dirs | ❌ Not Implemented | | -| `--prune-empty-dirs` | Prune empty dir chains | ❌ Not Implemented | Removed because it had no effect | +| `-m`, `--prune-empty-dirs` | Prune empty dir chains | ❌ Not Implemented | `-m` enables FastSync multithreading instead | ## 8. Metadata Preservation @@ -135,6 +141,12 @@ This document maps rsync's full feature set to FastSync's current implementation | `-J`, `--omit-link-times` | Omit symlinks from --times | ❌ Not Implemented | | | `--super` | Receiver attempts super-user activities | ❌ Not Implemented | | | `--fake-super` | Store/recover privileged attrs via xattrs | ❌ Not Implemented | | +| `--open-noatime` | Avoid changing access time when opening files | ❌ Not Implemented | | +| `--numeric-ids` | Do not map uid/gid by name | ❌ Not Implemented | | +| `--usermap=STRING` | Map usernames | ❌ Not Implemented | | +| `--groupmap=STRING` | Map group names | ❌ Not Implemented | | +| `--chown=USER:GROUP` | Map owner and group | ❌ Not Implemented | | +| `--copy-as=USER[:GROUP]` | Perform the copy as another user/group | ❌ Not Implemented | | ## 9. Symlink Handling @@ -160,19 +172,19 @@ This document maps rsync's full feature set to FastSync's current implementation | Flag | Rsync Description | FastSync Status | Notes | |------|-------------------|-----------------|-------| | `--checksum` | Skip based on checksum | ✅ Implemented | With `--incremental`, compares xxHash64 content checksums; `-c` remains compression | -| `--checksum-choice=STR` | Choose checksum algorithm | ❌ Not Implemented | xxHash used internally | +| `--checksum-choice=STR` (`--cc`) | Choose checksum algorithm | ❌ Not Implemented | xxHash used internally | | `--compare-dest=DIR` | Compare dest files relative to DIR | ❌ Not Implemented | Removed because it had no effect | | `--copy-dest=DIR` | Include copies of unchanged files | ❌ Not Implemented | Removed because it had no effect | | `--link-dest=DIR` | Hardlink to files when unchanged | ❌ Not Implemented | Removed because it had no effect | -| `--fuzzy`, `--no-fuzzy` | Find similar file for basis | ❌ Not Implemented | | +| `-y`, `--fuzzy`, `--no-fuzzy` | Find similar file for basis | ❌ Not Implemented | `--no-fuzzy` has no short alias | ## 12. Compression | Flag | Rsync Description | FastSync Status | Notes | |------|-------------------|-----------------|-------| | `-z`, `--compress` | Compress file data | 🔀 Alt Arg | Always uses zstd (rsync supports multiple algorithms) | -| `--compress-choice=STR` | Choose compression algorithm | ❌ Not Implemented | Removed because it had no effect; FastSync always uses zstd | -| `--compress-level=NUM` | Set compression level | ✅ Implemented | 1-22, default 5 | +| `--compress-choice=STR` (`--zc`) | Choose compression algorithm | ❌ Not Implemented | Removed because it had no effect; FastSync always uses zstd | +| `--compress-level=NUM` (`--zl`) | Set compression level | ✅ Implemented | 1-22, default 5 | | `--compress-threads=NUM` | Set compression threads | ❌ Not Implemented | | | `--skip-compress=LIST` | Skip compress for suffixes | ❌ Not Implemented | Internal skip for hardcoded types; not user-configurable | @@ -189,6 +201,7 @@ This document maps rsync's full feature set to FastSync's current implementation | `--address=ADDRESS` | Bind address for outgoing socket | ❌ Not Implemented | Removed because it had no effect | | `-4`, `--ipv4` | Prefer IPv4 | ❌ Not Implemented | Removed because it had no effect | | `-6`, `--ipv6` | Prefer IPv6 | ❌ Not Implemented | Removed because it had no effect | +| `--remote-option=OPT`, `-M` | Send an option only to the remote side | ❌ Not Implemented | `-M` is FastSync's metadata-preservation flag | ## 14. Daemon Mode @@ -214,6 +227,7 @@ This document maps rsync's full feature set to FastSync's current implementation | `--old-args` | Disable modern arg protection | ❌ Not Implemented | | | `--ignore-missing-args` | Ignore missing source args | ❌ Not Implemented | | | `--delete-missing-args` | Delete missing source args | ❌ Not Implemented | | +| `--max-alloc=SIZE` | Limit a single memory allocation | ❌ Not Implemented | | ## 16. Batch Operations @@ -234,7 +248,6 @@ This document maps rsync's full feature set to FastSync's current implementation | `--iconv=CONVERT_SPEC` | Charset conversion | ❌ Not Implemented | | | `--checksum-seed=NUM` | Set checksum seed | ❌ Not Implemented | | | `-s`, `--secluded-args` | Use protocol to send args | ❌ Not Implemented | | -| `--no-OPTION` | Turn off implied option | ❌ Not Implemented | | --- From 5c055960b81fb87b9bc2f86a61cdffc4e7837ea4 Mon Sep 17 00:00:00 2001 From: TapTap Date: Tue, 1 Sep 2026 21:18:23 +0200 Subject: [PATCH 2/3] docs: correct compatibility summary --- RSYNC_COMPAT.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/RSYNC_COMPAT.md b/RSYNC_COMPAT.md index 72d1c44..76d5e0f 100644 --- a/RSYNC_COMPAT.md +++ b/RSYNC_COMPAT.md @@ -8,8 +8,8 @@ This document maps rsync's full feature set to FastSync's current implementation |--------|-------|-------------| | ✅ Implemented | 34 | Feature works end-to-end | | 🔀 Alt Arg | 3 | Functionality exists but under different flag/semantics | -| ⚠️ Partial | 2 | Flag parsed/stored but behavior incomplete | -| ❌ Not Implemented | 111 | Flag not recognized or no behavior | +| ⚠️ Partial | 3 | Flag parsed/stored but behavior incomplete | +| ❌ Not Implemented | 110 | Flag not recognized or no behavior | | **Total** | **150** | | --- From 1517b3fc356b1dc4f92350e39ca0f77644393b4a Mon Sep 17 00:00:00 2001 From: TapTap Date: Tue, 1 Sep 2026 21:25:44 +0200 Subject: [PATCH 3/3] docs: add rsync feature implementation plan --- RSYNC_COMPAT.md | 102 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 102 insertions(+) diff --git a/RSYNC_COMPAT.md b/RSYNC_COMPAT.md index 76d5e0f..f1424fc 100644 --- a/RSYNC_COMPAT.md +++ b/RSYNC_COMPAT.md @@ -251,6 +251,108 @@ This document maps rsync's full feature set to FastSync's current implementation --- +## Implementation Difficulty Plan + +The estimates below cover the currently unimplemented features in this document. They assume one engineer familiar with the codebase, include implementation and focused tests, and exclude production rollout time. A feature should not be marked implemented until its behavior is tested in both local and SSH/TCP paths where applicable. + +| Effort | Typical duration | Meaning | +|--------|------------------|---------| +| XS | 0.5-1 day | CLI alias or a local formatting/validation change | +| S | 1-3 days | Isolated behavior with little or no protocol change | +| M | 3-7 days | Cross-cutting client, server, or scanner behavior | +| L | 1-3 weeks | Protocol, filesystem, privilege, or compatibility work | +| XL | 3+ weeks | New transfer mode, daemon subsystem, or broad interoperability effort | + +### Phase 1: Low-Risk CLI and Local Behavior + +These are the best first changes because they require limited wire-format work and can be tested with existing transfer fixtures. + +| Features | Effort | Implementation plan | +|----------|--------|--------------------| +| `--quiet`, `-q`; `--human-readable`, `-h`; `--8-bit-output`, `-8`; `--stderr=MODE`; `--info=FLAGS`; `--debug=FLAGS` | S | Extend logging and output formatting without changing transferred data. | +| `--no-OPTION`; `--old-args`; `--secluded-args`, `-s` | M | Add option implication/negation and safely serialize or protect remote arguments. `-s` currently has FastSync-specific semantics and needs a compatibility decision. | +| `-P`; `--del`; `--old-dirs`, `--old-d`; `--cc`; `--zc`; `--zl` | XS | Add aliases and composed behaviors after the underlying options exist. | +| `--whole-file`, `-W`; `--ignore-times`, `-I`; `--size-only`; `--modify-window`, `-@`; `--update`, `-u` | S | Extend the existing incremental comparison decision. | +| `--existing`; `--ignore-existing`; `--remove-source-files` | S | Add scanner/receiver eligibility checks and remove successfully synchronized source files. | +| `--executability`, `-E`; `--chmod=CHMOD` | M | Apply permission transformations safely while preserving current metadata behavior. | +| `--skip-compress=LIST`; `--compress-threads=NUM` | S | Make compression selection configurable and validate the thread setting against zstd behavior. | +| `--max-alloc=SIZE`; `--fsync` | S | Reuse existing allocation limits and add an explicit durability step after file writes. | + +### Phase 2: Filesystem Selection and Update Semantics + +These features are moderate because they affect traversal, temporary files, manifests, or the receiver's update policy. + +| Features | Effort | Implementation plan | +|----------|--------|--------------------| +| `--one-file-system`, `-x` | M | Track the source device during scanner traversal and skip mount-point crossings. | +| `--relative`, `-R`; `--no-implied-dirs`; `--dirs`, `-d`; `--mkpath` | M | Extend path-list construction and destination directory creation while preserving traversal safety. | +| `--temp-dir`, `-T` | M | Separate temporary-file placement from FastSync's timeout alias and define collision, permissions, and cleanup rules. | +| `--delay-updates` | L | Stage all successful updates and publish them at completion, including crash and cancellation cleanup. | +| `--files-from=FILE`; `--from0`, `-0`; `--filter=RULE`, `-f`; `-F`; `--cvs-exclude`, `-C` | L | Build a complete filter/parser layer and integrate it with scanner pruning, manifests, and delete behavior. `-f` conflicts with FastSync sendfile mode. | +| `--list-only`; `--itemize-changes`, `-i`; `--out-format=FORMAT`; `--log-file-format=FMT` | M | Add a structured change-event model so output modes share one source of truth. | + +### Phase 3: Deletion, Comparison, and Delta Compatibility + +These features require careful interaction with manifests, incremental checks, backups, and the existing delta protocol. + +| Features | Effort | Implementation plan | +|----------|--------|--------------------| +| `--delete-during`; `--delete-before`; `--delete-after`; `--delete-delay`; `--del` | L | Add deletion timing to the transfer state machine and ensure failures cannot remove files unexpectedly. | +| `--delete-excluded`; `--max-delete=NUM`; `--ignore-errors`; `--force`; `--prune-empty-dirs`, `-m` | M | Extend delete walks with policy limits, error handling, empty-directory pruning, and the `-m` short-flag conflict. | +| `--ignore-missing-args`; `--delete-missing-args` | M | Distinguish missing source arguments from traversal errors and apply explicit deletion policy. | +| `--compare-dest=DIR`; `--copy-dest=DIR`; `--link-dest=DIR` | L | Add alternate basis roots and hard-link handling, including metadata and cross-filesystem failures. | +| `--fuzzy`, `-y`; `--no-fuzzy` | L | Index candidate files and select a safe similar basis without making transfer time unbounded. | +| `--append`; `--append-verify` | M | Negotiate file length and verify the retained prefix before resuming. | +| `--checksum-choice=STR`, `--cc`; `--checksum-seed=NUM` | M | Negotiate checksum algorithms/seeds and preserve compatibility with existing xxHash checks. | + +### Phase 4: Metadata, Links, and Devices + +These features are platform-sensitive and need Linux permission, ACL, xattr, and special-file integration tests. + +| Features | Effort | Implementation plan | +|----------|--------|--------------------| +| `--numeric-ids`; `--usermap=STRING`; `--groupmap=STRING`; `--chown=USER:GROUP` | L | Define identity mapping, privilege failures, and wire representation before applying ownership. | +| `--open-noatime`; `--atimes`, `-U`; `--crtimes`, `-N`; `--omit-dir-times`, `-O`; `--omit-link-times`, `-J` | L | Extend metadata capture/apply with platform capability checks and explicit unsupported-attribute handling. | +| `--acls`, `-A`; `--xattrs`, `-X`; `--fake-super` | XL | Add portable serialization, size limits, privilege behavior, and security tests for ACL/xattr data. | +| `--hard-links`, `-H` | L | Preserve inode relationships across the file list and coordinate hard-link creation order. | +| `--munge-links`; `--copy-dirlinks`, `-k`; `--keep-dirlinks`, `-K` | L | Define symlink trust boundaries and receiver-side directory/link collision behavior. | +| `--devices`; `--specials`; `-D`; `--copy-devices`; `--write-devices` | XL | Add privileged special-file handling with strict type, path, and authorization checks. | +| `--super`; `--copy-as=USER[:GROUP]` | XL | Requires a deliberate privilege model, identity switching, and refusal paths; do not implement by blindly elevating the process. | +| `--preallocate` | S | Use platform allocation APIs before writes and fall back cleanly when unsupported. | + +### Phase 5: Connectivity and Daemon Compatibility + +These options affect process startup, authentication, sockets, and remote execution. They should follow the filesystem and protocol work rather than being added as parser-only flags. + +| Features | Effort | Implementation plan | +|----------|--------|--------------------| +| `--rsh=COMMAND`, `-e`; `--rsync-path=PROGRAM`; `--blocking-io`; `--outbuf=N\|L\|B` | M | Generalize SSH command construction and subprocess I/O while retaining argument escaping and timeout guarantees. | +| `--address=ADDRESS`; `--ipv4`, `-4`; `--ipv6`, `-6`; `--sockopts=OPTIONS`; `--port=PORT` daemon semantics | M | Add explicit socket-family/bind configuration and validate it independently for TCP client and daemon modes. | +| `--remote-option=OPT`, `-M`; `--trust-sender` | L | Add authenticated remote-option/config negotiation and reject unsafe sender-controlled values. `-M` conflicts with FastSync metadata mode. | +| `--daemon`; `--config=FILE`; `--dparam=OVERRIDE`; `--no-detach`; `--password-file=FILE`; `--early-input=FILE`; `--no-motd` | XL | Implement a real daemon lifecycle, module configuration, authentication, privilege separation, and process management. | + +### Phase 6: Batch, Encoding, and Protocol Interoperability + +These are the hardest compatibility items because they require durable formats or behavior that must interoperate with rsync itself. + +| Features | Effort | Implementation plan | +|----------|--------|--------------------| +| `--write-batch=FILE`; `--only-write-batch=FILE`; `--read-batch=FILE` | XL | Specify a versioned batch format, persist all required metadata, and test replay, corruption, and partial application. | +| `--protocol=NUM` | XL | Add protocol-version negotiation and compatibility branches without weakening current validation. | +| `--iconv=CONVERT_SPEC` | L | Convert filenames at the protocol boundary with invalid-sequence and normalization tests. | +| `--stop-after=MINS`; `--stop-at=TIME` | M | Add deadline propagation, interruptible I/O, and safe checkpoint/cleanup behavior. | +| `--early-input=FILE`; `--password-file=FILE` | M | Securely read startup credentials/input with permission checks and no secret disclosure in logs. | + +### Recommended Delivery Order + +1. Resolve short-option conflicts (`-m`, `-M`, `-T`, `-f`, `-s`) and define the compatibility contract. +2. Implement Phase 1 comparison, update, output, and alias features with unit and integration coverage. +3. Implement Phase 2 traversal/filtering and Phase 3 deletion semantics. +4. Implement metadata and link features that are safe on the supported platforms. +5. Treat daemon mode, special files, batch mode, and protocol-version compatibility as separate projects. + +The existing priority list below is a feature shortlist, not an implementation schedule; this plan supersedes it for effort and sequencing. + ## Recommendations: Top Features to Implement Next Ranked by user demand, implementation complexity, and interoperability impact: