Commit Graph

41 Commits

Author SHA1 Message Date
TapTap 348e72e706 Merge branch 'fix/rsync-parity-all' into integration/all-fixes
CI / lint (pull_request) Failing after 3s
CI / build-and-test (pull_request) Has been skipped
CI / sanitizers (address) (pull_request) Has been skipped
CI / sanitizers (undefined) (pull_request) Has been skipped
CI / fuzz-build (pull_request) Has been skipped
CI / coverage (pull_request) Has been skipped
CI / valgrind (pull_request) Has been skipped
2026-07-29 19:28:47 +02:00
TapTap 016c4030c5 feat: add rsync parity flags - symlink, metadata, output modes, delete options, file selection, connection options (#120, #121, #122, #127, #128, #129, #130) 2026-07-29 19:24:42 +02:00
TapTap 4f383cc5a5 fix: resolve all bug issues (#158, #163, #164, #165, #166, #167, #168, #169) 2026-07-29 19:10:22 +02:00
TapTap 52d5492dee fix: restore follow_symlinks and partial fields lost in merge, register ssh tests
CI / lint (pull_request) Successful in 11s
CI / sanitizers (address) (pull_request) Successful in 16s
CI / sanitizers (undefined) (pull_request) Successful in 16s
CI / coverage (pull_request) Successful in 10s
CI / fuzz-build (pull_request) Successful in 13s
CI / valgrind (pull_request) Successful in 12s
CI / build-and-test (pull_request) Successful in 55s
2026-07-29 18:19:50 +02:00
TapTap e68e33c0c5 Apply PR #143 content on top of latest main 2026-07-29 18:18:07 +02:00
TapTap d3dca6c2a5 Revert "Merge pull request 'Merge all 5 batch PRs: security, CLI features, protocol, performance, tests/docs' (#143) from merge-all-v2 into main"
CI / lint (pull_request) Successful in 9s
CI / sanitizers (address) (pull_request) Successful in 15s
CI / sanitizers (undefined) (pull_request) Successful in 15s
CI / coverage (pull_request) Successful in 11s
CI / fuzz-build (pull_request) Successful in 13s
CI / valgrind (pull_request) Successful in 13s
CI / build-and-test (pull_request) Successful in 54s
This reverts commit 29f4f8cde6, reversing
changes made to c6bf7bb84e.
2026-07-29 18:10:01 +02:00
TapTap c5acd13df2 Merge remote-tracking branch 'origin/main' into merge-all-v2
CI / lint (pull_request) Successful in 12s
CI / build-and-test (pull_request) Failing after 10s
CI / sanitizers (address) (pull_request) Failing after 14s
CI / sanitizers (undefined) (pull_request) Failing after 15s
CI / coverage (pull_request) Failing after 7s
CI / fuzz-build (pull_request) Failing after 15s
CI / valgrind (pull_request) Failing after 10s
# Conflicts:
#	src/client/client_cli.c
#	tests/test_transport_ssh.c
2026-07-29 18:07:03 +02:00
TapTap 30239c6f50 feat: add --fastsync-server-path flag to configure remote server binary path
CI / lint (pull_request) Successful in 10s
CI / sanitizers (address) (pull_request) Successful in 14s
CI / sanitizers (undefined) (pull_request) Successful in 16s
CI / fuzz-build (pull_request) Successful in 15s
CI / coverage (pull_request) Successful in 9s
CI / valgrind (pull_request) Successful in 12s
CI / build-and-test (pull_request) Successful in 54s
2026-07-21 18:13:21 +02:00
TapTap 081973c904 fix: security hardening (#112-#118) 2026-07-21 16:42:57 +02:00
TapTap 62f3df94ee Merge fix/refactoring into merge-all 2026-07-21 15:04:41 +02:00
TapTap 27e3ac11db fix: bump protocol version, add static_assert for metadata sizes
CI / lint (pull_request) Successful in 8s
CI / sanitizers (address) (pull_request) Successful in 14s
CI / sanitizers (undefined) (pull_request) Successful in 14s
CI / fuzz-build (pull_request) Successful in 12s
CI / coverage (pull_request) Successful in 9s
CI / build-and-test (pull_request) Successful in 55s
CI / valgrind (pull_request) Successful in 11s
2026-07-20 19:53:03 +02:00
TapTap 7ecba4e0d5 fix: adapt tests and fix bugs from enhancements rebase onto main
CI / lint (pull_request) Failing after 3s
CI / build-and-test (pull_request) Has been skipped
CI / sanitizers (address) (pull_request) Has been skipped
CI / sanitizers (undefined) (pull_request) Has been skipped
CI / fuzz-build (pull_request) Has been skipped
CI / coverage (pull_request) Has been skipped
CI / valgrind (pull_request) Has been skipped
- Fix transport_tcp.c: initialize server->ssl_ctx to NULL
  (prevents SSL_CTX_free on garbage when server_create_tls fails)
- Fix test_transport_tcp.c: client_create now sets fd=-1, family=AF_UNSPEC
- Fix test_transport_tcp.c: server address family may be AF_INET or AF_INET6
- Fix test_file_sendfile.c: send_path=false protocol includes file_type prefix
2026-07-20 19:42:53 +02:00
TapTap e3484939b6 Revert "Merge pull request 'Features and enhancements (#70, #36, #34, #33, #32, #57, #40, #37)' (#84) from fix/enhancements into main"
This reverts commit 8f25f6a6b6, reversing
changes made to 12ca4b13c8.
2026-07-20 19:36:09 +02:00
TapTap 2dd40d6a5b fix: features and enhancements — issues #70, #36, #34, #33, #32, #57, #40, #37
CI / lint (pull_request) Failing after 3s
CI / build-and-test (pull_request) Has been skipped
CI / sanitizers (address) (pull_request) Has been skipped
CI / sanitizers (undefined) (pull_request) Has been skipped
CI / fuzz-build (pull_request) Has been skipped
CI / coverage (pull_request) Has been skipped
CI / valgrind (pull_request) Has been skipped
2026-07-20 19:28:17 +02:00
TapTap 6a5a61b59f fix: address all remaining review warnings and style issues
CI / lint (push) Successful in 7s
CI / lint (pull_request) Successful in 7s
CI / build-and-test (push) Successful in 56s
CI / sanitizers (address) (push) Successful in 1m0s
CI / clang-tidy (push) Successful in 6s
CI / build-and-test (pull_request) Successful in 54s
CI / sanitizers (address) (pull_request) Successful in 59s
CI / clang-tidy (pull_request) Successful in 9s
- config.h: bump PROTOCOL_VERSION from "1.2.0" to "1.3.0" (WARNING-1)
- client_cli.c: fix realloc leak on exclude/include patterns (WARNING-2)
- file.c: document sendfile fallback as safety net only (WARNING-4)
- metadata.c: add warning log for utimensat failure (WARNING-5/STYLE-1)
- utils.c: fix is_dir_in_manifest false prefix match (WARNING-1)
- utils.c: fix double rmdir on recursive collapse, add errno.h (WARNING-2)
- client_send.c: guard unchecked send calls for manifest/finished (WARNING-3)
- client_send.c: send STATUS_NEXT on rc==2 without delta (WARNING-4)
2026-07-20 17:48:03 +02:00
TapTap 38be7c090a Fix 11 Gitea issues (#29, #30, #31, #35, #38, #41, #42, #43, #44, #45, #46)
CI / lint (push) Failing after 3s
CI / build-and-test (push) Has been skipped
CI / sanitizers (address) (push) Has been skipped
CI / clang-tidy (push) Has been skipped
CI / lint (pull_request) Failing after 2s
CI / build-and-test (pull_request) Has been skipped
CI / sanitizers (address) (pull_request) Has been skipped
CI / clang-tidy (pull_request) Has been skipped
#29 - compression_level now used in data_compress()
#30 - chunk_size no longer truncated to 32-bit
#31 - chmod/chown failures now logged
#35 - strtok -> strtok_r for thread safety
#38 - open_next_directory returns -1 on opendir failure
#41 - file_send_sendfile uses compression_level param
#42 - dirname() uses copy to avoid modifying input
#43 - delete_extras_walk checks manifest before rmdir
#44 - server_host/port moved into Config struct
#45 - SSH parse_remote_dest uses dynamic allocation
#46 - send_chunk refactored, reduced nesting/duplication
2026-07-20 17:09:12 +02:00
TapTap b9840b123d Merge remote-tracking branch 'origin/main' into ci/tier1-review-automation
CI / lint (push) Failing after 3s
CI / build-and-test (push) Has been skipped
CI / sanitizers (address) (push) Has been skipped
CI / lint (pull_request) Failing after 2s
CI / build-and-test (pull_request) Has been skipped
CI / sanitizers (address) (pull_request) Has been skipped
# Conflicts:
#	src/client/client_send.c
#	src/shared/config.c
#	src/shared/file.c
#	src/shared/file.h
#	src/shared/protocol.h
2026-07-19 16:58:53 +02:00
TapTap 971a070b65 fix: const-correct config_send and file_send_single_calls for cppcheck
CI / lint (push) Successful in 6s
CI / lint (pull_request) Successful in 6s
CI / build-and-test (push) Successful in 47s
CI / sanitizers (address) (push) Successful in 4s
CI / build-and-test (pull_request) Successful in 47s
CI / sanitizers (address) (pull_request) Successful in 4s
2026-07-19 16:44:01 +02:00
TapTap 23b1d6660c fix: reformat codebase and fix const-correctness for CI lint
CI / lint (push) Failing after 16s
CI / build-and-test (push) Has been skipped
CI / sanitizers (address) (push) Has been skipped
CI / sanitizers (thread) (push) Has been skipped
CI / lint (pull_request) Failing after 44s
CI / build-and-test (pull_request) Has been skipped
CI / sanitizers (address) (pull_request) Has been skipped
CI / sanitizers (thread) (pull_request) Has been skipped
- Reformat all C/H files to match .clang-format (LLVM style)
- Fix 26 cppcheck const-correctness warnings (constParameterPointer,
  constVariablePointer, constVariable)
- Update function declarations in headers to match const parameters
2026-07-19 15:57:31 +02:00
TapTap 598e4e7514 fix: address PR review - rolling adler32, extract helpers, configurable max file size
CI / build-and-test (push) Successful in 29s
CI / build-and-test (pull_request) Successful in 28s
- Fix rolling adler32: add missing -1 in s2 update formula (was producing
  wrong checksums, causing zero block matches)
- Fix file_send_sendfile signature to match typedef (add unused
  compression_level param)
- Extract receive_delta_file() from receive_incremental_check() to reduce
  nesting depth
- Extract send_file_incremental() helper in client_send.c
- Add delta_max_file_size to Config, serialized over wire
- Add --delta-max CLI flag
- Add test_large_file_delta (200KB) and extra delta_should_attempt cases
- Remove unused pos_in_block variable from delta_compute
2026-07-19 02:02:28 +02:00
TapTap e16db56580 feat: add delta transfer for incremental sync
CI / build-and-test (push) Successful in 28s
CI / build-and-test (pull_request) Successful in 28s
Implement rsync-style delta transfer using rolling checksums (Adler-32 +
xxHash32). When a file exists on both sides but has changed, only the
changed blocks are transmitted instead of the entire file.

- New status codes: STATUS_DELTA_SIGNATURE, STATUS_DELTA_DATA
- Protocol version bumped to 1.2.0
- Server generates block signature, client computes delta
- Auto-fallback to whole-file when delta >= 70% of file size
- Works with zstd compression on delta stream
- Configurable block size (default 8KB, --delta-block flag)
- 13 unit tests covering hashing, signature roundtrip, delta compute/apply,
  file growth/shrink, and decision logic
2026-07-18 20:00:15 +02:00
TapTap 864eb1316d Fix TLS code review issues: SSL cleanup, TLS 1.2 min, CA verify, deprecation guards, server --help, port validation, __thread io_ssl, shared accept loop 2026-07-18 17:39:37 +02:00
TapTap eff50346fd TLS transport: OpenSSL-based encrypted TCP
- New transport_tls.h/c: TLS server (server_create_tls, server_listen_tls)
  and client (client_connect_tls) using OpenSSL
- protocol.c: io_set_ssl() + SSL_read/SSL_write in send_n_data/receive_n_data
- transport_tcp.h: ssl/ssl_ctx fields added to Server/Client structs
- config.h/c: use_tls, tls_cert, tls_key fields
- client_cli.c: --tls, --cert, --key flags
- server.c: --tls, --cert, --key, -p flags with TLS support
- CMakeLists.txt: OpenSSL::SSL + OpenSSL::Crypto linkage
- shell.nix: openssl added to buildInputs
2026-07-18 17:38:55 +02:00
TapTap f1af88a25c Fix incremental-sync review: sendfile_no_path, reject -s+incremental, no data mutation, auto -M, STATUS_ERROR handling, PROTOCOL_VERSION bump 2026-07-18 17:19:12 +02:00
TapTap 1905369c9f Incremental sync: --incremental flag to skip unchanged files
- New STATUS_CHECK protocol status (value 6)
- Client sends path + size + mtime; server replies OK (skip) or NEXT (send)
- config.h/c: use_incremental field sent/received over wire
- file.c/h: file_send_single_calls_no_path helper for incremental path
- client_cli.c: --incremental flag
- client_send.c: per-file check before send in both single and chunk paths
- server.c: STATUS_CHECK handling in receive_files()
- multiprocessing.c: STATUS_CHECK handling in receive_thread()
- test.py: incremental sync test case
2026-07-18 17:19:12 +02:00
TapTap a62894be05 Merge main into server-improvements: resolve conflicts (protocol signatures, error returns) 2026-07-16 15:08:56 +02:00
TapTap bbdf027be8 Fix exit() in library code, add --include/--max-size/--min-size 2026-07-16 14:15:38 +02:00
TapTap efb98d8964 Multi-client server, protocol versioning, streaming decompression fixes 2026-07-16 13:59:41 +02:00
TapTap 284bf8f8bc rsync flags: -a, -n, -p, --exclude, --delete
Feature changes across 15 files:

- -a/--archive: enables -c -m -M (no -s, which slows transfers)
- -n/--dry-run: scan + print without connecting or transferring
- -p <port>: custom SSH port (passed as -p to ssh via execvp)
- --exclude <pattern>: glob-based filename filtering in scanner
- --delete: sender collects file manifest; receiver deletes unlisted files

Protocol: added STATUS_MANIFEST, use_delete field in Config wire format.
New utilities: glob_match(), delete_extras() with recursive directory walk.
Scanner: accepts exclude patterns; skips matching entries.
SSH transport: switched from execlp to execvp for dynamic port arg.
Server + multiprocessing: handle STATUS_MANIFEST in both single and
multithreaded receive paths.

Builds clean, all 7 tests pass.
2026-07-16 12:39:05 +02:00
TapTap d1a23fc4df quick-wins: ControlMaster, -z alias, --progress, fix write warning
- SSH ControlMaster auto with ControlPath for connection reuse
- -z as alias for -c (compression)
- --progress flag showing transfer rate in send_files
- Fixed unused-result warning on write() in transport_ssh.c
2026-07-16 12:28:16 +02:00
TapTap ca29109664 perf: downsized test data, increased SSH socket buffer, and made chunk size configurable
- Reduced test data from 50MB to 25MB (test.py)
- Increased SSH socketpair buffer to 1MB via setsockopt (transport_ssh.c)
- Made chunk size configurable: new Config field, --chunk-size CLI flag,
  sent over wire, and used in scanner instead of DESIRED_CHUNK_SIZE
- Fixed DESIRED_CHUNK_SIZE macro parentheses (chunk.h)
2026-07-16 12:20:02 +02:00
TapTap 8f4ea62ae7 Phase 1+2: bug fixes and dead code cleanup
Phase 1 — Bugs:
- receive_data: fix unsigned long long vs size_t mismatch
- chunk_deserialize: add NULL check in multiprocessing.c and server.c
- file_content_to_buffer: add missing fclose on error path
- send_files_multithreaded: propagate sender thread result
- server: add SIGPIPE handler
- to_disk: check fwrite return value
- scanner: use S_ISDIR instead of !S_ISREG
- scanner: free cur_path before early return

Phase 2 — Cleanup:
- Remove unused chunk_decompress function
- Remove unused array_list_clear function
- Remove unused num_connections config field
- Remove duplicate FILE_METADATA_WIRE_SIZE macro
- Update tests for removed APIs
2026-07-16 12:08:09 +02:00
TapTap b3f69208ce cleanup: remove dead code, fix bugs, refactor SSH dest parsing
- Remove debug printf calls from file.c
- Remove dead code: chunk_print, chunk_format, chunk_data_create/delete
- Remove unused config.use_single_send_per_file field
- Fix server_delete() no-op (double-pointer)
- Fix double cast (long)(long) -> ptrdiff_t in chunk.c
- Fix thread return value: thrd_error instead of 1
- Remove unused config param from receive_chunk_enqueue()
- Make queue_double_capacity() static
- Move SSH dest parsing into config.c as config_parse_ssh_dest()
- Fix test_config_ssh_dest to test parsing round-trip
- Remove chunk_format test (obsolete format)
- Replace chunk_data_delete with data_destroy in tests
2026-07-07 19:07:24 +02:00
TapTap fbd7ccf4dd feat: add SSH transport, rsync-style CLI, and --stdio server mode
- Replace send()/recv() with read()/write() + io_set_fds() for fd abstraction
- Add client_connect_ssh() via socketpair + fork/exec; reap child in client_disconnect()
- Add server --stdio flag for SSH invocation
- Replace handle_arg() with rsync-style positional args (fastsync <src> <dest>)
- Add SSH vs TCP auto-detection via is_remote_dest()
- Add TransportType and ssh_destination to Config, fix uninitialized fields in
  config_receive() (pre-existing multithreaded crash)
- Sender threads now wait for server STATUS_OK before disconnecting
- Add --help, error handling for sendfile+SSH combos
- Add pre-flight checks and Posix Args test case to test.py
- Add test_config_ssh_dest() unit test
2026-07-06 18:42:33 +02:00
TapTap 660834d453 Merge branch 'main' into metadata-transfer, resolve conflicts
Conflicts resolved:
- config_create signature: added both use_metadata and use_sendfile params
- config_send/config_receive: wire protocol includes both use_metadata and use_sendfile
- client.c: config_create call updated, arg parsing includes both -M/--preserve and -f/--sendfile
- test_config.c: all three config_create calls updated
- file_send_sendfile: use file->data->size instead of removed struct stat
2026-07-05 20:29:04 +02:00
TapTap 4a1e666365 refactor: merge File/FileReceive, add optional metadata transfer with -M flag
- Remove FileReceive struct; use File everywhere with nullable FileMetadata*
- Remove struct stat from File; file size lives in Data->size (data_create_reserve)
- Add FileMetadata struct (mode, uid, gid, mtime) sent conditionally over wire
- Add -M / --preserve flag to client
- Restore permissions, ownership, timestamps on disk write
- Wire format uses per-file present flag for metadata (zero overhead when off)
2026-07-05 20:21:46 +02:00
TapTap a48b3e3a17 refactor: pass use_sendfile to config_create 2026-07-05 16:38:13 +02:00
TapTap 01045d7d14 feat: add -f/--sendfile for zero-copy file transfer
- Add file_send_sendfile() using sendfile() syscall to send file
  content directly from fd to socket, bypassing userspace memory
- Add use_sendfile field to Config struct (default false)
- Add -f / --sendfile flag parsing in client.c
- sendfile path in send_chunk() used when -f is set without -c or -s
- send_files_multithreaded() falls back to single-threaded when
  sendfile is enabled (loader pipeline becomes unnecessary)
- Add Sendfile (-f) test case to test.py
2026-07-05 16:25:20 +02:00
Theo Tappe 75d5f91c6a add data compression and decompression 2026-06-14 16:52:07 +00:00
Theo Tappe f3e93d3fd9 improved testing capabilities and fixed bug that pre ended chunk sending 2026-06-10 22:11:12 +02:00
Theo Tappe 1e5eeb704f First Commit 2026-06-10 16:58:35 +02:00