Commit Graph

28 Commits

Author SHA1 Message Date
TapTap d7b9d4dbbc fix: apply clang-format to merged code from main
CI / lint (push) Failing after 8s
CI / build-and-test (push) Has been skipped
CI / sanitizers (address) (push) Has been skipped
CI / lint (pull_request) Failing after 7s
CI / build-and-test (pull_request) Has been skipped
CI / sanitizers (address) (pull_request) Has been skipped
2026-07-19 17:04:05 +02:00
TapTap b9840b123d Merge remote-tracking branch 'origin/main' into ci/tier1-review-automation
CI / lint (push) Failing after 3s
CI / build-and-test (push) Has been skipped
CI / sanitizers (address) (push) Has been skipped
CI / lint (pull_request) Failing after 2s
CI / build-and-test (pull_request) Has been skipped
CI / sanitizers (address) (pull_request) Has been skipped
# Conflicts:
#	src/client/client_send.c
#	src/shared/config.c
#	src/shared/file.c
#	src/shared/file.h
#	src/shared/protocol.h
2026-07-19 16:58:53 +02:00
TapTap 971a070b65 fix: const-correct config_send and file_send_single_calls for cppcheck
CI / lint (push) Successful in 6s
CI / lint (pull_request) Successful in 6s
CI / build-and-test (push) Successful in 47s
CI / sanitizers (address) (push) Successful in 4s
CI / build-and-test (pull_request) Successful in 47s
CI / sanitizers (address) (pull_request) Successful in 4s
2026-07-19 16:44:01 +02:00
TapTap 23b1d6660c fix: reformat codebase and fix const-correctness for CI lint
CI / lint (push) Failing after 16s
CI / build-and-test (push) Has been skipped
CI / sanitizers (address) (push) Has been skipped
CI / sanitizers (thread) (push) Has been skipped
CI / lint (pull_request) Failing after 44s
CI / build-and-test (pull_request) Has been skipped
CI / sanitizers (address) (pull_request) Has been skipped
CI / sanitizers (thread) (pull_request) Has been skipped
- Reformat all C/H files to match .clang-format (LLVM style)
- Fix 26 cppcheck const-correctness warnings (constParameterPointer,
  constVariablePointer, constVariable)
- Update function declarations in headers to match const parameters
2026-07-19 15:57:31 +02:00
TapTap 598e4e7514 fix: address PR review - rolling adler32, extract helpers, configurable max file size
CI / build-and-test (push) Successful in 29s
CI / build-and-test (pull_request) Successful in 28s
- Fix rolling adler32: add missing -1 in s2 update formula (was producing
  wrong checksums, causing zero block matches)
- Fix file_send_sendfile signature to match typedef (add unused
  compression_level param)
- Extract receive_delta_file() from receive_incremental_check() to reduce
  nesting depth
- Extract send_file_incremental() helper in client_send.c
- Add delta_max_file_size to Config, serialized over wire
- Add --delta-max CLI flag
- Add test_large_file_delta (200KB) and extra delta_should_attempt cases
- Remove unused pos_in_block variable from delta_compute
2026-07-19 02:02:28 +02:00
TapTap e16db56580 feat: add delta transfer for incremental sync
CI / build-and-test (push) Successful in 28s
CI / build-and-test (pull_request) Successful in 28s
Implement rsync-style delta transfer using rolling checksums (Adler-32 +
xxHash32). When a file exists on both sides but has changed, only the
changed blocks are transmitted instead of the entire file.

- New status codes: STATUS_DELTA_SIGNATURE, STATUS_DELTA_DATA
- Protocol version bumped to 1.2.0
- Server generates block signature, client computes delta
- Auto-fallback to whole-file when delta >= 70% of file size
- Works with zstd compression on delta stream
- Configurable block size (default 8KB, --delta-block flag)
- 13 unit tests covering hashing, signature roundtrip, delta compute/apply,
  file growth/shrink, and decision logic
2026-07-18 20:00:15 +02:00
TapTap 864eb1316d Fix TLS code review issues: SSL cleanup, TLS 1.2 min, CA verify, deprecation guards, server --help, port validation, __thread io_ssl, shared accept loop 2026-07-18 17:39:37 +02:00
TapTap eff50346fd TLS transport: OpenSSL-based encrypted TCP
- New transport_tls.h/c: TLS server (server_create_tls, server_listen_tls)
  and client (client_connect_tls) using OpenSSL
- protocol.c: io_set_ssl() + SSL_read/SSL_write in send_n_data/receive_n_data
- transport_tcp.h: ssl/ssl_ctx fields added to Server/Client structs
- config.h/c: use_tls, tls_cert, tls_key fields
- client_cli.c: --tls, --cert, --key flags
- server.c: --tls, --cert, --key, -p flags with TLS support
- CMakeLists.txt: OpenSSL::SSL + OpenSSL::Crypto linkage
- shell.nix: openssl added to buildInputs
2026-07-18 17:38:55 +02:00
TapTap 1905369c9f Incremental sync: --incremental flag to skip unchanged files
- New STATUS_CHECK protocol status (value 6)
- Client sends path + size + mtime; server replies OK (skip) or NEXT (send)
- config.h/c: use_incremental field sent/received over wire
- file.c/h: file_send_single_calls_no_path helper for incremental path
- client_cli.c: --incremental flag
- client_send.c: per-file check before send in both single and chunk paths
- server.c: STATUS_CHECK handling in receive_files()
- multiprocessing.c: STATUS_CHECK handling in receive_thread()
- test.py: incremental sync test case
2026-07-18 17:19:12 +02:00
TapTap a62894be05 Merge main into server-improvements: resolve conflicts (protocol signatures, error returns) 2026-07-16 15:08:56 +02:00
TapTap bbdf027be8 Fix exit() in library code, add --include/--max-size/--min-size 2026-07-16 14:15:38 +02:00
TapTap efb98d8964 Multi-client server, protocol versioning, streaming decompression fixes 2026-07-16 13:59:41 +02:00
TapTap 284bf8f8bc rsync flags: -a, -n, -p, --exclude, --delete
Feature changes across 15 files:

- -a/--archive: enables -c -m -M (no -s, which slows transfers)
- -n/--dry-run: scan + print without connecting or transferring
- -p <port>: custom SSH port (passed as -p to ssh via execvp)
- --exclude <pattern>: glob-based filename filtering in scanner
- --delete: sender collects file manifest; receiver deletes unlisted files

Protocol: added STATUS_MANIFEST, use_delete field in Config wire format.
New utilities: glob_match(), delete_extras() with recursive directory walk.
Scanner: accepts exclude patterns; skips matching entries.
SSH transport: switched from execlp to execvp for dynamic port arg.
Server + multiprocessing: handle STATUS_MANIFEST in both single and
multithreaded receive paths.

Builds clean, all 7 tests pass.
2026-07-16 12:39:05 +02:00
TapTap d1a23fc4df quick-wins: ControlMaster, -z alias, --progress, fix write warning
- SSH ControlMaster auto with ControlPath for connection reuse
- -z as alias for -c (compression)
- --progress flag showing transfer rate in send_files
- Fixed unused-result warning on write() in transport_ssh.c
2026-07-16 12:28:16 +02:00
TapTap ca29109664 perf: downsized test data, increased SSH socket buffer, and made chunk size configurable
- Reduced test data from 50MB to 25MB (test.py)
- Increased SSH socketpair buffer to 1MB via setsockopt (transport_ssh.c)
- Made chunk size configurable: new Config field, --chunk-size CLI flag,
  sent over wire, and used in scanner instead of DESIRED_CHUNK_SIZE
- Fixed DESIRED_CHUNK_SIZE macro parentheses (chunk.h)
2026-07-16 12:20:02 +02:00
TapTap 8f4ea62ae7 Phase 1+2: bug fixes and dead code cleanup
Phase 1 — Bugs:
- receive_data: fix unsigned long long vs size_t mismatch
- chunk_deserialize: add NULL check in multiprocessing.c and server.c
- file_content_to_buffer: add missing fclose on error path
- send_files_multithreaded: propagate sender thread result
- server: add SIGPIPE handler
- to_disk: check fwrite return value
- scanner: use S_ISDIR instead of !S_ISREG
- scanner: free cur_path before early return

Phase 2 — Cleanup:
- Remove unused chunk_decompress function
- Remove unused array_list_clear function
- Remove unused num_connections config field
- Remove duplicate FILE_METADATA_WIRE_SIZE macro
- Update tests for removed APIs
2026-07-16 12:08:09 +02:00
TapTap 02619cca5c refactor: split monolithic modules and extract shared components
- Remove dead socket.c/socket.h (accumulated duplicate symbols)
- Extract compression.h/c (data_compress/decompress from data.c)
- Extract io.h/c (low-level I/O from protocol.c)
- Extract metadata.h/c (unified metadata wire format from file.c, chunk.c, utils.c)
- Split client.c into client_cli.c (CLI parsing) + client_send.c (send logic)
- Move receiver pipeline threads from server.c to multiprocessing.c
- Move to_disk/file_restore_metadata from utils.c to file.c/metadata.c
- Fix const qualifiers on to_disk and str_dup signatures
- Suppress chown unused-result warning
2026-07-15 19:36:26 +02:00
TapTap b3f69208ce cleanup: remove dead code, fix bugs, refactor SSH dest parsing
- Remove debug printf calls from file.c
- Remove dead code: chunk_print, chunk_format, chunk_data_create/delete
- Remove unused config.use_single_send_per_file field
- Fix server_delete() no-op (double-pointer)
- Fix double cast (long)(long) -> ptrdiff_t in chunk.c
- Fix thread return value: thrd_error instead of 1
- Remove unused config param from receive_chunk_enqueue()
- Make queue_double_capacity() static
- Move SSH dest parsing into config.c as config_parse_ssh_dest()
- Fix test_config_ssh_dest to test parsing round-trip
- Remove chunk_format test (obsolete format)
- Replace chunk_data_delete with data_destroy in tests
2026-07-07 19:07:24 +02:00
TapTap fbd7ccf4dd feat: add SSH transport, rsync-style CLI, and --stdio server mode
- Replace send()/recv() with read()/write() + io_set_fds() for fd abstraction
- Add client_connect_ssh() via socketpair + fork/exec; reap child in client_disconnect()
- Add server --stdio flag for SSH invocation
- Replace handle_arg() with rsync-style positional args (fastsync <src> <dest>)
- Add SSH vs TCP auto-detection via is_remote_dest()
- Add TransportType and ssh_destination to Config, fix uninitialized fields in
  config_receive() (pre-existing multithreaded crash)
- Sender threads now wait for server STATUS_OK before disconnecting
- Add --help, error handling for sendfile+SSH combos
- Add pre-flight checks and Posix Args test case to test.py
- Add test_config_ssh_dest() unit test
2026-07-06 18:42:33 +02:00
TapTap 660834d453 Merge branch 'main' into metadata-transfer, resolve conflicts
Conflicts resolved:
- config_create signature: added both use_metadata and use_sendfile params
- config_send/config_receive: wire protocol includes both use_metadata and use_sendfile
- client.c: config_create call updated, arg parsing includes both -M/--preserve and -f/--sendfile
- test_config.c: all three config_create calls updated
- file_send_sendfile: use file->data->size instead of removed struct stat
2026-07-05 20:29:04 +02:00
TapTap 4a1e666365 refactor: merge File/FileReceive, add optional metadata transfer with -M flag
- Remove FileReceive struct; use File everywhere with nullable FileMetadata*
- Remove struct stat from File; file size lives in Data->size (data_create_reserve)
- Add FileMetadata struct (mode, uid, gid, mtime) sent conditionally over wire
- Add -M / --preserve flag to client
- Restore permissions, ownership, timestamps on disk write
- Wire format uses per-file present flag for metadata (zero overhead when off)
2026-07-05 20:21:46 +02:00
TapTap a48b3e3a17 refactor: pass use_sendfile to config_create 2026-07-05 16:38:13 +02:00
TapTap 01045d7d14 feat: add -f/--sendfile for zero-copy file transfer
- Add file_send_sendfile() using sendfile() syscall to send file
  content directly from fd to socket, bypassing userspace memory
- Add use_sendfile field to Config struct (default false)
- Add -f / --sendfile flag parsing in client.c
- sendfile path in send_chunk() used when -f is set without -c or -s
- send_files_multithreaded() falls back to single-threaded when
  sendfile is enabled (loader pipeline becomes unnecessary)
- Add Sendfile (-f) test case to test.py
2026-07-05 16:25:20 +02:00
TapTap f189adcbff fix: config_send proto mismatch (use_compression→compression_level) and format string %d→%zu for size_t 2026-07-04 21:11:34 +02:00
Theo Tappe 4d6171d6bf fix: bring project back to a working state 2026-07-04 17:55:03 +02:00
Theo Tappe 75d5f91c6a add data compression and decompression 2026-06-14 16:52:07 +00:00
Theo Tappe f3e93d3fd9 improved testing capabilities and fixed bug that pre ended chunk sending 2026-06-10 22:11:12 +02:00
Theo Tappe 1e5eeb704f First Commit 2026-06-10 16:58:35 +02:00