389 Commits

Author SHA1 Message Date
TapTap a48b3e3a17 refactor: pass use_sendfile to config_create 2026-07-05 16:38:13 +02:00
TapTap d0c57ba2bb docs: update README with sendfile details and examples 2026-07-05 16:33:21 +02:00
TapTap 2fb7203c82 feat: sendfile works with -m, error on -f + -c/-s
- load_files_multithreaded skips file_load_data when sendfile is active
- -f + -m works: sendfile bypasses loader, sender uses sendfile directly
- -f combined with -c or -s prints error and exits with rc=1
- Add Sendfile + Multithreading test case
2026-07-05 16:31:08 +02:00
TapTap 01045d7d14 feat: add -f/--sendfile for zero-copy file transfer
- Add file_send_sendfile() using sendfile() syscall to send file
  content directly from fd to socket, bypassing userspace memory
- Add use_sendfile field to Config struct (default false)
- Add -f / --sendfile flag parsing in client.c
- sendfile path in send_chunk() used when -f is set without -c or -s
- send_files_multithreaded() falls back to single-threaded when
  sendfile is enabled (loader pipeline becomes unnecessary)
- Add Sendfile (-f) test case to test.py
2026-07-05 16:25:20 +02:00
TapTap e7c23c135b docs: update README with --source-dir/--dest-dir/--save-to-disk and test.py 2026-07-04 22:58:53 +02:00
TapTap 1c5099b963 Merge pull request 'test: auto-generate test files, verify content, CLI args, test_data dir' (#7) from improve-test-py into main
Reviewed-on: #7
2026-07-04 22:56:58 +02:00
TapTap 7cacce24e3 test: remove explicit compression level 0 from test case 2026-07-04 22:51:34 +02:00
TapTap 88a4d91c2e chore: ignore __pycache__ 2026-07-04 22:51:34 +02:00
TapTap f65cd02627 refactor: --source-dir, --dest-dir, --save-to-disk as CLI args
- Replace env var control (FASTSYNC_SOURCE_DIR, FASTSYNC_DEST_DIR,
  FASTSYNC_SAVE_TO_DISK) with explicit --source-dir, --dest-dir,
  --save-to-disk CLI arguments
- Keep env vars as fallback for backward compatibility
- Fix test.py: split '-c 0' into separate args ['-c', '0']
- Fix test.py: throttled suite failed because systemd-run didn't
  inherit env vars; now all settings are passed as CLI args
- Add env_name to exception/timeout result entries for correct display
2026-07-04 22:51:34 +02:00
TapTap b9d1a8c281 chore: ignore test_data/ directory 2026-07-04 22:51:31 +02:00
TapTap 4867e04677 fix: scanner mid-directory resume bug, -c -s protocol fix, 50 MB test data
- Fix scanner: when chunk fills up mid-directory, save DIR handle so
  remaining files in that directory are not skipped (pre-existing bug)
- Fix -c -s: add STATUS_CHUNK to protocol, send it before chunk data,
  handle it on the server side for both single and multithreaded paths
- Extract chunk_serialize/chunk_deserialize from chunk_compress/decompress
- Remove dead declarations (file_receive_from_buffer, etc.)
- Fix memory leak in receive_file_receive (free -> data_destroy)
- test.py generates ~50 MB of test data across bulk files
- All 8 integration tests + 7 unit tests pass
2026-07-04 22:51:31 +02:00
TapTap 7f6fa968ee test: auto-generate test files, verify content, CLI args, test_data dir
- test.py generates its own test files in test_data/ (small.txt, medium.txt,
  binary.bin, nested subtree)
- source and dest directories configurable via --source-dir / --dest-dir
- verifies all transferred files match originals byte-for-byte after each test
- waits for server to finish before verification (fixes race in multithreaded)
- all test data lives under test_data/ and is cleaned up unless --keep-data
2026-07-04 22:51:27 +02:00
TapTap 42870ec719 Merge pull request 'docs: update README with STATUS_CHUNK protocol and -s flag details' (#6) from update-readme into main
Reviewed-on: #6
2026-07-04 21:51:05 +02:00
TapTap 3146cd7cde docs: update README with STATUS_CHUNK protocol and -s flag details 2026-07-04 21:50:45 +02:00
TapTap b8186d4b52 Merge pull request 'feat: implement chunk serialization protocol with -s flag' (#5) from chunk-serialization into main
Reviewed-on: #5
2026-07-04 21:49:55 +02:00
TapTap 1d1f93f800 bench: add chunk serialization combos (-s, -c -s, -m -s, -m -c -s) to test.py 2026-07-04 21:47:49 +02:00
TapTap c4aeed9f4d feat: implement chunk serialization protocol with -s flag
- Extract chunk_serialize/chunk_deserialize from chunk_compress/chunk_decompress
- Add STATUS_CHUNK to wire protocol for chunk-mode transfers
- Client: -s sends chunk via STATUS_CHUNK (serialized or compressed+serialized)
- Server: handle STATUS_CHUNK in both single-threaded and multithreaded paths
- Fix memory leak in receive_file_receive (free->data_destroy)
- Remove dead declarations: file_receive_from_buffer, file_receive_decompress,
  file_compress, chunk_data_to_disk, unused #defines
2026-07-04 21:38:46 +02:00
TapTap e0f64e8d43 Merge pull request 'Add compression round-trip and scanner tests; fix 2 bugs found along the way' (#4) from add-tests into main
Reviewed-on: #4
2026-07-04 21:29:29 +02:00
TapTap 43c0a1395d Add compression and scanner tests; fix chunk_decompress data ownership bug, data_destroy NULL safety, chunk_compress memory leak 2026-07-04 21:25:34 +02:00
TapTap 38019e7e32 Merge pull request 'Fix config protocol mismatch and format string bugs' (#3) from fix-proto-and-format into main
Reviewed-on: #3
2026-07-04 21:12:10 +02:00
TapTap f189adcbff fix: config_send proto mismatch (use_compression→compression_level) and format string %d→%zu for size_t 2026-07-04 21:11:34 +02:00
TapTap a242a6c031 Merge pull request 'Fix critical issues and all tests' (#2) from fix-critical-issues into main
Reviewed-on: #2
2026-07-04 21:09:21 +02:00
TapTap fe8fca47b4 fix: re-apply socket.c file descriptor check after merge 2026-07-04 21:08:30 +02:00
TapTap fefa968945 Merge remote-tracking branch 'origin/main' into fix-critical-issues
# Conflicts:
#	src/shared/chunk.c
2026-07-04 21:07:45 +02:00
TapTap 1486ac7088 fix: chunk_decompress compilation, chunk_compress data ptr, chunk_format serialization, file_destroy leak, test_chunk memcmp 2026-07-04 21:02:59 +02:00
TapTap 7c02b16325 Revert "Merge pull request 'Fix critical issues: implement chunk_decompress and fix socket file descriptor check' (#1) from fix-critical-issues into main"
This reverts commit d176507047, reversing
changes made to ae4c8ff9f0.
2026-07-04 20:51:35 +02:00
TapTap d176507047 Merge pull request 'Fix critical issues: implement chunk_decompress and fix socket file descriptor check' (#1) from fix-critical-issues into main
Reviewed-on: #1
2026-07-04 20:48:04 +02:00
TapTap ae4c8ff9f0 fix: remove some benchmarks so that its faster 2026-07-04 20:45:47 +02:00
TapTap f4f9adb007 Update opencode.json 2026-07-04 20:29:57 +02:00
TapTap 95e3bded60 Update shell.nix 2026-07-04 20:28:00 +02:00
TapTap e6c55356c8 Fix critical issues: implement chunk_decompress and fix file descriptor check in server_listen 2026-07-04 20:24:16 +02:00
TapTap 95568ae34e Add technical README file 2026-07-04 18:18:48 +02:00
TapTap 12ca9e736b feat: add rules for opencode and remove old irrelevant files 2026-07-04 18:09:16 +02:00
TapTap 1b7d623dfe fix: change to right compression level 2026-07-04 18:03:22 +02:00
Theo Tappe 4d6171d6bf fix: bring project back to a working state 2026-07-04 17:55:03 +02:00
Theo Tappe 75d5f91c6a add data compression and decompression 2026-06-14 16:52:07 +00:00
Theo Tappe 7478d050e3 Improved testing capabilities and fixed missing finished status in multithreading 2026-06-11 15:41:37 +02:00
Theo Tappe f3e93d3fd9 improved testing capabilities and fixed bug that pre ended chunk sending 2026-06-10 22:11:12 +02:00
Theo Tappe 1e5eeb704f First Commit 2026-06-10 16:58:35 +02:00