Commit Graph

21 Commits

Author SHA1 Message Date
TapTap 6a609c1b07 test: add text=True to rsync subprocess.run
Without text=True, stderr is bytes, and bytes.split('\n') throws
TypeError: a bytes-like object is required, not 'str'.
2026-07-05 18:42:35 +02:00
TapTap 1c26a7a712 test: robust netem setup and rsync daemon startup
- netem_apply now calls netem_reset() first to clear any leftover qdisc
- Rsync daemon startup polls TCP port instead of fixed sleep
- Dest dir cleared at start of run
2026-07-05 18:33:04 +02:00
TapTap 0c2364bd62 test: clear dest dir at start of run 2026-07-05 18:11:23 +02:00
TapTap 5170155112 bench: run rsync over localhost network for fair comparison
Previously rsync ran as a direct local copy (rsync -aH source/ dest/), bypassing the
loopback interface entirely. This meant tc netem latency/loss/rate limits were never
applied to rsync, making the comparison fundamentally unfair.

Now rsync goes through the network via an rsync daemon on localhost:
  rsync --daemon --no-detach --config=rsyncd.conf
  rsync rsync://localhost:PORT/source/ dest/

The daemon is started before rsync tests and killed afterward. Both the rsync client
and FastSync client run under the same systemd-run disk I/O limits when applicable.
2026-07-05 16:58:13 +02:00
TapTap 7a1c491f7f bench: default is LAN-only, --unlimited and --wan for exclusive profiles 2026-07-05 16:51:11 +02:00
TapTap a90e510424 bench: --wan runs only WAN, not also Unlimited 2026-07-05 16:48:45 +02:00
TapTap 39faff1c4c bench: LAN 20ms ±1ms, 0.1% loss 2026-07-05 16:47:35 +02:00
TapTap 89274c0fe2 bench: network profiles (LAN/WAN), jitter, packet loss, rsync comparison
- Replace single 'Throttled' suite with profiles: Unlimited, LAN (default),
  WAN (--wan flag)
- LAN: 1000mbit, 1ms ±0.1ms delay, 0% loss
- WAN: 100mbit, 50ms ±10ms delay, 1% loss (with --wan)
- Add jitter and packet loss to tc netem config
- Add rsync -aH (archive) and rsync -aHz (archive+compress) as comparison
- Fix verify_transfer to take the received directory directly
- --no-unlimited, --no-lan to skip profiles
- 36 tests total (10 FastSync + 2 rsync across 3 profiles)
2026-07-05 16:45:08 +02:00
TapTap 2fb7203c82 feat: sendfile works with -m, error on -f + -c/-s
- load_files_multithreaded skips file_load_data when sendfile is active
- -f + -m works: sendfile bypasses loader, sender uses sendfile directly
- -f combined with -c or -s prints error and exits with rc=1
- Add Sendfile + Multithreading test case
2026-07-05 16:31:08 +02:00
TapTap 01045d7d14 feat: add -f/--sendfile for zero-copy file transfer
- Add file_send_sendfile() using sendfile() syscall to send file
  content directly from fd to socket, bypassing userspace memory
- Add use_sendfile field to Config struct (default false)
- Add -f / --sendfile flag parsing in client.c
- sendfile path in send_chunk() used when -f is set without -c or -s
- send_files_multithreaded() falls back to single-threaded when
  sendfile is enabled (loader pipeline becomes unnecessary)
- Add Sendfile (-f) test case to test.py
2026-07-05 16:25:20 +02:00
TapTap 7cacce24e3 test: remove explicit compression level 0 from test case 2026-07-04 22:51:34 +02:00
TapTap f65cd02627 refactor: --source-dir, --dest-dir, --save-to-disk as CLI args
- Replace env var control (FASTSYNC_SOURCE_DIR, FASTSYNC_DEST_DIR,
  FASTSYNC_SAVE_TO_DISK) with explicit --source-dir, --dest-dir,
  --save-to-disk CLI arguments
- Keep env vars as fallback for backward compatibility
- Fix test.py: split '-c 0' into separate args ['-c', '0']
- Fix test.py: throttled suite failed because systemd-run didn't
  inherit env vars; now all settings are passed as CLI args
- Add env_name to exception/timeout result entries for correct display
2026-07-04 22:51:34 +02:00
TapTap 4867e04677 fix: scanner mid-directory resume bug, -c -s protocol fix, 50 MB test data
- Fix scanner: when chunk fills up mid-directory, save DIR handle so
  remaining files in that directory are not skipped (pre-existing bug)
- Fix -c -s: add STATUS_CHUNK to protocol, send it before chunk data,
  handle it on the server side for both single and multithreaded paths
- Extract chunk_serialize/chunk_deserialize from chunk_compress/decompress
- Remove dead declarations (file_receive_from_buffer, etc.)
- Fix memory leak in receive_file_receive (free -> data_destroy)
- test.py generates ~50 MB of test data across bulk files
- All 8 integration tests + 7 unit tests pass
2026-07-04 22:51:31 +02:00
TapTap 7f6fa968ee test: auto-generate test files, verify content, CLI args, test_data dir
- test.py generates its own test files in test_data/ (small.txt, medium.txt,
  binary.bin, nested subtree)
- source and dest directories configurable via --source-dir / --dest-dir
- verifies all transferred files match originals byte-for-byte after each test
- waits for server to finish before verification (fixes race in multithreaded)
- all test data lives under test_data/ and is cleaned up unless --keep-data
2026-07-04 22:51:27 +02:00
TapTap 1d1f93f800 bench: add chunk serialization combos (-s, -c -s, -m -s, -m -c -s) to test.py 2026-07-04 21:47:49 +02:00
TapTap ae4c8ff9f0 fix: remove some benchmarks so that its faster 2026-07-04 20:45:47 +02:00
TapTap 1b7d623dfe fix: change to right compression level 2026-07-04 18:03:22 +02:00
Theo Tappe 4d6171d6bf fix: bring project back to a working state 2026-07-04 17:55:03 +02:00
Theo Tappe 7478d050e3 Improved testing capabilities and fixed missing finished status in multithreading 2026-06-11 15:41:37 +02:00
Theo Tappe f3e93d3fd9 improved testing capabilities and fixed bug that pre ended chunk sending 2026-06-10 22:11:12 +02:00
Theo Tappe 1e5eeb704f First Commit 2026-06-10 16:58:35 +02:00