TapTap/FastSync
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
331 Commits 33 Branches 0 Tags
7d8f5938442ec0e0129631c627948ecedc5c9d09
Commit Graph

331 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
TapTap 478bdd6fe6 add docker as dependency
CI / build-and-test (push) Successful in 27s
Details
CI / build-and-test (pull_request) Successful in 26s
Details
2026-07-18 16:31:55 +02:00
TapTap e408b08443 fix: skip netem_reset when no sudo (light mode)
CI / build-and-test (push) Successful in 26s
Details
CI / build-and-test (pull_request) Successful in 27s
Details
2026-07-18 16:10:52 +02:00
TapTap 2c904bfac5 fix: handle missing ssh in preflight checks
CI / build-and-test (push) Failing after 4s
Details
CI / build-and-test (pull_request) Failing after 4s
Details
2026-07-18 16:08:26 +02:00
TapTap e9554c1e69 ci: add integration tests (light mode)
CI / build-and-test (push) Failing after 4s
Details
CI / build-and-test (pull_request) Failing after 4s
Details
2026-07-18 16:06:22 +02:00
TapTap 8f2205dd30 test: add --full flag, default mode runs 5 fast tests (~0.3s)
CI / build-and-test (push) Successful in 4s
Details
CI / build-and-test (pull_request) Successful in 4s
Details 
Default mode skips bulk data, network shaping, SSH/rsync
comparisons, and feature tests. --full restores original
31-test suite with LAN profiling and all benchmarks.
 2026-07-18 16:02:36 +02:00
TapTap 70ab76fe90 Merge pull request 'ci: add build and unit test pipeline' (#17) from actions into main
CI / build-and-test (push) Successful in 4s
Details 
Reviewed-on: #17
 2026-07-18 15:52:06 +02:00
TapTap ed7f8ee56e ci: add g++ and libc6-dev, use v4 image (verified locally)
CI / build-and-test (push) Successful in 4s
Details
CI / build-and-test (pull_request) Successful in 4s
Details
2026-07-18 15:31:41 +02:00
TapTap 8c7ebbfe32 ci: add make to Docker image, use v3 tag
CI / build-and-test (push) Failing after 10s
Details
2026-07-18 15:27:35 +02:00
TapTap cc09bedc07 ci: use v2 image tag to bypass runner cache
CI / build-and-test (push) Failing after 8s
Details
2026-07-18 15:20:57 +02:00
TapTap 5fe40a5242 ci: add node.js to Docker image for checkout action
CI / build-and-test (push) Failing after 1s
Details
2026-07-18 15:19:41 +02:00
TapTap 9130ee4b03 ci: use custom Docker image with build tools pre-installed
CI / build-and-test (push) Failing after 1s
Details
2026-07-18 15:15:26 +02:00
TapTap 36da0d6adb ci: restore apt-get update for package resolution
CI / build-and-test (push) Successful in 2m16s
Details
2026-07-18 14:45:03 +02:00
TapTap e7087d518b ci: skip apt-get update for faster installs
CI / build-and-test (push) Failing after 2s
Details
2026-07-18 14:43:02 +02:00
TapTap 3ea11b5984 ci: add build + unit tests workflow, lower cmake minimum to 3.22
CI / build-and-test (push) Successful in 2m17s
Details
2026-07-18 14:41:27 +02:00
TapTap 33f70f7de3 add demo action
Gitea Actions Demo / Explore-Gitea-Actions (push) Successful in 59s
Details
2026-07-18 14:22:58 +02:00
TapTap 45dd62f568 Merge pull request 'Bandwidth throttling: --bwlimit <KB/s> with token bucket' (#15) from bwlimit into main 
Reviewed-on: #15
 2026-07-17 10:22:46 +02:00
TapTap 4f8ae5bb70 fix: bw_tokens initialized to io_bwlimit, overflow check for --bwlimit 2026-07-17 10:07:18 +02:00
TapTap e2dac589a8 Merge pull request 'Add custom opencode agents for FastSync development' (#16) from add-custom-agents into main 
Reviewed-on: #16
 2026-07-17 10:02:04 +02:00
TapTap 1b87131e82 Add pr-review and pr-build skills 
- pr-review: read-only code review of PR branches, outputs findings report
- pr-build: builds, tests, and fixes PR branches, commits fixes
 2026-07-16 17:07:44 +02:00
TapTap 3dcaf0b79d Fix bwlimit review: remove __thread (breaks -m), validate >0, handle nanosleep EINTR 2026-07-16 17:00:14 +02:00
TapTap ef6aa143d0 Add custom opencode agents for FastSync development 
- c-reviewer: memory/thread safety, style review for C11 code
- test-writer: unit test generation using custom test framework
- cmake-expert: CMake build system management
- perf-analyst: transfer pipeline performance analysis
- protocol-designer: wire protocol design and extension
- doc-generator: API docs, protocol specs, usage examples
 2026-07-16 16:58:22 +02:00
TapTap 09a76e2a9a Bandwidth throttling: --bwlimit <KB/s> with token bucket 
- protocol.h/c: io_set_bwlimit() + token bucket in send_n_data
  (64KB chunks, nanosleep-based deficit compensation)
- client_cli.c: --bwlimit <KB/s> flag
- test.py: bandwidth limit test case
 2026-07-16 15:42:16 +02:00
TapTap c2436bef59 Merge pull request 'Multi-client server, protocol versioning, streaming decompression' (#11) from server-improvements into main 
Reviewed-on: #11
 2026-07-16 15:15:20 +02:00
TapTap a62894be05 Merge main into server-improvements: resolve conflicts (protocol signatures, error returns) 2026-07-16 15:08:56 +02:00
TapTap 93a5768bf0 Merge pull request 'Fix exit() in library code, add --include/--max-size/--min-size' (#12) from exit-and-select into main 
Reviewed-on: #12
 2026-07-16 15:03:38 +02:00
TapTap bbdf027be8 Fix exit() in library code, add --include/--max-size/--min-size 2026-07-16 14:15:38 +02:00
TapTap efb98d8964 Multi-client server, protocol versioning, streaming decompression fixes 2026-07-16 13:59:41 +02:00
TapTap 07207cd71d Merge pull request 'Fair benchmark: random data, exclude dry run from metrics' (#10) from fair-benchmark into main 
Reviewed-on: #10
 2026-07-16 13:46:27 +02:00
TapTap ea551c4a1d Fair benchmark: random data instead of zeros, exclude dry run from metrics 2026-07-16 13:43:33 +02:00
TapTap eff00fa021 Merge pull request 'Rsync-compatible flags: -a, -n, -p, --exclude, --delete' (#8) from rsync-flags into main 
Reviewed-on: #8
 2026-07-16 13:38:19 +02:00
TapTap ab24724cb0 Merge pull request 'Update README and add feature tests for rsync-compatible flags' (#9) from update-readme-and-tests into rsync-flags 
Reviewed-on: #9
 2026-07-16 13:38:13 +02:00
TapTap 2ff51246c3 Update README and add feature tests for rsync-compatible flags 2026-07-16 13:31:43 +02:00
TapTap 284bf8f8bc rsync flags: -a, -n, -p, --exclude, --delete 
Feature changes across 15 files:

- -a/--archive: enables -c -m -M (no -s, which slows transfers)
- -n/--dry-run: scan + print without connecting or transferring
- -p <port>: custom SSH port (passed as -p to ssh via execvp)
- --exclude <pattern>: glob-based filename filtering in scanner
- --delete: sender collects file manifest; receiver deletes unlisted files

Protocol: added STATUS_MANIFEST, use_delete field in Config wire format.
New utilities: glob_match(), delete_extras() with recursive directory walk.
Scanner: accepts exclude patterns; skips matching entries.
SSH transport: switched from execlp to execvp for dynamic port arg.
Server + multiprocessing: handle STATUS_MANIFEST in both single and
multithreaded receive paths.

Builds clean, all 7 tests pass.
 2026-07-16 12:39:05 +02:00
TapTap f778d1903c Merge pull request 'Streaming zstd: switch to ZSTD_compressStream2 / ZSTD_decompressStream' (#7) from streaming-zstd into main 
Reviewed-on: #7
 2026-07-16 12:31:49 +02:00
TapTap 04aad76481 Merge pull request 'Quick wins: ControlMaster, -z alias, --progress, fix write warning' (#6) from quick-wins into main 
Reviewed-on: #6
 2026-07-16 12:30:13 +02:00
TapTap 52571bc960 streaming zstd: switch to ZSTD_compressStream2 / ZSTD_decompressStream 
Replaced one-shot ZSTD_compress / ZSTD_decompress with streaming
API (ZSTD_compressStream2 + ZSTD_decompressStream) in both
data_compress and data_decompress. Interface unchanged.

Prepares for true streaming transfer where compression can be
interleaved with transmission.
 2026-07-16 12:29:31 +02:00
TapTap d1a23fc4df quick-wins: ControlMaster, -z alias, --progress, fix write warning 
- SSH ControlMaster auto with ControlPath for connection reuse
- -z as alias for -c (compression)
- --progress flag showing transfer rate in send_files
- Fixed unused-result warning on write() in transport_ssh.c
 2026-07-16 12:28:16 +02:00
TapTap 78cc735cff Merge pull request 'Performance tuning: test data, SSH buffer, and configurable chunk size' (#5) from perf-and-tuning into main 
Reviewed-on: #5
 2026-07-16 12:24:44 +02:00
TapTap ca29109664 perf: downsized test data, increased SSH socket buffer, and made chunk size configurable 
- Reduced test data from 50MB to 25MB (test.py)
- Increased SSH socketpair buffer to 1MB via setsockopt (transport_ssh.c)
- Made chunk size configurable: new Config field, --chunk-size CLI flag,
  sent over wire, and used in scanner instead of DESIRED_CHUNK_SIZE
- Fixed DESIRED_CHUNK_SIZE macro parentheses (chunk.h)
 2026-07-16 12:20:02 +02:00
TapTap 270623b90e Merge pull request 'Phase 1+2: bug fixes and dead code cleanup' (#4) from bugfix-cleanup into main 
Reviewed-on: #4
 2026-07-16 12:11:31 +02:00
TapTap 8f4ea62ae7 Phase 1+2: bug fixes and dead code cleanup 
Phase 1 — Bugs:
- receive_data: fix unsigned long long vs size_t mismatch
- chunk_deserialize: add NULL check in multiprocessing.c and server.c
- file_content_to_buffer: add missing fclose on error path
- send_files_multithreaded: propagate sender thread result
- server: add SIGPIPE handler
- to_disk: check fwrite return value
- scanner: use S_ISDIR instead of !S_ISREG
- scanner: free cur_path before early return

Phase 2 — Cleanup:
- Remove unused chunk_decompress function
- Remove unused array_list_clear function
- Remove unused num_connections config field
- Remove duplicate FILE_METADATA_WIRE_SIZE macro
- Update tests for removed APIs
 2026-07-16 12:08:09 +02:00
TapTap 81800fa661 Merge pull request 'refactor: split monolithic modules and extract shared components' (#1) from cleanup into main 
Reviewed-on: #1
 2026-07-16 11:58:37 +02:00
TapTap 56d26f083c Merge pull request 'fixup: address PR review comments' (#3) from fixup-pr-comments into cleanup 
Reviewed-on: #3
 2026-07-16 11:58:26 +02:00
TapTap d36768792a fixup: address PR review comments 
- Merge io.h/c back into protocol.h/c
- Change send_data to take Data* argument
- Remove redundant file_load_data from send_chunk
- Move compression into file_send_single_calls
- Move file_receive from multiprocessing.c to file.c
 2026-07-16 11:54:20 +02:00
TapTap 02619cca5c refactor: split monolithic modules and extract shared components 
- Remove dead socket.c/socket.h (accumulated duplicate symbols)
- Extract compression.h/c (data_compress/decompress from data.c)
- Extract io.h/c (low-level I/O from protocol.c)
- Extract metadata.h/c (unified metadata wire format from file.c, chunk.c, utils.c)
- Split client.c into client_cli.c (CLI parsing) + client_send.c (send logic)
- Move receiver pipeline threads from server.c to multiprocessing.c
- Move to_disk/file_restore_metadata from utils.c to file.c/metadata.c
- Fix const qualifiers on to_disk and str_dup signatures
- Suppress chown unused-result warning
 2026-07-15 19:36:26 +02:00
TapTap b3f69208ce cleanup: remove dead code, fix bugs, refactor SSH dest parsing 
- Remove debug printf calls from file.c
- Remove dead code: chunk_print, chunk_format, chunk_data_create/delete
- Remove unused config.use_single_send_per_file field
- Fix server_delete() no-op (double-pointer)
- Fix double cast (long)(long) -> ptrdiff_t in chunk.c
- Fix thread return value: thrd_error instead of 1
- Remove unused config param from receive_chunk_enqueue()
- Make queue_double_capacity() static
- Move SSH dest parsing into config.c as config_parse_ssh_dest()
- Fix test_config_ssh_dest to test parsing round-trip
- Remove chunk_format test (obsolete format)
- Replace chunk_data_delete with data_destroy in tests
 2026-07-07 19:07:24 +02:00
TapTap 783400d7e9 Merge pull request 'feat: add SSH transport, rsync-style CLI, and --stdio server mode' (#16) from transport-abstraction into main 
Reviewed-on: #16
 2026-07-07 18:59:45 +02:00
TapTap afca5e4f69 test: expand SSH tests to all valid flag combinations 2026-07-07 18:55:47 +02:00
TapTap e099dc4843 fix: skip wrappers dirs in SSH PATH scan; verify symlink with which 2026-07-06 20:39:59 +02:00
TapTap c2bdf68429 fix: check SSH symlink by file existence instead of PATH lookup 2026-07-06 20:33:51 +02:00