diff --git a/.gitignore b/.gitignore index a6030f3..d238172 100644 --- a/.gitignore +++ b/.gitignore @@ -8,3 +8,7 @@ build-*/ build2/ build3/ build_docker2/ + +# Test/run artifacts +root/ +test_partial_install_tmp/ diff --git a/.opencode/agents/architect.md b/.opencode/agents/architect.md index cce442f..878cf5a 100644 --- a/.opencode/agents/architect.md +++ b/.opencode/agents/architect.md @@ -128,7 +128,7 @@ Do not wait for the user to tell you CI failed — check proactively. The user s ## Branch Strategy -Never push directly to `main`. All changes must be developed on a feature branch and merged via a pull request. Always create a new branch (`git checkout -b `) before making changes, push it, and open a PR with `gh pr create --fill`. Wait for CI to pass before merging. +Never push directly to `dev` or `main`. All changes must be developed on a feature branch and merged via a pull request targeting `dev`. Create a branch (`git checkout -b `), push it, and open the PR with `tea pr create --repo TapTap/FastSync --base dev --head `. Wait for CI to pass before merging. ## Dependency Installation diff --git a/.opencode/agents/c-reviewer.md b/.opencode/agents/c-reviewer.md index 9c0ba24..07e3fd3 100644 --- a/.opencode/agents/c-reviewer.md +++ b/.opencode/agents/c-reviewer.md @@ -92,7 +92,7 @@ When using `tea` (the task execution agent) to run CI or tests, always set a suf ## Branch Strategy -Never push directly to `main`. All changes must be developed on a feature branch and merged via a pull request. Always create a new branch (`git checkout -b `) before making changes, push it, and open a PR with `gh pr create --fill`. Wait for CI to pass before merging. +Never push directly to `dev` or `main`. All changes must be developed on a feature branch and merged via a pull request targeting `dev`. Create a branch (`git checkout -b `), push it, and open the PR with `tea pr create --repo TapTap/FastSync --base dev --head `. Wait for CI to pass before merging. ## Dependency Installation diff --git a/.opencode/agents/cmake-expert.md b/.opencode/agents/cmake-expert.md index 430bfc9..211f104 100644 --- a/.opencode/agents/cmake-expert.md +++ b/.opencode/agents/cmake-expert.md @@ -84,7 +84,7 @@ tests/integration/ — Python pytest integration tests ### Dependencies - **zstd** — found via `find_library(ZSTD_LIBRARY zstd)` - **OpenSSL** — found via `find_package(OpenSSL REQUIRED)` (TLS 1.2+ transport) -- **xxHash** — fetched via `FetchContent` from GitHub (delta transfer hashing, v0.8.3) +- **xxHash** — fetched via `FetchContent` from the upstream repository (delta transfer hashing, v0.8.3) - **pthreads** — found via `find_package(Threads REQUIRED)` - **C11 standard** — required - **CMake 3.22+** — minimum version @@ -159,7 +159,7 @@ cmake -B build -S . -DCMAKE_BUILD_TYPE=RelWithDebInfo ```bash cmake -B build -S . cmake --build build -j$(nproc) -./build/server +./build/server -p 8080 --allow-unauthenticated ./build/client ./build/tests ``` @@ -187,7 +187,7 @@ When using `tea` (the task execution agent) to run CI or tests, always set a suf ## Branch Strategy -Never push directly to `main`. All changes must be developed on a feature branch and merged via a pull request. Always create a new branch (`git checkout -b `) before making changes, push it, and open a PR with `gh pr create --fill`. Wait for CI to pass before merging. +Never push directly to `dev` or `main`. All changes must be developed on a feature branch and merged via a pull request targeting `dev`. Create a branch (`git checkout -b `), push it, and open the PR with `tea pr create --repo TapTap/FastSync --base dev --head `. Wait for CI to pass before merging. ## Dependency Installation diff --git a/.opencode/agents/code-explainer.md b/.opencode/agents/code-explainer.md index b36ba1b..5039f7a 100644 --- a/.opencode/agents/code-explainer.md +++ b/.opencode/agents/code-explainer.md @@ -27,7 +27,7 @@ FastSync is a file synchronization tool (like rsync, but faster). It transfers f cmake -B build -S . && cmake --build build -j$(nproc) # Server (TCP mode) -./build/server +./build/server -p 8080 --allow-unauthenticated # Client (TCP mode) ./build/client --source-dir /path/to/send --dest-dir /path/to/receive --save-to-disk @@ -37,13 +37,13 @@ cmake -B build -S . && cmake --build build -j$(nproc) # Run tests ./build/tests # unit tests -python3 test.py # integration tests +python3 -m pytest tests/integration/ -n 4 --dist=load -m "not setpriv" # integration tests ``` ## Code Walkthrough ### Client Entry Point (`src/client/client_cli.c`) -- Parses CLI arguments using `getopt_long` +- Parses CLI arguments using a custom option-table parser (`OPTION_TABLE` in `src/client/client_cli.c`); there is no `getopt*` usage - Creates `Config` struct with all options - Detects SSH destinations (contains `:`) - Calls into `client_send.c` for the actual transfer @@ -109,7 +109,7 @@ Collection of files for batch transfer. Serialized with file count, then per-fil zstd streaming compression via `ZSTD_compressStream2`/`ZSTD_decompressStream`. Compression happens per-chunk in the sender stage. Level 1-22 (default 5). Streaming means memory usage stays bounded regardless of file size. ### "How does sendfile() work?" -On Linux, `sendfile()` copies data directly from kernel file buffer to socket, bypassing userspace. ~2x faster for large files. Enabled with `-f` flag. Only works with TCP (not SSH, not compression). +On Linux, `sendfile()` copies data directly from kernel file buffer to socket, bypassing userspace. ~2x faster for large files. Enabled with `--sendfile` (long form only). Only works with TCP (not SSH, not compression). ### "How does incremental sync work?" Client sends file metadata (path, size, mtime) to server. Server checks if destination file has same size+mtime. If match, server responds `STATUS_OK` (skip). If mismatch, server responds `STATUS_NEXT` (send). @@ -138,7 +138,7 @@ When using `tea` (the task execution agent) to run CI or tests, always set a suf ## Branch Strategy -Never push directly to `main`. All changes must be developed on a feature branch and merged via a pull request. Always create a new branch (`git checkout -b `) before making changes, push it, and open a PR with `gh pr create --fill`. Wait for CI to pass before merging. +Never push directly to `dev` or `main`. All changes must be developed on a feature branch and merged via a pull request targeting `dev`. Create a branch (`git checkout -b `), push it, and open the PR with `tea pr create --repo TapTap/FastSync --base dev --head `. Wait for CI to pass before merging. ## Dependency Installation diff --git a/.opencode/agents/code-quality-guardian.md b/.opencode/agents/code-quality-guardian.md index d28e2b9..e8aab2f 100644 --- a/.opencode/agents/code-quality-guardian.md +++ b/.opencode/agents/code-quality-guardian.md @@ -316,7 +316,7 @@ When using `tea` (the task execution agent) to run CI or tests, always set a suf ## Branch Strategy -Never push directly to `main`. All changes must be developed on a feature branch and merged via a pull request. Always create a new branch (`git checkout -b `) before making changes, push it, and open a PR with `gh pr create --fill`. Wait for CI to pass before merging. +Never push directly to `dev` or `main`. All changes must be developed on a feature branch and merged via a pull request targeting `dev`. Create a branch (`git checkout -b `), push it, and open the PR with `tea pr create --repo TapTap/FastSync --base dev --head `. Wait for CI to pass before merging. ## Dependency Installation diff --git a/.opencode/agents/debugger.md b/.opencode/agents/debugger.md index d5a9fd1..10c7822 100644 --- a/.opencode/agents/debugger.md +++ b/.opencode/agents/debugger.md @@ -14,10 +14,9 @@ Diagnose crashes, memory errors, hangs, and logic bugs. You use structured debug ### Memory Errors ```bash # AddressSanitizer (fast, recommended first) -cmake -B build -S . -DCMAKE_C_FLAGS="-fsanitize=address -fno-omit-frame-pointer" \ - -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=address" -cmake --build build -j$(nproc) -./build/client # or ./build/server +cmake -B build-asan -S . -DSANITIZER=address +cmake --build build-asan -j$(nproc) +./build-asan/client # or ./build-asan/server -p 8080 --allow-unauthenticated # Valgrind (slower, more thorough) valgrind --leak-check=full --show-leak-kinds=all --track-origins=yes \ @@ -32,10 +31,9 @@ valgrind --tool=drd ./build/client ... ### Thread Sanitizer ```bash -cmake -B build -S . -DCMAKE_C_FLAGS="-fsanitize=thread" \ - -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=thread" -cmake --build build -j$(nproc) -./build/tests +cmake -B build-tsan -S . -DSANITIZER=thread +cmake --build build-tsan -j$(nproc) +./build-tsan/tests ``` ### GDB @@ -143,7 +141,7 @@ gprof ./build/client gmon.out ### Step 5: Verify - Run `./build/tests` (unit tests) -- Run `python3 test.py` (integration tests) +- Run `python3 -m pytest tests/integration/ -n 4 --dist=load -m "not setpriv"` (integration tests) - Run under valgrind again to confirm clean - Test under ASan again @@ -162,7 +160,7 @@ When using `tea` (the task execution agent) to run CI or tests, always set a suf ## Branch Strategy -Never push directly to `main`. All changes must be developed on a feature branch and merged via a pull request. Always create a new branch (`git checkout -b `) before making changes, push it, and open a PR with `gh pr create --fill`. Wait for CI to pass before merging. +Never push directly to `dev` or `main`. All changes must be developed on a feature branch and merged via a pull request targeting `dev`. Create a branch (`git checkout -b `), push it, and open the PR with `tea pr create --repo TapTap/FastSync --base dev --head `. Wait for CI to pass before merging. ## Dependency Installation diff --git a/.opencode/agents/doc-generator.md b/.opencode/agents/doc-generator.md index 1062c6c..48d2ad0 100644 --- a/.opencode/agents/doc-generator.md +++ b/.opencode/agents/doc-generator.md @@ -96,7 +96,7 @@ When using `tea` (the task execution agent) to run CI or tests, always set a suf ## Branch Strategy -Never push directly to `main`. All changes must be developed on a feature branch and merged via a pull request. Always create a new branch (`git checkout -b `) before making changes, push it, and open a PR with `gh pr create --fill`. Wait for CI to pass before merging. +Never push directly to `dev` or `main`. All changes must be developed on a feature branch and merged via a pull request targeting `dev`. Create a branch (`git checkout -b `), push it, and open the PR with `tea pr create --repo TapTap/FastSync --base dev --head `. Wait for CI to pass before merging. ## Dependency Installation diff --git a/.opencode/agents/feature-scout.md b/.opencode/agents/feature-scout.md index 449e746..43521c4 100644 --- a/.opencode/agents/feature-scout.md +++ b/.opencode/agents/feature-scout.md @@ -16,12 +16,18 @@ Scan the codebase for patterns that suggest new feature opportunities. You ident ### Module Map ``` src/client/ Client-side: CLI parsing, scanning, sending - client_cli.c Entry point, argument parsing, config setup + client_cli.c Entry point, OPTION_TABLE parser, config setup + usage.c Usage/help text (authoritative CLI flag list) client_send.c Transfer orchestration, pipeline management + client_validation.c Destination/CLI validation scanner.c BFS directory traversal, chunk building + change_list.c File change-list bookkeeping src/server/ Server-side: listening, receiving, writing server.c TCP accept loop, per-connection handling + server_cli.c Server option-table CLI parsing + receiver.c Receiver-side file handling + receiver_pipeline.c Receiver worker pipeline src/shared/ Shared libraries (used by both client and server) protocol.c/h Wire protocol: status codes, send/receive primitives @@ -32,40 +38,63 @@ src/shared/ Shared libraries (used by both client and server) data.c/h Generic buffer type (Data) metadata.c/h File metadata (mode, uid, gid, mtime) file.c/h File representation + file_send.c/h Sender-side file transfer + file_receive.c/h Receiver-side file transfer + file_list.c/h File list model + file_store.c/h Destination file store array_list.c/h Dynamic array + delta.c/h Delta transfer algorithm + checksum.c/h Whole-file/block checksums (xxHash, md5) + filter.c/h rsync-style filter rules + batch.c/h Batch files (--write-batch/--read-batch) + charset.c/h Filename charset conversion (--iconv) + chmod.c/h Permission modification (--chmod) + xattr.c/h Extended attributes + hardlink.c/h Hard-link handling + identity.c/h uid/gid mapping (--usermap/--groupmap/--chown) + credentials.c/h Daemon credentials + daemon_conf.c/h Daemon module configuration + motd.c/h Daemon MOTD + delay_updates.c/h Delayed update staging + stop_condition.c/h Stop-after/stop-at handling transport_tcp.c/h TCP client/server with sendfile() zero-copy transport_ssh.c/h SSH transport with ControlMaster transport_tls.c/h TLS encryption via OpenSSL multiprocessing.c/h Fork-based concurrency log.c/h Logging utilities utils.c/h Shared utilities + file_types.h Shared file type definitions ``` -### Existing CLI Flags (from client_cli.c) +### Existing CLI Flags (authoritative source: `src/client/usage.c`) ``` ---source-dir Source directory to sync (required) ---dest-dir Destination directory on server (required) ---host Server hostname/IP (required) ---port Server TCP port ---server-mode Listen as server ---use-compression, -c Enable zstd compression ---use-multithreading, -m Enable multithreaded transfer ---use-sendfile, -s Use sendfile() zero-copy TCP ---use-ssh, -S Use SSH transport ---use-tls, -T Enable TLS encryption ---cert TLS certificate file ---key TLS key file ---ca TLS CA certificate file ---insecure Skip TLS verification ---bwlimit Bandwidth limit ---delete Delete files not in source ---include Include filter pattern ---exclude Exclude filter pattern ---dry-run Print what would be transferred ---save-to-disk Save transferred files to disk (for server tests) +--source-dir Source directory +--dest-dir Destination directory on server +--server-host Server IP address (default: 127.0.0.1) +--server-port Server port (default: 8080); --port is an alias +-c, --checksum Verify content by checksum instead of size+mtime +-z, --compress [level] Enable compression (level 1-22, default 5) +-j, --threads[=N] Enable multithreaded scanner/loader/sender pipeline +--chunk-serialization Enable chunk serialization (long form only) +--sendfile sendfile() zero-copy (TCP only; long form only) +-s, --secluded-args Protect-args compatibility option (no effect) +--tls Enable TLS encryption; --cert/--key/--ca give PEMs +--bwlimit Bandwidth limit in kilobytes per second +--delete Delete files on receiver not in source +--incremental Skip files unchanged since last transfer +--delta Delta transfer for changed files (needs --incremental) +-f, --filter=RULE rsync-style filter rule (+/- include/exclude) +--exclude Exclude files matching pattern +--include Only include files matching pattern +-m, --prune-empty-dirs Do not transfer empty directory entries +-n, --dry-run Show what would be transferred +--save-to-disk Write received files to disk --version Print version and exit ---help Print help +--help Show help ``` +> Always confirm the current flags with `./build/client --help`; the table above +> is a representative subset. `src/client/usage.c` is the authoritative list and +> `OPTION_TABLE` in `src/client/client_cli.c` is the parser (there is no `getopt*`). ## Feature Scout Checklist @@ -288,7 +317,7 @@ When using `tea` (the task execution agent) to run CI or tests, always set a suf ## Branch Strategy -Never push directly to `main`. All changes must be developed on a feature branch and merged via a pull request. Always create a new branch (`git checkout -b `) before making changes, push it, and open a PR with `gh pr create --fill`. Wait for CI to pass before merging. +Never push directly to `dev` or `main`. All changes must be developed on a feature branch and merged via a pull request targeting `dev`. Create a branch (`git checkout -b `), push it, and open the PR with `tea pr create --repo TapTap/FastSync --base dev --head `. Wait for CI to pass before merging. ## Dependency Installation diff --git a/.opencode/agents/integrator.md b/.opencode/agents/integrator.md index eea51ae..567729e 100644 --- a/.opencode/agents/integrator.md +++ b/.opencode/agents/integrator.md @@ -35,13 +35,14 @@ mkdir -p /tmp/fastsync_test/src echo "test content" > /tmp/fastsync_test/src/file.txt # Start server -./build/server & +./build/server -p 8080 --allow-unauthenticated & SERVER_PID=$! sleep 0.5 # Run client ./build/client --source-dir /tmp/fastsync_test/src \ --dest-dir /tmp/fastsync_test/dst \ + --server-port 8080 \ --save-to-disk # Verify @@ -156,7 +157,7 @@ When using `tea` (the task execution agent) to run CI or tests, always set a suf ## Branch Strategy -Never push directly to `main`. All changes must be developed on a feature branch and merged via a pull request. Always create a new branch (`git checkout -b `) before making changes, push it, and open a PR with `gh pr create --fill`. Wait for CI to pass before merging. +Never push directly to `dev` or `main`. All changes must be developed on a feature branch and merged via a pull request targeting `dev`. Create a branch (`git checkout -b `), push it, and open the PR with `tea pr create --repo TapTap/FastSync --base dev --head `. Wait for CI to pass before merging. ## Dependency Installation diff --git a/.opencode/agents/issue-creator.md b/.opencode/agents/issue-creator.md index c7296da..65ced1e 100644 --- a/.opencode/agents/issue-creator.md +++ b/.opencode/agents/issue-creator.md @@ -1,5 +1,5 @@ --- -description: Top-level orchestrator that analyzes the FastSync codebase by delegating to specialized sub-agents and creates GitHub issues from their findings. +description: Top-level orchestrator that analyzes the FastSync codebase by delegating to specialized sub-agents and creates Gitea issues from their findings. mode: subagent --- @@ -12,7 +12,7 @@ You are the primary orchestrator agent. Your job is to: 2. Decide which specialized sub-agents to dispatch for analysis 3. Delegate analysis work using the task tool 4. Receive structured findings from sub-agents -5. Create GitHub issues from those findings using `gh issue create` +5. Create Gitea issues from those findings using `tea issues create` 6. Coordinate the overall analysis workflow end-to-end > **Environment rule:** for CI, dependency installation must use the project's custom Docker image (repo-root `Dockerfile`, same as CI). For local development, use `nix-shell` (see `README.md`). See `AGENTS.md`. @@ -97,7 +97,7 @@ First, read the repository structure to understand what exists: ### Phase 2: Determine Analysis Scope Based on what the user requests or what needs attention: - **New features wanted?** → Dispatch `feature-scout` sub-agent -- **Security audit needed?** → Dispatch `security-screener` sub-agent +- **Security audit needed?** → Dispatch `security-auditor` sub-agent - **Code quality review?** → Dispatch `code-quality-guardian` sub-agent - **All of the above?** → Run all three in parallel @@ -110,7 +110,7 @@ Context: ``` ``` -Task: Ask the security-screener agent to analyze the codebase. +Task: Ask the security-auditor agent to analyze the codebase. Context: ``` @@ -138,14 +138,14 @@ Each sub-agent returns findings in this structured format: - **Labels**: comma-separated labels for the issue ``` -### Phase 5: Create GitHub Issues -For each finding, create a GitHub issue: +### Phase 5: Create Gitea Issues +For each finding, create a Gitea issue: ```bash -gh issue create \ +tea issues create --repo TapTap/FastSync \ --title "" \ - --label "" \ - --body "## Description + --labels "" \ + --description "## Description ## Location @@ -175,11 +175,13 @@ _This issue was automatically generated by the issue-creator agent._" ### Duplicate Detection Before creating an issue: -1. Check existing open issues: `gh issue list --state open --label "