Compare commits

..

7 Commits

Author SHA1 Message Date
TapTap 80768d64d4 fix: security issues #192 #191 #190 #189 #188 #187 2026-07-30 18:49:25 +02:00
TapTap e876055167 fix: remove Gitea Actions agent workflows (agents run locally, not in CI) 2026-07-30 18:25:45 +02:00
TapTap 3d2eb97205 ci: add automated agent workflows — review, fix, maintenance, batch-merge
CI / lint (push) Successful in 1m4s
CI / sanitizers (address) (push) Successful in 36s
CI / sanitizers (undefined) (push) Successful in 36s
CI / fuzz-build (push) Successful in 14s
CI / coverage (push) Successful in 31s
CI / build-and-test (push) Successful in 1m15s
CI / valgrind (push) Successful in 32s
2026-07-30 18:24:29 +02:00
TapTap da20bb4d37 docs: update workflow — dev branch, automated agents, batch orchestration 2026-07-30 18:23:54 +02:00
TapTap 3e4c571355 Merge pull request 'Comprehensive fix: security, bugs, refactoring, tests, and rsync parity features' (#173) from integration/all-fixes into main
CI / lint (push) Successful in 1m3s
CI / sanitizers (address) (push) Successful in 35s
CI / sanitizers (undefined) (push) Successful in 35s
CI / fuzz-build (push) Successful in 13s
CI / coverage (push) Successful in 31s
CI / build-and-test (push) Successful in 1m14s
CI / valgrind (push) Successful in 32s
2026-07-30 18:13:15 +02:00
TapTap 05dab758cf fix: use lstat instead of stat in receive_thread (symlink security)
CI / lint (pull_request) Successful in 1m4s
CI / sanitizers (address) (pull_request) Successful in 36s
CI / sanitizers (undefined) (pull_request) Successful in 36s
CI / fuzz-build (pull_request) Successful in 14s
CI / coverage (pull_request) Successful in 31s
CI / build-and-test (pull_request) Successful in 1m15s
CI / valgrind (pull_request) Successful in 33s
2026-07-30 18:10:29 +02:00
TapTap 8cc587b79e style: apply clang-format on modified files
CI / lint (pull_request) Successful in 1m5s
CI / sanitizers (undefined) (pull_request) Successful in 37s
CI / sanitizers (address) (pull_request) Successful in 37s
CI / fuzz-build (pull_request) Successful in 13s
CI / coverage (pull_request) Successful in 30s
CI / build-and-test (pull_request) Successful in 1m14s
CI / valgrind (pull_request) Successful in 32s
2026-07-29 20:05:12 +02:00
8 changed files with 175 additions and 49 deletions
+93 -42
View File
@@ -60,48 +60,6 @@ python3 -m pytest tests/ # integration tests
When running the CI workflow via `tea` (the task execution agent), always set a sufficient timeout (e.g., 600000ms) to allow CI to finish. After CI completes, check the results yourself — do not assume success. Use `gh run watch` or similar to monitor CI status, then inspect logs on failure. When running the CI workflow via `tea` (the task execution agent), always set a sufficient timeout (e.g., 600000ms) to allow CI to finish. After CI completes, check the results yourself — do not assume success. Use `gh run watch` or similar to monitor CI status, then inspect logs on failure.
## Branch Strategy
Never push directly to `main`. All changes must be developed on a feature branch and merged via a pull request. Always create a new branch before making changes:
```bash
git checkout -b <feature-branch-name>
```
After committing changes, push the branch and create a PR:
```bash
git push -u origin <feature-branch-name>
gh pr create --fill
```
Wait for CI to pass on the PR before merging.
## Batch PR Workflow
When handling multiple issues split across several PRs that target the same files:
1. **Group issues by logical category** into separate PR branches (e.g., memory-safety, refactoring, test-coverage).
2. **Fix and push** each branch independently. Let CI run on each PR.
3. **Run all 3 reviewer types** on each PR and post results to Gitea via `tea pr approve/reject` or the Gitea API:
- `reviewer` — general code correctness
- `code-quality-guardian` — code quality, duplication, complexity
- `security-auditor` — vulnerability assessment
4. **Iterate**: if any reviewer requests changes, fix, push, re-review. Repeat until all 3 approve.
5. **Merge approved PRs** one at a time into `main`.
6. **Create a combined merge branch** for the remaining PRs that conflict with the new `main`:
```bash
git checkout -b merge-all origin/main
for branch in branch1 branch2 branch3; do
git merge origin/$branch --no-edit || true
# Resolve conflicts, build, test
done
```
7. **Run review again** on the combined branch. Fix issues, push, re-review until approved.
8. **Merge** the combined PR, **close** the redundant individual PRs, and **close all resolved issues** via the Gitea API:
```bash
curl -s -X PATCH -H "Authorization: token $TOKEN" \
-H "Content-Type: application/json" \
-d '{"state":"closed"}' \
"https://gitea.tap-tap.win/api/v1/repos/owner/repo/issues/<number>"
```
## CI Troubleshooting ## CI Troubleshooting
### If lint (clang-format) fails ### If lint (clang-format) fails
@@ -124,6 +82,99 @@ Run locally before pushing:
python3 -m pytest tests/ -v --tb=short python3 -m pytest tests/ -v --tb=short
``` ```
## Branch Strategy
Two main branches: `dev` (integration) and `main` (stable releases).
### Rules
- **All PRs target `dev`** — never target `main` directly
- **`dev` is the default branch** in Gitea repo settings
- **`main` is protected** — only merged from `dev` via PR with 2 approvals + full CI pass
- **Feature/bug branches** branch from `dev`, PR back to `dev`
- **`dev``main` merges** happen on-demand or weekly, requiring full CI + review
```bash
# Start a new feature
git checkout dev && git pull
git checkout -b feat/my-feature
# ... work, commit, push
git push -u origin feat/my-feature
# Create PR targeting dev
```
### Creating the `dev` branch (one-time setup)
```bash
git checkout main && git pull
git checkout -b dev
git push origin dev
# Then in Gitea: Settings → Repository → Default Branch → dev
```
### Branch protection (Gitea repo settings)
**For `dev`:**
- ✅ Require PR for merging
- ✅ Require 1 approval
- ✅ Require status checks (all CI jobs must pass)
- ✅ Delete branch after merge
**For `main`:**
- ✅ Require PR from `dev` only
- ✅ Require CI
- ✅ Require 2 approvals
- ✅ No direct pushes
## Automated Agent Workflows
All agents run locally via the opencode CLI. There is no CI-based agent automation — agents are invoked on-demand by the developer or by this assistant.
### One-command batch workflow
For fixing a set of issues and creating one integration PR:
```bash
# 1. Run each subagent on its category
opencode run --agent security-auditor "Fix all open security issues"
opencode run --agent debugger "Fix all open bugs"
opencode run --agent test-writer "Add missing test coverage"
# 2. The assistant handles: merging branches, fixing CI failures,
# pushing, creating the integration PR, waiting for CI, iterating.
# The developer only reviews the final PR.
```
### Issue triage loop
When you want to fix a batch of issues autonomously:
1. Tell the assistant: *"Fix all open issues and create one big PR"*
2. The assistant delegates to subagents in parallel
3. Merges their branches, handles CI failures iteratively
4. Pushes and opens the final PR
5. You review the PR once CI passes — no intermediate check-ins
### Scheduling
For periodic maintenance (security audits, code quality scans), run:
```bash
opencode run --agent security-auditor "Audit the codebase for vulnerabilities"
opencode run --agent code-quality-guardian "Scan for code quality issues"
```
This can be cron'd locally if desired (e.g., `crontab -e` with `opencode run`).
## Is opencode a good option?
**Yes, for FastSync's needs.** The hybrid model works well:
- opencode's 17 specialized agents handle deep code analysis, fixes, tests, and reviews
- The assistant orchestrates subagents, merges branches, and iterates on CI
- You only review the final output
The key limitation: opencode is session-based, not a persistent daemon. But for the "fix all issues, one PR" workflow, this is fine — the assistant runs the full pipeline in one shot. Persistent webhook-driven automation isn't available for Gitea, but the one-shot batch approach is simpler and gives you full control over what gets merged.
### Recommendations for this project
- **Do** use the batch pattern: delegate to subagents, let the assistant merge + iterate CI, review once
- **Don't** try to run opencode in Gitea Actions — the CI container doesn't have your LLM keys or the interactive context agents need
- **If** you want fully hands-off periodic scans, set up a local cron job or systemd timer that runs `opencode run` and posts results to Gitea via API
## Gitea API & tea CLI ## Gitea API & tea CLI
### Check CI status via API ### Check CI status via API
+54 -2
View File
@@ -345,11 +345,63 @@ ParallelScanner* parallel_scanner_create(char* root_directory, bool use_metadata
char* cur_path = path_cat(root_directory, entry->d_name); char* cur_path = path_cat(root_directory, entry->d_name);
if (!cur_path) if (!cur_path)
continue; continue;
struct stat st; struct stat lstats;
if (stat(cur_path, &st) != 0) { if (lstat(cur_path, &lstats) != 0) {
free(cur_path); free(cur_path);
continue; continue;
} }
bool is_symlink = S_ISLNK(lstats.st_mode);
// Skip symlinks unless the user explicitly enabled following/copying them.
if (is_symlink && !follow_symlinks && !copy_links && !safe_links &&
!copy_unsafe_links) {
free(cur_path);
continue;
}
// --safe-links: reject symlinks pointing outside the source tree.
if (is_symlink && safe_links) {
char link_target[4096];
ssize_t len = readlink(cur_path, link_target, sizeof(link_target) - 1);
if (len < 0) {
free(cur_path);
continue;
}
link_target[len] = '';
if (link_target[0] == '/') {
free(cur_path);
continue;
}
}
// --copy-unsafe-links (without --copy-links): only copy absolute symlinks.
if (is_symlink && copy_unsafe_links && !copy_links) {
char link_target[4096];
ssize_t len = readlink(cur_path, link_target, sizeof(link_target) - 1);
if (len < 0) {
free(cur_path);
continue;
}
link_target[len] = '';
bool unsafe = (link_target[0] == '/');
if (!unsafe) {
free(cur_path);
continue;
}
}
// Determine whether to use lstat or stat results for the entry.
struct stat st;
bool use_lstat_res = is_symlink && follow_symlinks && !copy_links;
if (use_lstat_res) {
st = lstats;
} else {
if (stat(cur_path, &st) != 0) {
free(cur_path);
continue;
}
}
if (S_ISDIR(st.st_mode)) { if (S_ISDIR(st.st_mode)) {
array_list_add(subdirs, cur_path); array_list_add(subdirs, cur_path);
} else { } else {
+2 -1
View File
@@ -126,7 +126,8 @@ void handler(int file_descriptor) {
close(file_descriptor); close(file_descriptor);
return; return;
} }
PipelineContextReceiver* context = pipeline_context_receiver_create(config, q, file_descriptor, ssl); PipelineContextReceiver* context =
pipeline_context_receiver_create(config, q, file_descriptor, ssl);
if (context == NULL) { if (context == NULL) {
queue_destroy(q); queue_destroy(q);
config_delete(config); config_delete(config);
+20
View File
@@ -12,6 +12,9 @@
#include "metadata.h" #include "metadata.h"
#include "protocol.h" #include "protocol.h"
/* Maximum individual file data size within a chunk (64 MB) */
#define MAX_FILE_DATA_SIZE (64ULL * 1024 * 1024)
Chunk* chunk_create(File** items, int element_count) { Chunk* chunk_create(File** items, int element_count) {
Chunk* chunk = (Chunk*)malloc(sizeof(Chunk)); Chunk* chunk = (Chunk*)malloc(sizeof(Chunk));
if (chunk == NULL) { if (chunk == NULL) {
@@ -157,6 +160,14 @@ Chunk* chunk_deserialize(Data* data, bool use_metadata) {
return NULL; return NULL;
} }
// Reject individual file data larger than the maximum allowed size.
if (file_data_size > MAX_FILE_DATA_SIZE) {
log_message(LOG_LEVEL_ERROR, "File data size %zu exceeds maximum %llu",
file_data_size, (unsigned long long)MAX_FILE_DATA_SIZE);
array_list_delete(files);
return NULL;
}
void* file_data = malloc(file_data_size); void* file_data = malloc(file_data_size);
if (file_data == NULL) { if (file_data == NULL) {
perror("Could not allocate memory for file data"); perror("Could not allocate memory for file data");
@@ -210,6 +221,15 @@ Chunk* receive_chunk_data(int fd, const Config* config) {
return NULL; return NULL;
} }
} }
// Reject chunks larger than the maximum allowed size to prevent OOM.
if (data_to_process->size > MAX_CHUNK_SIZE) {
log_message(LOG_LEVEL_ERROR, "Chunk size %zu exceeds maximum %llu",
data_to_process->size, (unsigned long long)MAX_CHUNK_SIZE);
data_destroy(data_to_process);
return NULL;
}
Chunk* chunk = chunk_deserialize(data_to_process, config->use_metadata); Chunk* chunk = chunk_deserialize(data_to_process, config->use_metadata);
data_destroy(data_to_process); data_destroy(data_to_process);
if (chunk == NULL) if (chunk == NULL)
+2 -2
View File
@@ -54,7 +54,7 @@ void pipeline_context_sender_destroy(PipelineContextSender* context) {
} }
PipelineContextReceiver* pipeline_context_receiver_create(Config* config, Queue* queue, PipelineContextReceiver* pipeline_context_receiver_create(Config* config, Queue* queue,
int file_descriptor, SSL* ssl) { int file_descriptor, SSL* ssl) {
PipelineContextReceiver* context = malloc(sizeof(PipelineContextReceiver)); PipelineContextReceiver* context = malloc(sizeof(PipelineContextReceiver));
if (context == NULL) if (context == NULL)
return NULL; return NULL;
@@ -148,7 +148,7 @@ int receive_thread(void* pipeline_context) {
} }
char* full_path = path_cat(config->receive_root_directory, check_path); char* full_path = path_cat(config->receive_root_directory, check_path);
struct stat st; struct stat st;
bool has_old = full_path && stat(full_path, &st) == 0; bool has_old = full_path && lstat(full_path, &st) == 0;
bool match = has_old && (unsigned long long)st.st_size == check_size && bool match = has_old && (unsigned long long)st.st_size == check_size &&
(long long)st.st_mtime == check_mtime; (long long)st.st_mtime == check_mtime;
if (match) if (match)
+1 -1
View File
@@ -40,7 +40,7 @@ PipelineContextSender* pipeline_context_sender_create(Config* config, Queue* que
Queue* queue_loader); Queue* queue_loader);
void pipeline_context_sender_destroy(PipelineContextSender* context); void pipeline_context_sender_destroy(PipelineContextSender* context);
PipelineContextReceiver* pipeline_context_receiver_create(Config* config, Queue* queue_receiver, PipelineContextReceiver* pipeline_context_receiver_create(Config* config, Queue* queue_receiver,
int file_descriptor, SSL* ssl); int file_descriptor, SSL* ssl);
void pipeline_context_receiver_destroy(PipelineContextReceiver* context); void pipeline_context_receiver_destroy(PipelineContextReceiver* context);
int receive_thread(void* pipeline_context); int receive_thread(void* pipeline_context);
int write_thread(void* pipeline_context); int write_thread(void* pipeline_context);
+3
View File
@@ -11,6 +11,9 @@
/* Maximum allowed data payload size for receive_data (100 MB) */ /* Maximum allowed data payload size for receive_data (100 MB) */
#define MAX_DATA_PAYLOAD_SIZE (100ULL * 1024 * 1024) #define MAX_DATA_PAYLOAD_SIZE (100ULL * 1024 * 1024)
/* Maximum chunk size (64 MB) — prevents unbounded allocation from the wire */
#define MAX_CHUNK_SIZE (64ULL * 1024 * 1024)
typedef struct ssl_st SSL; typedef struct ssl_st SSL;
typedef int Status; typedef int Status;
-1
View File
@@ -116,7 +116,6 @@ static void test_receiver_fd_zero() {
pipeline_context_receiver_destroy(ctx); pipeline_context_receiver_destroy(ctx);
} }
/* Test that receive_thread completes cleanly when sent FINISHED immediately */ /* Test that receive_thread completes cleanly when sent FINISHED immediately */
static void test_receive_thread_finished() { static void test_receive_thread_finished() {
Config* cfg = config_create(); Config* cfg = config_create();