Add initial README file

README.md

@@ -1,137 +1,3 @@
-# FastFileTransfer
+# FastSync
 
-A high-performance file synchronization system that implements a custom client-server protocol for efficient file transfer with compression and multithreading support.
+A high-performance file synchronization tool for Nextcloud.
-
-## Technical Overview
-
-FastFileTransfer is a C implementation of a file synchronization system that:
-
-1. Uses a custom TCP-based protocol for client-server communication
-2. Implements chunked file transfer (10MB chunks by default)
-3. Supports zstd compression with configurable levels (1-22)
-4. Utilizes multithreading for parallel file processing
-5. Implements producer-consumer patterns with thread-safe queues
-6. Provides both in-memory and disk-based storage options
-
-## System Architecture
-
-The system consists of two main components:
-
-### Client
-- Scans source directories recursively
-- Creates file chunks with configurable size (10MB default)
-- Compresses data using zstd algorithm
-- Sends files to server using custom protocol
-- Supports both single-threaded and multi-threaded operation
-
-### Server
-- Listens for client connections on port 8080
-- Receives files using the custom protocol
-- Decompresses received data
-- Stores files either in memory or on disk
-- Implements thread pool for parallel processing
-
-## Protocol Details
-
-The client-server communication uses the following status codes:
-- `STATUS_OK`: Operation successful
-- `STATUS_ERROR`: Error occurred
-- `STATUS_FINISHED`: Transfer complete
-- `STATUS_NEXT`: Ready for next chunk
-
-## Configuration Options
-
-### Command Line Arguments
-| Argument | Description |
-|----------|-------------|
-| `-m` | Enable multithreading mode |
-| `-c [level]` | Enable compression with optional level (1-22, default: 5) |
-| `-s` | Enable chunk serialization |
-
-### Environment Variables
-| Variable | Description | Default |
-|----------|-------------|---------|
-| `FASTSYNC_SOURCE_DIR` | Source directory for files | Current user's documents directory |
-| `FASTSYNC_DEST_DIR` | Destination directory | `./data_copied` |
-| `FASTSYNC_SERVER_IP` | Server IP address | `127.0.0.1` |
-| `FASTSYNC_SERVER_PORT` | Server port | `8080` |
-| `FASTSYNC_SAVE_TO_DISK` | Save to disk (true/false) | `false` |
-
-## Implementation Details
-
-### Data Structures
-
-1. **Chunk**: Collection of files (default 10MB total size)
-2. **File**: File metadata with path and content
-3. **FileReceive**: Received file data structure
-4. **Config**: Configuration parameters structure
-5. **Queue**: Thread-safe queue implementation using condition variables
-
-### Key Algorithms
-
-1. **File Scanning**: Recursive directory traversal with BFS
-2. **Chunking**: Files grouped into chunks with size limit
-3. **Compression**: zstd compression with configurable levels
-4. **Network Protocol**: Custom TCP-based protocol with status codes
-5. **Thread Synchronization**: Condition variables and mutexes for thread coordination
-
-## Build Requirements
-
-- C11 compatible compiler
-- CMake 4.1 or later
-- zstd library
-- pthread support
-
-## Building
-
-```bash
-mkdir -p build && cd build
-cmake ..
-make
-```
-
-## Running
-
-### Server
-```bash
-./build/server
-```
-
-### Client
-```bash
-# Basic usage
-./build/client -m -c 10
-```
-
-## Testing
-
-The project includes comprehensive unit tests for core functionality:
-
-```bash
-./build/tests
-```
-
-## Code Organization
-
-```
-src/
-  client/    # Client implementation
-  server/    # Server implementation
-  shared/    # Shared data structures and utilities
-tests/      # Unit tests
-```
-
-## Performance Considerations
-
-1. Chunk size (10MB default) affects memory usage and transfer efficiency
-2. Compression level (1-22) trades CPU usage for space savings
-3. Multithreading improves performance on multi-core systems
-4. Thread-safe queues minimize contention between producer/consumer threads
-
-## Extensibility
-
-The system is designed with clear interfaces that allow for:
-1. Additional compression algorithms
-2. Different transport protocols
-3. Custom storage backends
-4. Extended metadata support

description.md

@@ -0,0 +1,24 @@
+# jetstream - A High-Performance File Transfer Utility
+
+## Features
+
+## Architecture ideas
+
+### Pipeline Stages Client
+
+- Directory Scanning -> Files
+- File Reading -> SendableBuffer
+- optional: Compression -> Sendable Buffer
+- Send Data
+
+### Pipeline Stages Server
+
+- Receive Data -> Sendable Buffer
+- optional: Decompress -> Files
+- File Writing
+
+### Passing Data between Steps
+
+- use Queue
+
+##

opencode.json

@@ -1,12 +0,0 @@
-{
-	"$schema": "https://opencode.ai/config.json",
-	"permission": {
-		"bash": {
-			"*": "allow",
-			"git push origin main": "deny",
-			"git push origin master": "deny",
-			"git push *": "ask",
-			"git commit *": "ask"
-		}
-	}
-}

shell.nix

@@ -1,8 +1,23 @@
-{ pkgs }: {
+{
-  shellHook = ''
+  pkgs ? import <nixpkgs> { },
-    echo "Setting up environment with gh CLI tool"
+}:
-  '';
+
-  buildInputs = [
+pkgs.mkShell {
-    pkgs.gh
+  nativeBuildInputs = with pkgs; [
+    gcc
+    cmake
+    gnumake
+    pkg-config
   ];
+
+  buildInputs = with pkgs; [
+    zstd
+  ];
+
+  NIX_ENFORCE_PURITY = 0;
+
+  shellHook = ''
+    export NIX_ENFORCE_PURITY=0
+    cmake -B build
+  '';
 }

src/shared/chunk.c

@@ -134,93 +134,18 @@ Data *chunk_compress(Chunk *chunk, int compression_level) {
 Chunk *chunk_decompress(Data *compressed_data) {
   log_message(LOG_LEVEL_DEBUG, "Starting to decompress chunk");
   Data *uncompressed_data = data_decompress(compressed_data);
-  if (uncompressed_data == NULL) {
-    log_message(LOG_LEVEL_ERROR, "Failed to decompress chunk data");
-    return NULL;
-  }
-  
   ArrayList *files = array_list_create(file_destroy);
-  char *data_pointer = uncompressed_data->data;
+  size_t *data_pointer = uncompressed_data->data;
-  size_t remaining_size = uncompressed_data->size;
+  while (data_pointer <
+         (size_t *)uncompressed_data->data + uncompressed_data->size) {
+    size_t path_len = data_pointer[0];
 
-  while (remaining_size > 0) {
+    printf("%zu, testing", path_len);
-    if (remaining_size < sizeof(size_t)) {
+    break;
-      log_message(LOG_LEVEL_ERROR, "Invalid chunk format: not enough data for path length");
+  }
-      array_list_destroy(files);
+
-      data_delete(uncompressed_data);
+  log_message(LOG_LEVEL_DEBUG, "Chunk succesfully decompressed");
   return NULL;
-    }
-    
-    size_t path_len = *(size_t *)data_pointer;
-    data_pointer += sizeof(size_t);
-    remaining_size -= sizeof(size_t);
-    
-    if (remaining_size < path_len) {
-      log_message(LOG_LEVEL_ERROR, "Invalid chunk format: not enough data for path");
-      array_list_destroy(files);
-      data_delete(uncompressed_data);
-      return NULL;
-    }
-    
-    char *path = malloc(path_len + 1);
-    if (path == NULL) {
-      perror("Could not allocate memory for file path");
-      array_list_destroy(files);
-      data_delete(uncompressed_data);
-      return NULL;
-    }
-    memcpy(path, data_pointer, path_len);
-    path[path_len] = '\0';
-    data_pointer += path_len;
-    remaining_size -= path_len;
-    
-    if (remaining_size < sizeof(size_t)) {
-      log_message(LOG_LEVEL_ERROR, "Invalid chunk format: not enough data for data size");
-      free(path);
-      array_list_destroy(files);
-      data_delete(uncompressed_data);
-      return NULL;
-    }
-    
-    size_t data_size = *(size_t *)data_pointer;
-    data_pointer += sizeof(size_t);
-    remaining_size -= sizeof(size_t);
-    
-    if (remaining_size < data_size) {
-      log_message(LOG_LEVEL_ERROR, "Invalid chunk format: not enough data for file content");
-      free(path);
-      array_list_destroy(files);
-      data_delete(uncompressed_data);
-      return NULL;
-    }
-    
-    File *file = file_create(path, data_size);
-    if (file == NULL) {
-      free(path);
-      array_list_destroy(files);
-      data_delete(uncompressed_data);
-      return NULL;
-    }
-    
-    memcpy(file->data, data_pointer, data_size);
-    data_pointer += data_size;
-    remaining_size -= data_size;
-    
-    array_list_add(files, file);
-    free(path);
-  }
-  
-  // Create the chunk from the files
-  File **file_array = (File **)array_list_to_array(files);
-  Chunk *chunk = chunk_create(file_array, array_list_size(files));
-  
-  // Clean up
-  free(file_array);
-  array_list_destroy(files);
-  data_delete(uncompressed_data);
-  
-  log_message(LOG_LEVEL_DEBUG, "Chunk successfully decompressed");
-  return chunk;
 }
 
 Data *chunk_data_create(void *data, unsigned long long data_size) {

src/shared/socket.c

@@ -61,7 +61,7 @@ void server_listen(Server *server, void (*handler)(int file_descriptor)) {
   int file_descriptor =
       accept(server->file_descriptor, (struct sockaddr *)&server->address,
              &server->address_length);
-  if (file_descriptor < 0) {
+  if (server->file_descriptor < 0) {
     perror("Could not accept the connection");
     exit(EXIT_FAILURE);
   }