Feature: Sparse file handling (--sparse) #186

New Issue

2026-07-30T18:33:38+02:00

TapTap commented

2026-07-30 18:33:38 +02:00

Corresponding rsync flags/behavior:

--sparse — handle sparse files efficiently by skipping holes

Current FastSync behavior:
to_disk() in src/shared/file.c:411 writes the entire file content sequentially with fwrite(). Sparse regions (long runs of zero bytes) are materialized as real disk blocks, wasting space and I/O.

Proposed changes:

CLI flag: --sparse
Config field: bool use_sparse
Implementation:
- Detect runs of zero bytes during file write
- Use lseek(fd, hole_size, SEEK_CUR) or fseek() + ftruncate() to create holes instead of writing zeros
- Alternatively, use SEEK_HOLE/SEEK_DATA on the source to preserve existing holes exactly
- Default behavior remains non-sparse to avoid surprises
Protocol: no wire changes; purely receiver-side write optimization

Priority: low
Rationale: Important for virtual machine images, databases, and other large sparse files. Without it, transfers can inflate disk usage dramatically.

Acceptance criteria:

client --sparse /src /dst produces a sparse destination file when source has holes
Transferred sparse file uses roughly the same disk blocks as the source
Non-sparse default still writes all bytes (backward compatible)
Integration test verifies disk-usage reduction for a sparse file

**Corresponding rsync flags/behavior:** - `--sparse` — handle sparse files efficiently by skipping holes **Current FastSync behavior:** `to_disk()` in `src/shared/file.c:411` writes the entire file content sequentially with `fwrite()`. Sparse regions (long runs of zero bytes) are materialized as real disk blocks, wasting space and I/O. **Proposed changes:** - CLI flag: `--sparse` - Config field: `bool use_sparse` - Implementation: - Detect runs of zero bytes during file write - Use `lseek(fd, hole_size, SEEK_CUR)` or `fseek()` + `ftruncate()` to create holes instead of writing zeros - Alternatively, use `SEEK_HOLE`/`SEEK_DATA` on the source to preserve existing holes exactly - Default behavior remains non-sparse to avoid surprises - Protocol: no wire changes; purely receiver-side write optimization **Priority:** low **Rationale:** Important for virtual machine images, databases, and other large sparse files. Without it, transfers can inflate disk usage dramatically. **Acceptance criteria:** - [ ] `client --sparse /src /dst` produces a sparse destination file when source has holes - [ ] Transferred sparse file uses roughly the same disk blocks as the source - [ ] Non-sparse default still writes all bytes (backward compatible) - [ ] Integration test verifies disk-usage reduction for a sparse file

TapTap added the enhancement needs-triage labels 2026-07-30 18:33:38 +02:00

TapTap closed this issue

2026-07-30 19:38:27 +02:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: TapTap/FastSync#186