--sparse — handle sparse files efficiently by skipping holes
Current FastSync behavior: to_disk() in src/shared/file.c:411 writes the entire file content sequentially with fwrite(). Sparse regions (long runs of zero bytes) are materialized as real disk blocks, wasting space and I/O.
Proposed changes:
CLI flag: --sparse
Config field: bool use_sparse
Implementation:
Detect runs of zero bytes during file write
Use lseek(fd, hole_size, SEEK_CUR) or fseek() + ftruncate() to create holes instead of writing zeros
Alternatively, use SEEK_HOLE/SEEK_DATA on the source to preserve existing holes exactly
Default behavior remains non-sparse to avoid surprises
Protocol: no wire changes; purely receiver-side write optimization
Priority: low Rationale: Important for virtual machine images, databases, and other large sparse files. Without it, transfers can inflate disk usage dramatically.
Acceptance criteria:
client --sparse /src /dst produces a sparse destination file when source has holes
Transferred sparse file uses roughly the same disk blocks as the source
Non-sparse default still writes all bytes (backward compatible)
Integration test verifies disk-usage reduction for a sparse file
**Corresponding rsync flags/behavior:**
- `--sparse` — handle sparse files efficiently by skipping holes
**Current FastSync behavior:**
`to_disk()` in `src/shared/file.c:411` writes the entire file content sequentially with `fwrite()`. Sparse regions (long runs of zero bytes) are materialized as real disk blocks, wasting space and I/O.
**Proposed changes:**
- CLI flag: `--sparse`
- Config field: `bool use_sparse`
- Implementation:
- Detect runs of zero bytes during file write
- Use `lseek(fd, hole_size, SEEK_CUR)` or `fseek()` + `ftruncate()` to create holes instead of writing zeros
- Alternatively, use `SEEK_HOLE`/`SEEK_DATA` on the source to preserve existing holes exactly
- Default behavior remains non-sparse to avoid surprises
- Protocol: no wire changes; purely receiver-side write optimization
**Priority:** low
**Rationale:** Important for virtual machine images, databases, and other large sparse files. Without it, transfers can inflate disk usage dramatically.
**Acceptance criteria:**
- [ ] `client --sparse /src /dst` produces a sparse destination file when source has holes
- [ ] Transferred sparse file uses roughly the same disk blocks as the source
- [ ] Non-sparse default still writes all bytes (backward compatible)
- [ ] Integration test verifies disk-usage reduction for a sparse file
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Corresponding rsync flags/behavior:
--sparse— handle sparse files efficiently by skipping holesCurrent FastSync behavior:
to_disk()insrc/shared/file.c:411writes the entire file content sequentially withfwrite(). Sparse regions (long runs of zero bytes) are materialized as real disk blocks, wasting space and I/O.Proposed changes:
--sparsebool use_sparselseek(fd, hole_size, SEEK_CUR)orfseek()+ftruncate()to create holes instead of writing zerosSEEK_HOLE/SEEK_DATAon the source to preserve existing holes exactlyPriority: low
Rationale: Important for virtual machine images, databases, and other large sparse files. Without it, transfers can inflate disk usage dramatically.
Acceptance criteria:
client --sparse /src /dstproduces a sparse destination file when source has holes