5 Commits
Author SHA1 Message Date
TapTap 0de859b302 docs: recount RSYNC_COMPAT summary after Phase-4 wave B (metadata times + hard links)
CI / lint (push) Successful in 53s
CI / sanitizers (address) (push) Successful in 49s
CI / sanitizers (undefined) (push) Successful in 49s
CI / fuzz-build (push) Successful in 20s
CI / coverage (push) Successful in 43s
CI / valgrind (push) Successful in 37s
CI / build-and-test (push) Successful in 4m28s
Wave B moved 6 rows: -U/--atimes, --open-noatime, -H/--hard-links -> ✅;
-N/--crtimes -> ⚠️; -O/--omit-dir-times, -J/--omit-link-times -> 🔄 (no-ops).
Summary: ✅91->94, ⚠️5->6, 🔄1->3, ❌47->41 (Total 147).
2026-09-08 21:02:34 +02:00
TapTap 4e7e84f947 Merge feat/p4-metadata-capture: atimes/crtimes/open-noatime/omit-dir-times/omit-link-times
# Conflicts:
#	src/shared/config.c
#	tests/integration/test_features.py
2026-09-08 21:00:07 +02:00
TapTap cf7cc5b8ca Merge feat/p4-hard-links: -H/--hard-links 2026-09-08 20:59:04 +02:00
TapTap e80888ce7b metadata times: -U/--atimes, -N/--crtimes, --open-noatime, -O/-J
CI / lint (pull_request) Successful in 52s
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
CI / build-and-test (pull_request) Successful in 1m19s
Capture+transmit source atime (pre-read stat; O_NOATIME sender guard) and birth
time (statx STATX_BTIME); receiver restores atime with mtime (crtime not settable
portably -> transmitted, explicitly not applied). --open-noatime is client-only.
-O/-J documented as accepted no-ops (FastSync never preserves dir/symlink times).
Wire: metadata frame gains atime/crtime val+sec+nsec; PROTOCOL_VERSION
2.11.0->2.12.0. Review fixes: gate atime capture to Linux (no epoch clobber on
non-Linux), close fd on fdopen failure, honest -O/-J status (Compat no-op).
2026-09-08 20:58:56 +02:00
TapTap f891cd0a6a hard-links: -H/--hard-links preserves inode relationships
CI / lint (pull_request) Successful in 50s
CI / sanitizers (address) (pull_request) Skipped
CI / sanitizers (undefined) (pull_request) Skipped
CI / fuzz-build (pull_request) Skipped
CI / coverage (pull_request) Skipped
CI / valgrind (pull_request) Skipped
CI / build-and-test (pull_request) Successful in 1m19s
Source files sharing (st_dev,st_ino) are recreated as hard links on the
destination; only the first member's data crosses the wire (siblings ride a
payload-less STATUS_HARDLINK frame). Ordering requires the single-FIFO-writer
receiver + forced sequential scan (documented). link()-failure falls back to a
byte-identical local copy. Rejects -s/--append. PROTOCOL_VERSION 2.11.0->2.12.0.
Review fixes: delete the dead HardLinkRegistry (ordering holds by FIFO writer),
and --existing no longer aborts when the first member is absent but the sibling
exists (leaves the sibling in place).
2026-09-08 20:58:56 +02:00
29 changed files with 1561 additions and 44 deletions

No files matched your search

+96 -10
View File
@@ -6,11 +6,11 @@ This document maps rsync's full feature set to FastSync's current implementation
| Status | Count | Description |
|--------|-------|-------------|
| ✅ Implemented | 91 | Feature works end-to-end |
| ✅ Implemented | 94 | Feature works end-to-end |
| 🔀 Alt Arg | 3 | Functionality exists but under different flag/semantics |
| ⚠️ Partial | 5 | Flag parsed/stored but behavior incomplete |
| 🔄 Compatibility No-op | 1 | Flag is accepted for CLI compatibility but has no effect |
| ❌ Not Implemented | 47 | Flag not recognized or no behavior |
| ⚠️ Partial | 6 | Flag parsed/stored but behavior incomplete |
| 🔄 Compatibility No-op | 3 | Flag is accepted for CLI compatibility but has no effect |
| ❌ Not Implemented | 41 | Flag not recognized or no behavior |
| **Total** | **147** | |
---
@@ -246,25 +246,76 @@ why plain `--append` works on the normal atomic path, not only with `--inplace`.
| `--chmod=CHMOD` | Affect file permissions | ✅ Implemented | Supports numeric and symbolic `ugo` `rwx` changes; retains receiver safety masking |
| `-A`, `--acls` | Preserve ACLs | ❌ Not Implemented | Removed because it had no effect |
| `-X`, `--xattrs` | Preserve extended attributes | ❌ Not Implemented | Removed because it had no effect |
| `-H`, `--hard-links` | Preserve hard links | ❌ Not Implemented | Removed because it had no effect |
| `-H`, `--hard-links` | Preserve hard links | ✅ Implemented | Files on the source that share an inode (`st_dev`+`st_ino`, e.g. a `cp -al` tree) are re-created as hard links to one another on the destination, so duplicate links stay deduplicated and only the first member's data is sent (later members are transmitted as payload-less `STATUS_HARDLINK` frames). The receiver links each sibling to the first member's installed file with an atomic link + rename; on `link()` failure it falls back to a byte-identical local copy of the first member, never a partial/corrupt file. Requires the sequential scan for ordering (the first member is always emitted and installed before any sibling is linked). Works single-threaded and under `-m`, `--inplace`, `--delay-updates` (links staged and published by rename) and `--partial`. Crosses the wire (`preserve_hard_links` bool; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0**, peers must match). Incompatible with `-s` (chunk serialization) and `--append`/`--append-verify`, rejected up front with a distinct error. See the Phase-4 hard-links notes below |
| `-D` | Same as --devices --specials | ❌ Not Implemented | Removed because device-file handling is not implemented |
| `--devices` | Preserve device files | ❌ Not Implemented | Removed because it had no effect |
| `--specials` | Preserve special files | ❌ Not Implemented | |
| `--copy-devices` | Copy device contents as file | ❌ Not Implemented | |
| `--write-devices` | Write to devices as files | ❌ Not Implemented | |
| `-U`, `--atimes` | Preserve access times | ❌ Not Implemented | |
| `-N`, `--crtimes` | Preserve create times | ❌ Not Implemented | |
| `-O`, `--omit-dir-times` | Omit dirs from --times | ❌ Not Implemented | |
| `-J`, `--omit-link-times` | Omit symlinks from --times | ❌ Not Implemented | |
| `-U`, `--atimes` | Preserve access times | ✅ Implemented | Captures the source access time (from the scanner's pre-read stat, so it is not clobbered by reading the file for transfer) and transmits it over the wire; the receiver restores it together with the mtime via `futimens`/`utimensat`. Implies metadata transmission (the times travel inside the `-M` metadata payload), but does not enable ownership application (that stays opt-in via the identity flags). Wire: new `atime` fields on the metadata frame + a `preserve_atimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** |
| `-N`, `--crtimes` | Preserve create times | ⚠️ Partial | Captures the source birth time via `statx(STATX_BTIME)` on Linux and transmits it (recorded as a wire field), but there is **no portable way to set a birth time** (`utimensat` can only set atime/mtime), so the receiver explicitly does NOT apply it: it logs a debug note and continues — never failing the transfer and never pretending it worked. On platforms without `statx` it parses as a documented no-op (flag accepted; nothing is captured). Implies metadata transmission. Wire: new `crtime` fields + a `preserve_crtimes` config boolean; `PROTOCOL_VERSION` bumped **2.11.0 → 2.12.0** (see the Phase-4 metadata-time notes) |
| `-O`, `--omit-dir-times` | Omit dirs from --times | 🔄 Compatibility No-op | Accepted and parsed for CLI compatibility, and the config boolean crosses the wire, but it has **no effect**: FastSync never preserves directory mtimes in the first place (directories are created via `mkdir` with no metadata, a documented divergence under `-d`/recursive), so there is nothing for an "omit" to suppress. It never breaks a normal run |
| `-J`, `--omit-link-times` | Omit symlinks from --times | 🔄 Compatibility No-op | Accepted and parsed for CLI compatibility, and the config boolean crosses the wire, but it has **no effect**: FastSync never sets symlink times (`-l`/`--links` still only includes symlinks without transmitting a target; `--copy-links` dereferences), so there is nothing for an "omit" to suppress. It never breaks a normal run |
| `--super` | Receiver attempts super-user activities | ❌ Not Implemented | |
| `--fake-super` | Store/recover privileged attrs via xattrs | ❌ Not Implemented | |
| `--open-noatime` | Avoid changing access time when opening files | ❌ Not Implemented | |
| `--open-noatime` | Avoid changing access time when opening files | ✅ Implemented | Sender-side policy: the sender opens source files with `O_NOATIME` (Linux) when reading them for transfer, so the open/read does NOT bump the source's on-disk access time. Degrades safely when `O_NOATIME` is unavailable (not defined) or refused (`EPERM`, since it needs `CAP_FOWNER` or file ownership): the code falls back to a normal open, so the data always transfers — only the atime-bump is skipped. It does not itself capture/preserve atime; it only avoids modifying it. **Client-only, never crosses the wire.** Exposed as `file_open_for_read()` and applied to both the buffered data path and the sendfile path |
| `--numeric-ids` | Do not map uid/gid by name | ✅ Implemented | Ownership is applied through FastSync's opt-in identity path (see the Phase-4 identity notes below). `--numeric-ids` is a mapping-policy modifier: when applying ownership it uses the transmitted numeric uid/gid directly, skipping the name lookup. Without an ownership-affecting option it is inert (FastSync only applies ownership when the user opts in). It does not need `-M` to be parsed, but ownership is only applied when metadata (hence the source uid/gid) is actually transmitted (see the notes) |
| `--usermap=STRING` | Map usernames | ✅ Implemented | Opt-in ownership application. rsync subset implemented: comma-separated `FROM:TO` rules evaluated in order, first match wins; `FROM`/`TO` are group/user names (resolved on the SOURCE machine at parse time), `*` (FROM matches any id / TO = the receiving process's current euid), and an `@N` or bare `N` numeric id. Rules are carried over the wire as resolved numeric id pairs; the receiver applies a matching rule (else falls back to `--chown`, `--numeric-ids`, then a best-effort name lookup) via an fd-relative `fchown`. Malformed/unresolvable specs are rejected with a clear error, never a silent no-op. Implies metadata preservation so the source uid/gid travel. Only effective when the receiver can actually change ownership (root or membership); otherwise it warns and continues |
| `--groupmap=STRING` | Map group names | ✅ Implemented | Same rsync subset and semantics as `--usermap` but for the group (gid) side and the group databases. See the Phase-4 identity notes |
| `--chown=USER:GROUP` | Map owner and group | ✅ Implemented | Opt-in ownership override applied receiver-side. Forms: `USER:GROUP`, `USER` (owner only), `:GROUP` (group only); a `*` for USER/GROUP means the current/root user or group as appropriate; an `@N`/bare `N` numeric id is accepted. A `:` inside a name may be escaped as `\:`. Equivalent to a trailing `*:*` usermap+groupmap rule (so an explicit `--usermap`/`--groupmap` match wins). Malformed or unresolvable specs are clear parse errors. Implies metadata preservation. Only effective when the receiver has permission to chown; otherwise it warns and continues (rsync parity) |
| `--copy-as=USER[:GROUP]` | Perform the copy as another user/group | ❌ Not Implemented | |
**Phase-4 metadata-time notes:** `-U/--atimes`, `-N/--crtimes`,
`-O/--omit-dir-times`, `-J/--omit-link-times`, and `--open-noatime` are new.
They change the wire: the per-file metadata frame grows `atime_valid` +
`atime_sec` + `atime_nsec` and `crtime_valid` + `crtime_sec` + `crtime_nsec`
(appended after the existing mode/uid/gid/mtime fields, preserving the exact
positions of every pre-existing field), and the config frame grows four
booleans — `preserve_atimes`, `preserve_crtimes`, `omit_dir_times`,
`omit_link_times` — that CROSS the wire so the receiver knows what to apply /
suppress. `--open-noatime` is **client-only** and is never serialized (it only
governs the sender's source reads). `PROTOCOL_VERSION` was bumped **2.11.0 →
2.12.0** (peers must match, exactly as prior phases did).
**Client-vs-wire split:** `-U` and `-N` affect both the sender (capture) and the
receiver (apply), so they and their metadata fields cross the wire;
`-O`/`-J` are receiver-side preferences and cross as config booleans;
`--open-noatime` is purely a client/sender open flag and stays off the wire
(mirroring the existing convention where `ignore_errors` is client-only while
`force_delete` crosses the wire).
**atime capture does not clobber the source atime:** the sender records the
access time from the **same pre-read stat the scanner already took** (inside
`file_metadata_create`), before any file data is read for transfer. So `-U`
alone captures the correct atime even without `--open-noatime`. `--open-noatime`
is orthogonal: it keeps the source's on-disk atime from being bumped by the read
that actually ships the data (only honoured where `O_NOATIME` works; it degrades
to a normal open otherwise, so the data always transfers).
**crtime handling:** `-N` captures the source birth time via `statx`/`STATX_BTIME`
(guarded `#ifdef STATX_BTIME` on Linux) and transmits it. On the receiver, **no
portable setter exists** (`utimensat` can only set atime/mtime), so the receiver
deliberately does **not** apply it: it logs a debug note and continues — it never
fails the transfer and never pretends the crtime was applied. This is the
explicit, documented unsupported-attribute handling. On platforms without
`statx` the flag is accepted but nothing is captured (a documented no-op).
**omit-dir-times / omit-link-times:** `-O` and `-J` are **accepted and parsed
for CLI compatibility** and their config booleans cross the wire, but they are
genuine **no-ops**: FastSync does not apply directory or symlink times at all
(directories are made via `mkdir` with no metadata; symlinks are dereferenced
or skipped, never written with a target), so there is nothing for an "omit" to
suppress. They never break a normal run. This is documented as a
divergence — the flags recognize the rsync interface but have no filtering
effect in FastSync.
**-U/-N and -M interaction:** because FastSync carries all metadata (mode, uid,
gid, mtime, and now atime/crtime) in one bounded payload that is only sent when
metadata transmission is on, `-U` and `-N` imply metadata transmission (the
times travel inside that payload). They do **not** enable ownership application,
which remains opt-in strictly through the identity flags (`--numeric-ids` /
`--usermap` / `--groupmap` / `--chown`).
**Phase-4 identity notes:** `--numeric-ids`, `--usermap`, `--groupmap`, and
`--chown` are real. They introduce a **controlled, opt-in, privilege-gated**
ownership-application path on the receiver: plain `-M`/`--preserve` still does
@@ -306,6 +357,41 @@ unlike rsync, plain `-M` never applies ownership and `--usermap`/`--groupmap`/
`--chown` each imply metadata preservation so the source uid/gid actually travel
(the flags only take effect where ownership is being preserved/applied).
**Phase-4 hard-links notes:** `-H`/`--hard-links` is real and introduces a
deduplicating wire path for files whose source entries share a filesystem inode.
On the sender, the scanner records each distinct `(st_dev, st_ino)` encounter and
assigns it a stable, run-local link-group id (`HardLinkTable`, mutex-guarded so a
multi-threaded scan could share one instance). The FIRST member of a group is
transferred normally and carries the data; each later (sibling) member is
transmitted as a payload-less `STATUS_HARDLINK` frame carrying its destination
path, the group id, and the first member's destination-relative wire path.
Ordering is guaranteed by forcing the sequential scanner whenever `-H` is on
(even under `-m`), so the first member is always emitted — and, on the receiver's
single write thread, installed — before any of its siblings; the receiver is
therefore always able to link to an already-present first member, including the
"first member already up-to-date/skipped" case (the sibling links to or copies
the existing file). Asymmetric existence policies are handled gracefully: under
`--existing`, if the first member's destination is absent (so it is skipped) but
a sibling's own destination already exists, that existing sibling is left in
place rather than the transfer aborting on the missing first member. The receiver
installs each sibling beneath its confined root
as an atomic hard link (temp link + rename); when `link()` fails (cross-device,
filesystem refuses links) it falls back to a byte-identical local copy of the
first member, never a partial/corrupt file. `--delay-updates` stages each sibling
as a hard link to the first member's STAGED file, so publication's renames
preserve the shared inode; `--inplace` and `--partial` are unaffected (a sibling
is a fresh link/copy). Because a hard link shares an inode, metadata is applied
exactly once on the first member and never re-written through the sibling (whose
members are byte-identical by construction), so all members agree.
Wire/version: `PROTOCOL_VERSION` was bumped **2.11.0 → 2.12.0** (peers must
match). The config frame already carried the `preserve_hard_links` boolean
(round-trips through `config_send`/`config_receive`); the only new wire element
is the `STATUS_HARDLINK` frame described above. Incompatibilities (rejected up
front with a distinct error on the client, and re-checked on receive): `-H` with
`-s` chunk serialization (the chunk wire has no per-file hard-link info) and `-H`
with `--append`/`--append-verify` (a payload-less sibling cannot be tail-resumed).
## 9. Symlink Handling
| Flag | Rsync Description | FastSync Status | Notes |
+20
View File
@@ -4,6 +4,7 @@
#include "compression.h"
#include "config.h"
#include "delta.h"
#include "file.h"
#include "file_list.h"
#include "filter.h"
#include "identity.h"
@@ -475,6 +476,7 @@ static const OptionEntry OPTION_TABLE[] = {
{"--copy-links", NULL, OPT_FLAG, offsetof(Config, copy_links)},
{"--safe-links", NULL, OPT_FLAG, offsetof(Config, safe_links)},
{"--copy-unsafe-links", NULL, OPT_FLAG, offsetof(Config, copy_unsafe_links)},
{"--hard-links", "-H", OPT_FLAG, offsetof(Config, preserve_hard_links)},
{"--sparse", "-S", OPT_FLAG, offsetof(Config, preserve_sparse)},
{"--inplace", NULL, OPT_FLAG, offsetof(Config, inplace)},
{"--preallocate", NULL, OPT_FLAG, offsetof(Config, preallocate)},
@@ -533,6 +535,11 @@ static const OptionEntry OPTION_TABLE[] = {
{"--cvs-exclude", "-C", OPT_FLAG, offsetof(Config, cvs_exclude)},
{"-F", NULL, OPT_FLAG, offsetof(Config, per_dir_filter)},
{"--numeric-ids", NULL, OPT_FLAG, offsetof(Config, numeric_ids)},
{"--atimes", "-U", OPT_FLAG, offsetof(Config, preserve_atimes)},
{"--crtimes", "-N", OPT_FLAG, offsetof(Config, preserve_crtimes)},
{"--omit-dir-times", "-O", OPT_FLAG, offsetof(Config, omit_dir_times)},
{"--omit-link-times", "-J", OPT_FLAG, offsetof(Config, omit_link_times)},
{"--open-noatime", NULL, OPT_FLAG, offsetof(Config, open_noatime)},
};
/* Only boolean options with no required argument are safe to negate. */
@@ -552,6 +559,7 @@ static const NegatableOption NEGATABLE_OPTIONS[] = {
{"copy-links", NULL, offsetof(Config, copy_links)},
{"safe-links", NULL, offsetof(Config, safe_links)},
{"copy-unsafe-links", NULL, offsetof(Config, copy_unsafe_links)},
{"hard-links", "H", offsetof(Config, preserve_hard_links)},
{"sparse", "S", offsetof(Config, preserve_sparse)},
{"inplace", NULL, offsetof(Config, inplace)},
{"preallocate", NULL, offsetof(Config, preallocate)},
@@ -794,6 +802,14 @@ int parse_args(Config* config, int argc, char* argv[], int* positional_args,
config. */
if (entry->offset == offsetof(Config, delete_missing_args))
config->ignore_missing_args = true;
/* -U/--atimes and -N/--crtimes carry their times inside the metadata
payload, which is only transmitted when use_metadata is set, so either
one implies metadata transmission. This is FastSync's broad -M bundle
(mode/mtime travel too); it does NOT enable ownership application,
which stays opt-in via the identity flags. */
if (entry->offset == offsetof(Config, preserve_atimes) ||
entry->offset == offsetof(Config, preserve_crtimes))
config->use_metadata = true;
continue;
}
@@ -1339,6 +1355,10 @@ int main(int argc, char* argv[]) {
goto cleanup;
}
/* --open-noatime is a sender-side policy: install it for every source read
(scan + data path) without touching the receiver. */
file_set_open_noatime(config->open_noatime);
/* Initialize TLS if needed */
if (config->use_tls)
tls_global_init();
+48 -7
View File
@@ -9,6 +9,7 @@
#include "file.h"
#include "file_list.h"
#include "filter.h"
#include "hardlink.h"
#include "metadata.h"
#include "log.h"
#include "multiprocessing.h"
@@ -44,10 +45,13 @@ static const char* display_bytes(unsigned long long bytes, bool human_readable,
/* Compiled scanner inputs that are shared read-only across scanner instances
* and, in -m mode, across worker threads. `base_filters` owns the compiled
* command-line + -C rules; the FileListSet allow-set lives in the Config. */
* command-line + -C rules; the FileListSet allow-set lives in the Config.
* `hardlinks` owns the --hard-links/-H link-group detection table (NULL when
* off) and is shared (mutex-guarded) across every scanner/worker of one scan. */
typedef struct {
ScannerOptions options;
FilterRuleList* base_filters; /* owned; may be NULL */
HardLinkTable* hardlinks; /* owned; may be NULL */
} PreparedScanner;
/* Build the scanner options for one scan. Returns false and logs on failure. */
@@ -55,6 +59,7 @@ static bool prepare_scanner(const Config* config, int num_threads, PreparedScann
if (!out)
return false;
out->base_filters = NULL;
out->hardlinks = NULL;
memset(&out->options, 0, sizeof(out->options));
int rule_count = config->filters ? config->filters->size : 0;
@@ -82,6 +87,8 @@ static bool prepare_scanner(const Config* config, int num_threads, PreparedScann
ScannerOptions* options = &out->options;
options->use_metadata = config->use_metadata;
options->preserve_atimes = config->preserve_atimes;
options->preserve_crtimes = config->preserve_crtimes;
options->chunk_size = config->chunk_size;
options->exclude_patterns = config->exclude_patterns;
options->exclude_count = config->exclude_count;
@@ -107,6 +114,16 @@ static bool prepare_scanner(const Config* config, int num_threads, PreparedScann
options->ignore_missing_args = config->ignore_missing_args || config->delete_missing_args;
options->excluded_paths = NULL;
options->excluded_mutex = NULL;
options->hardlinks = NULL;
if (config->preserve_hard_links) {
out->hardlinks = hardlink_table_create();
if (!out->hardlinks) {
filter_rule_list_free(out->base_filters);
out->base_filters = NULL;
return false;
}
options->hardlinks = out->hardlinks;
}
return true;
}
@@ -115,6 +132,8 @@ static void prepared_scanner_destroy(PreparedScanner* prepared) {
return;
filter_rule_list_free(prepared->base_filters);
prepared->base_filters = NULL;
hardlink_table_destroy(prepared->hardlinks);
prepared->hardlinks = NULL;
}
/* True when some --files-from entry is an ancestor-or-equal directory of
@@ -1216,6 +1235,19 @@ static int send_chunk_with_removal(Client* client, Chunk* chunk, Config* config,
change_emit_dir_sent(config, f);
continue;
}
/* --hard-links/-H sibling: a later member of a hard-link group that has no
data (its payload lives in the first member). Transmit a dedicated
STATUS_HARDLINK frame carrying the first member's destination-relative
wire path so the receiver links this entry to that installed file. */
if (f->link_group != 0 && !f->link_first && f->hardlink_target != NULL) {
if (!send_status(client->file_descriptor, STATUS_HARDLINK) ||
!send_str(client->file_descriptor, file_wire_path(f)) ||
!send_int(client->file_descriptor, f->link_group) ||
!send_str(client->file_descriptor, f->hardlink_target))
return -1;
change_emit_file_sent(config, f);
continue;
}
bool stream = f->data->data == NULL && f->data->size > 0;
bool use_sendfile =
(config->use_sendfile && !config->use_compression) || (stream && !config->use_compression);
@@ -1385,9 +1417,18 @@ static int scan_directory_multithreaded(void* pipeline_context) {
if (!context->early_delete)
prepared.options.excluded_paths = context->excluded_paths;
bool dirs_mode = prepared.options.dirs;
/* -H also selects the sequential scanner (see the comment at the branch),
* so the loop below must choose the scanner by which object exists, not by
* --dirs alone. */
bool use_dscanner = dirs_mode || prepared.options.hardlinks;
DirectoryScanner* dscanner = NULL;
ParallelScanner* scanner = NULL;
if (dirs_mode) {
/* --hard-links/-H forces the sequential scanner even in -m mode: a hard-link
group's first member must be emitted before any of its siblings so the
receiver always links to an already-installed first member. The parallel
scanner hands different subdirectories to different worker threads, which
can reorder a group whose members span directories. */
if (use_dscanner) {
dscanner =
directory_scanner_create_with_options(context->config->send_directory, &prepared.options);
} else {
@@ -1404,12 +1445,12 @@ static int scan_directory_multithreaded(void* pipeline_context) {
bool failed = false;
Chunk* current_chunk;
while (1) {
if (dirs_mode)
if (use_dscanner)
current_chunk = directory_scanner_next(dscanner);
else
current_chunk = parallel_scanner_next(scanner);
if (current_chunk == NULL) {
failed = dirs_mode ? directory_scanner_failed(dscanner) : parallel_scanner_failed(scanner);
failed = use_dscanner ? directory_scanner_failed(dscanner) : parallel_scanner_failed(scanner);
break;
}
if (context->config->use_delete && !context->early_delete) {
@@ -1434,9 +1475,9 @@ static int scan_directory_multithreaded(void* pipeline_context) {
/* Capture the scanner results BEFORE destroying the scanner objects (the
io_error flag lives on the scanner, so reading it after destroy would be a
use-after-free). */
bool had_io =
dirs_mode ? directory_scanner_had_io_error(dscanner) : parallel_scanner_had_io_error(scanner);
if (dirs_mode)
bool had_io = use_dscanner ? directory_scanner_had_io_error(dscanner)
: parallel_scanner_had_io_error(scanner);
if (use_dscanner)
directory_scanner_destroy(dscanner);
else
parallel_scanner_destroy(scanner);
+15
View File
@@ -69,6 +69,21 @@ bool validate_config(const Config* config) {
"full transfer)");
return false;
}
/* --hard-links/-H transmits each later group member as a dedicated per-file
STATUS_HARDLINK frame, which chunk serialization -s does not support; and a
hard-links sibling carries no payload, so the tail-resume of --append is
meaningless for it. Both combinations are rejected up front rather than
silently degrading. */
if (config->preserve_hard_links && config->use_chunk_serialization) {
log_message(LOG_LEVEL_ERROR,
"--hard-links/-H cannot be combined with -s (chunk serialization)");
return false;
}
if (config->preserve_hard_links && (config->append || config->append_verify)) {
log_message(LOG_LEVEL_ERROR,
"--hard-links/-H cannot be combined with --append/--append-verify");
return false;
}
if (config->log_file_format && !config->log_file) {
log_message(LOG_LEVEL_ERROR, "--log-file-format requires --log-file");
return false;
+59 -4
View File
@@ -165,6 +165,34 @@ static bool entry_passes_selection(const FileListSet* file_list, const FilterRul
return true;
}
/* Apply --hard-links (-H) detection to one regular File. On a sibling (a
* later member of an already-seen source inode) the File keeps the group id
* and the first member's wire path but carries NO data payload (size 0); the
* first member is left untouched (data present, link_first). Allocation
* failure is fatal: the scanner is marked failed. */
static void scanner_assign_hardlink(DirectoryScanner* scanner, HardLinkTable* table, File* file,
const struct stat* stats) {
if (!table || !file || !stats)
return;
int gid;
bool is_first;
char* first_path = NULL;
if (!hardlink_table_assign(table, file_wire_path(file), stats->st_dev, stats->st_ino, &gid,
&is_first, &first_path)) {
if (scanner)
scanner->failed = true;
return;
}
file->link_group = gid;
file->link_first = is_first;
if (!is_first) {
file->hardlink_target = first_path;
file->data->size = 0;
} else {
free(first_path);
}
}
/* Append `rel` to the caller's exclusion sink, taking `mtx` when shared across
parallel worker threads. Returns false on allocation failure (list left
unchanged). */
@@ -320,6 +348,8 @@ DirectoryScanner* directory_scanner_create_with_options(const char* root_directo
scanner->current_dir = NULL;
scanner->current_path = NULL;
scanner->use_metadata = options->use_metadata;
scanner->preserve_atimes = options->preserve_atimes;
scanner->preserve_crtimes = options->preserve_crtimes;
scanner->chunk_size = options->chunk_size > 0 ? options->chunk_size : DESIRED_CHUNK_SIZE;
scanner->exclude_patterns = options->exclude_patterns;
scanner->exclude_count = options->exclude_count;
@@ -356,6 +386,7 @@ DirectoryScanner* directory_scanner_create_with_options(const char* root_directo
scanner->io_error = false;
scanner->dirs_mode = options->dirs;
scanner->relative_mode = options->relative && options->file_list != NULL;
scanner->hardlinks = options->hardlinks;
scanner->prune_empty_dirs = options->prune_empty_dirs;
scanner->dirs_root_emitted = false;
scanner->list_index = 0;
@@ -564,7 +595,8 @@ static File* dirs_root_dir_file(DirectoryScanner* scanner) {
}
file->is_dir = true;
if (scanner->use_metadata) {
file->metadata = file_metadata_create(&st);
file->metadata = file_metadata_create(scanner->root_path, &st, scanner->preserve_atimes,
scanner->preserve_crtimes);
if (!file->metadata) {
file_destroy(file);
scanner->failed = true;
@@ -641,7 +673,8 @@ static File* dirs_file_for_entry(DirectoryScanner* scanner, const char* entry) {
}
}
if (scanner->use_metadata) {
file->metadata = file_metadata_create(&effective);
file->metadata = file_metadata_create(file->path, &effective, scanner->preserve_atimes,
scanner->preserve_crtimes);
if (!file->metadata) {
file_destroy(file);
scanner->failed = true;
@@ -892,8 +925,11 @@ Chunk* directory_scanner_next(DirectoryScanner* scanner) {
file->send_path = rel_copy;
rel_copy = NULL;
}
if (scanner->hardlinks && S_ISREG(stats.st_mode))
scanner_assign_hardlink(scanner, scanner->hardlinks, file, &stats);
if (scanner->use_metadata)
file->metadata = file_metadata_create(&stats);
file->metadata = file_metadata_create(file->path, &stats, scanner->preserve_atimes,
scanner->preserve_crtimes);
if (scanner->use_metadata && !file->metadata) {
free(rel_copy);
file_destroy(file);
@@ -1207,8 +1243,27 @@ static void scan_root_entry(const ScannerOptions* options, const FilterNode* roo
file->send_path = rel;
rel = NULL;
}
if (options->hardlinks && S_ISREG(st.st_mode)) {
int gid;
bool is_first;
char* first_path = NULL;
if (!hardlink_table_assign((HardLinkTable*)options->hardlinks, file_wire_path(file), st.st_dev,
st.st_ino, &gid, &is_first, &first_path)) {
ps->failed = true;
} else {
file->link_group = gid;
file->link_first = is_first;
if (!is_first) {
file->hardlink_target = first_path;
file->data->size = 0;
} else {
free(first_path);
}
}
}
if (options->use_metadata)
file->metadata = file_metadata_create(&st);
file->metadata =
file_metadata_create(file->path, &st, options->preserve_atimes, options->preserve_crtimes);
if (options->use_metadata && !file->metadata) {
free(rel);
file_destroy(file);
+15
View File
@@ -4,6 +4,7 @@
#include "chunk.h"
#include "file_list.h"
#include "filter.h"
#include "hardlink.h"
#include "protocol.h"
#include "queue.h"
#include <dirent.h>
@@ -14,6 +15,10 @@
typedef struct {
bool use_metadata;
/* Phase 4 metadata capture: -U/--atimes and -N/--crtimes tell the scanner to
* capture the source access / birth time into each entry's FileMetadata. */
bool preserve_atimes;
bool preserve_crtimes;
unsigned long long chunk_size;
char** exclude_patterns;
int exclude_count;
@@ -64,6 +69,11 @@ typedef struct {
* instead of failing (the --dirs generator is the only scanner path that
* observes a listed-but-missing entry). */
bool ignore_missing_args;
/* --hard-links (-H): shared, mutable (mutex-guarded) link-group detection
* table, NULL when -H is off. Owned by the caller (client_send), shared
* read-only here; the parallel scanner passes it unchanged to every worker so
* one table detects every group across all subdirectories. */
HardLinkTable* hardlinks;
} ScannerOptions;
/* Internal per-scanner filter state. FilterNode chains represent the ordered
@@ -75,6 +85,8 @@ typedef struct {
DIR* current_dir;
char* current_path;
bool use_metadata;
bool preserve_atimes;
bool preserve_crtimes;
unsigned long long chunk_size;
char** exclude_patterns;
int exclude_count;
@@ -124,6 +136,9 @@ typedef struct {
--ignore-errors the scan continues past it and the caller decides what to
do; `failed` is reserved for fatal errors that always abort the scan. */
bool io_error;
/* --hard-links (-H): shared link-group detection table (see ScannerOptions).
NULL when -H is off. */
HardLinkTable* hardlinks;
} DirectoryScanner;
typedef struct {
+1
View File
@@ -180,6 +180,7 @@ void print_usage(void) {
printf(" --copy-links Transform symlinks into referent files\n");
printf(" --safe-links Skip symlinks that point outside transfer tree\n");
printf(" --copy-unsafe-links Only transform unsafe symlinks into referent files\n");
printf(" -H, --hard-links Preserve hard-link relationships across the transfer\n");
printf(" -S, --sparse Handle sparse files efficiently\n");
printf(" --inplace Update files in-place (no temp+rename)\n");
printf(
+5 -1
View File
@@ -154,7 +154,7 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver
DeleteManifest* deferred_manifest = NULL;
while (status == STATUS_NEXT || status == STATUS_CHUNK || status == STATUS_CHECK ||
status == STATUS_KEEPALIVE || status == STATUS_ABORT || status == STATUS_CHECK_BATCH ||
status == STATUS_MKDIR || status == STATUS_MANIFEST) {
status == STATUS_MKDIR || status == STATUS_MANIFEST || status == STATUS_HARDLINK) {
if (status == STATUS_KEEPALIVE) {
if (!send_status(file_descriptor, STATUS_KEEPALIVE))
goto fail;
@@ -181,6 +181,10 @@ int receiver_process_pending(Config* config, int file_descriptor, const Receiver
File* dir = file_receive_directory(file_descriptor);
if (!dir || !sink->store_file(dir, sink->context))
goto receive_error;
} else if (status == STATUS_HARDLINK) {
File* file = file_receive_hardlink(file_descriptor);
if (!file || !sink->store_file(file, sink->context))
goto receive_error;
} else if (status == STATUS_MANIFEST) {
DeleteManifest* manifest = receive_manifest_entries(file_descriptor);
if (!manifest)
+29 -2
View File
@@ -148,6 +148,11 @@ static void config_set_defaults(Config* config) {
config->groupmap = NULL;
config->groupmap_count = 0;
config->delay_context = NULL;
config->preserve_atimes = false;
config->preserve_crtimes = false;
config->omit_dir_times = false;
config->omit_link_times = false;
config->open_noatime = false;
}
static bool valid_wire_bool(int value) {
@@ -195,6 +200,10 @@ static bool validate_received_config(const Config* config) {
which chunk serialization -s disables: reject on the receiver too
so a -s sender cannot negotiate an inert append mode. */
!((config->append || config->append_verify) && config->use_chunk_serialization) &&
!(config->preserve_hard_links && config->use_chunk_serialization) &&
!(config->preserve_hard_links && (config->append || config->append_verify)) &&
valid_wire_bool(config->preserve_atimes) && valid_wire_bool(config->preserve_crtimes) &&
valid_wire_bool(config->omit_dir_times) && valid_wire_bool(config->omit_link_times) &&
(!config->use_compression ||
(config->compression_level >= 1 && config->compression_level <= 22)) &&
config->chunk_size > 0 && config->chunk_size <= MAX_CHUNK_SIZE &&
@@ -746,6 +755,22 @@ static bool receive_identity_options(int fd, Config* c) {
receive_identity_map(fd, &c->groupmap_count, &c->groupmap);
}
/* -U/--atimes, -N/--crtimes (affect both sender capture and receiver apply)
* and -O/--omit-dir-times, -J/--omit-link-times (receiver-side prefs) all cross
* the wire so the receiver knows what to apply / suppress. --open-noatime is
* client-only (it only governs the sender's source reads) and is never
* serialized. Trailing fields; protocol 2.12.0. */
static bool send_metadata_times_options(int fd, const Config* c) {
return send_int(fd, c->preserve_atimes) && send_int(fd, c->preserve_crtimes) &&
send_int(fd, c->omit_dir_times) && send_int(fd, c->omit_link_times);
}
static bool receive_metadata_times_options(int fd, Config* c) {
return receive_wire_bool(fd, &c->preserve_atimes) &&
receive_wire_bool(fd, &c->preserve_crtimes) && receive_wire_bool(fd, &c->omit_dir_times) &&
receive_wire_bool(fd, &c->omit_link_times);
}
bool config_send(int file_descriptor, const Config* config) {
protocol_session_set_max_alloc(NULL, config->max_alloc);
if (!send_core_fields(file_descriptor, config) || !send_delta_fields(file_descriptor, config) ||
@@ -754,7 +779,8 @@ bool config_send(int file_descriptor, const Config* config) {
!send_resume_options(file_descriptor, config) ||
!send_basis_options(file_descriptor, config) || !send_fuzzy_option(file_descriptor, config) ||
!send_checksum_options(file_descriptor, config) ||
!send_identity_options(file_descriptor, config))
!send_identity_options(file_descriptor, config) ||
!send_metadata_times_options(file_descriptor, config))
return false;
Status status;
if (!receive_status(file_descriptor, &status))
@@ -790,7 +816,8 @@ Config* config_receive(int file_descriptor) {
!receive_basis_options(file_descriptor, config) ||
!receive_fuzzy_option(file_descriptor, config) ||
!receive_checksum_options(file_descriptor, config) ||
!receive_identity_options(file_descriptor, config))
!receive_identity_options(file_descriptor, config) ||
!receive_metadata_times_options(file_descriptor, config))
goto error;
if (config->compress_choice[0] != '\0' && strcmp(config->compress_choice, "zstd") != 0 &&
strcmp(config->compress_choice, "none") != 0) {
+20 -1
View File
@@ -291,9 +291,28 @@ typedef struct Config {
// Receiver-side runtime staging registry for --delay-updates. Never sent
// over the wire and never set on the sender side.
DelayUpdatesContext* delay_context;
// Phase 4: metadata time preservation. -U/--atimes and -N/--crtimes capture
// and transmit the source access / birth time (both sender and receiver
// effect, so they CROSS the wire). --omit-dir-times/-O and
// --omit-link-times/-J are receiver-side prefs (CROSS the wire). Their
// exact capture/transmit/apply semantics are documented in RSYNC_COMPAT.md.
/* -U/--atimes: preserve source access times on the destination. */
bool preserve_atimes;
/* -N/--crtimes: capture+transmit source birth time; see RSYNC_COMPAT for the
* receiver not-applied divergence. */
bool preserve_crtimes;
/* -O/--omit-dir-times: do not apply mtimes to directories. */
bool omit_dir_times;
/* -J/--omit-link-times: do not apply times to symlinks. */
bool omit_link_times;
/* --open-noatime: CLIENT-ONLY (never crosses the wire). The sender opens
* source files with O_NOATIME so reading for transfer does not bump the
* source access time. */
bool open_noatime;
} Config;
#define PROTOCOL_VERSION "2.11.0"
#define PROTOCOL_VERSION "2.12.0"
#define DEFAULT_CHUNK_SIZE (10 * 1024 * 1024)
/* Upper bound on total basis-dir entries (rsync caps --link-dest at 20). */
#define MAX_BASIS_DIRS 64
+79 -2
View File
@@ -1,3 +1,6 @@
#ifndef _GNU_SOURCE
#define _GNU_SOURCE /* statx + STATX_BTIME for --crtimes birth-time capture */
#endif
#include <errno.h>
#include <dirent.h>
#include <fcntl.h>
@@ -108,6 +111,9 @@ File* file_create(const char* path) {
file->skip = false;
file->is_dir = false;
file->basis_link = NULL;
file->link_group = 0;
file->link_first = false;
file->hardlink_target = NULL;
return file;
}
@@ -125,10 +131,13 @@ void file_destroy(void* item) {
file->send_path = NULL;
free(file->basis_link);
file->basis_link = NULL;
free(file->hardlink_target);
file->hardlink_target = NULL;
free(file);
}
FileMetadata* file_metadata_create(const struct stat* stats) {
FileMetadata* file_metadata_create(const char* path, const struct stat* stats, bool capture_atime,
bool capture_crtime) {
FileMetadata* m = protocol_alloc(sizeof(FileMetadata));
if (m == NULL) {
log_perror("ERROR: Could not allocate memory for file metadata");
@@ -143,6 +152,37 @@ FileMetadata* file_metadata_create(const struct stat* stats) {
#else
m->mtime_nsec = 0;
#endif
/* -U/--atimes: capture the access time from the same pre-read stat the
scanner already took, so the value is not clobbered by a later read for
transfer. The timestamp is populated (and atime_valid set) only on Linux,
where st_atim is populated; on other platforms the atime is left alone
rather than clobbered to the default 0/epoch by an unpopulated value. */
#ifdef __linux__
m->atime_valid = capture_atime;
m->atime_sec = stats->st_atim.tv_sec;
m->atime_nsec = stats->st_atim.tv_nsec;
#else
m->atime_valid = false;
m->atime_sec = 0;
m->atime_nsec = 0;
#endif
/* -N/--crtimes: birth time is not available via struct stat in general; on
Linux it needs statx STATX_BTIME. If unavailable it is captured as a
documented no-op (the flag stays accepted, crtime_valid stays false). */
m->crtime_valid = false;
m->crtime_sec = 0;
m->crtime_nsec = 0;
if (capture_crtime) {
#ifdef STATX_BTIME
struct statx stx;
if (path != NULL && statx(AT_FDCWD, path, AT_STATX_SYNC_AS_STAT, STATX_BTIME, &stx) == 0 &&
(stx.stx_mask & STATX_BTIME) != 0) {
m->crtime_valid = true;
m->crtime_sec = (time_t)stx.stx_btime.tv_sec;
m->crtime_nsec = (long)stx.stx_btime.tv_nsec;
}
#endif
}
return m;
}
@@ -150,6 +190,37 @@ void file_metadata_destroy(void* metadata) {
free(metadata);
}
/* --open-noatime: process-wide sender policy (client-only, never crosses the
* wire). When enabled, opening a source file for transfer uses O_NOATIME so
* the read does not bump the source's on-disk access time. It degrades safely
* to a normal open where O_NOATIME is unavailable (not defined) or refused
* (EPERM, because it needs CAP_FOWNER): the data path never silently changes,
* only the atime-bump is skipped. */
static bool file_open_noatime = false;
void file_set_open_noatime(bool enable) {
file_open_noatime = enable;
}
bool file_get_open_noatime(void) {
return file_open_noatime;
}
/* Open `path` read-only for transfer, honouring --open-noatime when set. */
int file_open_for_read(const char* path) {
int flags = O_RDONLY;
#ifdef O_NOATIME
if (file_get_open_noatime())
flags |= O_NOATIME;
#endif
int fd = open(path, flags);
#ifdef O_NOATIME
if (fd < 0 && (flags & O_NOATIME))
fd = open(path, O_RDONLY); /* degrade safely on EPERM / unsupported fs */
#endif
return fd;
}
bool file_load_data(File* file) {
if (file == NULL || !file->data)
return false;
@@ -176,8 +247,14 @@ bool file_load_data(File* file) {
size_t file_content_to_buffer(File* file) {
if (!file || !file->path || !file->data || (!file->data->data && file->data->size != 0))
return 0;
FILE* file_pointer = fopen(file->path, "rb");
int fd = file_open_for_read(file->path);
if (fd < 0) {
log_perror("Could not open the file!");
return 0;
}
FILE* file_pointer = fdopen(fd, "rb");
if (file_pointer == NULL) {
close(fd);
log_perror("Could not open the file!");
return 0;
}
+7 -1
View File
@@ -22,8 +22,14 @@ bool file_load_data(File* file);
bool file_checksum(File* file, ChecksumAlgo algo, uint64_t seed, uint8_t* out, size_t out_capacity,
size_t* out_len);
size_t file_content_to_buffer(File* file);
FileMetadata* file_metadata_create(const struct stat* stats);
FileMetadata* file_metadata_create(const char* path, const struct stat* stats, bool capture_atime,
bool capture_crtime);
void file_metadata_destroy(void* metadata);
/* --open-noatime process-wide sender policy; see file.c. */
void file_set_open_noatime(bool enable);
bool file_get_open_noatime(void);
/* Open `path` read-only for transfer, honouring --open-noatime when set. */
int file_open_for_read(const char* path);
bool file_write_to_disk(const char* path, const void* data, unsigned long long data_size,
bool inplace, bool sparse);
+247 -1
View File
@@ -102,6 +102,192 @@ static FileSaveResult file_stage_delayed_update(const char* root_directory,
return FILE_SAVE_WRITTEN;
}
/* Read the whole content of a confined regular file (used to fall back to a
byte-identical copy when a hard-link sibling's link() fails). Symlink-safe
(parent resolved via file_open_secure_parent + O_NOFOLLOW). A zero-length
file yields *out_size 0 and *out_buf NULL as a SUCCESS. Returns false only
on a real error/read failure, setting *source_absent to true when the reason
was that the path does not exist (ENOENT/ENOTDIR), so the caller can decide
between an abort and a graceful skip. */
static bool hardlink_read_source(const char* path, void** out_buf, unsigned long long* out_size,
bool* source_absent) {
*out_buf = NULL;
*out_size = 0;
*source_absent = false;
if (!path)
return false;
char* leaf = NULL;
int parent_fd = file_open_secure_parent(path, &leaf, false);
if (parent_fd < 0) {
*source_absent = errno == ENOENT || errno == ENOTDIR;
return false;
}
int fd = openat(parent_fd, leaf, O_RDONLY | O_CLOEXEC | O_NOFOLLOW);
int saved_errno = errno;
free(leaf);
close(parent_fd);
if (fd < 0) {
*source_absent = saved_errno == ENOENT || saved_errno == ENOTDIR;
return false;
}
struct stat st;
if (fstat(fd, &st) != 0 || !S_ISREG(st.st_mode)) {
close(fd);
return false;
}
unsigned long long size = (unsigned long long)st.st_size;
if (size > MAX_RECEIVE_WHOLE_FILE_SIZE || size > SIZE_MAX) {
close(fd);
return false;
}
if (size == 0) {
close(fd);
return true;
}
void* buf = protocol_alloc((size_t)size);
if (!buf) {
close(fd);
return false;
}
size_t got = 0;
while (got < (size_t)size) {
ssize_t n = read(fd, (char*)buf + got, (size_t)size - got);
if (n <= 0) {
free(buf);
close(fd);
return false;
}
got += (size_t)n;
}
close(fd);
*out_buf = buf;
*out_size = size;
return true;
}
/* The group's first member's installed file is absent, but its destination
path was validated (a sibling is only ever processed after its group's first
member). When the sibling's OWN destination already exists it should be
left alone -- a clean skip -- rather than aborting the whole transfer (the
asymmetric --existing case: the first member was skipped because its
destination was missing, while the sibling already has one). Only when the
sibling's destination is missing too is this a genuine failure to
link/copy, which aborts. */
static FileSaveResult hardlink_sibling_absent_first(const char* destination_path) {
if (destination_path && file_path_exists_secure(destination_path))
return FILE_SAVE_SKIPPED;
return FILE_SAVE_ERROR;
}
/* Install a --hard-links/-H sibling: the destination entry is atomically
replaced (temp + rename) with a hard link to the group's first member. The
first member is guaranteed already installed at `hardlink_target` under the
root because -H relies on the receiver's single-FIFO-writer pipeline (one
receive thread, one write thread, FIFO queue => wire order == write order)
plus the sender's forced sequential scan, so a sibling is always processed
after its group's first member. When link() fails (different filesystem,
filesystem refuses links) a byte-identical copy of the first member is
written instead, so the result is never partial or corrupt. With
--delay-updates the sibling is staged as a hard link to the first member's
STAGED file (publication's renames preserve the shared inode). The final
--existing/--ignore-existing/--update policies are decided against the final
destination like every normal write. */
static FileSaveResult file_save_hardlink_sibling(const char* root_directory, const File* file,
const Config* config) {
Config* cfg = (Config*)config;
if (!root_directory || !file || !file->path || !file->hardlink_target)
return FILE_SAVE_ERROR;
char* destination_path = path_cat(root_directory, file->path);
if (!destination_path)
return FILE_SAVE_ERROR;
if (cfg->existing && !file_path_exists_secure(destination_path)) {
free(destination_path);
return FILE_SAVE_SKIPPED;
}
if (cfg->ignore_existing && file_path_exists_secure(destination_path)) {
free(destination_path);
return FILE_SAVE_SKIPPED;
}
if (cfg->update && file_destination_is_newer_secure(destination_path, file->metadata)) {
free(destination_path);
return FILE_SAVE_SKIPPED;
}
bool preallocate = cfg && cfg->preallocate;
bool preserve_executability = cfg && cfg->use_executability;
bool use_fsync = cfg && cfg->use_fsync;
if (cfg->delay_updates) {
if (!cfg->delay_context) {
cfg->delay_context = delay_updates_context_create(root_directory);
if (!cfg->delay_context) {
free(destination_path);
return FILE_SAVE_ERROR;
}
}
if (!delay_updates_prepare(cfg->delay_context)) {
free(destination_path);
return FILE_SAVE_ERROR;
}
char* staged_first = path_cat(cfg->delay_context->staging_root, file->hardlink_target);
char* staged_sibling = path_cat(cfg->delay_context->staging_root, file->path);
if (!staged_first || !staged_sibling) {
free(staged_first);
free(staged_sibling);
free(destination_path);
return FILE_SAVE_ERROR;
}
void* content = NULL;
unsigned long long content_size = 0;
bool source_absent = false;
if (!hardlink_read_source(staged_first, &content, &content_size, &source_absent)) {
FileSaveResult absent_result =
source_absent ? hardlink_sibling_absent_first(destination_path) : FILE_SAVE_ERROR;
free(staged_first);
free(staged_sibling);
free(destination_path);
return absent_result;
}
bool ok =
file_to_disk_secure_link(staged_sibling, staged_first, content, content_size, preallocate,
file->metadata, preserve_executability, use_fsync, NULL);
free(content);
if (ok)
ok = delay_updates_record(cfg->delay_context, staged_sibling, destination_path, file->path);
if (!ok)
unlink(staged_sibling);
free(staged_first);
free(staged_sibling);
free(destination_path);
return ok ? FILE_SAVE_WRITTEN : FILE_SAVE_ERROR;
}
char* first_disk = path_cat(root_directory, file->hardlink_target);
if (!first_disk) {
free(destination_path);
return FILE_SAVE_ERROR;
}
void* content = NULL;
unsigned long long content_size = 0;
bool source_absent = false;
if (!hardlink_read_source(first_disk, &content, &content_size, &source_absent)) {
FileSaveResult absent_result =
source_absent ? hardlink_sibling_absent_first(destination_path) : FILE_SAVE_ERROR;
free(first_disk);
free(destination_path);
return absent_result;
}
const char* temp_dir = (cfg && cfg->temp_dir) ? cfg->temp_dir : NULL;
bool ok =
file_to_disk_secure_link(destination_path, first_disk, content, content_size, preallocate,
file->metadata, preserve_executability, use_fsync, temp_dir);
free(content);
free(first_disk);
free(destination_path);
return ok ? FILE_SAVE_WRITTEN : FILE_SAVE_ERROR;
}
FileSaveResult file_save_to_disk_full(const char* root_directory, const File* file,
const Config* config) {
/* Backups are incompatible with ignore-existing: moving the entry first
@@ -146,6 +332,14 @@ FileSaveResult file_save_to_disk_full(const char* root_directory, const File* fi
return ok ? FILE_SAVE_WRITTEN : FILE_SAVE_ERROR;
}
/* --hard-links/-H sibling: a later member of a link group arrives with no
payload and is installed as a hard link to (or, on link() failure, a
byte-identical copy of) the group's first member. Handled entirely here,
before the normal data-write paths (which would create an empty file). */
if (file->link_group != 0 && !file->link_first && file->hardlink_target != NULL) {
return file_save_hardlink_sibling(root_directory, file, config);
}
/* These options arrive from the client. They are names below the server
root, never independent filesystem roots. --temp-dir is confined exactly
like --backup-dir/--partial-dir: an absolute or `..`-escaping scratch
@@ -1222,7 +1416,7 @@ File* receive_incremental_check(int fd, const Config* config, bool* skipped) {
if (materialized && basis.content) {
materialized->data = basis.content;
basis.content = NULL;
materialized->metadata = file_metadata_create(&basis.st);
materialized->metadata = file_metadata_create(NULL, &basis.st, false, false);
materialized->skip = true; /* receiver must not ack this as a data file */
if (basis.type == BASIS_DEST_LINK) {
materialized->basis_link = basis.basis_path;
@@ -1622,6 +1816,58 @@ File* file_receive_directory(int file_descriptor) {
return file;
}
/* Receive a --hard-links/-H sibling frame (the leading STATUS_HARDLINK code has
already been consumed): the destination path, the run-local link-group id,
and the first (data-carrying) member's destination-relative wire path. The
created File carries no payload; it is installed beneath the receive root as
a hard link to (or, on link failure, a byte-identical copy of) the first
member. All paths are validated like every other received path (non-empty,
relative, no traversal). */
File* file_receive_hardlink(int file_descriptor) {
char* path = receive_str(file_descriptor);
if (path == NULL)
return NULL;
if (path[0] == '\0' || has_path_traversal(path)) {
char* escaped_path = output_escape(path, log_get_8_bit_output());
log_message(LOG_LEVEL_ERROR, "Invalid received hard-link path: %s",
escaped_path ? escaped_path : "<allocation failed>");
free(escaped_path);
free(path);
send_status(file_descriptor, STATUS_ERROR);
return NULL;
}
int gid;
if (!receive_int(file_descriptor, &gid) || gid <= 0) {
free(path);
return NULL;
}
char* target = receive_str(file_descriptor);
if (!target) {
free(path);
return NULL;
}
if (target[0] == '\0' || has_path_traversal(target)) {
char* escaped = output_escape(target, log_get_8_bit_output());
log_message(LOG_LEVEL_ERROR, "Invalid hard-link target path: %s",
escaped ? escaped : "<allocation failed>");
free(escaped);
free(target);
free(path);
send_status(file_descriptor, STATUS_ERROR);
return NULL;
}
File* file = file_create(path);
free(path);
if (file == NULL) {
free(target);
return NULL;
}
file->link_group = gid;
file->link_first = false;
file->hardlink_target = target;
return file;
}
/* Read a delete-manifest frame (the STATUS_MANIFEST leading code has already
been consumed): a keep-set entry count followed by that many
destination-relative paths, then a protected-prefix count followed by that
+1
View File
@@ -9,6 +9,7 @@
File* file_receive(const Config* config, int file_descriptor);
File* file_receive_directory(int file_descriptor);
File* file_receive_hardlink(int file_descriptor);
File* receive_incremental_check(int fd, const Config* config, bool* skipped);
/* A received delete-manifest frame: the keep-set (`keeps`, destination-relative
+1 -1
View File
@@ -78,7 +78,7 @@ bool file_send_sendfile_with_skip(File* file, int file_descriptor, bool use_meta
if (use_metadata && !metadata_send(file_descriptor, file->metadata))
return false;
int fd = open(file->path, O_RDONLY);
int fd = file_open_for_read(file->path);
if (fd == -1) {
log_perror("Could not open file for sendfile");
return false;
+21
View File
@@ -13,6 +13,18 @@ typedef struct {
gid_t gid;
time_t mtime_sec;
long mtime_nsec;
/* Optional access time (-U/--atimes) and creation/birth time (-N/--crtimes),
* appended for protocol 2.12.0. The SENDER sets the corresponding *_valid
* flag only when the preserve option is active (and, for crtime, only when
* the source platform exposed a birth time via statx STATX_BTIME). The wire
* always carries the fields and the flags; a false flag tells the receiver to
* ignore the value. */
bool atime_valid;
time_t atime_sec;
long atime_nsec;
bool crtime_valid;
time_t crtime_sec;
long crtime_nsec;
} FileMetadata;
typedef struct {
@@ -35,6 +47,15 @@ typedef struct {
* equals the incoming file, and `data` is kept as the cross-filesystem
* fallback (a local copy) if the hard link cannot be created. */
char* basis_link;
/* --hard-links (-H), sender + receiver wire state. link_group is a run-local
* id shared by every member of one source inode (0 = not part of a group).
* The FIRST member (link_first == true) carries its data on the wire and is
* written normally; every sibling (link_first == false) carries NO data and
* hardlink_target holds the first member's wire path so the receiver can link
* to (or copy from) the already-installed first member. */
int link_group;
bool link_first;
char* hardlink_target;
} File;
/* The path that should be sent on the wire and used for the receiver-side
+127
View File
@@ -0,0 +1,127 @@
#include "hardlink.h"
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include "log.h"
#include "utils.h"
/* ---- Sender-side detection table ---- */
HardLinkTable* hardlink_table_create(void) {
HardLinkTable* table = calloc(1, sizeof(HardLinkTable));
if (!table)
return NULL;
if (mtx_init(&table->mutex, mtx_plain) != thrd_success) {
free(table);
return NULL;
}
table->next_gid = 1;
return table;
}
static void hardlink_item_destroy(HardLinkItem* item) {
if (!item)
return;
free(item->first_path);
item->first_path = NULL;
}
void hardlink_table_destroy(HardLinkTable* table) {
if (!table)
return;
for (size_t i = 0; i < table->count; i++)
hardlink_item_destroy(&table->items[i]);
free(table->items);
table->items = NULL;
table->count = 0;
table->capacity = 0;
mtx_destroy(&table->mutex);
free(table);
}
static HardLinkItem* hardlink_table_find_locked(HardLinkTable* table, dev_t dev, ino_t ino) {
for (size_t i = 0; i < table->count; i++) {
if (table->items[i].dev == dev && table->items[i].ino == ino)
return &table->items[i];
}
return NULL;
}
static bool hardlink_table_add_locked(HardLinkTable* table, dev_t dev, ino_t ino, const char* path,
int gid, HardLinkItem** out) {
if (table->count == table->capacity) {
size_t new_capacity = table->capacity == 0 ? 8 : table->capacity * 2;
if (new_capacity < table->capacity)
return false;
HardLinkItem* grown = realloc(table->items, new_capacity * sizeof(HardLinkItem));
if (!grown)
return false;
table->items = grown;
table->capacity = new_capacity;
}
HardLinkItem* item = &table->items[table->count];
char* dup = str_dup(path);
if (!dup)
return false;
memset(item, 0, sizeof(*item));
item->dev = dev;
item->ino = ino;
item->gid = gid;
item->first_path = dup;
table->count++;
*out = item;
return true;
}
bool hardlink_table_assign(HardLinkTable* table, const char* wire_path, dev_t dev, ino_t ino,
int* gid, bool* is_first, char** first_path_out) {
if (!table || !wire_path || !gid || !is_first || !first_path_out)
return false;
if (mtx_lock(&table->mutex) != thrd_success)
return false;
bool ok = true;
const HardLinkItem* item = hardlink_table_find_locked(table, dev, ino);
int next_gid;
if (item) {
*is_first = false;
char* dup = str_dup(item->first_path);
if (!dup) {
ok = false;
} else {
*gid = item->gid;
*first_path_out = dup;
}
next_gid = -1;
} else {
if (table->next_gid <= 0) {
ok = false;
next_gid = -1;
} else {
next_gid = table->next_gid;
HardLinkItem* created = NULL;
if (!hardlink_table_add_locked(table, dev, ino, wire_path, next_gid, &created)) {
ok = false;
} else {
char* dup = str_dup(wire_path);
if (!dup) {
hardlink_item_destroy(created);
table->count--;
ok = false;
} else {
*is_first = true;
*gid = next_gid;
*first_path_out = dup;
}
}
}
}
if (ok && next_gid > 0)
table->next_gid++;
mtx_unlock(&table->mutex);
if (!ok) {
log_message(LOG_LEVEL_ERROR, "memory allocation failed while detecting hard links");
}
return ok;
}
+66
View File
@@ -0,0 +1,66 @@
#ifndef HARDLINK_H
#define HARDLINK_H
#include <stdbool.h>
#include <stddef.h>
#include <sys/types.h>
#include <threads.h>
/*
* --hard-links / -H support.
*
* Sender side: a HardLinkTable detects regular files on the source that share
* an (st_dev, st_ino) identity (a `cp -al`-style hard-linked tree) and assigns
* each distinct inode a stable, run-local link-group id. The first member
* encountered carries the file data; every later member is marked as a sibling
* (no data payload) that the receiver creates as a hard link to the first
* member's destination file. Grouping is scoped by st_dev so inode reuse
* across different filesystems is never conflated. The table is mutex-guarded
* so the parallel (multi-threaded) scanner COULD share one instance across its
* worker threads; the first-thread-to-call designates the data-carrying member,
* which is safe because a hard-link group's members are byte-identical. (In
* practice the sender forces the sequential scanner whenever -H is on; the
* mutex guards the shared table for any path that supplies one.)
*
* ORDERING (why there is no receiver-side handshake): the receiver stores every
* file - including a hard-link group's first member - through a SINGLE writer
* thread draining a single FIFO queue driven by a single receive thread, so
* wire order == write order and every sibling is processed AFTER its group's
* first member. The sender additionally forces the sequential scanner with -H
* so the first-member frame always precedes its siblings on the wire. Sibling
* install therefore needs no present/wait registry: it hard-links to the first
* member (or copies it) knowing that path is already installed - or that, if
* the first member was skipped (already up to date), its destination still
* exists. This guarantee is REQUIRED; do not introduce a concurrent
* multi-writer receiver for -H without re-adding an ordering mechanism.
*/
typedef struct HardLinkItem {
dev_t dev;
ino_t ino;
int gid;
char* first_path; /* wire path of the group's data-carrying first member */
} HardLinkItem;
typedef struct HardLinkTable {
mtx_t mutex;
HardLinkItem* items;
size_t count;
size_t capacity;
int next_gid;
} HardLinkTable;
HardLinkTable* hardlink_table_create(void);
void hardlink_table_destroy(HardLinkTable* table);
/* Assign a link-group id to the regular file at `wire_path` with (dev, ino).
* On the first encounter the file becomes the group's first (data-carrying)
* member (*is_first = true) and a fresh gid is allocated. On a later member
* *is_first = false and *first_path_out is set to a malloc'd copy of the first
* member's wire path (the caller stores it and owns it; on the first member
* path the returned *first_path_out is a malloc'd copy of its own wire path).
* Returns false on allocation failure (transfer should abort). */
bool hardlink_table_assign(HardLinkTable* table, const char* wire_path, dev_t dev, ino_t ino,
int* gid, bool* is_first, char** first_path_out);
#endif
+134 -3
View File
@@ -70,6 +70,24 @@ void metadata_to_buf(char** buf, const FileMetadata* m) {
int64_t mtime_nsec = (int64_t)m->mtime_nsec;
memcpy(*buf, &mtime_nsec, sizeof(mtime_nsec));
*buf += sizeof(mtime_nsec);
int32_t atime_valid = m->atime_valid ? 1 : 0;
memcpy(*buf, &atime_valid, sizeof(atime_valid));
*buf += sizeof(atime_valid);
int64_t atime_sec = (int64_t)m->atime_sec;
memcpy(*buf, &atime_sec, sizeof(atime_sec));
*buf += sizeof(atime_sec);
int64_t atime_nsec = (int64_t)m->atime_nsec;
memcpy(*buf, &atime_nsec, sizeof(atime_nsec));
*buf += sizeof(atime_nsec);
int32_t crtime_valid = m->crtime_valid ? 1 : 0;
memcpy(*buf, &crtime_valid, sizeof(crtime_valid));
*buf += sizeof(crtime_valid);
int64_t crtime_sec = (int64_t)m->crtime_sec;
memcpy(*buf, &crtime_sec, sizeof(crtime_sec));
*buf += sizeof(crtime_sec);
int64_t crtime_nsec = (int64_t)m->crtime_nsec;
memcpy(*buf, &crtime_nsec, sizeof(crtime_nsec));
*buf += sizeof(crtime_nsec);
}
FileMetadata* metadata_from_buf(char** buf) {
@@ -103,8 +121,34 @@ FileMetadata* metadata_from_buf(char** buf) {
memcpy(&mtime_nsec, *buf, sizeof(mtime_nsec));
*buf += sizeof(mtime_nsec);
m->mtime_nsec = (long)mtime_nsec;
int32_t atime_valid;
memcpy(&atime_valid, *buf, sizeof(atime_valid));
*buf += sizeof(atime_valid);
int64_t atime_sec;
memcpy(&atime_sec, *buf, sizeof(atime_sec));
*buf += sizeof(atime_sec);
int64_t atime_nsec;
memcpy(&atime_nsec, *buf, sizeof(atime_nsec));
*buf += sizeof(atime_nsec);
int32_t crtime_valid;
memcpy(&crtime_valid, *buf, sizeof(crtime_valid));
*buf += sizeof(crtime_valid);
int64_t crtime_sec;
memcpy(&crtime_sec, *buf, sizeof(crtime_sec));
*buf += sizeof(crtime_sec);
int64_t crtime_nsec;
memcpy(&crtime_nsec, *buf, sizeof(crtime_nsec));
*buf += sizeof(crtime_nsec);
m->atime_valid = atime_valid != 0;
m->atime_sec = (time_t)atime_sec;
m->atime_nsec = (long)atime_nsec;
m->crtime_valid = crtime_valid != 0;
m->crtime_sec = (time_t)crtime_sec;
m->crtime_nsec = (long)crtime_nsec;
if (present != 1 || mtime_nsec < 0 || mtime_nsec >= 1000000000LL || mode < 0 || uid < 0 ||
gid < 0) {
gid < 0 || atime_valid < 0 || atime_valid > 1 || crtime_valid < 0 || crtime_valid > 1 ||
(atime_valid && (atime_nsec < 0 || atime_nsec >= 1000000000LL)) ||
(crtime_valid && (crtime_nsec < 0 || crtime_nsec >= 1000000000LL))) {
free(m);
return NULL;
}
@@ -122,12 +166,24 @@ bool metadata_send(int file_descriptor, const FileMetadata* m) {
int32_t gid = (int32_t)m->gid;
int64_t mtime_sec = (int64_t)m->mtime_sec;
int64_t mtime_nsec = (int64_t)m->mtime_nsec;
int32_t atime_valid = m->atime_valid ? 1 : 0;
int64_t atime_sec = (int64_t)m->atime_sec;
int64_t atime_nsec = (int64_t)m->atime_nsec;
int32_t crtime_valid = m->crtime_valid ? 1 : 0;
int64_t crtime_sec = (int64_t)m->crtime_sec;
int64_t crtime_nsec = (int64_t)m->crtime_nsec;
return send_n_data(file_descriptor, &present, sizeof(present)) &&
send_n_data(file_descriptor, &mode, sizeof(mode)) &&
send_n_data(file_descriptor, &uid, sizeof(uid)) &&
send_n_data(file_descriptor, &gid, sizeof(gid)) &&
send_n_data(file_descriptor, &mtime_sec, sizeof(mtime_sec)) &&
send_n_data(file_descriptor, &mtime_nsec, sizeof(mtime_nsec));
send_n_data(file_descriptor, &mtime_nsec, sizeof(mtime_nsec)) &&
send_n_data(file_descriptor, &atime_valid, sizeof(atime_valid)) &&
send_n_data(file_descriptor, &atime_sec, sizeof(atime_sec)) &&
send_n_data(file_descriptor, &atime_nsec, sizeof(atime_nsec)) &&
send_n_data(file_descriptor, &crtime_valid, sizeof(crtime_valid)) &&
send_n_data(file_descriptor, &crtime_sec, sizeof(crtime_sec)) &&
send_n_data(file_descriptor, &crtime_nsec, sizeof(crtime_nsec));
}
FileMetadata* metadata_receive(int file_descriptor, int* ok) {
@@ -193,7 +249,58 @@ FileMetadata* metadata_receive(int file_descriptor, int* ok) {
return NULL;
}
m->mtime_nsec = (long)mtime_nsec;
if (mtime_nsec < 0 || mtime_nsec >= 1000000000LL || mode < 0 || uid < 0 || gid < 0) {
int32_t atime_valid;
if (!receive_n_data(file_descriptor, &atime_valid, sizeof(atime_valid))) {
free(m);
if (ok)
*ok = 0;
return NULL;
}
int64_t atime_sec;
if (!receive_n_data(file_descriptor, &atime_sec, sizeof(atime_sec))) {
free(m);
if (ok)
*ok = 0;
return NULL;
}
int64_t atime_nsec;
if (!receive_n_data(file_descriptor, &atime_nsec, sizeof(atime_nsec))) {
free(m);
if (ok)
*ok = 0;
return NULL;
}
int32_t crtime_valid;
if (!receive_n_data(file_descriptor, &crtime_valid, sizeof(crtime_valid))) {
free(m);
if (ok)
*ok = 0;
return NULL;
}
int64_t crtime_sec;
if (!receive_n_data(file_descriptor, &crtime_sec, sizeof(crtime_sec))) {
free(m);
if (ok)
*ok = 0;
return NULL;
}
int64_t crtime_nsec;
if (!receive_n_data(file_descriptor, &crtime_nsec, sizeof(crtime_nsec))) {
free(m);
if (ok)
*ok = 0;
return NULL;
}
m->atime_valid = atime_valid != 0;
m->atime_sec = (time_t)atime_sec;
m->atime_nsec = (long)atime_nsec;
m->crtime_valid = crtime_valid != 0;
m->crtime_sec = (time_t)crtime_sec;
m->crtime_nsec = (long)crtime_nsec;
if (mtime_nsec < 0 || mtime_nsec >= 1000000000LL || mode < 0 || uid < 0 || gid < 0 ||
atime_valid < 0 || atime_valid > 1 || crtime_valid < 0 || crtime_valid > 1 ||
(atime_valid && (atime_nsec < 0 || atime_nsec >= 1000000000LL)) ||
(crtime_valid && (crtime_nsec < 0 || crtime_nsec >= 1000000000LL))) {
free(m);
if (ok)
*ok = 0;
@@ -232,6 +339,16 @@ void file_restore_metadata(const char* path, const FileMetadata* metadata,
times[0].tv_nsec = UTIME_OMIT;
times[1].tv_sec = metadata->mtime_sec;
times[1].tv_nsec = metadata->mtime_nsec;
if (metadata->atime_valid) {
times[0].tv_sec = metadata->atime_sec;
times[0].tv_nsec = metadata->atime_nsec;
}
if (metadata->crtime_valid) {
log_message(LOG_LEVEL_DEBUG,
"crtime (birth time) %lld.%09ld transmitted for %s but not applied: no portable "
"setter exists",
(long long)metadata->crtime_sec, metadata->crtime_nsec, path);
}
if (utimensat(AT_FDCWD, path, times, 0) != 0) {
char* escaped_path = output_escape(path, log_get_8_bit_output());
log_message(LOG_LEVEL_WARNING, "Failed to set timestamps on %s: %s",
@@ -261,6 +378,20 @@ bool file_restore_metadata_fd(int fd, const FileMetadata* metadata, bool preserv
identity_apply_ownership(fd, (int32_t)metadata->uid, (int32_t)metadata->gid);
struct timespec times[2] = {{.tv_sec = 0, .tv_nsec = UTIME_OMIT},
{.tv_sec = metadata->mtime_sec, .tv_nsec = metadata->mtime_nsec}};
if (metadata->atime_valid) {
times[0].tv_sec = metadata->atime_sec;
times[0].tv_nsec = metadata->atime_nsec;
}
/* --crtimes captures and transmits the source birth time, but there is no
* portable way to set a birth time (utimensat can only set atime/mtime), so
* the receiver deliberately does NOT apply it. This is explicit, honest
* unsupported-attribute handling: log a debug note and continue — never fail
* the transfer and never pretend the crtime was applied. */
if (metadata->crtime_valid) {
log_message(LOG_LEVEL_DEBUG,
"crtime (birth time) %lld.%09ld transmitted but not applied: no portable setter",
(long long)metadata->crtime_sec, metadata->crtime_nsec);
}
if (futimens(fd, times) != 0)
ok = false;
return ok;
+8 -2
View File
@@ -15,6 +15,12 @@
* int32_t gid (was gid_t, platform-dependent)
* int64_t mtime_sec (was time_t, platform-dependent)
* int64_t mtime_nsec (was long, platform-dependent)
* int32_t atime_valid (-U/--atimes; protocol 2.12.0)
* int64_t atime_sec
* int64_t atime_nsec
* int32_t crtime_valid (-N/--crtimes; protocol 2.12.0)
* int64_t crtime_sec
* int64_t crtime_nsec
*
* Prior to 2.0.0 the wire format used the raw platform-dependent types,
* which broke compatiblity across different systems. All fields are now
@@ -23,8 +29,8 @@
/* Size of metadata fields on wire, excluding the int32_t `present` field that
* is always sent first. The total wire size for present metadata is
* sizeof(int32_t) + FILE_METADATA_WIRE_SIZE (32 bytes on most platforms). */
#define FILE_METADATA_WIRE_SIZE (sizeof(int32_t) * 3 + sizeof(int64_t) * 2)
* sizeof(int32_t) + FILE_METADATA_WIRE_SIZE (68 bytes on most platforms). */
#define FILE_METADATA_WIRE_SIZE (sizeof(int32_t) * 5 + sizeof(int64_t) * 6)
void metadata_to_buf(char** buf, const FileMetadata* m);
FileMetadata* metadata_from_buf(char** buf);
+2
View File
@@ -405,6 +405,8 @@ static const char* status_to_string(Status status) {
return "APPEND_OK";
case STATUS_APPEND_DATA:
return "APPEND_DATA";
case STATUS_HARDLINK:
return "HARDLINK";
default:
return "UNKNOWN";
}
+7 -1
View File
@@ -84,7 +84,13 @@ enum NET_STATUS {
STATUS_APPEND,
STATUS_APPEND_SIG,
STATUS_APPEND_OK,
STATUS_APPEND_DATA
STATUS_APPEND_DATA,
/* --hard-links/-H: a sibling (later member) of a source hard-link group.
* The sender transmits only the path, the run-local link-group id, and the
* first (data-carrying) member's destination-relative wire path; the receiver
* creates this entry as a hard link to the first member's installed file
* (falling back to a byte-identical copy if link() fails). Protocol 2.12.0. */
STATUS_HARDLINK
};
void io_set_fds(int read_fd, int write_fd);
+302
View File
@@ -3826,3 +3826,305 @@ class TestIdentityMapping:
st = os.stat(dst_file)
assert st.st_uid == 12345 and st.st_gid == 54321, \
f"--chown not applied: uid={st.st_uid} gid={st.st_gid}"
class TestHardLinks:
"""-H/--hard-links: source files sharing an inode are re-created as hard
links to one another on the destination (dedup preserved, first copy
transferred once, the rest linked/copied). No root required."""
STAGING = ".fastsync-stage"
def _make_source(self, name):
src = os.path.join(TEST_DATA_DIR, name)
clean_dir(src)
with open(os.path.join(src, "a.txt"), "wb") as fh:
fh.write(b"shared content\n" * 2000)
os.link(os.path.join(src, "a.txt"), os.path.join(src, "b.txt"))
with open(os.path.join(src, "c.txt"), "wb") as fh:
fh.write(b"independent content\n" * 2000)
return src
@pytest.mark.parametrize("flags", [[], ["-m"], ["--delay-updates"]])
def test_hard_links_preserved(self, shared_server, flags):
src = self._make_source("hl_src")
dest = os.path.join(TEST_DATA_DIR, "hl_dst")
clean_dir(dest)
result, _ = run_client(src, dest, flags=["-H"] + flags, port=shared_server.port)
assert result.returncode == 0, f"Exit {result.returncode}: {result.stderr[:300]}"
received = get_dest_received_dir(dest, src)
a = os.path.join(received, "a.txt")
b = os.path.join(received, "b.txt")
c = os.path.join(received, "c.txt")
assert os.path.isfile(a) and os.path.isfile(b) and os.path.isfile(c), \
"all three destination files exist"
with open(a, "rb") as fa, open(b, "rb") as fb:
assert fa.read() == fb.read(), "hard-linked pair content matches"
assert os.stat(a).st_ino == os.stat(b).st_ino, \
"source hard links were not preserved on the destination"
assert os.stat(a).st_ino != os.stat(c).st_ino, \
"independent files were incorrectly hard linked"
with open(a, "rb") as fa, open(c, "rb") as fc:
assert fa.read() != fc.read(), "independent files must differ in content"
assert not os.path.isdir(os.path.join(dest, self.STAGING)), \
"--delay-updates left a staging tree behind"
def test_hard_links_rejects_chunk_serialization(self, shared_server):
src = self._make_source("hl_reject_src")
dest = os.path.join(TEST_DATA_DIR, "hl_reject_dst")
clean_dir(dest)
result, _ = run_client(src, dest, flags=["-H", "-s"], port=shared_server.port)
assert result.returncode != 0, "-H with -s was accepted"
def test_hard_links_rejects_append(self, shared_server):
src = self._make_source("hl_reject_app_src")
dest = os.path.join(TEST_DATA_DIR, "hl_reject_app_dst")
clean_dir(dest)
result, _ = run_client(src, dest, flags=["-H", "--append"], port=shared_server.port)
assert result.returncode != 0, "-H with --append was accepted"
def test_hard_links_link_to_existing_first_member(self, shared_server):
"""A sibling whose first member is already up-to-date at the destination
must still be created as a hard link to that existing file."""
src = os.path.join(TEST_DATA_DIR, "hl_exist_src")
dest = os.path.join(TEST_DATA_DIR, "hl_exist_dst")
clean_dir(src)
clean_dir(dest)
with open(os.path.join(src, "a.txt"), "wb") as fh:
fh.write(b"seed content\n" * 1500)
result, _ = run_client(src, dest, port=shared_server.port)
assert result.returncode == 0, f"seed failed: {result.stderr[:200]}"
# Introduce a hard-link sibling to the already-transferred first member.
os.link(os.path.join(src, "a.txt"), os.path.join(src, "b.txt"))
result, _ = run_client(src, dest, flags=["-H"], port=shared_server.port)
assert result.returncode == 0, f"-H sync failed: {result.stderr[:300]}"
received = get_dest_received_dir(dest, src)
a = os.path.join(received, "a.txt")
b = os.path.join(received, "b.txt")
assert os.path.isfile(a) and os.path.isfile(b)
assert os.stat(a).st_ino == os.stat(b).st_ino, \
"new sibling was not linked to the existing first member"
with open(a, "rb") as fa, open(b, "rb") as fb:
assert fa.read() == fb.read()
def test_hard_links_existing_asymmetric_group(self, shared_server):
"""-H --existing with an asymmetric link group must succeed: when the
first member's destination is absent (so it is skipped by --existing)
but a sibling's destination already exists, the existing sibling is left
in place instead of the whole transfer aborting on the absent first
member."""
src = os.path.join(TEST_DATA_DIR, "hl_existing_src")
dest = os.path.join(TEST_DATA_DIR, "hl_existing_dst")
clean_dir(src)
clean_dir(dest)
with open(os.path.join(src, "a.txt"), "wb") as fh:
fh.write(b"asymmetric group content\n" * 1200)
# b.txt is a hard-link sibling of a.txt on the source.
os.link(os.path.join(src, "a.txt"), os.path.join(src, "b.txt"))
with open(os.path.join(src, "c.txt"), "wb") as fh:
fh.write(b"independent\n" * 1200)
# Pre-seed the destination with ONLY the sibling's file (the first
# member has no destination entry).
received = get_dest_received_dir(dest, src)
os.makedirs(received, exist_ok=True)
with open(os.path.join(received, "b.txt"), "wb") as fh:
fh.write(b"asymmetric group content\n" * 1200)
result, _ = run_client(src, dest, flags=["-H", "--existing"],
port=shared_server.port)
assert result.returncode == 0, \
f"-H --existing asymmetric group failed: {result.stderr[:300]}"
# The existing sibling was preserved and its content is intact.
with open(os.path.join(received, "b.txt"), "rb") as fh:
assert fh.read() == b"asymmetric group content\n" * 1200
# Under --existing the absent first member is not created.
assert not os.path.exists(os.path.join(received, "a.txt"))
class TestAtimes:
"""-U/--atimes preserves the source access time on the destination.
The sender captures atime during the scan (a stat, before any read for
transfer), so the value is not clobbered by reading the source. This is
verified by setting the source atime to a distinct value far in the past
and comparing the destination atime to it (with whole-second tolerance;
filesystems may round atime)."""
PAYLOAD = b"atime preservation payload\n"
@staticmethod
def _make_source(source, dest):
clean_dir(source)
clean_dir(dest)
path = os.path.join(source, "data.txt")
with open(path, "wb") as f:
f.write(TestAtimes.PAYLOAD)
atime = 946684800 # 2000-01-01 00:00:00 UTC (far from "now")
mtime = 951782400
os.utime(path, ns=(atime * 10**9 + 123456789, mtime * 10**9))
return path, atime
@pytest.mark.ci
@pytest.mark.parametrize("mt", [False, True])
def test_atimes_preserved(self, shared_server, mt):
source = os.path.join(TEST_DATA_DIR, f"atime_{'m' if mt else 's'}_src")
dest = os.path.join(TEST_DATA_DIR, f"atime_{'m' if mt else 's'}_dst")
src_file, atime = self._make_source(source, dest)
flags = ["-U"] + (["-m"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, \
f"-U failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
dst_file = os.path.join(received, "data.txt")
assert os.path.exists(dst_file)
dst_st = os.stat(dst_file)
assert abs(dst_st.st_atime - atime) < 1.5, \
f"dest atime {dst_st.st_atime} != source atime {atime}"
@pytest.mark.ci
@pytest.mark.parametrize("mt", [False, True])
def test_without_atimes_dest_differs(self, shared_server, mt):
"""Control: without -U the destination atime is not the source's old
value (it reflects the fresh write, i.e. now), proving -U is what
restores the source atime."""
source = os.path.join(TEST_DATA_DIR, f"atime_ctrl_{'m' if mt else 's'}_src")
dest = os.path.join(TEST_DATA_DIR, f"atime_ctrl_{'m' if mt else 's'}_dst")
src_file, atime = self._make_source(source, dest)
now = time.time()
flags = (["-m"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, f"control run failed: {(result.stderr or '')[:200]}"
received = get_dest_received_dir(dest, source)
dst_st = os.stat(os.path.join(received, "data.txt"))
# The fresh destination atime is ~now, not the source's year-2000 value.
assert abs(dst_st.st_atime - atime) > 24 * 3600, \
f"control dest atime {dst_st.st_atime} unexpectedly equals source atime {atime}"
assert abs(dst_st.st_atime - now) < 24 * 3600, \
f"control dest atime {dst_st.st_atime} not ~now ({now})"
class TestOpenNoatime:
"""--open-noatime opens the source with O_NOATIME so a transfer read does
not bump the source's access time. O_NOATIME is honoured for a file owned
by the reading process (or with CAP_FOWNER), so it works as non-root here;
where it is unavailable/refused FastSync degrades to a normal open and the
assertion below is skipped."""
@pytest.mark.skipif(not sys.platform.startswith("linux"),
reason="O_NOATIME is Linux-specific")
@pytest.mark.parametrize("mt", [False, True])
def test_open_noatime_preserves_source_atime(self, shared_server, mt):
source = os.path.join(TEST_DATA_DIR, f"noatime_{'m' if mt else 's'}_src")
dest = os.path.join(TEST_DATA_DIR, f"noatime_{'m' if mt else 's'}_dst")
clean_dir(source)
clean_dir(dest)
path = os.path.join(source, "data.txt")
with open(path, "wb") as f:
f.write(b"open-noatime payload\n")
atime = 730486800 # 1993-02-11, distinct and far from now
os.utime(path, ns=(atime * 10**9, atime * 10**9))
flags = ["--open-noatime"] + (["-m"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, \
f"--open-noatime failed: {(result.stderr or result.stdout)[:300]}"
after = os.stat(path)
assert abs(after.st_atime - atime) < 1.5, \
f"source atime {after.st_atime} was bumped by the readable read (wanted {atime})"
class TestCrtimes:
"""-N/--crtimes captures and transmits the source birth time. There is no
portable way to SET a birth time (utimensat only sets atime/mtime), so the
receiver deliberately does not apply it. The run must succeed without
crashing; we do not assert the destination birth time changed. When the
platform exposes a birth time (statx STATX_BTIME on Linux) we additionally
confirm a capture path exists."""
@pytest.mark.parametrize("mt", [False, True])
def test_crtimes_run_succeeds(self, shared_server, mt):
source = os.path.join(TEST_DATA_DIR, f"crtime_{'m' if mt else 's'}_src")
dest = os.path.join(TEST_DATA_DIR, f"crtime_{'m' if mt else 's'}_dst")
clean_dir(source)
clean_dir(dest)
path = os.path.join(source, "data.txt")
payload = b"crtime transfer payload\n"
with open(path, "wb") as f:
f.write(payload)
flags = ["-N"] + (["-m"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, \
f"-N failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
dst_file = os.path.join(received, "data.txt")
assert os.path.exists(dst_file)
with open(dst_file, "rb") as f:
assert f.read() == payload
def test_crtimes_combines_with_atimes(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "crtime_atime_combined_src")
dest = os.path.join(TEST_DATA_DIR, "crtime_atime_combined_dst")
clean_dir(source)
clean_dir(dest)
path = os.path.join(source, "data.txt")
with open(path, "wb") as f:
f.write(b"combined U N payload\n")
atime = 946684800
os.utime(path, ns=(atime * 10**9, 951782400 * 10**9))
result, _ = run_client(source, dest, flags=["-U", "-N"],
port=shared_server.port)
assert result.returncode == 0, \
f"-U -N failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
dst_st = os.stat(os.path.join(received, "data.txt"))
assert abs(dst_st.st_atime - atime) < 1.5, \
f"combined -U -N dest atime {dst_st.st_atime} != {atime}"
class TestOmitTimes:
"""-O/--omit-dir-times and -J/--omit-link-times are recognized and cross the
wire as receiver-side preferences. FastSync does not currently apply dir or
symlink times at all, so they are forward-compatible preferences: the run
must succeed and normal transfers must not break. A regular file's mtime
(from -M) is unaffected by -O/-J."""
@pytest.mark.parametrize("flag", ["-O", "-J"])
@pytest.mark.parametrize("mt", [False, True])
def test_omit_times_accepted(self, shared_server, flag, mt):
source = os.path.join(TEST_DATA_DIR, f"omit_{flag.strip('-')}_{'m' if mt else 's'}_src")
dest = os.path.join(TEST_DATA_DIR, f"omit_{flag.strip('-')}_{'m' if mt else 's'}_dst")
clean_dir(source)
clean_dir(dest)
with open(os.path.join(source, "a.txt"), "wb") as f:
f.write(b"omit times content\n")
flags = [flag] + (["-m"] if mt else [])
result, _ = run_client(source, dest, flags=flags, port=shared_server.port)
assert result.returncode == 0, \
f"{flag} failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
mismatches, missing = verify_transfer(source, received)
assert not missing, f"Missing: {missing}"
assert not mismatches, f"Mismatch: {mismatches}"
@pytest.mark.ci
def test_omit_times_with_dirs_and_regular_metadata(self, shared_server):
source = os.path.join(TEST_DATA_DIR, "omit_dirs_meta_src")
dest = os.path.join(TEST_DATA_DIR, "omit_dirs_meta_dst")
clean_dir(source)
clean_dir(dest)
os.makedirs(os.path.join(source, "subdir"))
with open(os.path.join(source, "f.txt"), "wb") as f:
f.write(b"regular mtime preserved under -O/-J\n")
result, _ = run_client(source, dest, flags=["-d", "--omit-dir-times"],
port=shared_server.port)
assert result.returncode == 0, \
f"-d -O failed: {(result.stderr or result.stdout)[:300]}"
result, _ = run_client(source, dest, flags=["-M", "-O", "-J"],
port=shared_server.port)
assert result.returncode == 0, \
f"-M -O -J failed: {(result.stderr or result.stdout)[:300]}"
received = get_dest_received_dir(dest, source)
mismatches, missing = verify_transfer(source, received)
assert not missing and not mismatches, f"missing={missing} mismatches={mismatches}"
+2 -2
View File
@@ -118,11 +118,11 @@ static void test_chunk_dir_entry_roundtrip() {
dir->is_dir = true;
if (use_metadata) {
reg->metadata = file_metadata_create(&st);
reg->metadata = file_metadata_create(file_path, &st, false, false);
EXPECT_NOT_NULL(reg->metadata);
struct stat dst;
EXPECT_EQ_INT(stat(dir_path, &dst), 0);
dir->metadata = file_metadata_create(&dst);
dir->metadata = file_metadata_create(dir_path, &dst, false, false);
EXPECT_NOT_NULL(dir->metadata);
}
+99 -2
View File
@@ -768,8 +768,6 @@ static void test_parse_args_delete_timing_without_delete_rejected() {
static void test_parse_args_rejects_unimplemented_options() {
static const char* const options[] = {"--silent",
"--queue-size",
"-H",
"--hard-links",
"-A",
"--acls",
"-X",
@@ -845,6 +843,52 @@ static void test_parse_args_update() {
config_delete(cfg);
}
static void test_parse_args_hard_links() {
Config* cfg = config_create();
char* argv_H[] = {"fastsync", "-H", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv_H, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->preserve_hard_links);
config_delete(cfg);
cfg = config_create();
positional_count = 0;
char* argv_long[] = {"fastsync", "--hard-links", "/src", "/dst"};
EXPECT_EQ_INT(parse_args(cfg, 4, argv_long, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->preserve_hard_links);
config_delete(cfg);
/* --no-hard-links clears the flag. */
cfg = config_create();
positional_count = 0;
char* argv_neg[] = {"fastsync", "--hard-links", "--no-hard-links", "/src", "/dst"};
EXPECT_EQ_INT(parse_args(cfg, 5, argv_neg, positional_args, &positional_count), 0);
EXPECT_FALSE(cfg->preserve_hard_links);
config_delete(cfg);
}
/* -H/--hard-links violates the per-file streaming requirement of -s and the
* payload-bearing tail-resume of --append: both combos are rejected up front. */
static void test_validate_config_hard_links_incompatible_modes() {
Config* cfg = config_create();
char* argv_s[] = {"fastsync", "-H", "-s", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 5, argv_s, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->preserve_hard_links);
EXPECT_FALSE(validate_config(cfg));
config_delete(cfg);
cfg = config_create();
positional_count = 0;
char* argv_append[] = {"fastsync", "-H", "--append", "/src", "/dst"};
EXPECT_EQ_INT(parse_args(cfg, 5, argv_append, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->preserve_hard_links);
EXPECT_FALSE(validate_config(cfg));
config_delete(cfg);
}
static void test_parse_args_info_flags() {
Config* cfg = config_create();
char* argv[] = {"fastsync", "--info=copy,skip", "/src", "/dst"};
@@ -2254,6 +2298,54 @@ static void test_parse_args_preallocate() {
config_delete(cfg);
}
/* Phase 4 metadata-time flags parse and set the expected config fields. -U and
* -N imply metadata transmission (they carry their times inside the metadata
* payload); -O/-J and --open-noatime do not. */
static void test_parse_args_metadata_times() {
Config* cfg = config_create();
cfg->send_directory = str_dup("/src");
cfg->receive_root_directory = str_dup("/dst");
char* argv[] = {"fastsync", "-U", "-N", "-O", "-J", "--open-noatime", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 8, argv, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->preserve_atimes);
EXPECT_TRUE(cfg->preserve_crtimes);
EXPECT_TRUE(cfg->omit_dir_times);
EXPECT_TRUE(cfg->omit_link_times);
EXPECT_TRUE(cfg->open_noatime);
/* -U/-N carry their times inside the metadata payload, so they imply it. */
EXPECT_TRUE(cfg->use_metadata);
EXPECT_TRUE(validate_config(cfg));
config_delete(cfg);
}
static void test_parse_args_atimes_long_and_short() {
Config* cfg = config_create();
cfg->send_directory = str_dup("/src");
cfg->receive_root_directory = str_dup("/dst");
char* argv[] = {"fastsync", "--atimes", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->preserve_atimes);
EXPECT_TRUE(cfg->use_metadata);
config_delete(cfg);
}
static void test_parse_args_omit_link_times_long() {
Config* cfg = config_create();
cfg->send_directory = str_dup("/src");
cfg->receive_root_directory = str_dup("/dst");
char* argv[] = {"fastsync", "--omit-link-times", "/src", "/dst"};
int positional_args[2];
int positional_count = 0;
EXPECT_EQ_INT(parse_args(cfg, 4, argv, positional_args, &positional_count), 0);
EXPECT_TRUE(cfg->omit_link_times);
EXPECT_FALSE(cfg->use_metadata);
config_delete(cfg);
}
void test_client_cli() {
test_validate_config_required_paths();
test_parse_args_numeric_ids();
@@ -2263,6 +2355,9 @@ void test_client_cli() {
test_parse_args_chown();
test_parse_args_rejects_malformed_identity();
test_parse_args_preallocate();
test_parse_args_metadata_times();
test_parse_args_atimes_long_and_short();
test_parse_args_omit_link_times_long();
test_parse_args_append();
test_parse_args_append_verify();
test_parse_args_append_both();
@@ -2314,6 +2409,8 @@ void test_client_cli() {
test_parse_args_rejects_unimplemented_options();
test_parse_args_quiet();
test_parse_args_human_readable();
test_parse_args_hard_links();
test_validate_config_hard_links_incompatible_modes();
test_parse_args_update();
test_parse_args_info_flags();
test_parse_args_info_verbose_order();
+49
View File
@@ -126,6 +126,7 @@ static void test_config_send_receive() {
send_cfg->use_compression = true;
send_cfg->use_metadata = true;
send_cfg->use_executability = true;
send_cfg->preserve_hard_links = true;
send_cfg->use_delta = true;
send_cfg->whole_file = true;
send_cfg->fuzzy = true;
@@ -181,6 +182,8 @@ static void test_config_send_receive() {
ok = false;
if (!recv_cfg->use_executability)
ok = false;
if (!recv_cfg->preserve_hard_links)
ok = false;
if (!recv_cfg->size_only)
ok = false;
if (!recv_cfg->ignore_times)
@@ -911,6 +914,51 @@ static void test_config_receive_rejects_invalid_checksum_algo() {
/* The identity-mapping fields (--numeric-ids / --usermap / --groupmap /
--chown) cross the config wire unchanged: the receiver needs them to apply
ownership with the same policy the client requested. */
static void test_config_metadata_times_wire_roundtrip() {
if (is_running_under_valgrind())
return;
Config* send_cfg = config_create();
EXPECT_NOT_NULL(send_cfg);
send_cfg->send_directory = str_dup("/send/src");
send_cfg->receive_root_directory = str_dup("/send/dst");
send_cfg->preserve_atimes = true;
send_cfg->preserve_crtimes = true;
send_cfg->omit_dir_times = true;
send_cfg->omit_link_times = true;
/* --open-noatime is client-only and must NOT cross the wire. */
send_cfg->open_noatime = true;
int p[2];
EXPECT_EQ_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, p), 0);
io_set_fds(p[0], p[1]);
io_set_bwlimit(0);
pid_t pid = fork();
if (pid == 0) {
close(p[1]);
io_set_fds(p[0], p[0]);
Config* recv = config_receive(p[0]);
bool ok = recv != NULL;
if (ok) {
ok = recv->preserve_atimes && recv->preserve_crtimes && recv->omit_dir_times &&
recv->omit_link_times && !recv->open_noatime;
}
config_delete(recv);
close(p[0]);
close(p[1]);
_exit(ok ? 0 : 1);
} else {
close(p[0]);
io_set_fds(p[1], p[1]);
bool sent = config_send(p[1], send_cfg);
int status;
waitpid(pid, &status, 0);
close(p[1]);
config_delete(send_cfg);
EXPECT_TRUE(sent);
EXPECT_TRUE(WIFEXITED(status) && WEXITSTATUS(status) == 0);
}
}
static void test_config_identity_wire_roundtrip() {
if (is_running_under_valgrind())
return;
@@ -1068,6 +1116,7 @@ void test_config() {
test_config_receive_rejects_invalid_checksum_algo();
test_config_identity_wire_roundtrip();
test_config_receive_rejects_invalid_identity();
test_config_metadata_times_wire_roundtrip();
test_config_preallocate_wire_roundtrip();
}
test_config_delete_timing_early_helper();
+3 -3
View File
@@ -596,7 +596,7 @@ static void test_file_metadata_create() {
struct stat st;
EXPECT_EQ_INT(stat("test_meta_file.txt", &st), 0);
FileMetadata* m = file_metadata_create(&st);
FileMetadata* m = file_metadata_create("test_meta_file.txt", &st, false, false);
EXPECT_NOT_NULL(m);
EXPECT_EQ_INT(m->mode, st.st_mode);
EXPECT_EQ_INT(m->uid, st.st_uid);
@@ -712,7 +712,7 @@ static void test_file_send_single_calls_metadata_and_path() {
file->data->data = malloc(len);
EXPECT_NOT_NULL(file->data->data);
memcpy(file->data->data, content, len);
file->metadata = file_metadata_create(&st);
file->metadata = file_metadata_create("test_meta_send.txt", &st, false, false);
EXPECT_NOT_NULL(file->metadata);
Config* cfg = config_create();
@@ -857,7 +857,7 @@ static void test_inplace_overwrite_metadata_strips_special_bits() {
EXPECT_NOT_NULL(f->data->data);
memcpy(f->data->data, new_content, strlen(new_content));
f->data->size = strlen(new_content);
f->metadata = file_metadata_create(&source_st);
f->metadata = file_metadata_create(source, &source_st, false, false);
EXPECT_NOT_NULL(f->metadata);
Config* cfg = config_create();
+1 -1
View File
@@ -106,7 +106,7 @@ static void test_fuzz_metadata_from_buf() {
struct stat st;
EXPECT_EQ_INT(stat("fuzz_meta_test.txt", &st), 0);
FileMetadata* meta = file_metadata_create(&st);
FileMetadata* meta = file_metadata_create("fuzz_meta_test.txt", &st, false, false);
EXPECT_NOT_NULL(meta);
EXPECT_EQ_INT((int)meta->mode, (int)st.st_mode);
EXPECT_EQ_INT((int)meta->mtime_sec, (int)st.st_mtime);
+97
View File
@@ -15,6 +15,12 @@ static void test_metadata_to_from_buf_roundtrip() {
original.gid = 1000;
original.mtime_sec = 1234567890;
original.mtime_nsec = 500000000;
original.atime_valid = true;
original.atime_sec = 1234567000;
original.atime_nsec = 250000000;
original.crtime_valid = true;
original.crtime_sec = 1200000000;
original.crtime_nsec = 750000000;
char* buf = malloc(FILE_METADATA_WIRE_SIZE + sizeof(int));
EXPECT_NOT_NULL(buf);
@@ -30,6 +36,14 @@ static void test_metadata_to_from_buf_roundtrip() {
EXPECT_EQ_INT(result->gid, 1000);
EXPECT_EQ_INT(result->mtime_sec, 1234567890);
EXPECT_EQ_INT(result->mtime_nsec, 500000000);
EXPECT_TRUE(result->atime_valid);
EXPECT_EQ_INT(result->atime_sec, 1234567000);
EXPECT_EQ_INT(result->atime_nsec, 250000000);
EXPECT_TRUE(result->crtime_valid);
EXPECT_EQ_INT(result->crtime_sec, 1200000000);
EXPECT_EQ_INT(result->crtime_nsec, 750000000);
EXPECT_EQ_INT((int)(read_ptr - buf), (int)FILE_METADATA_WIRE_SIZE + (int)sizeof(int));
free(result);
free(buf);
@@ -75,6 +89,12 @@ static void test_metadata_send_receive_roundtrip() {
original.gid = 1000;
original.mtime_sec = 1234567890;
original.mtime_nsec = 500000000;
original.atime_valid = false;
original.atime_sec = 0;
original.atime_nsec = 0;
original.crtime_valid = true;
original.crtime_sec = 1200000000;
original.crtime_nsec = 750000000;
EXPECT_TRUE(metadata_send(p[1], &original));
@@ -87,6 +107,10 @@ static void test_metadata_send_receive_roundtrip() {
EXPECT_EQ_INT(received->gid, 1000);
EXPECT_EQ_INT(received->mtime_sec, 1234567890);
EXPECT_EQ_INT(received->mtime_nsec, 500000000);
EXPECT_FALSE(received->atime_valid);
EXPECT_TRUE(received->crtime_valid);
EXPECT_EQ_INT(received->crtime_sec, 1200000000);
EXPECT_EQ_INT(received->crtime_nsec, 750000000);
free(received);
close(p[0]);
@@ -123,6 +147,77 @@ static void test_metadata_rejects_invalid_values() {
close(p[1]);
}
/* metadata_receive must reject an out-of-range atime/crtime nsec even when the
* flag would otherwise be valid (defense-in-depth on the -U/-N wire fields). */
static void test_metadata_receive_rejects_bad_optional_times() {
int p[2];
EXPECT_EQ_INT(pipe(p), 0);
io_set_fds(p[0], p[1]);
int32_t present = 1;
int32_t mode = 0644;
int32_t uid = 1000;
int32_t gid = 1000;
int64_t mtime_sec = 1;
int64_t mtime_nsec = 0;
int32_t atime_valid = 1;
int64_t atime_sec = 1;
int64_t atime_nsec = 2000000000; /* invalid: >= 1e9 */
EXPECT_TRUE(send_n_data(p[1], &present, sizeof(present)));
EXPECT_TRUE(send_n_data(p[1], &mode, sizeof(mode)));
EXPECT_TRUE(send_n_data(p[1], &uid, sizeof(uid)));
EXPECT_TRUE(send_n_data(p[1], &gid, sizeof(gid)));
EXPECT_TRUE(send_n_data(p[1], &mtime_sec, sizeof(mtime_sec)));
EXPECT_TRUE(send_n_data(p[1], &mtime_nsec, sizeof(mtime_nsec)));
EXPECT_TRUE(send_n_data(p[1], &atime_valid, sizeof(atime_valid)));
EXPECT_TRUE(send_n_data(p[1], &atime_sec, sizeof(atime_sec)));
EXPECT_TRUE(send_n_data(p[1], &atime_nsec, sizeof(atime_nsec)));
int32_t crtime_valid = 0;
int64_t crtime_sec = 0;
int64_t crtime_nsec = 0;
EXPECT_TRUE(send_n_data(p[1], &crtime_valid, sizeof(crtime_valid)));
EXPECT_TRUE(send_n_data(p[1], &crtime_sec, sizeof(crtime_sec)));
EXPECT_TRUE(send_n_data(p[1], &crtime_nsec, sizeof(crtime_nsec)));
int ok = 1;
EXPECT_NULL(metadata_receive(p[0], &ok));
EXPECT_EQ_INT(ok, 0);
close(p[0]);
close(p[1]);
}
/* file_restore_metadata applies the source atime alongside mtime when -U
* captured it (atime_valid set). */
static void test_file_restore_metadata_applies_atime() {
const char* path = "temp_meta_atime_test.txt";
EXPECT_TRUE(file_write_to_disk(path, "atime", 5, false, false));
FileMetadata m;
m.mode = 0644;
m.uid = getuid();
m.gid = getgid();
m.mtime_sec = 1234567890;
m.mtime_nsec = 0;
m.atime_valid = true;
m.atime_sec = 999999999;
m.atime_nsec = 123456789;
m.crtime_valid = false;
m.crtime_sec = 0;
m.crtime_nsec = 0;
file_restore_metadata(path, &m, false);
struct stat st;
EXPECT_EQ_INT(stat(path, &st), 0);
EXPECT_EQ_INT((int)st.st_mtime, 1234567890);
#ifdef __linux__
EXPECT_EQ_INT((int)st.st_atime, 999999999);
#else
EXPECT_EQ_INT((int)st.st_atime, 999999999);
#endif
unlink(path);
}
static void test_metadata_mtime_window() {
EXPECT_TRUE(metadata_mtime_matches(100, 100000000, 101, 600000000, 2));
EXPECT_FALSE(metadata_mtime_matches(100, 100000000, 102, 600000000, 2));
@@ -214,8 +309,10 @@ void test_metadata() {
test_metadata_send_receive_roundtrip();
test_metadata_send_null();
test_metadata_rejects_invalid_values();
test_metadata_receive_rejects_bad_optional_times();
test_metadata_mtime_window();
test_file_restore_metadata();
test_file_restore_metadata_applies_atime();
test_file_restore_executability_only();
test_directory_restore_executability_only();
test_chmod_changes();