Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions src/MeshWeaver.Documentation/Data/Architecture/GitHubSync.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,8 +150,28 @@ Import reuses the platform's content-addressed import pipeline (fingerprint gate
activity lock + canonical upsert + prune) — see
[StaticRepoImport.md](/Doc/Architecture/StaticRepoImport).

### Two-way — never overwrite changes made on the server

By default import is **git-first**: an update overwrites (and prunes) the live node from the
repo. That silently loses edits made on the server *between* syncs. Turn on **Two-way** (a
checkbox on the sync source) to change the conflict rule to **newest-writer-wins per node**:

- A node whose live `LastModified` is newer than the last recorded sync (`LastSyncedAt`) was
changed on the server since you last synced. An **Update to latest** **keeps** that node —
it is neither overwritten nor pruned — so your local change is carried back to GitHub on the
next **Commit** ("newer on the server wins → GitHub"). A node the server hasn't touched since
the last sync is still updated from the repo as usual.
- Two-way only takes effect once a first sync has recorded `LastSyncedAt`; the initial import is
git-first (there is nothing local to protect yet).

**Force update** is the escape hatch: it ignores two-way and overwrites/prunes from the repo
regardless — use it to deliberately discard local changes back to the repository state. Via MCP:
`git_hub_sync(space, op:"update", force:true)`.

> **Take-over edits survive.** A node you've edited and marked to exclude from sync is
> not overwritten or pruned by a re-import — that's how you "claim" content locally.
> Two-way generalizes this to *every* server-side edit (no explicit claim needed) as long as it
> post-dates the last sync.

---

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
Name: Two-way GitHub sync — your edits are kept, not overwritten
Category: What's New
Description: A GitHub sync source can now keep changes made on the server since the last sync, instead of overwriting them on update.
Icon: Sparkle
---

# Two-way GitHub sync

GitHub sync used to be strictly git-first: an **Update to latest** overwrote (and pruned) the live
Space from the repository, silently discarding edits you'd made on the server between syncs.

Turn on **Two-way** on a sync source and an update now keeps any node changed on the server since
the last sync — it is preserved, not overwritten or pruned, and is carried back to GitHub on your
next **Commit** ("newer on the server wins"). Nodes the server hasn't touched still update from the
repo as before.

Need the old behavior for a one-off? A **forced** update (`force`) overwrites local changes from the
repository regardless — the deliberate way to discard server edits and match the repo exactly.
15 changes: 9 additions & 6 deletions src/MeshWeaver.GitSync/GitHubActivityExtensions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -55,16 +55,19 @@ public static IObservable<string> CommitToGitHub(
/// <paramref name="sourceId"/> selects the sync source (null = the primary).</summary>
public static IObservable<string> UpdateToLatestFromGitHub(
this IMessageHub hub, string spacePath, string userId, Action<string>? onActivityCreated = null,
string? sourceId = null)
string? sourceId = null, bool force = false)
{
var pr = hub.ServiceProvider.GetRequiredService<PullRequestService>();
return hub.RunActivity(spacePath, ActivityCategory.Import, $"Update {spacePath} to latest",
return hub.RunActivity(spacePath, ActivityCategory.Import,
force ? $"Force-update {spacePath} to latest" : $"Update {spacePath} to latest",
ctx =>
{
ctx.Log("Fetching the branch HEAD from GitHub and importing the deltas…");
ctx.Log(force
? "Fetching the branch HEAD from GitHub and overwriting local changes (force)…"
: "Fetching the branch HEAD from GitHub and importing the deltas…");
// ctx.Log as the progress sink: files dropped from the import (parse failures)
// append an Error line here and flip the terminal status to Failed.
return pr.UpdateToLatest(spacePath, userId, sourceId, ctx.Log).Select(r =>
return pr.UpdateToLatest(spacePath, userId, sourceId, ctx.Log, force).Select(r =>
{
ctx.Log($"Imported {r.Outcome} ({r.Count} node(s)).");
return Unit.Default;
Expand All @@ -76,7 +79,7 @@ public static IObservable<string> UpdateToLatestFromGitHub(
/// <paramref name="sourceId"/> selects the sync source (null = the primary).</summary>
public static IObservable<string> ReimportFromGitHub(
this IMessageHub hub, string spacePath, string commitish, string userId,
Action<string>? onActivityCreated = null, string? sourceId = null)
Action<string>? onActivityCreated = null, string? sourceId = null, bool force = false)
{
var sync = hub.ServiceProvider.GetRequiredService<GitHubSyncService>();
return hub.RunActivity(spacePath, ActivityCategory.Import, $"Re-import {spacePath} at {commitish}",
Expand All @@ -85,7 +88,7 @@ public static IObservable<string> ReimportFromGitHub(
ctx.Log($"Fetching {commitish} from GitHub and importing the deltas…");
// ctx.Log as the progress sink: files dropped from the import (parse failures)
// append an Error line here and flip the terminal status to Failed.
return sync.ReimportAtCommit(spacePath, commitish, userId, sourceId, ctx.Log).Select(r =>
return sync.ReimportAtCommit(spacePath, commitish, userId, sourceId, ctx.Log, force).Select(r =>
{
ctx.Log($"Re-imported {r.Outcome} ({r.Count} node(s)) at {commitish}.");
return Unit.Default;
Expand Down
16 changes: 16 additions & 0 deletions src/MeshWeaver.GitSync/GitHubSyncConfig.cs
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,22 @@ public record GitHubSyncConfig
[Description("Sync direction")]
public SyncDirection Direction { get; init; } = SyncDirection.Bidirectional;

/// <summary>
/// Two-way conflict resolution on <b>import</b> ("Update to latest"). When true, an update
/// NEVER overwrites (or prunes) a node that was changed on the server SINCE THE LAST SYNC —
/// the server copy is preserved and carried back to GitHub on the next commit ("newer on the
/// server wins → GitHub"). When false (the default) import is git-first: the repo overwrites
/// the live node, which silently loses local edits made between syncs. A <b>forced</b> update
/// (<c>force: true</c>) overrides this and overwrites regardless — the escape hatch for
/// deliberately discarding local changes back to the repo state.
///
/// <para>"Since the last sync" is measured against <see cref="LastSyncedAt"/>: a node whose
/// <c>MeshNode.LastModified</c> is newer than the last recorded sync counts as a local
/// edit. Only takes effect once a first sync has recorded that timestamp.</para>
/// </summary>
[Description("Two-way (don't overwrite nodes changed on the server since the last sync — commit them back instead)")]
public bool TwoWay { get; init; }

/// <summary>Create <see cref="Branch"/> if it does not exist yet. Default true.</summary>
[Description("Create the branch if it doesn't exist")]
public bool CreateBranchIfMissing { get; init; } = true;
Expand Down
32 changes: 25 additions & 7 deletions src/MeshWeaver.GitSync/GitHubSyncService.cs
Original file line number Diff line number Diff line change
Expand Up @@ -337,10 +337,12 @@ public IObservable<StaticRepoImportResult> ImportFromGitHub(
/// <paramref name="progress"/> (when the caller runs as an activity: <c>ctx.Log</c>)
/// receives per-file problems — a repo file that failed to parse and was therefore
/// dropped from the import — so they land on the activity log, not only in Loki.
/// <paramref name="force"/> ignores the source's two-way setting and overwrites/prunes
/// from the repo regardless (the deliberate-discard escape hatch).
/// </summary>
public IObservable<StaticRepoImportResult> ReimportAtCommit(
string spacePath, string commitish, string userId, string? sourceId = null,
Action<string, LogLevel>? progress = null)
Action<string, LogLevel>? progress = null, bool force = false)
{
return ReadConfig(spacePath, sourceId).Take(1).SelectMany(config =>
{
Expand All @@ -351,13 +353,27 @@ public IObservable<StaticRepoImportResult> ReimportAtCommit(
return Observable.Throw<StaticRepoImportResult>(new InvalidOperationException(
$"This sync source is export-only (mesh → repo): importing from {repoUrl} is not allowed. " +
"Change the source's Sync direction to Bidirectional or Import-only to re-import."));
// Two-way (config.TwoWay): don't overwrite/prune nodes changed on the server since the last
// recorded sync (config.LastSyncedAt) — they are carried back on the next commit. `force`
// overrides. Git-first when TwoWay is off (unchanged legacy behavior).
var policy = new ImportConflictPolicy(config.TwoWay, config.LastSyncedAt, force);
return ResolveAuth(userId).SelectMany(auth =>
{
var token = auth.Token;
logger?.LogInformation("Re-importing {Space} at {Ref}", spacePath, commitish);
logger?.LogInformation("Re-importing {Space} at {Ref} (twoWay={TwoWay}, force={Force})",
spacePath, commitish, config.TwoWay, force);
return FetchAndImport(repoUrl, commitish, config.Subdirectory, token, spacePath,
SyncIgnore.For(config), progress)
.SelectMany(x => RecordLastSync(spacePath, x.CommitSha, sourceId).Select(_ => x.Result));
SyncIgnore.For(config), progress, policy)
// 🚨 Only advance the last-sync baseline when the mesh is now IN SYNC with the repo.
// A two-way import that PRESERVED server-newer nodes leaves the mesh AHEAD of the repo
// (those edits aren't committed back yet); advancing LastSyncedAt would move the
// baseline past them, so the NEXT update would no longer see them as "newer on the
// server" and would overwrite them — losing the very edits two-way just protected.
// Leave the baseline until a commit carries them back (which advances it). A clean
// import (nothing preserved — always the case git-first) records normally.
.SelectMany(x => x.Result.Preserved > 0
? Observable.Return(x.Result)
: RecordLastSync(spacePath, x.CommitSha, sourceId).Select(_ => x.Result));
});
});
}
Expand Down Expand Up @@ -411,14 +427,14 @@ private IObservable<ResolvedGitHubAuth> ResolveAuth(string userId) =>

private IObservable<(StaticRepoImportResult Result, string CommitSha)> FetchAndImport(
string repoUrl, string commitish, string? subdirectory, string token, string spaceId,
SyncIgnore ignore, Action<string, LogLevel>? progress = null)
SyncIgnore ignore, Action<string, LogLevel>? progress = null, ImportConflictPolicy? policy = null)
{
return repoClient.Fetch(repoUrl, commitish, subdirectory, token).SelectMany(snapshot =>
ParseSnapshot(snapshot, spaceId, ignore, progress).SelectMany(parsed =>
{
var source = new InMemoryStaticRepoSource(
spaceId, parsed.Children, parsed.Root, parsed.ContentSyncs);
return StaticRepoImporter.ImportSource(hub, source, logger)
return StaticRepoImporter.ImportSource(hub, source, logger, policy)
.Select(result => (result, snapshot.CommitSha));
}));
}
Expand Down Expand Up @@ -636,7 +652,8 @@ public IObservable<MeshNode> EnsureConfigNode(string spacePath, string? sourceId
public IObservable<MeshNode> SaveConfig(
string spacePath, string? repositoryUrl, string branch, string? subdirectory,
bool createBranchIfMissing, bool createRepoIfMissing,
SyncDirection direction = SyncDirection.Bidirectional, string? sourceId = null)
SyncDirection direction = SyncDirection.Bidirectional, string? sourceId = null,
bool twoWay = false)
=> UpdateConfig(spacePath, c => c with
{
RepositoryUrl = string.IsNullOrWhiteSpace(repositoryUrl) ? null : repositoryUrl.Trim(),
Expand All @@ -645,6 +662,7 @@ public IObservable<MeshNode> SaveConfig(
CreateBranchIfMissing = createBranchIfMissing,
CreateRepoIfMissing = createRepoIfMissing,
Direction = direction,
TwoWay = twoWay,
}, sourceId);

/// <summary>Read-modify-write a config field (current value from the synced query; null when absent).</summary>
Expand Down
8 changes: 5 additions & 3 deletions src/MeshWeaver.GitSync/PullRequestService.cs
Original file line number Diff line number Diff line change
Expand Up @@ -104,19 +104,21 @@ public IObservable<GitHubBranchResult> CreateBranch(
/// the checkout operation: the working Space is brought to the latest repo state.
/// <paramref name="progress"/> (when the caller runs as an activity: <c>ctx.Log</c>)
/// receives per-file import problems so they land on the activity log.
/// <paramref name="force"/> ignores the source's two-way setting and overwrites local
/// edits from the repo (deliberate discard).
/// </summary>
public IObservable<StaticRepoImportResult> UpdateToLatest(
string spacePath, string userId, string? sourceId = null,
Action<string, LogLevel>? progress = null)
Action<string, LogLevel>? progress = null, bool force = false)
{
return sync.ReadConfig(spacePath, sourceId).Take(1).SelectMany(config =>
{
if (config?.RepositoryUrl is not { Length: > 0 })
return Observable.Throw<StaticRepoImportResult>(new InvalidOperationException(
"No GitHub repository configured for this Space."));
var branch = string.IsNullOrWhiteSpace(config.Branch) ? "main" : config.Branch;
logger?.LogInformation("Updating {Space} to latest on {Branch}", spacePath, branch);
return sync.ReimportAtCommit(spacePath, branch, userId, sourceId, progress);
logger?.LogInformation("Updating {Space} to latest on {Branch} (force={Force})", spacePath, branch, force);
return sync.ReimportAtCommit(spacePath, branch, userId, sourceId, progress, force);
});
}

Expand Down
Loading
Loading