cairnquire/.project/specs/simplesync-protocol.md

SimpleSync Protocol Specification

Overview

SimpleSync is Cairnquire's intended synchronization model for asynchronous markdown collaboration. It covers three related surfaces:

1. The Go server watches the configured content directory and reflects local filesystem edits into the document index.
2. The Go-served browser UI caches documents in IndexedDB, queues offline edits, and resolves conflicts before writing over newer server content.
3. Native or external clients can use snapshot/delta endpoints for bidirectional file sync.

The protocol is content-addressed: every document version is identified by the SHA-256 hash of its bytes. The server never accepts a write that claims an older base hash without returning a conflict response.

Design Goals

• Automatic: Local filesystem edits and browser edits propagate without a manual import/export step.
• Conflict-aware: Concurrent edits to the same file are detected and surfaced; they are not silently overwritten.
• Offline-capable: Browser edits can be queued locally and synced when connectivity returns.
• File-level simple: Different files can be edited independently without CRDTs, branches, or a git mental model.
• Verifiable: Content hashes are recomputed by the server on every stored version.

Milestone 2 Scope

Included:

• Recursive filesystem watching for markdown documents.
• Debounced sync after editor save bursts.
• Ignoring hidden files, hidden directories, and common editor temp files.
• WebSocket document-change notifications.
• IndexedDB document cache and pending edit queue.
• HTTP save conflict detection using baseHash.
• Manual browser conflict resolution with server/queued diff and resolved Monaco editor.
• Native snapshot/delta/content endpoints for future external clients.
• Simple fixed-size cleanup for cached rendered pages/content.

Out of scope for Milestone 2:

• WebSocket authentication. Authentication and authorization belong to Milestone 3 and should be integrated with the final app session model, not a temporary shared secret.
• Playwright or browser-level automated sync tests.
• Performance benchmarking for large sync sets.
• Quota-aware IndexedDB eviction. Milestone 2 uses a fixed entry cap instead.

Content Addressing

All persisted document content is immutable and addressed by SHA-256:

hash = sha256(document_bytes)
store path = <store-dir>/<hash[0:2]>/<hash[2:4]>/<hash>

Identical content deduplicates naturally. The latest document table points at the current hash for each path.

Filesystem Watcher

The server watches the configured source directory recursively.

Watcher behavior:

• Watch existing directories on startup.
• Add watches for newly-created directories.
• Process markdown paths only: .md and .MD.
• Ignore hidden path segments such as .git, .cache, and .note.md.
• Ignore common editor temporary files such as *.swp, *.tmp, *.bak, *.orig, *~, and #file#.
• Debounce rapid event bursts before running a full SyncSourceDir().
• Keep the 30 second polling fallback so missed fsnotify events are eventually reconciled.

The watcher callback performs a full source-directory sync. This keeps the fsnotify layer simple and avoids depending on platform-specific event details for correctness.

Browser Editor Sync

The browser editor uses the document's current server hash as an optimistic concurrency token.

Load

{
  "path": "guide.md",
  "content": "# Guide\n",
  "hash": "server-hash"
}

The browser stores the loaded content and hash in IndexedDB.

Save

{
  "content": "# Guide\n\nLocal edit\n",
  "baseHash": "server-hash"
}

If baseHash still matches the server's current hash, the server writes the file, stores the new content hash, and returns the updated document.

If the server has changed since the browser loaded the document, the server returns 409 Conflict:

{
  "status": "conflict",
  "path": "guide.md",
  "baseHash": "server-hash",
  "currentHash": "new-server-hash",
  "currentContent": "# Guide\n\nServer edit\n"
}

The browser keeps the queued local edit in IndexedDB, shows a diff between the server copy and the queued edit, and lets the user produce a resolved document. Submitting the resolved document uses the latest currentHash as the next baseHash.

Offline Queue

When a save cannot reach the server, the browser records a pending edit:

{
  "path": "guide.md",
  "content": "# Guide\n\nQueued edit\n",
  "baseHash": "server-hash",
  "status": "queued",
  "updatedAt": "2026-05-14T12:00:00Z"
}

On reconnect or manual sync, queued edits are retried. Conflicts remain local until resolved by the user.

Cache Cleanup

The browser keeps pending edits until they sync or the user resolves them. Rendered page/content caches use a fixed entry cap and evict the oldest cachedAt records after successful cache writes. This gives predictable bounded growth without adding quota-estimation complexity before production hardening.

Realtime Notifications

WebSocket notifications are advisory. They tell connected browser clients that indexed server state changed; durable sync correctness still comes from hash-checked HTTP reads and writes.

Current intended message families:

{
  "type": "document_version",
  "path": "guide.md",
  "hash": "new-server-hash"
}

{
  "type": "folder_tree_changed"
}

Clients should reconnect automatically and refresh state after reconnect. If notifications are missed, the polling fallback and next document load still converge on server state.

Native Snapshot/Delta Sync

External clients use HTTP endpoints instead of browser IndexedDB state.

Initialize

POST /api/sync/init

{
  "deviceId": "macbook-pro-1",
  "rootPath": "/Users/alice/Documents/cairnquire"
}

Response:

{
  "snapshotId": "snap:device:timestamp",
  "serverSnapshot": [
    {
      "path": "guide.md",
      "hash": "sha256...",
      "size": 42,
      "modified": "2026-05-14T12:00:00Z"
    }
  ]
}

Delta

POST /api/sync/delta

{
  "snapshotId": "snap:device:timestamp",
  "clientDelta": [
    {
      "type": "update",
      "path": "guide.md",
      "hash": "client-hash",
      "content": "# Guide\n\nClient edit\n",
      "size": 58,
      "modified": "2026-05-14T12:05:00Z"
    }
  ]
}

Response:

{
  "serverDelta": [
    {
      "type": "create",
      "path": "new-note.md",
      "hash": "server-hash",
      "size": 100,
      "modified": "2026-05-14T12:03:00Z"
    }
  ],
  "newSnapshotId": "snap:device:new-timestamp",
  "conflicts": [
    {
      "path": "guide.md",
      "serverHash": "server-hash",
      "clientHash": "client-hash",
      "serverModified": "2026-05-14T12:03:00Z",
      "clientModified": "2026-05-14T12:05:00Z",
      "strategy": "manual-merge"
    }
  ]
}

For create/update deltas, native clients should send inline UTF-8 markdown in content. The server stores the bytes content-addressably, verifies any claimed hash, writes the markdown into the content tree, and returns a fresh newSnapshotId for the client's next sync request.

Resolve

POST /api/sync/resolve

{
  "snapshotId": "snap:device:timestamp",
  "resolutions": [
    {
      "path": "guide.md",
      "strategy": "server-wins"
    }
  ]
}

Response:

{
  "newSnapshotId": "snap:device:new-timestamp"
}

Content Fetch

GET /api/content/{hash}

Returns immutable raw bytes:

Content-Type: application/octet-stream
Cache-Control: public, max-age=31536000, immutable
ETag: "<hash>"

Conflict Rules

A same-file conflict exists when:

1. The client is editing from a known base hash.
2. The server's current hash for that path changed after that base hash.
3. The client attempts to write different content for the same path.

Different paths do not conflict. Deletions and renames are represented in the native delta model and should be resolved at file-path granularity.

Resolution strategies:

• manual-merge: User edits a resolved document and saves it against the latest server hash.
• server-wins: Keep the current server version.
• client-wins: Replace the server version with the client version after explicit user choice.
• rename-both: Preserve both versions using a conflict filename.
• last-write-wins: Allowed for automation, but not the default browser UX because it can hide data loss.

State Machine

                          online
        +----------------------------------+
        |                                  v
  +-----+-----+   edit/save ok       +-----+------+
  |  Reading  +--------------------->+   Synced   |
  +-----+-----+                      +-----+------+
        |                                  ^
        | edit while offline               |
        v                                  |
  +-----+------+   reconnect/manual sync   |
  |   Queued   +---------------------------+
  +-----+------+
        |
        | server hash changed
        v
  +-----+------+
  | Conflicted |
  +-----+------+
        |
        | user resolves against latest hash
        v
  +-----+------+
  |   Synced   |
  +------------+

Error Codes

Code	Description	Retryable
conflict	Write base hash is stale	No, user resolution required
not_found	Requested document or content hash is unknown	Sometimes
hash_mismatch	Content does not match claimed hash	Yes, after recompute
too_large	Content exceeds configured size limit	No
rate_limited	Too many requests	Yes, with backoff
server_error	Internal server error	Yes

Security Considerations

• Milestone 2 does not add temporary WebSocket auth.
• Milestone 3 should apply session/authz checks consistently to document reads, document writes, WebSocket upgrade, and sync endpoints.
• The server must canonicalize paths and reject traversal outside the configured content root.
• The server recomputes SHA-256 before storing or accepting content.
• Raw HTML markdown support is intended for trusted content repositories; untrusted public authoring requires a sanitizer policy before launch.

Version

Protocol Version: 1.0
Last Updated: 2026-05-14