cairnquire/.project/specs/simplesync-protocol.md

# SimpleSync Protocol Specification

## Overview

SimpleSync is Cairnquire's intended synchronization model for asynchronous markdown collaboration. It covers three related surfaces:

1. The Go server watches the configured content directory and reflects local filesystem edits into the document index.
2. The Go-served browser UI caches documents in IndexedDB, queues offline edits, and resolves conflicts before writing over newer server content.
3. Native or external clients can use snapshot/delta endpoints for bidirectional file sync.

The protocol is content-addressed: every document version is identified by the SHA-256 hash of its bytes. The server never accepts a write that claims an older base hash without returning a conflict response.

## Design Goals

- **Automatic:** Local filesystem edits and browser edits propagate without a manual import/export step.
- **Conflict-aware:** Concurrent edits to the same file are detected and surfaced; they are not silently overwritten.
- **Offline-capable:** Browser edits can be queued locally and synced when connectivity returns.
- **File-level simple:** Different files can be edited independently without CRDTs, branches, or a git mental model.
- **Verifiable:** Content hashes are recomputed by the server on every stored version.

## Milestone 2 Scope

Included:

- Recursive filesystem watching for markdown documents.
- Debounced sync after editor save bursts.
- Ignoring hidden files, hidden directories, and common editor temp files.
- WebSocket document-change notifications.
- IndexedDB document cache and pending edit queue.
- HTTP save conflict detection using `baseHash`.
- Manual browser conflict resolution with server/queued diff and resolved Monaco editor.
- Native snapshot/delta/content endpoints for future external clients.
- Simple fixed-size cleanup for cached rendered pages/content.

Out of scope for Milestone 2:

- WebSocket authentication. Authentication and authorization belong to Milestone 3 and should be integrated with the final app session model, not a temporary shared secret.
- Playwright or browser-level automated sync tests.
- Performance benchmarking for large sync sets.
- Quota-aware IndexedDB eviction. Milestone 2 uses a fixed entry cap instead.

## Content Addressing

All persisted document content is immutable and addressed by SHA-256:

```text
hash = sha256(document_bytes)
store path = <store-dir>/<hash[0:2]>/<hash[2:4]>/<hash>
```

Identical content deduplicates naturally. The latest document table points at the current hash for each path.

## Filesystem Watcher

The server watches the configured source directory recursively.

Watcher behavior:

- Watch existing directories on startup.
- Add watches for newly-created directories.
- Process markdown paths only: `.md` and `.MD`.
- Ignore hidden path segments such as `.git`, `.cache`, and `.note.md`.
- Ignore common editor temporary files such as `*.swp`, `*.tmp`, `*.bak`, `*.orig`, `*~`, and `#file#`.
- Debounce rapid event bursts before running a full `SyncSourceDir()`.
- Keep the 30 second polling fallback so missed fsnotify events are eventually reconciled.

The watcher callback performs a full source-directory sync. This keeps the fsnotify layer simple and avoids depending on platform-specific event details for correctness.

## Browser Editor Sync

The browser editor uses the document's current server hash as an optimistic concurrency token.

### Load

```json
{
  "path": "guide.md",
  "content": "# Guide\n",
  "hash": "server-hash"
}
```

The browser stores the loaded content and hash in IndexedDB.

### Save

```json
{
  "content": "# Guide\n\nLocal edit\n",
  "baseHash": "server-hash"
}
```

If `baseHash` still matches the server's current hash, the server writes the file, stores the new content hash, and returns the updated document.

If the server has changed since the browser loaded the document, the server returns `409 Conflict`:

```json
{
  "status": "conflict",
  "path": "guide.md",
  "baseHash": "server-hash",
  "currentHash": "new-server-hash",
  "currentContent": "# Guide\n\nServer edit\n"
}
```

The browser keeps the queued local edit in IndexedDB, shows a diff between the server copy and the queued edit, and lets the user produce a resolved document. Submitting the resolved document uses the latest `currentHash` as the next `baseHash`.

### Offline Queue

When a save cannot reach the server, the browser records a pending edit:

```json
{
  "path": "guide.md",
  "content": "# Guide\n\nQueued edit\n",
  "baseHash": "server-hash",
  "status": "queued",
  "updatedAt": "2026-05-14T12:00:00Z"
}
```

On reconnect or manual sync, queued edits are retried. Conflicts remain local until resolved by the user.

### Cache Cleanup

The browser keeps pending edits until they sync or the user resolves them. Rendered page/content caches use a fixed entry cap and evict the oldest `cachedAt` records after successful cache writes. This gives predictable bounded growth without adding quota-estimation complexity before production hardening.

## Realtime Notifications

WebSocket notifications are advisory. They tell connected browser clients that indexed server state changed; durable sync correctness still comes from hash-checked HTTP reads and writes.

Current intended message families:

```json
{
  "type": "document_version",
  "path": "guide.md",
  "hash": "new-server-hash"
}
```

```json
{
  "type": "folder_tree_changed"
}
```

Clients should reconnect automatically and refresh state after reconnect. If notifications are missed, the polling fallback and next document load still converge on server state.

## Native Snapshot/Delta Sync

External clients use HTTP endpoints instead of browser IndexedDB state.

### Initialize

`POST /api/sync/init`

```json
{
  "deviceId": "macbook-pro-1",
  "rootPath": "/Users/alice/Documents/cairnquire"
}
```

Response:

```json
{
  "snapshotId": "snap:device:timestamp",
  "serverSnapshot": [
    {
      "path": "guide.md",
      "hash": "sha256...",
      "size": 42,
      "modified": "2026-05-14T12:00:00Z"
    }
  ]
}
```

### Delta

`POST /api/sync/delta`

```json
{
  "snapshotId": "snap:device:timestamp",
  "clientDelta": [
    {
      "type": "update",
      "path": "guide.md",
      "hash": "client-hash",
      "size": 58,
      "modified": "2026-05-14T12:05:00Z"
    }
  ]
}
```

Response:

```json
{
  "serverDelta": [
    {
      "type": "create",
      "path": "new-note.md",
      "hash": "server-hash",
      "size": 100,
      "modified": "2026-05-14T12:03:00Z"
    }
  ],
  "conflicts": [
    {
      "path": "guide.md",
      "serverHash": "server-hash",
      "clientHash": "client-hash",
      "serverModified": "2026-05-14T12:03:00Z",
      "clientModified": "2026-05-14T12:05:00Z",
      "strategy": "manual-merge"
    }
  ]
}
```

### Resolve

`POST /api/sync/resolve`

```json
{
  "snapshotId": "snap:device:timestamp",
  "resolutions": [
    {
      "path": "guide.md",
      "strategy": "server-wins"
    }
  ]
}
```

Response:

```json
{
  "newSnapshotId": "snap:device:new-timestamp"
}
```

### Content Fetch

`GET /api/content/{hash}`

Returns immutable raw bytes:

```text
Content-Type: application/octet-stream
Cache-Control: public, max-age=31536000, immutable
ETag: "<hash>"
```

## Conflict Rules

A same-file conflict exists when:

1. The client is editing from a known base hash.
2. The server's current hash for that path changed after that base hash.
3. The client attempts to write different content for the same path.

Different paths do not conflict. Deletions and renames are represented in the native delta model and should be resolved at file-path granularity.

Resolution strategies:

- `manual-merge`: User edits a resolved document and saves it against the latest server hash.
- `server-wins`: Keep the current server version.
- `client-wins`: Replace the server version with the client version after explicit user choice.
- `rename-both`: Preserve both versions using a conflict filename.
- `last-write-wins`: Allowed for automation, but not the default browser UX because it can hide data loss.

## State Machine

```text
                          online
        +----------------------------------+
        |                                  v
  +-----+-----+   edit/save ok       +-----+------+
  |  Reading  +--------------------->+   Synced   |
  +-----+-----+                      +-----+------+
        |                                  ^
        | edit while offline               |
        v                                  |
  +-----+------+   reconnect/manual sync   |
  |   Queued   +---------------------------+
  +-----+------+
        |
        | server hash changed
        v
  +-----+------+
  | Conflicted |
  +-----+------+
        |
        | user resolves against latest hash
        v
  +-----+------+
  |   Synced   |
  +------------+
```

## Error Codes

| Code | Description | Retryable |
|------|-------------|-----------|
| `conflict` | Write base hash is stale | No, user resolution required |
| `not_found` | Requested document or content hash is unknown | Sometimes |
| `hash_mismatch` | Content does not match claimed hash | Yes, after recompute |
| `too_large` | Content exceeds configured size limit | No |
| `rate_limited` | Too many requests | Yes, with backoff |
| `server_error` | Internal server error | Yes |

## Security Considerations

- Milestone 2 does not add temporary WebSocket auth.
- Milestone 3 should apply session/authz checks consistently to document reads, document writes, WebSocket upgrade, and sync endpoints.
- The server must canonicalize paths and reject traversal outside the configured content root.
- The server recomputes SHA-256 before storing or accepting content.
- Raw HTML markdown support is intended for trusted content repositories; untrusted public authoring requires a sanitizer policy before launch.

## Version

**Protocol Version:** 1.0\
**Last Updated:** 2026-05-14