cairnquire/.project/specs/sync-protocol-v1.md

# Sync Protocol v1 Specification

## Overview

The Native Sync Protocol enables bidirectional filesystem synchronization between a native macOS client and the cairnquire server. The browser client continues to operate via the existing WebSocket + REST API.

## Data Model

### Snapshot

A point-in-time tree of all files in the sync scope:

```json
{
  "id": "snap:device-1:1234567890",
  "deviceId": "device-1",
  "createdAt": "2024-01-15T10:30:00Z",
  "files": [
    {
      "path": "hello.md",
      "hash": "a1b2c3...",
      "size": 42,
      "modified": "2024-01-15T10:00:00Z"
    }
  ]
}
```

### Delta

A list of changes since a snapshot:

```json
{
  "changes": [
    {
      "type": "create|update|delete|rename",
      "path": "new-file.md",
      "oldPath": "old-file.md",
      "hash": "a1b2c3...",
      "size": 42,
      "modified": "2024-01-15T10:00:00Z"
    }
  ]
}
```

### Conflict

When server and client both changed the same file since the last common snapshot:

```json
{
  "path": "hello.md",
  "serverHash": "a1b2c3...",
  "clientHash": "d4e5f6...",
  "serverModified": "2024-01-15T10:00:00Z",
  "clientModified": "2024-01-15T10:05:00Z",
  "strategy": "last-write-wins"
}
```

### Resolution Strategies

- `last-write-wins` (default): Use the most recently modified version
- `rename-both`: Keep both versions, renaming one as `file (conflict).md`
- `manual-merge`: Flag for UI resolution
- `server-wins`: Always use server version
- `client-wins`: Always use client version

## Endpoints

All endpoints require authentication via `X-API-Key` header.

### POST /api/sync/init

Initialize a new sync session.

**Request:**
```json
{
  "deviceId": "macbook-pro-1",
  "rootPath": "/Users/alice/Documents/md-hub"
}
```

**Response:**
```json
{
  "snapshotId": "snap:macbook-pro-1:1234567890",
  "serverSnapshot": [
    {
      "path": "hello.md",
      "hash": "a1b2c3...",
      "size": 42,
      "modified": "2024-01-15T10:00:00Z"
    }
  ]
}
```

### POST /api/sync/delta

Upload client changes and receive server changes + conflicts.

**Request:**
```json
{
  "snapshotId": "snap:macbook-pro-1:1234567890",
  "clientDelta": [
    {
      "type": "update",
      "path": "hello.md",
      "hash": "d4e5f6...",
      "size": 50,
      "modified": "2024-01-15T10:05:00Z"
    }
  ]
}
```

**Response:**
```json
{
  "serverDelta": [
    {
      "type": "create",
      "path": "new-guide.md",
      "hash": "g7h8i9...",
      "size": 100,
      "modified": "2024-01-15T10:02:00Z"
    }
  ],
  "conflicts": [
    {
      "path": "hello.md",
      "serverHash": "a1b2c3...",
      "clientHash": "d4e5f6...",
      "serverModified": "2024-01-15T10:00:00Z",
      "clientModified": "2024-01-15T10:05:00Z",
      "strategy": "last-write-wins"
    }
  ]
}
```

### POST /api/sync/resolve

Resolve conflicts and finalize the sync.

**Request:**
```json
{
  "snapshotId": "snap:macbook-pro-1:1234567890",
  "resolutions": [
    {
      "path": "hello.md",
      "strategy": "server-wins"
    }
  ]
}
```

**Response:**
```json
{
  "newSnapshotId": "snap:macbook-pro-1:1234567999"
}
```

### GET /api/content/{hash}

Fetch raw file content by content hash (immutable, cache-friendly).

**Response:** Raw file bytes with headers:
```
Content-Type: application/octet-stream
Cache-Control: public, max-age=31536000, immutable
ETag: "a1b2c3..."
```

## Protocol Flow

```
1. Client calls POST /api/sync/init
   → Receives server snapshot

2. Client computes local delta
   (fsevents + local state DB)

3. Client POSTs delta to /api/sync/delta
   → Receives serverDelta + conflicts

4. If conflicts exist:
   a. Client resolves conflicts (auto or UI)
   b. Client POSTs resolutions to /api/sync/resolve
   → Receives newSnapshotId

5. Client applies serverDelta to local filesystem
   (downloads content via /api/content/{hash})

6. Both sides now have the same snapshot
```

## Authentication

Sync endpoints require API key authentication:

```
X-API-Key: <api-key>
```

API keys are long-lived tokens scoped to a user and device. They are stored in the `api_keys` table with SHA-256 hashes.

## Implementation Details

### Server Architecture

```
internal/sync/
├── protocol.go    # Types and constants
├── service.go     # Business logic (conflict resolution, snapshot management)
├── store.go       # Database operations
```

### Database Schema

```sql
-- API keys for native client authentication
CREATE TABLE api_keys (
    id TEXT PRIMARY KEY,
    user_id TEXT NOT NULL,
    name TEXT NOT NULL,
    key_hash TEXT NOT NULL UNIQUE,
    scopes TEXT NOT NULL DEFAULT 'sync:read,sync:write',
    created_at TEXT NOT NULL,
    expires_at TEXT,
    last_used_at TEXT,
    FOREIGN KEY(user_id) REFERENCES users(id) ON DELETE CASCADE
);

-- Sync snapshots (point-in-time file trees)
CREATE TABLE sync_snapshots (
    id TEXT PRIMARY KEY,
    device_id TEXT NOT NULL,
    user_id TEXT NOT NULL,
    created_at TEXT NOT NULL,
    FOREIGN KEY(user_id) REFERENCES users(id) ON DELETE CASCADE
);

-- Files within a snapshot
CREATE TABLE sync_files (
    snapshot_id TEXT NOT NULL,
    path TEXT NOT NULL,
    hash TEXT NOT NULL,
    size_bytes INTEGER NOT NULL DEFAULT 0,
    modified_at TEXT NOT NULL,
    PRIMARY KEY (snapshot_id, path),
    FOREIGN KEY(snapshot_id) REFERENCES sync_snapshots(id) ON DELETE CASCADE
);
```

### Conflict Detection

A conflict occurs when:
1. The server file existed in the snapshot
2. The server file has changed since the snapshot (hash changed)
3. The client also changed the same file (different hash in delta)

### Content-Addressed Storage

All file content is stored in a content-addressable store using SHA-256 hashes:
- Storage path: `<store-dir>/<hash[0:2]>/<hash[2:4]>/<hash>`
- Immutable: once stored, content never changes
- Deduplication: identical content shares the same storage

## Testing

The sync protocol includes table-driven tests that simulate client/server scenarios:

- `TestInitSyncCreatesSnapshot`: Validates snapshot generation
- `TestApplyDeltaDetectsServerChanges`: Detects server-side modifications
- `TestApplyDeltaDetectsConflicts`: Identifies concurrent modifications
- `TestResolveConflictsCreatesNewSnapshot`: Validates conflict resolution
- `TestGetContentReturnsFileBytes`: Tests content retrieval