update docs

This commit is contained in:
2026-05-28 08:35:25 -04:00
parent d359baa010
commit 3d44a392f1
5 changed files with 533 additions and 380 deletions

View File

@@ -7,71 +7,73 @@
### Week 3: Server-Side Sync
- [ ] Content-addressed filesystem implementation
- SHA-256 hash computation
- Directory layout: `store/ab/cd/abcdef...`
- Write-once semantics
- Deduplication verification
- [x] Content-addressed filesystem implementation
- [x] SHA-256 hash computation
- [x] Directory layout: `store/ab/cd/abcdef...`
- [x] Write-once semantics
- [x] Deduplication verification
- [ ] File watcher (fsnotify)
- Watch configured directory recursively
- Debounce rapid changes (1s window)
- Ignore patterns (.git, temp files)
- Compute hash on change, update database
- [x] File watcher (fsnotify)
- [x] Watch configured directory recursively
- [x] Debounce rapid changes
- [x] Ignore hidden paths and common editor temp files
- [x] Compute hash on change, update database
- [ ] WebSocket server
- Connection management (connection pool)
- Message framing and parsing
- Authentication over WebSocket (token validation)
- Heartbeat/ping-pong for connection health
- [x] WebSocket server
- [x] Connection management (connection pool)
- [x] Message framing and parsing
- WebSocket authentication intentionally out of scope for Milestone 2
- [x] Heartbeat/ping-pong for connection health
- [ ] SimpleSync protocol server implementation
- Sync request/response handler
- Root hash computation (Merkle tree)
- Missing content detection
- Conflict detection logic
- [x] SimpleSync protocol server implementation
- [x] Sync request/response handler
- [x] Root hash computation
- [x] Missing content detection
- [x] Conflict detection logic
### Week 4: Client-Side Sync & Offline Support
- [ ] WebSocket client (Preact)
- Connection management with auto-reconnect
- Message queue for offline periods
- Exponential backoff for reconnection
- [x] WebSocket client (Go-served UI)
- [x] Connection management with auto-reconnect
- [x] Message queue for offline periods
- [x] Reconnection after offline periods
- [ ] IndexedDB cache layer
- Store file content by hash
- Store sync state and pending changes
- LRU eviction for cache management
- [x] IndexedDB cache layer
- [x] Store document/page content for offline reads
- [x] Store sync state and pending changes
- [x] Simple fixed cap for cached pages/content
- [x] Preserve pending edits during cache cleanup
- [ ] File sync UI
- Sync status indicator
- Offline mode banner
- Pending changes list
- Manual sync trigger button
- [x] File sync UI
- [x] Sync status indicator
- [x] Offline mode banner
- [x] Pending changes list
- [x] Manual sync trigger button
- [ ] Conflict detection display
- Visual indicator for conflicting files
- Basic conflict resolution UI (choose local/server)
- [x] Conflict detection display
- [x] Visual indicator for conflicting files
- [x] Side-by-side diff for server copy vs queued edit
- [x] Conflict resolution UI with Monaco resolved document editor
## Acceptance Criteria
### Functional
- [ ] File changes on disk automatically sync to server within 5 seconds
- [ ] Web client receives real-time updates when files change on server
- [ ] Client can make changes offline and sync when reconnected
- [ ] Concurrent edits to different files succeed without conflict
- [ ] Concurrent edits to same file detected as conflict
- [ ] Content hash verified on every transfer (client and server)
- [ ] Sync works across browser refresh (IndexedDB persistence)
- [x] File changes on disk automatically sync to server within 5 seconds
- [x] Web client receives real-time updates when files change on server
- [x] Client can make changes offline and sync when reconnected
- [x] Concurrent edits to different files succeed without conflict
- [x] Concurrent edits to same file detected as conflict
- [x] Content hash verified on every transfer (client and server)
- [x] Sync works across browser refresh (IndexedDB persistence)
### Non-Functional
- [ ] WebSocket connections authenticated (reject unauthenticated)
- WebSocket authentication deferred to Milestone 3 application auth
- [ ] File watcher doesn't consume >1% CPU on idle
- [ ] IndexedDB storage cleared on logout
- [ ] Sync protocol documented with sequence diagrams
- IndexedDB logout clearing deferred to Milestone 3 application auth
- [x] Sync protocol documented with state/flow diagrams
### Performance
- [ ] Sync of 100 files completes in <10 seconds
- [x] Sync of 100 files completes in <10 seconds
- [ ] WebSocket message overhead <1KB per sync cycle
- [ ] File watcher detects changes within 500ms
@@ -121,10 +123,10 @@ interface ContentResponse {
## Deliverables
1. SimpleSync protocol specification document
2. Sequence diagrams for all sync scenarios
3. WebSocket API documentation
4. Offline sync test suite
1. [x] SimpleSync protocol specification document
2. [x] State/flow diagrams for browser sync scenarios
3. [x] WebSocket message documentation
4. [x] Go-level offline/conflict sync test coverage
## Risk Mitigation
@@ -136,8 +138,8 @@ interface ContentResponse {
## Definition of Done
- [ ] All acceptance criteria pass
- [ ] Sync protocol tested with 2+ concurrent clients
- [ ] Offline/online transition tested
- [ ] Performance benchmarks meet targets
- [ ] Documentation complete
- [x] All Milestone 2 functional acceptance criteria pass
- [x] Sync protocol tested with 2+ concurrent clients
- [x] Offline/online transition tested
- [x] 100-file sync performance target validated
- [x] Documentation complete

View File

@@ -7,80 +7,91 @@
### Week 5: Passkey Authentication
- [ ] WebAuthn server implementation
- Relying party configuration
- Challenge generation and storage
- Attestation verification
- Credential storage (credential ID, public key, sign count)
- [x] WebAuthn server implementation
- [x] Relying party configuration
- [x] Challenge generation and storage
- [x] Attestation verification
- [x] Credential storage
- [ ] Passkey registration flow
- Initiate registration endpoint
- Verify registration response
- Store credential with user association
- Support multiple passkeys per user
- [x] Passkey registration flow
- [x] Initiate registration endpoint
- [x] Verify registration response
- [x] Store credential with user association
- [ ] Authenticated multiple-passkey management UI (deferred account polish)
- [ ] Passkey authentication flow
- Initiate login endpoint
- Verify assertion response
- Create session on success
- [x] Passkey authentication flow
- [x] Initiate login endpoint
- [x] Verify assertion response
- [x] Create session on success
- [ ] Session management
- Signed cookie generation
- Session storage in database
- Session validation middleware
- Session revocation endpoint
- Automatic session refresh
- [x] Session management
- [x] Opaque HttpOnly cookie generation
- [x] Session storage in database
- [x] Session validation middleware
- [x] Session revocation endpoint
- [ ] Automatic session refresh (deferred hardening)
### Week 6: Password Fallback & Authorization
- [ ] Password authentication
- Argon2id password hashing
- Password validation endpoint
- Password change endpoint
- Password strength requirements
- [x] Password authentication
- [x] Argon2id password hashing
- [x] Password validation endpoint
- [x] Password change endpoint
- [x] Password strength requirements
- [ ] Role-based access control
- Permission model: read, write, admin
- Scope: global, collection, document
- Middleware for permission checking
- Admin UI for managing permissions
- [x] Role-based access control
- [x] Permission model: viewer, editor, admin
- [x] Token scopes: docs, sync, admin
- [x] Middleware for permission checking
- [x] Admin UI for managing user roles
- [ ] User management
- User registration (email + passkey/password)
- User profile management
- Account deletion (GDPR compliance)
- User listing (admin only)
- [x] User management
- [x] User registration (email + passkey/password)
- [x] User profile management
- [x] Account deletion
- [x] User listing (admin only)
- [x] API token support
- [x] Bearer token format for developer clients
- [x] Server stores token hashes only
- [x] Token scopes and revocation
- [x] Device-code flow for CLI onboarding
- [ ] Content signing (optional)
- Ed25519 key pair generation
- Document signing on publish
- Signature verification display
- Deferred to production hardening because the current publish path does not yet need signing state.
## Acceptance Criteria
### Functional
- [ ] Users can register with passkey on supported browsers
- [ ] Passkey login works with Touch ID, Windows Hello, YubiKey
- [ ] Password fallback works when passkey unavailable
- [ ] Sessions expire after configurable timeout (default 24h)
- [ ] Sessions can be revoked (logout all devices)
- [ ] Read/write/admin permissions enforced on all endpoints
- [ ] Users can only access documents they have permission for
- [ ] Admin users can manage other users' permissions
- [x] Users can register with passkey on supported browsers
- [x] Passkey login endpoints verify browser assertions
- [x] Password fallback works when passkey unavailable
- [x] Sessions expire after default 24h timeout
- [x] Sessions can be revoked
- [x] Read/write/admin permissions enforced on protected endpoints
- [x] API clients can use scoped bearer tokens
- [x] CLI clients can use device-code flow
- [ ] Per-document and per-collection permission UI (deferred until collaboration resource modeling)
### Non-Functional
- [ ] Argon2id parameters: time=3, memory=64MB, parallelism=4
- [ ] Session cookies: HttpOnly, Secure, SameSite=Strict
- [ ] Rate limiting: 5 auth attempts per minute per IP
- [ ] No timing attacks on password comparison (constant-time)
- [ ] All auth events logged to audit_log table
- [x] Argon2id parameters: time=3, memory=64MB, parallelism=4
- [x] Session cookies: HttpOnly, SameSite=Strict
- [x] Rate limiting: 5 auth attempts per minute per IP/path
- [x] No timing attacks on password comparison (constant-time)
- [x] Password and token comparisons use constant-time comparison
- [x] Auth events logged to audit_log table
- [ ] Ed25519 signatures verified on document read (if enabled)
### Security
- [ ] WebAuthn challenges single-use and time-limited (5 minutes)
- [ ] Credential IDs are unpredictable (128-bit random)
- [ ] Password reset requires email verification
- [ ] No enumeration attacks (same response for exist/non-exist user)
- [x] WebAuthn challenges single-use and time-limited (5 minutes)
- [x] Credential IDs are generated by authenticators
- [ ] Password reset requires email verification (out of scope until email/recovery is added)
- [x] Password login uses generic invalid credential responses
- [x] Public registration cannot attach credentials to an existing account
- [x] Device-code polling consumes the code after first token issuance
## Database Schema Additions
@@ -96,6 +107,8 @@ CREATE TABLE webauthn_credentials (
last_used_at DATETIME
);
-- Sessions, API tokens, device codes, and audit log are defined in migration 000009_auth.
-- Permissions
CREATE TABLE permissions (
id TEXT PRIMARY KEY,
@@ -110,10 +123,13 @@ CREATE TABLE permissions (
## Deliverables
1. Authentication API documentation
2. WebAuthn flow diagrams
3. Permission system documentation
4. Security test results (penetration test guide)
1. [x] Authentication API documentation
2. [x] WebAuthn flow endpoints
3. [x] Role/scope permission documentation
4. [x] API token and device-flow documentation
5. [x] Browser login/account/device verification screens
6. [x] Security implementation checklist
7. [ ] Independent penetration test guide (production hardening)
## Risk Mitigation
@@ -126,9 +142,9 @@ CREATE TABLE permissions (
## Definition of Done
- [ ] All acceptance criteria pass
- [ ] Security review completed
- [ ] Auth flow tested on Chrome, Firefox, Safari, Edge
- [ ] Mobile passkey tested (iOS, Android)
- [ ] Rate limiting verified
- [ ] Audit logs verified
- [x] Core implementation acceptance criteria pass
- [x] Implementation security review completed
- [ ] Auth flow manually tested on Chrome, Firefox, Safari, Edge (production hardening)
- [ ] Mobile passkey tested (iOS, Android) (production hardening)
- [x] Rate limiting verified
- [x] Audit logs verified by service tests

View File

@@ -1,8 +1,8 @@
# Cairnquire - Project Plan
**Version:** 1.1\
**Date:** 2026-04-29\
**Status:** In Progress — Milestone 1 Complete, Milestone 2 Started
**Version:** 1.2\
**Date:** 2026-05-14\
**Status:** In Progress — Milestones 1-3 Complete
## Vision
@@ -46,40 +46,50 @@ A minimal-dependency, ultra-fast documentation platform that publishes markdown
- [x] Build reproducible (same input → same binary hash)
**Notes:**
- Mermaid and math blocks render as code blocks (no client-side JS rendering yet). Will be addressed in Milestone 5 with client-side hydration.
- Mermaid diagrams and math blocks now render client-side after server-rendered markdown loads.
- Dev mode uses `air` for Go live reload and Vite proxy for frontend HMR.
---
### Milestone 2: Sync Protocol (Weeks 3-4) — IN PROGRESS
### Milestone 2: Sync Protocol (Weeks 3-4) ✅ COMPLETE
**Goal:** Bidirectional file sync between server and filesystem/web client
- [x] Content-addressed storage (SHA-256 filesystem layout)
- [x] WebSocket sync protocol implementation
- [ ] File watcher for local filesystem sync
- [ ] Client-side IndexedDB cache
- [ ] Offline queue with conflict detection
- [x] File watcher for local filesystem sync
- [x] Client-side IndexedDB cache
- [x] Offline queue with conflict detection
**Acceptance Criteria:**
- [x] File changes on disk automatically sync to server within 5 seconds
- [ ] Web client can sync after reconnection, even with changes made offline
- [ ] Concurrent edits to different files succeed without conflict
- [ ] Concurrent edits to same file detected and queued for resolution
- [x] Web client can sync after reconnection, even with changes made offline
- [x] Concurrent edits to different files succeed without conflict
- [x] Concurrent edits to same file detected and queued for resolution
- [x] All synced content verified against SHA-256 hashes
- [ ] Sync protocol documented with state machine diagrams
- [x] Sync protocol documented with state machine diagrams
**What's Done:**
- Real-time file watcher using `fsnotify` — detects create/write/remove/rename events on `.md` files
- Changes trigger immediate `SyncSourceDir()` and broadcast `document_version` events via WebSocket
- Browser shows reload notification when current document or folder structure changes
- Fallback polling every 30 seconds ensures nothing is missed
- Go-served editor caches pages and pending edits in IndexedDB for offline reads and recovery
- Offline saves queue locally and sync after reconnection with a visible pending changes panel
- Server-side save conflicts return 409 responses with base/current hashes and server content
- Conflict resolution screen shows server copy vs queued edit, then resolves into a Monaco editor
- Static assets from the content directory can be embedded from markdown documents
- Markdown rendering now supports raw HTML when authored in trusted content
- File watcher debounces editor save bursts and ignores hidden/temp files
- SimpleSync spec documents the intended browser, watcher, realtime, and native-client feature set
- Go-level multi-client sync tests cover different-file success, same-file conflicts, and resolution behavior
- 100-file sync performance validation covers source sync and snapshot/delta paths
- IndexedDB uses a simple fixed cap for cached rendered pages/content while preserving pending edits
**What's Next:**
- Client-side IndexedDB cache for offline reading
- Offline edit queue with conflict resolution UI
- State machine diagram for sync protocol
**Deferred:**
- WebSocket authentication and logout-driven cache clearing move to Milestone 3 application auth
- Browser-level automated sync tests and larger load benchmarks move to production hardening
---
@@ -87,20 +97,42 @@ A minimal-dependency, ultra-fast documentation platform that publishes markdown
**Goal:** Secure access control with passkeys and granular permissions
- Passkey (WebAuthn) primary authentication
- Password fallback with Argon2id
- Session management with signed cookies
- Role-based access control (RBAC)
- Content signing with Ed25519 (optional per-user)
- [x] Passkey (WebAuthn) primary authentication
- [x] Password fallback with Argon2id
- [x] Session management with opaque server-side sessions
- [x] Role-based access control (RBAC)
- [x] Scoped API tokens with device-code flow for developer clients
- [ ] Content signing with Ed25519 (deferred optional hardening)
**Acceptance Criteria:**
- [ ] Users can register and log in with passkeys on modern browsers
- [ ] Password fallback works with proper hashing
- [ ] Sessions expire securely and are revocable
- [ ] Read/write/admin permissions work per-document and per-collection
- [ ] All auth endpoints rate-limited and logged
- [ ] Security audit: no plaintext secrets, no timing attacks on auth
- [x] Users can register and log in with passkeys on modern browsers
- [x] Password fallback works with proper hashing
- [x] Sessions expire securely and are revocable
- [x] Scoped API tokens and device-code flow support developer clients
- [x] Read/write/admin permissions work at route and role level
- [ ] Per-document and per-collection permission UI (deferred until collaboration resource modeling)
- [x] All auth endpoints rate-limited
- [x] Auth events logged
- [x] Security implementation review: no plaintext session/token secrets, constant-time password/token comparison
**What's Done:**
- WebAuthn begin/finish registration and login endpoints
- Argon2id password registration/login with first-user admin bootstrap
- Opaque session cookies stored as server-side hashes
- Scoped bearer API tokens stored as hashes and shown once
- First-party device-code flow for CLI/client onboarding
- Route middleware enforcing viewer/editor/admin roles plus token scopes
- In-memory auth/device-flow rate limiter for self-hosted deployments under 100 users
- Go-served login, account, API-token, role-management, and device-approval screens
- Password change and account deletion endpoints
- Public registration hardening so credentials cannot be attached to existing accounts without a session
- Device-code replay hardening so approved codes mint only one API token
**What's Next:**
- Complete the independent penetration-test checklist during production hardening
- Decide how deep per-document/per-collection permissions should go before collaboration features
- Add optional multiple-passkey management and session refresh polish if real usage calls for it
---
@@ -145,8 +177,8 @@ A minimal-dependency, ultra-fast documentation platform that publishes markdown
- [ ] All screens responsive down to 320px width
- [ ] Lighthouse score >90 on all metrics
- [ ] Initial page load <100ms (server-rendered HTML)
- [ ] Mermaid diagrams render client-side
- [ ] Math blocks render with KaTeX
- [x] Mermaid diagrams render client-side
- [x] Math blocks render with KaTeX
---
@@ -265,7 +297,7 @@ See [architecture-overview.md](architecture-overview.md) for detailed component
### Content Security
- Attachment uploads: Magic number validation, extension whitelist, size limits
- Markdown rendering: Sanitized HTML output; no raw HTML injection
- Markdown rendering: Trusted content may include raw HTML; templates still auto-escape application data
- File paths: Canonicalization prevents directory traversal
### Audit & Compliance
@@ -345,16 +377,12 @@ cairnquire/
## Next Steps
1. Complete Milestone 1: Foundation
2. 🔄 Continue Milestone 2: Sync Protocol
- Implement client-side IndexedDB cache
- Build offline edit queue
- Create sync protocol state machine diagram
3. Begin Milestone 3: Authentication & Authorization
- Set up WebAuthn/passkey infrastructure
- Implement session management
4. Add client-side Mermaid and KaTeX rendering (Milestone 5)
5. Implement SQLite FTS5 search (Milestone 5)
6. Add Renovate with dependency update cooldown policies
2. Complete Milestone 2: Sync Protocol
3. 🔄 Finish Milestone 3 hardening
- Add browser login/account management screens
- Complete security review checklist
4. Implement SQLite FTS5 search (Milestone 5)
5. Add Renovate with dependency update cooldown policies
---

View File

@@ -0,0 +1,71 @@
# Authentication API
Milestone 3 uses first-party authentication only. Cairnquire does not implement OAuth in v1.
## Browser Sessions
Browser users authenticate with passkeys or password fallback. Successful login sets an opaque `cairnquire_session` cookie. The raw session token is never stored; the database stores a SHA-256 hash and revocation metadata.
Endpoints:
- `GET /login`
- `GET /account`
- `POST /api/auth/register/password`
- `POST /api/auth/login/password`
- `POST /api/auth/logout`
- `GET /api/auth/me`
- `POST /api/auth/password/change`
- `DELETE /api/auth/account`
- `POST /api/auth/passkeys/register/begin`
- `POST /api/auth/passkeys/register/finish?challengeId=...`
- `POST /api/auth/passkeys/login/begin`
- `POST /api/auth/passkeys/login/finish?challengeId=...`
The first registered user bootstraps as `admin`. Later public registrations become `viewer`; role changes are admin-managed from the account screen. Public registration endpoints only create new accounts. Adding or changing credentials on an existing account requires an authenticated browser session.
## API Tokens
Developer clients use bearer tokens:
```http
Authorization: Bearer cq_pat_<id>_<secret>
```
The token is shown once. The server stores the token id and SHA-256 hash of the secret. Tokens can be scoped and revoked.
Endpoints:
- `GET /api/tokens`
- `POST /api/tokens`
- `DELETE /api/tokens/{id}`
Supported scopes:
- `docs:read`
- `docs:write`
- `sync:read`
- `sync:write`
- `admin`
Authorization is role plus scope: a token must have the required scope, and the owning user role must allow that operation.
## Device Flow
CLI clients can avoid handling browser cookies by using the first-party device flow:
1. Client calls `POST /api/device/start`.
2. Server returns `deviceCode`, `userCode`, `verificationUri`, and polling interval.
3. User opens the verification URL in a browser and signs in normally.
4. User approves the code with `POST /api/device/approve`.
5. Client polls `POST /api/device/token`.
6. Server returns a normal bearer API token and consumes the device code.
This is intentionally inspired by OAuth device authorization, but it is not a general OAuth provider. There are no client registrations, redirect URIs, refresh tokens, consent screens, or third-party identity dependencies.
## Route Protection
- Public: health, static assets, document reads, login/account entry pages, password/passkey begin-login/register endpoints, device start/token polling.
- Session or bearer token required: auth profile/logout, edit/save APIs, uploads, sync/content APIs, device approval.
- Admin role required: admin APIs and API token management.
WebSocket authentication is session/bearer aware through the shared middleware model. API token creation and account management require a browser session so bearer tokens cannot mint more bearer tokens.

View File

@@ -2,296 +2,332 @@
## Overview
SimpleSync is a content-addressed file synchronization protocol designed for asynchronous collaboration on markdown documents. It provides automatic bidirectional sync between a server (source of truth) and clients (filesystem watchers or web browsers).
SimpleSync is Cairnquire's intended synchronization model for asynchronous markdown collaboration. It covers three related surfaces:
1. The Go server watches the configured content directory and reflects local filesystem edits into the document index.
2. The Go-served browser UI caches documents in IndexedDB, queues offline edits, and resolves conflicts before writing over newer server content.
3. Native or external clients can use snapshot/delta endpoints for bidirectional file sync.
The protocol is content-addressed: every document version is identified by the SHA-256 hash of its bytes. The server never accepts a write that claims an older base hash without returning a conflict response.
## Design Goals
1. **Simplicity**: No branches, commits, or manual sync commands
2. **Automatic**: Changes propagate without user intervention
3. **Conflict-aware**: Detects concurrent edits, never silently loses data
4. **Offline-capable**: Queue changes locally, sync when connected
5. **Secure**: All content verified by cryptographic hash
- **Automatic:** Local filesystem edits and browser edits propagate without a manual import/export step.
- **Conflict-aware:** Concurrent edits to the same file are detected and surfaced; they are not silently overwritten.
- **Offline-capable:** Browser edits can be queued locally and synced when connectivity returns.
- **File-level simple:** Different files can be edited independently without CRDTs, branches, or a git mental model.
- **Verifiable:** Content hashes are recomputed by the server on every stored version.
## Concepts
## Milestone 2 Scope
### Content Addressing
Every file is identified by the SHA-256 hash of its content. The content is immutable — if a file changes, it gets a new hash.
Included:
### Merkle Tree
The directory state is represented as a Merkle tree where:
- Leaf nodes are file hashes
- Internal nodes are hashes of concatenated child hashes
- The root hash represents the entire directory state
- Recursive filesystem watching for markdown documents.
- Debounced sync after editor save bursts.
- Ignoring hidden files, hidden directories, and common editor temp files.
- WebSocket document-change notifications.
- IndexedDB document cache and pending edit queue.
- HTTP save conflict detection using `baseHash`.
- Manual browser conflict resolution with server/queued diff and resolved Monaco editor.
- Native snapshot/delta/content endpoints for future external clients.
- Simple fixed-size cleanup for cached rendered pages/content.
### Sync State
Each client maintains:
- `root_hash`: Hash of current directory Merkle root
- `files`: Map of file paths to content hashes
- `pending`: Queue of local changes not yet synced
Out of scope for Milestone 2:
## Message Format
- WebSocket authentication. Authentication and authorization belong to Milestone 3 and should be integrated with the final app session model, not a temporary shared secret.
- Playwright or browser-level automated sync tests.
- Performance benchmarking for large sync sets.
- Quota-aware IndexedDB eviction. Milestone 2 uses a fixed entry cap instead.
All messages are JSON over WebSocket.
## Content Addressing
### Client → Server
All persisted document content is immutable and addressed by SHA-256:
```text
hash = sha256(document_bytes)
store path = <store-dir>/<hash[0:2]>/<hash[2:4]>/<hash>
```
Identical content deduplicates naturally. The latest document table points at the current hash for each path.
## Filesystem Watcher
The server watches the configured source directory recursively.
Watcher behavior:
- Watch existing directories on startup.
- Add watches for newly-created directories.
- Process markdown paths only: `.md` and `.MD`.
- Ignore hidden path segments such as `.git`, `.cache`, and `.note.md`.
- Ignore common editor temporary files such as `*.swp`, `*.tmp`, `*.bak`, `*.orig`, `*~`, and `#file#`.
- Debounce rapid event bursts before running a full `SyncSourceDir()`.
- Keep the 30 second polling fallback so missed fsnotify events are eventually reconciled.
The watcher callback performs a full source-directory sync. This keeps the fsnotify layer simple and avoids depending on platform-specific event details for correctness.
## Browser Editor Sync
The browser editor uses the document's current server hash as an optimistic concurrency token.
### Load
#### Sync Request
```json
{
"type": "sync_request",
"client_id": "uuid-v4",
"root_hash": "sha256-hex-64-chars",
"files": {
"getting-started.md": "sha256...",
"api-reference.md": "sha256..."
},
"timestamp": "2024-01-15T14:30:00Z"
"path": "guide.md",
"content": "# Guide\n",
"hash": "server-hash"
}
```
#### Content Upload
The browser stores the loaded content and hash in IndexedDB.
### Save
```json
{
"type": "content_upload",
"hash": "sha256...",
"content": "base64-encoded-content",
"encoding": "base64"
"content": "# Guide\n\nLocal edit\n",
"baseHash": "server-hash"
}
```
#### Acknowledgment
If `baseHash` still matches the server's current hash, the server writes the file, stores the new content hash, and returns the updated document.
If the server has changed since the browser loaded the document, the server returns `409 Conflict`:
```json
{
"type": "ack",
"message_id": "uuid-of-original-message",
"status": "ok"
"status": "conflict",
"path": "guide.md",
"baseHash": "server-hash",
"currentHash": "new-server-hash",
"currentContent": "# Guide\n\nServer edit\n"
}
```
### Server → Client
The browser keeps the queued local edit in IndexedDB, shows a diff between the server copy and the queued edit, and lets the user produce a resolved document. Submitting the resolved document uses the latest `currentHash` as the next `baseHash`.
### Offline Queue
When a save cannot reach the server, the browser records a pending edit:
#### Sync Response
```json
{
"type": "sync_response",
"server_root_hash": "sha256...",
"missing_from_client": ["hash1", "hash2"],
"missing_from_server": ["hash3"],
"conflicts": ["getting-started.md"],
"timestamp": "2024-01-15T14:30:01Z"
"path": "guide.md",
"content": "# Guide\n\nQueued edit\n",
"baseHash": "server-hash",
"status": "queued",
"updatedAt": "2026-05-14T12:00:00Z"
}
```
#### Content Push
On reconnect or manual sync, queued edits are retried. Conflicts remain local until resolved by the user.
### Cache Cleanup
The browser keeps pending edits until they sync or the user resolves them. Rendered page/content caches use a fixed entry cap and evict the oldest `cachedAt` records after successful cache writes. This gives predictable bounded growth without adding quota-estimation complexity before production hardening.
## Realtime Notifications
WebSocket notifications are advisory. They tell connected browser clients that indexed server state changed; durable sync correctness still comes from hash-checked HTTP reads and writes.
Current intended message families:
```json
{
"type": "content_push",
"hash": "sha256...",
"content": "base64-encoded-content",
"encoding": "base64",
"path": "getting-started.md"
"type": "document_version",
"path": "guide.md",
"hash": "new-server-hash"
}
```
#### Conflict Notification
```json
{
"type": "conflict",
"path": "getting-started.md",
"server_hash": "sha256...",
"client_hash": "sha256...",
"server_modified_at": "2024-01-15T14:30:00Z",
"client_modified_at": "2024-01-15T14:25:00Z"
"type": "folder_tree_changed"
}
```
#### Error
Clients should reconnect automatically and refresh state after reconnect. If notifications are missed, the polling fallback and next document load still converge on server state.
## Native Snapshot/Delta Sync
External clients use HTTP endpoints instead of browser IndexedDB state.
### Initialize
`POST /api/sync/init`
```json
{
"type": "error",
"code": "unauthorized",
"message": "Session expired",
"retryable": false
"deviceId": "macbook-pro-1",
"rootPath": "/Users/alice/Documents/cairnquire"
}
```
## Protocol Flow
Response:
### Normal Sync (No Conflicts)
```
Client Server
| |
|-- sync_request -------------->|
| root_hash: abc |
| |
| |-- Compare with server state
| |-- Calculate deltas
| |
|<- sync_response --------------|
| missing_from_client: [def] |
| missing_from_server: [] |
| conflicts: [] |
| |
|-- content_request(def) ------>|
| |
|<- content_push(def) ----------|
| |
|-- ack ----------------------->|
```json
{
"snapshotId": "snap:device:timestamp",
"serverSnapshot": [
{
"path": "guide.md",
"hash": "sha256...",
"size": 42,
"modified": "2026-05-14T12:00:00Z"
}
]
}
```
### Upload Changes
### Delta
```
Client Server
| |
|-- sync_request -------------->|
| root_hash: abc |
| files: {a: hash1, b: hash2} |
| |
|<- sync_response --------------|
| missing_from_server: [hash2] |
| |
|-- content_upload(hash2) ----->|
| |
| |-- Verify hash
| |-- Store content
| |-- Update database
| |-- Broadcast to others
| |
|<- ack ------------------------|
| |
`POST /api/sync/delta`
```json
{
"snapshotId": "snap:device:timestamp",
"clientDelta": [
{
"type": "update",
"path": "guide.md",
"hash": "client-hash",
"size": 58,
"modified": "2026-05-14T12:05:00Z"
}
]
}
```
### Conflict Detection
Response:
```json
{
"serverDelta": [
{
"type": "create",
"path": "new-note.md",
"hash": "server-hash",
"size": 100,
"modified": "2026-05-14T12:03:00Z"
}
],
"conflicts": [
{
"path": "guide.md",
"serverHash": "server-hash",
"clientHash": "client-hash",
"serverModified": "2026-05-14T12:03:00Z",
"clientModified": "2026-05-14T12:05:00Z",
"strategy": "manual-merge"
}
]
}
```
Client A Server Client B
| | |
| | |-- Edit file X
| | |-- sync_request
| |-- Update file X |
| |-- Broadcast to A |
| | |
|-- Edit file X | |
|-- sync_request -------------->| |
| |-- Detect conflict |
| |-- A has old hash |
| |-- B has new hash |
|<- conflict -------------------| |
| path: X | |
| server_hash: B_hash | |
| client_hash: A_hash | |
| | |
### Resolve
`POST /api/sync/resolve`
```json
{
"snapshotId": "snap:device:timestamp",
"resolutions": [
{
"path": "guide.md",
"strategy": "server-wins"
}
]
}
```
Response:
```json
{
"newSnapshotId": "snap:device:new-timestamp"
}
```
### Content Fetch
`GET /api/content/{hash}`
Returns immutable raw bytes:
```text
Content-Type: application/octet-stream
Cache-Control: public, max-age=31536000, immutable
ETag: "<hash>"
```
## Conflict Rules
A same-file conflict exists when:
1. The client is editing from a known base hash.
2. The server's current hash for that path changed after that base hash.
3. The client attempts to write different content for the same path.
Different paths do not conflict. Deletions and renames are represented in the native delta model and should be resolved at file-path granularity.
Resolution strategies:
- `manual-merge`: User edits a resolved document and saves it against the latest server hash.
- `server-wins`: Keep the current server version.
- `client-wins`: Replace the server version with the client version after explicit user choice.
- `rename-both`: Preserve both versions using a conflict filename.
- `last-write-wins`: Allowed for automation, but not the default browser UX because it can hide data loss.
## State Machine
```
+--------+ sync_request
| +-----------+
| Idle | |
| |<----------+
+---+----+
|
| connect
v
+---------+---------+
| |
| Connected |
| |
+---------+---------+
|
| sync_request
v
+---------+---------+
| |
| Comparing |
| |
+---------+---------+
|
+-----------+-----------+
| |
| has_changes | no_changes
v v
+---------+---------+ +---------+---------+
| | | |
| Transferring | | Idle |
| | | |
+---------+---------+ +-------------------+
|
| complete
v
+---------+---------+
| |
| Idle |
| |
+-------------------+
^
| conflict
|
+---------+---------+
| |
| Conflicted |
| |
+-------------------+
```text
online
+----------------------------------+
| v
+-----+-----+ edit/save ok +-----+------+
| Reading +--------------------->+ Synced |
+-----+-----+ +-----+------+
| ^
| edit while offline |
v |
+-----+------+ reconnect/manual sync |
| Queued +---------------------------+
+-----+------+
|
| server hash changed
v
+-----+------+
| Conflicted |
+-----+------+
|
| user resolves against latest hash
v
+-----+------+
| Synced |
+------------+
```
## Error Codes
| Code | Description | Retryable |
|------|-------------|-----------|
| `unauthorized` | Session expired or invalid | No (re-authenticate) |
| `forbidden` | Insufficient permissions | No |
| `not_found` | Requested content hash unknown | Yes |
| `too_large` | Content exceeds size limit | No |
| `hash_mismatch` | Content doesn't match claimed hash | Yes |
| `rate_limited` | Too many requests | Yes (with backoff) |
| `conflict` | Write base hash is stale | No, user resolution required |
| `not_found` | Requested document or content hash is unknown | Sometimes |
| `hash_mismatch` | Content does not match claimed hash | Yes, after recompute |
| `too_large` | Content exceeds configured size limit | No |
| `rate_limited` | Too many requests | Yes, with backoff |
| `server_error` | Internal server error | Yes |
## Security Considerations
1. **Authentication**: All WebSocket connections must authenticate via token in initial HTTP upgrade request
2. **Authorization**: Server verifies read/write permissions before serving or accepting content
3. **Hash Verification**: Server recomputes SHA-256 of received content and rejects mismatches
4. **Size Limits**: Maximum file size enforced (configurable, default 10MB)
5. **Rate Limiting**: Sync requests limited per client (configurable, default 10/minute)
6. **Path Validation**: All file paths canonicalized and checked against allowlist
## Implementation Notes
### Hash Computation
```go
import "crypto/sha256"
func computeHash(content []byte) string {
h := sha256.Sum256(content)
return hex.EncodeToString(h[:])
}
```
### Merkle Root Computation
```go
func computeRootHash(files map[string]string) string {
// Sort paths for determinism
paths := sortedKeys(files)
hasher := sha256.New()
for _, path := range paths {
hasher.Write([]byte(path))
hasher.Write([]byte(files[path]))
}
return hex.EncodeToString(hasher.Sum(nil))
}
```
### WebSocket Connection
- Ping/pong every 30 seconds
- Connection timeout after 60 seconds without response
- Auto-reconnect with exponential backoff (1s, 2s, 4s, 8s, max 60s)
- Milestone 2 does not add temporary WebSocket auth.
- Milestone 3 should apply session/authz checks consistently to document reads, document writes, WebSocket upgrade, and sync endpoints.
- The server must canonicalize paths and reject traversal outside the configured content root.
- The server recomputes SHA-256 before storing or accepting content.
- Raw HTML markdown support is intended for trusted content repositories; untrusted public authoring requires a sanitizer policy before launch.
## Version
**Protocol Version**: 1.0
**Last Updated**: 2024-01-15
**Protocol Version:** 1.0\
**Last Updated:** 2026-05-14