# Cairnquire - Project Plan

**Version:** 1.3\
**Date:** 2026-06-01\
**Status:** In Progress — Milestones 1-3 Complete, M4-M5 In Progress

## Vision

A minimal-dependency, ultra-fast documentation platform that publishes markdown notes as a responsive web application. Features wiki-style linking, collaborative comments, offline-capable sync, and fine-grained access control — all deployable via a single `docker-compose.yml` with zero vendor lock-in.

## Design Principles

1. **Paranoid Security**: Cryptographic content verification, battle-tested dependencies only, supply-chain attack mitigation through minimal external code
2. **Operational Simplicity**: Single binary, single SQLite database, filesystem storage — nothing that can't be understood in an afternoon
3. **Performance First**: Sub-100ms initial page loads, instant client-side navigation, no JavaScript required for reading
4. **Offline Capability**: Read cached, sync when connected
5. **No Vendor Lock-in**: Works on any Docker host; no cloud-specific APIs

---

## Milestones

### Milestone 1: Foundation ✅ COMPLETE

**Goal:** Working server that can serve markdown files as HTML

- [x] Go HTTP server with chi router
- [x] libsql database with schema migration system
- [x] Markdown-to-HTML pipeline with most common extensions enabled
- [x] Static file serving for attachments
- [x] Basic Dockerfile and docker-compose.yml
- [x] Dev mode with `air` live reload
- [x] Miller column document browser with folder navigation
- [x] `index.md` default rendering for folders
- [x] Content-addressed attachment storage with secure serving
- [x] WebSocket real-time updates (document change notifications)

**Acceptance Criteria:**

- [x] Server starts and listens on configurable port
- [x] Database auto-migrates on startup
- [x] Markdown files with wiki-links, tags, callouts, mermaid, math render correctly
- [x] Attachments served securely with content-type headers
- [x] Docker compose brings up full stack with one command
- [x] All dependencies pinned with checksum verification
- [x] Build reproducible (same input → same binary hash)

**Notes:**
- Mermaid diagrams and math blocks now render client-side after server-rendered markdown loads.
- Dev mode uses `air` for Go live reload.

---

### Milestone 2: Sync Protocol (Weeks 3-4) ✅ COMPLETE

**Goal:** Bidirectional file sync between server and filesystem/web client

- [x] Content-addressed storage (SHA-256 filesystem layout)
- [x] WebSocket sync protocol implementation
- [x] File watcher for local filesystem sync
- [x] Client-side IndexedDB cache
- [x] Offline queue with conflict detection

**Acceptance Criteria:**

- [x] File changes on disk automatically sync to server within 5 seconds
- [x] Web client can sync after reconnection, even with changes made offline
- [x] Concurrent edits to different files succeed without conflict
- [x] Concurrent edits to same file detected and queued for resolution
- [x] All synced content verified against SHA-256 hashes
- [x] Sync protocol documented with state machine diagrams

**What's Done:**
- Real-time file watcher using `fsnotify` — detects create/write/remove/rename events on `.md` files
- Changes trigger immediate `SyncSourceDir()` and broadcast `document_version` events via WebSocket
- Browser shows reload notification when current document or folder structure changes
- Fallback polling every 30 seconds ensures nothing is missed
- Go-served editor caches pages and pending edits in IndexedDB for offline reads and recovery
- Offline saves queue locally and sync after reconnection with a visible pending changes panel
- Server-side save conflicts return 409 responses with base/current hashes and server content
- Conflict resolution screen shows server copy vs queued edit, then resolves into a Monaco editor
- Static assets from the content directory can be embedded from markdown documents
- Markdown rendering now supports raw HTML when authored in trusted content
- File watcher debounces editor save bursts and ignores hidden/temp files
- SimpleSync spec documents the intended browser, watcher, realtime, and native-client feature set
- Go-level multi-client sync tests cover different-file success, same-file conflicts, and resolution behavior
- 100-file sync performance validation covers source sync and snapshot/delta paths
- IndexedDB uses a simple fixed cap for cached rendered pages/content while preserving pending edits

**Deferred:**
- WebSocket authentication and logout-driven cache clearing move to Milestone 3 application auth
- Browser-level automated sync tests and larger load benchmarks move to production hardening

---

### Milestone 3: Authentication & Authorization (Weeks 5-6) ✅ COMPLETE

**Goal:** Secure access control with passkeys and granular permissions

- [x] Passkey (WebAuthn) primary authentication
- [x] Password fallback with Argon2id
- [x] Session management with opaque server-side sessions
- [x] Role-based access control (RBAC)
- [x] Scoped API tokens with device-code flow for developer clients
- [ ] Content signing with Ed25519 (deferred optional hardening)
- [ ] Automatic session refresh (deferred polish)
- [ ] Multiple passkey management UI (deferred polish)

**Acceptance Criteria:**

- [x] Users can register and log in with passkeys on modern browsers
- [x] Password fallback works with proper hashing
- [x] Sessions expire securely and are revocable
- [x] Scoped API tokens and device-code flow support developer clients
- [x] Read/write/admin permissions work at route and role level
- [ ] Per-document and per-collection permission UI (deferred until collaboration resource modeling)
- [x] All auth endpoints rate-limited
- [x] Auth events logged
- [x] Security implementation review: no plaintext session/token secrets, constant-time password/token comparison

**What's Done:**
- WebAuthn begin/finish registration and login endpoints
- Argon2id password registration/login with first-user admin bootstrap
- Opaque session cookies stored as server-side hashes
- Scoped bearer API tokens stored as hashes and shown once
- First-party device-code flow for CLI/client onboarding
- Route middleware enforcing viewer/editor/admin roles plus token scopes
- In-memory auth/device-flow rate limiter for self-hosted deployments under 100 users
- Go-served login, account, API-token, role-management, and device-approval screens
- Password change and account deletion endpoints
- Public registration hardening so credentials cannot be attached to existing accounts without a session
- Device-code replay hardening so approved codes mint only one API token

**What's Next:**
- Complete the independent penetration-test checklist during production hardening
- Implement resource permissions for pages, folders, and collections:
  - Content is secure/private by default unless a user is granted access directly or through a folder/collection grant
  - Supported levels remain view, edit, and admin; commenting is available to editors and admins only
  - Global admins and resource admins can change resource permissions
  - Unauthorized pages disappear from navigation, search, and sync results
  - Most permissive matching grant wins, so shared folder/collection access can expose otherwise-default-private pages
  - API tokens and native sync clients inherit the owning user's effective resource permissions
  - Public UUID/hash-style sharing remains supported for explicitly shared/uploaded assets
- Add optional multiple-passkey management and session refresh polish if real usage calls for it

---

### Milestone 4: Collaboration Features (Weeks 7-8) 🔄 IN PROGRESS

**Goal:** Comments, notifications, and conflict resolution

- [x] Comment system linked to file versions (by hash) - backend complete
- [x] Queued email notifications via Postmark, Cloudflare Email Service, or local SMTP
- [x] Web-based conflict resolution UI (Milestone 2)
- [ ] Notification preferences (per-file, per-folder, global, digest)
- [x] Comment threading schema and browser reply UI
- [ ] Email replies create comments

**Acceptance Criteria:**

- [x] Users can comment on specific lines/sections of a document (API)
- [x] Comments persist and are linked to document versions
- [x] Email notifications queued for watched file changes with retry
- [ ] Replying to notification email creates a comment
- [x] Conflicts display a unified diff with resolution actions (M2)
- [ ] Digest mode batches notifications by time window

---

### Milestone 5: Search & UI Polish (Weeks 9-10) 🔄 IN PROGRESS

**Goal:** Fast search, design system, and production readiness

- [x] Full-text search with SQLite FTS5
- [ ] Client-side search index sync
- [ ] Design system with browser-based CSS editor
- [ ] Responsive mobile layout
- [ ] Error boundaries and loading states
- [ ] Performance benchmarking
- [x] Client-side Mermaid and KaTeX rendering

**Acceptance Criteria:**

- [x] Search returns results in <50ms on client (server-side)
- [ ] Search works offline after initial sync
- [ ] Design system documented at `/design` route
- [ ] All screens responsive down to 320px width
- [ ] Lighthouse score >90 on all metrics
- [ ] Initial page load <100ms (server-rendered HTML)
- [x] Mermaid diagrams render client-side
- [x] Math blocks render with KaTeX

---

### Milestone 6: Production Hardening (Weeks 11-12)

**Goal:** Deploy with confidence

- Comprehensive test suite (unit, integration, e2e)
- Security audit and penetration testing guide
- Backup and restore procedures
- Monitoring and health checks
- Documentation for operators

**Acceptance Criteria:**

- [ ] 80% test coverage
- [ ] No critical or high-severity security findings
- [ ] Backup script exports database + attachments as tar.gz
- [ ] Health check endpoint returns status of all dependencies
- [ ] Operator docs cover: install, upgrade, backup, security incident response
- [ ] Load test: 1000 concurrent readers, 100 concurrent editors

---

## Architecture Overview

```
┌─────────────────────────────────────────────────────────────┐
│                        Client Browser                        │
│  ┌──────────────┐  ┌──────────────┐  ┌─────────────────┐   │
│  │ Go HTML UI   │  │  File Sync   │  │  Search Index   │   │
│  │ + scripts    │  │  (IndexedDB) │  │    (FTS5)       │   │
│  └──────────────┘  └──────────────┘  └─────────────────┘   │
└─────────────────────────────────────────────────────────────┘
                             │
                      WebSocket / HTTP
                             │
┌─────────────────────────────────────────────────────────────┐
│                      Go Application Server                   │
│  ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌──────────┐ │
│  │   HTTP     │ │ WebSocket  │ │  Markdown  │ │   Auth   │ │
│  │   Router   │ │   Sync     │ │  Renderer  │ │ (WebAuthn)│ │
│  └────────────┘ └────────────┘ └────────────┘ └──────────┘ │
│  ┌────────────┐ ┌────────────┐ ┌────────────┐              │
│  │   Email    │ │  Search    │ │ File Watch │              │
│  │  (Postmark)│ │   (FTS5)   │ │   (fsnotify)│             │
│  └────────────┘ └────────────┘ └────────────┘              │
└─────────────────────────────────────────────────────────────┘
                             │
          ┌──────────────────┴──────────────────┐
          │                                     │
   ┌──────────────┐                    ┌──────────────┐
   │   libsql     │                    │  Content-    │
   │  (SQLite)    │                    │  Addressed   │
   │              │                    │  Filesystem  │
   └──────────────┘                    └──────────────┘
```

See [architecture-overview.md](architecture-overview.md) for detailed component design.

---

## Technology Stack

### Backend (Go 1.26+)

| Component     | Library                                     | Justification                            |
| ------------- | ------------------------------------------- | ---------------------------------------- |
| HTTP Router   | `github.com/go-chi/chi`                     | Minimal, idiomatic, stdlib-compatible    |
| Database      | `github.com/tursodatabase/go-libsql`        | Embedded SQLite with replication option  |
| Migrations    | Internal migration runner                   | Simple, no extra dependency              |
| Auth/Passkeys | `github.com/go-webauthn/webauthn`           | Reference implementation, well-audited   |
| Password Hash | `golang.org/x/crypto/argon2`                | Winner of Password Hashing Competition   |
| Markdown      | `github.com/yuin/goldmark` + extensions     | Fast, extensible, CommonMark compliant   |
| WebSocket     | `github.com/gorilla/websocket`              | Most widely used Go WS library           |
| File Watch    | `github.com/fsnotify/fsnotify`              | Cross-platform, stdlib-adjacent          |
| Email         | `net/http` + `net/smtp`                     | Postmark/Cloudflare REST adapters + local SMTP |
| Crypto        | `crypto/ed25519`, `crypto/sha256`           | Standard library, no external deps       |
| Templates     | `html/template`                             | Stdlib, auto-escaping, SSR-safe          |

### Browser UI

| Component   | Technology                     | Justification                          |
| ----------- | ------------------------------ | -------------------------------------- |
| Rendering   | Go `html/template`             | Server-rendered, auto-escaped HTML     |
| Interaction | Embedded browser scripts       | Small progressive-enhancement modules |
| Styling     | Native CSS + custom properties | No build-time CSS processing needed   |
| Icons       | Inline SVG                     | No icon font dependency                |

### Infrastructure

| Component     | Technology         | Justification                                 |
| ------------- | ------------------ | --------------------------------------------- |
| Container     | Docker + Compose   | Universal deployment target                   |
| Reverse Proxy | Traefik (optional) | Auto-HTTPS, included in compose               |
| Database      | libsql/SQLite      | Single file, zero external dependency         |
| Storage       | Local filesystem   | Content-addressed, immutable, backup-friendly |

---

## Security Model

### Threats Addressed

1. **Supply Chain Attacks**: Minimal external dependencies; all pinned with go.sum; vendored if needed
2. **Data Tampering**: All document versions SHA-256 hashed; optional Ed25519 signatures
3. **Unauthorized Access**: RBAC with principle of least privilege; content never served without auth check
4. **Session Hijacking**: Signed cookies with HttpOnly, Secure, SameSite=Strict; short expiry with refresh
5. **CSRF**: Double-submit cookie pattern for state-changing operations
6. **XSS**: Go html/template auto-escaping; strict CSP headers; no inline scripts except nonce-tagged
7. **DoS**: Rate limiting on auth endpoints; file upload size limits; query timeouts

### Content Security

- Attachment uploads: Magic number validation, extension whitelist, size limits
- Markdown rendering: Trusted content may include raw HTML; templates still auto-escape application data
- File paths: Canonicalization prevents directory traversal

### Audit & Compliance

- Every data modification logged with user, timestamp, IP, and content hash
- Immutable history table — updates are inserts
- GDPR-compliant: full data export and deletion capability

---

## Performance Targets

| Metric              | Target                    | Measurement                      |
| ------------------- | ------------------------- | -------------------------------- |
| Initial Page Load   | <100ms                    | TTFB + First Contentful Paint    |
| Time to Interactive | <200ms                    | Hydration complete               |
| Search Query        | <50ms                     | Client-side after initial sync   |
| File Sync           | <5s                       | From disk save to server persist |
| Bundle Size         | <100KB                    | Gzipped JS + CSS                 |
| Concurrent Users    | 1000 readers, 100 editors | Load tested                      |

---

## Monorepo Structure

```
cairnquire/
├── README.md
├── docker-compose.yml
├── Dockerfile
├── mise.toml
├── docs/                          # Project documentation
│   ├── project-plan.md            # This file
│   ├── architecture-overview.md   # System architecture
│   ├── adrs/                      # Architecture Decision Records
│   ├── milestones/                # Detailed milestone plans
│   └── specs/                     # Technical specifications
├── apps/
│   ├── server/                    # Go HTTP application
│   │   ├── cmd/cairnquire/
│   │   ├── internal/
│   │   │   ├── app/               # Application orchestration
│   │   │   ├── config/            # Configuration management
│   │   │   ├── database/          # Database & migrations
│   │   │   ├── docs/              # Document service & repository
│   │   │   ├── httpserver/        # HTTP handlers & templates
│   │   │   ├── markdown/          # Markdown rendering pipeline
│   │   │   ├── realtime/          # WebSocket hub
│   │   │   └── store/             # Content-addressed filesystem
│   │   ├── static/                # Embedded static assets
│   │   └── templates/             # Go HTML templates
├── content/                       # Markdown source files
├── data/                          # SQLite database & attachments
└── packages/
    └── protocol/                  # Shared sync protocol spec
```

---

## Risk Register

| Risk                             | Likelihood | Impact | Mitigation                                                                          |
| -------------------------------- | ---------- | ------ | ----------------------------------------------------------------------------------- |
| WebAuthn/passkey adoption issues | Medium     | Medium | Password fallback always available; extensive browser testing                       |
| SQLite performance at scale      | Low        | High   | libsql replication path; FTS5 for search; benchmark early                           |
| Markdown extension parser bugs   | Medium     | Medium | Fuzz testing; strict CommonMark base; gradual feature rollout                       |
| Email deliverability             | Medium     | Medium | Provider reputation monitoring; SPF/DKIM/DMARC setup; provider fallback              |
| Frontend bundle bloat            | Medium     | Medium | Bundle analysis in CI; tree-shaking enforced; lazy loading                          |
| File sync conflicts in teams     | High       | Medium | Clear UX for conflict resolution; email notification; automatic merge when possible |

---

## Next Steps

1. ✅ Complete Milestone 1: Foundation
2. ✅ Complete Milestone 2: Sync Protocol
3. ✅ Complete Milestone 3: Authentication
4. 🔄 Finish Milestone 4: Collaboration
   - Complete inbound email replies, mention notifications, and conflict notifications
   - Add global/per-resource notification preferences and digest scheduling
   - Test watcher-driven notifications against live Postmark and Cloudflare accounts
5. 🔄 Finish Milestone 5: Search & UI Polish
   - Tag filtering in search
   - Design system /design route
   - Dark mode toggle
   - Mobile-responsive layout polish
   - Accessibility improvements
6. ⏳ Milestone 6: Production Hardening
   - Increase test coverage from 52.4% to 80%+
   - Add integration tests for auth, sync, collaboration
   - Security audit checklist
   - Backup/restore scripts
   - Complete dependency health checks and add monitoring endpoints

---

## Glossary

- **ADR**: Architecture Decision Record
- **CRDT**: Conflict-free Replicated Data Type (not used, but considered)
- **CSRF**: Cross-Site Request Forgery
- **CSP**: Content Security Policy
- **FTS5**: SQLite Full-Text Search version 5
- **RBAC**: Role-Based Access Control
- **SSR**: Server-Side Rendering
- **TTFB**: Time to First Byte
- **WebAuthn**: Web Authentication API (passkeys)
