cairnquire/.project/project-plan.md

Cairnquire - Project Plan

Version: 1.3
Date: 2026-06-01
Status: In Progress — Milestones 1-3 Complete, M4-M5 In Progress

Vision

A minimal-dependency, ultra-fast documentation platform that publishes markdown notes as a responsive web application. Features wiki-style linking, collaborative comments, offline-capable sync, and fine-grained access control — all deployable via a single docker-compose.yml with zero vendor lock-in.

Design Principles

1. Paranoid Security: Cryptographic content verification, battle-tested dependencies only, supply-chain attack mitigation through minimal external code
2. Operational Simplicity: Single binary, single SQLite database, filesystem storage — nothing that can't be understood in an afternoon
3. Performance First: Sub-100ms initial page loads, instant client-side navigation, no JavaScript required for reading
4. Offline Capability: Read cached, sync when connected
5. No Vendor Lock-in: Works on any Docker host; no cloud-specific APIs

---

Milestones

Milestone 1: Foundation ✅ COMPLETE

Goal: Working server that can serve markdown files as HTML

• Go HTTP server with chi router
• libsql database with schema migration system
• Markdown-to-HTML pipeline with most common extensions enabled
• Static file serving for attachments
• Basic Dockerfile and docker-compose.yml
• Dev mode with air live reload
• Miller column document browser with folder navigation
• index.md default rendering for folders
• Content-addressed attachment storage with secure serving
• WebSocket real-time updates (document change notifications)

Acceptance Criteria:

• Server starts and listens on configurable port
• Database auto-migrates on startup
• Markdown files with wiki-links, tags, callouts, mermaid, math render correctly
• Attachments served securely with content-type headers
• Docker compose brings up full stack with one command
• All dependencies pinned with checksum verification
• Build reproducible (same input → same binary hash)

Notes:

• Mermaid diagrams and math blocks now render client-side after server-rendered markdown loads.
• Dev mode uses air for Go live reload.

---

Milestone 2: Sync Protocol (Weeks 3-4) ✅ COMPLETE

Goal: Bidirectional file sync between server and filesystem/web client

• Content-addressed storage (SHA-256 filesystem layout)
• WebSocket sync protocol implementation
• File watcher for local filesystem sync
• Client-side IndexedDB cache
• Offline queue with conflict detection

Acceptance Criteria:

• File changes on disk automatically sync to server within 5 seconds
• Web client can sync after reconnection, even with changes made offline
• Concurrent edits to different files succeed without conflict
• Concurrent edits to same file detected and queued for resolution
• All synced content verified against SHA-256 hashes
• Sync protocol documented with state machine diagrams

What's Done:

• Real-time file watcher using fsnotify — detects create/write/remove/rename events on .md files
• Changes trigger immediate SyncSourceDir() and broadcast document_version events via WebSocket
• Browser shows reload notification when current document or folder structure changes
• Fallback polling every 30 seconds ensures nothing is missed
• Go-served editor caches pages and pending edits in IndexedDB for offline reads and recovery
• Offline saves queue locally and sync after reconnection with a visible pending changes panel
• Server-side save conflicts return 409 responses with base/current hashes and server content
• Conflict resolution screen shows server copy vs queued edit, then resolves into a Monaco editor
• Static assets from the content directory can be embedded from markdown documents
• Markdown rendering now supports raw HTML when authored in trusted content
• File watcher debounces editor save bursts and ignores hidden/temp files
• SimpleSync spec documents the intended browser, watcher, realtime, and native-client feature set
• Go-level multi-client sync tests cover different-file success, same-file conflicts, and resolution behavior
• 100-file sync performance validation covers source sync and snapshot/delta paths
• IndexedDB uses a simple fixed cap for cached rendered pages/content while preserving pending edits

Deferred:

• WebSocket authentication and logout-driven cache clearing move to Milestone 3 application auth
• Browser-level automated sync tests and larger load benchmarks move to production hardening

---

Milestone 3: Authentication & Authorization (Weeks 5-6) ✅ COMPLETE

Goal: Secure access control with passkeys and granular permissions

• Passkey (WebAuthn) primary authentication
• Password fallback with Argon2id
• Session management with opaque server-side sessions
• Role-based access control (RBAC)
• Scoped API tokens with device-code flow for developer clients
• Content signing with Ed25519 (deferred optional hardening)
• Automatic session refresh (deferred polish)
• Multiple passkey management UI (deferred polish)

Acceptance Criteria:

• Users can register and log in with passkeys on modern browsers
• Password fallback works with proper hashing
• Sessions expire securely and are revocable
• Scoped API tokens and device-code flow support developer clients
• Read/write/admin permissions work at route and role level
• Per-document and per-collection permission UI (deferred until collaboration resource modeling)
• All auth endpoints rate-limited
• Auth events logged
• Security implementation review: no plaintext session/token secrets, constant-time password/token comparison

What's Done:

• WebAuthn begin/finish registration and login endpoints
• Argon2id password registration/login with first-user admin bootstrap
• Opaque session cookies stored as server-side hashes
• Scoped bearer API tokens stored as hashes and shown once
• First-party device-code flow for CLI/client onboarding
• Route middleware enforcing viewer/editor/admin roles plus token scopes
• In-memory auth/device-flow rate limiter for self-hosted deployments under 100 users
• Go-served login, account, API-token, role-management, and device-approval screens
• Password change and account deletion endpoints
• Public registration hardening so credentials cannot be attached to existing accounts without a session
• Device-code replay hardening so approved codes mint only one API token

What's Next:

• Complete the independent penetration-test checklist during production hardening
• Implement resource permissions for pages, folders, and collections:
  • Content is secure/private by default unless a user is granted access directly or through a folder/collection grant
  • Supported levels remain view, edit, and admin; commenting is available to editors and admins only
  • Global admins and resource admins can change resource permissions
  • Unauthorized pages disappear from navigation, search, and sync results
  • Most permissive matching grant wins, so shared folder/collection access can expose otherwise-default-private pages
  • API tokens and native sync clients inherit the owning user's effective resource permissions
  • Public UUID/hash-style sharing remains supported for explicitly shared/uploaded assets
• Add optional multiple-passkey management and session refresh polish if real usage calls for it

---

Milestone 4: Collaboration Features (Weeks 7-8) 🔄 IN PROGRESS

Goal: Comments, notifications, and conflict resolution

• Comment system linked to file versions (by hash) - backend complete
• Email notifications via Postmark adapter - SMTP stub exists
• Web-based conflict resolution UI (Milestone 2)
• Notification preferences (per-file, per-folder, global, digest)
• Comment threading schema - UI needs wiring
• Email replies create comments

Acceptance Criteria:

• Users can comment on specific lines/sections of a document (API)
• Comments persist and are linked to document versions
• Email notifications sent for watched file changes (NoOp sender)
• Replying to notification email creates a comment
• Conflicts display side-by-side diff with resolution actions (M2)
• Digest mode batches notifications by time window

---

Milestone 5: Search & UI Polish (Weeks 9-10) 🔄 IN PROGRESS

Goal: Fast search, design system, and production readiness

• Full-text search with SQLite FTS5
• Client-side search index sync
• Design system with browser-based CSS editor
• Responsive mobile layout
• Error boundaries and loading states
• Performance benchmarking
• Client-side Mermaid and KaTeX rendering

Acceptance Criteria:

• Search returns results in <50ms on client (server-side)
• Search works offline after initial sync
• Design system documented at /design route
• All screens responsive down to 320px width
• Lighthouse score >90 on all metrics
• Initial page load <100ms (server-rendered HTML)
• Mermaid diagrams render client-side
• Math blocks render with KaTeX

---

Milestone 6: Production Hardening (Weeks 11-12)

Goal: Deploy with confidence

• Comprehensive test suite (unit, integration, e2e)
• Security audit and penetration testing guide
• Backup and restore procedures
• Monitoring and health checks
• Documentation for operators

Acceptance Criteria:

• 80% test coverage
• No critical or high-severity security findings
• Backup script exports database + attachments as tar.gz
• Health check endpoint returns status of all dependencies
• Operator docs cover: install, upgrade, backup, security incident response
• Load test: 1000 concurrent readers, 100 concurrent editors

---

Architecture Overview

┌─────────────────────────────────────────────────────────────┐
│                        Client Browser                        │
│  ┌──────────────┐  ┌──────────────┐  ┌─────────────────┐   │
│  │ Go HTML UI   │  │  File Sync   │  │  Search Index   │   │
│  │ + scripts    │  │  (IndexedDB) │  │    (FTS5)       │   │
│  └──────────────┘  └──────────────┘  └─────────────────┘   │
└─────────────────────────────────────────────────────────────┘
                             │
                      WebSocket / HTTP
                             │
┌─────────────────────────────────────────────────────────────┐
│                      Go Application Server                   │
│  ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌──────────┐ │
│  │   HTTP     │ │ WebSocket  │ │  Markdown  │ │   Auth   │ │
│  │   Router   │ │   Sync     │ │  Renderer  │ │ (WebAuthn)│ │
│  └────────────┘ └────────────┘ └────────────┘ └──────────┘ │
│  ┌────────────┐ ┌────────────┐ ┌────────────┐              │
│  │   Email    │ │  Search    │ │ File Watch │              │
│  │  (Postmark)│ │   (FTS5)   │ │   (fsnotify)│             │
│  └────────────┘ └────────────┘ └────────────┘              │
└─────────────────────────────────────────────────────────────┘
                             │
          ┌──────────────────┴──────────────────┐
          │                                     │
   ┌──────────────┐                    ┌──────────────┐
   │   libsql     │                    │  Content-    │
   │  (SQLite)    │                    │  Addressed   │
   │              │                    │  Filesystem  │
   └──────────────┘                    └──────────────┘

See architecture-overview.md for detailed component design.

---

Technology Stack

Backend (Go 1.24+)

Component	Library	Justification
HTTP Router	github.com/go-chi/chi	Minimal, idiomatic, stdlib-compatible
Database	github.com/tursodatabase/go-libsql	Embedded SQLite with replication option
Migrations	Internal migration runner	Simple, no extra dependency
Auth/Passkeys	github.com/go-webauthn/webauthn	Reference implementation, well-audited
Password Hash	golang.org/x/crypto/argon2	Winner of Password Hashing Competition
Markdown	github.com/yuin/goldmark + extensions	Fast, extensible, CommonMark compliant
WebSocket	github.com/gorilla/websocket	Most widely used Go WS library
File Watch	github.com/fsnotify/fsnotify	Cross-platform, stdlib-adjacent
Email	github.com/keighl/postmark + net/smtp	Postmark adapter + SMTP fallback
Crypto	crypto/ed25519, crypto/sha256	Standard library, no external deps
Templates	html/template	Stdlib, auto-escaping, SSR-safe

Browser UI

Component	Technology	Justification
Rendering	Go html/template	Server-rendered, auto-escaped HTML
Interaction	Embedded browser scripts	Small progressive-enhancement modules
Styling	Native CSS + custom properties	No build-time CSS processing needed
Icons	Inline SVG	No icon font dependency

Infrastructure

Component	Technology	Justification
Container	Docker + Compose	Universal deployment target
Reverse Proxy	Traefik (optional)	Auto-HTTPS, included in compose
Database	libsql/SQLite	Single file, zero external dependency
Storage	Local filesystem	Content-addressed, immutable, backup-friendly

---

Security Model

Threats Addressed

1. Supply Chain Attacks: Minimal external dependencies; all pinned with go.sum; vendored if needed
2. Data Tampering: All document versions SHA-256 hashed; optional Ed25519 signatures
3. Unauthorized Access: RBAC with principle of least privilege; content never served without auth check
4. Session Hijacking: Signed cookies with HttpOnly, Secure, SameSite=Strict; short expiry with refresh
5. CSRF: Double-submit cookie pattern for state-changing operations
6. XSS: Go html/template auto-escaping; strict CSP headers; no inline scripts except nonce-tagged
7. DoS: Rate limiting on auth endpoints; file upload size limits; query timeouts

Content Security

• Attachment uploads: Magic number validation, extension whitelist, size limits
• Markdown rendering: Trusted content may include raw HTML; templates still auto-escape application data
• File paths: Canonicalization prevents directory traversal

Audit & Compliance

• Every data modification logged with user, timestamp, IP, and content hash
• Immutable history table — updates are inserts
• GDPR-compliant: full data export and deletion capability

---

Performance Targets

Metric	Target	Measurement
Initial Page Load	<100ms	TTFB + First Contentful Paint
Time to Interactive	<200ms	Hydration complete
Search Query	<50ms	Client-side after initial sync
File Sync	<5s	From disk save to server persist
Bundle Size	<100KB	Gzipped JS + CSS
Concurrent Users	1000 readers, 100 editors	Load tested

---

Monorepo Structure

cairnquire/
├── README.md
├── docker-compose.yml
├── Dockerfile
├── mise.toml
├── docs/                          # Project documentation
│   ├── project-plan.md            # This file
│   ├── architecture-overview.md   # System architecture
│   ├── adrs/                      # Architecture Decision Records
│   ├── milestones/                # Detailed milestone plans
│   └── specs/                     # Technical specifications
├── apps/
│   ├── server/                    # Go HTTP application
│   │   ├── cmd/cairnquire/
│   │   ├── internal/
│   │   │   ├── app/               # Application orchestration
│   │   │   ├── config/            # Configuration management
│   │   │   ├── database/          # Database & migrations
│   │   │   ├── docs/              # Document service & repository
│   │   │   ├── httpserver/        # HTTP handlers & templates
│   │   │   ├── markdown/          # Markdown rendering pipeline
│   │   │   ├── realtime/          # WebSocket hub
│   │   │   └── store/             # Content-addressed filesystem
│   │   ├── static/                # Embedded static assets
│   │   └── templates/             # Go HTML templates
├── content/                       # Markdown source files
├── data/                          # SQLite database & attachments
└── packages/
    └── protocol/                  # Shared sync protocol spec

---

Risk Register

Risk	Likelihood	Impact	Mitigation
WebAuthn/passkey adoption issues	Medium	Medium	Password fallback always available; extensive browser testing
SQLite performance at scale	Low	High	libsql replication path; FTS5 for search; benchmark early
Markdown extension parser bugs	Medium	Medium	Fuzz testing; strict CommonMark base; gradual feature rollout
Email deliverability	Medium	Medium	Postmark reputation monitoring; SPF/DKIM setup; SMTP fallback
Frontend bundle bloat	Medium	Medium	Bundle analysis in CI; tree-shaking enforced; lazy loading
File sync conflicts in teams	High	Medium	Clear UX for conflict resolution; email notification; automatic merge when possible

---

Next Steps

1. ✅ Complete Milestone 1: Foundation
2. ✅ Complete Milestone 2: Sync Protocol
3. ✅ Complete Milestone 3: Authentication
4. 🔄 Finish Milestone 4: Collaboration
   • Wire up comment UI in browser (JS exists, needs integration)
   • Wire up notification bell UI
   • Implement actual email sending (Postmark or SMTP)
   • Add notification preferences page
   • Test watcher-driven notifications end-to-end
5. 🔄 Finish Milestone 5: Search & UI Polish
   • Tag filtering in search
   • Design system /design route
   • Dark mode toggle
   • Mobile-responsive layout polish
   • Accessibility improvements
6. ⏳ Milestone 6: Production Hardening
   • Increase test coverage from 39% to 80%+
   • Add integration tests for auth, sync, collaboration
   • Security audit checklist
   • Backup/restore scripts
   • Health check and monitoring endpoints

---

Glossary

• ADR: Architecture Decision Record
• CRDT: Conflict-free Replicated Data Type (not used, but considered)
• CSRF: Cross-Site Request Forgery
• CSP: Content Security Policy
• FTS5: SQLite Full-Text Search version 5
• RBAC: Role-Based Access Control
• SSR: Server-Side Rendering
• TTFB: Time to First Byte
• WebAuthn: Web Authentication API (passkeys)