393 lines
20 KiB
Markdown
393 lines
20 KiB
Markdown
# Cairnquire - Project Plan
|
|
|
|
**Version:** 1.2\
|
|
**Date:** 2026-05-14\
|
|
**Status:** In Progress — Milestones 1-3 Complete
|
|
|
|
## Vision
|
|
|
|
A minimal-dependency, ultra-fast documentation platform that publishes markdown notes as a responsive web application. Features wiki-style linking, collaborative comments, offline-capable sync, and fine-grained access control — all deployable via a single `docker-compose.yml` with zero vendor lock-in.
|
|
|
|
## Design Principles
|
|
|
|
1. **Paranoid Security**: Cryptographic content verification, battle-tested dependencies only, supply-chain attack mitigation through minimal external code
|
|
2. **Operational Simplicity**: Single binary, single SQLite database, filesystem storage — nothing that can't be understood in an afternoon
|
|
3. **Performance First**: Sub-100ms initial page loads, instant client-side navigation, no JavaScript required for reading
|
|
4. **Offline Capability**: Read cached, sync when connected
|
|
5. **No Vendor Lock-in**: Works on any Docker host; no cloud-specific APIs
|
|
|
|
---
|
|
|
|
## Milestones
|
|
|
|
### Milestone 1: Foundation ✅ COMPLETE
|
|
|
|
**Goal:** Working server that can serve markdown files as HTML
|
|
|
|
- [x] Go HTTP server with chi router
|
|
- [x] libsql database with schema migration system
|
|
- [x] Markdown-to-HTML pipeline with most common extensions enabled
|
|
- [x] Static file serving for attachments
|
|
- [x] Basic Dockerfile and docker-compose.yml
|
|
- [x] Dev mode with `air` live reload
|
|
- [x] Miller column document browser with folder navigation
|
|
- [x] `index.md` default rendering for folders
|
|
- [x] Content-addressed attachment storage with secure serving
|
|
- [x] WebSocket real-time updates (document change notifications)
|
|
|
|
**Acceptance Criteria:**
|
|
|
|
- [x] Server starts and listens on configurable port
|
|
- [x] Database auto-migrates on startup
|
|
- [x] Markdown files with wiki-links, tags, callouts, mermaid, math render correctly
|
|
- [x] Attachments served securely with content-type headers
|
|
- [x] Docker compose brings up full stack with one command
|
|
- [x] All dependencies pinned with checksum verification
|
|
- [x] Build reproducible (same input → same binary hash)
|
|
|
|
**Notes:**
|
|
- Mermaid diagrams and math blocks now render client-side after server-rendered markdown loads.
|
|
- Dev mode uses `air` for Go live reload.
|
|
|
|
---
|
|
|
|
### Milestone 2: Sync Protocol (Weeks 3-4) ✅ COMPLETE
|
|
|
|
**Goal:** Bidirectional file sync between server and filesystem/web client
|
|
|
|
- [x] Content-addressed storage (SHA-256 filesystem layout)
|
|
- [x] WebSocket sync protocol implementation
|
|
- [x] File watcher for local filesystem sync
|
|
- [x] Client-side IndexedDB cache
|
|
- [x] Offline queue with conflict detection
|
|
|
|
**Acceptance Criteria:**
|
|
|
|
- [x] File changes on disk automatically sync to server within 5 seconds
|
|
- [x] Web client can sync after reconnection, even with changes made offline
|
|
- [x] Concurrent edits to different files succeed without conflict
|
|
- [x] Concurrent edits to same file detected and queued for resolution
|
|
- [x] All synced content verified against SHA-256 hashes
|
|
- [x] Sync protocol documented with state machine diagrams
|
|
|
|
**What's Done:**
|
|
- Real-time file watcher using `fsnotify` — detects create/write/remove/rename events on `.md` files
|
|
- Changes trigger immediate `SyncSourceDir()` and broadcast `document_version` events via WebSocket
|
|
- Browser shows reload notification when current document or folder structure changes
|
|
- Fallback polling every 30 seconds ensures nothing is missed
|
|
- Go-served editor caches pages and pending edits in IndexedDB for offline reads and recovery
|
|
- Offline saves queue locally and sync after reconnection with a visible pending changes panel
|
|
- Server-side save conflicts return 409 responses with base/current hashes and server content
|
|
- Conflict resolution screen shows server copy vs queued edit, then resolves into a Monaco editor
|
|
- Static assets from the content directory can be embedded from markdown documents
|
|
- Markdown rendering now supports raw HTML when authored in trusted content
|
|
- File watcher debounces editor save bursts and ignores hidden/temp files
|
|
- SimpleSync spec documents the intended browser, watcher, realtime, and native-client feature set
|
|
- Go-level multi-client sync tests cover different-file success, same-file conflicts, and resolution behavior
|
|
- 100-file sync performance validation covers source sync and snapshot/delta paths
|
|
- IndexedDB uses a simple fixed cap for cached rendered pages/content while preserving pending edits
|
|
|
|
**Deferred:**
|
|
- WebSocket authentication and logout-driven cache clearing move to Milestone 3 application auth
|
|
- Browser-level automated sync tests and larger load benchmarks move to production hardening
|
|
|
|
---
|
|
|
|
### Milestone 3: Authentication & Authorization (Weeks 5-6)
|
|
|
|
**Goal:** Secure access control with passkeys and granular permissions
|
|
|
|
- [x] Passkey (WebAuthn) primary authentication
|
|
- [x] Password fallback with Argon2id
|
|
- [x] Session management with opaque server-side sessions
|
|
- [x] Role-based access control (RBAC)
|
|
- [x] Scoped API tokens with device-code flow for developer clients
|
|
- [ ] Content signing with Ed25519 (deferred optional hardening)
|
|
|
|
**Acceptance Criteria:**
|
|
|
|
- [x] Users can register and log in with passkeys on modern browsers
|
|
- [x] Password fallback works with proper hashing
|
|
- [x] Sessions expire securely and are revocable
|
|
- [x] Scoped API tokens and device-code flow support developer clients
|
|
- [x] Read/write/admin permissions work at route and role level
|
|
- [ ] Per-document and per-collection permission UI (deferred until collaboration resource modeling)
|
|
- [x] All auth endpoints rate-limited
|
|
- [x] Auth events logged
|
|
- [x] Security implementation review: no plaintext session/token secrets, constant-time password/token comparison
|
|
|
|
**What's Done:**
|
|
- WebAuthn begin/finish registration and login endpoints
|
|
- Argon2id password registration/login with first-user admin bootstrap
|
|
- Opaque session cookies stored as server-side hashes
|
|
- Scoped bearer API tokens stored as hashes and shown once
|
|
- First-party device-code flow for CLI/client onboarding
|
|
- Route middleware enforcing viewer/editor/admin roles plus token scopes
|
|
- In-memory auth/device-flow rate limiter for self-hosted deployments under 100 users
|
|
- Go-served login, account, API-token, role-management, and device-approval screens
|
|
- Password change and account deletion endpoints
|
|
- Public registration hardening so credentials cannot be attached to existing accounts without a session
|
|
- Device-code replay hardening so approved codes mint only one API token
|
|
|
|
**What's Next:**
|
|
- Complete the independent penetration-test checklist during production hardening
|
|
- Decide how deep per-document/per-collection permissions should go before collaboration features
|
|
- Add optional multiple-passkey management and session refresh polish if real usage calls for it
|
|
|
|
---
|
|
|
|
### Milestone 4: Collaboration Features (Weeks 7-8)
|
|
|
|
**Goal:** Comments, notifications, and conflict resolution
|
|
|
|
- Comment system linked to file versions (by hash)
|
|
- Email notifications via Postmark adapter
|
|
- Web-based conflict resolution UI
|
|
- Notification preferences (per-file, per-folder, global, digest)
|
|
- Comment threading and email replies
|
|
|
|
**Acceptance Criteria:**
|
|
|
|
- [ ] Users can comment on specific lines/sections of a document
|
|
- [ ] Comments persist and are linked to document versions
|
|
- [ ] Email notifications sent for watched file changes
|
|
- [ ] Replying to notification email creates a comment
|
|
- [ ] Conflicts display side-by-side diff with resolution actions
|
|
- [ ] Digest mode batches notifications by time window
|
|
|
|
---
|
|
|
|
### Milestone 5: Search & UI Polish (Weeks 9-10)
|
|
|
|
**Goal:** Fast search, design system, and production readiness
|
|
|
|
- Full-text search with SQLite FTS5
|
|
- Client-side search index sync
|
|
- Design system with browser-based CSS editor
|
|
- Responsive mobile layout
|
|
- Error boundaries and loading states
|
|
- Performance benchmarking
|
|
- Client-side Mermaid and KaTeX rendering
|
|
|
|
**Acceptance Criteria:**
|
|
|
|
- [ ] Search returns results in <50ms on client
|
|
- [ ] Search works offline after initial sync
|
|
- [ ] Design system documented at `/design` route
|
|
- [ ] All screens responsive down to 320px width
|
|
- [ ] Lighthouse score >90 on all metrics
|
|
- [ ] Initial page load <100ms (server-rendered HTML)
|
|
- [x] Mermaid diagrams render client-side
|
|
- [x] Math blocks render with KaTeX
|
|
|
|
---
|
|
|
|
### Milestone 6: Production Hardening (Weeks 11-12)
|
|
|
|
**Goal:** Deploy with confidence
|
|
|
|
- Comprehensive test suite (unit, integration, e2e)
|
|
- Security audit and penetration testing guide
|
|
- Backup and restore procedures
|
|
- Monitoring and health checks
|
|
- Documentation for operators
|
|
|
|
**Acceptance Criteria:**
|
|
|
|
- [ ] 80% test coverage
|
|
- [ ] No critical or high-severity security findings
|
|
- [ ] Backup script exports database + attachments as tar.gz
|
|
- [ ] Health check endpoint returns status of all dependencies
|
|
- [ ] Operator docs cover: install, upgrade, backup, security incident response
|
|
- [ ] Load test: 1000 concurrent readers, 100 concurrent editors
|
|
|
|
---
|
|
|
|
## Architecture Overview
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ Client Browser │
|
|
│ ┌──────────────┐ ┌──────────────┐ ┌─────────────────┐ │
|
|
│ │ Go HTML UI │ │ File Sync │ │ Search Index │ │
|
|
│ │ + scripts │ │ (IndexedDB) │ │ (FTS5) │ │
|
|
│ └──────────────┘ └──────────────┘ └─────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
│
|
|
WebSocket / HTTP
|
|
│
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ Go Application Server │
|
|
│ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌──────────┐ │
|
|
│ │ HTTP │ │ WebSocket │ │ Markdown │ │ Auth │ │
|
|
│ │ Router │ │ Sync │ │ Renderer │ │ (WebAuthn)│ │
|
|
│ └────────────┘ └────────────┘ └────────────┘ └──────────┘ │
|
|
│ ┌────────────┐ ┌────────────┐ ┌────────────┐ │
|
|
│ │ Email │ │ Search │ │ File Watch │ │
|
|
│ │ (Postmark)│ │ (FTS5) │ │ (fsnotify)│ │
|
|
│ └────────────┘ └────────────┘ └────────────┘ │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
│
|
|
┌──────────────────┴──────────────────┐
|
|
│ │
|
|
┌──────────────┐ ┌──────────────┐
|
|
│ libsql │ │ Content- │
|
|
│ (SQLite) │ │ Addressed │
|
|
│ │ │ Filesystem │
|
|
└──────────────┘ └──────────────┘
|
|
```
|
|
|
|
See [architecture-overview.md](architecture-overview.md) for detailed component design.
|
|
|
|
---
|
|
|
|
## Technology Stack
|
|
|
|
### Backend (Go 1.24+)
|
|
|
|
| Component | Library | Justification |
|
|
| ------------- | ------------------------------------------- | ---------------------------------------- |
|
|
| HTTP Router | `github.com/go-chi/chi` | Minimal, idiomatic, stdlib-compatible |
|
|
| Database | `github.com/tursodatabase/go-libsql` | Embedded SQLite with replication option |
|
|
| Migrations | Internal migration runner | Simple, no extra dependency |
|
|
| Auth/Passkeys | `github.com/go-webauthn/webauthn` | Reference implementation, well-audited |
|
|
| Password Hash | `golang.org/x/crypto/argon2` | Winner of Password Hashing Competition |
|
|
| Markdown | `github.com/yuin/goldmark` + extensions | Fast, extensible, CommonMark compliant |
|
|
| WebSocket | `github.com/gorilla/websocket` | Most widely used Go WS library |
|
|
| File Watch | `github.com/fsnotify/fsnotify` | Cross-platform, stdlib-adjacent |
|
|
| Email | `github.com/keighl/postmark` + `net/smtp` | Postmark adapter + SMTP fallback |
|
|
| Crypto | `crypto/ed25519`, `crypto/sha256` | Standard library, no external deps |
|
|
| Templates | `html/template` | Stdlib, auto-escaping, SSR-safe |
|
|
|
|
### Browser UI
|
|
|
|
| Component | Technology | Justification |
|
|
| ----------- | ------------------------------ | -------------------------------------- |
|
|
| Rendering | Go `html/template` | Server-rendered, auto-escaped HTML |
|
|
| Interaction | Embedded browser scripts | Small progressive-enhancement modules |
|
|
| Styling | Native CSS + custom properties | No build-time CSS processing needed |
|
|
| Icons | Inline SVG | No icon font dependency |
|
|
|
|
### Infrastructure
|
|
|
|
| Component | Technology | Justification |
|
|
| ------------- | ------------------ | --------------------------------------------- |
|
|
| Container | Docker + Compose | Universal deployment target |
|
|
| Reverse Proxy | Traefik (optional) | Auto-HTTPS, included in compose |
|
|
| Database | libsql/SQLite | Single file, zero external dependency |
|
|
| Storage | Local filesystem | Content-addressed, immutable, backup-friendly |
|
|
|
|
---
|
|
|
|
## Security Model
|
|
|
|
### Threats Addressed
|
|
|
|
1. **Supply Chain Attacks**: Minimal external dependencies; all pinned with go.sum; vendored if needed
|
|
2. **Data Tampering**: All document versions SHA-256 hashed; optional Ed25519 signatures
|
|
3. **Unauthorized Access**: RBAC with principle of least privilege; content never served without auth check
|
|
4. **Session Hijacking**: Signed cookies with HttpOnly, Secure, SameSite=Strict; short expiry with refresh
|
|
5. **CSRF**: Double-submit cookie pattern for state-changing operations
|
|
6. **XSS**: Go html/template auto-escaping; strict CSP headers; no inline scripts except nonce-tagged
|
|
7. **DoS**: Rate limiting on auth endpoints; file upload size limits; query timeouts
|
|
|
|
### Content Security
|
|
|
|
- Attachment uploads: Magic number validation, extension whitelist, size limits
|
|
- Markdown rendering: Trusted content may include raw HTML; templates still auto-escape application data
|
|
- File paths: Canonicalization prevents directory traversal
|
|
|
|
### Audit & Compliance
|
|
|
|
- Every data modification logged with user, timestamp, IP, and content hash
|
|
- Immutable history table — updates are inserts
|
|
- GDPR-compliant: full data export and deletion capability
|
|
|
|
---
|
|
|
|
## Performance Targets
|
|
|
|
| Metric | Target | Measurement |
|
|
| ------------------- | ------------------------- | -------------------------------- |
|
|
| Initial Page Load | <100ms | TTFB + First Contentful Paint |
|
|
| Time to Interactive | <200ms | Hydration complete |
|
|
| Search Query | <50ms | Client-side after initial sync |
|
|
| File Sync | <5s | From disk save to server persist |
|
|
| Bundle Size | <100KB | Gzipped JS + CSS |
|
|
| Concurrent Users | 1000 readers, 100 editors | Load tested |
|
|
|
|
---
|
|
|
|
## Monorepo Structure
|
|
|
|
```
|
|
cairnquire/
|
|
├── README.md
|
|
├── docker-compose.yml
|
|
├── Dockerfile
|
|
├── mise.toml
|
|
├── docs/ # Project documentation
|
|
│ ├── project-plan.md # This file
|
|
│ ├── architecture-overview.md # System architecture
|
|
│ ├── adrs/ # Architecture Decision Records
|
|
│ ├── milestones/ # Detailed milestone plans
|
|
│ └── specs/ # Technical specifications
|
|
├── apps/
|
|
│ ├── server/ # Go HTTP application
|
|
│ │ ├── cmd/cairnquire/
|
|
│ │ ├── internal/
|
|
│ │ │ ├── app/ # Application orchestration
|
|
│ │ │ ├── config/ # Configuration management
|
|
│ │ │ ├── database/ # Database & migrations
|
|
│ │ │ ├── docs/ # Document service & repository
|
|
│ │ │ ├── httpserver/ # HTTP handlers & templates
|
|
│ │ │ ├── markdown/ # Markdown rendering pipeline
|
|
│ │ │ ├── realtime/ # WebSocket hub
|
|
│ │ │ └── store/ # Content-addressed filesystem
|
|
│ │ ├── static/ # Embedded static assets
|
|
│ │ └── templates/ # Go HTML templates
|
|
├── content/ # Markdown source files
|
|
├── data/ # SQLite database & attachments
|
|
└── packages/
|
|
└── protocol/ # Shared sync protocol spec
|
|
```
|
|
|
|
---
|
|
|
|
## Risk Register
|
|
|
|
| Risk | Likelihood | Impact | Mitigation |
|
|
| -------------------------------- | ---------- | ------ | ----------------------------------------------------------------------------------- |
|
|
| WebAuthn/passkey adoption issues | Medium | Medium | Password fallback always available; extensive browser testing |
|
|
| SQLite performance at scale | Low | High | libsql replication path; FTS5 for search; benchmark early |
|
|
| Markdown extension parser bugs | Medium | Medium | Fuzz testing; strict CommonMark base; gradual feature rollout |
|
|
| Email deliverability | Medium | Medium | Postmark reputation monitoring; SPF/DKIM setup; SMTP fallback |
|
|
| Frontend bundle bloat | Medium | Medium | Bundle analysis in CI; tree-shaking enforced; lazy loading |
|
|
| File sync conflicts in teams | High | Medium | Clear UX for conflict resolution; email notification; automatic merge when possible |
|
|
|
|
---
|
|
|
|
## Next Steps
|
|
|
|
1. ✅ Complete Milestone 1: Foundation
|
|
2. ✅ Complete Milestone 2: Sync Protocol
|
|
3. 🔄 Finish Milestone 3 hardening
|
|
- Add browser login/account management screens
|
|
- Complete security review checklist
|
|
4. Implement SQLite FTS5 search (Milestone 5)
|
|
5. Add Renovate with dependency update cooldown policies
|
|
|
|
---
|
|
|
|
## Glossary
|
|
|
|
- **ADR**: Architecture Decision Record
|
|
- **CRDT**: Conflict-free Replicated Data Type (not used, but considered)
|
|
- **CSRF**: Cross-Site Request Forgery
|
|
- **CSP**: Content Security Policy
|
|
- **FTS5**: SQLite Full-Text Search version 5
|
|
- **RBAC**: Role-Based Access Control
|
|
- **SSR**: Server-Side Rendering
|
|
- **TTFB**: Time to First Byte
|
|
- **WebAuthn**: Web Authentication API (passkeys)
|