sirtimbly/cairnquire
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: add project planning documents

Browse Source
This commit is contained in:
Tim Bendt
2026-04-28 23:59:45 -04:00
commit 8e6646499f
16 changed files with 2343 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

53
docs/adrs/adr-001-tech-stack.md Normal file
View File

@@ -0,0 +1,53 @@
# ADR-001: Go Backend with Preact Frontend
## Status
Accepted
## Context
We need to choose a backend language/framework and frontend technology for a documentation platform with these requirements:
- Ultra-fast performance
- Minimal external dependencies
- No vendor lock-in
- Battle-tested security
- Server-side rendering for non-interactive pages
- Offline-capable web client
## Decision
**Backend**: Go 1.22+ with standard library plus minimal, well-audited packages
**Frontend**: Preact 10 with TypeScript, Vite build system
## Consequences
### Positive
- Go's standard library is comprehensive and security-audited
- Single static binary deployment — no runtime dependencies
- Preact is 10KB (vs 40KB+ React) with identical API
- Go's `html/template` provides safe SSR with auto-escaping
- TypeScript gives type safety without runtime overhead
- Both ecosystems have excellent supply-chain security (Go modules with checksums, npm audit)
### Negative
- Go template syntax is less expressive than JSX
- WebSocket sync logic must be implemented manually (no framework like Socket.io)
- Preact ecosystem smaller than React (mitigated by using mostly native APIs)
## Alternatives Considered
### Rust + Axum
- **Pros**: Memory safety guarantees, excellent performance
- **Cons**: Longer compile times, smaller talent pool, steeper learning curve for team
- **Rejected**: Go's pragmatic balance of safety and velocity better fits project timeline
### Deno Fresh
- **Pros**: Island architecture, native TypeScript, no build step
- **Cons**: Newer ecosystem, fewer audited libraries, Deno runtime less battle-tested than Go
- **Rejected**: Go's maturity and deployment simplicity preferred
### Next.js / React Server Components
- **Pros**: Mature ecosystem, built-in SSR
- **Cons**: Heavy bundle size, complex build system, RSC lock-in, many dependencies
- **Rejected**: Violates "minimal dependencies" and "no vendor lock-in" principles
## References
- Go Security Policy: https://go.dev/security
- Preact Size Comparison: https://bundlephobia.com/package/preact@10.19.3

81
docs/adrs/adr-002-sync-protocol.md Normal file
View File

@@ -0,0 +1,81 @@
# ADR-002: SimpleSync Protocol
## Status
Accepted
## Context
Users need to edit markdown files on their local filesystem (using any editor) and have changes reflected on the web automatically. Multiple users should be able to work on different files simultaneously without conflicts. We need a sync mechanism simpler than git but robust enough for production use.
## Decision
Implement **SimpleSync**: a content-addressed, Merkle-tree-inspired sync protocol over WebSocket with the following characteristics:
1. **Content-addressed storage**: All files identified by SHA-256 hash
2. **File-level granularity**: Parallel editing of different files is conflict-free
3. **Automatic sync**: File watcher detects changes and pushes to server
4. **Three-way merge**: Only when same file edited concurrently
5. **No branches/commits**: Just current state + history log
## Protocol Flow
### Client → Server: Sync Request
```json
{
"type": "sync_request",
"client_root_hash": "abc123...",
"client_files": {
"getting-started.md": "hash1",
"api-reference.md": "hash2"
}
}
```
### Server → Client: Sync Response
```json
{
"type": "sync_response",
"server_root_hash": "def456...",
"missing_from_client": ["hash3", "hash4"],
"missing_from_server": ["hash5"],
"conflicts": ["getting-started.md"]
}
```
### Content Transfer
Files transferred by hash: `GET /content/{hash}` or WebSocket binary frames
## Consequences
### Positive
- Dramatically simpler mental model than git
- Automatic — no user commands needed
- Works with any text editor
- Immutable file store provides natural deduplication
- Easy to implement offline queue: just store hashes and content
### Negative
- No offline branching (by design)
- Conflict resolution UI must be built
- No rebase/squash history manipulation (intentional simplicity)
- Large binary files could bloat content store (mitigated by size limits)
## Alternatives Considered
### Git-based sync
- **Pros**: Mature, well-understood, existing tools
- **Cons**: Requires git on client, complex merge UX, repo-level locking
- **Rejected**: Too complex for notes; overkill for target use case
### CRDTs (Yjs, Automerge)
- **Pros**: Real-time collaborative editing, automatic conflict resolution
- **Cons**: Complex to implement correctly, memory overhead, designed for real-time not async
- **Rejected**: Requirement is async-only; CRDTs are overkill
### rsync / Unison
- **Pros**: Battle-tested file sync
- **Cons**: No concept of "versions" or "comments", no web integration
- **Rejected**: Doesn't meet collaboration requirements
## References
- Content-addressable storage: https://en.wikipedia.org/wiki/Content-addressable_storage
- Merkle trees: https://en.wikipedia.org/wiki/Merkle_tree
- WebSocket RFC: https://tools.ietf.org/html/rfc6455

67
docs/adrs/adr-003-ssr-strategy.md Normal file
View File

@@ -0,0 +1,67 @@
# ADR-003: SSR Strategy - Go Templates with Preact Hydration
## Status
Accepted
## Context
The application needs to serve document pages that are readable without JavaScript (for performance and accessibility), while providing rich interactivity (comments, editing, search) when JavaScript is available. We need a server-side rendering strategy that works with our Go backend and Preact frontend.
## Decision
**Strategy C**: Go's `html/template` renders initial HTML for all pages. Preact hydrates interactive components on the client.
### Implementation Details
1. **Server Rendering**:
- Go handlers query database for document content
- Markdown parsed to HTML server-side (Goldmark)
- `html/template` renders full page with document HTML embedded
- No JavaScript required for reading
2. **Client Hydration**:
- Preact components mount into designated DOM containers
- Comments, search, editor are Preact islands
- Static content remains server-rendered HTML
- `preact-iso` handles client-side navigation between pages
3. **Data Passing**:
- Initial state embedded as JSON in `<script type="application/json">` tags
- Preact reads initial state during hydration, avoids extra API calls
- CSP nonce applied to all inline scripts
## Consequences
### Positive
- First contentful paint is instant (pure HTML)
- SEO-friendly without extra complexity
- Graceful degradation: works without JS
- Go templates are auto-escaping XSS-safe
- No V8/JS runtime embedded in Go binary
- Smaller client bundle (only interactive components)
### Negative
- Some code duplication: markdown parsing logic in both Go and JS (for preview)
- Template syntax is verbose compared to JSX
- Hydration can cause flicker if not carefully managed
- Client state must reconcile with server-rendered HTML
## Alternatives Considered
### Go Templates Only (No Preact)
- **Pros**: Zero client JS, maximum performance
- **Cons**: No interactive features (comments, real-time sync, search)
- **Rejected**: Doesn't meet collaboration requirements
### Embedded JS Runtime (QuickJS/V8)
- **Pros**: True isomorphic rendering, JSX templates
- **Cons**: Adds C dependency, complex build, larger binary, security surface area
- **Rejected**: Violates minimal dependency principle
### Preact SSR via WASM
- **Pros**: Same templates on server and client
- **Cons**: WASM overhead, complex build pipeline, slower than Go templates
- **Rejected**: Go templates are faster and simpler
## References
- Go html/template security: https://pkg.go.dev/html/template
- Preact hydration: https://preactjs.com/guide/v10/server-side-rendering
- Islands Architecture: https://jasonformat.com/islands-architecture/

74
docs/adrs/adr-004-auth-strategy.md Normal file
View File

@@ -0,0 +1,74 @@
# ADR-004: Authentication Strategy
## Status
Accepted
## Context
The platform needs secure authentication for both technical and non-technical users. Requirements include: primary passkey support, password fallback, session management, and resistance to common attacks (phishing, credential stuffing, session hijacking).
## Decision
**Passkeys as primary authentication method, Argon2id-hashed passwords as fallback.**
### Implementation
1. **Registration Flow**:
- User provides email
- System generates WebAuthn registration challenge
- Browser creates passkey ( Touch ID, Windows Hello, YubiKey)
- Server stores credential ID and public key
- Optional: user sets password fallback
2. **Login Flow (Passkey)**:
- User enters email
- Server returns WebAuthn assertion challenge
- Browser signs challenge with passkey
- Server verifies signature, creates session
3. **Login Flow (Password)**:
- User enters email + password
- Server verifies Argon2id hash (constant-time comparison)
- If valid, creates session
- Prompts to set up passkey (progressive enhancement)
4. **Session Management**:
- Signed cookies with 24-hour expiry
- Refresh tokens rotate on each use
- Sessions stored in database (revocable)
- HttpOnly, Secure, SameSite=Strict flags
## Consequences
### Positive
- Passkeys are phishing-resistant by design
- No password database to breach (if passkey-only)
- Argon2id is winner of Password Hashing Competition
- Session tokens are revocable server-side
- Progressive enhancement: passkey users never type passwords
### Negative
- Passkey support varies by browser/OS
- Users may lose access if they lose all devices (recovery flow needed)
- WebAuthn implementation complexity (mitigated by go-webauthn library)
- Backup codes or recovery email needed for account recovery
## Alternatives Considered
### OAuth 2.0 / SSO Only
- **Pros**: No password management, familiar UX
- **Cons**: Vendor lock-in to identity providers, privacy concerns, not self-hosted
- **Rejected**: Violates "no vendor lock-in" principle
### Passwords Only (with 2FA)
- **Pros**: Simple, universally supported
- **Cons**: Vulnerable to phishing, credential stuffing, SIM swapping
- **Rejected**: Passkeys are strictly superior
### Magic Links
- **Pros**: No passwords, simple UX
- **Cons**: Email dependency, slower, inbox security issues
- **Rejected**: Less secure than passkeys, poor for frequent login
## References
- WebAuthn spec: https://www.w3.org/TR/webauthn-2/
- go-webauthn library: https://github.com/go-webauthn/webauthn
- Argon2id RFC: https://datatracker.ietf.org/doc/html/rfc9106

68
docs/adrs/adr-005-storage-strategy.md Normal file
View File

@@ -0,0 +1,68 @@
# ADR-005: Storage Strategy
## Status
Accepted
## Context
The system needs to store: structured data (users, documents, comments), document content, file attachments, and search indexes. Requirements include: zero external dependencies, backup simplicity, and resistance to data corruption.
## Decision
**libsql (SQLite) for structured data + content-addressed filesystem for files**
### Implementation
1. **Database (libsql/SQLite)**:
- Single file: `data/db.sqlite`
- WAL mode for concurrent reads during writes
- Embedded — no separate process needed
- Backup: `cp db.sqlite db.sqlite.backup` (atomic with WAL checkpoint)
2. **File Storage (Content-Addressed)**:
- Directory: `data/files/`
- Layout: `data/files/ab/cd/abcdef1234567890abcdef1234567890abcdef12`
- SHA-256 hash as filename (hex encoded)
- First two characters as subdirectory (for filesystem performance)
- Immutable: write-once, never modify
3. **Document Mapping**:
- Database stores: `document_path → current_hash`
- History table stores all previous hashes
- Attachment metadata stores original filename + content hash
## Consequences
### Positive
- SQLite is the most tested database in the world (billions of deployments)
- libsql adds replication option without changing the API
- Content-addressed files are naturally deduplicated
- Filesystem storage is trivial to backup (rsync, tar)
- No network dependencies, works offline
- Single directory contains all state — easy to migrate
### Negative
- SQLite has write concurrency limits (mitigated by WAL + short transactions)
- Filesystem storage doesn't scale horizontally without shared storage
- No built-in file expiration/garbage collection (must implement orphan cleanup)
- Large attachments can bloat filesystem (mitigated by quotas)
## Alternatives Considered
### PostgreSQL
- **Pros**: Excellent concurrency, rich feature set, horizontal scaling
- **Cons**: Requires separate process, more complex backup, network dependency
- **Rejected**: Overkill for anticipated scale; SQLite is simpler and faster for read-heavy workloads
### Object Storage (S3/MinIO)
- **Pros**: Unlimited scale, CDN-friendly
- **Cons**: Network dependency, vendor lock-in, complex local development
- **Rejected**: Violates zero-dependency and simple backup requirements
### SQLite for Everything (BLOBs)
- **Pros**: Single file for everything
- **Cons**: Large BLOBs slow down database, harder to serve directly, backup size bloat
- **Rejected**: Separation of concerns — database for metadata, filesystem for content
## References
- SQLite WAL mode: https://sqlite.org/wal.html
- Content-addressable storage: https://git-scm.com/book/en/v2/Git-Internals-Git-Objects
- libsql: https://github.com/tursodatabase/libsql

79
docs/adrs/adr-006-email-strategy.md Normal file
View File

@@ -0,0 +1,79 @@
# ADR-006: Email Integration
## Status
Accepted
## Context
The platform needs to send email notifications for: file changes (to watchers), new comments, conflict alerts, and digest summaries. It must also receive email replies and convert them to comments. Requirement: use Postmark email gateway with plain-text only.
## Decision
**Postmark HTTP API for outgoing email, Postmark inbound webhook for incoming email.**
### Implementation
1. **Outgoing Email**:
- Go adapter using `github.com/keighl/postmark`
- Plain-text templates (no HTML)
- Message-ID generated per email for threading
- Templates: notification, digest, conflict-alert, welcome
2. **Incoming Email**:
- Postmark inbound webhook POSTs to `/webhooks/email`
- Webhook payload parsed for: from, subject, body, in-reply-to
- Reply matched to comment thread via In-Reply-To header
- Body converted to plain text comment
- Authentication: webhook signature verification
3. **Email Format**:
```
From: MD Hub Secure <notifications@example.com>
To: user@example.com
Subject: [docs/getting-started.md] Comment from Alice
Message-ID: <comment-123@example.com>
In-Reply-To: <comment-122@example.com>
Alice commented on "Getting Started":
> This section needs clarification...
Reply to this email to add a comment.
View online: https://example.com/docs/getting-started#comment-123
```
## Consequences
### Positive
- Postmark has excellent deliverability reputation
- Plain-text emails are accessible and lightweight
- Email threading works in all clients
- No HTML parsing/rendering security risks
- Webhook integration is simple and reliable
### Negative
- Postmark is a paid service (though inexpensive)
- Requires DNS setup (SPF, DKIM) for deliverability
- Plain-text limits formatting options (intentional trade-off)
- Rate limits on Postmark API (mitigated by queue + retry)
## Alternatives Considered
### SMTP Direct Delivery
- **Pros**: No third-party dependency, full control
- **Cons**: Complex deliverability management, IP reputation, spam folder issues
- **Rejected**: Operational burden too high for small team
### SendGrid / Mailgun
- **Pros**: Similar features to Postmark
- **Cons**: More expensive at scale, complex APIs
- **Rejected**: Postmark has better reputation for transactional email
### No Email (In-App Only)
- **Pros**: Zero email complexity
- **Cons**: Users must check app for notifications; poor for async collaboration
- **Rejected**: Email is core to collaboration workflow
## References
- Postmark API docs: https://postmarkapp.com/developer
- keighl/postmark Go library: https://github.com/keighl/postmark
- Email threading RFC: https://www.ietf.org/rfc/rfc2822.txt

69
docs/adrs/adr-007-security-model.md Normal file
View File

@@ -0,0 +1,69 @@
# ADR-007: Security Model
## Status
Accepted
## Context
The platform handles sensitive documentation and must resist supply-chain attacks, unauthorized access, data tampering, and common web vulnerabilities. The user explicitly requested a "paranoid" approach.
## Decision
Implement defense in depth with the following layers:
### 1. Supply Chain Security
- **Minimal dependencies**: Only well-established, security-audited libraries
- **Pinned versions**: All Go modules pinned with go.sum checksums
- **Vendoring option**: `go mod vendor` for air-gapped builds
- **No build scripts**: Makefile only — no npm install, no curl | bash
- **Reproducible builds**: Same source → same binary hash (via `-trimpath`)
### 2. Authentication Security
- **Passkeys primary**: Phishing-resistant WebAuthn/FIDO2
- **Argon2id fallback**: Password Hashing Competition winner
- **Constant-time comparison**: All secret verification uses `subtle.ConstantTimeCompare`
- **Rate limiting**: Auth endpoints: 5 attempts per minute per IP
- **Session hardening**: Signed, HttpOnly, Secure, SameSite=Strict cookies
### 3. Authorization Security
- **RBAC**: Read/write/admin per document and collection
- **Principle of least privilege**: Default to no access, explicitly grant
- **Content verification**: SHA-256 hash verified on every read
- **Audit logging**: Every access logged with actor, resource, timestamp, IP
### 4. Input Validation
- **Markdown sanitization**: Goldmark renders to safe HTML; no raw HTML injection
- **File upload validation**: Magic number check, extension whitelist, size limits
- **Path canonicalization**: All file paths resolved to absolute, checked against allowlist
- **Query parameterization**: All SQL uses parameterized queries (no string concatenation)
### 5. Transport Security
- **TLS 1.3**: Minimum version enforced
- **HSTS**: Strict-Transport-Security header with preload
- **CSP**: Content-Security-Policy restricts script sources to nonce-tagged inline and same-origin
- **CORS**: Restrictive — only same-origin by default
### 6. Operational Security
- **No secrets in logs**: All tokens, passwords redacted
- **No secrets in environment**: Configuration from file or encrypted env vars
- **Read-only container**: Filesystem mounted read-only except `/data`
- **Non-root user**: Container runs as unprivileged user
- **Health checks**: `/health` endpoint verifies all dependencies
## Consequences
### Positive
- Resistant to OWASP Top 10 vulnerabilities
- Supply-chain attacks mitigated by minimal dependencies
- Audit trail provides accountability
- Container hardening reduces attack surface
### Negative
- CSP can block legitimate resources (requires careful tuning)
- Rate limiting may affect legitimate users behind NAT (mitigated by user-aware limits)
- Security reviews required for any new dependency
- More complex local development (TLS certs, CSP headers)
## References
- OWASP Top 10: https://owasp.org/www-project-top-ten/
- Go Security Best Practices: https://go.dev/security
- WebAuthn Security: https://www.w3.org/TR/webauthn-2/#security-considerations
- Content Security Policy: https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP

316
docs/architecture-overview.md Normal file
View File

@@ -0,0 +1,316 @@
# Architecture Overview
## System Context
MD Hub Secure is a self-hosted documentation platform. A single Go binary serves HTTP requests, manages a libsql database, watches a filesystem directory, and synchronizes content with connected web clients.
## Component Diagram
```
┌─────────────────────────────────────────────────────────────────────┐
│ Client Layer │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────────┐ │
│ │ Browser │ │ File Editor │ │ Mobile / PWA │ │
│ │ (Preact SPA) │ │ (VS Code, │ │ (Responsive Web) │ │
│ │ │ │ Vim, etc.) │ │ │ │
│ └──────┬───────┘ └──────┬───────┘ └────────────┬─────────────┘ │
│ │ │ │ │
│ │ HTTP/WS │ File System │ HTTP/WS │
└─────────┼──────────────────┼────────────────────────┼────────────────┘
│ │ │
┌─────────┼──────────────────┼────────────────────────┼────────────────┐
│ │ │ Go Application Server │
│ │ │ │
│ ┌──────▼──────┐ ┌──────▼──────┐ │
│ │ HTTP Router │ │ File Watcher │ │
│ │ (chi) │ │ (fsnotify) │ │
│ └──────┬──────┘ └──────┬──────┘ │
│ │ │ │
│ ┌──────▼──────────────────▼──────┐ │
│ │ Request Processing │ │
│ │ ┌────────────┐ ┌───────────┐ │ │
│ │ │ Auth Layer │ │ Markdown │ │ │
│ │ │(WebAuthn) │ │ Renderer │ │ │
│ │ └────────────┘ └───────────┘ │ │
│ │ ┌────────────┐ ┌───────────┐ │ │
│ │ │ Sync Engine│ │ Search │ │ │
│ │ │(WebSocket) │ │ (FTS5) │ │ │
│ │ └────────────┘ └───────────┘ │ │
│ │ ┌────────────┐ ┌───────────┐ │ │
│ │ │ Email │ │ File │ │ │
│ │ │ (Postmark) │ │ Store │ │ │
│ │ └────────────┘ └───────────┘ │ │
│ └──────────┬─────────────────────┘ │
│ │ │
│ ┌──────────▼─────────────────────┐ │
│ │ Data Layer │ │
│ │ ┌────────────┐ ┌───────────┐ │ │
│ │ │ libsql │ │ Content- │ │ │
│ │ │ (SQLite) │ │ Addressed │ │ │
│ │ │ │ │ Files │ │ │
│ │ └────────────┘ └───────────┘ │ │
│ └────────────────────────────────┘ │
└────────────────────────────────────────────────────────────────┘
```
## Component Descriptions
### 1. HTTP Router (chi)
- Serves static assets (CSS, JS, fonts)
- Routes API requests to handlers
- Serves server-rendered HTML for document pages
- Enforces middleware chain: logging, recovery, CORS, CSP, auth
### 2. File Watcher (fsnotify)
- Watches configured directory for .md file changes
- Debounces rapid changes (1 second)
- Computes SHA-256 hash of new/changed files
- Triggers sync events to connected clients
### 3. Authentication Layer
- WebAuthn/passkey primary (FIDO2 compliant)
- Argon2id password fallback
- Session cookies: signed, HttpOnly, Secure, SameSite=Strict
- RBAC middleware checks permissions on every request
### 4. Markdown Renderer
- Goldmark parser with custom extensions:
- Wiki-links: `[[Page Name]]` and `[[Page Name|Display Text]]`
- Tags: `#tagname` parsed and linked
- Admonitions: `> [!NOTE]`, `> [!WARNING]`, etc.
- Mermaid diagrams: fenced code blocks with `mermaid` language
- Math: `$inline$` and `$$block$$` via KaTeX
- Server-side rendering produces semantic HTML
- Client-side hydration adds interactivity (folding, copy buttons)
### 5. Sync Engine (WebSocket)
- Implements SimpleSync protocol (see spec)
- Bidirectional: server pushes changes, clients request/pull
- Content-addressed: files identified by SHA-256 hash
- Conflict detection: three-way merge or manual resolution
- Offline queue: client stores pending changes in IndexedDB
### 6. Search (FTS5)
- SQLite FTS5 virtual table indexes document content
- Server provides search API for initial load
- Client syncs index to IndexedDB for offline search
- Index rebuilds incrementally on file changes
### 7. Email (Postmark)
- Go adapter using Postmark HTTP API
- Plain-text emails only (no HTML)
- Threading via Message-ID / In-Reply-To headers
- Webhook endpoint receives email replies, converts to comments
- Templates: notification, digest, conflict alert
### 8. File Store (Content-Addressed)
- Files stored by SHA-256 hash: `store/ab/cd/abcdef1234...`
- Immutable: once written, never modified
- Deduplication: identical content stored once
- Metadata in database maps filenames to current hash
- Attachments served with strict content-type validation
## Data Flow: Document Rendering
```
1. Request: GET /docs/getting-started
│
▼
2. Auth Middleware: Check session, verify read permission
│
▼
3. Document Lookup: Query DB for current hash of "getting-started.md"
│
▼
4. Cache Check: Is rendered HTML in cache? (TTL: 1 minute for dynamic)
│
├── Yes → Serve cached HTML
│
└── No → Continue
│
▼
5. File Read: Read content from `store/ab/cd/...hash...`
│
▼
6. Markdown Parse: Goldmark with extensions → AST
│
▼
7. Transform AST: Resolve wiki-links, extract tags, validate admonitions
│
▼
8. Render HTML: AST → html/template with layout
│
▼
9. Response: HTML with CSP nonce, Preact hydration marker
```
## Data Flow: File Sync
```
User saves file on laptop
│
▼
File Watcher detects change
│
▼
Compute SHA-256 hash
│
▼
Store in content-addressed filesystem
│
▼
Update database: filename → new hash
│
▼
Broadcast to WebSocket subscribers
│
▼
Connected clients receive update notification
│
▼
Client requests new content by hash (if not cached)
│
▼
Server sends file content
│
▼
Client verifies hash, updates UI
```
## Database Schema (Simplified)
```sql
-- Documents: current state and metadata
CREATE TABLE documents (
id TEXT PRIMARY KEY,
path TEXT UNIQUE NOT NULL, -- relative path like "getting-started.md"
current_hash TEXT NOT NULL, -- SHA-256 of current content
title TEXT NOT NULL,
created_at DATETIME NOT NULL,
updated_at DATETIME NOT NULL,
updated_by TEXT REFERENCES users(id),
permission_read TEXT NOT NULL DEFAULT 'public', -- public, authenticated, private
permission_write TEXT NOT NULL DEFAULT 'authenticated'
);
-- Document versions: immutable history
CREATE TABLE document_versions (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL REFERENCES documents(id),
hash TEXT NOT NULL, -- content hash
previous_hash TEXT, -- for diffing
created_at DATETIME NOT NULL,
created_by TEXT REFERENCES users(id),
change_summary TEXT, -- auto-generated or user-provided
signature TEXT -- optional Ed25519 signature
);
-- Users: minimal, no PII beyond email
CREATE TABLE users (
id TEXT PRIMARY KEY,
email TEXT UNIQUE NOT NULL,
display_name TEXT,
password_hash TEXT, -- NULL if passkey-only
passkey_credential_id TEXT, -- WebAuthn credential
created_at DATETIME NOT NULL,
last_seen_at DATETIME
);
-- Sessions: server-side state
CREATE TABLE sessions (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id),
token_hash TEXT UNIQUE NOT NULL, -- SHA-256 of session token
created_at DATETIME NOT NULL,
expires_at DATETIME NOT NULL,
ip_address TEXT,
user_agent TEXT
);
-- Comments: linked to specific versions
CREATE TABLE comments (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL REFERENCES documents(id),
version_hash TEXT NOT NULL, -- which version this comments on
parent_id TEXT REFERENCES comments(id), -- threading
author_id TEXT NOT NULL REFERENCES users(id),
content TEXT NOT NULL,
created_at DATETIME NOT NULL,
resolved_at DATETIME,
resolved_by TEXT REFERENCES users(id)
);
-- Notifications: user preferences and queue
CREATE TABLE notification_settings (
user_id TEXT PRIMARY KEY REFERENCES users(id),
global_enabled BOOLEAN NOT NULL DEFAULT true,
digest_mode TEXT CHECK(digest_mode IN ('instant', 'hourly', 'daily')) DEFAULT 'instant',
email_enabled BOOLEAN NOT NULL DEFAULT true
);
CREATE TABLE document_watchers (
user_id TEXT REFERENCES users(id),
document_id TEXT REFERENCES documents(id),
folder_path TEXT, -- watch entire folder
created_at DATETIME NOT NULL,
PRIMARY KEY (user_id, document_id, folder_path)
);
-- FTS5 Search Index
CREATE VIRTUAL TABLE document_search USING fts5(
content,
document_id UNINDEXED,
tokenize='porter'
);
-- Audit Log: everything
CREATE TABLE audit_log (
id INTEGER PRIMARY KEY AUTOINCREMENT,
occurred_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
actor_id TEXT REFERENCES users(id),
action TEXT NOT NULL, -- read, write, delete, login, logout, sync
resource_type TEXT NOT NULL, -- document, user, comment, etc.
resource_id TEXT NOT NULL,
details TEXT, -- JSON metadata
ip_address TEXT
);
```
## Deployment Architecture
```yaml
# docker-compose.yml (simplified)
version: "3.8"
services:
app:
build: .
ports:
- "8080:8080"
volumes:
- ./data:/data # Database + attachments
- ./notes:/notes:ro # Source markdown files (optional)
environment:
- DB_PATH=/data/db.sqlite
- FILESTORE_PATH=/data/files
- NOTES_PATH=/notes
- POSTMARK_API_KEY=${POSTMARK_API_KEY}
- DOMAIN=${DOMAIN}
healthcheck:
test: ["CMD", "wget", "-q", "--spider", "http://localhost:8080/health"]
```
## Scaling Considerations
This architecture is intentionally single-node:
- **SQLite** handles read-heavy workloads excellently; write concurrency managed via WAL mode
- **Content-addressed files** are immutable and cache-friendly
- **Server-rendered HTML** reduces client compute
If horizontal scaling becomes necessary:
1. libsql replication (read replicas)
2. NFS or object storage for content-addressed files
3. Load balancer with sticky sessions for WebSocket
These are documented as future enhancements, not current requirements.

118
docs/milestones/milestone-01-foundation.md Normal file
View File

@@ -0,0 +1,118 @@
# Milestone 1: Foundation
**Duration:** 2 weeks
**Goal:** Working server that can serve markdown files as HTML
## Tasks
### Week 1: Project Setup & Infrastructure
- [ ] Initialize monorepo structure
- `apps/server/` with Go module
- `apps/web/` with Vite + Preact + TypeScript
- `packages/protocol/` with protobuf schema
- `docs/` with existing documentation
- [ ] Go server skeleton
- `main.go` with graceful shutdown
- Chi router with middleware chain
- Configuration management (environment variables + config file)
- Structured logging (slog)
- [ ] Database layer
- libsql connection setup
- Migration system (golang-migrate)
- Initial schema (users, documents, document_versions)
- Query builder / repository pattern
- [ ] Docker setup
- Multi-stage Dockerfile for Go app
- `docker-compose.yml` with volume mounts
- Health check endpoint
- Non-root user in container
### Week 2: Markdown Rendering & Static Serving
- [ ] Markdown parser integration
- Goldmark with all extensions
- Wiki-link parser: `[[Page Name]]` → internal link
- Tag extractor: `#tagname` → tag link
- Admonition support: `> [!NOTE]` blocks
- Mermaid diagram containers
- Math markup containers (KaTeX)
- [ ] Template system
- Base layout template (header, nav, footer)
- Document template with rendered markdown
- Error page templates (404, 500)
- Design system CSS custom properties
- [ ] Static file serving
- Content-addressed filesystem setup
- File upload endpoint (basic)
- Security headers middleware
- MIME type detection
- [ ] Initial frontend
- Preact app setup with Vite
- Basic routing (preact-iso)
- Hydration entry point
- CSS design system foundation
## Acceptance Criteria
### Functional
- [ ] Server starts and listens on configurable port (default 8080)
- [ ] Database auto-migrates on startup with no manual intervention
- [ ] Markdown files render correctly with all extensions:
- [ ] Wiki-links resolve to internal document paths
- [ ] Tags are extracted and displayed
- [ ] Admonitions render with appropriate styling
- [ ] Mermaid diagrams have syntax-highlighted containers
- [ ] Math markup has proper delimiters for KaTeX
- [ ] Attachments served with correct Content-Type headers
- [ ] 404 and 500 pages render correctly
### Non-Functional
- [ ] Docker compose brings up full stack with `docker-compose up -d`
- [ ] Container health check passes within 30 seconds
- [ ] All Go dependencies pinned in go.sum
- [ ] Build is reproducible (same commit → same binary hash)
- [ ] No plaintext secrets in code or logs
- [ ] Security headers present on all responses:
- [ ] Content-Security-Policy
- [ ] X-Content-Type-Options: nosniff
- [ ] X-Frame-Options: DENY
- [ ] Referrer-Policy: strict-origin-when-cross-origin
### Performance
- [ ] Initial page load (TTFB) <100ms for cached documents
- [ ] Markdown rendering <50ms for documents <100KB
- [ ] Static file serving supports 1000 concurrent requests
## Deliverables
1. Working development environment (README with setup instructions)
2. CI pipeline (GitHub Actions or similar) with:
- Go tests
- TypeScript type checking
- Dockerfile build
- Security scan (govulncheck)
3. API documentation (OpenAPI spec)
4. Database migration files
## Risk Mitigation
| Risk | Mitigation |
|------|-----------|
| Goldmark extension conflicts | Test each extension in isolation, then combined |
| Docker volume permissions | Document UID/GID requirements, provide setup script |
| libsql compatibility | Test on both native SQLite and Turso cloud |
## Definition of Done
- [ ] All acceptance criteria pass
- [ ] Code review completed
- [ ] Documentation updated
- [ ] CI pipeline green
- [ ] Security scan shows no critical/high vulnerabilities

143
docs/milestones/milestone-02-sync-protocol.md Normal file
View File

@@ -0,0 +1,143 @@
# Milestone 2: Sync Protocol
**Duration:** 2 weeks
**Goal:** Bidirectional file sync between server and filesystem/web client
## Tasks
### Week 3: Server-Side Sync
- [ ] Content-addressed filesystem implementation
- SHA-256 hash computation
- Directory layout: `store/ab/cd/abcdef...`
- Write-once semantics
- Deduplication verification
- [ ] File watcher (fsnotify)
- Watch configured directory recursively
- Debounce rapid changes (1s window)
- Ignore patterns (.git, temp files)
- Compute hash on change, update database
- [ ] WebSocket server
- Connection management (connection pool)
- Message framing and parsing
- Authentication over WebSocket (token validation)
- Heartbeat/ping-pong for connection health
- [ ] SimpleSync protocol server implementation
- Sync request/response handler
- Root hash computation (Merkle tree)
- Missing content detection
- Conflict detection logic
### Week 4: Client-Side Sync & Offline Support
- [ ] WebSocket client (Preact)
- Connection management with auto-reconnect
- Message queue for offline periods
- Exponential backoff for reconnection
- [ ] IndexedDB cache layer
- Store file content by hash
- Store sync state and pending changes
- LRU eviction for cache management
- [ ] File sync UI
- Sync status indicator
- Offline mode banner
- Pending changes list
- Manual sync trigger button
- [ ] Conflict detection display
- Visual indicator for conflicting files
- Basic conflict resolution UI (choose local/server)
## Acceptance Criteria
### Functional
- [ ] File changes on disk automatically sync to server within 5 seconds
- [ ] Web client receives real-time updates when files change on server
- [ ] Client can make changes offline and sync when reconnected
- [ ] Concurrent edits to different files succeed without conflict
- [ ] Concurrent edits to same file detected as conflict
- [ ] Content hash verified on every transfer (client and server)
- [ ] Sync works across browser refresh (IndexedDB persistence)
### Non-Functional
- [ ] WebSocket connections authenticated (reject unauthenticated)
- [ ] File watcher doesn't consume >1% CPU on idle
- [ ] IndexedDB storage cleared on logout
- [ ] Sync protocol documented with sequence diagrams
### Performance
- [ ] Sync of 100 files completes in <10 seconds
- [ ] WebSocket message overhead <1KB per sync cycle
- [ ] File watcher detects changes within 500ms
## Protocol Specification
### Message Types
```typescript
// Client → Server
interface SyncRequest {
type: 'sync_request';
clientId: string;
rootHash: string;
files: Record<string, string>; // path → hash
}
// Server → Client
interface SyncResponse {
type: 'sync_response';
serverRootHash: string;
missingFromClient: string[]; // hashes to download
missingFromServer: string[]; // hashes to upload
conflicts: string[]; // file paths with conflicts
}
// Content request
interface ContentRequest {
type: 'content_request';
hash: string;
}
// Content response
interface ContentResponse {
type: 'content_response';
hash: string;
content: string; // base64 encoded
}
```
### State Machine
```
[Idle] --sync_request--> [Comparing] --has_differences--> [Transferring] --complete--> [Idle]
--no_differences--> [Idle]
[Idle] --file_change--> [LocalChange] --sync--> [Comparing]
```
## Deliverables
1. SimpleSync protocol specification document
2. Sequence diagrams for all sync scenarios
3. WebSocket API documentation
4. Offline sync test suite
## Risk Mitigation
| Risk | Mitigation |
|------|-----------|
| WebSocket connection drops frequently | Auto-reconnect with exponential backoff, queue offline |
| Large files cause sync delays | Chunked transfer, progress indicators, size limits |
| Conflicts in frequently edited files | Optimistic UI, clear conflict resolution, email notification |
## Definition of Done
- [ ] All acceptance criteria pass
- [ ] Sync protocol tested with 2+ concurrent clients
- [ ] Offline/online transition tested
- [ ] Performance benchmarks meet targets
- [ ] Documentation complete

134
docs/milestones/milestone-03-authentication.md Normal file
View File

@@ -0,0 +1,134 @@
# Milestone 3: Authentication & Authorization
**Duration:** 2 weeks
**Goal:** Secure access control with passkeys and granular permissions
## Tasks
### Week 5: Passkey Authentication
- [ ] WebAuthn server implementation
- Relying party configuration
- Challenge generation and storage
- Attestation verification
- Credential storage (credential ID, public key, sign count)
- [ ] Passkey registration flow
- Initiate registration endpoint
- Verify registration response
- Store credential with user association
- Support multiple passkeys per user
- [ ] Passkey authentication flow
- Initiate login endpoint
- Verify assertion response
- Create session on success
- [ ] Session management
- Signed cookie generation
- Session storage in database
- Session validation middleware
- Session revocation endpoint
- Automatic session refresh
### Week 6: Password Fallback & Authorization
- [ ] Password authentication
- Argon2id password hashing
- Password validation endpoint
- Password change endpoint
- Password strength requirements
- [ ] Role-based access control
- Permission model: read, write, admin
- Scope: global, collection, document
- Middleware for permission checking
- Admin UI for managing permissions
- [ ] User management
- User registration (email + passkey/password)
- User profile management
- Account deletion (GDPR compliance)
- User listing (admin only)
- [ ] Content signing (optional)
- Ed25519 key pair generation
- Document signing on publish
- Signature verification display
## Acceptance Criteria
### Functional
- [ ] Users can register with passkey on supported browsers
- [ ] Passkey login works with Touch ID, Windows Hello, YubiKey
- [ ] Password fallback works when passkey unavailable
- [ ] Sessions expire after configurable timeout (default 24h)
- [ ] Sessions can be revoked (logout all devices)
- [ ] Read/write/admin permissions enforced on all endpoints
- [ ] Users can only access documents they have permission for
- [ ] Admin users can manage other users' permissions
### Non-Functional
- [ ] Argon2id parameters: time=3, memory=64MB, parallelism=4
- [ ] Session cookies: HttpOnly, Secure, SameSite=Strict
- [ ] Rate limiting: 5 auth attempts per minute per IP
- [ ] No timing attacks on password comparison (constant-time)
- [ ] All auth events logged to audit_log table
- [ ] Ed25519 signatures verified on document read (if enabled)
### Security
- [ ] WebAuthn challenges single-use and time-limited (5 minutes)
- [ ] Credential IDs are unpredictable (128-bit random)
- [ ] Password reset requires email verification
- [ ] No enumeration attacks (same response for exist/non-exist user)
## Database Schema Additions
```sql
-- WebAuthn credentials
CREATE TABLE webauthn_credentials (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id),
credential_id BLOB NOT NULL UNIQUE,
public_key BLOB NOT NULL,
sign_count INTEGER NOT NULL DEFAULT 0,
created_at DATETIME NOT NULL,
last_used_at DATETIME
);
-- Permissions
CREATE TABLE permissions (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id),
resource_type TEXT NOT NULL CHECK(resource_type IN ('global', 'collection', 'document')),
resource_id TEXT, -- NULL for global
permission TEXT NOT NULL CHECK(permission IN ('read', 'write', 'admin')),
granted_by TEXT REFERENCES users(id),
created_at DATETIME NOT NULL
);
```
## Deliverables
1. Authentication API documentation
2. WebAuthn flow diagrams
3. Permission system documentation
4. Security test results (penetration test guide)
## Risk Mitigation
| Risk | Mitigation |
|------|-----------|
| Passkey not supported on user's device | Password fallback always available |
| User loses all passkeys | Recovery email + backup codes |
| Permission system too complex | Start with simple roles, iterate |
| Session hijacking | Short expiry, rotation, revocation |
## Definition of Done
- [ ] All acceptance criteria pass
- [ ] Security review completed
- [ ] Auth flow tested on Chrome, Firefox, Safari, Edge
- [ ] Mobile passkey tested (iOS, Android)
- [ ] Rate limiting verified
- [ ] Audit logs verified

171
docs/milestones/milestone-04-collaboration.md Normal file
View File

@@ -0,0 +1,171 @@
# Milestone 4: Collaboration Features
**Duration:** 2 weeks
**Goal:** Comments, notifications, email integration, and conflict resolution
## Tasks
### Week 7: Comments & Notifications
- [ ] Comment system
- Comment creation endpoint (web)
- Comment threading (parent/child)
- Line/section anchoring (hash-based)
- Comment resolution (mark as resolved)
- Comment display in Preact UI
- [ ] Notification system
- Notification queue in database
- Per-file watcher subscriptions
- Per-folder watcher subscriptions
- Global notification settings
- Digest mode (hourly/daily batches)
- [ ] Web notification UI
- Notification bell with unread count
- Notification list dropdown
- Mark as read functionality
- Notification preferences page
### Week 8: Email Integration & Conflict Resolution
- [ ] Postmark integration
- Go adapter setup
- Plain-text email templates
- Outgoing email queue with retry
- Webhook endpoint for inbound email
- [ ] Email notification types
- File changed notification
- New comment notification
- Mention notification (@username)
- Conflict alert notification
- Digest summary email
[ ] Email reply handling
- Parse inbound email (from, subject, body)
- Match to comment thread via In-Reply-To
- Create comment from email body
- Validate sender permission
- [ ] Conflict resolution UI
- Side-by-side diff view
- Accept local/server/merge options
- Manual merge editor
- Conflict notification email
## Acceptance Criteria
### Functional
- [ ] Users can add comments to specific lines in a document
- [ ] Comments are linked to document version (hash)
[ ] Comment threading works (reply to reply)
- [ ] Users can watch files or folders for changes
- [ ] Email notifications sent within 1 minute of event
- [ ] Replying to notification email creates a comment
- [ ] Digest emails batch notifications by time window
- [ ] Conflicts display side-by-side diff
- [ ] Users can resolve conflicts via web UI
- [ ] Resolved comments are hidden but accessible in history
### Non-Functional
- [ ] Emails are plain-text only (no HTML)
- [ ] Email threading works in Gmail, Outlook, Apple Mail
- [ ] Webhook signature verified for inbound email
- [ ] Failed emails retried 3 times with exponential backoff
- [ ] Email queue doesn't block web requests (async processing)
### Email Template Examples
**File Change Notification:**
```
Subject: [docs/api-reference.md] Updated by Alice
Alice updated "API Reference" at 2024-01-15 14:30 UTC.
Changed sections:
- Authentication
- Rate Limiting
View changes: https://example.com/docs/api-reference?v=abc123
Unsubscribe: https://example.com/unsubscribe/123
```
**Comment Notification:**
```
Subject: [docs/getting-started.md] Comment from Bob
Bob commented on "Getting Started":
> This step is unclear. Can we add an example?
Reply to this email to respond.
View online: https://example.com/docs/getting-started#comment-456
```
## Database Schema Additions
```sql
-- Comments
CREATE TABLE comments (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL REFERENCES documents(id),
version_hash TEXT NOT NULL,
parent_id TEXT REFERENCES comments(id),
author_id TEXT NOT NULL REFERENCES users(id),
content TEXT NOT NULL,
anchor_line INTEGER, -- line number for anchoring
anchor_hash TEXT, -- hash of specific content for anchoring
created_at DATETIME NOT NULL,
resolved_at DATETIME,
resolved_by TEXT REFERENCES users(id)
);
-- Notifications
CREATE TABLE notifications (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id),
type TEXT NOT NULL, -- file_changed, comment, mention, conflict
resource_type TEXT NOT NULL,
resource_id TEXT NOT NULL,
message TEXT NOT NULL,
read_at DATETIME,
emailed_at DATETIME,
created_at DATETIME NOT NULL
);
-- Watcher subscriptions
CREATE TABLE watchers (
user_id TEXT REFERENCES users(id),
document_id TEXT REFERENCES documents(id),
folder_path TEXT,
created_at DATETIME NOT NULL,
PRIMARY KEY (user_id, document_id, folder_path)
);
```
## Deliverables
1. Email template documentation
2. Webhook integration guide
3. Comment API documentation
4. Conflict resolution flow diagrams
## Risk Mitigation
| Risk | Mitigation |
|------|-----------|
| Email deliverability issues | Postmark reputation monitoring, SPF/DKIM setup |
| Comment spam | Rate limiting, auth required, moderation tools |
| Email parsing errors | Robust parser, fallback to plain text extraction |
| Conflict resolution UI confusion | Clear visual design, undo option, help text |
## Definition of Done
- [ ] All acceptance criteria pass
- [ ] Email flows tested with real Postmark account
- [ ] Comment threading tested with 3+ levels
- [ ] Conflict resolution tested with 2+ concurrent editors
- [ ] Notification preferences persist across sessions
- [ ] Digest mode tested with 24-hour window

195
docs/milestones/milestone-05-search-ui.md Normal file
View File

@@ -0,0 +1,195 @@
# Milestone 5: Search & UI Polish
**Duration:** 2 weeks
**Goal:** Fast search, design system, responsive layout, and performance optimization
## Tasks
### Week 9: Search Implementation
- [ ] Server-side search index
- FTS5 virtual table setup
- Index population from documents
- Incremental index updates on file changes
- Search query endpoint
- [ ] Client-side search
- Flexsearch or Pagefind integration
- Index sync from server on load
- Incremental index updates via WebSocket
- Search UI (input, results, filters)
- [ ] Search features
- Full-text content search
- Tag filtering
- Title search (boosted)
- Recent results caching
- Search suggestions (autocomplete)
### Week 10: Design System & Performance
- [ ] Design system
- CSS custom properties for theming
- Component library (buttons, inputs, cards, navigation)
- Typography scale
- Color system (light/dark mode)
- Spacing scale
- `/design` route for browser-based design tool
- [ ] Responsive layout
- Mobile-first CSS
- Container queries for components
- Navigation adapts to screen size
- Touch-friendly targets (min 44px)
- Print styles for documents
- [ ] Performance optimization
- Bundle analysis and optimization
- Lazy loading for non-critical components
- Image optimization (if applicable)
- CSS critical path extraction
- Service worker for offline caching
- [ ] Accessibility
- ARIA labels on interactive elements
- Keyboard navigation
- Focus management
- Color contrast compliance (WCAG AA)
- Screen reader testing
## Acceptance Criteria
### Functional
- [ ] Search returns results in <50ms after initial sync
- [ ] Search works offline after first load
- [ ] Search supports exact phrases (quoted)
- [ ] Tag filtering narrows search results
- [ ] `/design` route displays all components with editable CSS
- [ ] Dark mode toggles correctly (no flash)
- [ ] All pages responsive down to 320px width
- [ ] Keyboard navigation works throughout app
### Non-Functional
- [ ] Lighthouse score >90 on all metrics
- [ ] Bundle size <100KB gzipped (JS + CSS)
- [ ] Time to Interactive <200ms
- [ ] No layout shift during hydration
- [ ] CSS custom properties work without JS
### Performance Targets
| Metric | Target | Measurement |
|--------|--------|-------------|
| Initial Page Load | <100ms | TTFB + FCP |
| Time to Interactive | <200ms | Hydration complete |
| Search Query | <50ms | Client-side after sync |
| Bundle Size | <100KB | Gzipped JS + CSS |
| Lighthouse Performance | >90 | Chrome DevTools |
| Accessibility | >90 | Lighthouse a11y |
## Design System Specification
### CSS Custom Properties
```css
:root {
/* Colors */
--color-primary: #2563eb;
--color-primary-hover: #1d4ed8;
--color-text: #1f2937;
--color-text-muted: #6b7280;
--color-background: #ffffff;
--color-surface: #f9fafb;
--color-border: #e5e7eb;
/* Typography */
--font-sans: system-ui, -apple-system, sans-serif;
--font-mono: ui-monospace, monospace;
--text-sm: 0.875rem;
--text-base: 1rem;
--text-lg: 1.125rem;
--text-xl: 1.25rem;
/* Spacing */
--space-1: 0.25rem;
--space-2: 0.5rem;
--space-4: 1rem;
--space-8: 2rem;
/* Shadows */
--shadow-sm: 0 1px 2px rgba(0,0,0,0.05);
--shadow-md: 0 4px 6px rgba(0,0,0,0.1);
/* Radii */
--radius-sm: 0.25rem;
--radius-md: 0.5rem;
}
@media (prefers-color-scheme: dark) {
:root {
--color-text: #f9fafb;
--color-text-muted: #9ca3af;
--color-background: #111827;
--color-surface: #1f2937;
--color-border: #374151;
}
}
```
### Component Examples
**Button:**
```html
<button class="btn btn-primary">Click me</button>
```
```css
.btn {
display: inline-flex;
align-items: center;
gap: var(--space-2);
padding: var(--space-2) var(--space-4);
border-radius: var(--radius-md);
font-size: var(--text-base);
font-weight: 500;
cursor: pointer;
transition: all 0.2s;
}
.btn-primary {
background: var(--color-primary);
color: white;
border: none;
}
.btn-primary:hover {
background: var(--color-primary-hover);
}
```
## Deliverables
1. Design system documentation
2. Component showcase at `/design`
3. Performance audit report
4. Accessibility audit report
5. Search API documentation
## Risk Mitigation
| Risk | Mitigation |
|------|-----------|
| Bundle size exceeds target | Tree-shaking, code splitting, lazy loading |
| Search index too large for client | Pagination, server-side fallback, index pruning |
| Design system inconsistent | CSS custom properties, component tests, design tokens |
| Mobile layout issues | Test on real devices, container queries, touch targets |
## Definition of Done
- [ ] All acceptance criteria pass
- [ ] Lighthouse audit green on all metrics
- [ ] Manual testing on Chrome, Firefox, Safari, Edge
- [ ] Mobile testing on iOS Safari and Android Chrome
- [ ] Screen reader tested (VoiceOver, NVDA)
- [ ] Performance benchmarks documented
- [ ] Design system reviewed by team

150
docs/milestones/milestone-06-production.md Normal file
View File

@@ -0,0 +1,150 @@
# Milestone 6: Production Hardening
**Duration:** 2 weeks
**Goal:** Deploy with confidence through testing, security audit, and operational documentation
## Tasks
### Week 11: Testing & Security
- [ ] Unit tests
- Go: >80% coverage for all packages
- Preact: Component testing with uvu
- Mock external dependencies (email, filesystem)
- [ ] Integration tests
- API endpoint testing
- Database operation testing
- Auth flow testing
- Sync protocol testing
- [ ] End-to-end tests
- User registration and login
- Document creation and editing
- Comment and notification flow
- Conflict resolution
- Search functionality
- [ ] Security audit
- Dependency vulnerability scan (govulncheck, npm audit)
- OWASP ZAP scan
- Manual penetration testing guide
- Rate limiting verification
- Session security testing
- File upload security testing
### Week 12: Operations & Documentation
- [ ] Backup and restore
- Automated backup script (database + files)
- Point-in-time recovery procedure
- Backup encryption (optional)
- Restore testing
- [ ] Monitoring
- Health check endpoint (`/health`)
- Metrics endpoint (`/metrics`) with Prometheus format
- Error tracking and alerting
- Log aggregation (structured JSON)
- [ ] Documentation
- Installation guide
- Upgrade procedures
- Backup/restore procedures
- Security incident response guide
- Troubleshooting guide
- API reference
- [ ] Load testing
- 1000 concurrent readers
- 100 concurrent editors
- File sync stress test
- Search performance under load
## Acceptance Criteria
### Functional
- [ ] >80% test coverage (Go + TypeScript)
- [ ] All integration tests pass
- [ ] All e2e tests pass
- [ ] Backup script creates restorable archive
- [ ] Restore procedure tested and documented
- [ ] Health check verifies: database, filesystem, email connectivity
### Non-Functional
- [ ] No critical or high-severity vulnerabilities
- [ ] No medium-severity vulnerabilities without documented mitigations
- [ ] Rate limiting prevents brute force attacks
- [ ] File uploads cannot execute code
- [ ] SQL injection impossible (parameterized queries only)
- [ ] XSS prevented (auto-escaping, CSP)
### Performance
- [ ] 1000 concurrent readers: p99 latency <500ms
- [ ] 100 concurrent editors: no sync conflicts lost
- [ ] Search performance: p99 <100ms under load
- [ ] Memory usage stable over 24-hour test (no leaks)
## Security Checklist
### Authentication
- [ ] Passkey registration works securely
- [ ] Passkey login validates signature
- [ ] Password hashing uses Argon2id
- [ ] Session cookies are signed and HttpOnly
- [ ] Session expiry is enforced
- [ ] Rate limiting on auth endpoints
### Authorization
- [ ] Permission checks on all endpoints
- [ ] No horizontal privilege escalation
- [ ] No vertical privilege escalation
- [ ] Content only served to authorized users
### Input Validation
- [ ] Markdown sanitized (no raw HTML injection)
- [ ] File uploads validated (magic numbers, size)
- [ ] Path traversal prevented
- [ ] SQL injection impossible
- [ ] XSS prevented
### Transport
- [ ] TLS 1.3 enforced
- [ ] HSTS header present
- [ ] CSP header restricts scripts
- [ ] CORS restrictive
### Operational
- [ ] No secrets in logs
- [ ] No secrets in environment (config file or encrypted)
- [ ] Container runs as non-root
- [ ] Read-only filesystem except /data
- [ ] Resource limits configured
## Deliverables
1. Test suite (unit, integration, e2e)
2. Security audit report
3. Backup/restore scripts
4. Monitoring dashboard configuration
5. Operator documentation
6. Load test results
## Risk Mitigation
| Risk | Mitigation |
|------|-----------|
| Security vulnerability found | Patch immediately, assess impact, notify users |
| Data loss | Automated backups, point-in-time recovery tested |
| Performance degradation | Monitoring alerts, load testing, scaling plan |
| Dependency vulnerability | Automated scans, rapid patching process |
## Definition of Done
- [ ] All acceptance criteria pass
- [ ] Security audit signed off
- [ ] Load tests meet targets
- [ ] Documentation complete and reviewed
- [ ] CI/CD pipeline green
- [ ] Rollback procedure tested
- [ ] Team trained on operations procedures

328
docs/project-plan.md Normal file
View File

@@ -0,0 +1,328 @@
# MD Hub Secure - Project Plan
**Version:** 1.0
**Date:** 2026-04-28
**Status:** Draft
## Vision
A minimal-dependency, ultra-fast documentation platform that publishes markdown notes as a responsive web application. Features wiki-style linking, collaborative comments, offline-capable sync, and fine-grained access control — all deployable via a single `docker-compose.yml` with zero vendor lock-in.
## Design Principles
1. **Paranoid Security**: Cryptographic content verification, battle-tested dependencies only, supply-chain attack mitigation through minimal external code
2. **Operational Simplicity**: Single binary, single SQLite database, filesystem storage — nothing that can't be understood in an afternoon
3. **Performance First**: Sub-100ms initial page loads, instant client-side navigation, no JavaScript required for reading
4. **Offline Capability**: Read cached, sync when connected
5. **No Vendor Lock-in**: Works on any Docker host; no cloud-specific APIs
---
## Milestones
### Milestone 1: Foundation (Weeks 1-2)
**Goal:** Working server that can serve markdown files as HTML
- Go HTTP server with chi router
- libsql database with schema migration system
- Markdown-to-HTML pipeline with all extensions
- Static file serving for attachments
- Basic Dockerfile and docker-compose.yml
**Acceptance Criteria:**
- [ ] Server starts and listens on configurable port
- [ ] Database auto-migrates on startup
- [ ] Markdown files with wiki-links, tags, callouts, mermaid, math render correctly
- [ ] Attachments served securely with content-type headers
- [ ] Docker compose brings up full stack with one command
- [ ] All dependencies pinned with checksum verification
- [ ] Build reproducible (same input → same binary hash)
---
### Milestone 2: Sync Protocol (Weeks 3-4)
**Goal:** Bidirectional file sync between server and filesystem/web client
- Content-addressed storage (SHA-256 filesystem layout)
- WebSocket sync protocol implementation
- File watcher for local filesystem sync
- Client-side IndexedDB cache
- Offline queue with conflict detection
**Acceptance Criteria:**
- [ ] File changes on disk automatically sync to server within 5 seconds
- [ ] Web client can sync after reconnection, even with changes made offline
- [ ] Concurrent edits to different files succeed without conflict
- [ ] Concurrent edits to same file detected and queued for resolution
- [ ] All synced content verified against SHA-256 hashes
- [ ] Sync protocol documented with state machine diagrams
---
### Milestone 3: Authentication & Authorization (Weeks 5-6)
**Goal:** Secure access control with passkeys and granular permissions
- Passkey (WebAuthn) primary authentication
- Password fallback with Argon2id
- Session management with signed cookies
- Role-based access control (RBAC)
- Content signing with Ed25519 (optional per-user)
**Acceptance Criteria:**
- [ ] Users can register and log in with passkeys on modern browsers
- [ ] Password fallback works with proper hashing
- [ ] Sessions expire securely and are revocable
- [ ] Read/write/admin permissions work per-document and per-collection
- [ ] All auth endpoints rate-limited and logged
- [ ] Security audit: no plaintext secrets, no timing attacks on auth
---
### Milestone 4: Collaboration Features (Weeks 7-8)
**Goal:** Comments, notifications, and conflict resolution
- Comment system linked to file versions (by hash)
- Email notifications via Postmark adapter
- Web-based conflict resolution UI
- Notification preferences (per-file, per-folder, global, digest)
- Comment threading and email replies
**Acceptance Criteria:**
- [ ] Users can comment on specific lines/sections of a document
- [ ] Comments persist and are linked to document versions
- [ ] Email notifications sent for watched file changes
- [ ] Replying to notification email creates a comment
- [ ] Conflicts display side-by-side diff with resolution actions
- [ ] Digest mode batches notifications by time window
---
### Milestone 5: Search & UI Polish (Weeks 9-10)
**Goal:** Fast search, design system, and production readiness
- Full-text search with SQLite FTS5
- Client-side search index sync
- Design system with browser-based CSS editor
- Responsive mobile layout
- Error boundaries and loading states
- Performance benchmarking
**Acceptance Criteria:**
- [ ] Search returns results in <50ms on client
- [ ] Search works offline after initial sync
- [ ] Design system documented at `/design` route
- [ ] All screens responsive down to 320px width
- [ ] Lighthouse score >90 on all metrics
- [ ] Initial page load <100ms (server-rendered HTML)
---
### Milestone 6: Production Hardening (Weeks 11-12)
**Goal:** Deploy with confidence
- Comprehensive test suite (unit, integration, e2e)
- Security audit and penetration testing guide
- Backup and restore procedures
- Monitoring and health checks
- Documentation for operators
**Acceptance Criteria:**
- [ ] >80% test coverage
- [ ] No critical or high-severity security findings
- [ ] Backup script exports database + attachments as tar.gz
- [ ] Health check endpoint returns status of all dependencies
- [ ] Operator docs cover: install, upgrade, backup, security incident response
- [ ] Load test: 1000 concurrent readers, 100 concurrent editors
---
## Architecture Overview
```
┌─────────────────────────────────────────────────────────────┐
│ Client Browser │
│ ┌──────────────┐ ┌──────────────┐ ┌─────────────────┐ │
│ │ Preact UI │ │ File Sync │ │ Search Index │ │
│ │ (hydrated) │ │ (IndexedDB) │ │ (FTS5) │ │
│ └──────────────┘ └──────────────┘ └─────────────────┘ │
└─────────────────────────────────────────────────────────────┘
│
WebSocket / HTTP
│
┌─────────────────────────────────────────────────────────────┐
│ Go Application Server │
│ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌──────────┐ │
│ │ HTTP │ │ WebSocket │ │ Markdown │ │ Auth │ │
│ │ Router │ │ Sync │ │ Renderer │ │ (WebAuthn)│ │
│ └────────────┘ └────────────┘ └────────────┘ └──────────┘ │
│ ┌────────────┐ ┌────────────┐ ┌────────────┐ │
│ │ Email │ │ Search │ │ File Watch │ │
│ │ (Postmark)│ │ (FTS5) │ │ (fsnotify)│ │
│ └────────────┘ └────────────┘ └────────────┘ │
└─────────────────────────────────────────────────────────────┘
│
┌──────────────────┴──────────────────┐
│ │
┌──────────────┐ ┌──────────────┐
│ libsql │ │ Content- │
│ (SQLite) │ │ Addressed │
│ │ │ Filesystem │
└──────────────┘ └──────────────┘
```
See [architecture-overview.md](architecture-overview.md) for detailed component design.
---
## Technology Stack
### Backend (Go 1.22+)
| Component | Library | Justification |
|-----------|---------|---------------|
| HTTP Router | `github.com/go-chi/chi` | Minimal, idiomatic, stdlib-compatible |
| Database | `github.com/tursodatabase/libsql-client-go` | Embedded SQLite with replication option |
| Migrations | `github.com/golang-migrate/migrate` | Battle-tested, version-controlled schema |
| Auth/Passkeys | `github.com/go-webauthn/webauthn` | Reference implementation, well-audited |
| Password Hash | `golang.org/x/crypto/argon2` | Winner of Password Hashing Competition |
| Markdown | `github.com/yuin/goldmark` + extensions | Fast, extensible, CommonMark compliant |
| WebSocket | `github.com/gorilla/websocket` | Most widely used Go WS library |
| File Watch | `github.com/fsnotify/fsnotify` | Cross-platform, stdlib-adjacent |
| Email | `github.com/keighl/postmark` + `net/smtp` | Postmark adapter + SMTP fallback |
| Crypto | `crypto/ed25519`, `crypto/sha256` | Standard library, no external deps |
| Templates | `html/template` | Stdlib, auto-escaping, SSR-safe |
### Frontend (Preact + TypeScript)
| Component | Library | Justification |
|-----------|---------|---------------|
| Framework | `preact` (10KB) | React API, tiny bundle, fast hydration |
| Routing | `preact-iso` | Lightweight isomorphic routing |
| Markdown | `marked` + `micromark` | Server+client compatible |
| Search | `flexsearch` | Client-side full-text, zero server load |
| Styling | Native CSS + custom properties | No build-time CSS processing needed |
| Icons | Inline SVG | No icon font dependency |
| Build | `vite` | Fast HMR, modern output, minimal config |
### Infrastructure
| Component | Technology | Justification |
|-----------|-----------|---------------|
| Container | Docker + Compose | Universal deployment target |
| Reverse Proxy | Traefik (optional) | Auto-HTTPS, included in compose |
| Database | libsql/SQLite | Single file, zero external dependency |
| Storage | Local filesystem | Content-addressed, immutable, backup-friendly |
---
## Security Model
### Threats Addressed
1. **Supply Chain Attacks**: Minimal external dependencies; all pinned with go.sum; vendored if needed
2. **Data Tampering**: All document versions SHA-256 hashed; optional Ed25519 signatures
3. **Unauthorized Access**: RBAC with principle of least privilege; content never served without auth check
4. **Session Hijacking**: Signed cookies with HttpOnly, Secure, SameSite=Strict; short expiry with refresh
5. **CSRF**: Double-submit cookie pattern for state-changing operations
6. **XSS**: Go html/template auto-escaping; strict CSP headers; no inline scripts except nonce-tagged
7. **DoS**: Rate limiting on auth endpoints; file upload size limits; query timeouts
### Content Security
- Attachment uploads: Magic number validation, extension whitelist, size limits
- Markdown rendering: Sanitized HTML output; no raw HTML injection
- File paths: Canonicalization prevents directory traversal
### Audit & Compliance
- Every data modification logged with user, timestamp, IP, and content hash
- Immutable history table — updates are inserts
- GDPR-compliant: full data export and deletion capability
---
## Performance Targets
| Metric | Target | Measurement |
|--------|--------|-------------|
| Initial Page Load | <100ms | TTFB + First Contentful Paint |
| Time to Interactive | <200ms | Hydration complete |
| Search Query | <50ms | Client-side after initial sync |
| File Sync | <5s | From disk save to server persist |
| Bundle Size | <100KB | Gzipped JS + CSS |
| Concurrent Users | 1000 readers, 100 editors | Load tested |
---
## Monorepo Structure
```
md-hub-secure/
├── README.md
├── docker-compose.yml
├── Dockerfile
├── Makefile
├── docs/ # Project documentation
│ ├── project-plan.md # This file
│ ├── architecture-overview.md # System architecture
│ ├── adrs/ # Architecture Decision Records
│ ├── milestones/ # Detailed milestone plans
│ └── specs/ # Technical specifications
├── apps/
│ ├── server/ # Go HTTP application
│ │ ├── main.go
│ │ ├── internal/
│ │ │ ├── api/ # HTTP handlers
│ │ │ ├── auth/ # Authentication & authorization
│ │ │ ├── config/ # Configuration management
│ │ │ ├── db/ # Database models & migrations
│ │ │ ├── markdown/ # Markdown rendering pipeline
│ │ │ ├── search/ # Full-text search
│ │ │ ├── sync/ # SimpleSync protocol
│ │ │ ├── email/ # Postmark integration
│ │ │ └── fs/ # Content-addressed filesystem
│ │ └── web/static/ # Built frontend assets
│ └── web/ # Preact frontend application
│ ├── src/
│ │ ├── components/ # Reusable UI components
│ │ ├── pages/ # Route-level components
│ │ ├── lib/ # Utilities & API client
│ │ ├── stores/ # State management
│ │ └── styles/ # CSS custom properties & themes
│ ├── design-system.html # Browser-based design tool
│ └── index.html
└── packages/
└── protocol/ # Shared sync protocol spec
├── sync.proto # Protocol buffer definition
└── go/ # Go generated types
```
---
## Risk Register
| Risk | Likelihood | Impact | Mitigation |
|------|-----------|--------|------------|
| WebAuthn/passkey adoption issues | Medium | Medium | Password fallback always available; extensive browser testing |
| SQLite performance at scale | Low | High | libsql replication path; FTS5 for search; benchmark early |
| Markdown extension parser bugs | Medium | Medium | Fuzz testing; strict CommonMark base; gradual feature rollout |
| Email deliverability | Medium | Medium | Postmark reputation monitoring; SPF/DKIM setup; SMTP fallback |
| Frontend bundle bloat | Medium | Medium | Bundle analysis in CI; tree-shaking enforced; lazy loading |
| File sync conflicts in teams | High | Medium | Clear UX for conflict resolution; email notification; automatic merge when possible |
---
## Next Steps
1. Review and approve this plan
2. Set up repository structure and CI pipeline
3. Begin Milestone 1: Foundation
4. Schedule weekly architecture review meetings during Milestone 2 (sync protocol)
---
## Glossary
- **ADR**: Architecture Decision Record
- **CRDT**: Conflict-free Replicated Data Type (not used, but considered)
- **CSRF**: Cross-Site Request Forgery
- **CSP**: Content Security Policy
- **FTS5**: SQLite Full-Text Search version 5
- **RBAC**: Role-Based Access Control
- **SSR**: Server-Side Rendering
- **TTFB**: Time to First Byte
- **WebAuthn**: Web Authentication API (passkeys)

297
docs/specs/simplesync-protocol.md Normal file
View File

@@ -0,0 +1,297 @@
# SimpleSync Protocol Specification
## Overview
SimpleSync is a content-addressed file synchronization protocol designed for asynchronous collaboration on markdown documents. It provides automatic bidirectional sync between a server (source of truth) and clients (filesystem watchers or web browsers).
## Design Goals
1. **Simplicity**: No branches, commits, or manual sync commands
2. **Automatic**: Changes propagate without user intervention
3. **Conflict-aware**: Detects concurrent edits, never silently loses data
4. **Offline-capable**: Queue changes locally, sync when connected
5. **Secure**: All content verified by cryptographic hash
## Concepts
### Content Addressing
Every file is identified by the SHA-256 hash of its content. The content is immutable — if a file changes, it gets a new hash.
### Merkle Tree
The directory state is represented as a Merkle tree where:
- Leaf nodes are file hashes
- Internal nodes are hashes of concatenated child hashes
- The root hash represents the entire directory state
### Sync State
Each client maintains:
- `root_hash`: Hash of current directory Merkle root
- `files`: Map of file paths to content hashes
- `pending`: Queue of local changes not yet synced
## Message Format
All messages are JSON over WebSocket.
### Client → Server
#### Sync Request
```json
{
"type": "sync_request",
"client_id": "uuid-v4",
"root_hash": "sha256-hex-64-chars",
"files": {
"getting-started.md": "sha256...",
"api-reference.md": "sha256..."
},
"timestamp": "2024-01-15T14:30:00Z"
}
```
#### Content Upload
```json
{
"type": "content_upload",
"hash": "sha256...",
"content": "base64-encoded-content",
"encoding": "base64"
}
```
#### Acknowledgment
```json
{
"type": "ack",
"message_id": "uuid-of-original-message",
"status": "ok"
}
```
### Server → Client
#### Sync Response
```json
{
"type": "sync_response",
"server_root_hash": "sha256...",
"missing_from_client": ["hash1", "hash2"],
"missing_from_server": ["hash3"],
"conflicts": ["getting-started.md"],
"timestamp": "2024-01-15T14:30:01Z"
}
```
#### Content Push
```json
{
"type": "content_push",
"hash": "sha256...",
"content": "base64-encoded-content",
"encoding": "base64",
"path": "getting-started.md"
}
```
#### Conflict Notification
```json
{
"type": "conflict",
"path": "getting-started.md",
"server_hash": "sha256...",
"client_hash": "sha256...",
"server_modified_at": "2024-01-15T14:30:00Z",
"client_modified_at": "2024-01-15T14:25:00Z"
}
```
#### Error
```json
{
"type": "error",
"code": "unauthorized",
"message": "Session expired",
"retryable": false
}
```
## Protocol Flow
### Normal Sync (No Conflicts)
```
Client Server
| |
|-- sync_request -------------->|
| root_hash: abc |
| |
| |-- Compare with server state
| |-- Calculate deltas
| |
|<- sync_response --------------|
| missing_from_client: [def] |
| missing_from_server: [] |
| conflicts: [] |
| |
|-- content_request(def) ------>|
| |
|<- content_push(def) ----------|
| |
|-- ack ----------------------->|
```
### Upload Changes
```
Client Server
| |
|-- sync_request -------------->|
| root_hash: abc |
| files: {a: hash1, b: hash2} |
| |
|<- sync_response --------------|
| missing_from_server: [hash2] |
| |
|-- content_upload(hash2) ----->|
| |
| |-- Verify hash
| |-- Store content
| |-- Update database
| |-- Broadcast to others
| |
|<- ack ------------------------|
| |
```
### Conflict Detection
```
Client A Server Client B
| | |
| | |-- Edit file X
| | |-- sync_request
| |-- Update file X |
| |-- Broadcast to A |
| | |
|-- Edit file X | |
|-- sync_request -------------->| |
| |-- Detect conflict |
| |-- A has old hash |
| |-- B has new hash |
|<- conflict -------------------| |
| path: X | |
| server_hash: B_hash | |
| client_hash: A_hash | |
| | |
```
## State Machine
```
+--------+ sync_request
| +-----------+
| Idle | |
| |<----------+
+---+----+
|
| connect
v
+---------+---------+
| |
| Connected |
| |
+---------+---------+
|
| sync_request
v
+---------+---------+
| |
| Comparing |
| |
+---------+---------+
|
+-----------+-----------+
| |
| has_changes | no_changes
v v
+---------+---------+ +---------+---------+
| | | |
| Transferring | | Idle |
| | | |
+---------+---------+ +-------------------+
|
| complete
v
+---------+---------+
| |
| Idle |
| |
+-------------------+
^
| conflict
|
+---------+---------+
| |
| Conflicted |
| |
+-------------------+
```
## Error Codes
| Code | Description | Retryable |
|------|-------------|-----------|
| `unauthorized` | Session expired or invalid | No (re-authenticate) |
| `forbidden` | Insufficient permissions | No |
| `not_found` | Requested content hash unknown | Yes |
| `too_large` | Content exceeds size limit | No |
| `hash_mismatch` | Content doesn't match claimed hash | Yes |
| `rate_limited` | Too many requests | Yes (with backoff) |
| `server_error` | Internal server error | Yes |
## Security Considerations
1. **Authentication**: All WebSocket connections must authenticate via token in initial HTTP upgrade request
2. **Authorization**: Server verifies read/write permissions before serving or accepting content
3. **Hash Verification**: Server recomputes SHA-256 of received content and rejects mismatches
4. **Size Limits**: Maximum file size enforced (configurable, default 10MB)
5. **Rate Limiting**: Sync requests limited per client (configurable, default 10/minute)
6. **Path Validation**: All file paths canonicalized and checked against allowlist
## Implementation Notes
### Hash Computation
```go
import "crypto/sha256"
func computeHash(content []byte) string {
h := sha256.Sum256(content)
return hex.EncodeToString(h[:])
}
```
### Merkle Root Computation
```go
func computeRootHash(files map[string]string) string {
// Sort paths for determinism
paths := sortedKeys(files)
hasher := sha256.New()
for _, path := range paths {
hasher.Write([]byte(path))
hasher.Write([]byte(files[path]))
}
return hex.EncodeToString(hasher.Sum(nil))
}
```
### WebSocket Connection
- Ping/pong every 30 seconds
- Connection timeout after 60 seconds without response
- Auto-reconnect with exponential backoff (1s, 2s, 4s, 8s, max 60s)
## Version
**Protocol Version**: 1.0
**Last Updated**: 2024-01-15