13 KiB
13 KiB
Architecture Overview
System Context
MD Hub Secure is a self-hosted documentation platform. A single Go binary serves HTTP requests, manages a libsql database, watches a filesystem directory, and synchronizes content with connected web clients.
Component Diagram
┌─────────────────────────────────────────────────────────────────────┐
│ Client Layer │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────────┐ │
│ │ Browser │ │ File Editor │ │ Mobile / PWA │ │
│ │ (Preact SPA) │ │ (VS Code, │ │ (Responsive Web) │ │
│ │ │ │ Vim, etc.) │ │ │ │
│ └──────┬───────┘ └──────┬───────┘ └────────────┬─────────────┘ │
│ │ │ │ │
│ │ HTTP/WS │ File System │ HTTP/WS │
└─────────┼──────────────────┼────────────────────────┼────────────────┘
│ │ │
┌─────────┼──────────────────┼────────────────────────┼────────────────┐
│ │ │ Go Application Server │
│ │ │ │
│ ┌──────▼──────┐ ┌──────▼──────┐ │
│ │ HTTP Router │ │ File Watcher │ │
│ │ (chi) │ │ (fsnotify) │ │
│ └──────┬──────┘ └──────┬──────┘ │
│ │ │ │
│ ┌──────▼──────────────────▼──────┐ │
│ │ Request Processing │ │
│ │ ┌────────────┐ ┌───────────┐ │ │
│ │ │ Auth Layer │ │ Markdown │ │ │
│ │ │(WebAuthn) │ │ Renderer │ │ │
│ │ └────────────┘ └───────────┘ │ │
│ │ ┌────────────┐ ┌───────────┐ │ │
│ │ │ Sync Engine│ │ Search │ │ │
│ │ │(WebSocket) │ │ (FTS5) │ │ │
│ │ └────────────┘ └───────────┘ │ │
│ │ ┌────────────┐ ┌───────────┐ │ │
│ │ │ Email │ │ File │ │ │
│ │ │ (Postmark) │ │ Store │ │ │
│ │ └────────────┘ └───────────┘ │ │
│ └──────────┬─────────────────────┘ │
│ │ │
│ ┌──────────▼─────────────────────┐ │
│ │ Data Layer │ │
│ │ ┌────────────┐ ┌───────────┐ │ │
│ │ │ libsql │ │ Content- │ │ │
│ │ │ (SQLite) │ │ Addressed │ │ │
│ │ │ │ │ Files │ │ │
│ │ └────────────┘ └───────────┘ │ │
│ └────────────────────────────────┘ │
└────────────────────────────────────────────────────────────────┘
Component Descriptions
1. HTTP Router (chi)
- Serves static assets (CSS, JS, fonts)
- Routes API requests to handlers
- Serves server-rendered HTML for document pages
- Enforces middleware chain: logging, recovery, CORS, CSP, auth
2. File Watcher (fsnotify)
- Watches configured directory for .md file changes
- Debounces rapid changes (1 second)
- Computes SHA-256 hash of new/changed files
- Triggers sync events to connected clients
3. Authentication Layer
- WebAuthn/passkey primary (FIDO2 compliant)
- Argon2id password fallback
- Session cookies: signed, HttpOnly, Secure, SameSite=Strict
- RBAC middleware checks permissions on every request
4. Markdown Renderer
- Goldmark parser with custom extensions:
- Wiki-links:
[[Page Name]]and[[Page Name|Display Text]] - Tags:
#tagnameparsed and linked - Admonitions:
> [!NOTE],> [!WARNING], etc. - Mermaid diagrams: fenced code blocks with
mermaidlanguage - Math:
$inline$and$$block$$via KaTeX
- Wiki-links:
- Server-side rendering produces semantic HTML
- Client-side hydration adds interactivity (folding, copy buttons)
5. Sync Engine (WebSocket)
- Implements SimpleSync protocol (see spec)
- Bidirectional: server pushes changes, clients request/pull
- Content-addressed: files identified by SHA-256 hash
- Conflict detection: three-way merge or manual resolution
- Offline queue: client stores pending changes in IndexedDB
6. Search (FTS5)
- SQLite FTS5 virtual table indexes document content
- Server provides search API for initial load
- Client syncs index to IndexedDB for offline search
- Index rebuilds incrementally on file changes
7. Email (Postmark)
- Go adapter using Postmark HTTP API
- Plain-text emails only (no HTML)
- Threading via Message-ID / In-Reply-To headers
- Webhook endpoint receives email replies, converts to comments
- Templates: notification, digest, conflict alert
8. File Store (Content-Addressed)
- Files stored by SHA-256 hash:
store/ab/cd/abcdef1234... - Immutable: once written, never modified
- Deduplication: identical content stored once
- Metadata in database maps filenames to current hash
- Attachments served with strict content-type validation
Data Flow: Document Rendering
1. Request: GET /docs/getting-started
│
▼
2. Auth Middleware: Check session, verify read permission
│
▼
3. Document Lookup: Query DB for current hash of "getting-started.md"
│
▼
4. Cache Check: Is rendered HTML in cache? (TTL: 1 minute for dynamic)
│
├── Yes → Serve cached HTML
│
└── No → Continue
│
▼
5. File Read: Read content from `store/ab/cd/...hash...`
│
▼
6. Markdown Parse: Goldmark with extensions → AST
│
▼
7. Transform AST: Resolve wiki-links, extract tags, validate admonitions
│
▼
8. Render HTML: AST → html/template with layout
│
▼
9. Response: HTML with CSP nonce, Preact hydration marker
Data Flow: File Sync
User saves file on laptop
│
▼
File Watcher detects change
│
▼
Compute SHA-256 hash
│
▼
Store in content-addressed filesystem
│
▼
Update database: filename → new hash
│
▼
Broadcast to WebSocket subscribers
│
▼
Connected clients receive update notification
│
▼
Client requests new content by hash (if not cached)
│
▼
Server sends file content
│
▼
Client verifies hash, updates UI
Database Schema (Simplified)
-- Documents: current state and metadata
CREATE TABLE documents (
id TEXT PRIMARY KEY,
path TEXT UNIQUE NOT NULL, -- relative path like "getting-started.md"
current_hash TEXT NOT NULL, -- SHA-256 of current content
title TEXT NOT NULL,
created_at DATETIME NOT NULL,
updated_at DATETIME NOT NULL,
updated_by TEXT REFERENCES users(id),
permission_read TEXT NOT NULL DEFAULT 'public', -- public, authenticated, private
permission_write TEXT NOT NULL DEFAULT 'authenticated'
);
-- Document versions: immutable history
CREATE TABLE document_versions (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL REFERENCES documents(id),
hash TEXT NOT NULL, -- content hash
previous_hash TEXT, -- for diffing
created_at DATETIME NOT NULL,
created_by TEXT REFERENCES users(id),
change_summary TEXT, -- auto-generated or user-provided
signature TEXT -- optional Ed25519 signature
);
-- Users: minimal, no PII beyond email
CREATE TABLE users (
id TEXT PRIMARY KEY,
email TEXT UNIQUE NOT NULL,
display_name TEXT,
password_hash TEXT, -- NULL if passkey-only
passkey_credential_id TEXT, -- WebAuthn credential
created_at DATETIME NOT NULL,
last_seen_at DATETIME
);
-- Sessions: server-side state
CREATE TABLE sessions (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id),
token_hash TEXT UNIQUE NOT NULL, -- SHA-256 of session token
created_at DATETIME NOT NULL,
expires_at DATETIME NOT NULL,
ip_address TEXT,
user_agent TEXT
);
-- Comments: linked to specific versions
CREATE TABLE comments (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL REFERENCES documents(id),
version_hash TEXT NOT NULL, -- which version this comments on
parent_id TEXT REFERENCES comments(id), -- threading
author_id TEXT NOT NULL REFERENCES users(id),
content TEXT NOT NULL,
created_at DATETIME NOT NULL,
resolved_at DATETIME,
resolved_by TEXT REFERENCES users(id)
);
-- Notifications: user preferences and queue
CREATE TABLE notification_settings (
user_id TEXT PRIMARY KEY REFERENCES users(id),
global_enabled BOOLEAN NOT NULL DEFAULT true,
digest_mode TEXT CHECK(digest_mode IN ('instant', 'hourly', 'daily')) DEFAULT 'instant',
email_enabled BOOLEAN NOT NULL DEFAULT true
);
CREATE TABLE document_watchers (
user_id TEXT REFERENCES users(id),
document_id TEXT REFERENCES documents(id),
folder_path TEXT, -- watch entire folder
created_at DATETIME NOT NULL,
PRIMARY KEY (user_id, document_id, folder_path)
);
-- FTS5 Search Index
CREATE VIRTUAL TABLE document_search USING fts5(
content,
document_id UNINDEXED,
tokenize='porter'
);
-- Audit Log: everything
CREATE TABLE audit_log (
id INTEGER PRIMARY KEY AUTOINCREMENT,
occurred_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
actor_id TEXT REFERENCES users(id),
action TEXT NOT NULL, -- read, write, delete, login, logout, sync
resource_type TEXT NOT NULL, -- document, user, comment, etc.
resource_id TEXT NOT NULL,
details TEXT, -- JSON metadata
ip_address TEXT
);
Deployment Architecture
# docker-compose.yml (simplified)
version: "3.8"
services:
app:
build: .
ports:
- "8080:8080"
volumes:
- ./data:/data # Database + attachments
- ./notes:/notes:ro # Source markdown files (optional)
environment:
- DB_PATH=/data/db.sqlite
- FILESTORE_PATH=/data/files
- NOTES_PATH=/notes
- POSTMARK_API_KEY=${POSTMARK_API_KEY}
- DOMAIN=${DOMAIN}
healthcheck:
test: ["CMD", "wget", "-q", "--spider", "http://localhost:8080/health"]
Scaling Considerations
This architecture is intentionally single-node:
- SQLite handles read-heavy workloads excellently; write concurrency managed via WAL mode
- Content-addressed files are immutable and cache-friendly
- Server-rendered HTML reduces client compute
If horizontal scaling becomes necessary:
- libsql replication (read replicas)
- NFS or object storage for content-addressed files
- Load balancer with sticky sessions for WebSocket
These are documented as future enhancements, not current requirements.